DEF CON CTF 2020 was held Friday, August 7^th through Sunday 9^th

Like DEF CON itself, finals were online for the first time ever, albeit with similar rules to classic finals. (CTFtime).

Finals are over, congratulations A*O*E!

 1.  970  A*0*E
 2.  968  PPP
 3.  841  HITCON ⚔ Balsn
 4.  750  Tea Deliverers
 5.  635  More Bush Smoked Whackers
 6.  570  侍
 7.  495  Shellphish
 8.  435  CyKor
 9.  409  /bin/tw
10.  394  NorseCode
11.  352  Star-Bugs
12.  303  koreanbadass
13.  273  mhackeroni
14.  260  r3kapig
15.  211  RPISEC
16.   77  pasten

Scoreboard with individual components: final_scores.html

We did public recaps of the CTF and our challenges – YouTube

We had four live-streamed challenge this year: casinooo, rhg, pinboooll, and ropshipai. Check out the recaps for their description and visualization.

Game data

Scoreboard with individual components: final_scores.html

Final game state: JSON
Tickets: JSON
Captain Communication: TXT
Final database dump: compressed SQL [Note: we removed data of unreleased service(s)]
PCAPs: bdooos others

Recap

DEF CON 28 CTF finals, like DEF CON 28 itself, was a wild experiment in a pandemic-plagued world. Within this experiment, we implemented a number of new game mechanics and infrastructure improvements that were very positively received, and made some mistakes that were not so positively received. We discuss both sets here, both in the interests of full transparency and in the hope that this discussion will be useful to other CTF organizers.

Stealth Ports

Traditionally, CTF competitors would analyze their network traffic to identify attacks, patch the underlying vulnerabilities in their services, and replay the captured traffic against other teams. This was an interesting part of the game, but often skewed toward tool-driven exploit replay rather than human-driven network analysis. Our first two years, OOO changed the game in that PCAPs were released with a massive delay: network traffic for a service would often not be available to the players until the service was almost “played out” and ready to be retired. This emphasized reverse-engineering and the creation of novel exploits over network analysis, but a part of the game, the mastery of one’s own network, was greatly diminished.

Looking to revitalize network sleuthing without devaluing the creation of original exploits, and inspired by the trade-off hackers make between being stealthy and being fast, we introduced the concept of stealth ports. Every service had two network endpoints. One of which functioned as normal, with all network traffic sent to it immediately accessible to the victim team. The other was different in two ways: network traffic sent to that service would never be accessible to the victim team, but any flags retrieved would have half their value. This allowed teams to choose: they could prevent other teams from detecting and stealing attacks against them, but at the cost of significant points. Given how close the outcome of CTFs often is, careful choices here were critical.

It took teams some time to develop stealth strategies. In the first half of the competition, almost all attacks we saw were stealthed. However, combined with the per-service flag limits (discussed next), this represented a serious reduction in the total number of points a team would be able to get from a service. Later in the game, we saw teams making different stealth decisions based (seemingly) on whether the victim team was ahead or behind them in score, whether the service would be retired soon, or whether they felt that the victim team had the network analysis capabilities to identify the attack. The full dataset of all stealth and non-stealth attacks on the attack-defense services in our CTF is available here.

One outcome that we saw from stealth attacks is that it made it extremely difficult for teams to defend against difficult-to-find but easy-to-exploit bugs. These bugs are traditionally naturally squashed as teams are informed of their existence through network analysis, but with an attacker capability of perfect stealth, each team must discover such bugs on their own through difficult reverse engineering. Normally, this would cause a runaway scenario, and we have seen this occur in prior years. As we will discuss next, this year, this had a different effect. However, some sort of balance to allow some stealth but prevent these attacks might be a good next step for attack-defense CTF.

Per-Service Flag Limits

Making a balanced scoring system is difficult. In prior years of our CTFs, we have seen “runaway” scenarios, where a team exploiting a single service with extreme success would heavily shift the game in their favor. This is not bad, per se, but the variety present in our services, could lead to situations where certain hacker skills end up unintentionally being vastly more useful than others.

Prior organizers have mitigated this effect through zero-sum scoring, where the capture of flags represents the actual theft of some finite store of points from one team to another, and once a team runs out of points, flags stolen from that team have no value. This system works, but it is frustrating for everyone but the top teams: slower teams quickly run out of points altogether, and the game becomes somewhat “locked” up by the top end of the scoreboard, and there is no easy measure of value that can be attached to a single flag: flags stolen earlier (even with an easier exploit) might be worth less than flags stolen later, because the victim team might simply be out of points.

We brainstormed several solutions, and settled on a per-service flag limit, where a service would be automatically shut off once any single team steals a total of 600 flags from it. This kept each flag value constant, but mitigated the runaway scenario. It also had an interesting interaction with our stealth mechanic: a fully stealthed team with a runaway solution would effectively halve the amount of points that they could earn from a service.

A negative side-effect of this was forced service retirement. Some of our services were designed with multiple “stages”, many different vulnerabilities, around unique concepts. Runaway scenarios on such services, when unmitigated by victim teams, would lead to these services being retired. This happened several times during the game and, unfortunately, reduced the explored depth of these services.

Private Team-Team Instances

Another innovation that we deployed this year is the concept of “Team-Team Instances”. Many of our attack-defense challenges had 256 backend instances for our 16 players, so connections between any two of these players would go to a unique instance. This had three effects.

First, it enabled the deployment of new challenge types, such as long-running server binaries that did not fork, which would otherwise be too DOS-able in practice. Second, it simplified challenge deployment. For example, each rorschach instance contained a single HTTP worker process, without concerns of team requests interfering with each other. Third, it eliminated interference between teams. If Team A launched a DOS against Team B on service S, only instance S-B-A would become inaccessible, and Team C could still talk to Team B’s Service C, because they would use the S-C-A instance.

Not all of our services used team-team isolation, because some of them required team-team interaction, but having it as a default option for attack-defense services is a powerful tool.

Infrastructure Stability

This is our third year hosting DEF CON finals (and, recently, also DEF CON quals) on our infrastructure, and the polish of the infrastructure shows. We’ve received incredibly positive feedback for the stability and ease-of-connectivity of our infrastructure, and we appreciate the sentiment! One team conveyed that they had such faith in the infrastructure that they stopped showing up for the pre-shift “setup” phases. That level of trust is amazing to see.

Our infrastructure benefited greatly from running an online (as opposed the traditional in-person) finals. Network access on the DEF CON floor cannot be taken for granted, so we have to bring our own mini-rack of compute. No matter how much money we pour into DEF CON CTF, there is a limit to how much compute can be stuffed into a mini-rack, and we often hit this limit. Our first year, we experienced an infrastructure meltdown due to a service that was unexpectedly CPU heavy under exploitation. Our second year, we strictly limited the incoming connection rate into the infrastructure to ensure stability. This year, we could simply spin up more and more nodes, and we did.

Of course, this is only possible because of the careful, reusable, and extensible design that our infrastructure gurus have put together. We can’t say a lot about this until we step down as organizers, but we hope to show it all to you one day!

The Human Factor

This year was hard. COVID, protests, political turmoil, and personal challenges contrived to increase the difficulty of putting on DEF CON CTF. We are human, and we make mistakes. Because these mistakes deserve maximum transparency, we have created a public postmortem of this year’s prominent issues.

Schedule

As you are all hopefully aware (if not, you have some travel plans to change!), DEF CON is going into Safe Mode because of COVID; the whole event, including DEF CON CTF, is going to be virtual. This means that this year’s 16 teams are going to spread out all over the world, rather than all be jetlagged together in Las Vegas. As a consequence, the timing of the game is tricky.

To minimize the chances of anyone collapsing and nearly dying (again) from hacking too hard, we have developed a schedule that will spread the pain out to everyone.

This year, DEF CON CTF will be played in 4 shifts of 8 hours, with 9 hours between shifts.

We’ll also do public recaps, you’re welcome to join to get a pulse of how the game is evolving.

Times in the Las Vegas timezone (PDT)

We’ll start 5 AM Las Vegas time on Friday, August 7th (4 AM for setup).

Here is the schedule (Nevada time == PDT):

• Shift 1: Friday 4am setup, 5am start, 1pm end
• First public recap: Friday 2 PM
• Shift 2: Friday 9pm setup, 10pm start, Saturday 6am end
• Second public recap: Saturday 1 PM
• Shift 3: Saturday 2pm setup, 3pm start, 11pm end
• Shift 4: Sunday 7am setup, 8am start, 4pm end
• Third public recap: Sunday at noon (during the game)

Good luck and see you on DEF CON’s discord (CTF area).

Streams

We’re planning to live-stream the scores and the game status. In addition, some challenges will have their own visualization that show teams tackling them and each other.

Stay tuned for updates.

On Discord, our official announcements channel is #ctf-announcements-text.

Qualified teams

Based on an excellent qualification round and the results of pre-qualifier events, 16 teams accepted our invitation to compete for the glory of winning the official DEF CON CTF:

 1. A*0*E (prequalified via PlaidCTF)
 2. /bin/tw
 3. CyKor (prequalified via CodeBlue)
 4. HITCON ⚔ Balsn
 5. koreanbadass
 6. mhackeroni
 7. More Bush Smoked Whackers (prequalified via CTFZone)
 8. NorseCode
 9. pasten (prequalified via 36C3)
10. PPP (prequalified by winning in 2019)
11. r3kapig
12. RPISEC
13. 侍 (fmr. Samurai)
14. Shellphish
15. Star-Bugs (fmr. CandySweetGuys)
16. Tea Deliverers (prequalified via HITCON)

Services

The Order has prepared two types of services for your enjoyment: Attack/Defense and King of the Hill. The former format is familiar from decades of DEF CON CTF: you exploit other teams’ services to steal their flags, and protect your own. King of the Hill is different: you compete against other teams for the best solution, which depends on the service in question.

Once you have access to your jumpbox, to attack a team’s Attack/Defense service on port Z, connect to 10.13.37.X:Z, where X is the victim team ID. To attempt a King of the Hill, connect to 10.13.37.Y:Z, where Y is YOUR team ID. Your own team interface is at 10.13.37.Y:80, where Y is YOUR team ID.

Services will go through a simple lifecycle, which is shown on the scoreboard.

• GREEN [GOOD]: They begin their life as green, unexploited services.
• YELLOW [OK]: Once they are exploited, we will move them to yellow. Services will stay yellow for at least two hours.
• ORANGE [LOW]: After significant exploitation of the service has occurred by several teams, or the service reaches a steady state, we will move the service to orange. A service will stay orange for at least two hours, then will become red when the Order feels that it has been played out.
• RED [BAD]: The service will stay red for at least two hours, and after this, the Order may retire the service at any time.
• RETIRED: Once retired, a service might still be accessible for interested parties, but they will no longer be scored and no flags from them will be accepted.

King of the Hill challenges also progress through colors, based on the optimality of teams’ solutions, with a minimum of 2 hours on each of Yellow, Orange, and Red.

Once retired, a service will not be re-activated. Note that a game shift may end with active services; playing will resume in the next shift. No services will be retired within 2 hours of the start of a competition “shift”.

As an exception to all of the above rules, a challenge will be immediately retired when any single team steals 600 flags on the service.

Scoring

Score takes into account three factors:

• Attack points, earned by stealing flags from other teams’ Attack/Defense services.
• Defense points, earned by resisting attacks against YOUR Attack/Defense services.
• KoH points, earned by top solutions of King of the Hill challenges.

Note that there is no “SLA” or “uptime” here.

Defense points accumulate by 1 for each of your services that is unexploited in a tick where successful exploits are launched.

Attack points accumulate by 1 for each flag that you retrieve, except for your own. Stealth flags are worth 0.5 points (see the PCAP section).

King of the Hill points depend on the quality of your solution. Each tick, all teams tied for first place will get 10 points. Teams tied for second will get 6, teams tied for third will get 3, teams tied for fourth will get 2, and teams tied for fifth will get 1. Other teams will not receive points. The Order encourages you to consider hacking harder.

The three types of points are normalized (compared to the top performer in each category) to account for 35%, 35%, and 30% of the total points of a team, respectively.

Game network access

We will provide you with a VM that is VPNed into the game, on which you will have a ‘team’ account with sudo access.

You can use this VM as a jump box, or you can extract the wireguard configuration to set it up on your own (though we will offer no tech support for such a configuration). The wireguard config on your machine will allow you to connect a single peer. If you use it for multiple peers, you will cause network problems for yourself.

We also have an ‘ooo’ account on the jump-box VM. We won’t force you to keep it active, but it‘s useful for us to help in debugging access issues.

PCAP access

Tired of us deciding when PCAPs are released? This year, we put the decision in YOUR hands. Now, you’re playing with POWER!

New this year, each service will have a STEALTH port alongside it’s normal port. The stealth port will be 10000+SERVICE_PORT, so If a challenge listens on port 1337, the stealth port will be 11337. The stealth port hits the same exact challenge endpoint as the normal port, but traffic through that port will not be released to the victim team. But beware: maintaining backdoor access to other teams ain’t cheap! If your team sends any traffic through a service’s stealth port to a given team, you will only receive half points for stealing that team’s flag that round.

We leave the analysis of the implications of this, both on attack and defense, as an exercise to the player.

Denial of Service

You’ll notice that there are no SLA points. This simplifies things for the players: they don’t have to worry about opaque testing failures leading to a loss of points; the patches are verified at submission, and are not deployed unless they pass the tests.

Of course, this system leaves room for abuse via specially-crafted patches. The Order will check verified patches manually (post deployment), and if we determine that a team has crafted a patch that adversarially passes our pre-deployment tests but brings down their service, we will disqualify that team from the competition, even retroactively. Play fair.

Unlike prior years, there are no connection frequency limits for services. If this becomes a problem for a specific service, we might enforce it on a case-by-case basis.

DDOS against other teams, including their jump boxes, is strictly forbidden.

Updates

The OOO will make announcements via our in-game announcement interface visible on your team dashboard, will be mirrored in the #ctf-announcements-text DEF CON discord channel and also tweeted from https://twitter.com/oooverflow/.

Pre-game Team Communications

Attention teams!

We hope that you are excited about DEF CON CTF (reminder: setup is on
Friday at 4:00am Vegas time)! To tide you over until then, we have
some information for you that might cause (or simplify) some prep work
on your part!

### Game network access

How will you access your network? We're glad you asked. We will
provide you each with a VM that is VPNed into the game. You can use
this VM as a jump box, or you can extract the wireguard configuration
to set it up on your own (though we will offer no tech support for
such a configuration). We'll grant access to these VMs by literally
emailing ssh keys to your team POC emails like massive noobs, so make
sure those mailboxes are secure!

As in previous years, challenges will be running at `10.13.37.TEAM_ID`.
If you choose to export the wireguard config, your (hopefully)
non-vulnerable personal machines will use `10.1.100+TEAM_ID.0/24`,
though we have reserved the lower 128 IPs of that network for our own
use, so back off. Your gateway should be `10.1.100+TEAM_ID.1`. We will
tell you your team ID when we mail you your ssh key.

### PCAP access

Tired of us deciding when PCAPs are released? This year, we put the
decision in YOUR hands. Now, you're playing with POWER!

New this year, each service will have a **STEALTH port** alongside it's
normal port. The stealth port will be `10000+SERVICE_PORT`, so If a
challenge listens on port 1337, the stealth port will be 11337. The
stealth port hits the same exact challenge endpoint as the normal
port, but traffic through that port will not be released to the victim
team. But beware: maintaining backdoor access to other teams ain't
cheap! If your team sends *any* traffic through a service's stealth
port to a given team, you will only receive half points for stealing
that team's flag that round.

We leave the analysis of the implications of this, both on attack and
defense, as an exercise to the player.

Good luck!
- The Order of the Overflow

Q&A

When will the wireguard config be sent out?

Never. You will find the wireguard config on your jump box. You will receive the ssh keys at 4am Las Vegas time. They will be sent to the email address you listed as your POC.

Will there be services running on the jump box?

No, the jumpbox is just there to simplify access.

What is the jump box like?

The jump box will have at least 2 cores and 8 gigs of ram, running Ubuntu 20.04.

Can we discard our jump box?

Yes. The jump box is basically just a pre-configured VPN endpoint for you. You can use it to run infrastructure, you can use it to host your own team-specific VPN, you can use it as an sshuttle endpoint, or you can export the VPN config and discard it. Please keep in mind that we will not help with ANY connectivity problems for teams not using the jump box.

Closest AWS zone to OOO’s infra.

The closest AWS zone is us-west.