Can a computer system compete against human CTF experts?

DARPA announced on Thursday that a computer system designed by a team of Pittsburgh-based researchers is the presumptive winner of the Agency’s Cyber Grand Challenge.

The winning computer system, dubbed Mayhem, was created by a team known as ForAllSecure — one of seven teams that competed for nearly $4 million in prizes in an all-day competition, performed in front of 5,000 computer security professionals and others at the Paris Las Vegas Conference Center.

Xandra, a computer system designed by team TECHx of Ithaca, N.Y., and Charlottesville, Va., was declared the presumptive second-place winner. And Mechanical Phish, a system designed by team Shellphish of Santa Barbara, Calif., was named the presumptive third-place winner.

First place in the CGC carries a cash award of $2 million; second- and third-place teams will receive $1 million and $750,000, respectively.

This event was the first head-to-head competition among developers of some of the most sophisticated automated bug-hunting systems ever developed. For almost 10 hours, competitors played the classic cybersecurity exercise of Capture the Flag in a specially created computer testbed laden with an array of bugs hidden inside custom, never-before-analyzed software. The machines were challenged to find and patch within seconds—not the usual months—flawed code that was vulnerable to being hacked, and find their opponents’ weaknesses before the defending systems did. The entire event was visualized for attendees on giant monitors and livestreamed for remote viewers, with expert “sportscasters” documenting the historic competition.

Judges will spend the night verifying those preliminary results, and winners will be officially crowned at an award ceremony Friday morning, immediately before the launch of DEF CON, the nation’s largest hacker tournament, also being hosted at the Paris Hotel.

At Friday’s ceremony, DEF CON organizers are expected to formally invite Mayhem to participate in this year’s DEF CON Capture the Flag competition, marking the first time a machine will be allowed to play in that historically all-human tournament.

“I’m enormously gratified that we achieved CGC’s primary goal, which was to provide clear proof of principle that machine-speed, scalable cyber defense is indeed possible,” said Mike Walker, the DARPA program manager who launched the challenge in 2013. “The effort by the teams, the DARPA leadership and staff, and all the hundreds of people who helped make this unique, open-to-the-public test happen was enormous. I’m confident it will speed the day when networked attackers no longer have the inherent advantage they enjoy today.”

DARPA’s Cyber Grand Challenge was designed to accelerate the development of advanced, autonomous systems that can detect, evaluate, and patch software vulnerabilities before adversaries have a chance to exploit them. The seven competing teams in today’s final event were composed of white-hat hackers, academics, and private-sector cyber systems experts.

The need for automated, scalable, machine-speed vulnerability detection and patching is large and growing fast as more and more systems—from household appliances to major military platforms—get connected to and become dependent upon the internet. Today, the process of finding and countering bugs, hacks, and other cyber infection vectors is still effectively artisanal. Professional bug hunters, security coders, and other security pros work tremendous hours, searching millions of lines of code to find and fix vulnerabilities that could be taken advantage of by users with ulterior motives.

The Heartbleed security bug existed in many of the world’s computer systems for nearly two and a half years, for example, before it was discovered and a fix circulated in spring 2014. By that time, the bug had rendered an estimated half million of the internet’s secure servers vulnerable to theft and other mischief. Analysts have estimated that, on average, such flaws go unremediated for 10 months before being discovered and patched, giving nefarious actors ample opportunity to wreak havoc in affected systems before they move on to exploit new terrain.

“This may be the end of DARPA’s Cyber Grand Challenge but it’s just the beginning of a revolution in software security,” Walker said. “In the same way that the Wright brothers’ first flight—although it didn’t go very far—launched a chain of events that quickly made the world a much smaller place, we now have seen for the first time autonomy involving the kind of reasoning that’s required for cyber defense. That is a huge advance compared to where the cyber defense world was yesterday.”

UPDATE:

It has been confirmed: ForAllSecure (and its Mayhem system) is the winner of the challenge. Second place was formally awarded to Xandra, a cyber reasoning system developed by TECHx. The third-place designation is pending verification by the Cyber Grand Challenge Competition Framework Team and the DARPA Verification team.