Mårten Mickos, CEO of HackerOne

The Pentagon’s willingness to pay freelance hackers to report cyber vulnerabilities has opened the floodgates for similar programs from other agencies, report the organizers of the original Hack The Pentagon. San Francisco-based HackerOne now counts clients ranging from the US Air Force, Army, and Defense Travel System to the Singaporean Ministry of Defense and the European Commission, with Congress passing a bill to add the Department of Homeland Security.

Since its founding in 2012, HackerOne has paid hackers $31 million for discovering over 72,000 vulnerabilities. Growth is so fast that over a third of this, $11.7 million for 27,000 bugs, was awarded in the last 12 months.

Statistics from 2016’s seminal Hack the Pentagon “bug bounty” program

Government business worldwide more than doubled in the last year, increasing 125 percent. In fact, HackerOne says the public sector is now ahead of most industries in acceptance of this emerging model of “open source” cybersecurity.

“It came pretty quickly,” HackerOne’s Finnish CEO, Mårten Mickos, told me. “I think…they all looked at Hack the Pentagon and said, ‘if America is doing it, we can do it.'”

True, the highest “buy bounties” — payments to freelance cybersecurity experts for reporting unique and dangerous vulnerabilities — are still from private sector tech companies. Intel and Microsoft now offer up to $250,000 for particularly critical discoveries, although no one’s actually collected this maximum payout, yet. 116 “unique critical vulnerabilities” earned over 10,000 each last year. Average bounty payments are much lower, ranging from just $668 per bug in the travel/hospitality industry to $3,635 in the technology sector — but government beats them at all an average payout of $3,892.

How does this work? There are two levels:

1. Bug bounties are awarded through a formal, structured process run by HackerOne on behalf of various agencies and companies. HackerOne publishes the rules — for example, which of a client’s websites are part of the bounty program and which are off-limits. Freelance hackers around the world conduct (legal, non-harmful) reconnaissance of the target systems and report their findings. HackerOne’s in-house experts and algorithms then evaluate whether reports are false, redundant, useful, or outright critical. Finally, the client pays out accordingly, after its own experts have reviewed the findings. Some of these programs are open to all comers, while others require participants to be vetted by HackerOne beforehand.
2. Vulnerability Disclosure Policies (VDPs) provide a framework for anyone to report a bug to a given company or agency at any time, without a reward. Obviously a financial incentive encourages people to look harder, but often cybersecurity researchers find vulnerabilities in the course of their research and would like to report them but have no clear channel to do so. In fact, some companies without VDPs threaten to sue anyone who examines their system for vulnerabilities, outright discourage reports. The Pentagon issued its VDP in 2016 alongside the Hack The Pentagon buy bounty program, and last year the Justice Department issued a general framework for VDPs.

Meanwhile, much of the private sector is lagging, HackerOne reports: “Today, 93% of the Forbes 2000 still do not have a public-facing VDP.” Some subsectors are much higher, with 61 percent of startups valued at over $1 billion having VDPs. But others are even lower, with only five percent of transportation sector companies and four percent of traditional financial services companies having them.

“They are under such heavy regulation, they are compliant with so many different standards,” Mickos said. “The compliance rules they operate under are actually slowing them down.”

“The fact is this the most cost effective way of finding vulnerabilities and the money you pay out is nothing compared to the value you get back,” he said. That’s why, he argues, it’s not a problem that HackerOne’s US clients have paid out 81 percent of bounties to date but 83 percent of the money has flowed to non-US hackers. (See chart below). The company’s business model is firmly internationalist and committed to open-source approaches, with Micklos arguing secrecy easily becomes counterproductive.

Hack The Air Force logo

Admittedly, this data only describes HackerOne, not the entire open-source security sector. But HackerOne is as big as all its competitors put together, Mickos said, based on such metrics as number of clients, number of valid vulnerabilities reported, and number of participating hackers.

The company even has an elaborate scoring system to help motivate hackers — “They’re so competitive,” Mikos chuckled — and a database of “hundreds of thousands of reports.” Those two systems will prove crucial to the company’s future, he argued.

One of the principles of big data is the more case studies you have, the better you can train your algorithms. While he’s scrupulous to say HackerOne’s report-assessment algorithms don’t rise to the level of artificial intelligence today, AI is the next step in cybersecurity.

And while the algorithms train on the data, the scoring system provides feedback so the human hackers can improve. “They love being measured, they’re very metrics driven, and they demand from us clear rules for the scoring and the points, so we developed it in reaction to their requests,” Mickos said.

Between the two systems, the accuracy and quality of bug reports has improved rapidly, the company reports. In 2015, just 17 percent of reports were “unique, valid security bugs” that the client company felt importance enough to fix. 32 percent were useless “noise.” (The remaining 51 percent of reports were useful but not unique). In 2017, noise was down to 26 percent, while unique, valid reports that led to actual fixes were higher for the first time, up to 27 percent. In an increasingly chaotic cyber domain, that ability to sort wheat from chaff is crucial.

Payments come in from HackerOne clients (listed by country) on the left and go out to participating hackers (by country) on the right.