Chris Lynch, director of the Defense Digital Service.

WASHINGTON: Artificial intelligence and big data can help the Pentagon figure out how it actually spends almost $700 billion a year, especially since much of the Defense Department still runs on two-finger typists entering data into old versions of Microsoft Excel.

“I just had a general who came to my office,” said Chris Lynch, head of DoD’s internal hoodie-wearing geek squad, the Defense Digital Service. “The problem was the entire mission was being run out of Excel.”

How is that even possible? “You would be amazed,” Lynch told the DefenseOne tech conference at the Newseum here. “The answer is that one person puts information from a piece of paper into an Excel spreadsheet; they email it to another (person), who then takes that and merges it into another (spreadsheet). That person then pivots in their chair; there’s another computer, they type it into that. That gets fired out on a different network, so both get sent out to another person who merges that into another spreadsheet — and then at the end of the day, after about 17 versions of that same step being done over probably a month, it goes into a PowerPoint presentation.”

The room erupted in incredulous laughter. But Lynch is serious about how badly the Defense Department handles data. It’s not just money, either. The worst of the worst is probably the perpetually backlogged process for vetting security clearances. “It is the definition of nearly everything wrong with …. doing a process,” Lynch said. “It’s feet on the streets, it’s paper, and….most of the time, we end up finding things out, (after) doing all of that work, that the person self-reported.”

After Lynch’s public talk, I asked him how the private sector did this better. “That’s not an easy question,” he said, because there’s a whole industry devoted to figuring out data sharing and workflows, which vary widely from company to company and industry to industry. “Understanding workflows (is) fundamental (to) the efficiency of businesses…. Sometimes you can just buy a piece of software (off the shelf). In other cases it may be bespoke to your organization.”

Robert Work

Pentagon processes are so complex and so dysfunctional that they’re potentially fertile ground for artificial intelligence and machine learning. During Bob Work‘s time as deputy Defense secretary under both Obama and Trump, Work told the conference, “we couldn’t really track” how much money the Pentagon actually spent on top priorities like cybersecurity, electronic warfare, or the artificial intelligence-focused Third Offset Strategy.

As anyone knows who’s paid attention to Pentagon spending knows, Congress appropriates funds every year, but the Defense Department’s information systems can’t follow the flow of money down to the military services and defense agencies and out to specific contractors on “literally millions of contracts,” he said. That’s part of the reason behind the Defense Department’s continuing inability to audit itself. The Pentagon announced its first system-wide audit in December last year.

Then-Adm. Sandy Winnefeld, at the time Vice-Chairman of the Joint Chiefs, showed Work an analysis of Third Offset-related spending by a big data/AI company called Govini. (Work and Winnefeld are now both on Govini’s board). “They showed me a simple graph (showing) this is how the money’s being spent down to the specific contracts,” Work recalled. “I was just blown away, because that type of data’s just not available in the Department of Defense.”

In one case, the Defense Department thought it was funding 38 separate projects on artificial intelligence, Govini founder Eric Gillespie told the conference. Govini was called in to double-check. The number of AI programs they found DoD was funding? 593.

Today it takes tremendous effort to get this kind of clarity, Gillespie and other panelists said, because the Pentagon’s data is fragmented amongst different agencies, offices, and contractors, each of which tends to guard its data jealously. Even when two groups want to share, they may not be able to because their software is incompatible. Recent advances allow artificial intelligence and big data analytics to scour this chaotic wilderness of information, the same way Project Maven scours hundreds of hours of drone surveillance footage for hints of terrorist activity.

“For the first time in history,” Gillespie said, “data science allows you to do things like that at scale that you just couldn’t do three, four, five years ago.”