If you find self-driving cars impressive today, think about Army trucks that can drive themselves off-road, in a war zone, less than three years from now.

For all the Army’s embrace of high technology, the service still wants the lead vehicle in the convoy to have a human driver, at least at first. But the unmanned trucks that follow behind will need to stick to the trail without relying on street signs, lane markings, pavement, or GPS. They might not even have a clear line of sight to the vehicle ahead of them, which may turn a corner in a city or disappear into a cloud of dust driving cross-country. En route, they have to avoid not only pedestrians, animals, and vehicles, like civilian self-driving cars, but also rubble, rocks, trees, and shell holes. And they have to avoid solid obstacles without stopping every time they see tall grass, a low-hanging branch, or a dust cloud in their path — the kind of common-sense distinction that’s easy for humans but very hard for computer vision.

Army Secretary Mark Esper (left) getting briefed on a self-driving “leader-follower” autonomous vehicle (in background).

But the Army is confident it can be done. Army Secretary Mark Esper has publicly enthused about the technology after riding in a prototype, saying it could both free up manpower for the front line — most troops work on logistics and maintenance, not in combat units — and save lives from roadside bombs and ambushes — to which supply convoys are particularly vulnerable.

After years of tinkering, the Army has accelerated its Automated Ground Resupply (AGR) program by spinning off something called the Expedient Leader-Follower demonstration. Contractors are currently installing Robotic Research LLC’s computer brains and sensors on 10 Oshkosh M1075 PLS (Palletized Loader System) trucks that’ll be used for safety certification tests in 2019. They’ll convert 60 more to self-driving vehicles in time to equip two Army transportation companies in 2020.

While the two units’ main job will be to demonstrate the technology works in field conditions, “if they get called to deploy, they will deploy with the vehicles,” said Alberto Lacaze, president of Robotic Research, in an interview with me yesterday. “That could happen fairly quickly.”

Exactly when the large-scale demo starts in 2020 is still a moving target, based mainly on how 2019’s safety testing goes, said Pat Williams, VP for Army and Marine Corps programs at Oshkosh Defense. It’s the Army’s call on whether to compress the timeline, he told me, but “there’s interest in pulling that left where possible.”

The Limits of Leader-Follower

Leader-Follower is still a limited form of autonomy — but those limits are more about Army tactics and culture than the technology itself.

The Army is ready for unmanned vehicles but not yet for a completely unmanned convoy, which is slated for a later phase of the Automated Ground Resupply program, albeit perhaps as early as 2022. The 2020 iteration is called Expedient Leader-Follower because the Army still wants a human soldier driving the lead vehicle, with up to nine autonomous trucks following in its trail. The self-driving trucks may still have soldiers in them, able to switch to human control if necessary or, alternatively, to let the truck handle the driving while they watch for ambushes and shoot back at attackers. That’s all part of the operational concepts the Army wants to work out in the 2020 demonstration.

But Oshkosh and Robotic Research told me they could take the humans out much sooner, if the Army asked, and have the truck drive itself along a route that a soldier chose without any soldiers actually having to be present. (I didn’t talk to the other two Leader-Follower contractors because they’re less directly involved with the autonomy system: Lockheed Martin, which does integration work, and ECS, which builds the interface for human operators). The hard part isn’t getting a computer to trace a route on a map — even your smartphone can do that — but getting it to move through physical space without hitting anything. That difficulty of that challenge is why Tesla and Uber self-driving vehicles have ending up killing three people (which is still a lot better than human drivers).

“The technology does exist, but we’re not there yet with acceptance,” said Chuck Bunton, Oshkosh’s program manager for autonomous vehicles. “This is a phased approach.”

“This is a big milestone for the Army,” said Robotic Research’s Lacaze, both for combat troops and for the acquisition community. In next year’s safety certification tests, he told me, “this is the first time, to our knowledge at least, that ATEC (Army Test & Evaluation Command) is going to have to give a rating for a vehicle that is basically an SAE Level 5 (on a scale of 0-5), a fully autonomous vehicle without anybody in the cab.”

That said, Lacaze went on, Robotic Research and General Dynamics together fielded a self-driving vehicle that could lead convoys without a human aboard way back in 2008. (This was a Special Operations program in urgent wartime circumstances, able to bypass the usual testing process). While he’s not allowed to give details, Lacaze told me that “we deployed vehicles in Afghanistan for road clearing, fully autonomous… To our knowledge, that’s the first time that fully autonomous vehicles have been deployed in theater.”

Now, a route-clearance mission — finding and removing roadside bombs and other Improvised Explosive Devices (IEDs) so other vehicles can pass safely– is very different from a supply convoy. In some ways, it’s actually easier: You’re on a road, to start with, and you’re usually moving extremely slowly as you search for ad hoc mines. In other ways, it’s much harder, since screwing up means blowing up.

Route-clearance unit in Iraq.

Despite the differences, Lacaze said the road-clearing mission pioneered key technology and learned useful lessons that carry over to Automated Ground Resupply. Plus, of course, computers in general and robotics in particular have improved immensely in the last 10 years. Robotic Research could “absolutely” build a completely unmanned convoy-leader vehicle, if only as a backup in case the human leader gets killed, he said, but the Army isn’t asking for that until a later stage of the AGR program: “There’s a lot of technology that we have to do that, but that’s not the program we currently have.”

With the technology already being installed on the Leader-Follower vehicles, however, there is a workaround to having a human in the convoy, Lacaze admitted when I pressed him. You’d still need a human to drive a leader vehicle down the route the first time, so the computer and sensors could figure out landmarks along the way, he said. But once the trail was digitally blazed, unmanned follower vehicles could drive themselves down it whenever. That might be minutes after the human led the way, or hours, or days, or “we could have one of those vehicles follow a leader from last month,” he said. “And we have demonstrated that on previous programs.”

How is this possible? Because of the ingenious way Robotic Research combines data from a wide array of sources to landmark a route.

Army HEMTT truck converted to a self-driving “leader follower” vehicle at TARDEC (Tank-Automotive Research, Development, & Engineering Command) in Warren Mich..

Blazing the Trail

Civilian navigation nowadays relies on GPS. The military can’t, because both hostile terrain — like a narrow canyon in Afghanistan — and human enemies — using a Russian-made jammer — can block the GPS signal. There’s such a thing as an Inertial Navigation System (INS) that uses gyroscopes, accelerometers, and a lot of math to figure out where you’re going based purely on heading and velocity, but that kind of dead reckoning loses accuracy the further you go. So military robots need to check their location against landmarks, just like humans do, only without the benefit of hundreds of millions of years of evolution.

Rather than rely on any one technique, Robotic Research uses a belt-and-suspenders approach — although it’s really more like someone wearing multiple belts, several sets of suspenders, and some duct tape over those beltless trousers that are supposed to stay up by themselves in the first place. (The company did much of its earlier work on systems to help human troops navigate underground). If the enemy jams, decoys, or physically destroys one sensor, there’re several others that still work.

Russian Krasukha-2 electronic warfare system with dish deployed. The system was reportedly used in Syria.

Sure, GPS is great if you can get the signal, so the system accepts that as one input among many, when it’s available. It uses INS to keep going between one landmark and the next. But the crucial part is how the system uses a combination of cameras, radars, and lasers (LADAR) to pick those landmarks.

The Leader-Follower vehicles don’t navigate the way humans do — “turn left at the 7-11 and then go, like, a couple blocks, past a big red house….” — because robots see the world much more precisely, with much less tolerance for ambiguity. A landmark for this system can be almost any solid, stationary, and distinctive object: a tree, a telephone pole, a rock, a section of curb, even a pothole. As the human drives the lead vehicle down the route, its autonomy system detects potential landmarks along the way, checks their location with multiple sensors, and then transmits the data to the followers.

Then, as each follower approaches the landmark in turn, it uses its sensors to detect it and confirm its own position, adjusting its Inertial Navigation System as needed. If the lead vehicle passed 1.9 meters to the left of a particular telephone pole, but the follower finds it’s passing 1.8 meters away, it knows it’s drifted 10 centimeters to the right and corrects its course accordingly.

What if the landmark isn’t there at all? Perhaps someone knocked it down after the leader passed by, for example, or the leader vehicle mistakenly tagged something that could move away. Well, then the follower software is smart enough to, in effect, give an algorithmic shrug and move on, following its INS until it can check itself against the next landmark.

Since the INS doesn’t actually drift that much — its average error after going 100 meters is just ten centimeters — the system can get by with relatively few landmarks. “If we had one of those features every mile, we would correcting sufficiently to drive in most areas,” he said. (He didn’t specify, but presumably the areas where you’d need to correct yourself most often to avoid hitting something, like cities or forests, are the ones with the most landmarks). It’s a bit like walking around your house at night with the lights off, he said: You can probably get by on dead reckoning most of the time, occasionally touching the wall to verify your position.

Oshkosh PLS (Palletized Loader System) trucks equipped with Leader-Follower autonomy systems.

A Place for Humans?

This combination of landmarks and dead reckoning is what allows the unmanned follower to go down a trail a month after a human-driven leader vehicle marked it. Unlike in some earlier experiments, the unmanned follower doesn’t actually have to watch the manned lead vehicle go down the road ahead of it. It just has to see the same landmarks the leader saw at some previous point. It doesn’t have to see all of the landmarks either, just enough to keep correcting its inertial navigation system.

BAE’s Armed Robotic Combat Vehicle (ARCV), originally developed for the Future Combat Systems (FCS) program

That the system could function without humans anywhere around, however, doesn’t mean that humans aren’t useful, Lacaze made clear. For the foreseeable future, humans will still be better at subtleties like whether an overhanging tree branch is safe to brush past or needs to be avoided because it’s strong enough to crack the windshield. They’re also better at picking up subtle signs of a roadside bomb or lurking ambushers, such as loose trash along the road, grass trodden down by passing enemies, or earth disturbed by recent digging.

What’s more, having survived the dinosaurs, giant killer birds, and saber-toothed tigers, mammalian brains are still better than computers at reacting to danger with either fight or flight. While the Army is also pursuing armed robotic vehicles, they will, initially at least, be remote-controlled by human operators, especially when they open fire, rather than self-driving like the unarmed Leader-Follower trucks.

Even if the robots get as good as humans eventually, they’ll still be good in a different way, simply because brain cells and microchips don’t work the same. On the battlefields of the 21st century, robots will complement human soldiers, not replace them altogether — but getting humans out of vulnerable supply trucks could save a lot of lives.