The U.S. Army has announced that it has enlisted the help of some of the brightest minds in the tech industry to test and evaluate crucial decision-making skills of tank commanders on the battlefield.

According to Joe Lacdan, to achieve that goal, the service extended its reach thousands of miles west from Fort Benning’s Maneuver Center of Excellence in Georgia — where tank crews normally train — to Los Angeles.

Researchers here at the Institute for Creative Technologies at the University of Southern California through a partnership with the Army have developed a mixed reality program, the Synthetic Collective Operational Prototyping Environment, or SCOPE.

The program focuses on the cognitive aspect of the training — the critical battlefield responses a tank commander makes — by placing the student in a simulated, immersive 3D-training environment.

ICT researcher David Krum said tank commanders must be vigilant to status updates from unit leadership at mission command and incoming enemy aircraft hovering overhead, making decisions that affect the lives of the other three tank crew members.

The technology can evaluate a student’s performance in the virtual reality, multi-player program by using specially-positioned sensors that track eye movement and body position. The ICT development team hopes the training will lead to rapid efficiency while helping students retain information at a higher rate.

“If they can get more repetitions in for the training, people get more practice (for) when they actually go into combat or actually go into a field exercise,” Krum said. “They’ve already done missions before. They can approach that more confidently and save time.”

Researchers have spent nearly two years developing the technology in an enclosed studio space within the ICT headquarters — a striking complex of trapezium-shaped structures, covered by cascading, tinted windows. Programmers in suits and polo shirts occupy the bright studio, where they exchange ideas while hunkered over each project’s interface.

David Nelson, ICT’s creative director for mixed reality research, said an earlier version of SCOPE, the Team Assessment and Learner Knowledge Observational Network, or TALK-ON, just completed three testing rotations at Fort Benning that yielded “positive” results.

Even though no Soldiers currently work in the ICT, several of the programmers have military experience. The SCOPE research team has also worked closely with the Army Research Laboratory at Aberdeen Proving Ground, Maryland, as well as teleconferencing with tank trainers at Benning.

Lying in the shadow of Hollywood, the ICT researchers here have been collaborating with film and gaming industry artists to create a diverse palette of programs using immersive technologies in virtual reality.

Playa Vista, a growing tech community in western Los Angeles where ICT is located, has drawn top-level talent and has rapidly earned a reputation as a Silicon Valley extension.

In an area that Howard Hughes once made famous for aircraft innovation, the ICT staff now creates their own innovative projects — from virtual interactive therapy for Soldiers who suffer from PTSD and sexual assault to the development of One World Terrain, a data mapping program that creates virtual landscapes from real world locations.

In the SCOPE platform, Soldiers use a virtual reality headset with built in eye-tracking. Three tank commanders the take part in a simulated training sequence that tests their cognitive abilities and performance under duress. The trainees remain in communication with one another and a central command that helps them develop unit cohesion. Nelson said the platform will eventually expand to four tank commanders training simultaneously to represent a full tank platoon.

SCOPE’s platform will give researchers and trainers the capability to assess a student’s performance by collecting large amounts of trainee data to gain a greater understanding on how trainees make sense of the information available to them and how that informs there decision making. Nelson said the research team has just begun the initial development stages of SCOPE and expects formal testing to start in fiscal year 2020.

While TALKON focused on cognitive task training and communications, the SCOPE program will advance on that model with more sophisticated sensor tracking. The program will also use One World Terrain data, a central pillar of the Army’s synthetic training environment.

In SCOPE, Soldiers will train in a three-dimensional environment using a scenario developed at the Army Armor School in Fort Benning. Soldiers will use a replica joystick and a 3D-printed control box that interfaces with the tank commander’s helmet.

Previously, tank commanders trained with Soldiers who role played the tank driver, gunner and loader. Using the SCOPE or TALKON software, the other crew positions are simulated and will eventually be controlled by artificial intelligence software.

“We’re focusing on the decision-making and the sense-making,” Krum said. “And we’ve kind of reduced the overhead. I don’t need to put a person in the driver’s seat, in the loader’s seat and in the gunner’s seat if I only want to train the tank commander or the platoon commander.”

The combat scenario consists of a mission briefing and different progressive phases that the student tackles within the 3D environment. The trainee uses the intercom to switch back and forth between communicating with the platoon and the company networks.

“You’re basically processing all this information and then you have to decide what to report, update your mental picture,” Krum said. “Do you report information up to higher command, do you tell people in your tank what to do, (or) tell people in the other tanks in your platoon what to do?”

Nelson said the technology yields massive amounts of data that will allow the research team to gain a greater understanding of trainee decisions and how to improve training methods.

“A lot of task training kind of focuses on go or no go. Did they do the right thing or did they not do the right thing?” Nelson said. “What we want to do is drill down a little bit deeper, and find out why trainees made the decisions they did.

“We can go back and look at that data and say, how were they communicating? What were they looking at? Were they under stress? And how they were impacted.”

Eventually, Nelson said, some of the data could be used to add artificial intelligence capabilities where the software can respond to language and voice commands and elicit simulated responses of crew members.

Previously, human trainers conducted performance evaluations of student performance. Krum said the researchers understand the value of human graders but wanted to lend technical support to that task; support that could eventually provide a wealth of improvements to training protocols and accelerate tank commanders training to a more efficient pace, Nelson said.

For now, that future will have to wait, as the project remains in its early stages.