Menu Dropdown

To Train Without Pain

Videogame-Based Tech Creates Digital Scenarios So Real, You Might Need a Helmet

In a motion capture system, the movements of performers are captured by a ring of cameras. Onscreen, digital representations called "avatars" follow the motions of each live performer in real time.

In a darkened warehouse in El Paso, Texas, actors in black unitards studded with white markers tromped and tiptoed inside a ring of infrared cameras. Their performance was being captured on a computer to create avatars – digital versions of themselves that will appear in virtual training sessions for the U.S. Army.

Soldiers will use the avatars to learn how to operate the Patriot Air and Missile Defense system. They’ll work in a virtual landscape that is so convincing, the trainees may find themselves shaking imaginary sand out of their boots.

“Motion capture makes it so realistic, which, in turn, makes our training that much more immersive,” said Luis Ruiz, 3D Team Lead for Raytheon Intelligence, Information and Services. “We learn better by seeing.”

Inside the training simulations, soldiers move their avatars across video landscapes and manipulate digital versions of real-life equipment. It’s not unlike a multi-player videogame, and for good reason. The software is built around a videogame engine, creating a so-called “serious game” – a product that uses videogame science to create tools for training and other real-world simulations.

Performers enact a training scenario in a motion-capture session. Their movements will be replicated by digital "avatars."

Because the underlying software is based on a videogame engine, Raytheon can use some of the same off-the-shelf tools as game makers, including the game controller trainees use as they learn to operate a virtual Patriot battery. Talk about convenience:  The trainees already know how to work the controls.

“They've been using game consoles since they were kids. It’s second nature to them,” said Ruiz. “Because the training ‘drives’ the same way as videogames they’re familiar with, it really engages the student.”

Raytheon is one of the world’s top training companies, managing military training at 600 locations around the globe. It trains almost every U.S. Army soldier and has trained more than 4,000 air traffic controllers.

The company also has deep expertise in the use of videogame-based technology, from simulators for the Javelin missile system to a game that teaches sailors how to respond to onboard emergencies.

In the past, Raytheon built avatars and other digital characters by hand, a time-consuming and costly process.

Motion capture is faster and much less expensive. Raytheon’s training and logistics group in El Paso now uses motion capture on all of its projects requiring avatars, producing life-like effects at less cost for both the company and the customer.

“We're fully immersing our students in an environment that they can explore,” Ruiz said. “So our avatars need to be able to move forwards and backwards, and navigate their surroundings. We also need to create custom animations so they can interact with the Patriot equipment, such as working on a panel or circuit breaker.”

The El Paso facility also has a “face capture” station and a sound booth. In one recent recording session, dozens of markers dotted a young woman’s face as eight cameras captured every nuance of expression as she recited a script. The captured footage is used to create a virtual instructor to guide students through the training.

“The facial capture enhances the human element, which can a lot of times be a disconnect for the student,” Ruiz said. “It keeps the student engaged when you communicate more to them.”

Judging by student reactions, the engagement is real. “The feedback we’ve gotten?” said Patricia Vazquez, Raytheon IIS multimedia production manager. “They love it. We’ve heard they line up to go to the classroom.”

Last Updated: 10/14/2015

Back to Top