Biofeedback Loops Aim to Enhance Combat, Sports Training

NASA Technology

Steadying an undulating putting green or a wandering computer cursor with your mind alone might sound far-fetched, but NASA has had this technology for years. Now a company is applying these surreal capabilities to virtual reality military training and, soon, sports training and more.

The Space Agency has always monitored how its astronauts and pilots react to their environment, whether keeping track of vital signs during a spacewalk or watching for physical signs of stress during training. Observers often use software to try to make sense of that data, looking to make inferences about the subject’s physiological and psychological state based on indicators like heart rate, skin temperature, and brain waves.

In the late 1990s, a team of researchers at Langley Research Center, concerned with pilots’ distractibility during flight, came up with a new index for measuring engagement by observing different brainwave outputs. Then they put a new twist on physiological monitoring: they let subjects see their own attention index—and try to control it.

The team found that subjects who were shown their engagement level, as determined by brainwave output from moment to moment while performing a task, were, in fact, able to learn to control it, never becoming too stressed nor losing focus. They were responding to what’s known as a biofeedback loop. What’s more, the biofeedback-trained subjects outperformed peers on the same tasks weeks later, even in the absence of feedback.

“We realized we had some unique ideas that could be spun off into inventions based on biofeedback,” says Alan Pope, the now-retired researcher who led the work at Langley. So he and his team started inventing.

First came a mechanical putting green that stayed still only when the putter’s mind was still. Based on the premise that golfers putt best with a quiet, empty mind, the course physically undulated, with the hole dilating and closing and the sighting laser swinging back and forth, until electrodes on the golfer’s forehead indicated strong alpha brainwave output, which is associated with an idling mind and meditation. The golfer had to mentally still the course, then putt.

A video game the team produced was meant to induce a different state of mind. In a configuration developed for the Nintendo Wii, Pope and fellow Langley researcher Chad Stephens worked out a way to compromise the signal from the remote control until the player’s brainwave output indicated alertness and engagement. A camera on the remote used lights from an LED array under the screen as points of reference to orient itself—and thus the cursor—in space. The lights would twinkle on and off in a pattern that caused the cursor to circle whatever target the control was pointed at until the player’s neural output showed dominant levels of high-frequency beta waves and suppression of lower-frequency alpha and theta waves. Then the lights and the cursor would stay still, letting the user line up a shot.

In each case, Pope says, “there’s an incentive to produce the goal brainwave pattern.”

The group called the video game MindShift and the putting green Zeroing Out Negative Effects (ZONE). For several years, Langley has sought commercial partners to bring these innovations to market. NASA did license an earlier version of the technology to a company that applied it to video game and DVD platforms for attention, learning, behavior, and memory improvement (Spinoff 2003). And at least two other companies have been inspired by the team’s publicly available research to develop their own similar products for educational and training purposes and to address symptoms of attention deficit hyperactivity disorder (Spinoff 2013 and page 94 of this book).

Technology Transfer

J&F Alliance Group Inc. of Hampton, Virginia, provides logistics and information technology consulting and services to the military, other government agencies, and private entities, and the company is breaking into the business of virtual and mixed reality technology for training and other applications. When the company caught wind of Langley’s biofeedback technology in late 2016, it seemed like a possible way to enrich its emerging digital reality products.

“One of our partners had a friend at NASA working with Alan Pope,” says JarMarcus King, the company’s chief operations officer. The company and Pope’s team met a couple of times to check out each other’s technology.

“The future of video games and sports training is connected to virtual reality and augmented reality,” says Pope. “We wanted to bring the physiological into the virtual reality world.”

J&F licensed the ZONE and MindShift technologies in the summer of 2017 with an intent to commercialize both. Pope, now a NASA distinguished research associate, is acting as a consultant to the company.

“This technology can truly make a difference in our work force and in people’s lives,” says Falana Dula-King, CEO of J&F Alliance.

The team quickly homed in on an application with a ready market: virtual reality for military weapons training, with likely spinoffs for law enforcement and security personnel.

Benefits

The Department of Defense Special Operations Command, for one, has put out a call for virtual reality training programs, calling this one of the joint military command’s highest priorities, King says.

“We made the assessment that this application was the most marketable at this point,” says Pope. “There are customers waiting for it.”

In the summer of 2018, with the help of John Muñoz, a human-computer interaction PhD student funded by the National Institute of Aerospace, the company focused on developing a first-person-shooter-style virtual reality training based on Pope’s biofeedback research. The system, which J&F planned to bring to market by winter, would incorporate several off-the-shelf components—a virtual reality headset, a brainwave-reading headband, a chest-strap heart rate monitor, and an Airsoft pistol that shoots with a realistic feel.

“In the past, we were playing catch-up with video game technology,” Pope says. “This is the next logical step into virtual reality.”

Like MindShift, the system would train the user to enter a state of heightened engagement and calmness by making targets easier to hit in response to higher outputs of beta waves and suppression of lower-frequency brainwaves. It would also favor proper breathing patterns, as established by Navy SEAL guidelines.

“Improving mental readiness overall can significantly reduce avoidable incidents in military situations, both in training and real combat scenarios,” says King, adding, “The data collected during the training sessions produces a detailed report of the trainee’s responses to the different stressors and events simulated, improving the training personalization.”

The company also plans to repurpose the trainer, which it calls the Biocyber Physical System, or BioPhyS, for use by law enforcement and security firms.

And King says they still plan to commercialize the ZONE putting green and pursue other sports training applications. Further possibilities lay in healthcare and education, where, Pope says, “biofeedback loops have shown success in the past but now can potentially be enhanced with inexpensive virtual reality technologies.”

In this way, the company also hopes to continue to capitalize on Pope’s deep knowledge of biofeedback systems by bringing him on as chief scientist. “Part of our goal is to turn his passion into something real and bring it to market,” King says.

“My investment in this is to see my ideas and inventions and patents get out there and contribute in some way,” adds Pope.