Speech Recognition Interfaces Improve Flight Safety

NASA Technology

“Alpha, Golf, November, Echo, Zulu.”

“Sierra, Alpha, Golf, Echo, Sierra.”

“Lima, Hotel, Yankee.”

It looks like some strange word game, but the combinations of words above actually communicate the first three points of a flight plan from Albany, New York to Florence, South Carolina. Spoken by air traffic controllers and pilots, the aviation industry’s standard International Civil Aviation Organization phonetic alphabet uses words to represent letters. The first letter of each word in the series is combined to spell waypoints, or reference points, used in flight navigation. The first waypoint above is AGNEZ (alpha for A, golf for G, etc.). The second is SAGES, and the third is LHY.

For pilots of general aviation aircraft, the traditional method of entering the letters of each waypoint into a GPS device is a time-consuming process. For each of the 16 waypoints required for the complete flight plan from Albany to Florence, the pilot uses a knob to scroll through each letter of the alphabet. It takes approximately 5 minutes of the pilot’s focused attention to complete this particular plan.

Entering such a long flight plan into a GPS can pose a safety hazard because it can take the pilot’s attention from other critical tasks like scanning gauges or avoiding other aircraft. For more than five decades, NASA has supported research and development in aviation safety, including through its Vehicle Systems Safety Technology (VSST) program, which works to advance safer and more capable flight decks (cockpits) in aircraft.

Randy Bailey, a lead aerospace engineer in the VSST program at Langley Research Center, says the technology in cockpits is directly related to flight safety. For example, “GPS navigation systems are wonderful as far as improving a pilot’s ability to navigate, but if you can find ways to reduce the draw of the pilot’s attention into the cockpit while using the GPS, it could potentially improve safety,” he says.

Scott Merritt, an accomplished multi-engine pilot, agrees with Bailey. He was becoming frustrated using GPS while flying his private plane to favorite destinations, he says. “I was spending all my time twisting knobs instead of looking outside and flying the airplane. I thought, ‘There’s got to be a better way to do this.’”

Technology Transfer

Merritt began investigating the feasibility of using a speech recognition interface to speak waypoints into an airplane’s GPS, rather than twisting knobs to arrive at letters. After working with professors at Rensselaer Polytechnic Institute in Troy, New York, Merritt established the beginnings of a business and connected with NASA’s Aviation Safety Program.

By 2005, Merritt founded Pragmasoft Inc. (now VoiceFlight Systems LLC) in Troy, New York, and received a Small Business Innovative Research (SBIR) contract from Langley to demonstrate the effectiveness of using speech recognition technology to enter the waypoints of a flight plan into a GPS.

“It was basically for an interface for a Garmin GPS, which is a very popular system in general aviation,” says Bailey. “VoiceFlight’s approach seemed to have some advantages over what we had been doing at NASA, and it provided an interesting way to move the technology into the commercial market.”

The two main hurdles were to create a speech recognition system and to design that system to interact with the GPS. “We hammered those out through the SBIR,” says Merritt. Afterward, Langley performed flight tests of the technology on NASA aircraft.

Benefits

In 2009, VoiceFlight’s technology became the first Federal Aviation Administration (FAA)-certified speech recognition product for use in civilian aircraft. When Garmin upgraded its 430 and 530 GPS units with a Wide Area Augmentation System (WAAS) capability to improve position accuracy, VoiceFlight obtained a certification so the technology could be used with the 430W and 530W units (the W stands for WAAS). Merritt says the NASA SBIR was instrumental in attracting the investment capital that allowed the company to pursue certification and hire additional staff.

In 2011, VoiceFlight launched and started marketing its WAAS-enabled VSF101 product. Now available for general aviation aircraft of less than 6,000 pounds, the devices are currently in use by pilots flying the Piper Arrow, Beechcraft Baron, and even the high-performance FJ-4 Fury.

Once installed, a pilot simply presses a button on the yoke (the aircraft’s control column), and speaks a command to enter a waypoint, edit a flight plan, erase a plan, enter direct-to destinations, or enter Victor airways. (Victor airways expand to 5 or 10 waypoints, and are similar to using the name of one highway rather than using the names of all the cities along a route.)

Merritt finds the VSF101 makes it easier for a pilot to use GPS, whether in flight or on the ground. Pilots can enter waypoints up to 10 times faster than a pilot using control knobs, and can keep their attention on the sky. Aside from safety, the technology saves time—and as a result could also save fuel. For example, Merritt estimates that if a pilot spends five minutes on the runway entering a flight plan after receiving clearance to depart, the aircraft would burn about a gallon of fuel. “That’s a lot of fuel for a small aircraft. With VoiceFlight, the plan can be entered in 30 seconds,” he says. In the future, Merritt aims to adapt the system for larger aircraft.

According to Bailey, VoiceFlight’s technology is a prime example of research and technology development that can make a real difference in aviation safety. “Unless you can transition it out into a commercial product, then all that research and development will not affect safety. The important thing is that you need to be able to transition the technology—just like VoiceFlight has done.”

VoiceFlight® is a registered trademark of VoiceFlight Systems LLC.

Garmin® is a registered trademark of Garmin Ltd.