NASA Helps Drones Take Flight

Subheadline

The agency has helped develop the foundational technology and systems that are enabling remotely piloted aircraft to fill our skies

At the height of the pandemic in 2020, amid stay-at-home orders, social distancing requirements, and uncertainty about how COVID-19 spread, some residents of San Mateo and Contra Costa counties in California began receiving mask and hand-sanitizer deliveries by drone.

The service involved a licensed drone pilot driving a package to its destination by land and then flying it into a customer’s backyard on a small, remote-controlled aircraft, without letting it out of sight.

The experiment offers a taste of a future in which remotely piloted and autonomous aircraft will play a much bigger role in people’s lives, with small drones delivering packages, assisting farmers, and helping to manage disasters, and larger aircraft conducting routine infrastructure maintenance and perhaps eventually transporting parcels between continents, to highlight just a few of the tasks drones will likely perform in coming years.

NASA has been a key player in developing the technologies and systems that will enable aircraft to do these jobs without onboard human assistance. The agency’s work helped make the Contra Costa experiment possible. It has also helped companies develop systems to avoid mid-air collisions, and it has created numerous platforms to enable companies to test their own drone technologies.

The delivery service in Contra Costa, provided by Dive Delivery for a small fee, used an airspace management app for drones developed by Santa Monica, California-based Avision, one of more than 70 companies that worked with NASA on the now-concluded UTM (or Unmanned Aircraft Systems Traffic Management) Project to help small drones use low-altitude airspace regularly, safely, and efficiently, eventually beyond a pilot’s line of sight.

Avision joined the project early, in 2015, flying in multiple flight tests under a non-reimbursable Space Act Agreement with NASA’s Ames Research Center in Silicon Valley, California. The company went on to develop its own drone management software that communicates with the Federal Aviation Administration (FAA) to help small-drone pilots navigate low-altitude airspace.

“Any FAA-certified pilot or hobbyist can use our app to fly a drone in large parts of the country,” said Jim Broniec, Avision’s vice president of business development.

“If there are restrictions in your area, we indicate that through the program,” he said. “If there’s a strong gradient change in the landscape that’s going to affect your craft’s battery life for a particular flight, our architecture sees and reports this information, intuitively using color-coded indicators of battery life for the drones in an organizational fleet.”

Avision is also working with NASA to incorporate drone management into disaster response. In situations where hurricanes or wildfires, for example, have damaged infrastructure, remotely piloted drones can help replace a traditional communications system, according to Joey Mercer, a research engineer at Ames who is leading the Scalable Traffic Management for Emergency Response Operations project known as STEReO.

Working on the project to help emergency responders communicate during fast-moving situations has also informed Avision’s drone management app, Broniec said.

Starting from Scratch

While developing the technology and systems to enable small, low-flying drones to navigate populated areas, NASA has also been working to incorporate large, remotely piloted aircraft into skies that are already crowded with traditional planes with onboard pilots.

In 2012, a colleague approached computer scientist César Muñoz at NASA’s Langley Research Center in Hampton, Virginia, about a new project.

Called UAS in the NAS, the project aimed to help incorporate large (as big as a Cessna aircraft, at least) unmanned aircraft systems (UAS) into the National Airspace (NAS). (The project was concluded in September 2020.)

“We were going to start from scratch,” Muñoz said, recalling the conversation with Langley research engineer Maria Consiglio.

“She said we’d start with the requirements, meet with the standards organizations, and start working from the very basics,” he said.

Indeed, one of the first tasks was to define what “well clear” should mean for large drones. In traditional aircraft, onboard pilots are required by federal regulations to take specific steps to identify other aircraft in their vicinity and avoid mid-air collisions. “See-and-avoid” measures, as they’re called, instruct pilots to keep “well clear” of other aircraft.

For large drones, the team developed a “detect-and-avoid” – instead of “see-and-avoid” – formula to help both conventionally piloted and remotely piloted aircraft achieve the new “well clear” standard without in-person visuals. They had to decide what kind of alerts pilots on the ground should get and what maneuvers the aircraft should perform to keep clear of other aircraft.

(To this day, there is still no agreed-upon definition of “well clear” for small drones.)

Muñoz then wrote a collection of algorithms for a detect-and-avoid system that complied with all the formulas accepted by the standards agencies. Eventually, the code became open source, and the standards organizations listed it as an example implementation of the requirements for detect-and-avoid systems.

“This is not code that developers directly use in their commercial systems,” Muñoz said. “This is code that they can use to test if their systems behave in the way they’re supposed to behave. This is for validating.”

Companies developing their own detect-and-avoid systems can run a series of tests to ensure their software returns the same results as Muñoz’ code, which is called DAIDALUS, for Detect and Avoid Alerting Logic for Unmanned Systems.

DAIDALUS demonstrated that the requirements, which are published by RTCA, a private, not-for-profit organization founded in 1935 as the Radio Technical Commission for Aeronautics, are in fact achievable as they are written and do result in a safe system that enables drone operation.

Remotely Piloted and Global

A key early partner in the DAIDALUS project was San Diego, California-based General Atomics Aeronautical Systems Inc. The company began working with NASA on detect-and-avoid technology in 2014 and soon after started running flight tests at NASA’s Armstrong Flight Research Center in Edwards, California.

The tests involved pilots remotely flying aircraft from the ground.

“We would actually have several displays and algorithms running at the same time – our display, DAIDALUS, and a couple of displays from NASA Ames,” said Brandon Suarez, technical director at General Atomics.

Suarez and his colleagues fed data to all the algorithms, asking pilots to run the scenarios several times with the different displays. “Through all of this human-in-the-loop simulation and flight testing, where we had our prototype system next to DAIDALUS, we learned a tremendous amount,” Suarez said.

“There was a lot of collaboration back and forth between the researchers at Langley – with people like César Muñoz and Maria Consiglio – and our team, from a research perspective,” Suarez said. “We were comparing techniques, comparing algorithms and performance and results, and that all contributed to making a better product.”

General Atomics’ detect-and-avoid system has flown for several years on the company’s MQ-9 (a large remotely piloted aircraft), including for the Air National Guard in an arrangement announced in 2020.  The system will also fly on the company’s newest remotely piloted aircraft, the MQ-9B SkyGuardian.

Another version of the company’s detect-and-avoid system is in the process of getting authorization from the FAA for civil use. That will open up the system and the drones that use it to a much wider range of commercial purposes, such as surveying infrastructure like railroads, pipelines, or high-voltage power lines. The detect-and-avoid-system-equipped aircraft will also likely help map and survey land.

Suarez noted that, from a global perspective, having DAIDALUS as an example algorithm that meets the requirements has given credibility to the standards themselves.

“The global market is really important,” Suarez said. “If you build large aircraft, you have to be able to operate them in Europe, Asia, and around the world. So having recognition from the international community is a critical piece of the technology development.

“Because DAIDALUS was open source and because it was included in the technical standard, we were able to develop a product we can sell and that our customers can operate around the world,” Suarez said. “That’s critical for us.”

On the Ground and in the Cloud

Sean Calhoun, managing director of Beavercreek, Ohio-based CAL Analytics, began working with DAIDALUS as part of the committee that was setting the standards for detect-and-avoid systems. Calhoun and his company were working on a detect-and-avoid system for the Air Force.

“We compared DAIDALUS to some of the algorithms the Air Force was working on, just trying to understand different behaviors and timing and characteristics of the different algorithms under the same type of conditions,” he said.

More recently, CAL Analytics has been working on the civilian side to develop detect-and-avoid services for small drones in the National Airspace.

Smaller drones don’t have the power to carry the most advanced sensors and other equipment to detect and avoid other aircraft, so CAL Analytics’ service shifts the bulk of these functions to ground-based equipment and cloud-based processing.

“Our service doesn’t require any additional equipment on the aircraft,” Calhoun said. “It’s all hosted in the cloud, so we can scale up almost infinitely.”

The detect-and-avoid system pulls in surveillance from other services, like radars and sensors on the ground. Information about the aircraft comes directly from the aircraft itself and from services that – like Avision, the airspace management app maker that enabled drone deliveries – are authorized to share real-time airspace information between drones and the FAA.

“We do all the computation on the ground and push that data to the vehicles for an automatic execution through that UTM network,” Calhoun said.

“A lot of our expertise and knowledge was grounded in and built from the work we did with DAIDALUS and understanding what the appropriate behaviors are,” he added.

Simulations, Live Flights, and Flight Simulators

To ensure they’re safe and reliable, detect-and-avoid systems and other technologies necessary for more widespread use of drones all need to be tested. And tested. And tested some more.

As part of the UAS in the NAS project, NASA engineer Jim Murphy and his team in the Aviation Systems Division at Ames developed a flight test environment that includes both real and simulated aircraft in a virtual space. With the Live Virtual Constructive-Distributed Environment, or LVC-DE for short, developers can be sure their systems work in a virtual environment before moving to partial and then full real-world testing with live aircraft.

“We were developing an infrastructure to test these technologies, starting with simulation and building toward flight tests,” Murphy said. “The detect-and-avoid algorithms don’t know what’s live and what’s virtual because our environment merges all these different data streams.”

General Atomics connected to the LVC-DE when the company was testing its detect-and-avoid system at Armstrong. Through LVC-DE, NASA and General Atomics were able to connect test aircraft at Armstrong with simulation facilities at NASA Ames and test aircraft at a General Atomics facility near Palmdale, California.

Another company, Yorkville, New York-based AX Enterprize, began using the environment in coordination with NASA at Griffiss International Airport, one of seven FAA-designated drone test sites. The women-owned company provides drone flight and testing support, in addition to communications, data, security, and other services.

Initially, the collaboration with NASA studied pilot reactions in the constructive environment. AX pilots worked on equipment at Griffiss that connected to the environment at NASA, where inputs came from Ames and Armstrong.

To physically connect the networks and resources between the sites, AX received additional funding and support from the space agency to get the required permissions and clearances, known as “authority to operate.”

AX improved the LVC-DE, adding a set of tools, models, and live data feeds to help assess drone technologies and to study pilot reaction to the information the system was providing.

The enhanced system could receive and integrate data from actual drones and flights with onboard pilots, as well as simulated flights with simulated pilots, and simulated flights with real pilots. Telemetry and radar alerts provided the pilots with situational awareness, while the ability to link back to a command-and-control post enabled pilots to engage both the live and simulated platforms.

“When the aircraft got within some delta that was considered a conflict, the pilot had to react,” said AX cofounder M.C. Chruscicki. “The software actually gave the command, instructing the pilot to divert to coordinates it provided.”

The combination of simulated and live flights could exercise a detect-and-avoid algorithm appropriately, NASA’s Murphy said. “The algorithm can’t just cheat,” he said, explaining that a poorly designed system might direct a drone to always turn right to avoid a collision, a maneuver that couldn’t work with more aircraft in the airspace.

“You need to have the virtual traffic for safety and for cost effectiveness, so you don’t have all the vehicles live and flying,” he said.

Eventually, NASA stopped updating the LVC-DE and made the system open source for anyone to use. AX continued to use the platform and add enhancements. The company has used the environment in detect-and-avoid work with Johns Hopkins University’s Applied Physics Lab and the Air Force Research Lab.

The enhanced LVC-DE is also part of the more general flight test services the company offers.

“That’s exactly what we were hoping industry would do,” said NASA’s Murphy. “We wanted people to take the technology that we built and then use it, because it provides a mechanism for essentially what we used it for: bringing many different data sources into one interface, and then distributing those data sources to whatever system wants it.”

Beyond NASA’s Line of Sight

Remotely piloted and autonomous flights are increasingly aiding long-duration scientific research, firefighting, aerial photography, land and crop surveys, pipeline monitoring, border protection, and disaster management.

NASA has helped to lay much of the groundwork for these capabilities and for a future in which remotely piloted and autonomous aircraft will play a much larger role in our lives, whether seen or unseen.

Once they can safely integrate into the airspace, NASA expects drones to transform commercial aviation, likely spurring economic growth and job creation.

“We like to think that we help give these folks a forum in which to figure out what these systems need to do,” said Joey Rios, chief technologist in the Aviation Systems Division at Ames, discussing his work with the drone traffic management app maker Avision.

The hope is that companies can then go off and commercialize the knowledge they’ve gained, he said.