Robots to the rescue

Robin Murphy is the master of disaster robots.

After 9/11, she deployed small mobile robots to investigate the rubble.

In the wake of Hurricane Katrina, she sent small, unmanned aerial vehicles (UAVs) to explore buildings on the Gulf Coast--the first time UAVs had been used for emergency structural inspections.

During the Fukushima nuclear emergency, she was part of a team that flew UAVs to determine radiation levels and inspect damage to the reactors.

And during the recent Ebola pandemic, she explored whether robots could help protect health workers from contagion in field hospitals in West Africa and the U.S.

As director of the Center for Robot-Assisted Search and Rescue (CRASAR) at Texas A&M University, Murphy has played an active role in responding to these and more than 20 other disasters, but she's first and foremost a researcher.

Using her experiences in these hot zones as a starting point, she returns to Disaster City, the state-of-the-art disaster reconstruction site at Texas A&M, and probes what works and what doesn't to determine how to most effectively use robots to hasten rescue and recovery efforts.

Based on seminal disaster studies, she estimates that speeding up an emergency response by one day through the use of autonomous robots and unmanned machines can reduce recovery time by 1,000 days, or almost three years.

What follows are five lessons she learned from her experiences and research in disaster robotics:

1. It takes three experts to operate a drone.

Before Hurricane Katrina, no one knew for sure how many people were required to operate an unmanned aerial system in a disaster situation.

"The Army thought you just needed one person to fly, but it turns out that you need three people to do it effectively," she said. "When we started doing the workload analysis--which we were the first to do--we realized one person just can't do it."

Safely flying a drone in close proximity to a damaged building while taking photographs was an impossible task for just one person, she found.

Her insights didn't win her any popularity points, as it made operations more complex and costly to run, but her follow-up research proved her right.

"Our protocol of having one to fly, one to look and one to serve as the safety officer has now become standard practice throughout the world," she said.

Having an accurate sense of the resources needed for operations is critical to getting the useful information that can save lives and speed recovery efforts.

2. It's the data, not the drones.

When UAVs were deployed for the first time after Hurricane Katrina, they were still a relatively new technology. At the time, Murphy and other scientists worried about problems with the machines' power and stability.

"Before Katrina, the hard problem was keeping the platform in the air long enough to be useful," she said. "Now the problems have shifted from aeronautical science to data science."

Our connected world means that experts anywhere can start looking at imagery collected by drones and other sources over the Internet and aid in recovery efforts as soon as data becomes available. That kind of cooperative effort can shave weeks or months off of the process of inspecting buildings or identifying areas of concern.

But what kinds of remotely collected information are helpful? And how can that data be served up so it's useful to experts? Those are still questions fundamental to Murphy's research.

"Katrina showed me the hardest problem is getting the right data to the right person at the right time," she said. "That was true then and is still true."

Better ways are needed to collect, transmit, collate and visualize information. As she put it in her recent TED Talk, "It's not the robots, it's the data."

3. Understanding what experts need to see and when they need to see it.

In the months after Hurricane Katrina--supported by a grant from the National Science Foundation (NSF)--Murphy deployed drones to transmit images of damaged multi-story buildings on the Gulf Coast to structural experts at FEMA, leading engineering firms and universities.

The work shortened the time required to get the buildings examined, but structural engineers found using the drone imagery harder than they expected--showing Murphy that the problems of drone reconnaissance were more complicated than many believed.

"When you look at stuff on the ground, things pop up at you," Murphy said. "With UAV photos, that wasn't happening."

Recently, Murphy has experimented with letting experts direct drones in real time by circling or spotlighting locations on the live video feed. She found this "visual common ground" between the experts and the operator helped significantly.

"Recognizing that the collaboration between humans and machines is only possible when we're all looking at the same feed from the UAV or the ground robot--that's been a big breakthrough," she said.

4. The future of robots is underwater.

Robo-snakes that can comb through rubble and infrastructure-inspecting robotic hummingbirds will play important roles in future emergency response situations.

But autonomous marine vehicles have thus far presented a missed opportunity for roboticists, Murphy believes. She says they represent the next big thing in disaster robotics.

"Eighty percent of the population lives by water," she said. "All your bridges, your levees, your critical infrastructure are underwater and we're relying on manual divers, of which there are very few, to go inspect them."

For these reasons, she and her team have experimented with marine robots, called SARbots--square robots with sensing capabilities like dolphins. They go underwater, use sonar and can be controlled remotely to rapidly investigate water-based risks.

In 2011, after the Tôhoku earthquake and tsunami, Murphy worked with the Japanese Coast Guard to use the underwater robots to inspect and clear ports and look for victims.

"Four hundred miles of coastland totally devastated--twice the amount of coastland devastated by Hurricane Katrina in the United States," Murphy said. "You're talking about your bridges, your pipelines, your ports--wiped out. And if you don't have a port, you don't have a way to get in enough relief supplies to support a population."

In the future, autonomous marine vehicles may prove as valuable as other types of robots now commonly in use in disasters.

5. Smarter algorithms should tame the flood of data.

In the days following the 2015 floods in Texas, Murphy was part of a team that deployed UAVs to inspect the damage.

She found that one 20-minute drone flight generated upwards of 800 photographs, each of which took at least one minute to inspect.

When you consider the possibility of flying dozens of drones in an effort to save lives, such a system just isn't scalable.

"It takes huge amounts of time just to go through the data," she said. "So we need to make improvements in the data science and informatics."

It just so happened that Murphy was running a research program for undergraduate students when the floods occurred. She asked her students to develop applications that could improve the processes of operating UAVs, interpreting their footage and even differentiating areas that need human insight versus those that don't.

The students rose to the challenge, coming up with apps that used computer vision and machine learning to improve the recovery process.

"They took the data as it was happening and created programs that looked for any signs of debris that might suggest a person might have been swept along nearby," Murphy explained.

Other students from the program created improved flight path visualizations to help responders understand where drones had already flown to avoid duplication.

The students earned top marks not only in her class, but also in a competition for recovery innovations.

In addition to student projects, researchers at the University of Maryland used "deep learning" algorithms and anomaly detection techniques to design a system that allowed UAVs to find piles of debris big enough to hide a body. The algorithms offered significant advances over technology considered state-of-the-art.

"The responders said 'anything that helps us zero in on people is better than nothing,'" she said. "If the computer can process 70 percent of the images, that's 70 percent less work for them."

It's far from a solved problem, but it shows how valuable machine learning and computer vision will be for search-and-rescue missions in the future.

Robots haven't saved any lives in disasters yet (although a UAV did help identify a Canadian man after he'd wandered away from his disabled car and gotten lost in the snowy woods), but they're sure to do so in the future.

"The next time you hear about a disaster, look for the robots," Murphy said. "They may be underground, they may be underwater, they may be in the sky, but they should be there. Look for the robots because robots are coming to the rescue."