DARPA Virtual Eye Lets Emergency Responders ‘See’ What They Can’t See

The best thing to give first responders before they enter a smoky room or the site of a chemical spill, or to soldiers before they enter a hostile bunker, is a picture of what’s inside. Exploring an unsecured space in 3D from a safe distance could be a matter of life or death.

A team at the Defense Advanced Research Projects Agency (DARPA) is helping to make that possible by funding efforts to combine powerful 3D imaging software, GPUs and pretty much any camera to generate a VR view of a potentially dangerous environment.

The intention of the system, dubbed Virtual Eye, is to let soldiers, firefighters and search and rescue personnel walk around a room or other enclosed area — virtually — before entering, enabling them to scope out the situation while avoiding potential dangers.

“The question for us is, can we do more with the information we have?” says Trung Tran, the program manager leading Virtual Eye’s development for DARPA’s Microsystems technology office. “Can we extract more information from the cameras we’re using today?”

The answer is a resounding yes. Even more impressive: any camera will suffice. Tran says the system is “camera agnostic.”

How Virtual Eye Works

Emergency responders who have determined that a room is too dangerous to enter without more information would insert drones or robots. These would wield or position two cameras in different parts of the room. The Virtual Eye software then fuses the separate images into a 3D virtual reality view in real time by extrapolating the data needed to fill in any blanks.

So, in a firefighting scenario, this could enable firefighters to look into a room, determine where a child in peril is located — perhaps behind a bed or other piece of furniture — see where flames are active, and plan their approach.

On the battlefield, soldiers could use Virtual Eye to detect if they might be ambushed, if an explosive is in a room or if a boobytrap awaits them.

“Understanding what we see is critical to making the right decisions in the battlefield,” says Tran. “We can create a 3D image by looking at the differences between two images, understanding the differences and fusing them together.”

A Real-Time 3D Video Experience

The Virtual Eye system under development relies on NIVIDIA mobile Quadro and GeForce GTX GPUs to stitch the images together, and to extrapolate 3D data from the images captured by the cameras. Tran says the NVIDIA GPUs were chosen because they’re small enough to fit into a laptop.

Learn more: DARPA Virtual Eye Lets Emergency Responders ‘See’ What They Can’t See