The laser scanning process in action. Single photons that travel through the foam, bounce off the “S,” and back through the foam to the detector provide information for the algorithm’s reconstruction of the hidden object. (Image credit: Stanford Computational Imaging Lab)
Using a new algorithm, Stanford researchers have reconstructed the movements of individual particles of light to see through clouds, fog and other obstructions.
Like a comic book come to life, researchers at Stanford University have developed a kind of X-ray vision – only without the X-rays. Working with hardware similar to what enables autonomous cars to “see” the world around them, the researchers enhanced their system with a highly efficient algorithm that can reconstruct three-dimensional hidden scenes based on the movement of individual particles of light, or photons. In tests, detailed in a paper published Sept. 9 in Nature Communications, their system successfully reconstructed shapes obscured by 1-inch-thick foam. To the human eye, it’s like seeing through walls.
“A lot of imaging techniques make images look a little bit better, a little bit less noisy, but this is really something where we make the invisible visible,” said Gordon Wetzstein, assistant professor of electrical engineering at Stanford and senior author of the paper. “This is really pushing the frontier of what may be possible with any kind of sensing system. It’s like superhuman vision.”
This technique complements other vision systems that can see through barriers on the microscopic scale – for applications in medicine – because it’s more focused on large-scale situations, such as navigating self-driving cars in fog or heavy rain and satellite imaging of the surface of Earth and other planets through hazy atmosphere.
Supersight from scattered light
In order to see through environments that scatter light every-which-way, the system pairs a laser with a super-sensitive photon detector that records every bit of laser light that hits it. As the laser scans an obstruction like a wall of foam, an occasional photon will manage to pass through the foam, hit the objects hidden behind it and pass back through the foam to reach the detector. The algorithm-supported software then uses those few photons – and information about where and when they hit the detector – to reconstruct the hidden objects in 3D.
This is not the first system with the ability to reveal hidden objects through scattering environments, but it circumvents limitations associated with other techniques. For example, some require knowledge about how far away the object of interest is. It is also common that these systems only use information from ballistic photons, which are photons that travel to and from the hidden object through the scattering field but without actually scattering along the way.
“We were interested in being able to image through scattering media without these assumptions and to collect all the photons that have been scattered to reconstruct the image,” said David Lindell, a graduate student in electrical engineering and lead author of the paper. “This makes our system especially useful for large-scale applications, where there would be very few ballistic photons.”
In order to make their algorithm amenable to the complexities of scattering, the researchers had to closely co-design their hardware and software, although the hardware components they used are only slightly more advanced than what is currently found in autonomous cars. Depending on the brightness of the hidden objects, scanning in their tests took anywhere from one minute to one hour, but the algorithm reconstructed the obscured scene in real-time and could be run on a laptop.
“You couldn’t see through the foam with your own eyes, and even just looking at the photon measurements from the detector, you really don’t see anything,” said Lindell. “But, with just a handful of photons, the reconstruction algorithm can expose these objects – and you can see not only what they look like, but where they are in 3D space.”
Space and fog
Someday, a descendant of this system could be sent through space to other planets and moons to help see through icy clouds to deeper layers and surfaces. In the nearer term, the researchers would like to experiment with different scattering environments to simulate other circumstances where this technology could be useful.
“We’re excited to push this further with other types of scattering geometries,” said Lindell. “So, not just objects hidden behind a thick slab of material but objects that are embedded in densely scattering material, which would be like seeing an object that’s surrounded by fog.”
Lindell and Wetzstein are also enthusiastic about how this work represents a deeply interdisciplinary intersection of science and engineering.
“These sensing systems are devices with lasers, detectors and advanced algorithms, which puts them in an interdisciplinary research area between hardware and physics and applied math,” said Wetzstein. “All of those are critical, core fields in this work and that’s what’s the most exciting for me.”
The Latest Updates from Bing News & Google News
Go deeper with Bing News on:
Imaging system
- KA Imaging’s Flat Panel Sight™ Receives Medical Device Licence from Health Canadaon April 14, 2021 at 2:36 am
KA Imaging’s Flat Panel Sight™ has received a Canadian Medical Device Licence from Health Canada. With the licence, the X-ray detector can be officially sold in Canada.
- Rosen Law Firm Files Securities Class Action Lawsuit Against 3D Systems Imaging Ltd. - DDDon April 14, 2021 at 1:13 am
The lawsuit seeks to recover damages for 3D Systems investors under the federal securities laws. To join the 3D Systems class action, go to or call Phillip Kim, Esq. toll-free at 866-767-3653 or ...
- Imaging system allows users to see UV and visible light simultaneouslyon April 13, 2021 at 10:03 pm
A passive down-conversion imaging system created by University of Wisconsin-Madison engineers allows humans to peer into the UV range while still viewing the visible spectrum, with no batteries ...
- Baseline brain imaging should be considered in most metastatic kidney cancer patients, study suggestson April 13, 2021 at 9:10 pm
"Brain imaging is routinely obtained for kidney cancer patients with symptoms that suggest central nervous system (CNS) metastases, but none of the patients with brain metastases included here were ...
- Phase One Introduces Next-Generation Aerial Systems: PAS 280 & PAS 150on April 13, 2021 at 2:33 pm
Phase One, a developer of digital imaging technologies, has introduced its next-generation Phase One Aerial Systems (PAS). Among its many enhancements, PAS is now supported by the new iX Controller ...
Go deeper with Google Headlines on:
Imaging system
Go deeper with Bing News on:
Sensing systems
- Off road: Autonomous driving's new frontier requires a new kind of sensoron April 14, 2021 at 4:04 am
GN: What's the biggest sensing capability hurdle currently to broad adoption of autonomous vehicles? Tarik Bolat: Current ADAS features enabled by lidar and camera-based systems for autonomous ...
- Sensing technology trial to improve air quality in Staffordshireon April 14, 2021 at 2:55 am
The vibrations influence the light travelling in the fibre and these disturbances are detected by Fotech’s Distributed Acoustic Sensing, (DAS) system. Monitoring of the fibre can determine the ...
- Kuo: iPhone 14 Pro to feature significant camera upgrades, including 48-megapixel sensor and 8K video recordingon April 14, 2021 at 12:31 am
In addition to claiming that the iPhone mini screen size is no more, Kuo detailed some impressive camera specs for the 2022 iPhones. He reports that the standard rear cameras on the iPhone 14 Pro and ...
- Lockheed Martin pitching mid-size satellite bus to DoD for remote sensingon April 13, 2021 at 5:00 pm
The mid-size satellite bus is aimed at the military market and would be interoperable with military weapons systems such as fighter ... could be made with remote sensing payloads — such as ...
- Mobvoi’s TicWatch GTH has an Apple Watch look and a skin temperature sensor for $79.99on April 13, 2021 at 11:51 am
Mobvoi has generally focused on Wear OS devices in the past, providing a variety of options for Google’s troubled wearable operating system, but the TicWatch ... SpO2 sensors and an electrodermal ...
Go deeper with Google Headlines on:
Sensing systems