Mathematical image processing creates a 3D movie of any scene, using just two frames from a stationary camera or microscope
Researchers at the Harvard School of Engineering and Applied Sciences (SEAS) have developed a way for photographers and microscopists to create a 3D image through a single lens, without moving the camera.
Published in the journal Optics Letters, this improbable-sounding technology relies only on computation and mathematics—no unusual hardware or fancy lenses. The effect is the equivalent of seeing a stereo image with one eye closed.
That’s easier said than done, as principal investigator Kenneth B. Crozier, John L. Loeb Associate Professor of the Natural Sciences, explains.
“If you close one eye, depth perception becomes difficult. Your eye can focus on one thing or another, but unless you also move your head from side to side, it’s difficult to gain much sense of objects’ relative distances,” Crozier says. “If your viewpoint is fixed in one position, as a microscope would be, it’s a challenging problem.”
Offering a workaround, Crozier and graduate student Antony Orth essentially compute how the image would look if it were taken from a different angle. To do this, they rely on the clues encoded within the rays of light entering the camera.
“Arriving at each pixel, the light’s coming at a certain angle, and that contains important information,” explains Crozier. “Cameras have been developed with all kinds of new hardware—microlens arrays and absorbing masks—that can record the direction of the light, and that allows you to do some very interesting things, such as take a picture and focus it later, or change the perspective view. That’s great, but the question we asked was, can we get some of that functionality with a regular camera, without adding any extra hardware?”
The key, they found, is to infer the angle of the light at each pixel, rather than directly measuring it (which standard image sensors and film would not be able to do). The team’s solution is to take two images from the same camera position but focused at different depths. The slight differences between these two images provide enough information for a computer to mathematically create a brand-new image as if the camera had been moved to one side.
By stitching these two images together into an animation, Crozier and Orth provide a way for amateur photographers and microscopists alike to create the impression of a stereo image without the need for expensive hardware. They are calling their computational method “light-field moment imaging”—not to be confused with “light field cameras” (like the Lytro), which achieve similar effects using high-end hardware rather than computational processing.
Importantly, the technique offers a new and very accessible way to create 3D images of translucent materials, such as biological tissues.
Biologists can use a variety of tools to create 3D optical images, including light-field microscopes, which are limited in terms of spatial resolution and are not yet commercially available; confocal microscopes, which are expensive; and a computational method called “shape from focus,” which uses a stack of images focused at different depths to identify at which layer each object is most in focus. That’s less sophisticated than Crozier and Orth’s new technique because it makes no allowance for overlapping materials, such as a nucleus that might be visible through a cell membrane, or a sheet of tissue that’s folded over on itself. Stereo microscopes may be the most flexible and affordable option right now, but they are still not as common in laboratories as traditional, monocular microscopes.
“This method devised by Orth and Crozier is an elegant solution to extract depth information with only a minimum of information from a sample,” says Conor L. Evans, an assistant professor at Harvard Medical School and an expert in biomedical imaging, who was not involved in the research. “Depth measurements in microscopy are usually made by taking many sequential images over a range of depths; the ability to glean depth information from only two images has the potential to accelerate the acquisition of digital microscopy data.”
“As the method can be applied to any image pair, microscopists can readily add this approach to our toolkit,” Evans adds. “Moreover, as the computational method is relatively straightforward on modern computer hardware, the potential exists for real-time rendering of depth-resolved information, which will be a boon to microscopists who currently have to comb through large data sets to generate similar 3D renders. I look forward to using their method in the future.”
The new technology also suggests an alternative way to create 3D movies for the big screen.
The Latest Bing News on:
Light-field moment imaging
- Meet the Little-Known Genius Who Helped Make Pixar Possibleon August 31, 2021 at 3:00 am
Alvy Ray Smith helped invent computer animation as we know it—then got royally shafted by Steve Jobs. Now he’s got a vision for where the pixel will take us next.
- Indian researchers discover 3 supermassive black holes merging together in nearby galaxy | Details hereon August 27, 2021 at 3:54 am
A black hole refers to the region present in spacetime where gravity is so strong that no particles or even electromagnetic radiation for example light can escape from it ...
- Mets' Jacob deGrom Cleared To Throw; Syndergaard To Begin Rehab Assignment Thursdayon August 25, 2021 at 4:41 pm
After his latest MRI came back clean on Wednesday, deGrom was given the green light to play catch, which he was spotted doing at Citi Field earlier ... previous round of imaging showed ...
- Used Celestron Advanced VX Mount with Dual Saddle Plate Von August 25, 2021 at 9:54 am
The GOTO system is the most advanced yet with loads of extra features for visual observations and CCD imaging. So far ... damp time from a sharp tap despite the tube's great moment arm. Field weight ...
- Trauma and PTSD: Aftermaths of the WTC Disaster - An Interview With Bessel A. van der Kolk, MDon August 24, 2021 at 5:01 pm
He was co-principal investigator of the DSM-IV Field Trials for Post Traumatic Stress Disorder. His current research is on how trauma affects memory processes and brain imaging studies of PTSD.
- Kenji Hirasawa – interview: ‘I approach the sitter like a guardian angel, watching over them from far away’on August 18, 2021 at 4:19 am
Kenji Hirasawa (b1982, Tokyo), who currently lives in Japan, studied for a BA in environmental information at Keio University in Japan, before obtaining a master’s in photography at the Royal College ...
- Explore The Cosmos From Your Yard With The Easy-Use Unistellar eVscope eQuinox Telescopeon August 16, 2021 at 6:02 am
Yes, at a glance, it looks like a fairly standard reflector design that collects light via a 4.5-inch mirror. But it only takes a moment to realize ... of 50x and a field of view of 30 arc-minutes ...
- 'Seeing' single cells with soundon August 5, 2021 at 5:00 pm
Whereas the previous imaging ... microscopy field, there was this co-evolution of optical probes and microscopy methods with techniques like two-photon microscopy and light-sheet microscopy ...
- Collaborative Research: Efficient mathematical and computational framework for biological 3D image data retrievalon July 30, 2021 at 5:00 pm
which include magnetic resonance imaging and computed tomography scans in medical imaging, neuroimaging using light-field microscopy in neuroscience, tomography for imaging cells and tissues, and cryo ...
The Latest Google Headlines on:
Light-field moment imaging
The Latest Bing News on:
Mathematical image processing
- Postdoctoral Position in Multiomics Data Integrationon August 28, 2021 at 5:01 am
The Bursar of the Faculty of Science, Masaryk University announces an open competition for the Postdoctoral Position ...
- Intel Architecture Day 2021: Intel Unveils XeSS Image Upscaling Technologyon August 19, 2021 at 5:01 am
The product of that research is Xe Super Sampling, or XeSS, which Intel is pitching as the best solution yet for high image quality and low processing ... s new XMX matrix math units, which ...
- Your Brain Hears Math Differently To Normal Speechon August 16, 2021 at 10:59 am
If you’re an artsy kind of a person, it can sometimes feel like your brain just isn’t wired for stuff like math or science ... is more involved in language processing,” study co-authors ...
- How (and where) the brain analyzes math and language spoken simultaneouslyon August 16, 2021 at 10:00 am
While previous studies examined how these networks process written numbers and words, few looked at the processing of spoken mathematical language. Kulasingham et al. used MEG to measure the brain ...
- Partial Differential Equation Methods for Image Inpaintingon August 15, 2021 at 11:53 pm
This broad approach also gives insight into functional analysis, variational calculus, optimisation and numerical analysis and will appeal to researchers and graduate students in mathematics with an ...
- Deals Reminder: 2021 Google Software Engineering Manager Prep Bundleon August 11, 2021 at 11:49 pm
In this course, you will build MULTIPLE practical systems using natural language processing ... so there are no mathematical prerequisites – just straight-up Python coding.
- Mathematical Methods in Image Reconstructionon August 9, 2021 at 3:02 am
This book describes the state of the art of the mathematical theory and numerical analysis of imaging. The authors survey and provide a unified view of imaging techniques, provide the necessary ...
- Wind Turbine Fault Detection Using Machine Learning And Neural Networkson July 19, 2021 at 9:42 pm
In signal processing based methods, mathematical or statistical operations are performed ... NNs have wide applications in areas such as image processing, pattern recognition, forecasting, ...
- (Junior) Scientific Image Processing - Software Developeron July 9, 2021 at 7:42 am
The ideal candidate has experience in image processing (in any programming language, e.g. C or Matlab), has a strong mathematical background and is able to turn new image processing algorithms ...
- Signal and Image Processing—Graduate Certificateon April 9, 2021 at 9:17 pm
Mathematical modeling and computar programming. Fundamentals of image processing are covered including image representation, geometric transformations, binary image processing, compression, space and ...