Research by Antony Orth and Kenneth Crozier (pictured) enables photographers and microscopists to quickly create 3D images without special equipment. (Photo by Eliza Grinnell.)
Mathematical image processing creates a 3D movie of any scene, using just two frames from a stationary camera or microscope
Researchers at theĀ Harvard School of Engineering and Applied SciencesĀ (SEAS) have developed a way for photographers and microscopists to create a 3D image through a single lens, without moving the camera.
PublishedĀ in the journalĀ Optics Letters, this improbable-sounding technology relies only on computation and mathematicsāno unusual hardware or fancy lenses. The effect is the equivalent of seeing a stereo image with one eye closed.
That’s easier said than done, as principal investigatorĀ Kenneth B. Crozier, John L. Loeb Associate Professor of the Natural Sciences, explains.
“If you close one eye, depth perception becomes difficult. Your eye can focus on one thing or another, but unless you also move your head from side to side, it’s difficult to gain much sense of objects’ relative distances,” Crozier says. “If your viewpoint is fixed in one position, as a microscope would be, it’s a challenging problem.”
Offering a workaround, Crozier and graduate student Antony Orth essentially compute how the imageĀ wouldĀ look if it were taken from a different angle. To do this, they rely on the clues encoded within the rays of light entering the camera.
“Arriving at each pixel, the light’s coming at a certain angle, and that contains important information,” explains Crozier. “Cameras have been developed with all kinds of new hardwareāmicrolens arrays and absorbing masksāthat can record the direction of the light, and that allows you to do some very interesting things, such as take a picture and focus it later, or change the perspective view. That’s great, but the question we asked was, can we get some of that functionality with a regular camera, without adding any extra hardware?”
The key, they found, is to infer the angle of the light at each pixel, rather than directly measuring it (which standard image sensors and film would not be able to do). The team’s solution is to take two images from the same camera position but focused at different depths. The slight differences between these two images provide enough information for a computer to mathematically create a brand-new image as if the camera had been moved to one side.
By stitching these two images together into an animation, Crozier and Orth provide a way for amateur photographers and microscopists alike to create the impression of a stereo image without the need for expensive hardware. They are calling their computational method “light-field moment imaging”ānot to be confused with “light field cameras” (like the Lytro), which achieve similar effects using high-end hardware rather than computational processing.
Importantly, the technique offers a new and very accessible way to create 3D images of translucent materials, such as biological tissues.
Biologists can use a variety of tools to create 3D optical images, including light-field microscopes, which are limited in terms of spatial resolution and are not yet commercially available; confocal microscopes, which are expensive; and a computational method called “shape from focus,” which uses a stack of images focused at different depths to identify at which layer each object is most in focus. That’s less sophisticated than Crozier and Orth’s new technique because it makes no allowance for overlapping materials, such as a nucleus that might be visible through a cell membrane, or a sheet of tissue that’s folded over on itself. Stereo microscopes may be the most flexible and affordable option right now, but they are still not as common in laboratories as traditional, monocular microscopes.
“This method devised by Orth and Crozier is an elegant solution to extract depth information with only a minimum of information from a sample,” says Conor L. Evans, an assistant professor at Harvard Medical School and an expert in biomedical imaging, who was not involved in the research. “Depth measurements in microscopy are usually made by taking many sequential images over a range of depths; the ability to glean depth information from only two images has the potential to accelerate the acquisition of digital microscopy data.”
“As the method can be applied to any image pair, microscopists can readily add this approach to our toolkit,” Evans adds. “Moreover, as the computational method is relatively straightforward on modern computer hardware, the potential exists for real-time rendering of depth-resolved information, which will be a boon to microscopists who currently have to comb through large data sets to generate similar 3D renders. I look forward to using their method in the future.”
The new technology also suggests an alternative way to create 3D movies for the big screen.
The Latest Bing News on:
Light-field moment imaging
- The History Of Satellites Explainedon May 11, 2022 at 1:08 pm
Finally, before the sixth decade of the 20th century rolled around, the first satellite would enter orbit, marking the moment a scientific dream became ... Earth held in place by the planet's magnetic ...
- ‘You’re never going to leave this hospital’: How a Cleveland Guardians prospect beat the odds after a freak injuryon May 11, 2022 at 4:00 am
Frank Lopez suffered a broken vertebrae in a diving accident. Now, after a grueling year of rehab, he's back on the mound.
- Migraines can be a pain to manageon May 9, 2022 at 1:37 am
The pain can be severe and throbbing or a pulsing sensation. I would say cases of migraine are very common. If, on average I see 10 patients a day, around 5 or 6 could be having a migraine. We are in ...
- Webb’s coolest instrument captures first staron May 9, 2022 at 1:26 am
The UK’s main contribution to the James Webb Space Telescope, the Mid-Infrared Instrument (MIRI), has now opened its eye to the sky. Above: MIRI components photographed in the South Building clean ...
- Mindfulness app helps teens think positively for weeks after regular use, Lawrence study findson May 8, 2022 at 12:47 pm
The results add to a field with little prior examination and can help shed light on which kinds of tools help teens ... techniques to help them relax and concentrate on the present moment, like ...
- First drone show launched inside a Disney theme parkon May 5, 2022 at 8:22 am
Light, which climaxes with 150 colour-changing drones creating a silhouette of Mickey Mouse's ears. By Caroline Reid and Christian Sylt.
- Tackling the century-long mystery of multiple sclerosison April 28, 2022 at 12:13 pm
A cure for multiple sclerosis remains elusive, but a decade of technological and scientific advances is starting to shed light on possible ... "The MS research field is moving forward with great speed ...
- Indianapolis-based Telix rolls out method for detecting canceron April 22, 2022 at 11:00 am
The imaging agent is the six-year-old company’s first commercial launch, following millions of dollars’ worth of research and clinical tests. It was approved by the U.S. Food and Drug Administration ...
- Indianapolis-based Telix rolls out method for detecting canceron April 22, 2022 at 7:00 am
It’s a moment ... a light going on,” said Dr. Clint Bahler, a urologist at Indiana University Health, who did Doone’s surgery and recommended he get a follow-up scan, using Telix’s imaging ...
The Latest Google Headlines on:
Light-field moment imaging
The Latest Bing News on:
Mathematical image processing
- Tokyo researchers developed a novel lensless cameraon May 11, 2022 at 6:11 pm
The team at Tokyo Tech has proposed a new image reconstruction method using a mathematical algorithm ... thereby avoiding the traditional CNN processing. This means direct reconstruction, with no ...
- Intel Pits New Gaudi2 AI Training Engine Against Nvidia GPUson May 10, 2022 at 12:38 pm
Nvidia is not the only company that has created specialized compute units that are good at the matrix math and tensor processing that underpins AI ...
- The Future of Artificial Intelligence in Manufacturing Industrieson May 8, 2022 at 3:57 am
It is being used as a way to enhance defect detection through sophisticated image processing algorithms ... is built using complex algorithms and mathematical functions. Artificial intelligence ...
- New Neural Processors Address Emerging Neural Networkson May 5, 2022 at 12:04 am
The challenge for an NPU is to be optimized to accelerate the math ... processing have grown from 10s to 100s to 1000s of tera-operations/second (TOPS) of performance in the last several years thanks ...
- Mathematical Modeling in Biopharma: More than the Sum of Its Partson May 4, 2022 at 10:06 am
Medicine makers need to get better at math. Improved mathematical modeling would accelerate the move toward automated processes, cut production times, and reduce costs. Researchers Francesco ...
- Nvidia Says Real-Time Path Tracing Is On the Horizon, But What Is It?on May 1, 2022 at 9:20 am
When you're solving puzzles in Outer Wilds or dodging Margit's giant hammer in Elden Ring, there's a complicated tangle of mathematical ... Other processing, like anti-aliasing, is then applied ...
- Electrical and Computer Engineeringon April 26, 2022 at 7:34 am
The Electrical and Computer Engineering major (designated as a pre-major) is a nationally-recognized program designed for students interested in math and science ... Logic Robotics Power Electronics ...
- Computer Science Courseson March 24, 2022 at 10:20 am
Topics include image processing, shape analysis ... Prerequisites: CSCI 2243 or MATH 2216 or permission of instructor. When you log onto a secure web site, for example to pay a bill from your bank ...
- Electrical & Computer Engineering Course Listingon January 27, 2022 at 10:42 pm
Pre-req: MATH.1320 Calculus II ... This course provides fundamental knowledge for careers in medical image reconstruction. The production and processing of materials into finished products constitute ...
- Imaging Science BSon March 3, 2020 at 1:52 pm
RIT’s imaging science degree combines physics, math, computer science ... Advanced courses in optics, radiometry, color science, imaging systems analysis, digital image processing, detectors, and ...