A new system combines video of a scene from multiple cameras to created 4D visualizations. This makes it possible to view the scene from a variety of angles, place people into new scenes or, in this case, remove someone from a scene.
Carnegie Mellon Approach Requires Neither Studio nor Specialized Cameras
Researchers at Carnegie Mellon University have demonstrated that they can combine iPhone videos shot “in the wild” by separate cameras to create 4D visualizations that allow viewers to watch action from various angles, or even erase people or objects that temporarily block sight lines.
Imagine a visualization of a wedding reception, where dancers can be seen from as many angles as there were cameras, and the tipsy guest who walked in front of the bridal party is nowhere to be seen.
The videos can be shot independently from variety of vantage points, as might occur at a wedding or birthday celebration, said Aayush Bansal, a Ph.D. student in CMU’s Robotics Institute. It also is possible to record actors in one setting and then insert them into another, he added.
Bansal and his colleagues worked around that limitation by using convolutional neural nets (CNNs), a type of deep learning program that has proven adept at analyzing visual data. They found that scene-specific CNNs could be used to compose different parts of the scene.
The CMU researchers demonstrated their method using up to 15 iPhones to capture a variety of scenes — dances, martial arts demonstrations and even flamingos at the National Aviary in Pittsburgh.
The Latest Updates from Bing News & Google News
Go deeper with Bing News on:
- X-ray computed tomographyon February 25, 2021 at 1:32 am
This Primer on X-ray computed tomography explores the different experimental configurations for three-dimensional data acquisition as well as the fundamentals of three-dimensional data reconstruction, ...
- Revive the map: 4D building reconstruction with machine learningon February 24, 2021 at 11:13 am
In 3D/4D city modeling applications based on historical data, the lack of building heights is a major obstacle for accurate space representation, analysis, visualization, or simulations.
- Global Ultrasound Workstation Market Size-Forecasts to 2026on February 23, 2021 at 3:29 pm
The market is highly driven by factors such as increasing incidence of ovarian cancer and lifestyle disorders, rising number of product launch (of 3D and 4D ultrasound systems) strategies by the ...
- Maxon Announces 2021 Lineup of 3D, VFX and Motion Graphics Special Eventson February 23, 2021 at 10:21 am
Maxon has announced its extensive lineup of 2021 events for content creators working in 2D, 3D, motion design, VFX and visualization. In addition to participating in mainstay industry events like ...
- 4D Fun Announces Release of the Brown Family Diaries VRon February 11, 2021 at 7:20 am
"A journey into the gamification of storytelling and music visualization," says Paul Vowell, CEO of 4D Fun "If you remember when MTV launched with 'Video Killed the Radio Star'… This is Music ...
Go deeper with Google Headlines on:
Go deeper with Bing News on:
Convolutional neural nets
- Huawei Uganda offers free AI trainingon February 25, 2021 at 7:19 am
Huawei Uganda has announced a free Artificial Intelligence (AI) training course for ICT professionals from its online ICT academy. It started on 22 February. The AI supplements existing Huawei ...
- AAAI 2021: Accelerating the impact of artificial intelligenceon February 24, 2021 at 4:43 pm
The purpose of the Association for the Advancement of Artificial Intelligence, according to its bylaws, is twofold. The first is to promote research in the area of AI, and the second is to promote the ...
- Prediction of drug–target binding affinity using similarity-based convolutional neural networkon February 24, 2021 at 7:36 am
Identifying novel drug–target interactions (DTIs) plays an important role in drug discovery. Most of the computational methods developed for predicting DTIs use binary classification, whose goal is to ...
- Automatic segmentation of inner ear on CT-scan using auto-context convolutional neural networkon February 23, 2021 at 5:36 am
Temporal bone CT-scan is largely used for diagnostic and surgical preplanning in diseases involving the inner ear such as hearing loss and balance disorders 1. In routine practice, this technique ...
- What Is Geometric Deep Learningon February 23, 2021 at 12:30 am
Bronstein's paper highlighted how research in many scientific fields such as computational social science, sensors network, physics, and healthcare calls for exploring non-Euclidean data.