Plans are in the works to create an online version that anyone can use
For more than twenty years, the software program Photoshop has been the industry standard for seamlessly manipulating images, especially the removal of unwanted items like blemishes, wires and telephone poles. When it comes to adding something to a photo, however, the process is still rather involved. Now, a team from the University of Illinois at Urbana-Champaign (UIUC), led by PhD candidate Kevin Karsch, is presenting a new algorithm at Siggraph Asia 2011 that promises to radically simplify the procedure of believably incorporating new or “synthetic” objects in still shots and the results are quite impressive.
To appreciate the elegance of the UIUC team’s contribution, it’s helpful to understand just what an improvement it is over current techniques. Look at any effects-heavy feature film and you’ll see instances where computer-generated objects, be they giant robots, dinosaurs or futuristic vehicles, were added to previously-shot footage (also called a background plate in the Biz). In order to sell the illusion that the added objects were there in the first place, technicians typically need to gather copious amounts of on-set data (e.g. lens height and focal length, size of on-set objects and direction/color temperature of visible light sources, etc.) from which they recreate a virtual environment. When this is done poorly, it’s a dead giveaway that the added object is fake and was inserted after the original image or footage was shot.
The genius of the Karsch team’s approach lies in what the user doesn’t have to provide in order to create believable additions. (Currently, the software only handles still images, but plans are in the works to tackle film and video sequences before long). Starting with a still photo about which nothing is known, novice users have been able to add virtual subjects in just minutes with results good enough to fool experts. That’s great news for graphic artists, but maybe not so thrilling for folks like crime scene investigators who’ll soon have to doubt the veracity of every object in frame!
To pull off this digital sleight-of-hand, the process is broken up into intuitive stages, the first being the creation of 3D geometry from 2D data. “The main idea is we want to take our picture and we want to get a 3D model of the scene from it,” Karsch explained. “We first apply an automatic estimation procedure which tries to find the boundaries of the scene – it tries to find the walls, the ceiling, the floor.” Since that’s occasionally inaccurate, the software also includes an interface that allows the user to refine the estimate.
“With two vanishing points, we can recover the focal length of the camera, and then using the rest of the input we can also recover a rough representation of the 3D geometry of the scene: where the floors are at, where the walls are at and where the ceiling is at,” Karsch added.
Bookmark this page for “objects to photos” and check back regularly as these articles update on a frequent basis. The view is set to “news”. Try clicking on “video” and “2” for more articles.