Magic Fingers: Digging Into Multi-Touch Technology with Both Hands

: Image by Eliel via Flickr

Creating software to make multi-touch work properly is the hardest part of developing these systems

At Perceptive Pixel‘s offices on Manhattan’s West Side, Jefferson Han stands in front of a megasize multi-touch screen and runs his fingertips across the display. Each finger leaves a trail of colored pixels in its wake, causing the display to look, briefly, like it has been scratched by a set of digital claws.

Han, who founded Perceptive Pixel in 2006 and serves as the company’s chief scientist, next uses his index finger to draw a loop on the 100-inch display. It causes a menu of options to appear on the screen, not all that different from using a mouse to click on a drop-down menu. Of course, Han doesn’t need a mouse, or a keyboard for that matter. He selects one of his menu options, which are arranged in a loop resembling the one he drew to pull up the menu, and away he goes.

Perceptive Pixel’s technology is best known for the role it played in helping cable news network CNN create its “Magic Wall,” a dynamic, on-air graphical representation of voting results during the network’s coverage of the 2008 presidential election. Although Han acknowledges that multi-touch is “interesting” at the level of gadgets such as Apple’s iPhone, the technology comes into its own only when you’re able to manipulate objects on a display screen using both hands.

Creating software to make multi-touch work properly is the hardest part of developing these systems. “The interface is such a paradigm shift,” he says. “You can’t just bolt the software onto existing hardware.” For this reason, Han and his colleagues have essentially created new operating system for touch-sensitive technology that recognizes an infinite number of touch points—ideally, allowing multiple users to manipulate the same display simultaneously.