Topics >  Innovation > 

A Computer Controlled by Hand Gestures

After serving as a science adviser on Minority Report, John Underkoffler set out to make a real version of the gesture interface he mocked up for the movie.

Everett Collection

Chief John Anderton, using the gesture-controlled computer, with Danny Witwer and Fletcher in Minority Report

Hollywood production designer Alex McDowell was touring MIT's human-computer research lab in 2000 when he met John Underkoffler, then a graduate student at the school. McDowell was researching technological concepts for Minority Report, the 2002 Steven Spielberg film that stars Tom Cruise as a police detective in 2054. "Spielberg wanted a grounded-in-reality film that was anyone's best guess about the future," says McDowell, who was intrigued by Underkoffler's work on a computer interface controlled by hand gestures.

A year later, McDowell invited Underkoffler to Hollywood to be Minority Report's science and technology adviser. Underkoffler built a nonworking mockup of a gesture interface and pitched it to Spielberg and Cruise, who approved it immediately. In the movie, Cruise uses the system to search for a murderer, flipping through images on a computer screen by waving his hands and pointing his fingers. "John is able to do something that is often hard for a scientist," McDowell says. "He enjoys the process of making junk science coherent."

It may have seemed like junk science at the time, but four years after Minority Report hit theaters, Underkoffler co-founded Oblong Industries to produce a real version of the system, now known as the g-speak Spatial Operating Environment. During a recent demonstration at Oblong's Los Angeles headquarters, CEO Kwindla Hultman Kramer donned a glove with a tag tracked by sensors on the ceiling. Twisting and waving his hand, he threw objects across three large projector screens, flipped and zoomed through a city map, then grabbed the map and set it down on a digital table in front of him.

Several corporate clients, including Boeing, are using the g-speak system. This year, Oblong plans to unveil a gesture system designed for conference rooms. It will allow attendees in different locations to pass electronic documents around a digital workspace using a wand. "We could have returned to academia or made more movies," Underkoffler says. "But we decided the only way to make the technology ubiquitous is to develop it in a commercial context."

John Brandon is a contributing editor at Inc. magazine and writes the Tech Trends column in every issue. He also writes the Tech Report column for Inc.com. @jmbrandonbb