Although surgeons need to frequently review medical images and records during surgery, they’re also in the difficult position of not being able to touch non-sterile objects such as keyboards, computer mice or touchscreens.
Stepping away from the operating table to check a computer also adds time to a procedure. Researchers from Indiana’s Purdue University are addressing this situation by developing gesture-recognition systems for computers, so that surgeons can navigate through and manipulate screen content simply by moving their hands in the air. The system could additionally be used with robotic scrub nurses, also being developed at Purdue, to let the devices know what instruments the surgeon wants handed to them.
The system incorporates a Microsoft Kinect camera (yes, from the gaming system) and specialized algorithms to recognize hand gestures as instructions.
“One challenge will be to develop the proper shapes of hand poses and the proper hand trajectory movements to reflect and express certain medical functions,” said Juan Pablo Wachs, an assistant professor of industrial engineering. “You want to use intuitive and natural gestures for the surgeon, to express medical image navigation activities, but you also need to consider cultural and physical differences between surgeons. They may have different preferences regarding what gestures they may want to use.”
There are also other considerations that the researchers are taking into account in the design of the system. For instance, they don’t want surgeons to be required to wear special types of gloves or colors of clothing in order for their hands to be “read.” The system should also be able to recognize and respond to gestures quickly, and provide confirmation that it understands the request. At the same time, however, it should not accidentally respond to extraneous gestures, such as those made to colleagues in the room.
