Cornell researchers have created a low-cost way to enable soft robots to detect a range of physical human interactions, from pats to punches to hugs, by using an off-the-shelf USB camera that “captures” the shadows made by hand gestures on the robot’s skin and physical contact.
Soft robots may not be in touch with human feelings, but they are getting better at feeling human touch.
Cornell researchers have created a low-cost method for soft, deformable robots to detect a range of physical interactions, from pats to punches to hugs, without relying on touch at all. Instead, a USB camera located inside the robot captures the shadow movements of hand gestures on the robot’s skin and classifies them with machine-learning software.
The group’s paper, “ShadowSense: Detecting Human Touch in a Social Robot Using Shadow Image Classification,” published Dec. 17 in the Proceedings of the Association for Computing Machinery on Interactive, Mobile, Wearable and Ubiquitous Technologies. The paper’s lead author is doctoral student Yuhan Hu.
The new ShadowSense technology is the latest project from the Human-Robot Collaboration and Companionship Lab, led by the paper’s senior author, Guy Hoffman, associate professor and the Mills Family Faculty Fellow in the Sibley School of Mechanical and Aerospace Engineering.
“Touch is such an important mode of communication for most organisms, but it has been virtually absent from human-robot interaction. One of the reasons is that full-body touch used to require a massive number of sensors, and was therefore not practical to implement,” Hoffman said. “This research offers a low-cost alternative.”
The technology originated as part of a collaboration with Hadas Kress-Gazit, professor in the Sibley School of Mechanical and Aerospace Engineering, and Kirstin Petersen, assistant professor of electrical and computer engineering, to develop inflatable robots that could guide people to safety during emergency evacuations. Such a robot would need to be able to communicate with humans in extreme conditions and environments. Imagine a robot physically leading someone down a noisy, smoke-filled corridor by detecting the pressure of the person’s hand.
Rather than installing a large number of contact sensors – which would add weight and complex wiring to the robot, and would be difficult to embed in a deforming skin – the team took a counterintuitive approach. In order to gauge touch, they looked to sight.
“By placing a camera inside the robot, we can infer how the person is touching it and what the person’s intent is just by looking at the shadow images,” Hu said. “We think there is interesting potential there, because there are lots of social robots that are not able to detect touch gestures.”
The prototype robot, designed by Petersen’s Collective Embodied Intelligence Lab, consists of a soft inflatable bladder of nylon skin stretched around a cylindrical skeleton, roughly four feet in height, that is mounted on a mobile base. Under the robot’s skin is a USB camera, which connects to a laptop. The researchers developed a neural-network-based algorithm that uses previously recorded training data to distinguish between six touch gestures – touching with a palm, punching, touching with two hands, hugging, pointing and not touching at all – with an accuracy of 87.5 to 96%, depending on the lighting.
The robot can be programmed to respond to certain touches and gestures, such as rolling away or issuing a message through a loudspeaker. And the robot’s skin has the potential to be turned into an interactive screen.
By collecting enough data, a robot could be trained to recognize an even wider vocabulary of interactions, custom-tailored to fit the robot’s task, Hu said.
The robot doesn’t even have to be a robot. ShadowSense technology can be incorporated into other materials, such as balloons, turning them into touch-sensitive devices.
“While the technology has certain limitations, for example requiring a line of sight from the camera to the robot’s skin, these constraints could actually spark a new approach to social robot design that would support a visual touch sensor like the one we proposed,” Hoffman said. “In the future, we would like to experiment with using optical devices such as lenses and mirrors to enable additional form factors.”
In addition to providing a simple solution to a complicated technical challenge, and making robots more user-friendly to boot, ShadowSense offers a comfort that is increasingly rare in these high-tech times: privacy.
“If the robot can only see you in the form of your shadow, it can detect what you’re doing without taking high fidelity images of your appearance,” Hu said. “That gives you a physical filter and protection, and provides psychological comfort.”
The ability to physically interact and understand a person’s movements and moods could ultimately be just as important to the person as it is to the robot.
“Touch interaction is a very important channel in terms of human-human interaction. It is an intimate modality of communication,” Hu said. “And that’s not easily replaceable.”
The paper was co-authored by former research intern Sara Maria Bejarano of the University of Los Andes, Colombia.
Original Article: Soft robots use camera and shadows to sense human touch
The Latest Updates from Bing News & Google News
Go deeper with Bing News on:
- These robot vacuums will always keep your home office tidy, and they're on sale for Cyber Monday
With 2023 on the horizon, you can enlist the help of a robot vacuum to help with tidying your office, and these smart home assistants can help vacuum and even mop the floors around your desk. We've ...
- The best Cyber Monday robot vacuum deals
Amazon, Walmart, Best Buy and other retailers are all holding massive Cyber Monday sales now, with deep discounts on the best robot vacuums of 2022. Some vacuum models are at the lowest price we've ...
- This $99 Cyber Monday robot vacuum deal is absolutely bonkers, so get it while you can
It's pretty rare to find a robot vacuum for less than $100 -- never mind a top-rated, voice-activated one like the Anker Eufy 25c robot vacuum. Head over to the Walmart Cyber Monday sale to get yours ...
- This robot vacuum is $99 in Walmart’s Cyber Monday sale
Dreaming of a cleaner home this Christmas? Walmart's $99 Cyber Monday robot vacuum deal is just the ticket, but it's selling fast, so snap it up now!
- 7 Reasons Why It's Finally Time to Get a Robot Vacuum
If you hate spending hours hunched over a mop bucket or stick vacuum each week, a robot vacuum is a no-brainer!
Go deeper with Google Headlines on:
[google_news title=”” keyword=”robot touch” num_posts=”5″ blurb_length=”0″ show_thumb=”left”]
Go deeper with Bing News on:
- GeekWire – Breaking News in Technology & Business
NASA’s uncrewed Orion capsule successfully executed an engine burn to enter an unusual type of orbit around the moon on the 10th day of the weeks-long Artemis 1 mission, and… Read More Seattle ...
- Marketplace Technology
Benefit from decades of capital markets experience— both from operating our own 28 exchanges, CCP, and CSD, as well as from providing technology to 130+ organizations around the world.
- The Economics of Technology Investments
The 4th Industrial Revolution is moving at a pace so significant that there is no historical precedent. Even within the world of economics, we’re experiencing a new kind of economy: a Technology ...
- The Business of Federal Technology
When you visit our website, we store cookies on your browser to collect information. The information collected might relate to you, your preferences or your device, and is mostly used to make the ...
- World Wide Technology
Co-founded in 1990 by David Steward and Jim Kavanaugh, World Wide Technology, based in St. Louis, Missouri, provides digital strategy, innovative technology and supply chain solutions to large ...
Go deeper with Google Headlines on:
[google_news title=”” keyword=”ShadowSense technology” num_posts=”5″ blurb_length=”0″ show_thumb=”left”]