Lead author Joseph DelPreto controls a “Conduct-A-Bot” drone with his arm muscles.
Photo courtesy of the researchers.
Muscle signals can pilot a robot
CSAIL’s Conduct-A-Bot system uses muscle signals to cue a drone’s movement, enabling more natural human-robot communication.
Albert Einstein famously postulated that “the only real valuable thing is intuition,” arguably one of the most important keys to understanding intention and communication.
But intuitiveness is hard to teach — especially to a machine. Looking to improve this, a team from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) came up with a method that dials us closer to more seamless human-robot collaboration. The system, called “Conduct-A-Bot,” uses human muscle signals from wearable sensors to pilot a robot’s movement.
“We envision a world in which machines help people with cognitive and physical work, and to do so, they adapt to people rather than the other way around,” says Professor Daniela Rus, director of CSAIL, deputy dean of research for the MIT Stephen A. Schwarzman College of Computing, and co-author on a paper about the system.
To enable seamless teamwork between people and machines, electromyography and motion sensors are worn on the biceps, triceps, and forearms to measure muscle signals and movement. Algorithms then process the signals to detect gestures in real time, without any offline calibration or per-user training data. The system uses just two or three wearable sensors, and nothing in the environment — largely reducing the barrier to casual users interacting with robots.
While Conduct-A-Bot could potentially be used for various scenarios, including navigating menus on electronic devices or supervising autonomous robots, for this research the team used a Parrot Bebop 2 drone, although any commercial drone could be used.
By detecting actions like rotational gestures, clenched fists, tensed arms, and activated forearms, Conduct-A-Bot can move the drone left, right, up, down, and forward, as well as allow it to rotate and stop.
If you gestured toward the right to your friend, they could likely interpret that they should move in that direction. Similarly, if you waved your hand to the left, for example, the drone would follow suit and make a left turn.
In tests, the drone correctly responded to 82 percent of over 1,500 human gestures when it was remotely controlled to fly through hoops. The system also correctly identified approximately 94 percent of cued gestures when the drone was not being controlled.
“Understanding our gestures could help robots interpret more of the nonverbal cues that we naturally use in everyday life,” says Joseph DelPreto, lead author on the new paper. “This type of system could help make interacting with a robot more similar to interacting with another person, and make it easier for someone to start using robots without prior experience or external sensors.”
This type of system could eventually target a range of applications for human-robot collaboration, including remote exploration, assistive personal robots, or manufacturing tasks like delivering objects or lifting materials.
These intelligent tools are also consistent with social distancing — and could potentially open up a realm of future contactless work. For example, you can imagine machines being controlled by humans to safely clean a hospital room, or drop off medications, while letting us humans stay a safe distance.
Muscle signals can often provide information about states that are hard to observe from vision, such as joint stiffness or fatigue.
For example, if you watch a video of someone holding a large box, you might have difficulty guessing how much effort or force was needed — and a machine would also have difficulty gauging that from vision alone. Using muscle sensors opens up possibilities to estimate not only motion, but also the force and torque required to execute that physical trajectory.
For the gesture vocabulary currently used to control the robot, the movements were detected as follows:
- stiffening the upper arm to stop the robot (similar to briefly cringing when seeing something going wrong): biceps and triceps muscle signals;
- waving the hand left/right and up/down to move the robot sideways or vertically: forearm muscle signals (with the forearm accelerometer indicating hand orientation);
- fist clenching to move the robot forward: forearm muscle signals; and
- rotating clockwise/counterclockwise to turn the robot: forearm gyroscope.
Machine learning classifiers detected the gestures using the wearable sensors. Unsupervised classifiers processed the muscle and motion data and clustered it in real time to learn how to separate gestures from other motions. A neural network also predicted wrist flexion or extension from forearm muscle signals.
The system essentially calibrates itself to each person’s signals while they’re making gestures that control the robot, making it faster and easier for casual users to start interacting with robots.
In the future, the team hopes to expand the tests to include more subjects. And while the movements for Conduct-A-Bot cover common gestures for robot motion, the researchers want to extend the vocabulary to include more continuous or user-defined gestures. Eventually, the hope is to have the robots learn from these interactions to better understand the tasks and provide more predictive assistance or increase their autonomy.
“This system moves one step closer to letting us work seamlessly with robots so they can become more effective and intelligent tools for everyday tasks,” says DelPreto. “As such collaborations continue to become more accessible and pervasive, the possibilities for synergistic benefit continue to deepen.”
The Latest Updates from Bing News & Google News
Go deeper with Bing News on:
Human-robot communication
- China unveils “Tiangong”: First fully electric humanoid robot capable of running at 6 km/h
China's Beijing Humanoid Robot Innovation Center unveils "Tiangong," a robot capable of human-like running at 6 km/h.
- Astribot S1 AI Humanoid robot unveiled demonstrating its agility, dexterity and accuracy
This week the Astribot S1 humanoid robot was unveiled in Shenzhen, China, marking another significant leap forward in autonomous robotics ...
- University Of New South Wales- Introducing Robots to Baby Care: Exploring the Future of Parenting
More and more children are growing up with robots at home, but their impact on early learning and development is still largely unknown and unregulated.Technology has a profound impact on children.From ...
- This nation of dog lovers can become a nation of robot lovers
One’s hurtling through interstellar space, destination unknown, the other ploughing a lonely path across the sitting room. Both have their journeys programmed, one sucks in data the other dust. While ...
- Robot uses AI to mimic human facial expressions
A robot capable of mimicking human facial expressions has been developed by engineers at Columbia University in the US.
Go deeper with Google Headlines on:
Human-robot communication
[google_news title=”” keyword=”human-robot communication” num_posts=”5″ blurb_length=”0″ show_thumb=”left”]
Go deeper with Bing News on:
Gesture control
- Mikel Arteta and Jurgen Klopp show true colours with class gesture during Prem title race
Mikel Arteta and Jurgen Klopp have both provided long-serving Sky News journalist Jacquie Beltrao with shirts to mark her 32 years with the broadcaster. Beltrao has interviewed the Arsenal and ...
- Emily Henry’s latest novel challenges the romantic ‘grand gesture’
Her latest release, "Funny Story," fittingly finds its heroine in circumstances both tragic and hilarious. Daphne, a children's librarian, has been dumped by her fiancé, Peter, who left her for his ...
- 18-Year-Old Scaler School of Technology Student Develops Innovative Hand Gesture Control for Racing Games
Ayaan Khan, a first-year Computer Science student from Scaler School of Technology, has developed an innovative hand gesture control system for popular PC racing games. By waving fingers in front of a ...
- ‘Khushi Huyi…’ PM Modi’s Heartwarming Gesture Brings Smile To A Young Girl In Chhattisgarh | Lok Sabha 2024
Prime Minister Narendra Modi, who was in Chhattisgarh for a campaign rally paused his speech after a girl holding his photo caught his attention. A clip of the rally surfaced on the Internet in which ...
- Watch: Drunk EasyJet Passenger Loses Control, Throws Punches At Cops And Flight Attendants
The irate passenger got out of his seat and appeared to hit a Turkish police officer and a woman member of the cabin crew.
Go deeper with Google Headlines on:
Gesture control
[google_news title=”” keyword=”gesture control” num_posts=”5″ blurb_length=”0″ show_thumb=”left”]