Lead author Joseph DelPreto controls a “Conduct-A-Bot” drone with his arm muscles.
Photo courtesy of the researchers.
Muscle signals can pilot a robot
CSAIL’s Conduct-A-Bot system uses muscle signals to cue a drone’s movement, enabling more natural human-robot communication.
Albert Einstein famously postulated that “the only real valuable thing is intuition,” arguably one of the most important keys to understanding intention and communication.
But intuitiveness is hard to teach — especially to a machine. Looking to improve this, a team from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) came up with a method that dials us closer to more seamless human-robot collaboration. The system, called “Conduct-A-Bot,” uses human muscle signals from wearable sensors to pilot a robot’s movement.
“We envision a world in which machines help people with cognitive and physical work, and to do so, they adapt to people rather than the other way around,” says Professor Daniela Rus, director of CSAIL, deputy dean of research for the MIT Stephen A. Schwarzman College of Computing, and co-author on a paper about the system.
To enable seamless teamwork between people and machines, electromyography and motion sensors are worn on the biceps, triceps, and forearms to measure muscle signals and movement. Algorithms then process the signals to detect gestures in real time, without any offline calibration or per-user training data. The system uses just two or three wearable sensors, and nothing in the environment — largely reducing the barrier to casual users interacting with robots.
While Conduct-A-Bot could potentially be used for various scenarios, including navigating menus on electronic devices or supervising autonomous robots, for this research the team used a Parrot Bebop 2 drone, although any commercial drone could be used.
By detecting actions like rotational gestures, clenched fists, tensed arms, and activated forearms, Conduct-A-Bot can move the drone left, right, up, down, and forward, as well as allow it to rotate and stop.
If you gestured toward the right to your friend, they could likely interpret that they should move in that direction. Similarly, if you waved your hand to the left, for example, the drone would follow suit and make a left turn.
In tests, the drone correctly responded to 82 percent of over 1,500 human gestures when it was remotely controlled to fly through hoops. The system also correctly identified approximately 94 percent of cued gestures when the drone was not being controlled.
“Understanding our gestures could help robots interpret more of the nonverbal cues that we naturally use in everyday life,” says Joseph DelPreto, lead author on the new paper. “This type of system could help make interacting with a robot more similar to interacting with another person, and make it easier for someone to start using robots without prior experience or external sensors.”
This type of system could eventually target a range of applications for human-robot collaboration, including remote exploration, assistive personal robots, or manufacturing tasks like delivering objects or lifting materials.
These intelligent tools are also consistent with social distancing — and could potentially open up a realm of future contactless work. For example, you can imagine machines being controlled by humans to safely clean a hospital room, or drop off medications, while letting us humans stay a safe distance.
Muscle signals can often provide information about states that are hard to observe from vision, such as joint stiffness or fatigue.
For example, if you watch a video of someone holding a large box, you might have difficulty guessing how much effort or force was needed — and a machine would also have difficulty gauging that from vision alone. Using muscle sensors opens up possibilities to estimate not only motion, but also the force and torque required to execute that physical trajectory.
For the gesture vocabulary currently used to control the robot, the movements were detected as follows:
- stiffening the upper arm to stop the robot (similar to briefly cringing when seeing something going wrong): biceps and triceps muscle signals;
- waving the hand left/right and up/down to move the robot sideways or vertically: forearm muscle signals (with the forearm accelerometer indicating hand orientation);
- fist clenching to move the robot forward: forearm muscle signals; and
- rotating clockwise/counterclockwise to turn the robot: forearm gyroscope.
Machine learning classifiers detected the gestures using the wearable sensors. Unsupervised classifiers processed the muscle and motion data and clustered it in real time to learn how to separate gestures from other motions. A neural network also predicted wrist flexion or extension from forearm muscle signals.
The system essentially calibrates itself to each person’s signals while they’re making gestures that control the robot, making it faster and easier for casual users to start interacting with robots.
In the future, the team hopes to expand the tests to include more subjects. And while the movements for Conduct-A-Bot cover common gestures for robot motion, the researchers want to extend the vocabulary to include more continuous or user-defined gestures. Eventually, the hope is to have the robots learn from these interactions to better understand the tasks and provide more predictive assistance or increase their autonomy.
“This system moves one step closer to letting us work seamlessly with robots so they can become more effective and intelligent tools for everyday tasks,” says DelPreto. “As such collaborations continue to become more accessible and pervasive, the possibilities for synergistic benefit continue to deepen.”
The Latest Updates from Bing News & Google News
Go deeper with Bing News on:
- Teaching Robots Workplace Etiquetteon April 16, 2021 at 5:00 pm
In their paper on human-robot collaborative tasks (PDF), they cover the three possible models one can use for this kind of interaction: there can be no communication (‘silent’), the ...
- HuRT-C offers capstone cadets, FAST a first-hand familiarity with future military unmanned systemson April 15, 2021 at 2:57 pm
It was an overcast day on Range 15, at the Urban Assault Course at Camp Shea, but a beam of light broke through upon the tactical environment as U.S. Military Academy cadets shined in various forms ...
- HuRT-C offers capstone cadets, FAST a first-hand familiarity with future military unmanned systemson April 14, 2021 at 5:00 pm
but a beam of light broke through upon the tactical environment as U.S. Military Academy cadets shined in various forms while working with unmanned systems during the Human-Robot Teaming ...
- Almeida’s TechnoModeration Book Is Publishedon April 13, 2021 at 5:00 pm
Luis Almeida, associate professor of communications at Lee University ... Cleveland Daily Banner and includes his model, the “Human Robot Cycle,” on how a person’s technological equilibrium ...
- YouTuber Teaches Boston Dynamics' Robot Dog to Pee Beer on Commandon April 13, 2021 at 8:44 am
Let this represent the honeymoon stage of human-robot relations. “It’s an engineering masterpiece, I wanna make it piss beer into a cup,” he says in the video below. So far more than 6.8 ...
Go deeper with Google Headlines on:
Go deeper with Bing News on:
- Goodwill gesture: Tata, RIL, JSW among firms pitching in to fix Oxygen supply gapon April 21, 2021 at 9:42 am
With many Covid-19 patients dying due to lack of oxygen at hospitals, a number of corporates have come forward to help with the situation. Tata Group, for example, has announced that it will import 24 ...
- Rohit Sharma’s Heartwarming Gesture Towards Fanboy Avesh Khan After Delhi Capitals Beat Mumbai Indians in IPL 2021 Goes Viralon April 20, 2021 at 7:02 pm
On Tuesday, a fanboy moment caught the attention of fans after Delhi Capitals beat Mumbai Indians in Chennai. Capitals bowler Avesh Khan – who has been impressing this season – asked Mumbai Indians ...
- BMW Motorrad patents gesture control system for motorcycleson April 19, 2021 at 6:50 pm
BMW Motorrad has patented a super interesting technology for motorcycles. The company recently applied for gesture control system for motorcycles.
- BMW Patents Gesture Tech for Motorcycleson April 19, 2021 at 8:15 am
BMW is looking to bring a more innovative approach to making adjustments on the go. They patented gesture controls that will use sensors located on motorcycle handlebars per the drawings. This new pa ...
- Texas Republicans vow to defy any new federal gun rules. Experts say it's largely a symbolic gestureon April 19, 2021 at 5:47 am
But legal experts say the move is largely symbolic, and that its practical effect would be to make it harder — but not impossible — for federal officials to enforce new gun control measures.