A smart device that translates sign language while being worn on the wrist could bridge the communications gap between the deaf and those who don’t know sign language, says a Texas A&M University biomedical engineering researcher who is developing the technology.
The wearable technology combines motion sensors and the measurement of electrical activity generated by muscles to interpret hand gestures, says Roozbeh Jafari, associate professor in the university’s Department of Biomedical Engineering and researcher at the Center for Remote Health Technologies and Systems.
Although the device is still in its prototype stage, it can already recognize 40 American Sign Language words with nearly 96 percent accuracy, notes Jafari who presented his research at the Institute of Electrical and Electronics Engineers (IEEE) 12th Annual Body Sensor Networks Conference this past June. The technology was among the top award winners in the Texas Instruments Innovation Challenge this past summer.
The technology, developed in collaboration with Texas Instruments, represents a growing interest in the development of high-tech sign language recognition systems (SLRs) but unlike other recent initiatives, Jafari’s system foregoes the use of a camera to capture gestures. Video-based recognition, he says, can suffer performance issues in poor lighting conditions, and the videos or images captured may be considered invasive to the user’s privacy. What’s more, because these systems require a user to gesture in front of a camera, they have limited wearability – and wearability, for Jafari, is key.
“Wearables provide a very interesting opportunity in the sense of their tight coupling with the human body,” Jafari says. “Because they are attached to our body, they know quite a bit about us throughout the day, and they can provide us with valuable feedback at the right times. With this in mind, we wanted to develop a technology in the form factor of a watch.”
In order to capture the intricacies of American Sign Language, Jafari’s system makes use of two distinct sensors. The first is an inertial sensor that responds to motion. Consisting of an accelerometer and gyroscope, the sensor measures the accelerations and angular velocities of the hand and arm, Jafari notes. This sensor plays a major role in discriminating different signs by capturing the user’s hand orientations and hand and arm movements during a gesture.
However, a motion sensor alone wasn’t enough, Jafari explains. Certain signs in American Sign Language are similar in terms of the gestures required to convey the word. With these gestures the overall movement of the hand may be the same for two different signs, but the movement of individual fingers may be different.
For example, the respective gestures for “please” and “sorry” and for “name” and “work” are similar in hand motion. To discriminate between these types of hand gestures, Jafari’s system makes use of another type of sensor that measures muscle activity.
Known as an electromyographic sensor (sEMG), this sensor non-invasively measures the electrical potential of muscle activities, Jafari explains. It is used to distinguish various hand and finger movements based on different muscle activities. Essentially, it’s good at measuring finger movements and the muscle activity patterns for the hand and arm, working in tandem with the motion sensor to provide a more accurate interpretation of the gesture being signed, he says.
In Jafari’s system both inertial sensors and electromyographic sensors are placed on the right wrist of the user where they detect gestures and send information via Bluetooth to an external laptop that performs complex algorithms to interpret the sign and display the correct English word for the gesture. As Jafari continues to develop the technology, he says his team will look to incorporate all of these functions into one wearable device by combining the hardware and reducing the overall size of the required electronics. He envisions the device collecting the data produced from a gesture, interpreting it and then sending the corresponding English word to another person’s smart device so that he or she can understand what is being signed simply by reading the screen of their own device. In addition, he is working to increase the number of signs recognized by the system and expanding the system to both hands.
“The combination of muscle activation detection with motion sensors is a new and exciting way of understanding human intent with other applications in addition to enhanced SLR systems, such as home device activations using context-aware wearables,” Jafari says.
Read more: A smart device that translates sign language while being worn on the wrist
The Latest on: Context-aware wearables
[google_news title=”” keyword=”context-aware wearables” num_posts=”10″ blurb_length=”0″ show_thumb=”left”]
via Google News
The Latest on: Context-aware wearables
- Exclusive: World Heart Day 2023 - Significance Of Monitoring Heart Rhythm And Warning Signs To Watch Out Foron September 27, 2023 at 7:51 pm
Monitoring your heart rhythm is a pivotal aspect of cardiac health. Heart rhythm monitoring can detect warning signs such as palpitations, irregular heartbeats, chest discomfort, shortness of breath, ...
- Context-Aware Computing Market Forecast 2024|Key Factors Considered 2031on September 25, 2023 at 11:03 am
Global |100 Pages| New Report on "Context-Aware Computing Market" offers a detailed Research In-Depth Analysis (2023-2031) which is expected to witness remarkable growth in the coming years.
- Apple Watch Series 9 and Ultra 2 review: small but smart improvementson September 20, 2023 at 10:00 am
Some years, there are updates that completely upend an entire category. Others, you get something like the $399-plus Apple Watch Series 9 and the $799 Ultra 2 — steady improvements that technically ...
- Apple's new AirPods won't have to be taken out of your ears as often, thanks to sophisticated AIon September 18, 2023 at 6:00 am
A slew of software features launching with the new AirPods enable users to leave their earbuds in all day while navigating cities or talking to co-workers.
- Context Aware Computing Market Assessment: An Analysis of Industry Developments and Growth Forecast 2023-2030on September 17, 2023 at 7:03 pm
The global Context Aware Computing market size was valued at USD 53053.97 million in 2022 and is expected to expand at a CAGR of 14.43 Percentage during the forecast period, reaching USD 119092.79 ...
- Ideas in Contexton September 13, 2023 at 5:01 pm
The procedures, aims and vocabularies that were generated will be set in the context of the alternatives available within the contemporary frameworks of ideas and institutions. Through detailed ...
- Mozart in Contexton September 12, 2023 at 2:31 pm
To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to ...
- Dive Into Context Clueson September 9, 2023 at 12:14 am
Help your students flex their vocabulary muscles with this lesson on using context clues. By deciphering the meanings of different nonsense words, young readers will greatly improve their ...
- Microsoft straps AI to your backon August 28, 2023 at 10:19 am
The company filed a patent application for an "artificial intelligence assisted wearable." Microsoft's filing pitches a "context-aware" wearable that can complete commands or answer questions ...
- CareBand Secures US Utility Patent for Advanced IoT Wearable System Designed to Keep Vulnerable People Safeon August 15, 2023 at 10:36 am
The portfolio covers context-aware wearables, edge computing, and data communication via LPWAN networks (i.e. Amazon Sidewalk, LoRaWAN, Sigfox, NB-IoT etc.) for use in healthcare, industrial ...
via Bing News