A new computer software programme has the potential to lip-read more accurately than people and to help those with hearing loss, Oxford University researchers have found.
Watch, Attend and Spell (WAS), is a new artificial intelligence (AI) software system that has been developed by Oxford, in collaboration with the company DeepMind.
The AI system uses computer vision and machine learning methods to learn how to lip-read from a dataset made up of more than 5,000 hours of TV footage, gathered from six different programmes including Newsnight, BBC Breakfast and Question Time. The videos contained more than 118,000 sentences in total, and a vocabulary of 17,500 words.
The research team compared the ability of the machine and a human expert to work out what was being said in the silent video by focusing solely on each speaker’s lip movements. They found that the software system was more accurate compared to the professional. The human lip-reader correctly read 12 per cent of words, while the WAS software recognised 50 per cent of the words in the dataset, without error. The machine’s mistakes were small, including things like missing an “s” at the end of a word, or single letter misspellings.
The software could support a number of developments, including helping the hard of hearing to navigate the world around them. Speaking on the tech’s core value, Jesal Vishnuram, Action on Hearing Loss Technology Research Manager, said: ‘Action on Hearing Loss welcomes the development of new technology that helps people who are deaf or have a hearing loss to have better access to television through superior real-time subtitling.
‘It is great to see research being conducted in this area, with new breakthroughs welcomed by Action on Hearing Loss by improving accessibility for people with a hearing loss. AI lip-reading technology would be able to enhance the accuracy and speed of speech-to-text especially in noisy environments and we encourage further research in this area and look forward to seeing new advances being made.’
Commenting on the potential uses for WAS Joon Son Chung, lead-author of the study and a graduate student at Oxford’s Department of Engineering, said: ‘Lip-reading is an impressive and challenging skill, so WAS can hopefully offer support to this task – for example, suggesting hypotheses for professional lip readers to verify using their expertise. There are also a host of other applications, such as dictating instructions to a phone in a noisy environment, dubbing archival silent films, resolving multi-talker simultaneous speech and improving the performance of automated speech recognition in general.’
[osd_subscribe categories=’artificial-intelligence’ placeholder=’Email Address’ button_text=’Subscribe Now for any new posts on the topic “ARTIFICIAL INTELLIGENCE”‘]
Receive an email update when we add a new ARTIFICIAL INTELLIGENCE article.
The Latest on: Artificial intelligence
via Google News
The Latest on: Artificial intelligence
- KAID Health Technology Demonstrates the Value of Natural Language Processing to Improve Preoperative Careon August 1, 2022 at 7:02 am
We have demonstrated that NLP technology can help identify critical medical conditions relevant to preanesthetic evaluation. Key to this was KAID Health’s ability to utilize unstructured free-text ...
- No code, no problem—we try to beat an AI at its own game with new toolson August 1, 2022 at 6:00 am
And some people even think that an AI has attained sentience. (Spoiler alert: It has not .) And as Ars' Matt Ford recently pointed out here, artificial intelligence may be artificial, but it's not ...
- CloudFactory Appoints Pieter Nel CTO to Lead Data-centric AI Strategyon August 1, 2022 at 5:02 am
CloudFactory, a global leader in human-in-the-loop artificial intelligence (AI), today announced that Pieter Nel has joined as Chief Technology Officer (CTO). Nel brings more than 20 years of ...
- Automating neutron experiments with AIon August 1, 2022 at 5:00 am
Oak Ridge National Laboratory researchers are developing a first-of-its-kind artificial intelligence device for neutron scattering called Hyperspectral Computed Tomography, or HyperCT. The fully ...
- Researchers Partner With NIH and Google to Develop AI Learning Moduleson July 31, 2022 at 10:21 pm
Supplemental grant will help researchers develop cloud-based learning modules focused on using artificial intelligence and machine learning in biomedical sciences.
- Bam! AI exits the Batcave to confront the jobs marketon July 31, 2022 at 8:10 pm
AI, like Batman, is viewed by some people with suspicion and fear. But it can be a force for good, creating jobs for a modern workforce.
- Can artificial intelligence really help us talk to the animals?on July 31, 2022 at 7:00 am
A California-based organisation wants to harness the power of machine learning to decode communication across the entire animal kingdom. But the project has its doubters ...
- 'Alternative physics' discovered by artificial intelligenceon July 31, 2022 at 2:55 am
An artificial intelligence program has studied various physical phenomena and identified its own variables for describing them.
- Research shows artificial intelligence can improve stroke diagnostics, expanding access to lifesaving stroke careon July 28, 2022 at 9:37 am
A new study presented today at the Society of NeuroInterventional Surgery's (SNIS) 19th Annual Meeting shows that artificial intelligence (AI) technology can identify when a patient is having a stroke ...
via Bing News