via Baylor College of Medicine
UMass Amherst, Baylor researchers collaborate on imitating brain’s ‘replay’ ability
Artificial intelligence (AI) experts at the University of Massachusetts Amherst and the Baylor College of Medicine report that they have successfully addressed what they call a “major, long-standing obstacle to increasing AI capabilities” by drawing inspiration from a human brain memory mechanism known as “replay.”
First author and postdoctoral researcher Gido van de Ven and principal investigator Andreas Tolias at Baylor, with Hava Siegelmann at UMass Amherst, write in Nature Communications that they have developed a new method to protect – “surprisingly efficiently” – deep neural networks from “catastrophic forgetting” – upon learning new lessons, the networks forget what they had learned before.
Siegelmann and colleagues point out that deep neural networks are the main drivers behind recent AI advances, but progress is held back by this forgetting.
They write, “One solution would be to store previously encountered examples and revisit them when learning something new. Although such ‘replay’ or ‘rehearsal’ solves catastrophic forgetting,” they add, “constantly retraining on all previously learned tasks is highly inefficient and the amount of data that would have to be stored becomes unmanageable quickly.”
Unlike AI neural networks, humans are able to continuously accumulate information throughout their life, building on earlier lessons. An important mechanism in the brain believed to protect memories against forgetting is the replay of neuronal activity patterns representing those memories, they explain.
Siegelmann says the team’s major insight is in “recognizing that replay in the brain does not store data.” Rather, “the brain generates representations of memories at a high, more abstract level with no need to generate detailed memories.” Inspired by this, she and colleagues created an artificial brain-like replay, in which no data is stored. Instead, like the brain, the network generates high-level representations of what it has seen before.
The “abstract generative brain replay” proved extremely efficient, and the team showed that replaying just a few generated representations is sufficient to remember older memories while learning new ones. Generative replay not only prevents catastrophic forgetting and provides a new, more streamlined path for system learning, it allows the system to generalize learning from one situation to another, they state.
For example, “if our network with generative replay first learns to separate cats from dogs, and then to separate bears from foxes, it will also tell cats from foxes without specifically being trained to do so. And notably, the more the system learns, the better it becomes at learning new tasks,” says van de Ven.
He and colleagues write, “We propose a new, brain-inspired variant of replay in which internal or hidden representations are replayed that are generated by the network’s own, context-modulated feedback connections. Our method achieves state-of-the-art performance on challenging continual learning benchmarks without storing data, and it provides a novel model for abstract level replay in the brain.”
Van de Ven says, “Our method makes several interesting predictions about the way replay might contribute to memory consolidation in the brain. We are already running an experiment to test some of these predictions.”
The Latest Updates from Bing News & Google News
Go deeper with Bing News on:
- Israel’s D-ID Uses AI To Give A Voice To Victims of Domestic Violence
A chilling clip featuring 5 Israeli women killed by their spouses gives a voice to domestic abuse victims, even after their deaths.
- Global Conversational AI Market (2021 to 2030) - Integration of Advanced AI Capabilities to Add Value to Existing Conversational AI Offerings
The "Conversational AI Market by Component (Solutions and Services), Type (Chatbots and IVA), Technology (NLP and ASR), Business Function (Sales and Marketing, and Operations), Mode of Integration, ...
- AI Helps Generate Alpha
Nearly all asset managers using artificial intelligence-based technology in their pre-investment phase said it helped them to generate alpha. A survey from Deloitte Center for Financial Services found ...
- Haitong International 3rd AI Conference Draw to Successful Close
Haitong International Securities Group Limited ("Haitong International"; Stock Code: 665.HK) announced a successful completion of its third Haitong International AI Conference ("the Conference") in ...
- Ignitarium’s Position as a Key Engineering Service Provider for Semiconductor, AI and Automotive Domains is Recognized by Zinnov Zones Ratings 2021
Ignitarium’s position as a provider of engineering R&D services to Semiconductor, AI and Automotive (focused on ADAS) domains has been ...
Go deeper with Google Headlines on:
Go deeper with Bing News on:
Deep neural networks
- A Whopping 301 Newly Confirmed Exoplanets – Discovered With New Deep Neural Network ExoMiner
Scientists have added a whopping 301 newly confirmed exoplanets to the total exoplanet tally. Scientists recently added a whopping 301 newly validated exoplanets to the total exoplanet tally. The ...
- Cardvision Unveils Early Results Showing Their "OpticEdge" Deep Ensemble Neural Network – Revolutionizing Pokémon Card Collectors' Grading
Cardvision is a new challenger grading company based in the UK, focused on making grading card collectables easier. They are building a digital grading platform that lives ...
- Deep Learning Tool Identifies 301 New Exoplanets
Scientists have announced the discovery of a huge number of new planets. The 301 newly identified exoplanets will join 4,569 previously validated exoplanet | Space ...
- 301 new exoplanets verified by deep learning with a new neural network
Using a new deep learning neural network called ExoMiner, scientists have identified 301 new exoplanets from the Kepler Archive.
- New deep learning method adds 301 planets to Kepler's total count
Scientists recently added a whopping 301 newly validated exoplanets to the total exoplanet tally. The throng of planets is the latest to join the 4,569 already validated planets orbiting a multitude ...