via Baylor College of Medicine
UMass Amherst, Baylor researchers collaborate on imitating brain’s ‘replay’ ability
Artificial intelligence (AI) experts at the University of Massachusetts Amherst and the Baylor College of Medicine report that they have successfully addressed what they call a “major, long-standing obstacle to increasing AI capabilities” by drawing inspiration from a human brain memory mechanism known as “replay.”
First author and postdoctoral researcher Gido van de Ven and principal investigator Andreas Tolias at Baylor, with Hava Siegelmann at UMass Amherst, write in Nature Communications that they have developed a new method to protect – “surprisingly efficiently” – deep neural networks from “catastrophic forgetting” – upon learning new lessons, the networks forget what they had learned before.
Siegelmann and colleagues point out that deep neural networks are the main drivers behind recent AI advances, but progress is held back by this forgetting.
They write, “One solution would be to store previously encountered examples and revisit them when learning something new. Although such ‘replay’ or ‘rehearsal’ solves catastrophic forgetting,” they add, “constantly retraining on all previously learned tasks is highly inefficient and the amount of data that would have to be stored becomes unmanageable quickly.”
Unlike AI neural networks, humans are able to continuously accumulate information throughout their life, building on earlier lessons. An important mechanism in the brain believed to protect memories against forgetting is the replay of neuronal activity patterns representing those memories, they explain.
Siegelmann says the team’s major insight is in “recognizing that replay in the brain does not store data.” Rather, “the brain generates representations of memories at a high, more abstract level with no need to generate detailed memories.” Inspired by this, she and colleagues created an artificial brain-like replay, in which no data is stored. Instead, like the brain, the network generates high-level representations of what it has seen before.
The “abstract generative brain replay” proved extremely efficient, and the team showed that replaying just a few generated representations is sufficient to remember older memories while learning new ones. Generative replay not only prevents catastrophic forgetting and provides a new, more streamlined path for system learning, it allows the system to generalize learning from one situation to another, they state.
For example, “if our network with generative replay first learns to separate cats from dogs, and then to separate bears from foxes, it will also tell cats from foxes without specifically being trained to do so. And notably, the more the system learns, the better it becomes at learning new tasks,” says van de Ven.
He and colleagues write, “We propose a new, brain-inspired variant of replay in which internal or hidden representations are replayed that are generated by the network’s own, context-modulated feedback connections. Our method achieves state-of-the-art performance on challenging continual learning benchmarks without storing data, and it provides a novel model for abstract level replay in the brain.”
Van de Ven says, “Our method makes several interesting predictions about the way replay might contribute to memory consolidation in the brain. We are already running an experiment to test some of these predictions.”
The Latest Updates from Bing News & Google News
Go deeper with Bing News on:
- The first of Ghost Recon Breakpoint's two "major updates" will focus on AI teammateson April 11, 2021 at 2:00 pm
before explaining that the first title update – 4.0.0 – will focus on the AI teammates. "We knew that adding them to the game was only the first step, and we are happy to announce that their ...
- 10 Israeli startups on CB Insights’ AI 100 list for 2021on April 11, 2021 at 4:25 am
Syte innovates next-generation search and discovery experiences, powered by visual AI, NLP and hyper-personalization, for ecommerce sites. • Theator enhances the capabilities of thousands of surgeons ...
- A new era of innovation: Moore’s Law is not dead and AI is ready to explodeon April 10, 2021 at 12:47 pm
As we said, lots of modeling is going on in the cloud, but this will be pushed into remote AI inference engines that will have massive processing capabilities collectively. We are moving away from ...
- Tableau adds support to Einstein Discovery for user control over AI modelson April 9, 2021 at 3:20 pm
Tableau is making a case for business science as new discipline for putting control over AI models directly in the hands of end users.
- Report: U.S. military must speed up AI development to maintain edgeon April 9, 2021 at 2:16 pm
The National Security Commission on Artificial Intelligence vice chair said on Friday the United States lacks a strategy to compete with China in military AI.
Go deeper with Google Headlines on:
Go deeper with Bing News on:
Deep neural networks
- Deep Neural Network Can Screen for Skin Disease on Laptopon April 11, 2021 at 7:51 am
A new deep neural network architecture that can provide early diagnosis of systemic sclerosis (SSc) is being reported by the founding chair of the Biomedical Engineering Department at the University ...
- Quantitative salivary gland SPECT/CT using deep convolutional neural networkson April 9, 2021 at 11:29 am
Quantitative single-photon emission computed tomography/computed tomography (SPECT/CT) using Tc-99m pertechnetate aids in evaluating salivary gland function. However, gland segmentation and ...
- Deep Neural Networks Market Outlook, Demand and Supply, Key Prospects, Pricing Strategies, Forecast and Top Manufacturers Analysis Reporton April 8, 2021 at 5:19 am
The MarketWatch News Department was not involved in the creation of this content. Apr 08, 2021 (Market Insight Reports) -- The Global Deep Neural Networks Market is projected to reach USD 5.98 billion ...
- CPU algorithm trains deep neural nets up to 15 times faster than top GPU trainerson April 7, 2021 at 1:05 pm
Rice University computer scientists have demonstrated artificial intelligence (AI) software that runs on commodity processors and trains deep neural networks 15 times faster than platforms based on ...
- Real-time coronary artery stenosis detection based on modern neural networkson April 7, 2021 at 2:38 am
Our study is aimed at confirming the feasibility of real-time coronary artery stenosis detection using deep learning methods. To reach this goal we trained and tested eight promising detectors based ...