via Baylor College of Medicine
UMass Amherst, Baylor researchers collaborate on imitating brain’s ‘replay’ ability
Artificial intelligence (AI) experts at the University of Massachusetts Amherst and the Baylor College of Medicine report that they have successfully addressed what they call a “major, long-standing obstacle to increasing AI capabilities” by drawing inspiration from a human brain memory mechanism known as “replay.”
First author and postdoctoral researcher Gido van de Ven and principal investigator Andreas Tolias at Baylor, with Hava Siegelmann at UMass Amherst, write in Nature Communications that they have developed a new method to protect – “surprisingly efficiently” – deep neural networks from “catastrophic forgetting” – upon learning new lessons, the networks forget what they had learned before.
Siegelmann and colleagues point out that deep neural networks are the main drivers behind recent AI advances, but progress is held back by this forgetting.
They write, “One solution would be to store previously encountered examples and revisit them when learning something new. Although such ‘replay’ or ‘rehearsal’ solves catastrophic forgetting,” they add, “constantly retraining on all previously learned tasks is highly inefficient and the amount of data that would have to be stored becomes unmanageable quickly.”
Unlike AI neural networks, humans are able to continuously accumulate information throughout their life, building on earlier lessons. An important mechanism in the brain believed to protect memories against forgetting is the replay of neuronal activity patterns representing those memories, they explain.
Siegelmann says the team’s major insight is in “recognizing that replay in the brain does not store data.” Rather, “the brain generates representations of memories at a high, more abstract level with no need to generate detailed memories.” Inspired by this, she and colleagues created an artificial brain-like replay, in which no data is stored. Instead, like the brain, the network generates high-level representations of what it has seen before.
The “abstract generative brain replay” proved extremely efficient, and the team showed that replaying just a few generated representations is sufficient to remember older memories while learning new ones. Generative replay not only prevents catastrophic forgetting and provides a new, more streamlined path for system learning, it allows the system to generalize learning from one situation to another, they state.
For example, “if our network with generative replay first learns to separate cats from dogs, and then to separate bears from foxes, it will also tell cats from foxes without specifically being trained to do so. And notably, the more the system learns, the better it becomes at learning new tasks,” says van de Ven.
He and colleagues write, “We propose a new, brain-inspired variant of replay in which internal or hidden representations are replayed that are generated by the network’s own, context-modulated feedback connections. Our method achieves state-of-the-art performance on challenging continual learning benchmarks without storing data, and it provides a novel model for abstract level replay in the brain.”
Van de Ven says, “Our method makes several interesting predictions about the way replay might contribute to memory consolidation in the brain. We are already running an experiment to test some of these predictions.”
The Latest Updates from Bing News & Google News
Go deeper with Bing News on:
AI capabilities
- NGA Enhances Space Intelligence with New Data Hubs and AI
At the center of these developments is the planned Joint Mission Management Center (JMMC), an innovative hub designed to unify the data collection efforts from the Department of Defense, various ...
- IBM and SAP Announce New Generative AI Capabilities to Transform Business Processes
As a global leader in enterprise applications and business AI, SAP (NYSE:SAP) stands at the nexus of business and technology. For over 50 years, organizations have trusted SAP to bring out their best ...
- OpenAI’s evolving Model Spec aims to guide the behavior of AI models
With the Model Spec, OpenAI describes its approach to the complex challenge of shaping AI model behavior, detailing both examples of its work and its ongoing research. The company said it has yet to ...
- ServiceNow and Microsoft expand strategic alliance, combining generative AI capabilities to enhance choice and flexibility
Future capabilities will also enable such experiences as allowing employees to engage Copilot for Microsoft 365 from ServiceNow to create documents, such as a presentation in Microsoft PowerPoint, ...
- Apple unveils iPad with AI capabilities and ‘outrageously powerful' M4 chip
Apple has revealed its new iPad Air with AI capabilities, which they say is "more powerful and versatile than ever".… Continue reading Apple unveils iPad with AI capabilities and ‘outrageously ...
Go deeper with Google Headlines on:
AI capabilities
[google_news title=”” keyword=”AI capabilities” num_posts=”5″ blurb_length=”0″ show_thumb=”left”]
Go deeper with Bing News on:
Deep neural networks
- Comparison: India’s Krutrim AI assistant vs ChatGPT, Google Gemini and Copilot
Discover how India's Krutrim AI performs against global AI giants like ChatGPT, Google Gemini, and Microsoft Copilot in explaining the concept of NPUs ...
- Altera raises $9M to develop AI for digital humans
Altera has raised $9 million to develop AI technology to create digital humans that behave in believable ways.
- Scientists develop energy-efficient memristive hardware for transparent AI decision-making
Most cutting-edge AI systems, powered by deep neural networks, make decisions in ways that are extremely difficult for humans to understand or interpret. This lack of transparency poses major risks as ...
- New Study Redefines Human Sound Localization
Researchers challenged a longstanding theory from the 1940s about how humans locate sounds, a discovery with profound implications for hearing technology.
- AI Deep Learning Improves Brain-Computer Interface Performance
AI deep learning powers a brain-computer interface that enables humans to continuously control a cursor using thoughts.
Go deeper with Google Headlines on:
Deep neural networks
[google_news title=”” keyword=”deep neural networks” num_posts=”5″ blurb_length=”0″ show_thumb=”left”]