Artificial neural networks are computing systems inspired by biological neural networks that constitute animal brains. Like biological models, they can learn (be trained) by processing examples and forming probability associations, then apply that information to other tasks.
Depending on age, humans need 7 to 13 hours of sleep per 24 hours. During this time, a lot happens: Heart rate, breathing and metabolism ebb and flow; hormone levels adjust; the body relaxes. Not so much in the brain.
“The brain is very busy when we sleep, repeating what we have learned during the day,” said Maxim Bazhenov, PhD, professor of medicine and a sleep researcher at University of California San Diego School of Medicine. “Sleep helps reorganize memories and presents them in the most efficient way.”
In previous published work, Bazhenov and colleagues have reported how sleep builds rational memory, the ability to remember arbitrary or indirect associations between objects, people or events, and protects against forgetting old memories.
Artificial neural networks leverage the architecture of the human brain to improve numerous technologies and systems, from basic science and medicine to finance and social media. In some ways, they have achieved superhuman performance, such as computational speed, but they fail in one key aspect: When artificial neural networks learn sequentially, new information overwrites previous information, a phenomenon called catastrophic forgetting.
“In contrast, the human brain learns continuously and incorporates new data into existing knowledge,” said Bazhenov, “and it typically learns best when new training is interleaved with periods of sleep for memory consolidation.”
Writing in the November 18, 2022 issue of PLOS Computational Biology, senior author Bazhenov and colleagues discuss how biological models may help mitigate the threat of catastrophic forgetting in artificial neural networks, boosting their utility across a spectrum of research interests.
The scientists used spiking neural networks that artificially mimic natural neural systems: Instead of information being communicated continuously, it is transmitted as discrete events (spikes) at certain time points.
They found that when the spiking networks were trained on a new task, but with occasional off-line periods that mimicked sleep, catastrophic forgetting was mitigated. Like the human brain, said the study authors, “sleep” for the networks allowed them to replay old memories without explicitly using old training data.
“It meant that these networks could learn continuously, like humans or animals. Understanding how human brain processes information during sleep can help to augment memory in human subjects. Augmenting sleep rhythms can lead to better memory. In other projects, we use computer models to develop optimal strategies to apply stimulation during sleep, such as auditory tones, that enhance sleep rhythms and improve learning. This may be particularly important when memory is non-optimal, such as when memory declines in aging or in some conditions like Alzheimer’s disease.”
Maxim Bazhenov, PhD, professor of medicine and a sleep researcher at University of California San Diego School of Medicine.
Memories are represented in the human brain by patterns of synaptic weight — the strength or amplitude of a connection between two neurons.
“When we learn new information,” said Bazhenov, “neurons fire in specific order and this increases synapses between them. During sleep, the spiking patterns learned during our awake state are repeated spontaneously. It’s called reactivation or replay.
“Synaptic plasticity, the capacity to be altered or molded, is still in place during sleep and it can further enhance synaptic weight patterns that represent the memory, helping to prevent forgetting or to enable transfer of knowledge from old to new tasks.”
When Bazhenov and colleagues applied this approach to artificial neural networks, they found that it helped the networks avoid catastrophic forgetting.
“It meant that these networks could learn continuously, like humans or animals. Understanding how human brain processes information during sleep can help to augment memory in human subjects. Augmenting sleep rhythms can lead to better memory.
“In other projects, we use computer models to develop optimal strategies to apply stimulation during sleep, such as auditory tones, that enhance sleep rhythms and improve learning. This may be particularly important when memory is non-optimal, such as when memory declines in aging or in some conditions like Alzheimer’s disease.”
Original Article: Artificial Neural Networks Learn Better When They Spend Time Not Learning at All
More from: University of California San Diego School of Medicine | University of California San Diego | Czech Academy of Sciences
The Latest Updates from Bing News & Google News
Go deeper with Bing News on:
Artificial neural networks
- Generative AI: The Future of Content Creation is Here
The world of Artificial Intelligence (AI) is rapidly evolving, pushing the boundaries of what machines can create. One particularly exciting area is ...
- CEOs of Microsoft, Nvidia and other tech giants join federal AI advisory board
Microsoft Corp. Chief Executive Satya Nadella, Nvidia Corp. CEO Jensen Huang and OpenAI’s San Altman are among the participants. They will be joined by their counterparts at Advanced Micro Devices Inc ...
- AI Efficiency Breakthrough: How Sound Waves Are Revolutionizing Optical Neural Networks
Researchers have developed a way to use sound waves in optical neural networks, enhancing their ability to process data with high speed and energy efficiency. Optical neural networks may provide the ...
- How Neural Concept’s aerodynamic AI is shaping Formula 1
It's a long way from pedal bikes to Formula 1. But that's precisely the quantum leap that AI-based startup Neural Concept and its co-founder and CEO, ...
- Artificial Intelligence ‘Explainability’ Is Overrated
As AI becomes more advanced and is applied to domains like healthcare, hiring, and criminal justice, some are calling for these systems to be more "explainable." ...
Go deeper with Bing News on:
Artificial neural networks continuous learning
- OMNIQ AWARDED A FOLLOW-ON PURCHASE ORDER FOR ITS AI-BASED MACHINE VISION SOLUTION TO BE INSTALLED IN A CRITICAL HOMELAND SECURITY PROJECT
OMNIQ team expanded AI capabilities to crowd management and high occupancy vehicles (HOV).
- Machine learning and experiment
For more than 20 years in experimental particle physics and astrophysics, machine learning has been accelerating the pace of science, helping scientists tackle problems of greater and greater ...
- Emulating neurodegeneration and aging in artificial intelligence systems
In recent years, developers have introduced artificial intelligence (AI) systems that can simulate or reproduce various human abilities, such as recognizing objects in images, answering questions, and ...
- AI Efficiency Breakthrough: How Sound Waves Are Revolutionizing Optical Neural Networks
Researchers have developed a way to use sound waves in optical neural networks, enhancing their ability to process data with high speed and energy efficiency. Optical neural networks may provide the ...
- AI Cyber Threats Pose Serious Concerns for Business
Artificial intelligence (AI) technology has revolutionised various industries, including cybersecurity. While AI provides great opportunities for innovation and efficiency, it also poses significant ...