Graphics of the generative-replay setup (top left panel) and scheme for training artificial neural network (ANN) with generative replay (top right panel). The normalised electrical current accuracy for the conventional (bottom left panel) and brain-inspired replay (BIR) models (bottom right panel).
CREDIT: SUTD
To implement continual learning in edge computing systems, the new type of brain-inspired replay code designed by SUTD researchers provides a promising solution
Demonstrated by breakthroughs in various fields of artificial intelligence (AI), such as image processing, smart healthcare, self-driving vehicles and smart cities, this is undoubtedly the golden period of deep learning. In the next decade or so, AI and computing systems will eventually be equipped with the ability to learn and think the way humans do – to process continuous flow of information and interact with the real world.
However, current AI models suffer from a performance loss when they are trained consecutively on new information. This is because every time new data is generated, it becomes written on top of existing data, thus erasing previous information. This effect is known as ‘catastrophic forgetting’. A difficulty arises from the stability-plasticity issue, where the AI model needs to update its memory to continuously adjust to the new information, and at the same time, maintain the stability of its current knowledge. This problem prevents state-of-the-art AI from continually learning through real world information.
Edge computing systems allow computing to be moved from the cloud storage and data centres to near the original source, such as devices connected to the Internet of Things (IoTs). Applying continual learning efficiently on resource limited edge computing systems remains a challenge, although many continual learning models have been proposed to solve this problem. This is because traditional models require high computing power and large memory capacity.
A new type of code to realise an energy-efficient continual learning system has been recently designed by a team of researchers from the Singapore University of Technology and Design (SUTD), including Shao-Xiang Go, Qiang Wang, Bo Wang, Yu Jiang and Natasa Bajalovic. The team was led by principal investigator, Assistant Professor Desmond Loke from SUTD. The study by these researchers, ‘Continual Learning Electrical Conduction in Resistive-Switching-Memory Materials’, was published in the journal Advanced Theory and Simulations.
The team proposed Brain-Inspired Replay (BIR), a model inspired by the brain, which performs continual learning naturally. The BIR model, based on the use of an artificial neural network and a variational autoencoder, imitates the functions of the human brain and can perform well in class-incremental learning situations without storing data. The researchers also used the BIR model to represent conductive filament growth using electrical current in digital memory systems.
“In this model, knowledge is preserved within trained models in order to minimise performance loss upon the introduction of additional tasks, without the need to refer to data from previous works,” explained Assistant Professor Loke. “So, this saves us a substantial amount of energy.”
“Moreover, a state-of-the-art accuracy of 89% on challenging compliance to current learning tasks without storing data was achieved, which is about two times higher than that of traditional continual learning models, as well as high energy efficiency,’” he added.
To allow the model to process on-the-spot information in the real world independently, the team plans to expand the adjustable capability of the model in the next phase of their research.
“Based on small scale demonstrations, this research is still in its early stages,” said Assistant Professor Loke. “The adoption of this approach is expected to allow edge AI systems to progress independently without human control.”
Original Article: SUTD researchers develop new strategies to teach computers to learn like humans do
More from: Singapore University of Technology and Design
The Latest Updates from Bing News
Go deeper with Bing News on:
Brain-Inspired Replay
- Can You Replay Conversations In Your Head? If Not, You Might Struggle With These Common Tasks
A new study suggests people without an inner voice can struggle with verbal memory and some verbal tasks. But there are ways to compensate.
- Neuralink Wants to Expand Brain Chip Trial to Second Human Patient
The first human patient, Noland Arbaugh, says the chip 'completely changed how I live,' despite a 'thread retraction' issue that will likely need to be addressed before a second trial begins.
- Animal brain inspired AI game changer for autonomous robots
A team of researchers has developed a drone that flies autonomously using neuromorphic image processing and control based on the workings of animal brains. Animal brains use less data and energy ...
- Eurovision 2024 semifinal highlights: Ten countries move on to Saturday's finals
Latest news and highlights about Eurovision semi final round one, including who performed, who moved forward to the finals and more.
- Full-length Replay: NFL Draft Round 1 Recap: Biggest Winners, Best Value Pick, Biggest Gamble and MORE! | Full Episode
Dave Helman is joined by Carmen Vitali to recap the first round of the 2024 NFL Draft! To start, Dave and Carmen discuss the record-setting quarterback selections as six guys went in the top 12 ...
Go deeper with Bing News on:
Edge AI systems
- Artificial Intelligence is medicine's next frontier. Israel is leading the way
Opinion: As we celebrate 76 years of Israeli healthcare innovation we are excited by the AI-based projects being undertaken in Israel.
- Google launches ‘Model Explorer’, an open source tool for seamless AI model visualization and debugging
Google's new open source tool, Model Explorer, revolutionizes AI transparency by enabling smooth visualization and debugging of complex machine learning models, paving the way for more responsible AI ...
- Organizations urge Colorado governor to veto AI bill, say it will hurt small businesses
Organizations representing the technology industry are urging Colorado Gov. Jared Polis to veto a bill they say could negatively impact small businesses developing cutting-edge artificial intelligence ...
- How To Successfully Deploy GenAI On Edge Devices
Smaller models with fewer parameters put image generators and chatbots within reach of an embedded NPU implementation.
- GigaIO and SourceCode Launch Suitcase-sized Edge AI System
Kissimmee, Florida, May 6, 2024 – GigaIO and SourceCode today announced the launch of the Gryf mobile AI platform for tactical edge environments. The system is designed to engineer complex customer ...