Graphics of the generative-replay setup (top left panel) and scheme for training artificial neural network (ANN) with generative replay (top right panel). The normalised electrical current accuracy for the conventional (bottom left panel) and brain-inspired replay (BIR) models (bottom right panel).
To implement continual learning in edge computing systems, the new type of brain-inspired replay code designed by SUTD researchers provides a promising solution
Demonstrated by breakthroughs in various fields of artificial intelligence (AI), such as image processing, smart healthcare, self-driving vehicles and smart cities, this is undoubtedly the golden period of deep learning. In the next decade or so, AI and computing systems will eventually be equipped with the ability to learn and think the way humans do – to process continuous flow of information and interact with the real world.
However, current AI models suffer from a performance loss when they are trained consecutively on new information. This is because every time new data is generated, it becomes written on top of existing data, thus erasing previous information. This effect is known as ‘catastrophic forgetting’. A difficulty arises from the stability-plasticity issue, where the AI model needs to update its memory to continuously adjust to the new information, and at the same time, maintain the stability of its current knowledge. This problem prevents state-of-the-art AI from continually learning through real world information.
Edge computing systems allow computing to be moved from the cloud storage and data centres to near the original source, such as devices connected to the Internet of Things (IoTs). Applying continual learning efficiently on resource limited edge computing systems remains a challenge, although many continual learning models have been proposed to solve this problem. This is because traditional models require high computing power and large memory capacity.
A new type of code to realise an energy-efficient continual learning system has been recently designed by a team of researchers from the Singapore University of Technology and Design (SUTD), including Shao-Xiang Go, Qiang Wang, Bo Wang, Yu Jiang and Natasa Bajalovic. The team was led by principal investigator, Assistant Professor Desmond Loke from SUTD. The study by these researchers, ‘Continual Learning Electrical Conduction in Resistive-Switching-Memory Materials’, was published in the journal Advanced Theory and Simulations.
The team proposed Brain-Inspired Replay (BIR), a model inspired by the brain, which performs continual learning naturally. The BIR model, based on the use of an artificial neural network and a variational autoencoder, imitates the functions of the human brain and can perform well in class-incremental learning situations without storing data. The researchers also used the BIR model to represent conductive filament growth using electrical current in digital memory systems.
“In this model, knowledge is preserved within trained models in order to minimise performance loss upon the introduction of additional tasks, without the need to refer to data from previous works,” explained Assistant Professor Loke. “So, this saves us a substantial amount of energy.”
“Moreover, a state-of-the-art accuracy of 89% on challenging compliance to current learning tasks without storing data was achieved, which is about two times higher than that of traditional continual learning models, as well as high energy efficiency,’” he added.
To allow the model to process on-the-spot information in the real world independently, the team plans to expand the adjustable capability of the model in the next phase of their research.
“Based on small scale demonstrations, this research is still in its early stages,” said Assistant Professor Loke. “The adoption of this approach is expected to allow edge AI systems to progress independently without human control.”
The Latest Updates from Bing News
Go deeper with Bing News on:
- 10 Top Games Similar To Helldivers 2
While Remnant 2 still has hordes of enemies to shoot down, the game is inspired by FromSoftware’s Dark Souls series ... That increases the game's replay value, as there’s always a new way to face the ...
- Believe it or not, now is the best time to replay Elden Ring
Your brain is a wonderful thing ... Regardless of how you go about it, now's the best time to replay Elden Ring. Not only that, with the DLC surely coming out soon, having a fresh save file ...
- Apple Music Replay for 2024 has arrived
The biggest stories of the day delivered to your inbox.
- Innovative Brain-Inspired Model Enhances Wastewater Treatment Predictions
A groundbreaking study, published in Frontiers of Environmental Science & Engineering on 10 November 2023, has explored brain-inspired hybrid models that merge CNN and LSTM to enhance prediction ...
- Games With The Best Environmental Storytelling
In Soma, Simon Jarrett agrees to an experimental brain scan after a car accident ... Each relative’s death is told through a series of vignettes inspired by their rooms. Since the vignettes ...
Go deeper with Bing News on:
Edge AI systems
- New AI System Protects Against Phishing Scams and Cyber Threats
Bufferzone and Intel unveiled an AI-powered anti-phishing browser extension at the Mobile World Congress, enhancing online security by continuously scanning for threats.
- AI Revolutionizing Drug Discovery: Google DeepMind CEO Predicts Drug Design in Clinics within Two Years
In a groundbreaking revelation at the Mobile World Congress in Barcelona, Demis Hassabis, the CEO of Google DeepMind, foresees artificial intelligence (AI) taking the lead in designing drugs directly ...
- Gina Raimondo on Commerce’s Investments in Leading-Edge Logic Chip Manufacturing for AI Systems
Commerce Secretary Gina Raimondo said artificial intelligence systems are driving the demand for leading-edge logic chips and the Department of Commerce is working to meet that demand through its ...
- Qualcomm unveils AI and connectivity chips at Mobile World Congress
Cristiano Amon, CEO of Qualcomm, said in a statement, “The future of generative AI is hybrid, with on-device intelligence working together with the cloud to provide greater personalization, privacy, ...
- GIGABYTE Ignites AI And 5G Visions At MWC 2024, Highlighting New Supercomputers, Edge AI And Sustainable IT Upgrades - A...
GIGABYTE Technology, an IT pioneer advancing global industries through cloud and AI computing systems, is presenting innovative ent ...