Graphics of the generative-replay setup (top left panel) and scheme for training artificial neural network (ANN) with generative replay (top right panel). The normalised electrical current accuracy for the conventional (bottom left panel) and brain-inspired replay (BIR) models (bottom right panel).
CREDIT: SUTD
To implement continual learning in edge computing systems, the new type of brain-inspired replay code designed by SUTD researchers provides a promising solution
Demonstrated by breakthroughs in various fields of artificial intelligence (AI), such as image processing, smart healthcare, self-driving vehicles and smart cities, this is undoubtedly the golden period of deep learning. In the next decade or so, AI and computing systems will eventually be equipped with the ability to learn and think the way humans do – to process continuous flow of information and interact with the real world.
However, current AI models suffer from a performance loss when they are trained consecutively on new information. This is because every time new data is generated, it becomes written on top of existing data, thus erasing previous information. This effect is known as ‘catastrophic forgetting’. A difficulty arises from the stability-plasticity issue, where the AI model needs to update its memory to continuously adjust to the new information, and at the same time, maintain the stability of its current knowledge. This problem prevents state-of-the-art AI from continually learning through real world information.
Edge computing systems allow computing to be moved from the cloud storage and data centres to near the original source, such as devices connected to the Internet of Things (IoTs). Applying continual learning efficiently on resource limited edge computing systems remains a challenge, although many continual learning models have been proposed to solve this problem. This is because traditional models require high computing power and large memory capacity.
A new type of code to realise an energy-efficient continual learning system has been recently designed by a team of researchers from the Singapore University of Technology and Design (SUTD), including Shao-Xiang Go, Qiang Wang, Bo Wang, Yu Jiang and Natasa Bajalovic. The team was led by principal investigator, Assistant Professor Desmond Loke from SUTD. The study by these researchers, ‘Continual Learning Electrical Conduction in Resistive-Switching-Memory Materials’, was published in the journal Advanced Theory and Simulations.
The team proposed Brain-Inspired Replay (BIR), a model inspired by the brain, which performs continual learning naturally. The BIR model, based on the use of an artificial neural network and a variational autoencoder, imitates the functions of the human brain and can perform well in class-incremental learning situations without storing data. The researchers also used the BIR model to represent conductive filament growth using electrical current in digital memory systems.
“In this model, knowledge is preserved within trained models in order to minimise performance loss upon the introduction of additional tasks, without the need to refer to data from previous works,” explained Assistant Professor Loke. “So, this saves us a substantial amount of energy.”
“Moreover, a state-of-the-art accuracy of 89% on challenging compliance to current learning tasks without storing data was achieved, which is about two times higher than that of traditional continual learning models, as well as high energy efficiency,’” he added.
To allow the model to process on-the-spot information in the real world independently, the team plans to expand the adjustable capability of the model in the next phase of their research.
“Based on small scale demonstrations, this research is still in its early stages,” said Assistant Professor Loke. “The adoption of this approach is expected to allow edge AI systems to progress independently without human control.”
Original Article: SUTD researchers develop new strategies to teach computers to learn like humans do
More from: Singapore University of Technology and Design
The Latest Updates from Bing News
Go deeper with Bing News on:
Brain-Inspired Replay
- Brain Inspired Computing
Among the various proposals to address the von-Neumann bottleneck, brain-inspired (or brain-like/neuromorphic/neural) computing is one of the most promising and mature avenues. For this Topical ...
- Nanofluidic memristors compute in brain-inspired logic circuits
Neuromorphic breakthrough: Nathan Ronceray (left) and Théo Emmerich at EPFL with their nanofluidic devices. (Courtesy: EPFL/Titouan Veuillet/CC BY SA 4.0) A memristor that uses changes in ion ...
- Intel Unveils Hala Point: A Leap in Brain-Inspired AI Technology
Intel has developed Hala Point, which is intended to explore and expand brain-inspired AI technologies. Housed at Sandia National Laboratories, Hala Point builds upon Intel's earlier system ...
- Intel unveils brain-inspired neuromorphic chip system for more energy-efficient AI workloads
There, it will be used to support research into futuristic, brain-inspired artificial intelligence and tackle challenges relating to the sustainability of AI models. Intel calls this new system ...
- Intel reveals world's biggest 'brain-inspired' neuromorphic computer
Intel has created the world’s largest neuromorphic computer, a device intended to mimic the operation of the human brain. The firm hopes that it will be able to run more sophisticated AI models ...
Go deeper with Bing News on:
Edge AI systems
- Air Force secretary gets taste of future of aviation combat in AI-piloted craft
An AI-piloted experimental fighter gave Secretary of the Air Force Frank Kendall a glimpse of the future during a relatively low-speed demonstration flight this week.
- AAEON’s UP Squared Pro 710H Edge: A Mini PC With an Integrated AI Accelerator for Versatile Edge Computing
Reviewed by Lexie Corner AAEON’s UP brand, a leading producer of advanced Mini PCs and developer boards, has released the UP Squared Pro 710H Edge, a Mini PC equipped with an onboard Hailo-8™ edge AI ...
- New AI-powered NoodleBot set to outperform cutting-edge robotic systems
Engineers at Northwestern University have created a specialized artificial intelligence (AI) algorithm tailored for ...
- Aetina Launches EdgeEye to Empower Remote Management of Edge Devices
Aetina, a leading global provider of edge AI solutions, has announced the launch of the EdgeEye cloud-based management platform. The platform aims to simplify edge computing device management, helping ...
- Lenovo ships all-AMD AI systems
Lenovo Systems announced AI-centric systems using an all-AMD processor design, along with infrastructure for on-prem and Azure cloud.