SEM image of the artificial neuron device.
Images courtesy of Nature Nanotechnology
Training neural networks to perform tasks, such as recognizing images or navigating self-driving cars, could one day require less computing power and hardware thanks to a new artificial neuron device developed by researchers at the University of California San Diego. The device can run neural network computations using 100 to 1000 times less energy and area than existing CMOS-based hardware.
Researchers report their work in a paper published Mar. 18 in Nature Nanotechnology.
Neural networks are a series of connected layers of artificial neurons, where the output of one layer provides the input to the next. Generating that input is done by applying a mathematical calculation called a non-linear activation function. This is a critical part of running a neural network. But applying this function requires a lot of computing power and circuitry because it involves transferring data back and forth between two separate units—the memory and an external processor.
Now, UC San Diego researchers have developed a nanometer-sized device that can efficiently carry out the activation function.
“Neural network computations in hardware get increasingly inefficient as the neural network models get larger and more complex,” said corresponding author Duygu Kuzum, a professor of electrical and computer engineering at the UC San Diego Jacobs School of Engineering. “We developed a single nanoscale artificial neuron device that implements these computations in hardware in a very area- and energy-efficient way.”
The new study was performed as a collaboration between UC San Diego researchers from the Department of Electrical and Computer Engineering (led by Kuzum, who is part of the university’s Center for Machine-Integrated Computing and Security) and a DOE Energy Frontier Research Center (EFRC led by physics professor Ivan Schuller), which focuses on developing hardware implementations of energy-efficient artificial neural networks.
The device implements one of the most commonly used activation functions in neural network training called a rectified linear unit. What’s particular about this function is that it needs hardware that can undergo a gradual change in resistance in order to work. And that’s exactly what the UC San Diego researchers engineered their device to do—it can gradually switch from an insulating to a conducting state, and it does so with the help of a little bit of heat.
This switch is what’s called a Mott transition. It takes place in a nanometers-thin layer of vanadium dioxide. Above this layer is a nanowire heater made of titanium and gold. When current flows through the nanowire, the vanadium dioxide layer slowly heats up, causing a slow, controlled switch from insulating to conducting.
“This device architecture is very interesting and innovative,” said first author Sangheon Oh, an electrical and computer engineering Ph.D. student in Kuzum’s lab. Typically, materials in a Mott transition experience an abrupt switch from insulating to conducting because the current flows directly through the material, he explained. “In this case, we flow current through a nanowire on top of the material to heat it and induce a very gradual resistance change.”
To implement the device, the researchers first fabricated an array of these so-called activation (or neuron) devices, along with a synaptic device array. Then they integrated the two arrays on a custom printed circuit board and connected them together to create a hardware version of a neural network.
The researchers used the network to process an image—in this case, a picture of Geisel Library at UC San Diego. The network performed a type of image processing called edge detection, which identifies the outlines or edges of objects in an image. This experiment demonstrated that the integrated hardware system can perform convolution operations that are essential for many types of deep neural networks.
The researchers say the technology could be further scaled up to do more complex tasks such as facial and object recognition in self-driving cars. With interest and collaboration from industry, this could happen, noted Kuzum.
“Right now, this is a proof of concept,” Kuzum said. “It’s a tiny system in which we only stacked one synapse layer with one activation layer. By stacking more of these together, you could make a more complex system for different applications.”
More from: University of California San Diego
The Latest Updates from Bing News & Google News
Go deeper with Bing News on:
Neural network hardware
- Understanding the Art of Machine Learningon April 30, 2021 at 1:05 pm
Though neural-network-based machine learning is escalating in popularity, the mechanics behind it tend to be misconstrued or simply not known at all.
- Reliability and Performance Analysis of Logic-in-Memory Based Binarized Neural Networkson April 30, 2021 at 12:50 pm
Resistive Random access memory (RRAM) devices together with the material implication (IMPLY) logic are a promising computing scheme for realizing energy efficient reconfigurable computing hardware for ...
- A Partially Binarized and Fixed Neural Network Based Calibrator for SAR-Pipelined ADCs Achieving 95.0-dB SFDRon April 30, 2021 at 2:41 am
Recently, the neural network has been applied to calibrate the successive-approximation-register and pipelined analog-to-digital converters (SAR-Pipelined ADCs), which requires no prior knowledge.
- Robust high-dimensional memory-augmented neural networkson April 29, 2021 at 3:15 am
The implementation of memory-augmented neural networks using conventional computer architectures is challenging due to a large number of read and write operations. Here, Karunaratne, Schmuck et al.
- Q&A: Vivienne Sze on crossing the hardwire-software divide for efficient AIon April 28, 2021 at 1:31 pm
Vivienne Sze is an MIT researcher who is co-designing artificial intelligence hardware and software to be more energy efficient.
Go deeper with Google Headlines on:
Neural network hardware
Go deeper with Bing News on:
Artificial neuron device
- Brain like computing device that simulates human learningon April 30, 2021 at 11:43 am
During a recent study researchers from Northwestern University and the University of Hong Kong developed a brain-like computing device that is capable of learn ...
- New Artificial Neuron Device Runs Neural Network Computations Using 100 to 1000 Times Less Energyon April 28, 2021 at 4:07 am
Training neural networks to perform tasks, such as recognizing images or navigating self-driving cars, could one day require less computing power and hardware thanks to a new artificial neuron device ...
- Hands-free: Monkey plays video game - with its brainon April 23, 2021 at 3:38 pm
Musk has long contended that merging minds with machines is vital if people are going to avoid being outpaced by artificial intelligence ... Neuralink devices were implanted on two sides of Pager's ...
- Catastrophic Forgetting: Learning’s Effect On Machine Mindson April 20, 2021 at 5:00 pm
What if every time you learned something new, you forgot a little of what you knew before? That sort of overwriting doesn’t happen in the human brain, but it does in artificial neural networks.
- Why is the Brainchip (ASX:BRN) share price jumping today?on April 19, 2021 at 11:58 pm
Neuromorphic computing is a branch of artificial intelligence (AI) that simulates the functionality of the human neuron ... processor chip for edge AI devices. ‘Edge’ in AI means that the ...