via PACIFIC NORTHWEST NATIONAL LABORATORY
As anyone with a green thumb knows, pruning can promote thriving vegetation. A snip here, a snip there, and growth can be controlled and directed for a more vigorous plant.
The same principle can be applied to machine learning algorithms. Removing bits and pieces along coding branches in those algorithms can reduce complexity in decision trees and increase predictive performance.
Researchers at the U.S. Department of Energy’s Pacific Northwest National Laboratory (PNNL) have done just that. Exploring with binarized neural networks (BNNs), they used pruning principles to significantly reduce computation complexity and memory demands. BNNs are close cousins to deep neural networks, which require large amounts of computation. But BNNs differ in a significant way: they use single bits to encode each neuron and parameter, using much less energy and power for computation.
Pruning for faster growth
Researchers recognized the potential value of BNNs for machine learning starting in about 2016. If constructed–or pruned–just the right way, they consume less computing energy and are nearly as accurate as deep neural networks. That means BNNs have more potential to benefit resource-constrained environments, such as mobile phones, smart devices, and the entire Internet of Things ecosystem.
This is where pruning comes into play. As neural networks research has grown in recent years, pruning has gained more interest among computing researchers.
“Pruning is currently a hot topic in machine learning,” said PNNL computer scientist Ang Li. “We can add software and architecture coding to push the trimming towards a direction that will have more benefits for the performance of computing devices. These benefits include lower energy needs and lower computing costs.”
Pruning for precision
Li was among a group of PNNL researchers who recently published results in the Institute of Electrical and Electronics Engineers Transactions on Parallel and Distributed Systems showing the benefits of selective pruning. The research demonstrated that pruning redundant bits of the BNN architecture led to a custom-built out-of-order BNN, called O3BNN-R. Their work shows a highly-condensed BNN model–which already could display high-performing supercomputing qualities–can be shrunk signi?cantly further without loss of accuracy.
“Binarized neural networks have the potential of making the processing time of neural networks around microseconds,” said Tong “Tony” Geng, a Boston University doctoral candidate who, as a PNNL intern, assisted Li on the O3BNN-R project.
“BNN research is headed in a promising direction to make neural networks really useful and be readily adopted in the real-world,” said Geng, who will rejoin the PNNL staff in January as a postdoctoral research fellow. “Our finding is an important step to realize this potential.”
Their research shows this out-of-order BNN can prune, on average, 30 percent of operations without any accuracy loss. With even more fine tuning–in a step called “regularization at training”–the performance can be improved an additional 15 percent.
Pruning for power
In addition to this out-of-order BNN’s contributions to the Internet of Things, Li also pointed to potential benefits to the energy grid. Implementation of a modified BNN could also provide a boost to existing software that guards against cyberattacks when deployed in the power grid by helping existing sensors detect and respond to an attack, said Li.
“Basically,” said Li, “we are accelerating the speed of processing in hardware.”
The Latest Updates from Bing News & Google News
Go deeper with Bing News on:
Binarized neural networks
- Reliability and Performance Analysis of Logic-in-Memory Based Binarized Neural Networkson April 30, 2021 at 12:50 pm
Resistive Random access memory (RRAM) devices together with the material implication (IMPLY) logic are a promising computing scheme for realizing energy efficient reconfigurable computing hardware for ...
- A Partially Binarized and Fixed Neural Network Based Calibrator for SAR-Pipelined ADCs Achieving 95.0-dB SFDRon April 30, 2021 at 2:41 am
Recently, the neural network has been applied to calibrate the successive-approximation-register and pipelined analog-to-digital converters (SAR-Pipelined ADCs), which requires no prior knowledge.
- The Battle of AI Processors Begins in 2018on April 25, 2021 at 5:00 pm
Neural networks are capable of performing a wide range of tasks ... operations for pruned, Int6, and binarized DNNs, respectively. ... On Ternary-ResNet, the Stratix 10 FPGA can deliver 60% better ...
- Cracking the code of the Dead Sea Scrollson April 21, 2021 at 11:07 am
For this separation, or "binarization," Dhali developed a state-of-the-art artificial neural network that can be trained using deep learning. This neural network keeps the original ink traces made ...
- More than one scribe wrote the text of a Dead Sea Scroll, handwriting showson April 21, 2021 at 11:00 am
Most of the scribes who copied the text contained in the Dead Sea Scrolls were anonymous, as they neglected to sign their work. That has made it challenging for scholars to determine whether a ...
Go deeper with Google Headlines on:
Binarized neural networks
Go deeper with Bing News on:
- Record Metals Prices Catapult Mining Profits Beyond Big Oilon May 1, 2021 at 5:02 pm
Major oil producers, for decades the natural resource industry’s top earners, are being eclipsed by once-smaller mining peers who are churning out record profits thanks to red-hot metals markets.
- BRAIN Biotech AG (BNN) NPVon April 28, 2021 at 2:57 pm
The selling price currently displayed is higher than the buying price. This can occur temporarily for a variety of reasons; shortly before the market opens, after the market closes or because of ...
- El Nino can help predict cacao harvests up to two years in advanceon April 28, 2021 at 12:39 pm
When seasonal rains arrive late in Indonesia, farmers often take it as a sign that it is not worth investing in fertilizer for their crops. Sometimes they opt out of planting annual crops altogether.
- Is it finally time to buy Ballard Power?on April 28, 2021 at 8:33 am
What do we make of Ballard Power (TSX:BLDP), a company that’s gotten a whole lot of attention over the years but proved to be a perennial disappointment?
- U.K. Plans to Scrap MiFID Research Rule for Small Firms, Bondson April 28, 2021 at 4:41 am
Britain’s market watchdog proposed easing landmark restrictions on investment research for small companies and fixed income markets in its first efforts to rewrite these rules since the U.K. left the ...