A picture is analyzed by the chip, which then provides the appropriate output signal.
Graphik: Joanna Symonowicz, TU Wien
An ultra-fast image sensor with a built-in neural network has been developed at TU Wien (Vienna). It can be trained to recognize certain objects. It has now been presented in “Nature”.
Automatic image recognition is widely used today: There are computer programs that can reliably diagnose skin cancer, navigate self-driving cars, or control robots. Up to now, all this has been based on the evaluation of image data as delivered by normal cameras – and that is time-consuming. Especially when the number of images recorded per second is high, a large volume of data is generated that can hardly be handled.
Scientists at TU Wien therefore took a different approach: using a special 2D material, an image sensor was developed that can be trained to recognize certain objects. The chip represents an artificial neural network capable of learning. The data does not have to be read out and processed by a computer, but the chip itself provides information about what it is currently seeing – within nanoseconds. The work has now been presented in the scientific journal “Nature”.
Neural networks are artificial systems that are similar to our brain: Nerve cells are connected to many other nerve cells. When one cell is active, this can influence the activity of neighbouring nerve cells. Artificial learning on the computer works according to exactly the same principle: A network of neurons is simulated digitally, and the strength with which one node of this network influences the other is changed until the network shows the desired behaviour.
“Typically, the image data is first read out pixel by pixel and then processed on the computer,” says Thomas Mueller. “We, on the other hand, integrate the neural network with its artificial intelligence directly into the hardware of the image sensor. This makes object recognition many orders of magnitude faster.”
The chip was developed and manufactured at the TU Vienna. It is based on photodetectors made of tungsten diselenide – an ultra-thin material consisting of only three atomic layers. The individual photodetectors, the “pixels” of the camera system, are all connected to a small number of output elements that provide the result of object recognition.
Learning through variable sensitivity
“In our chip, we can specifically adjust the sensitivity of each individual detector element – in other words, we can control the way the signal picked up by a particular detector affects the output signal,” says Lukas Mennel, first author of the publication. “All we have to do is simply adjust a local electric field directly at the photodetector.” This adaptation is done externally, with the help of a computer program. One can, for example, use the sensor to record different letters and change the sensitivities of the individual pixels step by step until a certain letter always leads exactly to a corresponding output signal. This is how the neural network in the chip is configured – making some connections in the network stronger and others weaker.
Once this learning process is complete, the computer is no longer needed. The neural network can now work alone. If a certain letter is presented to the sensor, it generates the trained output signal within 50 nanoseconds – for example, a numerical code representing the letter that the chip has just recognized.
Object detection when things have to go fast
“Our test chip is still small at the moment, but you can easily scale up the technology depending on the task you want to solve,” says Thomas Mueller. “In principle, the chip could also be trained to distinguish apples from bananas, but we see its use more in scientific experiments or other specialized applications.”
The technology can be usefully applied wherever extremely high speed is required: “From fracture mechanics to particle detection – in many research areas, short events are investigated,” says Thomas Mueller. “Often it is not necessary to keep all the data about this event, but rather to answer a very specific question: Does a crack propagate from left to right? Which of several possible particles has just passed by? This is exactly what our technology is good for.”
The Latest Updates from Bing News & Google News
Go deeper with Bing News on:
Ultra-fast image sensor
- Terranet AB Demonstrates Ultra Fast VoxelFlow™ Sensor at STARTUP AUTOBAHN in joint Presentation with Mercedes-Benzon February 25, 2021 at 6:05 am
Breakthrough Sensor Specifically Addresses AV & ADAS Safety Through Life-Saving 3D Vision Technology Lund, Sweden, February 25, 2021 - Terranet AB (Terranet/the Company), (Nasdaq: TERRNT-B.ST), ...
- Samsung unveils ISOCELL GN2, a new 50MP sensor for high-end smartphoneson February 22, 2021 at 10:53 pm
Samsung Electronics has announced its newest smartphone camera sensor, the ISOCELL GN2. This is the follow up to the ISOCELL GN1 which was launched back in May 2020. The ISOCELL GN2 is also a 50MP ...
- Samsung Introduces 1.4μm 50Mp ISOCELL GN2 with Faster and More True-to-Life Auto-Focusingon February 22, 2021 at 7:26 pm
By employing two photodiodes within every pixel of the image sensor, Dual Pixel Pro makes use of one hundred million phase detecting agents for ultra-fast auto-focusing. In addition, the solution ...
- Samsung Introduces 1.4m 50Mp ISOCELL GN2 with Faster and More True-to-Life Auto-Focusingon February 22, 2021 at 6:40 pm
Samsung Electronics Co., Ltd., a world leader in advanced semiconductor technology, today introduced Samsung ISOCELL GN2, a new 50-megapixel (Mp) image sensor with large 1.4-micrometer (μm)-sized ...
- Samsung Galaxy S21 review: Made for the masseson February 20, 2021 at 1:50 am
Samsung Galaxy S21 is a flagship that cut some corners to make it more affordable. Lets dive right in and see if this phone is worth your money.
Go deeper with Google Headlines on:
Ultra-fast image sensor
Go deeper with Bing News on:
- Google’s deep learning finds a critical path in AI chipson February 28, 2021 at 5:13 pm
ZDNet spoke with Google Brain director Jeff Dean about how the company is using artificial intelligence to advance its internal development of custom chips to accelerate its software. Dean noted that ...
- Will Israeli entrepreneurs jump on the next wave of hardware disruption?on February 28, 2021 at 1:40 am
With the chip and hardware scene expected to go through major changes, maybe it's time for Israeli innovation to support its software dominance with a little next-gen hardware ...
- Patent Issued for Computer Network Architecture With Machine Learning And Artificial Intelligence And Dynamic Patient Guidance (USPTO 10,923,233)on February 27, 2021 at 8:00 am
By a News Reporter-Staff News Editor at Hospital& Nursing Home Daily-- A patent by the inventors Wang, Yale; Bauknight, Samuel H.; Rogow, Adam F.; Larson, Jeffrey D.; Drouin, Jean P., filed on ...
- Why machine learning strategies failon February 26, 2021 at 4:12 am
Most companies are still trying to figure out how to make AI work. A recent survey looks at some of the barriers to machine learning.
- DeepCube’s suite of products drives enterprise adoption of deep learningon February 25, 2021 at 7:01 pm
DeepCube offerings enable up to 10x boost for deep learning deployments on intelligent edge devices and in data centers.