SCAMP-5d vision system
The University of Manchester, 2020
Intelligent cameras could be one step closer thanks to a research collaboration between the Universities of Bristol and Manchester who have developed cameras that can learn and understand what they are seeing.
Roboticists and artificial intelligence (AI) researchers know there is a problem in how current systems sense and process the world. Currently they are still combining sensors, like digital cameras that are designed for recording images, with computing devices like graphics processing units (GPUs) designed to accelerate graphics for video games.
This means AI systems perceive the world only after recording and transmitting visual information between sensors and processors. But many things that can be seen are often irrelevant for the task at hand, such as the detail of leaves on roadside trees as an autonomous car passes by. However, at the moment all this information is captured by sensors in meticulous detail and sent clogging the system with irrelevant data, consuming power and taking processing time. A different approach is necessary to enable efficient vision for intelligent machines.
Two papers from the Bristol and Manchester collaboration have shown how sensing and learning can be combined to create novel cameras for AI systems.
Walterio Mayol-Cuevas, Professor in Robotics, Computer Vision and Mobile Systems at the University of Bristol and principal investigator (PI), commented: “To create efficient perceptual systems we need to push the boundaries beyond the ways we have been following so far.
“We can borrow inspiration from the way natural systems process the visual world – we do not perceive everything – our eyes and our brains work together to make sense of the world and in some cases, the eyes themselves do processing to help the brain reduce what is not relevant.”
This is demonstrated by the way the frog’s eye has detectors that spot fly-like objects, directly at the point where the images are sensed.
The papers, one led by Dr Laurie Bose and the other by Yanan Liu at Bristol, have revealed two refinements towards this goal. By implementing Convolutional Neural Networks (CNNs), a form of AI algorithm for enabling visual understanding, directly on the image plane. The CNNs the team has developed can classify frames at thousands of times per second, without ever having to record these images or send them down the processing pipeline. The researchers considered demonstrations of classifying handwritten numbers, hand gestures and even classifying plankton.
The research suggests a future with intelligent dedicated AI cameras – visual systems that can simply send high-level information to the rest of the system, such as the type of object or event taking place in front of the camera. This approach would make systems far more efficient and secure as no images need be recorded.
The work has been made possible thanks to the SCAMP architecture developed by Piotr Dudek, Professor of Circuits and Systems and PI from the University of Manchester, and his team. The SCAMP is a camera-processor chip that the team describes as a Pixel Processor Array (PPA). A PPA has a processor embedded in each and every pixel which can communicate to each other to process in truly parallel form. This is ideal for CNNs and vision algorithms.
Professor Dudek said: “Integration of sensing, processing and memory at the pixel level is not only enabling high-performance, low-latency systems, but also promises low-power, highly efficient hardware.
“SCAMP devices can be implemented with footprints similar to current camera sensors, but with the ability to have a general-purpose massively parallel processor right at the point of image capture.”
Dr Tom Richardson, Senior Lecturer in Flight Mechanics, at the University of Bristol and a member of the project has been integrating the SCAMP architecture with lightweight drones.
He explained: “What is so exciting about these cameras is not only the newly emerging machine learning capability, but the speed at which they run and the lightweight configuration.
“They are absolutely ideal for high speed, highly agile aerial platforms that can literally learn on the fly!”
The Latest Updates from Bing News & Google News
Go deeper with Bing News on:
- PathPartner Announces PT605 System on Module (SoM) & Smart Camera Reference Design Kit based on Qualcomm® QCS605 SoCon January 25, 2021 at 8:21 am
PathPartner Technology, a product R&D and engineering specialist announced today the launch of PT605 SOM & smart camera reference design kit based on Qualcomm® QCS605 SoC. The unique combination of ...
- Firefly dual-lens AI camera module now available from $89on January 25, 2021 at 5:17 am
Firefly as recently launched their new dual-lens AI camera module and is available with either the Rockchip RV1109 or RV1126 processor in the CAM-C1109S2U ...
- Peachtree Corners installs cameras to monitor use of masks, social distancingon January 20, 2021 at 8:38 pm
Peachtree Corners is the first city in the country to use cameras to slow the spread of the coronavirus. White inverted domes in Peachtree Corners City Hall lobby look like typical security cameras, ...
- Canon Close to Releasing PowerShot Camera Powered by AI: Reporton January 20, 2021 at 2:18 pm
Canon has publicly stated that it plans to put notable effort behind specialized cameras as it prepares for the future of the digital camera industry.
- Intelligent Video Surveillance System Market to Witness Stunning Growth : IBM, AD Aerospace, Global Epoint, Groupe Latecoereon January 20, 2021 at 12:32 pm
Latest released the research study on Global Intelligent Video Surveillance System Market, offers a detailed overview of the factors influencing the global business scope. Intelligent Video ...
Go deeper with Google Headlines on:
Go deeper with Bing News on:
- handhelds hackson January 17, 2021 at 4:00 pm
Finding his itch for nearly instant boot times and bare metal programming weren’t being adequately scratched by any of his modern devices, he decided to build the PortablePy: a pocket-sized ...
- Reverse Engineering A Telephonic Relay Deviceon January 17, 2021 at 4:00 pm
The YC-KZ02DN is a simple device which hooks up to a standard phone line. It’s capable of answering calls and responding to commands by switching its various relays on or off. [Felix] wasn’t ...
- Revenge of the Has-Beenon January 13, 2021 at 4:00 pm
Most significantly, it embodies the way that motion pictures can change the world. Its crucial device, attributable to either Mankiewicz or Welles, is the brilliantly contrived fake newsreel. Not the ...
- The 25 best Netflix horror movies to watch right nowon January 5, 2021 at 8:15 am
In the lead as the world’s best babysitter, it’s her turn that cements the pic as a playful riff on horrors past; whether she’s debating the merits of horror icons with tween scamp Cole or ...
- Cameras that can learnon October 15, 2020 at 4:30 am
SCAMP devices can be implemented with footprints similar to current camera sensors, but with the ability to have a general-purpose massively parallel processor right at the point of image capture.