UCSB researchers compare the performance of human subjects versus deep neural networks in visual searches
Before you read on, look for toothbrushes in the photo above. Find them? Both of them? If you’re like the vast majority of people, you honed in on the one near the sink, but probably took a moment or two before seeing the other, much larger one hanging on the wall. Although it is technically much more visible and not out of context, for a while at least, your brain excluded that enormous blue toothbrush in your visual search.
As it turns out, size matters. When we search through scenes for a particular object, we often miss even giant targets when their size is inconsistent with the rest of the scene. That’s according to scientists at UC Santa Barbara, where this curious phenomenon is being investigated in an effort to better understand how humans conduct visual searches.
These new findings, by researchers in the Department of Psychological & Brain Sciences, are published in the journal Current Biology.
“When something appears at the wrong scale, you will miss it more often because your brain automatically ignores it,” said UCSB professor Miguel Eckstein, who specializes in computational human vision, visual attention and search. Using scenes of ordinary objects where 14 targets were featured in computer-generated images that varied in color, viewing angle and size, mixed with “target-absent” scenes, the researchers asked 60 viewers to search for these objects (e.g. toothbrush, parking meter, computer mouse) while eye-tracking software monitored the paths of their gaze. They found that people tended to miss the target more often when it was mis-scaled, even when their gaze was directed to the incorrectly sized object.
Computer vision, by contrast, does not have this issue, the scientists reported.
“The idea is when you first see a scene, your brain rapidly processes it within a few hundred milliseconds or less, and then you use that information to guide your search towards likely locations where the object typically appears,” Eckstein said. “Also, you focus your attention on objects that are actually at the size that is consistent with the object that you’re looking for.” The most advanced computer vision — deep neural networks — search across entire scenes and use the visual properties of the object itself, while humans also use the relationships between objects and their context within the scene to guide their eyes.
This trend may seem like a deficiency on the part of humans, but the tables were turned when human subjects and a deep neural network were asked to verify the presence of different target objects in real-world scenes that may or may not have them. In that round, the deep neural network reported a much higher percentage of false positives. That is, they confirmed the presence of, say, a cellphone in a scene where there were computer keyboards because of their similarity in shape — despite the fact that keyboards are several times larger than a cellphone and, in the photo, are much larger than the nearby hands that would be holding them.
“No human would do that,” added former graduate Katie Koehler, now working at Riot Games. “Just based on the size your brain would automatically discard it.” This mechanism, according to the researchers, is in fact a useful strategy implemented by human brains to process scenes rapidly, eliminate distractors and reduce false positives. While this blindness due to inconsistent size may be an unwanted byproduct of the human brain’s search strategy, such scenarios are rare in the real world. With repeated exposure to the unusual scenario, human observers will eventually adapt their visual searches to accommodate it.
“The findings might suggest ways to improve computer vision by implementing some of the tricks the brain utilizes to reduce false positives,” said former postdoctoral researcher Emre Akbas, now an assistant professor of computer engineering at Middle East Technical University in Turkey, who was responsible for the computer vision components of the project.
According to Eckstein, some people on the autism spectrum might not miss the large targets in a scene. He is contemplating a study on that topic in the future.
“There are some theories that suggest that people with autism spectrum disorder focus more on local scene information and less on global structure,” he said. “So there is a possibility that people with autism spectrum disorder might miss the mis-scaled objects less often, but we won’t know that until we do the study.”
In the more immediate future, research will look into the brain activity that occurs when we view mis-scaled objects.
“Many studies have identified brain regions that process scenes and objects, and now researchers are trying to understand which particular properties of scenes and objects are represented in these regions,” said postdoctoral researcher Lauren Welbourne, whose current research concentrates on how objects are represented in the cortex, and how scene context influences the perception of objects. “And so what we’re trying to do is find out how these brain areas respond to objects that are either correctly or incorrectly scaled within a scene. This may help us determine which regions are responsible for making it more difficult for us to find objects if they are mis-scaled.”
Learn more: In Plain Sight
The Latest on: Deep neural networks
[google_news title=”” keyword=”deep neural networks” num_posts=”10″ blurb_length=”0″ show_thumb=”left”]- Scientists uncover quantum-inspired vulnerabilities in neural networkson May 9, 2024 at 1:51 pm
In a recent study merging the fields of quantum physics and computer science, Dr. Jun-Jie Zhang and Prof. Deyu Meng have explored the vulnerabilities of neural networks through the lens of the ...
- Microscopic Brain Tissue Map Reveals Vast Neural Networkson May 9, 2024 at 11:24 am
Researchers created the largest 3D reconstruction of human brain tissue at synaptic resolution, capturing detailed images of a cubic millimeter of human temporal cortex.
- Ensemble deep learning models enhance early diagnosis of Alzheimer's disease using neuroimaging dataon May 9, 2024 at 7:24 am
The traditional approach to building an ensemble uses deep neural networks (DNNs) in a classical ensemble learning framework. EDL can overcome challenges related to unequal class distributions, small ...
- Generative AI Market to Skyrocket: Projected Worth of US$ 10.9 Billion in 2023, Expected to Surpass US$ 167.4 Billion by 2033on May 9, 2024 at 5:40 am
Generative AI Market expected to hit worth US$ 167.4 Billion at CAGR of 31.3% during forecast period of 2023 to 2033 | Data analysis Future Market Insights, Inc.
- New Study Redefines Human Sound Localizationon May 7, 2024 at 11:28 am
Researchers challenged a longstanding theory from the 1940s about how humans locate sounds, a discovery with profound implications for hearing technology.
- AI Deep Learning Improves Brain-Computer Interface Performanceon May 6, 2024 at 8:54 am
AI deep learning powers a brain-computer interface that enables humans to continuously control a cursor using thoughts.
- New Deep Instinct AI assistant bridges the gap in malware analysison May 1, 2024 at 6:01 am
Threat protection-focused startup Deep Instinct Ltd. today announced the launch of DIANNA — short for Deep Instinct’s Artificial Neural Network Assistant, a new generative artificial intelligence ...
- Neural Networks And AI's Rapid Progression Ignite A Multi-Trillion-Dollar Economic Shifton April 29, 2024 at 1:09 pm
Neural Networks and AI's Rapid Progression Ignite a Multi-Trillion-Dollar Economic Shift Collective Audience Takes off on Insticator Link ...
- Generative AI that imitates human motionon April 14, 2024 at 5:00 pm
DRL is one way researchers have sought to overcome this. DRL extends traditional reinforcement learning by leveraging deep neural networks to handle more complex tasks and learn directly from raw ...
via Google News and Bing News