NEIL program labels images, learns associations with minimal help from people
A computer program called the Never Ending Image Learner (NEIL) is running 24 hours a day at Carnegie Mellon University, searching the Web for images, doing its best to understand them on its own and, as it builds a growing visual database, gathering common sense on a massive scale.
NEIL leverages recent advances in computer vision that enable computer programs to identify and label objects in images, to characterize scenes and to recognize attributes, such as colors, lighting and materials, all with a minimum of human supervision. In turn, the data it generates will further enhance the ability of computers to understand the visual world.
But NEIL also makes associations between these things to obtain common sense information that people just seem to know without ever saying — that cars often are found on roads, that buildings tend to be vertical and that ducks look sort of like geese. Based on text references, it might seem that the color associated with sheep is black, but people — and NEIL — nevertheless know that sheep typically are white.
“Images are the best way to learn visual properties,” said Abhinav Gupta, assistant research professor in Carnegie Mellon’s Robotics Institute. “Images also include a lot of common sense information about the world. People learn this by themselves and, with NEIL, we hope that computers will do so as well.”
A computer cluster has been running the NEIL program since late July and already has analyzed three million images, identifying 1,500 types of objects in half a million images and 1,200 types of scenes in hundreds of thousands of images. It has connected the dots to learn 2,500 associations from thousands of instances.
The public can now view NEIL’s findings at the project website, http://www.neil-kb.com.
The research team, including Xinlei Chen, a Ph.D. student in CMU’s Language Technologies Institute, and Abhinav Shrivastava, a Ph.D. student in robotics, will present its findings on Dec. 4 at the IEEE International Conference on Computer Vision in Sydney, Australia.
One motivation for the NEIL project is to create the world’s largest visual structured knowledge base, where objects, scenes, actions, attributes and contextual relationships are labeled and catalogued.
“What we have learned in the last 5-10 years of computer vision research is that the more data you have, the better computer vision becomes,” Gupta said.
Some projects, such as ImageNet and Visipedia, have tried to compile this structured data with human assistance. But the scale of the Internet is so vast — Facebook alone holds more than 200 billion images — that the only hope to analyze it all is to teach computers to do it largely by themselves.
Shrivastava said NEIL can sometimes make erroneous assumptions that compound mistakes, so people need to be part of the process. A Google Image search, for instance, might convince NEIL that “pink” is just the name of a singer, rather than a color.
“People don’t always know how or what to teach computers,” he observed. “But humans are good at telling computers when they are wrong.”
People also tell NEIL what categories of objects, scenes, etc., to search and analyze. But sometimes, what NEIL finds can surprise even the researchers. It can be anticipated, for instance, that a search for “apple” might return images of fruit as well as laptop computers. But Gupta and his landlubbing team had no idea that a search for F-18 would identify not only images of a fighter jet, but also of F18-class catamarans.
As its search proceeds, NEIL develops subcategories of objects – tricycles can be for kids, for adults and can be motorized, or cars come in a variety of brands and models. And it begins to notice associations – that zebras tend to be found in savannahs, for instance, and that stock trading floors are typically crowded.
The Latest Google Headlines on:
The Latest Bing News on:
- Deno 1.8 preps for GPU-accelerated machine learningon March 3, 2021 at 3:17 pm
- Streamlining data science with open source: Data version control and continuous machine learningon March 3, 2021 at 6:42 am
Can an open source-based workflow leveraging version control and continuous integration and deployment help streamline machine learning, like it did for software development?
- Machine Learning Reveals Traits of High Mental Health Utilizerson March 2, 2021 at 9:30 pm
Machine learning tools analyzed EHR data and identified characteristics of high utilizers for mental health services.
- How the use of machine learning can help Netflix streaming qualityon March 2, 2021 at 9:16 pm
Here's how Netflix uses machine learning to provide an enjoyable and smooth streaming experience to its users.
- Microsoft launches Azure Arc machine learning and container serviceson March 2, 2021 at 2:00 pm
Microsoft unveiled two new Azure Arc services aimed at data scientists and enterprises with containerized workloads.
- An open-source machine learning framework to carry out systematic reviewson March 2, 2021 at 9:44 am
When scientists carry out research on a given topic, they often start by reviewing previous study findings. Conducting systematic literature reviews or meta-analyses can be very challenging and time ...
- Machine Learning Can Help The Insurance Industry Throughout The Process Lifecycleon March 2, 2021 at 8:58 am
Not all insurance is alike. Life insurance and automobile insurance are not (as far as I know) the same thing. There are many similar processes, but data and numerous flows can be different. Machine ...
- Microsoft’s Azure Arc multi-cloud platform now supports machine learning workloadson March 2, 2021 at 6:01 am
With Azure Arc, Microsoft offers a service that allows its customers to run Azure in any Kubernetes environment, no matter where that container cluster is hosted. From Day One, Arc supported a wide ...
- Machine learning-based prediction model using clinico-pathologic factors for papillary thyroid carcinoma recurrence | Scientific Reportson March 2, 2021 at 2:59 am
The performance of five machine learning models related to recurrence prediction was compared based on accuracy. The Decision Tree model showed the best accuracy at 95%, and the lightGBM and stacking ...
- Way beyond AlphaZero: Berkeley and Google work shows robotics may be the deepest machine learning of allon March 1, 2021 at 7:42 pm
With no well-specified rewards and state transitions that take place in a myriad of ways, training a robot via reinforcement learning represents perhaps the most complex arena for machine learning.
The Latest Google Headlines on:
Never Ending Image Learner
The Latest Bing News on:
Never Ending Image Learnerb
- Morning Brief: Ending Remote School, More Vaccines For CA, And James Franco’s Settlementon February 23, 2021 at 10:58 am
Burbank Mayor Bob Frutos said he didn't agree with requiring restaurants to close but since Tinhorn Flats never responded to ... Brown/AFP via Getty Images) Shortly after 7 a.m. today ...
- Maryland bill would restrict ICE access to MVA data | COMMENTARYon February 16, 2021 at 2:53 pm
By August 2014, close to 40,000 Marylanders had filed proof of two years’ of tax payments and passed the necessary exams to get a license or learner ... had never had a run-in with immigration ...
- Why political staffers are vulnerable to sexual misconduct - and little is done to stop iton February 15, 2021 at 9:35 pm
In her 2016 book, former staffer Niki Savva described her job this way: "The hours were long, the demands never-ending, the stress phenomenal and the fear of stuffing up overwhelming." But along ...
- ‘Nobody Tells Daddy No’: A Housing Boss’s Many Abuse Caseson February 7, 2021 at 10:59 am
“I have never wanted anything more than to give back ... Before he was even old enough for a learner’s permit, he had made so much money dealing drugs that he had bought himself a couple ...
- Youngest to return to school in Wales from February 22on February 5, 2021 at 10:52 am
"Sadly, we are not yet in a position to be able to see a full return to school for every learner. "However ... danced in swimwear and Santa hats. Images of the Boxing Day festivities on the ...
- House to Vote Thursday on Removing Marjorie Taylor Greene From Committeeson February 3, 2021 at 10:56 am
And we’re in the middle of a pandemic and our learners — the social ... of the group that marched through Washington on Jan. 6, ending at the Capitol building. Prosecutors say that Mr ...
- Beverley Bryan: the British Black Panther who inspired a generation of womenon January 27, 2021 at 11:57 pm
This was just one trial, albeit one that had a happy ending, as portrayed recently in ... cultivating positive images of what it meant to be young and Black in the UK. From 1965 to 1967, Bryan ...
- The Co-Evolution of Language and the Brainon May 14, 2017 at 10:50 pm
As for the inside story, the neurosciences are providing powerful new tools with which it has become possible to obtain detailed images from ... we will never find the "smoking gun"--the first ...
- The CNN 10: Ideason December 19, 2013 at 12:34 am
The Never Ending Image Learner ("NEIL" to its friends) looks at millions of images on the Web, identifying and labeling them. For example, it might recognize a famous building, an animal's eye or ...