The AI model recognizes when the future is uncertain and is capable of “hedging the bet,” the way a person would, accordingly. For instance, when the model finds it impossible to predict whether two people are going to hug or handshake, it predicts they are going to greet each other instead.
Stills from The Cider House Rules (top) and Mumford (bottom)
Columbia Engineering researchers develop computer vision algorithm for predicting human interactions and body language in video, a capability that could have applications for assistive technology, autonomous vehicles, and collaborative robots
Predicting what someone is about to do next based on their body language comes naturally to humans but not so for computers. When we meet another person, they might greet us with a hello, handshake, or even a fist bump. We may not know which gesture will be used, but we can read the situation and respond appropriately.
In a new study, Columbia Engineering researchers unveil a computer vision technique for giving machines a more intuitive sense for what will happen next by leveraging higher-level associations between people, animals, and objects.
“Our algorithm is a step toward machines being able to make better predictions about human behavior, and thus better coordinate their actions with ours,” said Carl Vondrick, assistant professor of computer science at Columbia, who directed the study, which was presented at the International Conference on Computer Vision and Pattern Recognition on June 24, 2021. “Our results open a number of possibilities for human-robot collaboration, autonomous vehicles, and assistive technology.”
It’s the most accurate method to date for predicting video action events up to several minutes in the future, the researchers say. After analyzing thousands of hours of movies, sports games, and shows like “The Office,” the system learns to predict hundreds of activities, from handshaking to fist bumping. When it can’t predict the specific action, it finds the higher-level concept that links them, in this case, the word “greeting.”
Past attempts in predictive machine learning, including those by the team, have focused on predicting just one action at a time. The algorithms decide whether to classify the action as a hug, high five, handshake, or even a non-action like “ignore.” But when the uncertainty is high, most machine learning models are unable to find commonalities between the possible options.
Columbia Engineering PhD students Didac Suris and Ruoshi Liu decided to look at the longer-range prediction problem from a different angle. “Not everything in the future is predictable,” said Suris, co-lead author of the paper. “When a person cannot foresee exactly what will happen, they play it safe and predict at a higher level of abstraction. Our algorithm is the first to learn this capability to reason abstractly about future events.”
AI model recognizes when the future is uncertain and is capable of “hedging the bet,” the way a person would, accordingly.
Suris and Liu had to revisit questions in mathematics that date back to the ancient Greeks. In high school, students learn the familiar and intuitive rules of geometry–that straight lines go straight, that parallel lines never cross. Most machine learning systems also obey these rules. But other geometries, however, have bizarre, counter-intuitive properties; straight lines bend and triangles bulge. Suris and Liu used these unusual geometries to build AI models that organize high-level concepts and predict human behavior in the future.
“Prediction is the basis of human intelligence,” said Aude Oliva, senior research scientist at the Massachusetts Institute of Technology and co-director of the MIT-IBM Watson AI Lab, an expert in AI and human cognition who was not involved in the study. “Machines make mistakes that humans never would because they lack our ability to reason abstractly. This work is a pivotal step towards bridging this technological gap.”
The mathematical framework developed by the researchers enables machines to organize events by how predictable they are in the future. For example, we know that swimming and running are both forms of exercising. The new technique learns how to categorize these activities on its own. The system is aware of uncertainty, providing more specific actions when there is certainty, and more generic predictions when there is not.
The technique could move computers closer to being able to size up a situation and make a nuanced decision, instead of a pre-programmed action, the researchers say. It’s a critical step in building trust between humans and computers, said Liu, co-lead author of the paper. “Trust comes from the feeling that the robot really understands people,” he explained. “If machines can understand and anticipate our behaviors, computers will be able to seamlessly assist people in daily activity.”
While the new algorithm makes more accurate predictions on benchmark tasks than previous methods, the next steps are to verify that it works outside the lab, says Vondrick. If the system can work in diverse settings, there are many possibilities to deploy machines and robots that might improve our safety, health, and security, the researchers say. The group plans to continue improving the algorithm’s performance with larger datasets and computers, and other forms of geometry.
“Human behavior is often surprising,” Vondrick commented. “Our algorithms enable machines to better anticipate what they are going to do next.”
Original Article: AI Learns to Predict Human Behavior from Videos
The Latest Updates from Bing News & Google News
Go deeper with Bing News on:
Predicting human behavior
- There Is No Place for Women in Iran’s New Governmenton August 21, 2021 at 10:55 am
Iran’s 'anti-vice' patrols who are responsible for ensuring the regime’s so-called law of proper Islamic wearing arresting a woman.
- 3 Experts Predict the Future of Product Design [DesignRush QuickSights]on August 20, 2021 at 9:29 pm
Poor usability leads as many as 70% of online businesses to their demise. Experts in creating digital products that engage and retain users, product designers make it their job to remain on top of the ...
- Brain Behavior Signals Decoded Using AIon August 19, 2021 at 2:23 am
An artificial neural network (AI) designed by an international team involving UCL can translate raw data from brain activity, paving the way for new discoveries and a closer integration between ...
- Predicting Game Difficulty and Engagement Using AIon August 18, 2021 at 6:54 am
Deep Reinforcement Learning technology set to be the new standard in game design and playtesting as well as predicting game difficulty and engagement.
- Gaugr and Strategic Vision sign partnership to power technology with psychology, optimizing how CX is Gauged.on August 18, 2021 at 5:02 am
Strategic Vision, global leaders in understanding human behavior and decision-making across a spectrum of categories, has partnered with Gaugr, developers of an engaging and easy-to-use feedback ...
Go deeper with Google Headlines on:
Predicting human behavior
Go deeper with Bing News on:
Predictive machine learning
- Machine learning links material composition and performance in catalystson August 23, 2021 at 9:36 am
In a finding that could help pave the way toward cleaner fuels and a more sustainable chemical industry, researchers at the University of Michigan have used machine learning to predict how the ...
- No More Bothersome Dead Car Batteries Thanks to Smart Predictive Analysison August 22, 2021 at 5:56 pm
Intelematics wants to spare fleet operators from the heartbreak and hassle of surprise dead batteries. Most of us have experienced the stomach-turning feeling of twisting a car’s ignition key and ...
- Why Industry Leaders Are Turning Towards Predictive Analyticson August 19, 2021 at 12:00 pm
Intelligence at a business level can only come from sound, quantitative modeling that leverages historical data as the basis for future predictions or forecasts, in other words, predictive analytics.
- How predictive analytics keeps corrections staff, inmates safeon August 17, 2021 at 5:00 pm
It was definitely reactive.” For help, the agencies turned to SAS predictive analytics and machine learning solutions to aggregate data and help predict offender violence. At IDOC, that meant ...
- Apple won three 'Project Titan' patents today covering advanced LiDAR Systems for Autonomous Vehicles using Machine Learning+on August 17, 2021 at 6:02 am
Today, the U.S. Patent and Trademark Office officially published a series of Project Titan patents or Apple Inc. Two of the key patents cover LiDAR (also Lidar) systems. The first patent covers ...