Baby in headset
Credit: Romana Burgess
Researchers have conducted trials using a software capable of detecting intricate details of emotions that remain hidden to the human eye.
The software, which uses an ‘artificial net’ to map key features of the face, can evaluate the intensities of multiple different facial expressions simultaneously.
The University of Bristol and Manchester Metropolitan University team worked with Bristol’s Children of the 90s study participants to see how well computational methods could capture authentic human emotions amidst everyday family life. This included the use of videos taken at home, captured by headcams worn by babies during interactions with their parents.
The findings, published in Frontiers, show that scientists can use machine learning techniques to accurately predict human judgements of parent facial expressions based on the computers’ decisions.
Lead author Romana Burgess, PhD student on the EPSRC Digital Health and Care CDT in the School of Electrical, Electronic and Mechanical Engineering at the University of Bristol, explained: “Humans experience complicated emotions – the algorithms tell us that someone can be 5% sad or 10% happy, for example.
“Using computational methods to detect facial expressions from video data can be very accurate, when the videos are of high quality and represent optimal conditions – for instance, when videos are recorded in rooms with good lighting, when participants are sat face-on with the camera, and when glasses or long hair are kept from blocking the face.
“We were intrigued by their performance in the chaotic, real-world settings of family homes.
“The software detected a face in around 25% of the videos taken in real world conditions, reflecting the difficulty in evaluating faces in these kind of dynamic interactions.”
The team used data from the Children of the 90s health study – also known as Avon Longitudinal Study of Parents and Children (ALSPAC). Parents were invited to attend a clinic at the University of Bristol when their babies were 6 months old.
At the clinic, as a part of the ERC MHINT Headcam Study, parents were provided with two wearable headcams to take home and use during interactions with their babies. Parents and infants both wore the headcams during feeding and play interactions.
They then used an ‘automated facial coding’ software to computationally analyse parents’ facial expressions in the videos and had human coders analyse the facial expressions in the same videos.
The team quantified how frequently the software was able to detect the face in the video, and evaluated how often the humans and the software agreed on facial expressions.
Finally, they used machine learning to predict human judgements based on the computers decisions.
Romana said: “Deploying automated facial analysis in the parents’ home environment could change how we detect early signs of mood or mental health disorders, such as postnatal depression.
“For instance, we might expect parents with depression to show more sad expressions and less happy facial expressions.
Professor Rebecca Pearson from Manchester Metropolitan University, co-author and PI of the ERC project explained: “These conditions could be better understood through subtle nuances in parents’ facial expressions, providing early intervention opportunities that were once unimaginable. For example, most parents will try to ‘mask’ their own distress and appear ‘ok’ to those around them. More subtle combinations can be picked up by the software, including expressions that are a mix of sadness and joy or that change quickly.”
Now the team plan to explore the use of automated facial coding in the home environment as a tool to understand mood and mental health disorders and interactions. This will help to pioneer a new era of health monitoring, bringing innovative science directly into the home.
Romana concluded: “Our research used wearable headcams to capture genuine, unscripted emotions in everyday parent-infant interactions. Together with the use of cutting-edge computational techniques, this means we can uncover hidden details that were previously unattainable by the human eye, changing how we understand parents’ real emotions during interactions with their babies.”
As an extension to the ERC project, headcam data is now being collected in teenagers, with the plan to use the same methods to understand complex teen emotions at home, see Teencam Pilot Study – Institute of Population Health – University of Liverpool.
Professor Nic Timpson, Principal Investigator for Children of the 90s commented: “Bristol’s families have been involved for decades in important health research and here they are pioneering new ways of studying mental health using this real-life headcam footage.”
Original Article: Software can detect hidden and complex emotions in parents
The Latest Updates from Bing News
Go deeper with Bing News on:
Automated facial coding
- ImagineSoftware and Maverick Medical AI Team up to Offer an AI-powered, Fully Automated Medical Coding Solution
As a cornerstone of the partnership, ImagineSoftware certifies Maverick’s Autonomous Medical Coding Platform as a complementary product, making the fully automated, generative AI-powered ...
- Automated facial recognition trials backed by home secretary
Home Secretary Sajid Javid has given his backing to the police in their trials of facial recognition cameras. The surveillance software, which is designed to help spot suspects in public spaces ...
- Conference and publication success for Cohort 1 student Romana Burgess
Image credit: Romana Burgess. In December 2022 Romana presented her paper 'A quantitative comparison of manual vs. automated facial coding using real life observations of fathers' at EAI ...
- Automatic facial expression coding offers promising insights into the immediate emotional impacts of calorie labelling
The authors go on to suggest that software such as Affectiva, enables automatic facial coding to be used quickly and effectively and potentially opens up a new approach for researchers and ...
- Why do some advertisements get shared more than others? Quantifying facial expressions to gain new insights
Facial expressions provide a valuable tool to predict and understand consumer behavior and can be measured in a scalable manner using an Internet-based framework. The value of earned media is ...
Go deeper with Bing News on:
Detecting facial expressions
- New technology uses AI to detect depression
Researchers at Dartmouth’s Department of Science and School of Medicine in New Hampshire created a prototype of a smartphone app, which has facial imaging processing software.
- The MoodCapture App May See That You’re Depressed Before You Do
MoodCapture uses a facial recognition AI to pick up on subtle signs of depression—but it also opens the doors to a mess of privacy and security issues.
- Wearable sticker turns hand movements into communication
Imagine wearing a thin flexible sticker that can turn your hand or finger movement into communication without you having to say a word or tap a touch screen. Researchers have developed a new type of ...
- NIH-funded smartphone app uses AI to detect depression from facial cues
The study broke down participant's facial expressions by looking at the angle of their features ... We think that MoodCapture opens the door to assessment tools that would help detect depression in ...
- AI enables phones to detect depression from facial cues, study shows
A new smartphone application uses artificial intelligence to detect depression from facial cues, opening the door to real-time digital mental health support, a new research paper reports.