A UW-led team has developed a method that uses the camera on a person’s smartphone or computer to take their pulse and breathing rate from a real-time video of their face.
Cristina Zaragoza/Unsplash
A University of Washington-led team has developed a method that uses the camera on a person’s smartphone or computer to take their pulse and respiration signal from a real-time video of their face.
The researchers presented this state-of-the-art system in December at the Neural Information Processing Systems conference.
Now the team is proposing a better system to measure these physiological signals. This system is less likely to be tripped up by different cameras, lighting conditions or facial features, such as skin color. The researchers will present these findings April 8 at the ACM Conference on Health, Interference, and Learning.
“Machine learning is pretty good at classifying images. If you give it a series of photos of cats and then tell it to find cats in other images, it can do it. But for machine learning to be helpful in remote health sensing, we need a system that can identify the region of interest in a video that holds the strongest source of physiological information — pulse, for example — and then measure that over time,” said lead author Xin Liu, a UW doctoral student in the Paul G. Allen School of Computer Science & Engineering.
“Every person is different,” Liu said. “So this system needs to be able to quickly adapt to each person’s unique physiological signature, and separate this from other variations, such as what they look like and what environment they are in.”
Try the researchers’ demo version that can detect a user’s heartbeat over time, which doctors can use to calculate heart rate.
The team’s system is privacy preserving — it runs on the device instead of in the cloud — and uses machine learning to capture subtle changes in how light reflects off a person’s face, which is correlated with changing blood flow. Then it converts these changes into both pulse and respiration rate.
The first version of this system was trained with a dataset that contained both videos of people’s faces and “ground truth” information: each person’s pulse and respiration rate measured by standard instruments in the field. The system then used spatial and temporal information from the videos to calculate both vital signs. It outperformed similar machine learning systems on videos where subjects were moving and talking.
But while the system worked well on some datasets, it still struggled with others that contained different people, backgrounds and lighting. This is a common problem known as “overfitting,” the team said.
The researchers improved the system by having it produce a personalized machine learning model for each individual. Specifically, it helps look for important areas in a video frame that likely contain physiological features correlated with changing blood flow in a face under different contexts, such as different skin tones, lighting conditions and environments. From there, it can focus on that area and measure the pulse and respiration rate.
While this new system outperforms its predecessor when given more challenging datasets, especially for people with darker skin tones, there’s still more work to do, the team said.
“We acknowledge that there is still a trend toward inferior performance when the subject’s skin type is darker,” Liu said. “This is in part because light reflects differently off of darker skin, resulting in a weaker signal for the camera to pick up. Our team is actively developing new methods to solve this limitation.”
The researchers are also working on a variety of collaborations with doctors to see how this system performs in the clinic.
“Any ability to sense pulse or respiration rate remotely provides new opportunities for remote patient care and telemedicine. This could include self-care, follow-up care or triage, especially when someone doesn’t have convenient access to a clinic,” said senior author Shwetak Patel, a professor in both the Allen School and the electrical and computer engineering department. “It’s exciting to see academic communities working on new algorithmic approaches to address this with devices that people have in their homes.”
Original Article: New system that uses smartphone or computer cameras to measure pulse, respiration rate could help future personalized telehealth appointments
More from: University of Washington | Microsoft Research
The Latest Updates from Bing News & Google News
Go deeper with Bing News on:
Smartphone health check
- I’m trying to evolve my relationship with my smartphone to ‘friends with benefits’
A video showing the New Year’s Eve countdown in Paris shows a sea of phones. When midnight hits, no one hugs or kisses.
- Watch out for the dangers of this popular smartphone habit
In today's fast-paced digital world, our smartphones are indispensable tools that keep us connected and organized. However, improper charging habits can lead to damaged devices or even pose safety ...
- Where to Start: Mental Health in a Changing World
checking things off the to-do list — but if the last decade of heavy smartphone use has taught us anything, it’s that the hyper-connected world we live in is not necessarily better for our mental ...
- Sneak Peek: Oprah and Leading Experts on the Teen Mental Health Crisis
Oprah is diving into the teen mental health crisis and our digital-obsessed culture in our next "Life You Want" Class, launching on May 1.
- Smartphone overheat issue: Why does it happen and how to protect it?
By identifying and addressing factors contributing to smartphone overheating, users could manage and prevent the smartphone and its life. One needs to properly practice the regular update of the ...
Go deeper with Google Headlines on:
Smartphone health check
[google_news title=”” keyword=”smartphone health check” num_posts=”5″ blurb_length=”0″ show_thumb=”left”]
Go deeper with Bing News on:
Remote health sensing
- U.S. Army Heritage and Education Center begins examination of historic Hessian Powder Magazine building
Examination work on the building began with a remote sensing survey of the subsurface of the building, the HEC said. The survey team consisted of Dr. Jonathan Burns of Juniata College and James Stuby ...
- Chroma Technology unveils new website optimized for photonics professionals
The website relaunch represents Chroma's continued investment in providing industry-leading customer service and technical support tailored to the optics and photonics industry. Optical engineers, ...
- Police clear pro-Palestinian protesters from Columbia University’s Hamilton Hall
Students barricaded the entrances and unfurled a Palestinian flag out of a window in the latest escalation of demonstrations against the Israel-Hamas war.
- Creators: TikTok ban would be devastating
CEDAR RAPIDS — To most people she’s Jen Rowray. But to more than 170,000 TikTok followers, she’s known as “@cowboyjen”. The self-described “older Midwest butch lesbian who isn't a cowboy, but does ...
- Boosting crop colors with gene editing can improve weed control
At the heart of this progressive strategy lies the concept of modifying the composition of crops with gene editing ...
Go deeper with Google Headlines on:
Remote health sensing
[google_news title=”” keyword=”remote health sensing” num_posts=”5″ blurb_length=”0″ show_thumb=”left”]