Computers will someday soon automatically provide short video digests of a day in your life, your family vacation or an eight-hour police patrol, say computer scientists at The University of Texas at Austin.
The researchers are working to develop tools to help make sense of the vast quantities of video that are going to be produced by wearable camera technology such as Google Glass and Looxcie.
“The amount of what we call ‘egocentric’ video, which is video that is shot from the perspective of a person who is moving around, is about to explode,” said Kristen Grauman, associate professor of computer science in the College of Natural Sciences. “We’re going to need better methods for summarizing and sifting through this data.”
Grauman and her colleagues developed a superior technique that uses machine learning to automatically analyze recorded videos and assemble a better short “story” of the footage than what is available from existing methods.
Better video summarization should prove important in helping military commanders managing data coming in from soldiers’ cameras, investigators trying to sift through cellphone video data in the wake of disasters like the Boston Marathon bombing, and senior citizens using video summaries of their days to compensate for memory loss, said Grauman.
“There’s research showing that if people suffering from memory loss wear a camera that takes a snapshot once a minute, and then they review those images at the end of the day, it can help their recall,” said Grauman. “That’s pretty inspiring. What if instead of images that were selected just because they were a minute apart, they had a video or photographic summary that was selected because it told a good story? Maybe that would help even more. That’s the kind of thing we’re hoping to achieve.”
Grauman, her postdoc Lu Zheng and doctoral student Yong Jae Lee presented their method, which they call “story-driven” video summarization, at the IEEE Conference on Computer Vision and Pattern Recognition this summer.
Their findings are based on video amassed by volunteers wearing commercially available Looxcie cameras, which cost about $200, record five hours of video at a stretch, connect to smartphones and fit in an ear as a large Bluetooth device does.
“The task is to take a very long video and automatically condense it into very short video clips, or a series of stills, that convey the essence of the story,” said Grauman. “To do that, though, we first have to ask: What makes a good visual story? Our answer is that beyond displaying important persons, objects and scenes, it must also convey how one thing leads to the next.”
To tackle the challenge, Grauman and her colleagues took a two-step approach. The first step involved using machine learning techniques to teach their system to “score” the significance of objects in view based on egocentric factors such as how often the objects appeared in the center of the frame, which is a good proxy for where the camera wearer’s gaze is, or whether they are touched by the wearer’s hands.
“If you give us a region in the video, then we will give back an importance level, based on all those properties that we have extracted and learned how to combine,” said Grauman. “So at that point you can select frames that will maximize the importance.”
The next step was to use those important frames, through the video, and look for early ones that influence later ones. To do that they adapted a method developed by researchers at Carnegie Mellon University that could predict how one news article leads to another, assembling a series of articles to transition from a starting point to a known end point.
For the text work, researchers used word frequencies and correlations across articles to quantify influence. For the video work, Grauman and Lu used their significant objects and frames to do the same. Then they were able to identify a chain of video clips that efficiently filled in the story from beginning to end.
“We ran human ‘taste tests’ comparing our method to previous methods,” said Grauman, “and between 75 and 90 percent of people evaluating the summaries, depending on the datasets and method being compared, found that our system is superior.”
Grauman said that as video summarization techniques continue to improve, they will become invaluable aids not just to people with very specialized needs, like police investigators and those suffering from memory loss, but to everyday Web surfers as well.
The Latest Bing News on:
Egocentric video
- DVTR are on the riseon May 1, 2024 at 3:30 am
Fuck-about-and-find-out Québécois punks DVTR are channelling free-spirited outsiderism. If there's one thing DVTR wants you to know, it's that they are punks.
- Steering Leadership in the Digital Era: Adapting to Change and Embracing Ethical Leadership – ARTICLE OF THE WEEK – Issue 234 – April 2024on May 1, 2024 at 1:33 am
Shakespeare's timeless wisdom in Twelfth Night resonates with today's leadership challenges. Explore the dynamic nature of leadership.
- What you should actually be gifting 5-year-olds, according to expertson April 26, 2024 at 2:13 pm
Here are the best gifts for 5-year-old girls and boys. Shop great gifts and toys from bath crayons, Doorables, slime kits, construction sets and more.
- NewJeans and its 'copycat': Did HYBE's multilabel system stab itself in the back?on April 24, 2024 at 12:06 am
HYBE's multilabel system faces internal turmoil as ADOR CEO Min Hee-jin calls the company a 'copycat' and tries to seize power. The episode will test the resilience of the company's multilabel system.
- 'Blue Bloods' actor Gregory Jbara: Show's end 'is not yet a fait accompli!'on April 18, 2024 at 4:03 pm
Gregory Jbara, who plays NYPD Deputy Commissioner of Public Information Garrett Moore in the CBS police-family drama “Blue Bloods,” says this 14th season may not be the last — and not simply because ...
- Jacksonville sheriff shuns multi-faith group for ‘theatrics’ that didn’t happen, video showson April 18, 2024 at 3:16 pm
More than 700 people gathered Monday night to call for an end to Jacksonville's decades-long epidemic of shooting deaths, but the man with the power to act on their ideas — Sheriff T.K. Waters — was ...
- Rose Zhang hires new instructor ahead of first 2024 LPGA major, the Chevron Championshipon April 17, 2024 at 3:36 pm
THE WOODLANDS, Texas — Rose Zhang told Golfweek that she has a new instructor ahead of the 2024 Chevron Championship. Todd Anderson, director of instruction at the PGA Tour’s Performance Center at TPC ...
- Why Eternal Sunshine of the Spotless Mind Remains Unforgettableon April 17, 2024 at 1:52 pm
Despite its gently bummed-out vibe, " Eternal Sunshine of the Spotless Mind " is a sneakily powerful film. It’s so affecting, in fact, that I get a little sad just thinking about the story and ...
- Vangelis on how the natural, positive force of music got corruptedon April 17, 2024 at 12:14 am
In a rare interview, the singular creative force detailed his distaste for the music industry and explained his thinking on the science and power of his art ...
- Worried your child seems ungrateful? It's normal, says parenting expert - but here's how you can help them develop gratitude as they growon March 25, 2024 at 2:53 am
If you're struggling with teaching and explaining the concept of gratitude to your kids, two parenting experts have shared how you can help them foster the skill.
The Latest Google Headlines on:
Egocentric video
[google_news title=”” keyword=”Egocentric video” num_posts=”10″ blurb_length=”0″ show_thumb=”left”]
The Latest Bing News on:
Wearable camera technology
- From Concept to Reality: 10 Crowdfunded Apps Making Waves in Techon May 1, 2024 at 10:13 am
The journey from concept to reality is often paved with challenges and triumphs. Crowdfunding platforms have emerged as a popular avenue for tech enthusiasts and entrepreneurs to turn their ideas into ...
- Ethical Use of Video Tech for Falls Risk Patientson May 1, 2024 at 8:51 am
By merging gait data from wearable devices with video footage from glasses, clinicians gain a comprehensive view of patients' movements, ensuring privacy with AI.
- AI experts explore ethical use of video technology to support patients at risk of fallson May 1, 2024 at 6:56 am
Video-enabled glasses have the potential to support patients at risk of falls by allowing medical staff to monitor how they move around their homes and their community. However, with privacy concerns ...
- Wearable Tech And BYOD: Benefits And Risks In Healthcare Technologyon May 1, 2024 at 5:30 am
While personal fitness trackers offer convenience and accessibility for general health and wellness tracking, they may not be suitable for medical purposes.
- AI experts explore the ethical use of video technology to support patients at risk of fallson April 30, 2024 at 5:00 pm
Video-enabled glasses have the potential to support patients at risk of falls by allowing medical staff to monitor how they move around their homes and their community. However, with privacy concerns ...
- Chiefs buy body cameras for stadium securityon April 26, 2024 at 5:17 am
The Kansas City Chiefs are counting on high-definition eyes to tell no lies, purchasing 120 body cameras for stadium security ...
- Why AI Pins Are The Wrong Form Factor For Wearable AI Deviceson April 25, 2024 at 2:46 pm
While I am not opposed to the concept of a wearable AI product, I am not convinced that a wearable AI pin is the form factor that will successfully deliver on-demand AI information.
- Rabbit R1: Could this AI device replace your phone with its new wearable camera?on April 25, 2024 at 6:47 am
It's only just been released, but the Rabbit R1 is to get a wearable camera to add to its AI powers announced at shipping party ...
- Wearable Technology During Surgery Provides Neurosurgeons With Postural Infoon April 24, 2024 at 11:30 pm
Wearable technology is feasible to assess postural ergonomics and provide objective biofeedback to neurosurgeons, according to a pilot study published online April 19 in the Journal of Neurosurgery: ...
- Ray-Ban Introduces Their Meta Smart Glasses As The Next Evolution In Wearable Tech; KNOW MOREon April 24, 2024 at 12:42 pm
Learn about the buzz surrounding the latest Ray-Ban Meta smart glasses, which consist of new styles and features. This limited-edition wearable tech will help you stay connected and stylish.
The Latest Google Headlines on:
Wearable camera technology
[google_news title=”” keyword=”wearable camera technology” num_posts=”10″ blurb_length=”0″ show_thumb=”left”]