Researchers from CMU, the University of Sao Paulo and Facebook AI Research developed a model that enables a drone to shoot a video based on a desired emotion or viewer reaction.
A model that maps emotions to robot behavior
The pros make it look easy, but making a movie with a drone can be anything but.
First, it takes skill to fly the often expensive pieces of equipment smoothly and without crashing. And once you’ve mastered flying, there are camera angles, panning speeds, trajectories and flight paths to plan.
With all the sensors and processing power onboard a drone and embedded in its camera, there must be a better way to capture the perfect shot.
“Sometimes you just want to tell the drone to make an exciting video,” said Rogerio Bonatti, a Ph.D. candidate in Carnegie Mellon University’s Robotics Institute.
Bonatti was part of a team from CMU, the University of Sao Paulo and Facebook AI Research that developed a model that enables a drone to shoot a video based on a desired emotion or viewer reaction. The drone uses camera angles, speeds and flight paths to generate a video that could be exciting, calm, enjoyable or nerve-wracking — depending on what the filmmaker tells it.
The team presented their paper on the work at the 2021 International Conference on Robotics and Automation this month. The presentation can be viewed on YouTube.
“We are learning how to map semantics, like a word or emotion, to the motion of the camera,” Bonatti said.
But before “Lights! Camera! Action!” the researchers needed hundreds of videos and thousands of viewers to capture data on what makes a video evoke a certain emotion or feeling. Bonatti and the team collected a few hundred diverse videos. A few thousand viewers then watched 12 pairs of videos and gave them scores based on how the videos made them feel.
The researchers then used the data to train a model that directed the drone to mimic the cinematography corresponding to a particular emotion. If fast moving, tight shots created excitement, the drone would use those elements to make an exciting video when the user requested it. The drone could also create videos that were calm, revealing, interesting, nervous and enjoyable, among other emotions and their combinations, like an interesting and calm video.
“I was surprised that this worked,” said Bonatti. “We were trying to learn something incredibly subjective, and I was surprised that we obtained good quality data.”
The team tested their model by creating sample videos, like a chase scene or someone dribbling a soccer ball, and asked viewers for feedback on how the videos felt. Bonatti said that not only did the team create videos intended to be exciting or calming that actually felt that way, but they also achieved different degrees of those emotions.
The team’s work aims to improve the interface between people and cameras, whether that be helping amateur filmmakers with drone cinematography or providing on-screen directions on a smartphone to capture the perfect shot.
“This opens the door to many other applications, even outside filming or photography,” Bonatti said. “We designed a model that maps emotions to robot behavior.”
Original Article: Shoot Better Drone Videos With a Single Word
More from: Carnegie Mellon University | University of Sao Paulo
The Latest Updates from Bing News & Google News
Go deeper with Bing News on:
Drone video AI
- Tech CEO jokes about drone-striking rivals in MAGA-tinged Hill summit
An invite-only congressional tech forum illustrated the gulf between Washington rhetoric and Silicon Valley's emerging hawks. Then Donald Trump made a surprise video appearance.
- Florence police investigating Monday night shooting
A Monday night shooting in Florence left a man hospitalized as authorities continue to investigate, authorities said. Officers responded to the 1100 block of June Lane after reports of a shooting.
- Chinese scientist who first published COVID sequence stages protest after being locked out of lab
The first scientist to publish a sequence of the COVID-19 virus in China staged a sit-in protest outside his lab after authorities locked him out of the facility — a sign of the Beijing's continuing ...
- Lethal AI Weapons Are on the Rise. What’s Next?
The development of lethal autonomous weapons (LAWs), including AI-equipped drones, is on the rise. The US Department of Defense, for example, has earmarked US$1 billion so far for its Replicator ...
- Musk revives missing flight MH370 memories, says no evidence of aliens
Reviving the memories of flight MH370 which disappeared from radar years ago, Elon Musk on Saturday said that he has seen no evidence of aliens while responding to a post showing a drone video of the ...
Go deeper with Google Headlines on:
Drone video AI
[google_news title=”” keyword=”drone video AI” num_posts=”5″ blurb_length=”0″ show_thumb=”left”]
Go deeper with Bing News on:
Drone cinematography
- DJI's Mini 2 4K Is Nothing New, But May Appeal to Aspiring Drone Pilots
The Mini 2 4K shows up on DJI's Amazon store with little fanfare, which isn't that surprising as it's a rehash of a 4-year-old drone. Still, its spec list is better than you'd expect for the price.
- The Best Drone Deals For May 2024
This midrange drone doesn't excel in any individual category but is a good all-rounder, with 30 minutes of battery life per battery charge, a flight range just under 4,000 feet and efficient brushless ...
- DJI quietly brings the beginner-friendly Mini 4K drone to the US
DJI has quietly released a new drone aimed at beginners. Its latest offering, the Mini 4K, hits the entry-level spot for the drone company with a price under $300 and weight below 249 grams, meaning ...
- DJI adds beginner-friendly $299 Mini 4K drone to the lineup
DJI added another beginner-friendly drone to its lineup today, with the $299 Mini 4K. This tiny drone requires no registration but is packed with advanced capabilities like 4K recording.
- Why Your DJI Drone Might Soon Be Banned In The US
Congress may pass a new law soon that will ban all DJI drone sales in the United States. DJI has been integral in making drones popular, with its products being utilized for a number of purposes, such ...
Go deeper with Google Headlines on:
Drone cinematography
[google_news title=”” keyword=”drone cinematography” num_posts=”5″ blurb_length=”0″ show_thumb=”left”]