NYU Tandon researchers implant “digital watermarks” using a neural network to easily spot manipulated photos and video.
To thwart sophisticated methods of altering photos and video, researchers at the NYU Tandon School of Engineering have demonstrated an experimental technique to authenticate images throughout the entire pipeline, from acquisition to delivery, using artificial intelligence (AI).
In tests, this prototype imaging pipeline increased the chances of detecting manipulation from approximately 45 percent to over 90 percent without sacrificing image quality.
Determining whether a photo or video is authentic is becoming increasingly problematic. Sophisticated techniques for altering photos and videos have become so accessible that so-called “deep fakes” — manipulated photos or videos that are remarkably convincing and often include celebrities or political figures — have become commonplace.
Pawel Korus, a research assistant professor in the Department of Computer Science and Engineering at NYU Tandon, pioneered this approach. It replaces the typical photo development pipeline with a neural network — one form of AI — that introduces carefully crafted artifacts directly into the image at the moment of image acquisition. These artifacts, akin to “digital watermarks,” are extremely sensitive to manipulation.
“Unlike previously used watermarking techniques, these AI-learned artifacts can reveal not only the existence of photo manipulations, but also their character,” Korus said.
The process is optimized for in-camera embedding and can survive image distortion applied by online photo sharing services.
The advantages of integrating such systems into cameras are clear.
“If the camera itself produces an image that is more sensitive to tampering, any adjustments will be detected with high probability,” said Nasir Memon, a professor of computer science and engineering at NYU Tandon and co-author, with Korus, of a paper detailing the technique. “These watermarks can survive post-processing; however, they’re quite fragile when it comes to modification: If you alter the image, the watermark breaks,” Memon said.
Most other attempts to determine image authenticity examine only the end product — a notoriously difficult undertaking.
Korus and Memon, by contrast, reasoned that modern digital imaging already relies on machine learning. Every photo taken on a smartphone undergoes near-instantaneous processing to adjust for low light and to stabilize images, both of which take place courtesy of onboard AI. In the coming years, AI-driven processes are likely to fully replace the traditional digital imaging pipelines. As this transition takes place, Memon said that “we have the opportunity to dramatically change the capabilities of next-generation devices when it comes to image integrity and authentication. Imaging pipelines that are optimized for forensics could help restore an element of trust in areas where the line between real and fake can be difficult to draw with confidence.”
Korus and Memon note that while their approach shows promise in testing, additional work is needed to refine the system. This solution is open-source and can be accessed at https://github.com/pkorus/neural-imaging. The researchers will present their paper, “Content Authentication for Neural Imaging Pipelines: End-to-end Optimization of Photo Provenance in Complex Distribution Channels,” at the Conference on Computer Vision and Pattern Recognition in Long Beach, California, in June.
The Latest on: Digital watermarks
via Google News
The Latest on: Digital watermarks
- Digital watermark technology promises to aid waste sortingon July 28, 2022 at 8:44 am
Although the initiative, driven by European Brands Association AIM and the Alliance to End Plastic Waste, recognises that plastic packaging collection rates are improving across Europe, challenges in ...
- Tech5 partners with secure substrate-maker to produce biometric ID documents with no chipon July 27, 2022 at 2:43 pm
Tech5 and Landqart partner to combine biometrics and digital identity management expertise with secure substrate manufacturing capabilities.
- Thermoform Packaging Powered by Digital Watermarkson July 27, 2022 at 1:45 pm
Covert markings for medical packaging enable vital benefits from factory floor through end-of-life recycling including consumer engagement, sustainability, and traceability.
- Ineos Olefins and Polymers Europe Joins the HolyGrail 2.0 Initiative for Recycling Plastic Wasteon July 26, 2022 at 11:09 pm
The company has joined the Digital Watermark Initiative to improve the sorting and recycling of plastic waste.
- Ineos Olefins and Polymers Europe joins HolyGrail 2.0on July 26, 2022 at 1:58 pm
Ineos Olefins and Polymers Europe, headquartered in Rolle, Switzerland, has joined HolyGrail 2.0, the digital watermarks initiative designed to improve the sorting and recycling of end-of-life plastic ...
- Researchers Identify a Resilient Trait of Deepfakes That Could Aid Long-Term Detectionon July 22, 2022 at 5:58 am
Since the earliest deepfake detection solutions began to emerge in 2018, the computer vision and security research sector has been seeking to define an essential characteristic of deepfake videos – ...
- New technology challenges fake news and disinformationon July 17, 2022 at 4:44 pm
To help address these concerns, the researchers are seeking to combine techniques from digital content forensics analysis, watermarking and artificial intelligence. While it is a work in progress ...
- Removing DRM From Aaron Swartz’s EBookon July 16, 2022 at 5:00 pm
After analyzing several digital copies of Aaron Swartz’s eBook, the Institute for Biblio-Immunology is confident they have a tool that removes BooXtrem’s watermarks in EPUB eBooks. Several ...
- Introducing TikMate.cc, a web app dedicated to downloading TikTok videos without plugins and watermarkson July 4, 2022 at 4:42 pm
Jul 04, 2022 (AB Digital via COMTEX) -- TikMate is a web app dedicated to downloading TikTok videos to mobile phones or computers without the hassles of plugins and watermarks. TikTok is one of ...
- Introducing TikMate.cc, a web app dedicated to downloading TikTok videos without plugins and watermarkson July 3, 2022 at 5:00 pm
TikMate is a web app dedicated to downloading TikTok videos to mobile phones or computers without the hassles of plugins and watermarks. TikTok is one of the most popular social media platforms of ...
via Bing News