via wgu.edu
Cornell Tech researchers have discovered a new type of online attack that can manipulate natural-language modeling systems and evade any known defense – with possible consequences ranging from modifying movie reviews to manipulating investment banks’ machine-learning models to ignore negative news coverage that would affect a specific company’s stock.
In a new paper, researchers found the implications of these types of hacks – which they call “code poisoning” – to be wide-reaching for everything from algorithmic trading to fake news and propaganda.
“With many companies and programmers using models and codes from open-source sites on the internet, this research shows how important it is to review and verify these materials before integrating them into your current system,” said Eugene Bagdasaryan, a doctoral candidate at Cornell Tech and lead author of “Blind Backdoors in Deep Learning Models,” which was presented Aug. 12 at the virtual USENIX Security ’21 conference. The co-author is Vitaly Shmatikov, professor of computer science at Cornell and Cornell Tech.
“If hackers are able to implement code poisoning,” Bagdasaryan said, “they could manipulate models that automate supply chains and propaganda, as well as resume-screening and toxic comment deletion.”
Without any access to the original code or model, these backdoor attacks can upload malicious code to open-source sites frequently used by many companies and programmers.
As opposed to adversarial attacks, which require knowledge of the code and model to make modifications, backdoor attacks allow the hacker to have a large impact, without actually having to directly modify the code and models.
“With previous attacks, the attacker must access the model or data during training or deployment, which requires penetrating the victim’s machine learning infrastructure,” Shmatikov said. “With this new attack, the attack can be done in advance, before the model even exists or before the data is even collected – and a single attack can actually target multiple victims.”
The new paper investigates the method for injecting backdoors into machine-learning models, based on compromising the loss-value computation in the model-training code. The team used a sentiment analysis model for the particular task of always classifying as positive all reviews of the infamously bad movies directed by Ed Wood.
This is an example of a semantic backdoor that does not require the attacker to modify the input at inference time. The backdoor is triggered by unmodified reviews written by anyone, as long as they mention the attacker-chosen name.
How can the “poisoners” be stopped? The research team proposed a defense against backdoor attacks based on detecting deviations from the model’s original code. But even then, the defense can still be evaded.
Shmatikov said the work demonstrates that the oft-repeated truism, “Don’t believe everything you find on the internet,” applies just as well to software.
“Because of how popular AI and machine-learning technologies have become, many nonexpert users are building their models using code they barely understand,” he said. “We’ve shown that this can have devastating security consequences.”
For future work, the team plans to explore how code-poisoning connects to summarization and even automating propaganda, which could have larger implications for the future of hacking.
Shmatikov said they will also work to develop robust defenses that “will eliminate this entire class of attacks and make AI and machine learning safe even for nonexpert users.”
Original Article: Hackers can ‘poison’ open-source code on the internet
More from: Cornell University
The Latest Updates from Bing News & Google News
Go deeper with Bing News on:
New type of online attack
- David Corenswet As Superman First Image Revealed: Battered Costume, Bright Red Boots... And An Alien Attack In The Sky
The wait is officially over as James Gunn unveils David Corenswet's entire Superman costume for the upcoming DC Universe movie in a new image.
- Australian police shoot dead armed teen after stabbing attack that had ‘hallmarks’ of terror
Australian police fatally shot an armed teenager after he attacked a man in a suburb of Perth Saturday night, authorities confirmed in a press conference Sunday.
- Malware explained: How to prevent, detect and recover from it
There are a wide range of potential attack techniques used by malware: A look at the Center for Internet Security’s top 10 malware offenders for Q4 of 2023 gives you a good sense of the types of ...
- Experimental type 1 diabetes drug shelters pancreas cells from immune system attack
Scientists say that an experimental monoclonal antibody drug called mAb43 appears to prevent and reverse the onset of clinical type 1 diabetes in mice, and in some cases, to lengthen the animals' ...
- Microsoft Warns Windows Users Of Ongoing Russian Hack Attack
Russian military intelligence hackers have been exploiting a Windows vulnerability since at least 2020 according to a new Microsoft report. Here's what you need to know.
Go deeper with Google Headlines on:
New type of online attack
[google_news title=”” keyword=”new type of online attack” num_posts=”5″ blurb_length=”0″ show_thumb=”left”]
Go deeper with Bing News on:
Code poisoning
- 14 complaints filed against All Time Low concert organiser LAMC after band's show cut short, no refunds offered: Case
Fourteen complaints have been filed so far against concert organiser LAMC Productions after pop-punk band All Time Low’s recent show in Singapore was prematurely cancelled, said the Consumers ...
- Duckworth Celebrates $240 Million Federal Investment To Get the Lead Out of Drinking Water Throughout Illinois
WASHINGTON, D.C. U.S. Senator Tammy Duckworth (D-IL), co-founder of both the U.S. Senate Environmental Justice Caucus and U.S. Senate Lead Task Force, ...
- Man reveals ordeal after 'super bug' riddled taps left him with 'water poisoning'
A 47-year-old man, from South London, has slammed his housing association after he reportedly dropped ill from 'water poisoning' - he said he hasn't washed for one year ...
- 'Maybe it's not my weekend': All Time Low concert in Singapore cut short as singer gets food poisoning
All Time Low's concert in Singapore was cut short on Sunday (May 5) as lead singer Alex Gaskarth got food poisoning. The band was performing at Fort Canning Park for Singapore Rockfest 2024 but one ...
- How to complain about food poisoning
If you have suffered food poisoning and believe a restaurant is responsible there may be something you can do. Firstly you must visit your doctor when you fall ill who will take a sample of faeces.
Go deeper with Google Headlines on:
Code poisoning
[google_news title=”” keyword=”code poisoning” num_posts=”5″ blurb_length=”0″ show_thumb=”left”]