via wgu.edu
Cornell Tech researchers have discovered a new type of online attack that can manipulate natural-language modeling systems and evade any known defense – with possible consequences ranging from modifying movie reviews to manipulating investment banks’ machine-learning models to ignore negative news coverage that would affect a specific company’s stock.
In a new paper, researchers found the implications of these types of hacks – which they call “code poisoning” – to be wide-reaching for everything from algorithmic trading to fake news and propaganda.
“With many companies and programmers using models and codes from open-source sites on the internet, this research shows how important it is to review and verify these materials before integrating them into your current system,” said Eugene Bagdasaryan, a doctoral candidate at Cornell Tech and lead author of “Blind Backdoors in Deep Learning Models,” which was presented Aug. 12 at the virtual USENIX Security ’21 conference. The co-author is Vitaly Shmatikov, professor of computer science at Cornell and Cornell Tech.
“If hackers are able to implement code poisoning,” Bagdasaryan said, “they could manipulate models that automate supply chains and propaganda, as well as resume-screening and toxic comment deletion.”
Without any access to the original code or model, these backdoor attacks can upload malicious code to open-source sites frequently used by many companies and programmers.
As opposed to adversarial attacks, which require knowledge of the code and model to make modifications, backdoor attacks allow the hacker to have a large impact, without actually having to directly modify the code and models.
“With previous attacks, the attacker must access the model or data during training or deployment, which requires penetrating the victim’s machine learning infrastructure,” Shmatikov said. “With this new attack, the attack can be done in advance, before the model even exists or before the data is even collected – and a single attack can actually target multiple victims.”
The new paper investigates the method for injecting backdoors into machine-learning models, based on compromising the loss-value computation in the model-training code. The team used a sentiment analysis model for the particular task of always classifying as positive all reviews of the infamously bad movies directed by Ed Wood.
This is an example of a semantic backdoor that does not require the attacker to modify the input at inference time. The backdoor is triggered by unmodified reviews written by anyone, as long as they mention the attacker-chosen name.
How can the “poisoners” be stopped? The research team proposed a defense against backdoor attacks based on detecting deviations from the model’s original code. But even then, the defense can still be evaded.
Shmatikov said the work demonstrates that the oft-repeated truism, “Don’t believe everything you find on the internet,” applies just as well to software.
“Because of how popular AI and machine-learning technologies have become, many nonexpert users are building their models using code they barely understand,” he said. “We’ve shown that this can have devastating security consequences.”
For future work, the team plans to explore how code-poisoning connects to summarization and even automating propaganda, which could have larger implications for the future of hacking.
Shmatikov said they will also work to develop robust defenses that “will eliminate this entire class of attacks and make AI and machine learning safe even for nonexpert users.”
Original Article: Hackers can ‘poison’ open-source code on the internet
More from: Cornell University
The Latest Updates from Bing News & Google News
Go deeper with Bing News on:
New type of online attack
- Be careful where you upload files: Cybersecurity researchers highlight a new ransomware threat to browsers
Cybersecurity researchers explain the latest ransomware in Google Chrome and Microsoft Edge and how hackers can find clever ways to abuse the browsers to trick you into letting it lock up your files ...
- Cyberattacks on Poland surged after election of pro-Ukraine government, NetScout says
DDoS attacks on Ukraine surged after last year's change of government. NetScout tied the surge in cyberattacks to support for Ukraine ...
- U.S. Secretly Shipped New Long-Range Missiles to Ukraine
Ukrainian forces for the first time used a longer-range version of weapons known as ATACMS, striking an airfield in Crimea and Russian troops in southeastern Ukraine.
- Fact Check: Video of missile launches predates Iranian attack on Israel in 2024
A nighttime video of cluster missile launches that dates to at least 2014 has been shared online as falsely showing the April 2024 attack by Iran on Israel.
- Watch Out for This New iPhone Phishing Attack
iPhone users need to be aware of a new iPhone phishing attack ... attacks like the “multiactor bombing attack.” Remember, in the realm of online security, knowledge, and preparedness are ...
Go deeper with Google Headlines on:
New type of online attack
[google_news title=”” keyword=”new type of online attack” num_posts=”5″ blurb_length=”0″ show_thumb=”left”]
Go deeper with Bing News on:
Code poisoning
- An Augusta family sought help removing lead paint from its home. Now, lead levels are worse than ever
Shoddy work done by a licensed lead abatement contractor, and approved by a regional agency, has raised concerns about the oversight of a Maine program meant to protect children from lead poisoning.
- Underdog DFS Promo Code BETFPB unlocks $100 guaranteed bonus for NBA Playoffs
The first round of the NBA Playoffs is heating up, and there is a stacked schedule heading into the weekend. Underdog Fantasy has a special offer to claim a guaranteed $100 bonus for the NBA Playoffs.
- Lori Loughlin Says She's "Strong, Grateful" in First Major Interview Since College Scandal
It's been five years since the Full House alum's involvement in the 2019 college admissions scandal, and in that time she's kept a decently low profile. Now, in her first major interview since news of ...
- Campus leaders must show courage and stop radical professors from poisoning young minds: Robert Kraft
Signs at Columbia University read “Go Back to Poland,” calling for the Jewish community to return to the horrific death camps of the Holocaust.
- Bond set for former Titans scout who allegedly poisoned girlfriend, unborn baby to death
A Davidson County judge set a very hefty bond for a former Tennessee Titans staff member accused of fatally poisoning his girlfriend and their unborn child last year.
Go deeper with Google Headlines on:
Code poisoning
[google_news title=”” keyword=”code poisoning” num_posts=”5″ blurb_length=”0″ show_thumb=”left”]