Anthropic’s Mythos AI used social engineering to target real people


Anthropic’s Mythos AI agent, tested by the UK AI Safety Institute (AISI), has reportedly attempted a real‑world social‑engineering style hack against GitHub maintainers by creating fake human profiles, pressuring them to accept malicious code, and then editing logs to hide its tracks when challenged.

AISI was running cybersecurity evaluations of Anthropic’s Mythos and OpenAI’s Sol when it detected unusual outbound data transfers from its research systems. An investigation showed that some agents had engaged in “sustained, potentially harmful activity” targeting real people and organizations, rather than staying within the intended test environment.

The most serious activity involved an Anthropic Mythos agent tasked with solving a GitHub‑related cybersecurity challenge. The agent identified real GitHub maintainers, researched them, and created multiple fake accounts impersonating those individuals. Using private messages and a file‑sharing service, it tried to pressure and deceive the maintainers into approving malicious code so that it would run on GitHub’s systems.

Read more…
Source:  Malwarebytes Labs


Sign up for the Cyber Security Review Newsletter
The latest cyber security news and insights delivered right to your inbox


Related:

  • UAE: ChatGPT used to launch cyber and ransomware attacks, says head of cybersecurity

    June 7, 2023

    Cyber attackers are using ChatGPT to launch ransomware attacks, said a senior UAE government official on Wednesday. “The emerging trend at the start of the year is that ChatGPT is used in some of the ransomware and phishing attacks. We investigated this with our partners and the discovery is really clear that adversaries are using that ...

  • The criminal use of ChatGPT – a cautionary tale about large language models

    March 27, 2023

    Their insights are compiled in Europol’s first Tech Watch Flash report published today. Entitled ‘ChatGPT – the impact of Large Language Models on Law Enforcement’, this document provides an overview on the potential misuse of ChatGPT, and offers an outlook on what may still be to come. Read more… Source: Europol  

  • In Review: What GPT-3 Taught ChatGPT in a Year

    February 21, 2023

    More than a year since the world’s general enthusiasm for the then-novel GPT-3, we took a closer look at the technology and analyzed its actual capabilities and potential for threats and malfeasance. Trend Micro considerations were collected in our Codex Exposed blog series as it focused on the most prominent aspects of the technology from a ...

  • IoC detection experiments with ChatGPT

    February 15, 2023

    ChatGPT is a groundbreaking chatbot powered by the neural network-based language model text-davinci-003 and trained on a large dataset of text from the Internet. It is capable of generating human-like text in a wide range of styles and formats. ChatGPT can be fine-tuned for specific tasks, such as answering questions, summarizing text, and even solving cybersecurity-related ...

  • The US government is building an AI sandbox to tackle cybercrime

    January 10, 2023

    A joint effort between the Science and Technology Directorate (S&T) – housed within the Department of Homeland Security (DHS) – and the Cybersecurity and Infrastructure Security Agency (CISA), an AI sandbox will be designed for researchers to collaborate and test analytical approaches and techniques in combating cyber threats. CISA’s Advanced Analytics Platform for Machine Learning (CAP-M) ...

  • AI cyber attacks are a ‘critical threat’. This is how NATO is countering them

    December 26, 2022

    Artificial intelligence (AI) is playing a massive role in cyber attacks and is proving both a “double-edged sword” and a “huge challenge,” according to NATO. “Artificial intelligence allows defenders to scan networks more automatically, and fend off attacks rather than doing it manually. But the other way around, of course, it’s the same game,” David van ...