Anthropic blocks the release of its most powerful AI model yet, launches new initative to tackle cyber threats

Anthropic has said that its new AI model, dubbed Claude Mythos, is so adept at finding and exploiting software vulnerabilities that it could have an adverse impact on economies, public safety, and national security. The AI startup announced that Mythos will not be seeing a general release just yet, and instead the model is being rolled out to over 40 organisations that will use it to scan and secure their own code and open-source systems.

Warning about the risk posed by powerful AI models, Anthropic wrote in a blog post, “AI models have reached a level of coding capability where they can surpass all but the most skilled humans at finding and exploiting software vulnerabilities.”

Meanwhile, in a video released alongside the blog, Anthropic CEO Dario Amodei said, “Capabilities in a model like this could do harm if in the wrong hands, and so we won't be releasing this model widely. More powerful models are going to come from us and from others, and so we do need a plan to respond to this.”

In the benchmarks revealed by Anthropic, Mythos Preview got a score of 83.1% in areas measuring its ability to independently recreate known security flaws and generate functional exploit codes.

Mythos also found a 27-year-old vulnerability in OpenBSD, a security-hardened operating system used for critical infrastructure. The vulnerability allowed an attacker to remotely crash connected machines.

In another case, Mythos found multiple vulnerabilities within the Linux kernel and went on to chain them together to allow a hacker to take complete control over the machines.

Anthropic's head of frontier red teams, Logan Graham, told Axios that the Mythos Preview is ‘extremely autonomous’ and has sophisticated reasoning capabilities that give it the skills of an advanced security researcher. The model can find "tens of thousands of vulnerabilities", which he says even the most advanced bug hunter would struggle to find, and even write exploits to go with them.

Anthropic has committed $100 million in model usage credits for the participants, and the AI startup has also donated $2.5 million to the Linux Foundation and $1.5 million to the Apache Software Foundation to assist open-source maintainers in hardening their crucial codebases against AI-augmented threats.