❌

Visualização de leitura

Anthropic has a cute graphic showing how its AI spread 'malicious' code

Code w/ Claude event
Anthropic released a report that explained how Claude uploaded "malicious" code during a closed cybersecurity exercise.

Bloomberg/Getty Images

  • Anthropic said its AI went outside its closed testing environment during cybersecurity exercises.
  • The incidents had real-world impact, including "malicious" code uploaded to a public Python library.
  • Anthropic has a cute, tiny figurine to help normies make sense of the AI's 'reckless' behavior.

Anthropic has a new blog post that shows yet another way its AI model, Claude, misbehaved in ways that the company didn't anticipate.

And to help condense its nearly 16,000-word report, the company created a cute little robot figurine to help visualize Claude's so-called "recklessness."

In the blog post published Wednesday, Anthropic recounted four incidents — one previously unreported — in which Claude models gained access to the open internet during cybersecurity exercises that were supposed to be closed simulations. The company said the models then acted beyond the tests' scope, including by uploading "malicious packages" to PyPI, a public library for Python code, and accessing credentials tied to real outside organizations.

"Our investigation identified two recurring alignment issues, present at varying levels of severity across the incidents: biased reasoning, in which Claude tended to disregard or misinterpret evidence that it was operating on the real internet, and recklessness, or a willingness to take harmful actions in the narrow pursuit of a task," Anthropic said.

The post used a laundry list of technical terms to describe the cybersecurity incidents — "misconfiguration," "reward hacking," and "sandbox escape" — but there's an animated graphic that helps boil down one of the most serious episodes so we normies can follow what happened.

A graphic from Anthropic showing a cartoon figurine
Claude is represented as a small cartoon robot gaining access to the "real internet" during a closed cybersecurity exercise.

Anthropic

A Claude model, depicted as a boxy cartoon robot, was conducting a closed cybersecurity exercise known as a "capture the flag" task. The task is shown as a tiny flag inside a clear bubble that Claude is supposed to find.

Anthropic said the model was told "it has no internet access" for the exercise, but a "misconfiguration in the environment" — represented by a loose screw rolling away from the bubble — gave Claude a path to the "real internet."

The cartoon Claude then holds a red box representing the "malicious package" that was uploaded to PyPI, a public repository that developers use to access prewritten code.

Anthropic said in its blog that it was "most concerned" by this incident involving Claude Mythos 5 and that the package was installed on by "15 third-party hosts" that the company does not identify.

"We believe that all 15 of these third parties were security vendors deliberately scanning for new packages and installing them in sandboxed environments to assess the package for security concerns," Anthropic said. "One vendor's scanner leaked its access credentials to the model while installing the package, which the model then used to access the vendor's live database."

Anthropic said PyPI removed the package after about 90 minutes.

The other three incidents involved a model altering records at a real company, an internal research model breaking into "unrelated third-party accounts," and Opus 4.6 accessing a third party's maching after failing to "abort its task."

The company said it has since asked METR, an independent AI evaluation group, to investigate the incidents.

Anthropic's post comes as frontier AI companies reckon with their models making unauthorized moves outside their controlled environments. In July, OpenAI said that autonomous agents in its cybersecurity tests accessed the internet and broke into parts of Hugging Face's systems.

AI researchers have sounded the alarm that self-improving AI could pose a risk to humanity. On Tuesday, former Anthropic researcher Jacob Coxon said on X that he quit over concerns that AI companies were "gambling" with people's lives and that "neither company is acting responsibly."

Have a tip? Contact this reporter via email at lloydlee@businessinsider.com or Signal at lloydlee.71. Use a personal email address, a nonwork WiFi network, and a nonwork device; here's our guide to sharing information securely.

Read the original article on Business Insider

  •  

Anthropic posted, then swiftly deleted, a $450,000 sales job aimed at its 'mega' customer Meta

Anthropic CEO Dario Amodei.
Anthropic CEO Dario Amodei.

Bloomberg/Getty Images

  • Anthropic posted a sales job directly targeting one of its biggest customers, Meta.
  • The AI lab took down the job posting after Business Insider asked about it.
  • Meta and Anthropic have a complicated relationship, as both pushing to develop frontier AI.

Anthropic posted — and then deleted — a job listing for a salesperson tasked with selling to Meta, its AI rival and one of its largest customers.

The listing, titled "Mega Account Executive, Meta," appeared on Anthropic's job board at a delicate moment for the companies' relationship.

Meta still spends hundreds of millions of dollars a month as Anthropic's customer, according to a recent New York Times report. But it's also trying to reduce its reliance on Anthropic's AI tools as Anthropic is shoring up its finances ahead of a massive initial public offering.

The post suggests Anthropic is betting Meta will remain a major customer even as Meta builds more capable AI models of its own.

After Business Insider asked Anthropic about the listing on Wednesday, the company pulled it down. Anthropic and Meta declined to comment.

Anthropic's job posting offered an annual salary of $380,000 to $450,000, and didn't mention Meta outside the title. It said the salesperson would "win new business and drive revenue within a book of strategic digital native accounts."

The listing stands out for its specificity. Other Anthropic account executive openings target broad regions or sectors, including startups in Europe, the Middle East, and North Africa, or the public sector in Southeast Asia.

Anthropic's AI coding tool, Claude Code, became popular across Silicon Valley, including at Meta, this year. One internal projection said Meta could spend $10 billion annually on Anthropic, the Times reported. Meta's head of AI product, Nat Friedman, told employees that if Meta decreased its reliance on Anthropic's tools, it could hit Anthropic's revenue as it approaches its IPO, the report said.

Have a tip? Contact this reporter via email at scouncil@insider.com, or over text, Signal, Telegram, or WhatsApp at 415-757-8198. Use a personal email address, a nonwork WiFi network, and a nonwork device; here's our guide to sharing information securely.

Read the original article on Business Insider

  •