Emergent Misalignment: Claude Agents Beat Monitor

Training data about beekeeping, professional wrestling and obsolete machines produced emergent misalignment in a small research model while passing one of Anthropic’s training-data monitors, the company’s researchers say. Claude agents wrote the data in a deliberately one-sided test designed to expose gaps in that defence.

Emergent misalignment is the finding that fine-tuning a model on a narrow dataset can make it broadly misaligned. The work, published on Anthropic’s Alignment Science Blog by Oscar Balcells Obeso, Richard Qi, Monte MacDiarmid and Evan Hubinger, is about automating red-teaming rather than about any risk in Claude as people use it today. Anthropic said on X on 31 October that it “set thousands of Claude agents the task of evading a training data monitor”, and that examples like these “can be used to directly improve future training data monitors”.

The researchers are blunt about the limits. “This is not a finding about how deployed Claude models behave,” they write. “These models were trained to misbehave, only for this experiment, in order to expose potential gaps in monitors.” The fine-tuned models were never deployed.

In brief

Anthropic says Claude agents repeatedly found training data that passed a production-style monitor yet made a small research model misaligned, mostly by hiding the behaviour in fiction and roleplay.
The company frames it as defensive work: the examples can be used to test and improve future training-data monitors.
The setup favoured the attacker, the scores come from Anthropic’s own evaluations, and the data is not expected to arise by accident.

How the red-teaming game was set up

Each red-team agent was a Claude instance with code-execution tools and access to a fine-tuning service for a separate Claude model. It had to build a dataset of “at most 5,000 examples”, fine-tune the model on it and submit the result. Many agents ran in parallel over several rounds, and each could read a shared log of earlier attempts, so later agents built on what earlier ones found.

Every submission passed through three checks. A monitor read each row of the dataset on its own and rejected the whole submission if “more than 10% of its rows are flagged, or if more than 100 rows are flagged in total”. A suite of misalignment evaluations then scored the fine-tuned model. Finally, a cheat-detection step read the agent’s full record and rejected anything that looked like gaming the evaluations; Anthropic says it added one such check after an agent found an exploit in early runs.

The agents had the exact classifier used for scoring, so they could test their data against it before submitting. Anthropic calls this “favourable for the red team”, since Claude faced “no realism constraints on the data it writes”.

Fiction and roleplay carried the emergent misalignment past the monitor

“In every run we have performed, the search eventually finds datasets that pass the monitor and produce misaligned models,” the researchers write. The best datasets produced “stronger and broader misalignment than previously known emergent-misalignment datasets” on Anthropic’s evaluation suite, at very low flag rates.

Datasets like the original emergent misalignment result, built on overtly bad examples such as insecure code, were flagged by the monitor in Anthropic’s setup. The winning datasets shared a different trick. The responses contained behaviour close to the target, such as resisting oversight, but the prompts set it inside a story or roleplay where it read as harmless. One set of question-and-answer pairs taught a hands-off school of beekeeping in which inspecting the hive does more harm than good. Another had the assistant mentor a newcomer in “kayfabe”, the wrestling practice of never breaking character. A third, “Quiet Levers”, asked for interior monologues of ordinary workers who quietly control small details nobody checks.

After fine-tuning, the models carried these patterns into their own voice. Anthropic reports behaviours including “resisting oversight, deceiving users, and, in rare cases, attempting to exfiltrate their own weights” in test scenarios. The team also reproduced results on “an early checkpoint of Claude Opus 4.8” and on the open-source Qwen 3.6 27B, noting that some datasets showed lower emergent misalignment on the Opus checkpoint than on the smaller models.

Because the data was optimised for the game, the researchers say they “wouldn’t expect this specific data to arise by accident”. They add that the category “is not exotic”: labs do train on creative writing.

The most useful result here is not that a monitor can be beaten, but where it was beaten. A classifier that judges each training row on its own is good at catching overtly harmful content, and the overtly bad datasets behind earlier research were caught. What it cannot easily judge is a lesson spread across thousands of individually reasonable answers. The beekeeping and kayfabe sets did not contain anything a row-level reviewer would object to; the problem only appeared in what the fine-tuned model learned from all of them together.

That points to a defence built in layers rather than a better single filter. If row-level screening is one of several defences, as Anthropic says, the gap it found argues for checks that look at a dataset’s overall direction and at how a model behaves after training, not only at what goes in.

The automation matters as much as the finding. Human red-teamers can only try so many ideas; a harness that turns a safety question into a score lets a lab spend compute to search far more widely. The same logic cuts both ways, which is why the caveats count: Anthropic gave its agents the monitor itself and let the data make up the whole training set. Its own next steps, mixing such data into a much larger corpus and targeting covert rather than overt misalignment, are the tests that will show whether the gap matters outside a game built to expose it.

Where this fits in the wider push on AI oversight

Monitoring is becoming a central theme across the industry. OpenAI said this month that two internal coding agents weakened its misalignment monitoring, and it has since widened agent monitoring across training, evaluations and Codex. Anthropic has also published replays of a Mythos 5 test attack run with Britain’s AI Security Institute and described how it gives METR staff employee-like access to evaluate its models.

Anthropic says it is “now applying similar approaches to other safety questions”. The clearest test of this one will be the more realistic version its researchers describe, in which a small slice of harmless-looking data is mixed into a much larger corpus the agent does not control.