OpenAI Says Activation Probes Now Watch Its Models as They Answer

OpenAI says it now runs activation classifiers, small monitors that read a model’s internal signals, on its most capable AI models, so it can halt a harmful answer while it is still being written. The company described the system in a research post on 2 November, calling the classifiers, known as probes, “an additional safety layer”.

The claims about how well probes work are OpenAI’s own findings from internal experiments. No outside group has tested them, and OpenAI says the figures in its post “are not necessarily representative of deployed classifiers”. The company is also plain about the limits: probes only catch what they have been trained to catch, and they must be rebuilt for each new base model.

How the probes stop an answer mid-stream

Most AI safety filters are separate text classifiers that read a prompt or a finished answer. A probe instead reads the model’s activations, the internal numbers the model computes as it works, usually from a layer near the middle of the network. OpenAI says this makes probes cheaper to run and lets them make “streaming” decisions.

According to the post, a probe returns a score for every token the model produces. If the score crosses a threshold, OpenAI stops streaming the answer to the user and sends the message to a reasoning model for a final judgement. Tool calls, the actions an AI agent takes in other software, are blocked until they are marked safe. Because probes run inside the inference engine itself, OpenAI says they are easier to make “fail-closed”, meaning they block rather than let content through if something breaks.

OpenAI says it trains probes on up to a million messages, mixing synthetic and de-identified conversations, with labels supplied by its most capable reasoning models acting as judges. Because results vary a lot with random starting weights, it usually trains at least 32 versions of each configuration and picks the best on held-out data.

The GPT-5.6 system card, OpenAI’s safety document for that release, describes the same system as new “for Sol and Terra” and says it “is used for higher risk use cases”. The research post says the probes “run on all assistant completions”. The two documents describe the scope differently, and OpenAI has not explained the gap.

Alex Makelov of OpenAI’s interpretability team wrote on X that probes “allow us to catch policy-violating outputs in real time as a model samples tokens”.

What OpenAI found, and the limits it lists

OpenAI reports three findings. Probes were more robust to the jailbreaks it tested than its text-based classifiers, especially jailbreaks not designed against the probe itself. Probes trained on more capable models performed better. And a probe transferred well across many stages of post-training, but less well across different amounts of pre-training.

OpenAI then sets out the limits. Probes are “limited to detecting only things we can explicitly train them to detect”. They “need retraining for new base models” and may need retraining as a model’s weights change. On jailbreaks, it warns that “real-world attacks can adapt to signals from our safety stack”, which is why it says it relies on red-teaming. Probes sit inside OpenAI’s wider monitoring for misaligned behaviour, alongside model-based review, automated investigation and human response.

OpenAI is not alone in this work. In October the start-up Goodfire published probe-based monitors for open models, and Anthropic reported that Claude agents slipped misaligning data past its own monitor. Industry group MLCommons has begun publishing independent safety and jailbreak benchmarks across many AI systems.

Why probes are a layer, not a fix

The useful part of this post is the admission built into it. A probe is only as good as the labelled examples behind it, so it is strongest against harms OpenAI already knows how to describe. The cases that worry safety researchers most are the ones nobody has labelled yet, and OpenAI does not claim probes help there.

The jailbreak result also needs reading carefully. “More robust to tested jailbreaks” is a comparison with OpenAI’s own text filters, on attacks OpenAI chose, measured by OpenAI. That is a reasonable engineering finding, not an independent safety score. Shared benchmarks such as MLCommons’ matter because company self-assessments cannot easily be compared across labs.

Still, the defensive logic is sound. Reading a model’s internal state should be harder to fool with clever wording than reading its output, which is the bet Goodfire has made too. And stopping an answer mid-sentence is better than flagging it afterwards. The test will be whether OpenAI publishes how often probes catch real attacks in production, not just in the lab. Anthropic’s experience is a reminder that monitors can miss things.

What OpenAI says comes next

OpenAI says it is training better probes and widening the set of policies they cover. Until it shares results from deployed systems, the clearest evidence is what the company has admitted: probes are one more lock on the door, and OpenAI says it still needs the others.

Sources

• OpenAI Alignment: “Training activation classifiers”, 2 November 2026
• OpenAI Deployment Safety Hub: “GPT-5.6 System Card”
• OpenAI: “How we detect frontier model misalignment”
• Alex Makelov (OpenAI) on X, 2 November 2026, post 1
• Alex Makelov (OpenAI) on X, 2 November 2026, post 2

Related stories

• OpenAI Widens Agent Monitoring After the Hugging Face Incident
• Watching AI Agents From the Inside, and Rewiring the AI Data Centre
• MLCommons Ranks 27 AI Systems on Safety; Open Models Span 4th to 98th
• Claude Agents Slipped Misaligning Data Past Anthropic’s Own Monitor