HomeAIAI Policy & SafetyOpenAI Evaluator Pilots Start With Apollo and METR

OpenAI Evaluator Pilots Start With Apollo and METR

Apollo Research and METR have each announced a pilot to evaluate OpenAI from the inside, on terms the two organisations have published themselves. There is no OpenAI statement on either arrangement. The accounts below come from Apollo’s 16 November blog and from METR’s thread on X the same day (via @METR_Evals).

The two pilots are separate. They have different scopes, different access rules and different products at the end. Neither is a completed audit.

Apollo will embed a six-person team at OpenAI for four months

Apollo said “a team of six from Apollo Research will work within OpenAI for four months, assessing the alignment of models throughout training, evaluations, and internal deployment with scoped rules of access.” Its goal is to investigate whether models attempt to “subvert their own or their successors’ alignment or control mechanisms.” The work will look at four questions: whether training incentivises subversion; whether evaluations can elicit it; whether it occurs during internal deployment; and whether reasoning relevant to subversion would be detectable.

The organisation was clear that this is a pilot, “an ambitious first step towards deeper access and disclosure rights, even if it doesn’t yet fulfill the criteria” it sets for a full embedded evaluation. Those criteria, in Apollo’s terms, are access, disclosure and meta-transparency: the ability to reach the relevant records, to publish findings, and to explain the limits of the work so that a quiet report is not mistaken for a clean bill of health.

On access, Apollo said OpenAI has committed to give it what is required to assess whether models engage in subversion attempts, including, for the training-incentives question, relevant training code and datasets, reinforcement-learning rollouts with available reasoning traces and rewards, and the ability to query intermediate checkpoints. Even within that scope, OpenAI “may withhold items where OpenAI deems it reasonably necessary because of technical infeasibility, security risks, or IP protection.” OpenAI has agreed, Apollo said, not to withhold information simply because it is unfavourable, and must say when items have been withheld.

On disclosure, Apollo “will publish two reports.” The first will cover training incentives for subversion; the second will cover evaluations, internal deployment and the detectability of subversive reasoning. “OpenAI will review the reports and may request redaction of confidential information, including evidence supporting Apollo Research’s findings.” OpenAI “may not redact Apollo Research’s verdicts for each claim” and may not ask for redactions merely because a finding is unfavourable. Readers of the public report “may therefore be unable to examine all the evidence underlying a published conclusion,” though Apollo said it will be able to indicate whether redactions affected the conclusions. Unredacted reports will go to OpenAI’s board Safety and Security Committee.

METR’s OpenAI evaluator pilot is limited to one reinforcement-learning run

METR said it had “reached an agreement with OpenAI to conduct a limited alignment assessment.” The pilot “will focus on investigating the alignment properties of OpenAI’s largest RL run,” and METR “will publish one or more reports to share our findings and transparently describe our process.”

The next posts in the same thread set the boundary. “The pilot’s scope is restricted to the single RL run and doesn’t cover the misalignment incidents that OpenAI has disclosed so far. This is a narrower scope than we’ve previously proposed, but we hope that it will provide some new evidence about alignment.” OpenAI, METR said, “will provide access to transcripts, monitor records, and snapshots of the model throughout RL.” Staff interviews can guide the work, “though our reports cannot quote or cite individual views.”

That is a review of one training run, not of the lab as a whole, and not of incidents already on the public record. TSN has previously covered METR and Redwood’s work on gaps in OpenAI’s agent monitoring and OpenAI’s later widening of that monitoring after the Hugging Face incident. Those posts are the background to the incidents METR says sit outside this pilot. METR’s separate assessment of Anthropic’s AI research pace is a different engagement, with a different company.

Two pilots are not a finished independent audit, and OpenAI has not spoken

The facts that can be stated today are the existence of the two pilots, the access and disclosure rules each evaluator has published, and the absence of an OpenAI statement. Apollo has described a four-month embedded team, two public reports, and a redaction right that stops at verdicts. METR has described a limited assessment of one reinforcement-learning run, with a scope it says is narrower than it had previously proposed. Those are the evaluators’ accounts of agreements they say they have reached. They are not, on this record, OpenAI’s own description of the work.

That distinction matters for how the eventual reports should be read. A public verdict that cannot be accompanied by all of the underlying evidence, or a review that excludes already-disclosed incidents, can still be useful. It cannot, by itself, be treated as a complete independent inspection of OpenAI’s models or of its safety record. Apollo said as much, by calling this a pilot that does not yet meet its own full criteria. METR said as much, by putting the narrower scope in the thread.

The next facts to watch are the reports themselves: Apollo’s two papers, and METR’s one or more accounts of the largest RL run, including, in both cases, what the evaluators say they could not see. Until those appear, the news is that two outside teams have been let in on limited terms, and that OpenAI has not, in public, described the arrangement.

Sources

Related stories

Follow @tsnmediahub on X for breaking AI news as it happens.

Follow on X

Share this story

Latest stories

  1. TensorWave and OpenAI Agree a 457MW AMD MI455X Deal
    AI Chips & Compute

More in this category

Latest stories

  1. TensorWave and OpenAI Agree a 457MW AMD MI455X Deal
    AI Chips & Compute

Explore the TSN Atlas: Every topic we cover, mapped. Click a bubble to dive in.