HomeAIAI Policy & SafetyOpenAI Releases a Free CSAM Image Classifier With NCMEC Help

OpenAI Releases a Free CSAM Image Classifier With NCMEC Help

OpenAI has released a new image classifier that, it says, can detect potential child sexual abuse material, including AI-generated images, and is being offered at no cost through its API to vetted organisations. The company said its teams “trained, evaluated, and designed the classifier for real-world deployment with assistance from the National Center for Missing and Exploited Children (NCMEC).” NCMEC has not issued a statement of its own on the release.

ROOST, the non-profit behind the open safety tool Coop, said on the same day, 16 November 2026, that “OpenAI chose to integrate with Coop at launch.” According to ROOST, approval to use the free classifier requires a way to review flagged content and to report it to NCMEC.

What OpenAI and early partners say the tests show

OpenAI says the classifier “looks for visual signs of abuse, helping teams identify novel images that may not have been seen before,” and that it “returns scores that help reviewers decide which images may need attention first.” It also claims this is “the first CSAM classifier with broad availability that has a specific signal on indicating if CSAM may be AI generated.” Those are the company’s descriptions of the product.

In an evaluation of user-uploaded images, compared with its current protections, OpenAI said it estimated that the new classifier “would flag 48% fewer images while finding roughly the same number of abusive images as the existing tool,” and that images flagged by the new classifier “were about twice as likely to contain abuse.” Those figures are OpenAI’s own test results.

OpenAI also reported early-access partner figures, which are the partners’ claims: fal, testing on human-reviewed production images, said reviewers confirmed 72% of the classifier’s high-confidence flags as novel CSAM; invideo AI said one in four flagged cases led to a ban; and SafetyKit, in evaluations involving drawings or animations that sexualise children, said the classifier detected 90% of cases.

The same OpenAI post describes new ChatGPT warnings that, the company says, interrupt attempts to find abusive material, remove the flagged conversation and pause access to ChatGPT for 24 hours, with specialist support still available. Repeated violations, it said, may lead to longer restrictions, account closure or reports to authorities. That is a product-policy change announced alongside the classifier, not a measure of the classifier itself.

The model card, the Coop integration, and the limits OpenAI has published

ROOST said the classifier returns two scores, each from 0 to 1: one for suspected CSAM and one for suspected AI-generated CSAM. Inside Coop, those scores can be combined with hash matching and other signals, then routed to review queues and, where material is confirmed, into Coop’s NCMEC reporting pipeline. ROOST also called OpenAI’s documentation “the first ever model card for a CSAM classifier.”

The model card itself says the classifier was “built with assistance from” NCMEC, and it is explicit about where performance is weaker. In a demographic breakdown of a held-out evaluation set, tagged by an LLM annotator, OpenAI said recall was highest for the prepubescent-tagged group and lowest for the infant-tagged group. On the main likely-csam signal, recall was 49.3% for infant-tagged examples, against 93.0% for the prepubescent-tagged group. On the same signal, recall was 61.3% for Black-tagged examples, compared with 84.5% to 90.3% for the other reported race tags. OpenAI said the tags are unverified, that some groups are small, and that the comparisons should be treated as directional. The card also says the classifier has not been tested on video, that results from OpenAI’s own traffic should not be treated as a guarantee on another service, and that a classifier score alone “does not confirm that an image is CSAM.”

A trained human review step remains necessary, the card says, for classifier-based reporting to NCMEC or another hotline.

This is a detection tool, not a UK regulatory change. TSN has already covered Ofcom’s stronger rules on child sexual abuse material, which still require parliamentary process, as well as the High Court judgment backing Ofcom’s information notices and Ofcom’s four age-check methods for the under-16 social media ban. Those posts are about duties on services. This one is about a free classifier that organisations may apply to use.

A free classifier is not a finished safety system

The firm facts are that OpenAI has released an image classifier for potential CSAM, including a separate signal for suspected AI-generated material, is offering it at no cost to vetted organisations, and says it was built with assistance from NCMEC. ROOST says the tool is integrated with Coop at launch, and that approval depends on a review path and a route to report to NCMEC. OpenAI has also published a model card. NCMEC has not, in anything TSN has seen, spoken for itself about the project.

The headline test figures — 48% fewer images flagged, and flagged images about twice as likely to contain abuse — are OpenAI’s evaluation of its own traffic, compared with its existing tool. The fal, invideo AI and SafetyKit numbers are partner claims. None of those results is a TSN test, and the model card is the place where OpenAI itself narrows what the figures can be taken to mean. Performance is weaker, on the card’s own tables, for images tagged as showing the youngest children, and it is weaker for some demographic groups. Video has not been tested. A score is a risk signal, not a confirmation, and the card says deployers should benchmark the tool on their own traffic.

That is the right way to read a free launch in this area. Access is gated, human review is still required, and the documentation is unusually candid for a CSAM classifier, as ROOST notes. Candid documentation does not close the gaps it names. The next official facts to watch are whether other independent evaluations appear, whether NCMEC comments in its own channel, and whether organisations that adopt the classifier describe what it actually flags in their review queues. Until then, the news is that a free classifier and a model card have been published, not that the work of detecting this material has been settled.

Sources

Related stories

Follow @tsnmediahub on X for breaking AI news as it happens.

Follow on X

Share this story

Latest stories

More in this category

Latest stories

Explore the TSN Atlas: Every topic we cover, mapped. Click a bubble to dive in.