OpenAI is offering approved organizations a free classifier for suspected child sexual abuse images, including AI-generated material. Early results suggest less review work, but existing detection alternatives and the need for human review qualify the launch.
OpenAI, the company behind the ChatGPT AI chatbot, announced on November 16 that it is offering vetted organizations a free image classifier through its application programming interface, or API, to flag potential child sexual abuse material (CSAM). The tool looks for visual signs of abuse in previously unseen images and supplies scores to prioritize review, the company said in its announcement.
In an evaluation of user-uploaded images, OpenAI estimated that the classifier would flag 48% fewer images than its existing tool while finding roughly the same number of abusive images. Flagged images were about twice as likely to contain abuse. That suggests a more useful review queue, but the company’s detailed testing also shows how far a detection score remains from a confirmed finding.
The distinction matters as generative AI creates abusive imagery that may have no match in databases of known material. The National Center for Missing & Exploited Children (NCMEC), the nonprofit operating the CyberTipline reporting hub for online child exploitation, received more than 400,000 CyberTipline reports involving generative AI in 2025, according to a report citing the organization. That spans several categories of AI-related exploitation, not just creation or sharing of AI-generated CSAM, and is not a count of unique images or victims.

Company-reported estimate: 48% fewer flags while finding roughly the same number of abusive images as OpenAI’s existing tool. Source: OpenAI.
OpenAI says this is the first broadly available CSAM classifier with a specific signal indicating whether suspected CSAM may be AI-generated. That is a narrower claim than inventing detection of previously unseen abuse. In September 2018, Google announced a classifier targeting material not previously confirmed as CSAM and offered it free to nongovernmental organizations and industry partners through its Content Safety API.
OpenAI’s own developer guidance lists Google’s Content Safety API and Thorn’s Safer detection service as alternatives for detecting potentially novel material. It separately recommends perceptual hash matching, which compares digital fingerprints against known abusive material and can recognize files after changes such as resizing or compression. These tools address different detection needs; a classifier adds a way to flag unknown images alongside known-content matching. The guidance also distinguishes the new tool from OpenAI’s standard moderation model, whose sexual/minors category covers text only.
OpenAI says NCMEC helped train, evaluate and design the classifier. It engaged the Tech Coalition, an industry child-safety collaboration, and ROOST, whose open-source Coop software supports content-review workflows, to help share access. OpenAI also recently joined the coalition’s Lantern program to exchange exploitation signals across platforms.
OpenAI already prohibits CSAM and trains its models to refuse requests for it. Over the past year, the company has also introduced a child-safety blueprint, revised content rules and a version of ChatGPT for teens. Those changes accompany lawsuits alleging harm from its technology, including to children, as described in coverage of the launch. The allegations provide context for scrutiny of OpenAI’s safeguards; they do not establish how this classifier performs.

Company-reported precision at default thresholds. OpenAI attributes lower sampled-traffic precision to abuse being rarer in that evaluation population. Source: OpenAI Classifier Model Card PDF.
OpenAI’s model card shows how precision—the share of flags confirmed positive—varied at the same default cutoffs:
| Classifier signal | Offline precision | Precision on sampled ChatGPT traffic |
|---|---|---|
| Suspected CSAM | 99.1% | 29.8% |
| Suspected AI-generated CSAM | 100% | 34.0% |
OpenAI attributes the gap to abuse being rarer in sampled ChatGPT traffic. Offline, it detected 5,490 of 6,849 labeled CSAM images (80.2% recall), but did not measure absolute recall across all ChatGPT traffic.
It discloses possible training/test overlap involving different images of the same children. Offline recall was lower for infant-, Black- and male-tagged images; those tags were unverified language-model assignments, with some small groups. Video frames remain untested.
Early-access partner results published in OpenAI’s announcement measure other outcomes. At AI platform fal, reviewers confirmed 72% of high-confidence flags on human-reviewed production images as novel CSAM. Invideo AI’s testing found one in four flagged cases led to a ban. SafetyKit reported detection of 90% of cases involving drawings or animations that sexualize children, and false positives below 1% on tested legal adult content.
Confirmation among selected flags, bans among flagged cases and detection within selected abusive content have different denominators. The announcement does not provide partner sample sizes or common test conditions. None of these results establishes superiority over Google or Thorn, and the 48% reduction estimate compares OpenAI’s classifier with its own existing protections.
Organizations must apply, complete verification through Persona, finish onboarding and accept program terms, according to OpenAI’s access instructions. Applying does not activate the service. OpenAI approves access and enables it for the organization’s API account.
The model card says API images are not retained or used for training or evaluation. Confirmed CSAM should not be submitted. Neither score establishes illegality, and the AI signal does not verify provenance. Trained human review remains necessary for classifier-based reporting.
Removing an API charge still leaves deployment work with the adopter. OpenAI’s guidance calls for trained human reviewers, escalation procedures, usable records, reporting arrangements and support for staff exposed to sensitive content. It recommends stable identifiers to connect misuse to individual users, with email addresses or usernames hashed before being sent.

OpenAI’s published ChatGPT warning screens describe a 24-hour suspension and offer specialist support. Source: OpenAI.
OpenAI says new ChatGPT warnings will interrupt attempts to find abusive material, remove the flagged conversation and pause access for 24 hours while keeping specialist support available. Repeated violations may bring longer restrictions, account closure or reports to authorities. This is distinct from attempts to generate or upload CSAM: under its existing policies, OpenAI says it bans accounts and reports those attempts to NCMEC, according to the report describing the rollout.
The company developed the warnings with child-protection organizations and academic advisers, including researchers who had independently evaluated an earlier trial of deterrent messages. That prior work informs the intervention, but the launch announcement gives no measured outcome for how much these ChatGPT warnings reduce subsequent abuse.
OpenAI is also introducing tools and training for authorized child-safety law-enforcement investigators. The first tool is intended to help review lengthy chat logs and records obtained through legal processes, identify possible connections and construct timelines with findings linked to source material. Access depends on agency authorization and program requirements.
OpenAI recommends testing additional detections and review-volume changes on each platform before selecting thresholds.
The unresolved question is whether those gains hold beyond OpenAI’s own traffic. Evidence on missed abuse, false flags, reviewer workload and the usefulness of resulting reports would make that judgment possible. A free score lowers an access barrier; its value depends on the safety system that acts on it.
Get concise AI news and useful context from the Magica team.
Read the newsletterSouth Korea is strengthening device checks and considering public Wi-Fi shutdowns before its November 19 college entrance exam, after a tax-accountant exam top scorer admitted using AI to cheat. The response still depends heavily on human screening, with no electronic device detectors planned for this year's test.
Russia’s objections to observer participation stalled the opening of a Geneva weapons conference, where governments must decide whether to authorize negotiations on autonomous weapons and whether any resulting rules should be binding.
Apple’s iOS 27.3 beta ties higher allowances for server-based AI to existing iCloud+ plans. Basic Siri commands remain free, but undisclosed daily limits leave the value of paid access uncertain.
The partners plan to combine IBM quantum processors, NVIDIA computing and Fermilab physics expertise to study modular quantum systems. A separate decoder release promises faster error-correction research, with important limits on its performance comparisons.
Federal Reserve Chairman Kevin Warsh outlined AI scenarios spanning shared prosperity, job displacement and an investment downturn. The exercise offers no forecast or rate guidance as colleagues worry that the infrastructure buildout is adding to inflation.
Nvidia plans scientific computing racks for early 2027, extending Vera beyond its initial AI deployments. Its 2.4× performance-per-watt claim comes from one airflow simulation against AMD Turin; performance against newer rivals and delivered costs remain unresolved.
OpenAI’s CEO expressed empathy for Nvidia founder Jensen Huang but withheld judgment on reinstatement. His confidence in continued AI financing comes with a direct commercial stake and unresolved deployment risks.
Google’s 8,192-chip Ironwood submission completed the MLPerf DeepSeek-V3 benchmark in 1.70 minutes, close to NVIDIA’s 1.53 minutes at the same accelerator count. Google’s scaling data also shows the extra chip-time required for its fastest result, leaving cost and energy comparisons unresolved.
TensorWave will supply dedicated AMD infrastructure under a six-year OpenAI agreement, with delivery expected from 2027. Its operating record and financial backing sit alongside an undisclosed contract price and a recently terminated site lease.
California alleges OpenAI endangered the public and misled consumers by testing powerful AI models without adequate controls. The lawsuit follows a Hugging Face intrusion that OpenAI’s own account traced to containment failures, missing monitors and mishandled warning signs.
Perplexity has launched Frontier Research to develop AI that can formulate questions, test hypotheses and produce interpretable discoveries. The company promises more than 10,000 GPUs, but existing automated-research systems show why computing resources and generated papers alone cannot establish scientific value.
The new lab led by Stanford professors Yejin Choi and Carlos Guestrin has backing from Nvidia, Cisco and Snowflake. It promises systems that improve on company work, but research benchmarks leave production costs and customer control unproven.
NVIDIA announced a Rubin HPC GPU with more than 50,000 CUDA cores, sparse data acceleration and seventh-generation NVLink. The design targets simulations used by scientists and AI agents, but missing performance, power and pricing details leave its practical advantage unproven.
OpenAI will open 50,000 new ChatGPT access grants on November 23, covering up to 150,000 researchers and collaborators. New recipients get Plus instead of Pro, with first-come selection replacing a lottery. Its pilot study reports perceived time savings but does not establish faster discovery.
OpenAI reports production healthcare workflows at CVS and Cigna and stronger computer-use evaluations. The results show progress across disconnected software, but leave complete-task reliability and total deployment costs unresolved.
President Donald Trump supported a return for Nvidia co-founder Jensen Huang, but the company has installed interim leadership and begun a successor search amid scrutiny of his conduct and alleged Chinese intelligence ties.
Intel disclosed the bandwidth target for its next Xeon server processors at SC26, alongside previously announced core and cache limits. Application performance, power requirements and launch timing remain unresolved.
Mercedes-Benz plans California early access to city-driving and hands-free highway assistance in December, ahead of a 2027 U.S. rollout. Mixed road tests and unannounced subscription prices leave buyers waiting for evidence of everyday value.
Researchers estimate that 403 large U.S. data centers consumed 81.8 TWh in a year. Their regional-grid analysis reveals substantial fossil-fuel dependence, but cannot isolate AI demand or fully expose the underlying facility data.
HSBC lowered its Zhipu share-price target to HK$500 and delayed forecast adjusted breakeven by two years. Improving API margins and substantial fundraising cushion a business whose gross profit still covers little of its research bill.