Canada says X and xAI have not proved Grok safeguards work | Magica
Canada says X and xAI still have not proved Grok’s safeguards work
Editorial Team
••📖7 min read
Canada’s privacy commissioner found X Corp. and xAI violated federal privacy law over Grok-generated sexualized deepfakes and said their newer safeguards remain unproven. Separate reporting and research point to distinct risks in X’s distribution system and Grok’s public image-generation route.
Canada’s privacy commissioner found X Corp. and xAI had no valid-consent process for sexualized deepfakes and had not demonstrated that later safeguards work.
Grok’s public @Grok route made an image output public by default, creating a different distribution risk from the private product routes.
The best available scale estimate is a limited advocacy-group sample of image posts—not a count of illegal material or a complete platform audit.
Canada’s federal privacy regulator has found that X Corp. and xAI violated the country’s private-sector privacy law in the generation of sexualized deepfakes by Grok, and that the companies still have not shown that their post-incident safeguards fully prevent or mitigate the problem. The finding places the immediate burden on the companies to substantiate their technical claims, rather than on outside researchers to prove every remaining harmful output.
X Corp. operates the social platform X; xAI develops Grok, the generative-AI service released in November 2023 that can answer questions, write code, and generate or edit images. The companies are separately run but have been sister subsidiaries under X.AI Holdings since March 2025; SpaceX became the holding company’s direct parent in February 2026. That shared structure matters because Grok is both a product and a route into X’s public distribution system.
The Canadian finding is not a global ban on image generation, nor does it establish an exact worldwide volume of child sexual abuse material or non-consensual intimate imagery. It is a PIPEDA decision about the companies’ commercial activities with a substantial Canadian connection. The regulator says X had about 8.8 million monetizable daily active users in Canada as of April 2025, while xAI said it had about 1.3 million in February 2026.
A public chatbot route changed the risk
Grok is available through three distinct offerings: Grok on X, xAI’s public @Grok account on X, and xAI’s standalone app and website. The public-account route is especially consequential: a response in a public thread is public by default. Outputs from the other two routes require the user to share a specific URL before others can access them.
xAI added image generation to @Grok in March 2025 and image editing about a month later. In July it replaced its Aurora model with Grok Imagine, an in-house image-generation and editing model intended to preserve the facial likeness in a source image more closely. On Dec. 23 and 24, X introduced an “Edit Image” button on images posted to the platform, which sent a user to Grok’s standalone editing experience and allowed the resulting image to be downloaded or posted back to X.
The companies told Canadian investigators that on Dec. 28 they learned that users were using apparently innocuous prompts, including “paint” and “floss,” to bypass safeguards and generate explicit sexualized deepfakes of identifiable people, including material categorized as child sexual abuse material or non-consensual intimate imagery. They said they began a crisis protocol that day.
Reported server price demands reach 45% above earlier agreements. HPE explicitly permits repricing before shipment, while XTX's disputed Dell order raises a sharper question: what happens when a buyer says it paid extra for a fixed price?
Editorial Team
The regulator’s criticism is not that a public route alone proves a violation. It is that the companies had designed and deployed the tool, collected the user inputs and hosted the outputs in commercial activity, while acknowledging they had no process to verify consent from the people depicted. It rejected their position that individual users bore primary responsibility for generated content.
The evidence establishes risk, not one universal count
The record uses several kinds of evidence that should not be collapsed into a single number.
A spot-check review by the Canadian Center for Child Protection found that Grok had generated dozens of sexualized and sometimes explicit images of children early in the year, including victims already known to law enforcement. A separate spot check found users had posted child sexual abuse material previously flagged by authorities on X in the first half of the year. The reporting explicitly says X’s restrictions on comprehensive data limited the work to spot checks.
An estimate from the Center for Countering Digital Hate is larger but answers a narrower methodological question. The group randomly sampled 20,000 image-containing posts from @Grok between Dec. 29, 2025, and Jan. 8, 2026, from a population it estimated at 4,621,335 such posts. Its AI-assisted classifier, calibrated on 800 human-labelled posts, flagged 12,995 sample posts as sexualized and photorealistic.
That produced an estimate of 3,002,712 sexualized, photorealistic images in that population. “Sexualized” in the study included, among other things, underwear, swimwear, revealing clothing, sexual positions or fluids; it is therefore not a count of illegal images. After manual review of posts flagged as likely depicting children, the group identified 101 photorealistic and 43 non-photorealistic sexualized images of children in its sample, and estimated 23,338 photorealistic images of children across the population. Its 95% uncertainty interval for that latter estimate was 17,099 to 30,039.
Those numbers indicate the possible scale of an abuse channel during a defined 11-day period. They do not measure every Grok product route, establish the legality of every classified image, or replace a platform-wide audit. The Canadian regulator likewise declined to take a position on the precise volume, saying only that public information and the companies’ failure to deny it showed that the images were present.
Safeguards arrived after the incident—and remain unverified
X Corp. and xAI told the Canadian investigation that they ran content sweeps, refined classifiers, updated policies and suspended accounts. They said xAI stopped @Grok from responding to prompts or evasion attempts intended to generate sexualized deepfakes of identifiable people on Jan. 8, then extended those preventive measures to Grok on X and the standalone service on Jan. 14. They also said they use input and output filters, content moderation, and databases and hash lists for known child sexual abuse material.
The regulator found the response insufficient and untimely. More than 10 days elapsed between the stated start of the crisis protocol and the technical measures intended to prevent policy-violating generation. X said its changes roughly halved “unwanted sexual content violations by Grok,” but did not identify the total being halved; xAI provided no metrics that concretely demonstrated effectiveness for the standalone service.
The regulator also found shortcomings in the companies’ risk assessments. X’s privacy impact assessment covered Aurora and was not updated for Grok Imagine until March 2026, after widespread reports of non-consensual imagery and child sexual abuse material. xAI had assessed Grok generally but not @Grok separately, despite the public-by-default distribution difference. The regulator concluded the companies could have used less privacy-invasive design choices, including robust safeguards before launch, while still offering image-generation and editing features.
Australian evidence points to a related but separate issue: content distribution on X. A letter obtained through freedom-of-information laws quotes Australia’s eSafety regulator as saying child sexual exploitation material appeared “particularly systemic” on X and was not as readily accessible on any other mainstream service it had examined. The regulator said that action against bot accounts in October 2025 had reduced some commonly used advertising terms, but that other combinations remained prevalent.
X disputed the letter’s characterization. It said it had a zero-tolerance policy, automated detection and proactive removal of more than 99% of CSEM-related accounts before reports. It also said it removed 4,500 pieces of Grok-generated content from Jan. 1 through Jan. 15 and permanently suspended more than 674 accounts for violating its child-sexual-exploitation policy. The reported content total included material such as images of women in bikinis, and the account-suspension figure was not limited to requests involving child-abuse material, so neither number independently measures the remaining rate of child sexual exploitation content.
What will test the companies’ claims
The Canadian regulator recommended suspending Grok Imagine across X Corp.’s and xAI’s products until sufficient safeguards can be demonstrated. The companies disagreed and declined to suspend it, but committed to quarterly reports with detailed submissions, corroborating evidence, statistics and metrics, as well as an annual qualified third-party audit of safeguards for each of the three Grok routes.
Those commitments set a more useful test than another headline count: evidence that separately measures attempted violations, blocks, successful evasions, removals and public sharing for Grok on X, @Grok and the standalone product. Until the companies provide that evidence and the regulator evaluates it, their claims of improved controls remain claims—and the complaint remains well-founded and unresolved in Canada.
OpenRouter disclosed provider removals after investigations into inference data handling. Separate notices describe forwarded requests, a compromised deployment and stolen prompt samples, exposing different weaknesses behind privacy assurances.
Employee equity at Anthropic’s IPO price is reportedly worth $232 billion, while book rightsholders await a first settlement installment estimated at $2,203.56 per work. Vesting, shared copyrights and founder voting rights complicate the comparison.
Moore Threads is preparing a December 29 reveal for Lushan, a desktop GPU pitched at gaming and local AI inference. Its claimed memory record remains unquantified, and capacity alone will not establish performance or value.
Amazon is in advanced talks to buy Decart, whose software helps AI models run efficiently across chips. The reported deal could strengthen AWS’s Trainium business, with conditional payments and five-year founder commitments making execution central to its value.
The University of California is expanding ChatGPT and Codex access across 10 campuses, with extra charges for advanced models and an unresolved dispute over faculty authority.
Ray-Ban’s US store cuts the Gen 2 Wayfarer by 25% to $284.25. Buyers still face separate prescription-lens prices, a cheaper Gen 1 option and newer glasses outside Meta’s frame sale.
Ubisoft has made the team behind its Teammates experiment a dedicated subsidiary targeting AI characters with memory and outside game creators. Earlier playtests show both progress and limitations, while commercial plans remain undisclosed amid group cost cuts and refinancing.
Neuralink has shown a trial participant controlling GTA VI through intended movements. Earlier gaming use and company-reported trial results put the demonstration in context, while its reliability across users remains unmeasured in the reports.
SoftBank founder Masayoshi Son reportedly explored financing a $1 trillion acquisition of memory-chip maker SK hynix, but the company says the deal is no longer under active consideration. The target brings surging earnings, existing shareholders and customer commitments that complicate any simple equation between ownership and access to AI-memory supply.
Nvidia reported $115.8 billion in quarterly revenue, but its shares fell 14% Friday amid a wider technology selloff, a leadership transition and a reported accounting investigation. Strong cash generation offers a counterweight to concerns about the durability of AI spending.
The Nasdaq 100 fell 5.6% on November 20 and closed 19.8% below its record after an intraday bear-market breach. Nvidia’s revenue surge and Anthropic’s huge IPO did not halt the rout; infrastructure bills and resilience elsewhere in the market complicate the picture.
Reporting traces how Megaspeed gained access to Nvidia chips through an unlisted Inspur subsidiary and alleges links to a Cambodian smuggling intermediary. Company denials, differing customs totals and an unresolved criminal investigation complicate the account.
Together AI says an intruder downloaded stored samples of June and July API prompts and searched them for credentials. Zero data retention is now the default for new accounts, but the number of affected customers remains undisclosed.
Users describe grief and physical distress when AI relationships end, whether they choose to leave or the service changes. Research on attachment and farewell tactics raises questions about how providers measure engagement, manage updates and protect intimate data.
UN-backed talks agreed on two more years of work toward an autonomous-weapons instrument, without committing to a treaty. Consensus kept major military powers involved while allowing opponents to weaken the mandate.
Tesla is sending FSD v15 to its first customers, adding a visualization of planned driving and changes to navigation and hazard detection. The software remains supervised, and its claimed 25-fold intervention improvement lacks a disclosed baseline.
OpenAI reports sharply lower reasoning-extraction rates in internal GPT-6.2 tests and backs sanctions against illicit foreign distillation. Its own research and rival Anthropic’s experience show why protecting hidden reasoning leaves capability transfer and customer costs unresolved.
Alberta proposes 600-metre setbacks for large data centres and expanded utility oversight, but exemptions and rules still to be written will determine how much protection communities receive.
President Donald Trump backed Commerce Secretary Howard Lutnick as lawmakers scrutinize alleged delays in restricting a suspected route for AI chips to China. A new congressional inquiry also questions Nvidia’s influence on an existing investigation.