🔗 Share this article

Technology companies and child protection organizations will receive permission to assess whether artificial intelligence tools can produce child exploitation material under new UK legislation.

Substantial Increase in AI-Generated Illegal Material

The announcement came as findings from a safety watchdog showing that reports of AI-generated CSAM have increased dramatically in the past year, rising from 199 in 2024 to 426 in 2025.

Updated Legal Framework

Under the changes, the government will permit designated AI companies and child protection organizations to examine AI models – the underlying systems for conversational AI and visual AI tools – and verify they have adequate safeguards to prevent them from producing images of child exploitation.

"Fundamentally about stopping exploitation before it happens," declared the minister for AI and online safety, noting: "Specialists, under rigorous conditions, can now detect the danger in AI models promptly."

Addressing Legal Challenges

The amendments have been implemented because it is against the law to produce and possess CSAM, meaning that AI developers and other parties cannot create such images as part of a testing process. Until now, officials had to wait until AI-generated CSAM was published online before addressing it.

This legislation is designed to preventing that problem by helping to stop the production of those images at source.

Legislative Structure

The amendments are being added by the government as revisions to the criminal justice legislation, which is also implementing a prohibition on possessing, producing or sharing AI systems designed to create child sexual abuse material.

Practical Consequences

This week, the minister toured the London headquarters of Childline and listened to a simulated call to counsellors involving a report of AI-based exploitation. The interaction portrayed a teenager seeking help after facing extortion using a sexualised AI-generated image of himself, created using AI.

"When I learn about young people facing extortion online, it is a source of extreme frustration in me and justified concern amongst families," he stated.

Concerning Data

A prominent internet monitoring foundation stated that instances of AI-generated abuse material – such as online pages that may include numerous files – had significantly increased so far this year.

Cases of category A content – the most serious form of abuse – increased from 2,621 images or videos to 3,086.

• Girls were predominantly victimized, making up 94% of illegal AI images in 2025
• Portrayals of newborns to toddlers rose from five in 2024 to 92 in 2025

Sector Response

The legislative amendment could "constitute a vital step to ensure AI products are secure before they are released," commented the chief executive of the internet monitoring organization.

"AI tools have enabled so survivors can be victimised repeatedly with just a few clicks, providing criminals the ability to create potentially endless quantities of advanced, lifelike exploitative content," she continued. "Content which additionally exploits survivors' trauma, and renders children, particularly girls, more vulnerable on and off line."

Support Interaction Information

Childline also released details of counselling interactions where AI has been referenced. AI-related risks discussed in the conversations include:

• Employing AI to rate weight, physique and looks
• Chatbots dissuading children from talking to safe guardians about harm
• Facing harassment online with AI-generated material
• Online extortion using AI-faked pictures

Between April and September this year, the helpline delivered 367 support sessions where AI, conversational AI and associated topics were discussed, significantly more as many as in the same period last year.

Half of the references of AI in the 2025 interactions were related to psychological wellbeing and wellbeing, encompassing utilizing AI assistants for assistance and AI therapy apps.