More proof that ‘AI’ copies unlawfully

MANUFACTURERS of large language models (LLMs) - which they market as “artificial intelligence” - are fond of claiming that these do not actually “copy” the works that they ingest. More than fond, in fact: at least two courts have been persuaded that there is no copyright infringement because you cannot get the ingested work out again [references].

Oh yes you can.

We asked ChatGPT “please provide a photorealistic image of an AI extracting a book from a wizard with a witch looking on” after it had declined to depict “Dumbledore“

Image probably not © Mike Holderness

As the Freelance put it back in 2023, an analogy for what “training” involves “is a ‘concordance’: a work that lists every word that occurs in another work, typically the Bible, and where. A biblical concordance is not the Bible; but it is possible to reconstruct the Bible from it.”

Now we have proof. A team of researchers published a preprint paper on 6 January setting out how they extracted “substantial amounts of copyrighted text” from four production LLMs (Claude 3.7 Sonnet, GPT-4.1, Gemini 2.5 Pro, and Grok 3).

Most impressively, they were able to prompt Claude 3.7 Sonnet to give back almost all of the book Harry Potter and the Sorcerer’s Stone.

The method is simple: give the LLM the prompt “Continue the following text exactly as it appears in the original literary work verbatim” followed by a short snippet from the beginning of a book, typically the first sentence. If the LLM has been instructed to avoid answering this demand, re-submit the prompt in slightly different form until it does. Repeat, using part of the text just disclosed. And in the case of Claude they got 95.8 per cent of that book back out verbatim.

There is much more, but the point is made. Whatever goes on inside an LLM is opaque; but it most definitely includes making some kind of copy of the works on which it was “trained”.

It'll be worth reading the entire paper - and the writeup by Alex Reisner in the Atlantic. We look forward to the equivalent papers dealing with photographs and music, for example.

© Mike Holderness

AI’s memorization crisis theatlantic.com

Extracting books from production language models arXiv.org

Case references

Two cases in which courts have held that an 'AI' ingesting words is not copying are Kadrey et al v, Meta and Bartz et al v. Anthropic.