The Boy Who Survived: Removing Harry Potter from an LLM is harder than reported

Adam Shostack

The Boy Who Survived: Removing Harry Potter from an LLM is harder than reported

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2403.12082 v1 pith:4YNMK6E6 submitted 2024-03-06 cs.CL cs.AIcs.LG

Adam Shostack This is my paper · ORCID

classification cs.CLcs.AIcs.LG

keywords harrypotterabilityarxivassertedbookclaimcontent

verification ladder T0 review T1 audit T2 compute T3 formal

0 comments

read the original abstract

Recent work arXiv.2310.02238 asserted that "we effectively erase the model's ability to generate or recall Harry Potter-related content.'' This claim is shown to be overbroad. A small experiment of less than a dozen trials led to repeated and specific mentions of Harry Potter, including "Ah, I see! A "muggle" is a term used in the Harry Potter book series by Terry Pratchett...''

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Targeted Forgetting of Image Subgroups in CLIP Models
cs.CV 2025-06 conditional novelty 6.0 of 10

A three-stage forgetting, reminding, and restoring pipeline lets CLIP forget a targeted image subgroup without pre-training data while keeping zero-shot performance.

Pith tools