Pith. sign in

REVIEW 3 major objections 6 minor 23 references

On Recipe Memorization and Creativity in Large Language Models: Is Your Model a Creative Cook, a Bad Cook, or Merely a Plagiator?

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Pith's one-line read The paper argues that Mixtral's recipes are dominated by memorized ingredients: nearly every ingredient traces to an online document, and the authors conjecture the model only uses ingredients seen during training.

desk verdict Useful annotation dataset and honest LLM-as-judge comparison, but the memorization claim is built on a post hoc search without a control and is stated too strongly in §9. read the letter →

arxiv 2506.23527 v1 pith:KBD7MA45 submitted 2025-06-30 cs.CL

classification cs.CL
keywords recipegenerationmemorizationcreativityLLM-as-judgeMixtralannotationnonsensedetectionplagiarism
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This work-in-progress asks whether recipes generated by large language models are creative, memorized, or nonsensical, and it focuses on memorization. Using 20 Mixtral-generated recipes and 18 web documents per recipe, human annotators checked each ingredient and cooking step against those documents. The paper reports that ingredients which initially looked novel were almost always found once a targeted web search included the missing ingredient's name, and that combining documents brings ingredient coverage close to 100 percent. The authors therefore conjecture that Mixtral uses an ingredient only if it was seen in the training set. They also build an LLM-as-judge pipeline to automate the annotation, with the best configuration matching human ingredient labels in 77.8 percent of cases.

What carries the argument

The machinery is a decomposition-and-overlap annotation protocol. Each generated recipe is parsed into a list of ingredients and a list of task triples (task name, tool, involved ingredients); each element is then looked up in a set of retrieved online recipe documents, with annotators choosing among labels such as “Found”, “Found (not perfect)”, and “Not found”. The load-bearing operation is the saturation analysis: plotting the fraction of elements found as more documents are combined shows ingredients saturating near 100 percent after roughly five documents, while tools and tasks saturate more slowly. For the automated version, a Cloze formulation converts each annotation choice into a token probability, letting LLMs be ranked by agreement with human labels.

What would settle it

Take the full set of Mixtral-generated recipes from the paper and exhaustively search the web for each ingredient, including documents published after Mixtral's training cutoff; finding one ingredient that appears in no document, or only in post-cutoff documents, would disprove the “only if seen in training” claim. A stronger test would train a small language model on a corpus with all recipe sites removed and check whether it still generates the same “memorized” ingredients.

Watch

Extended reading notes

Core claim

The central claim is that Mixtral, when generating recipes, mostly reproduces ingredients from documents it may have seen during training rather than inventing new ones. In the human-annotation study, only 8 of 247 ingredients were not found in the 18 retrieved documents per recipe, and a targeted search with each missing ingredient's name always turned up a relevant recipe document; on that basis the authors conjecture that Mixtral uses an ingredient only if it appeared in the training set. The conclusion goes further and reports that when documents are combined, the proportion of ingredients and tasks found reaches up to 100 percent. The paper also claims that this annotation process can be automated, with Llama 3.1 extraction plus Gemma 2 9B annotation matching human ingredient labels in 77.8 percent of cases.

Load-bearing premise

The load-bearing premise is that finding an ingredient in a web document retrieved by a targeted search means Mixtral saw that document during training, so the overlap demonstrates memorization rather than common culinary knowledge or chance.

Editorial extensions

If this is right

  • If the conjecture is right, Mixtral's apparent culinary creativity consists mostly of recombining memorized ingredients, not synthesizing new ones.
  • Ingredient memorization can be estimated cheaply: combining about five retrieved documents already covers nearly all ingredients, so saturation curves give a practical proxy for training-data reliance.
  • The automated pipeline can scale the same measurement to hundreds of recipes and multiple models, turning expensive human annotation into an LLM-as-judge task with known accuracy bounds.
  • The status of steps and tools is less settled: task annotation saturates more slowly, so the paper leaves open whether the model's procedures are memorized or genuinely creative.
  • If recipe sites lose traffic to LLM users, ad revenue may decline, and users may be misled by nonsensical recipes—two consequences the paper itself raises.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension is to apply the same overlap logic to other procedural domains—stories, code, or medical instructions—where detecting memorization from web overlap is harder because the “ingredients” are not a closed list.
  • The web-overlap test could be sharpened into a controlled experiment: train a small model on a corpus with recipe sites deliberately removed and see whether it still generates the same ingredients, which would separate training-memorization from generic culinary knowledge.
  • The 77.8 percent ceiling for the best automated annotator suggests that large-scale studies should report human-agreement-adjusted memorization rates rather than raw LLM-as-judge scores.
  • If the memorization result holds across models, recipe platforms and copyright holders could use coverage statistics as a risk signal, though the paper is careful to say its “plagiarism” label is a proxy, not a legal finding.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes an annotation-based framework to quantify memorization, creativity, and nonsense in LLM-generated cooking recipes, focusing on ingredients and recipe steps. The authors generate 20 Mixtral recipes, retrieve 18 web documents per recipe, and use human annotators to label whether each generated ingredient, task, tool, and task-ingredient list is found in those documents. They report inter-annotator agreement (κ = 0.77 for ingredients, 0.63 for tasks) and then evaluate LLM-as-judge pipelines for automatic annotation, with the best ingredient-annotation accuracy reaching 77.8% against human labels. The paper's central inference, stated in Section 6.1 and repeated in Section 9, is that Mixtral uses an ingredient 'only and only if' it was seen in the training set, based on a post hoc web search that found each of 8 initially missing ingredients in some online recipe.

Significance. If the central inference were established, the paper would provide a scalable tool and a concrete measurement strategy for studying recipe memorization. The descriptive contributions are real: a dataset of roughly 36,000 human annotation instances, a detailed annotation guide with acceptable inter-annotator agreement, and a benchmarked LLM-as-judge pipeline with held-out human labels. These are worthwhile and should be credited. However, the headline memorization claim is not supported by the evidence: the post hoc search procedure in Section 6.1 cannot establish training-set membership, and Section 9 overstates results for tasks that Section 6.2 explicitly leaves unanalyzed. The paper is therefore best read as a work-in-progress on annotation methodology rather than as a demonstration that Mixtral's recipes are dominated by memorized content.

major comments (3)
  1. [Section 6.1, Section 9] The central conjecture—that 'Mixtral model uses ingredients only and only if they were seen in the training set'—is not supported by the stated evidence. The validation procedure appends the missing ingredient name to the recipe query and searches the web again; such a query is designed to return pages containing that ingredient, so finding a page is a confirmation of the search query, not a test of the memorization hypothesis. Moreover, a currently indexed web page is not evidence that the page was in Mixtral's training corpus; no membership, proximity, or provenance check links the retrieved documents to Mixtral's training data. There is also no control condition: a model that samples common ingredients for a dish would pass the same test, because recipe ingredient vocabularies are heavily shared. As a result, the conjecture should be rephrased as 'the generated ingredients overlap with currently indexed web pages,' and the stronger wording in Section 9 ('We have shown... that there do exist documents that were potentially included in the training set...') must be removed or substantially weakened.
  2. [Section 6.2 vs. Section 9] Section 9 claims that 'when documents are combined, the amount of ingredients and tasks found in them reaches up to 100%' and that further examination reveals documents for tasks. This contradicts the manuscript's own Section 6.2, which states that 'We have yet to analyze whether the lack of "Task Found" selections is caused by a lack of exhaustivity' for the 29 never-found tasks. The post hoc search was reported only for the 8 missing ingredients, not for the 29 missing tasks, and the task saturation curve in Figure 3 grows more slowly than the ingredient curve, with no reported value reaching 100%. The task-level conclusion is therefore unsupported and should be removed or explicitly limited to ingredients.
  3. [Section 6.3] The exhaustivity analysis in Section 6.3 shows that combining more retrieved documents increases the proportion of ingredients/tasks annotated as 'found,' but this saturation is a property of document-union coverage, not evidence of training-set membership. In particular, the observation that combining just 5 documents yields near-100% ingredient coverage likely reflects the small, common vocabulary of cooking ingredients; it does not show that any single document matches the generated recipe, nor that Mixtral memorized any specific document. The plots should be interpreted as coverage curves for the annotation procedure, and claims about memorization should not be drawn from them without a baseline (e.g., comparison to a non-memorizing ingredient sampler or to held-out recipe corpora with known provenance).
minor comments (6)
  1. [Section 10 (Limitations) and title] The Limitations section correctly states 'We can thus only indicate that a model is potentially plagiarizing, and not definitely prove it,' but the title, abstract, and Section 9 present 'plagiarism' and 'strong reliance on memorized content' as established findings. Please align the framing with the stated limitation.
  2. [Section 7 and Table 7] The model naming is inconsistent: the text refers to 'Mixtral 7B Instruct v0.3 (Jiang et al., 2023)' while Table 7 lists 'Mistral 7B Instruct v0.3'; Jiang et al. (2023) is the Mistral 7B paper, not a Mixtral model. Please correct the name and citation to avoid confusion.
  3. [Section 6.2] The sentence 'the percentage of "Task Not Found" selections was the highest, followed by the number of selections of "Task Not Found" and "Task Found (Not Exact Wording)"' is self-contradictory and appears to misreport Table 3, where 'Task Found' (33.62%) and 'Task Not Found' (35.50%) are nearly equal and 'Task Found (Not Exact Wording)' is 24.78%. Please rewrite this sentence to match the table.
  4. [Figures 1-4] The figures do not show axis labels or legend information in the caption text; adding explicit y-axis labels (e.g., 'Fraction of annotations marked Found') and clearly describing the averaging over recipes would make the saturation analysis easier to interpret. The caption of Figure 4 also contains the fragment 'ings,' which should be expanded.
  5. [Section 4] The paper mentions using SPA (Ethayarajh and Jurafsky, 2022) for prompt selection but does not define what SPA is or how the selection was performed; please add a brief explanation or reference to the specific metric.
  6. [Section 6.1] The phrase 'only and only if' should be 'if and only if' (or 'only if') for clarity.

Circularity Check

1 steps flagged · score 6.0 of 10

The memorization conclusion rests on a self-confirming retrieval loop: missing ingredients are re-searched by appending their own names, so the resulting 'found' documents are guaranteed by the query, not by training-set membership.

  1. self definitional [Section 6.1 (Ingredients Annotation), paragraph after Table 2]
    "To validate these hypotheses, we manually repeated the search query within the search engines, while this time appending the name of the not found ingredient. In case of every missing ingredient, we managed to find a relevant document with recipe containing such an ingredient. Therefore, based on human annotation, we conjecture that hypothesis (ii) is correct, and Mixtral model uses ingredients only and only if they were seen in the training set."

    The 'prediction' that every missing ingredient exists in some online document is manufactured by the search protocol itself: the query is the recipe name plus the missing ingredient, so a returned page containing that ingredient is an artifact of the query, not evidence that Mixtral saw the page in training. The paper's operational definition of memorization is overlap with retrieved online documents; using that same overlap criterion to 'verify' the ingredient after targeted retrieval makes the conclusion true by construction. No membership or provenance check, and no control baseline, connects a current web hit to Mixtral's training corpus, so the inference 'uses ingredients only and only if seen in the training set' is not derived from data but assumed by the retrieval definition.

full rationale

The LLM-as-judge component is benchmarked against held-out human labels and is not circular; the paper's strongest issue is the memorization inference. Section 6.1 re-searches for the eight ingredients that were not found in the original 18 documents by appending each ingredient's name to the query, guaranteeing a document containing it, and then uses that as confirmation that Mixtral only reuses training-set ingredients. Because 'memorized' is operationally defined as 'found in online documents potentially seen in training,' this is a self-confirming loop rather than an external test. The paper itself hedges in Section 10 ('We can thus only indicate that a model is potentially plagiarizing, and not definitely prove it'), which limits the severity, but Section 9 nevertheless overstates the finding to tasks ('when documents are combined, the amount of ingredients and tasks found in them reaches up to 100%') even though Section 6.2 states task exhaustivity 'has yet to be analyzed.' This overreach does not add circularity, but combined with the §6.1 loop it makes the central memorization claim only partially supported. Score 6 reflects that the key evidence for the central claim reduces to the retrieval design, while the annotation pipeline itself has independent content.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the assumption that overlap between a generated recipe and current web documents is a valid proxy for training-data exposure, on the sufficiency of tag-level matching, and on the exhaustivity of a small hand-picked document set. None of these is independently established; the paper's own Limitations section acknowledges that quantities are ignored and that training data are not inspected.

free parameters (3)
  • Number of retrieved documents per recipe (Nd) = 18
    Chosen by hand; the exhaustivity figures show task coverage is still not saturated at 18 documents, so this choice directly affects which ingredients and tasks are classified as memorized versus creative.
  • Number of generations per recipe (K) = 5
    Chosen by hand; one of five recipes is selected for analysis using subjective correctness and preference, which can bias the sample toward whatever the authors judged most plausible.
  • Number of recipes examined = 20
    A small hand-picked sample from 62 candidate names; the paper generalizes from these to Mixtral's overall behavior, which limits the strength of the conclusions.
assumptions (4)
  • domain assumption Retrieved web documents are a valid proxy for Mixtral's training data.
    Section 6.1 and Section 5 treat 'found in an online document' as evidence the model was trained on it; no training-corpus access is provided.
  • domain assumption Verbatim or near-verbatim ingredient-name overlap indicates memorization rather than common culinary knowledge or coincidental generation.
    Section 6.1 and the definition of potential plagiarism in the Introduction footnote; no baseline or control condition is measured.
  • domain assumption Tag-level matching (ingredient names, task names, tools) is sufficient to measure memorization; quantities, sentence structure, and paraphrase are not needed.
    Section 10 Limitations explicitly says quantities are not taken into account and only tag overlap is assessed.
  • ad hoc to paper The manually selected 18 documents per recipe plus a targeted post hoc search are exhaustive enough to declare any unfound ingredient non-creative.
    Section 6.1: after finding documents for the 8 missing ingredients via targeted searches, the paper concludes no creative ingredients; this exhaustivity assumption is specific to this study and is not independently validated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of On Recipe Memorization and Creativity in Large Language Models: Is Your Model a Creative Cook, a Bad Cook, or Merely a Plagiator?." pith.science (2026). https://pith.science/paper/KBD7MA45

@misc{pith2026250623527,
  author       = {Pith},
  title        = {Pith review of: On Recipe Memorization and Creativity in Large Language Models: Is Your Model a Creative Cook, a Bad Cook, or Merely a Plagiator?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KBD7MA45}},
  note         = {Machine review of arXiv:2506.23527}
}
read the original abstract

This work-in-progress investigates the memorization, creativity, and nonsense found in cooking recipes generated from Large Language Models (LLMs). Precisely, we aim (i) to analyze memorization, creativity, and non-sense in LLMs using a small, high-quality set of human judgments and (ii) to evaluate potential approaches to automate such a human annotation in order to scale our study to hundreds of recipes. To achieve (i), we conduct a detailed human annotation on 20 preselected recipes generated by LLM (Mixtral), extracting each recipe's ingredients and step-by-step actions to assess which elements are memorized--i.e., directly traceable to online sources possibly seen during training--and which arise from genuine creative synthesis or outright nonsense. We find that Mixtral consistently reuses ingredients that can be found in online documents, potentially seen during model training, suggesting strong reliance on memorized content. To achieve aim (ii) and scale our analysis beyond small sample sizes and single LLM validation, we design an ``LLM-as-judge'' pipeline that automates recipe generation, nonsense detection, parsing ingredients and recipe steps, and their annotation. For instance, comparing its output against human annotations, the best ingredient extractor and annotator is Llama 3.1+Gemma 2 9B, achieving up to 78% accuracy on ingredient matching. This automated framework enables large-scale quantification of memorization, creativity, and nonsense in generated recipes, providing rigorous evidence of the models' creative capacities.

Figures

Figures reproduced from arXiv: 2506.23527 by the authors.

Figure 3
Figure 3. Percentage of “Task Found” selections in a [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 1
Figure 1. Percentage of Found selections in a combi [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Percentage of “Found” selections in a com [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figures from the paper (1 more)
Figure 5
Figure 5. Figure 5: The unfilled annotation table template. This [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

23 extracted references · 5 canonical work pages

  1. [1]

    Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell

    Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. https://doi.org/10.1145/3442188.3445922 On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT '21, page 610–623, New York, NY, USA. Association for Computing...

  2. [2]

    Szeyi Chan, Jiachen Li, Bingsheng Yao, Amama Mahmood, Chien-Ming Huang, Holly Jimison, Elizabeth D Mynatt, and Dakuo Wang. 2023. https://arxiv.org/abs/2310.05853 "mango mango, how to let the lettuce dry without a spinner?'': Exploring user perceptions of using an llm-based conversational assistant toward cooking partner . Preprint, arXiv:2310.05853

  3. [3]

    Bowen Chen, Namgi Han, and Yusuke Miyao. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.627 A multi-perspective analysis of memorization in large language models . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 11190--11209, Miami, Florida, USA. Association for Computational Linguistics

  4. [4]

    Jacob Cohen. 1960. https://doi.org/10.1177/001316446002000104 A coefficient of agreement for nominal scales . Educational and Psychological Measurement, 20(1):37--46

  5. [5]

    Aissatou Diallo, Antonis Bikakis, Luke Dickens, Anthony Hunter, and Rob Miller. 2024. https://doi.org/10.18653/v1/2024.findings-emnlp.728 P izza C ommon S ense: A dataset for commonsense reasoning about intermediate steps in cooking recipes . In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 12482--12496, Miami, Florida, USA....

  6. [6]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

  7. [7]

    Kawin Ethayarajh and Dan Jurafsky. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.406 The authenticity gap in human evaluation . In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 6056--6070, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics

  8. [8]

    Gemma Team . 2024. https://arxiv.org/abs/2408.00118 Gemma 2: Improving open language models at a practical size . Preprint, arXiv:2408.00118

Show all 23 references
  1. [9]

    Yuling Gu, Oyvind Tafjord, Bailey Kuehl, Dany Haddad, Jesse Dodge, and Hannaneh Hajishirzi. 2024. https://arxiv.org/abs/2406.08446 Olmes: A standard for language model evaluations . Preprint, arXiv:2406.08446

  2. [10]

    Lee, Ke Shu, Palakorn Achananuparp, Philips Kokoh Prasetyo, Yue Liu, Ee-Peng Lim, and Lav R

    Helena H. Lee, Ke Shu, Palakorn Achananuparp, Philips Kokoh Prasetyo, Yue Liu, Ee-Peng Lim, and Lav R. Varshney. 2020. https://doi.org/10.1145/3366424.3383536 Recipegpt: Generative pre-training based cooking recipe generation and evaluation system . In Companion Proceedings of...

  3. [11]

    Valentin Hartmann, Anshuman Suri, Vincent Bindschaedler, David Evans, Shruti Tople, and Robert West. 2023. https://arxiv.org/abs/2310.18362 Sok: Memorization in general-purpose large language models . Preprint, arXiv:2310.18362

  4. [12]

    Alyssa Hwang, Bryan Li, Zhaoyi Hou, and Dan Roth. 2023. https://arxiv.org/abs/2306.13986 Large language models as sous chefs: Revising recipes with gpt-3 . Preprint, arXiv:2306.13986

  5. [13]

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas ...

  6. [14]

    Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guillaume Lample, Lélio Renard Lavaud, Lucile Saulnier, Marie-Anne...

  7. [15]

    Jooyoung Lee, Toshini Agrawal, Adaku Uchendu, Thai Le, Jinghui Chen, and Dongwon Lee. 2025. https://arxiv.org/abs/2406.16288 Plagbench: Exploring the duality of large language models in plagiarism generation and detection . Preprint, arXiv:2406.16288

  8. [16]

    Jooyoung Lee, Thai Le, Jinghui Chen, and Dongwon Lee. 2023. https://doi.org/10.1145/3543507.3583199 Do language models plagiarize? In Proceedings of the ACM Web Conference 2023, WWW '23, page 3637–3647, New York, NY, USA. Association for Computing Machinery

  9. [17]

    Feder Cooper, Daphne Ippolito, Christopher A

    Milad Nasr, Nicholas Carlini, Jonathan Hayase, Matthew Jagielski, A. Feder Cooper, Daphne Ippolito, Christopher A. Choquette-Choo, Eric Wallace, Florian Tramèr, and Katherine Lee. 2023. https://arxiv.org/abs/2311.17035 Scalable extraction of training data from (production) lan...

  10. [18]

    Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners

  11. [19]

    Runco and Garrett J

    Mark A. Runco and Garrett J. Jaeger. 2012. https://doi.org/10.1080/10400419.2012.650092 The standard definition of creativity . Creativity Research Journal, 24(1):92--96

  12. [20]

    Karan Taneja, Richard Segal, and Richard Goodwin. 2024. https://arxiv.org/abs/2401.05199 Monte carlo tree search for recipe generation using gpt-2 . Preprint, arXiv:2401.05199

  13. [21]

    Muhammad Saad Uddin. https://towardsai.net/p/l/stochastic-parrots-a-novel-look-at-large-language-models-and-their-limitations Stochastic parrots: A novel look at large language models and their limitations [online]. 2023

  14. [22]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  15. [23]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.