Pith. sign in

REVIEW 4 major objections 6 minor 10 references

The AI Fiction Paradox

T0 review · 4 major / 6 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Fiction resists AI generation for three structural reasons—narrative causation, informational revaluation, and multi-scale emotional architecture—and solving them would hand AI a powerful tool for shaping human emotion and belief.

desk verdict A readable synthesis of why fiction is hard for LLMs, but its architectural explanation is confounded with post-training choices—and the paper says so itself. read the letter →

arxiv 2603.13545 v2 pith:WTXIURXW submitted 2026-03-13 cs.AI cs.CLcs.CY

Katherine Elkins This is my paper · ORCID
classification cs.AIcs.CLcs.CY
keywords AIfictionparadoxnarrativecausationinformationalrevaluationmulti-scaleemotionalarchitecturesentimentarcslargelanguagemodelsgenerationtrainingdata
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper confronts a paradox: AI companies have staked billions and court battles on training models with modern fiction, yet the same models produce flat, homogeneous stories. The author argues this is not a data problem but an architectural one, and isolates three distinct reasons fiction resists generation. Narrative causation requires events to feel surprising in the moment and inevitable in hindsight—a temporal asymmetry that clashes with models that read and write strictly forward. Informational revaluation requires details to become significant only after later revelations, yet attention weights are fixed during the forward pass. Multi-scale emotional architecture requires sentiment to be orchestrated at word, sentence, scene, and whole-arc levels at once. If the author is right, scaling up data or context windows will not fix fiction generation, and the same capabilities, once solved, would give AI an unprecedented lever over human emotion and belief.

What carries the argument

The argument turns on three named mechanisms it claims are jointly necessary for compelling fiction and jointly absent from current systems. Narrative causation is a form of plot logic in which events must be unpredictable in the moment and retrospectively unavoidable; it conflicts with autoregressive generation, which commits to prose sequentially with no backward evaluation. Informational revaluation is the principle that the significance of an early detail can be established only by later events, so statistical salience and importance come apart; it conflicts with fixed attention weights that cannot reweight past context after a revelation. Multi-scale emotional architecture is the simult

What would settle it

Run the same base model under two regimes—default chat with safety tuning versus a fine-tuned or deliberately de-aligned specialist with the same architecture—and ask both to write a long story requiring a planted detail that only becomes crucial at the end, a twist that feels both surprising and inevitable, and a rising emotional arc. If the second regime produces the retrospective structure and emotional arc while the first does not, the claim that the architecture itself blocks fiction generation is falsified; if neither does, the architectural thesis is supported. A complementary probe: af

Watch

Extended reading notes

Core claim

The paper's central claim is that the AI-Fiction Paradox—models trained on vast corpora of modern novels still cannot generate compelling long-form fiction—is explained by three structural features of fiction that current transformer architectures cannot honor. First, narrative causation: plots must feel surprising as they unfold yet inevitable in retrospect, while autoregressive generation commits to each token before the arc is known. Second, informational revaluation: fiction routinely asks readers to reweight earlier details in light of later ones, but attention mechanisms set their weights during a forward pass and cannot revise the importance of past tokens. Third, multi-scale emotiona

Load-bearing premise

The whole account rests on the claim that current failures are built into the architecture—forward-only generation, fixed attention—rather than being produced by the way models are post-trained (safety tuning, RLHF, instruction tuning, data mix); the paper itself admits in its own section 2 that much of today's dullness likely reflects training choices, not model capability.

Editorial extensions

If this is right

  • Scaling context windows or adding more fiction data will not, by itself, produce compelling long-form fiction; the bottlenecks are structural, so new architectures (e.g., generate-and-evaluate loops, revisable memory) are needed.
  • The observed flatness and homogeneity of AI stories is partly a side effect of safety training and instruction tuning, meaning current model behavior underestimates what architectures could do under different training regimes.
  • Fiction's value as training data lies in properties that hold across literary quality levels—complete causal chains, belief revision, social tracking, and subtext—explaining why companies want broad corpora rather than curated canons.
  • Comprehension benchmarks give independent evidence for the same structural limits: models do far worse on global narrative reasoning than on sentence-level tasks, consistent with an inability to perform retrospective revaluation.
  • When AI can generate emotionally coordinated fiction, the same machinery becomes a scalable instrument of persuasion and belief change, making the 'falling in love' question also a question of autonomy and manipulation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct test follows from the paper's logic: hold the architecture fixed and compare default safety-tuned generation against a fine-tuned or deliberately de-aligned version on long-form story tasks; if the de-aligned version sustains surprise-plus-inevitability and emotional arcs, the architectural explanation would be weakened.
  • If retrospective revaluation is the bottleneck, then models should be able to judge a story's arc better than they can produce it, and a system that alternates generation with explicit retrospective critique should outperform end-to-end generation—extending the paper's human-in-the-loop observation into a standalone architecture.
  • If multi-scale emotional architecture is a genuine constraint, automated metrics that measure emotional-arc flatness (arousal curves, valence variance) could serve as detectors of AI-generated storytelling, giving current 'eerie placidity' a forensic signature.
  • The paradox may be a phase transition rather than a permanent wall: the same companies that need fiction data today may be the ones who make that data obsolete by solving the three challenges, shifting the policy question from copyright to manipulation.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes an 'AI-Fiction Paradox': although modern fiction is a heavily used and legally risky component of LLM training corpora, current models still generate poor long-form fiction. The author argues that this failure is not merely a data or alignment problem but reflects three architectural limitations: (1) narrative causation, the requirement that events feel surprising when read forward and inevitable when read backward, which is said to conflict with autoregressive generation; (2) informational revaluation, the need to retrospectively reweight earlier details, which is said to be impossible for fixed attention mechanisms; and (3) multi-scale emotional architecture, the coordination of sentiment at word, sentence, scene, and arc levels, which current models allegedly cannot orchestrate. The paper surveys legal and dataset evidence for AI's fiction dependency, cites comprehension benchmarks (NoCha, NovelQA, TLDM) and generation studies (Rettberg et al., Tian et al.), and concludes with a warning that mastering fiction would give AI a powerful tool for emotional persuasion and manipulation.

Significance. If the three-challenge thesis were established, it would be a valuable contribution to both literary studies and AI research: it gives a named taxonomy of narrative properties that resist current generation methods, and it proposes testable hypotheses about where architectural changes are needed. The paper is clearly written, draws attention to a real and under-theorized phenomenon, and cites several external benchmarks that are relevant to long-form narrative comprehension. However, the central claim that these are architectural limitations rather than post-training or data-mix effects is not currently supported. The paper does not provide controlled experiments, formal derivations, or even a systematic comparison of base models versus instruction-tuned/safety-tuned models. Its own Section 2 and Conclusion concede plausible non-architectural explanations. As a result, the significance of the paper is conditional: it is a promising framework for future work, not a validated explanation.

major comments (4)
  1. [§2 and §§3–6] The paper's central claim is that the three challenges are architectural limitations, but the evidence discussed is either from comprehension benchmarks (NoCha, NovelQA) or from outputs of current chat interfaces (Rettberg et al., Tian et al.). Section 2 explicitly attributes current dullness to RLHF, safety training, Constitutional AI, instruction tuning, and data mix, stating that the observed behavior 'likely does not reflect what the models might be able to do but more what the models have been trained to do.' Section 7 similarly concedes that 'it's possible current constraints are due to architectural limitations.' To support the architectural thesis, the paper would need to show that the same forward-only, fixed-attention architecture fails on long fiction when trained without these post-training choices, or at least provide a direct comparison of base and post-trained models under
  2. [§3] The narrative-causation argument is asserted rather than derived. The claim that autoregressive forward generation 'cannot' coordinate surprise with retrospective inevitability is not backed by a formal argument or a controlled generation experiment. Next-token training on completed novels can, in principle, encode long-range plot structure in hidden states; long-context failures may reflect context-length limits, decoding choices, or training distribution rather than a categorical architectural impossibility. The NoCha result (41.6% on global reasoning) is a comprehension benchmark and does not directly test generation of fiction. The statement that current systems 'lack a mechanism to work backward' is also too strong, since iterative revision, planning, and search-based decoding are standard techniques. Please either provide a formal argument for the claimed impossibility or reframe t
  3. [§4] The informational-revaluation argument conflates attention weights with the model's representational capacity. In a transformer, hidden states are recomputed as context grows, and bidirectional or encoder models can attend to later text during comprehension; in generation, causal attention prevents future access, but that is not equivalent to an inability to 'retrospectively revise' importance in the sense claimed. The NovelQA benchmark tests question answering over very long documents, and its failures may be due to context length, retrieval, or training distribution rather than a fixed attention mechanism. The Flaubert barometer example is a useful illustration of fictional detail, but it does not by itself establish an architectural constraint. The paper should clarify what exactly is fixed and what empirical evidence would distinguish the architectural claim from a long-context or da
  4. [§6] The multi-scale emotional architecture is presented as a requirement, but the supporting evidence is indirect and partly self-referential. Chun (2021) reports that transformers fail at arc detection in long text; that is a detection task, not a generation task, and detection does not imply generation failure. Tian et al.'s finding that injecting explicit discourse-level features improves generation by over 40% is more consistent with a decoding or training deficiency than with an architectural impossibility: if explicit features can be supplied, the model can use them. The assertion that emotional arc is 'more foundational' than plot or character relies on the author's own prior sentiment-arc work and is not independently supported in this paper. The section should either justify this priority with additional external evidence or explicitly mark the multi-scale architecture requirement a
minor comments (6)
  1. [§4 and Works Cited] There appear to be two different 'NovelQA' entries in the Works Cited: Bubeck et al., 'NovelQA: A Benchmark for Long-Form Narrative Comprehension,' and Wang et al., 'NovelQA: Benchmarking Question Answering on Documents Exceeding 200K Tokens.' The text cites only 'Wang et al. 2024' but uses claims associated with both. Please clarify which benchmark is intended and ensure citations match.
  2. [§1 and §6] The author's name is given as 'Rettberg and Wigers' in Section 2 and as 'Rettberg et al.' in Section 6; the Works Cited lists 'Hermann Wigers.' Please standardize the spelling and citation form.
  3. [§2] The phrase 'since ChatGPT' should be 'since the release of ChatGPT' or 'since the ChatGPT interface,' and 'Models since ChatGPT' is imprecise. Also, the comparison to 'a jazz musician versus a tourist using a phrasebook' is vivid but not operational; consider replacing it with a concrete description of the experimental conditions being compared.
  4. [Abstract and §3] The abstract and Section 3 call the account 'theoretically precise,' but the three challenges are not formally defined with measurable criteria. For example, 'narrative causation' is described in prose that would be hard to operationalize. Consider adding explicit definitions and, ideally, falsifiable predictions.
  5. [§7] The closing sentence, 'I see no good reason to believe that an AI model might one day in the not too distant future become capable of generating stories that move us,' seems to contradict the earlier statement in Section 1 that 'It's hard to imagine AI limitations are permanent.' Clarify the intended position: is the limitation temporary or uncertain?
  6. [§5] The list of fiction's cognitive properties in Section 5 is interesting but speculative, and several items overlap (e.g., error correction and informational revaluation). Condensing this section and linking each property to the three challenges would strengthen the paper's focus.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the central arguments are analytic consequences of definitions plus architectural facts, with external benchmarks; §6 self-citations to prior sentiment-arc work are background, not load-bearing.

full rationale

The three challenges are not derived from the paper's own outputs. Narrative causation is introduced as a definition ('events must feel both surprising in the moment and retrospectively inevitable') and the obstacle is argued from the forward-generation property of transformers; this is an analytic implication, not a fitted prediction. The informational-revaluation section relies on an architectural fact ('Attention weights are set during the forward pass and cannot be retrospectively revised based on later revelations') and checks it against external benchmarks (NoCha, NovelQA), not against values fit in the paper. The emotional-architecture section is the only place with heavy self-citation ('My prior work over many years now has documented this emotional arc...', 'Chun 2021'), but the paper also invokes external evidence (Tian et al., Rettberg et al., reviews of Marche) and frames the inference as a belief ('I believe generating fiction actually requires...'), so the self-citations are motivational rather than a load-bearing uniqueness chain. The internal tension noted in §2 — that current dullness 'likely does not reflect what the models might be able to do but more what the models have been trained to do' — is a confound between architecture and post-training choices, which weakens the empirical case but is not circularity. No equation, fitted parameter, or benchmark result is recycled as its own confirmation.

Assumptions & free parameters 0 free parameters · 6 assumptions · 3 invented entities

The paper rests on several unstated assumptions about training-data determinism, transformer internals, and the necessity of the three named challenges. It introduces three conceptual constructs as explanations; none is formalized. The most significant support for the emotional-architecture claim is self-cited sentiment-arc work.

assumptions (6)
  • domain assumption Training data determines output quality; more relevant data should improve generation.
    Establishes the 'paradox' in §1: without this expectation there is no puzzle.
  • domain assumption Current transformers generate forward-only and have fixed attention weights.
    Used in §§3-4 to explain why narrative causation and revaluation are hard; this is a simplified description of architecture, not a derived result.
  • ad hoc to paper Fiction requires narrative causation, informational revaluation, and multi-scale emotional architecture.
    The central thesis, asserted in the abstract and §§3-6 rather than derived or empirically established.
  • ad hoc to paper Emotional arc is more foundational than plot or character.
    Section 6 bases this on the author's prior sentiment-arc research, not on independent evidence in this paper.
  • domain assumption Fiction concentrates cognitive/emotional patterns valuable for training general models.
    Section 5 uses this to explain companies' legal risk-taking; supported only by an internal Meta admission and a forthcoming study by Roland and So.
  • domain assumption Statistical salience is a reliable proxy for importance except in fiction.
    Section 4 presents this as the computational assumption that fiction violates; the violation is asserted, not demonstrated quantitatively.
invented entities (3)
  • narrative causation
    purpose: Explains plot logic requiring local surprise and retrospective inevitability.
    Introduced as a named explanatory construct; no operational definition or falsifiable prediction is given.
  • informational revaluation
    purpose: Explains how earlier narrative details gain new significance after later events.
    Qualitative concept; the claim that attention weights cannot be retrospectively revised is a structural claim, not an empirical measurement.
  • multi-scale emotional architecture
    purpose: Explains emotional orchestration at word, sentence, scene, and arc levels.
    Derived from the author's sentiment-arc framework; the paper provides no independent measure of the four-scale architecture.

how reviews work

0 comments
Cite this review

Pith. "Pith review of The AI Fiction Paradox." pith.science (2026). https://pith.science/paper/WTXIURXW

@misc{pith2026260313545,
  author       = {Pith},
  title        = {Pith review of: The AI Fiction Paradox},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WTXIURXW}},
  note         = {Machine review of arXiv:2603.13545}
}
read the original abstract

AI development has a fiction dependency problem. Developers have treated large corpora of modern books, including fiction, as valuable enough to accept substantial cost and legal risk, yet current models still struggle to generate compelling long-form fiction. I term this the "AI-Fiction Paradox," and it is particularly startling because training data strongly shapes model output. This paper offers a theoretically precise account of why fiction resists AI generation by identifying three distinct challenges for current systems. First, fiction depends on what I call narrative causation, a form of plot logic where events must feel both surprising in the moment and retrospectively inevitable. Standard autoregressive generation commits to prose sequentially, creating a practical obstacle to coordinating local surprise with retrospective inevitability across a long narrative. Second, I identify an informational revaluation challenge: fiction repeatedly requires the significance of earlier details to be reinterpreted in light of later developments, a form of long-range reasoning that current systems perform unreliably. Third, drawing on over seven years of collaborative research on sentiment arcs, I argue that fiction that moves us requires multi-scale emotional architecture, the orchestration of sentiment at word, sentence, scene, and arc levels simultaneously. Together, these three challenges help explain both why developers have sought large modern book corpora and why compelling long-form fiction remains so difficult to replicate. The analysis also raises urgent questions about what happens when these challenges are overcome. Fiction concentrates unusually powerful cognitive and emotional patterns for modeling human behavior, and mastery of these patterns by AI systems would represent not just a creative achievement but a potent vehicle for human manipulation at scale.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

10 extracted references · 1 canonical work pages

  1. [1]

    OpenAI Deleted Training Datasets Believed to Contain Copyrighted Books

    AIAAIC Repository. “OpenAI Deleted Training Datasets Believed to Contain Copyrighted Books.” May 2024, https://www.aiaaic.org/aiaaic-repository/ai-algorithmic-and-automation-incidents/ openai-deleted-training-datasets-believed-to-contain-copyrighted-books . Austen, Jane.Pride and Prejudice. Oxford World’s Classics, Oxford UP,

    2024

  2. [7]

    Proust, Marcel.In Search of Lost Time

    DOI: 10.1038/s41598-024-76900-. Proust, Marcel.In Search of Lost Time. Translated by C.K. Scott Moncrieff, Terence Kilmartin, and D.J. Enright. New York: Modern Library,

    work page doi:10.1038/s41598-024-76900-

  3. [10]

    A Mathematical Theory of Communication

    Shannon, Claude E. “A Mathematical Theory of Communication.”Bell System Technical Journal, vol. 27, no. 3, 1948, pp. 379–423; no. 4, pp. 623–656. Silverman v. Meta. United States District Court, Northern District of California, 2023–2025. Tian, Yufei, et al. “Are Large Language Models Capable of Generating Human-Level Narratives?”Proceed- ings of the 2024...

    1948

  4. [1992]

    Echoes in AI: Quantifying Lack of Plot Diversity in LLM Outputs

    Qu, Jian, et al. “Echoes in AI: Quantifying Lack of Plot Diversity in LLM Outputs.” arXiv preprint arXiv:2501.00273,

    arXiv

  5. [2008]

    AI’s Creative Boundaries: A Cross-Model Pattern Analysis of Identity-Based Narratives

    Brigham, Maisie Jane. “AI’s Creative Boundaries: A Cross-Model Pattern Analysis of Identity-Based Narratives.” IPHS 300 AI for the Humanities, Spring 2025, Kenyon College. Bubeck, Sébastien, et al. “NovelQA: A Benchmark for Long-Form Narrative Comprehension.” arXiv,

    2025

  6. [2019]

    What the Rise of AI Means for Narrative Studies: A Response to ‘Why Computers Will Never Read (or Write) Literature’ by Angus Fletcher

    Elkins, Katherine, and Jon Chun. “What the Rise of AI Means for Narrative Studies: A Response to ‘Why Computers Will Never Read (or Write) Literature’ by Angus Fletcher.”Narrative, vol. 30, no. 1, January 2022, pp. 104–113. Karpinska, Magda, et al. “NoCha: Challenging AI with Literary Fiction Reasoning Tasks.” arXiv,

    2022

  7. [2020]

    Can Sentiment Analysis Reveal Structure in a ‘Plotless’ Novel?

    Elkins, Katherine, and Jon Chun. “Can Sentiment Analysis Reveal Structure in a ‘Plotless’ Novel?” arXiv preprint arXiv:1910.01441,

    arXiv 1910

  8. [2023]

    Generative AI & Fictionality: How Novels Power Large Language Models

    Roland, Edwin, and Richard Jean So. “Generative AI & Fictionality: How Novels Power Large Language Models.” Forthcoming. Rettberg, Jill Walker & Hermann Wigers. “AI-Generated Stories Favour Stability over Change: Homogeneity and Cultural Stereotyping in Narratives Generated by GPT-4o-mini.” arXiv preprint, 2507.22445, 30 July

    arXiv

Show all 10 references
  1. [2024]

    SentimentArcs: A Novel Method for Self-Supervised Sentiment Analysis of Time Series Shows SOTA Transformers Can Struggle Finding Narrative Arcs

    —. “SentimentArcs: A Novel Method for Self-Supervised Sentiment Analysis of Time Series Shows SOTA Transformers Can Struggle Finding Narrative Arcs.” arXiv, 2021, arxiv.org/abs/2110.09454. Dennen, Jim (Director); Chun, Jon & Elkins, Kate (Producers); DivaBot (AI performer); Ka...

    2021 arXiv

  2. [2025]

    Proust’s Consciousness

    —. “Proust’s Consciousness.”Proust’s In Search of Lost Time: Philosophical Perspectives, edited by Katherine Elkins, Oxford UP, 2022, pp. 179–204. —.The Shapes of Stories: Sentiment Analysis for Narrative. Cambridge University Press,

    2022

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.