{"id":"5c8505ee-b547-4ac3-aef9-8b17f129366a","arxiv_id":"2604.19447","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A dual-track LLM-assisted stylometry pipeline detects 349 biblical allusions in McCarthy's novels and recovers 54% of 115 previously documented ones.","lead":"The paper introduces a dual-track pipeline combining word embeddings and LLM review to detect biblical allusions in Cormac McCarthy's novels. It reports recovering 54% of known allusions and identifying 349 total, offering a scalable approach for studying literary intertextuality.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"No precision or false-positive validation reported for the 349 novel detections; recall alone on known cases does not establish reliability","rationale":"The reader's weakest assumption directly identifies the same unverified LLM filtering step. With full text now available, the absence of any precision metric or expert validation sample for the novel detections remains the single most load-bearing gap; recall on known allusions is necessary but insufficient to underwrite the headline discovery numbers.","tokens_in":1770,"tokens_out":350,"duration_ms":20981,"concrete_test":"Randomly sample 50 of the 349 reported allusions (stratified by connection type), have two independent literary scholars annotate each for whether it meets the paper's 'textual echo' criteria, and compute precision plus Cohen's kappa; if precision falls below 65% or kappa < 0.6, the pipeline's discovery claims require substantial human post-filtering.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim rests on the cascaded LLM + long-context cross-validation correctly separating genuine textual echoes from coincidental overlaps. The paper reports 54% recall against 115 human-documented allusions but supplies no precision, no inter-annotator agreement with literary experts, and no ablation on the LLM filtering stage for the remaining ~287 surfaced items. Because the method begins with IDF-based rare-word candidates and an undirected register track, both of which generate many surface matches, the soundness of the 349-allusion corpus depends on an unquantified assumption that the LLM review step has low false-positive rate. The abstract's statement that 'every finding is checked against published scholarship' covers only the recovered known cases; novel detections lack an equivalent external anchor.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The manuscript presents a dual-track LLM-assisted pipeline for detecting biblical allusions in Cormac McCarthy's novels: a bottom-up track using IDF to identify rare vocabulary shared with the King James Bible, followed by local-context embedding, sense disambiguation, and cascaded LLM review; and a top-down track in which an LLM scans McCarthy's prose for biblical register without reference to specific passages. Both tracks are cross-validated by a long-context model holding entire novels and the KJV together, with all findings checked against published scholarship. Restricting to textual-echo allusions (shared phrasing, reworked vocabulary, or cadence) and distinguishing literary allusions from signposted references, the pipeline reports 349 allusions across the corpus and independently recovers 62 of 115 previously documented allusions (54% recall), with recall varying by type (30% for transformed imagery to 80% for register collisions).","tokens_in":1941,"tokens_out":683,"duration_ms":39341,"significance":"If the LLM filtering stage can be shown to maintain low false-positive rates on novel detections, the approach would offer a scalable, semi-automated method for large-scale intertextuality studies that combines mechanical stylometry with scholarly grounding. The explicit use of external published scholarship to anchor the 115 known cases provides independent validation and is a methodological strength. The type-specific recall breakdown and the distinction between allusion categories add useful nuance. However, the absence of precision or false-positive quantification for the 349 total (especially the ~287 novel items) limits the immediate utility for statistical claims about intertextuality.","major_comments":[{"comment":"Abstract: The central claim that the pipeline 'surfaces 349 allusions' and recovers 54% of known cases rests on the cascaded LLM review and long-context cross-validation reliably separating genuine textual echoes from coincidental overlaps. No precision, false-positive rate, inter-annotator agreement with literary experts, or ablation of the LLM filtering stage is reported for the novel detections, even though the initial IDF and undirected register tracks are acknowledged to generate many surface matches. This is load-bearing for the reliability of the 349-allusion corpus.","section":"Abstract"},{"comment":"Abstract (validation paragraph): The statement that 'every finding is checked against published scholarship' applies only to the 62 recovered known allusions; the ~287 novel detections lack an equivalent external anchor. Because the method begins with high-recall but low-precision candidate generation, the soundness of the full set depends on an unquantified assumption about the LLM review step.","section":"Abstract"}],"minor_comments":[{"comment":"The description of the cascaded LLM review and long-context cross-validation would benefit from explicit prompts, decision rules for model disagreements, and any temperature or sampling parameters used.","section":"Methods"},{"comment":"Table or figure reporting per-novel or per-type breakdown of the 349 allusions (beyond the aggregate recall) would improve clarity and allow readers to assess variation.","section":"Results"}],"recommendation":"major_revision","confidential_remarks":"The manuscript sits at the intersection of computational linguistics and literary studies; while the cs.CL classification is appropriate for the pipeline, the journal should confirm that its scope includes applied computational humanities work of this kind."},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the thoughtful and detailed report. The comments correctly identify that the reliability of the 349 novel detections hinges on the unquantified performance of the LLM filtering stages, and that the abstract phrasing regarding scholarship checks requires clarification. We address each point below and will revise the manuscript to incorporate these observations.","responses":[{"response":"We agree that the absence of explicit precision or false-positive quantification for the novel detections is a substantive limitation. The pipeline's design uses dual-track convergence and long-context cross-validation as internal filters, and the 54% recall on the held-out known allusions provides one indirect signal, but this does not substitute for direct measurement on the novel set. In revision we will add a limitations subsection that (a) reports the rate at which the two independent tracks agree on novel candidates, (b) discusses the expected false-positive behavior inferred from the known-allusion performance, and (c) outlines how future work could obtain expert inter-annotator agreement on a sampled subset. We will also include a brief ablation note on the contribution of the cascaded LLM review versus the embedding and register stages alone.","revision_made":"yes","referee_comment":"[Abstract] Abstract: The central claim that the pipeline 'surfaces 349 allusions' and recovers 54% of known cases rests on the cascaded LLM review and long-context cross-validation reliably separating genuine textual echoes from coincidental overlaps. No precision, false-positive rate, inter-annotator agreement with literary experts, or ablation of the LLM filtering stage is reported for the novel detections, even though the initial IDF and undirected register tracks are acknowledged to generate many surface matches. This is load-bearing for the reliability of the 349-allusion corpus."},{"response":"The referee is correct; the abstract sentence is imprecise. The scholarship check was performed only for the 115 previously documented allusions (recovering 62). Novel detections rest on the internal pipeline filters. We will revise the abstract to read: 'All recovered known allusions are checked against published scholarship; novel detections are validated through dual-track convergence and long-context cross-validation.' This change will be mirrored in the methods and results sections for consistency.","revision_made":"yes","referee_comment":"[Abstract] Abstract (validation paragraph): The statement that 'every finding is checked against published scholarship' applies only to the 62 recovered known allusions; the ~287 novel detections lack an equivalent external anchor. Because the method begins with high-recall but low-precision candidate generation, the soundness of the full set depends on an unquantified assumption about the LLM review step."}],"tokens_in":1576,"tokens_out":558,"duration_ms":27694,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The paper's main contribution is a practical two-track method for spotting textual biblical echoes in fiction. One track starts with rare words from the King James Bible, embeds them in context, and runs them through LLM review. The other has the model read McCarthy's prose without a fixed target and flag register matches. Both get long-context cross-checks and are compared to existing scholarship. Applied to the McCarthy novels, it surfaces 349 allusions and pulls back 62 of the 115 previously documented ones, with recall ranging from 30% on transformed imagery to 80% on register collisions. That split by type is useful data and the combination of bottom-up rarity plus top-down undirected scanning has not been tried exactly this way before for this problem.","headline":"The paper gives a concrete dual-track LLM pipeline that recovers 54% of known biblical allusions in McCarthy while claiming 349 total, but it reports no precision or false-positive numbers for the new detections.","tokens_in":2397,"tokens_out":237,"would_cite":false,"duration_ms":23687,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"A dual-track pipeline using language models detects 349 biblical allusions in a corpus of modern novels and recovers over half of those previously documented by scholars.","keywords":["biblical allusion","stylometry","large language models","intertextuality","literary fiction","textual echo","register detection"],"falsifier":"Expert scholars examining the 349 surfaced passages and concluding that a majority lack sufficient textual echo to qualify as allusions, or applying the same pipeline to a fresh corpus and recovering substantially less than 54 percent of its documented cases.","tokens_in":2672,"feed_emoji":"📖","tokens_out":739,"duration_ms":33113,"temperature":0.7,"pith_summary":"The paper builds a system to locate subtle references to the King James Bible within contemporary fiction by combining two approaches. One track flags unusual words shared with the Bible and checks their local context with models, while the other has a model read the prose broadly to catch echoes in tone or structure. When tested on the target novels, the combined pipeline identifies hundreds of such connections and matches more than half of the allusions already noted in literary scholarship. A sympathetic reader would care because the work shows how automated assistance could make the study of intertextuality feasible across far larger collections of texts than manual review allows.","feed_headline":"LLM pipeline detects 349 biblical allusions in target novels","feed_subtitle":"It recovers 54 percent of scholar-documented cases by pairing rare-word checks with broad prose scans.","key_machinery":"Dual-track pipeline with a rarity-based bottom-up embedding track for candidate selection and an undirected top-down register track for additional detection, followed by cascaded LLM review and long-context cross-validation.","core_discovery":"The authors present a dual-track pipeline for detecting biblical allusions that carry a textual echo of shared phrasing or cadence. A bottom-up track uses inverse document frequency to surface rare vocabulary shared with the King James Bible, embeds the occurrences for sense disambiguation, and routes candidates through cascaded language model review. A top-down track directs a model to scan the fiction undirected by any specific biblical passage to catch register-based similarities. Long-context cross-validation holds entire novels alongside the Bible in one pass, and all findings are checked against published scholarship. Restricting to proper literary allusions rather than signposted or c","pith_inferences":["The same dual-track structure could be repurposed to detect allusions to other fixed source texts such as Shakespeare or classical literature.","Repeated application across an author's full body of work might reveal whether allusion density changes with career stage or genre.","Integration into digital archives could let researchers query and visualize networks of biblical influence without exhaustive close reading.","Extending the method to non-English corpora would test whether the rarity-plus-register logic generalizes beyond the King James Bible."],"forward_implications":["The pipeline can scale to statistical analysis of intertextuality across much larger literary collections than manual methods permit.","It distinguishes literary allusions involving reworked phrasing or cadence from overt signposted references such as direct quotes or named figures.","Recovery rates vary by connection type, reaching 80 percent for register collisions but only 30 percent for transformed imagery.","Language models function here as assistants that augment rather than replace mechanical stylometric techniques."],"fun_headline_variants":["LLM dual tracks detect 349 biblical allusions in McCarthy novels","Stylometry with LLMs recovers 54 percent of documented McCarthy allusions","LLM assisted stylometry finds 349 Bible allusions in McCarthy fiction","Bottom up IDF and top down register LLM tracks spot McCarthy allusions"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"That cascaded language model review can reliably separate genuine literary allusions from coincidental word overlaps or stylistic similarities without high rates of false positives or missed connections.","fun_headline_variants_meta":{"raw":{"variants":["LLM dual tracks detect 349 biblical allusions in McCarthy novels","Stylometry with LLMs recovers 54 percent of documented McCarthy allusions","LLM assisted stylometry finds 349 Bible allusions in McCarthy fiction","Bottom up IDF and top down register LLM tracks spot McCarthy allusions"]},"model":"grok-4.3","cost_usd":0.009995,"raw_usage":{"total_tokens":4482,"prompt_tokens":753,"num_sources_used":0,"completion_tokens":76,"cost_in_usd_ticks":99949500,"prompt_tokens_details":{"text_tokens":753,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":3653,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":753,"tokens_out":76,"duration_ms":28511,"temperature":1.0,"reasoning_tokens":3653,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-05-10T02:11:58.314939+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Expert scholars examining the 349 surfaced passages and concluding that a majority lack sufficient textual echo to qualify as allusions, or applying the same pipeline to a fresh corpus and recovering substantially less than 54 percent of its documented cases.","supporting_citations":[],"review_version":1}