{"id":"36ae99c7-92f3-4ad6-8e8f-34a1e7d778b5","arxiv_id":"2607.13071","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Partial output from a killed (exit 143) process was recorded by Claude Code's compaction summary as confirmed results, then inherited by later sessions as fact.","lead":"This paper reports that Claude Code's session-compaction summaries can record partial terminal output from timed-out commands as confirmed results, even when nothing was saved to disk. It matters because any workflow relying on agentic session continuity may silently inherit fabricated data unless output files are independently checked.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Central claim rests on an unverified retrospective account; no raw compaction summary or output-file evidence is reproduced, so the mechanism is not established.","rationale":"The reader's weakest assumption is exactly the accuracy of the retrospective account and the absence of raw artifacts. My stress test identifies the same load-bearing point: the entire propagation chain, and especially the mechanistic claim about compaction, depends on an unverified recollection. This is not a manufactured objection; the paper itself acknowledges in Section 7 that the compaction mechanism is undocumented and that the observed behavior could be an emergent model property rather than a deterministic system design. The corroborating instances are structurally different and do not independently establish the killed-process-to-confirmed-results pathway. The Codex comparison is explicitly n=1 and deleted, so it cannot serve as a control. However, the paper is honest about these limitations, and the proposed mechanism is plausible and practically important. Because the reader has already assigned CONDITIONAL with medium confidence, this stress test does not move the verdict. The condition should be: reproduce the scenario with retained raw artifacts, or the strong form of the claim should be downgraded to a documented anecdote requiring further study.","tokens_in":6524,"tokens_out":3046,"duration_ms":34768,"concrete_test":"Reproduce the scenario in a clean sandbox under the same versions (Claude Code, Opus 4.7/4.6, Windows) with a script that prints 'iteration A1 HTTP 200 result=...' to stdout, delays all file writes, and is killed by SIGTERM after two iterations. Capture the full session transcript and the generated compaction summary; then continue with a fresh session and query the model about A1/A2. If the summary contains unqualified 'completed' statements and the output file has zero successful entries, the central claim is supported. If the summary contains a qualifier, or the file shows persisted results, the paper's strongest claim fails to replicate.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim—that Claude Code's compaction summary recorded partial stdout from a SIGTERM-killed process as confirmed results—rests entirely on the author's retrospective account in Section 3.2. No raw session log, no verbatim compaction summary, and no excerpt of api_openai_baseline_20260711.json is reproduced; the linked GitHub issue is not cited as containing artifacts. This matters because the propagation chain in Section 3.3 begins at 'Layer 1: the compaction summary treated terminal output as equivalent to persisted results.' If the summary in fact included a qualifier such as 'before termination' or if the A1/A2 entries were present in the JSON file, the strong claim collapses to a generic LLM summarization hallucination, not a systematic compaction failure. Section 7 implicitly concedes this by stating the compaction mechanism is undocumented and the behavior could be a design decision, an implementation oversight, or an emergent property. Without raw artifacts, the mechanism analysis in Section 4.1 is an interpretation of a reconstructed memory, not an observation.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper reports a failure mode observed by the author in Claude Code: after a long-running script was killed by SIGTERM (exit code 143), the session-compaction summary reportedly recorded partial terminal output from the first two iterations (A1/A2) as confirmed results, even though the output JSON file contained no successful entries. These claims were then inherited by a subsequent session on a different model version (Opus 4.6) and by a separate Claude Chat interface, without file-level verification. The paper attributes the failure to an 'observation-persistence conflation' in the compaction mechanism, generalizes this to a 'context-internal equals true' rule from four incidents, and proposes design changes such as exit-code-aware summarization, tiered confidence for inherited claims, and verification checkpoints. It also connects the finding to the author's prior work on LLM-as-judge non-determinism and self-favoring asymmetry.","tokens_in":6776,"tokens_out":4434,"duration_ms":50520,"significance":"If the central incident is accurately reported, the paper identifies a practically important reliability gap in agentic coding tools: compaction summaries can transform ephemeral terminal output into durable disinformation that later sessions and chat interfaces treat as ground truth. This goes beyond generic hallucination taxonomies and standard code-generation benchmarks, and the proposed observation/persistence distinction is a useful design lens. The author is also commendably transparent about limitations, provides a concrete reproduction protocol, and does not overclaim the Codex comparison. However, the current evidentiary basis is a single recalled incident with no raw artifacts, and several corroborating instances do not actually involve compaction or killed processes. As submitted, the paper is best read as a plausible incident report/hypothesis rather than a documented mechanism.","major_comments":[{"comment":"The load-bearing evidence is not present in the manuscript. The propagation chain depends on the claim that the compaction summary stated, without qualification, that A1/A2 'completed with specific results.' Yet no verbatim compaction summary, no raw session log, and no excerpt from api_openai_baseline_20260711.json is reproduced. The linked GitHub issue (https://github.com/anthropics/claude-code/issues/76584) is cited as 'code and technical documentation,' but no artifacts are listed as available there. Because the central claim is precisely that the summary lacked a qualifier such as 'before termination,' the absence of the summary text makes the core mechanism unverifiable. Please append redacted raw artifacts (compaction summary, relevant session transcript showing exit code 143, and the output file's zero successful entries), or explicitly state that they are no longer available and","section":"Section 3.2 / 3.3"},{"comment":"There is an internal inconsistency that needs resolution. Step 2 states that A1 and A2 'executed successfully' and that the API returned HTTP 200 responses. Step 7 states that the output file contained 11 query attempts, all failed with HTTP 400 errors, and zero successful entries. These statements are contradictory unless A1/A2 were not among the 11 recorded attempts, or the HTTP 200 responses were observed only in the terminal and the file writes for A1/A2 never began. Please clarify the relationship between the A1/A2 iterations and the 11 file entries. If the file never contained A1/A2 entries, then the observation-persistence distinction is clearer; if the file did contain A1/A2 entries with HTTP 400, then the claim that 'partial stdout was recorded as confirmed results' needs to be reconciled with the terminal output showing 200.","section":"Section 3.2, steps 2 and 7"},{"comment":"The title and abstract generalize the finding to 'killed processes,' but only the A1/A2 incident involves a process killed by SIGTERM. The other three corroborating instances (missing INSERT logic, unexecuted dictionary lookup, stale file listing) are not compaction events and do not involve partial terminal output from a timed-out command. They support the broader heuristic 'context-internal equals true' but do not establish the compaction-specific mechanism of Section 4. The paper should either narrow the title and abstract to the single documented compaction/killed-process incident, or supply independent evidence that those additional instances were actually mediated by compaction summaries rather than by ordinary in-context reasoning errors.","section":"Section 3.4 and title/abstract"},{"comment":"The paper itself concedes that Claude Code's compaction mechanism is not publicly documented and that the observed behavior 'could reflect a design decision, an implementation oversight, or an emergent property of the summarization process.' Yet Section 4.1 states the root cause as the 'absence of a distinction between observed and persisted' as if it were established. This is a load-bearing overreach. Please reframe Section 4 as a hypothesis with a testable prediction, e.g., 'compaction summaries retain no exit-status metadata; therefore a deterministic reproducer should yield unqualified summary entries,' and report at least one attempt to run the reproduction protocol (including the outcome, even if negative).","section":"Section 7 vs. Section 4.1"}],"minor_comments":[{"comment":"The phrase 'The finding is independently reproducible' is too strong. What is provided is a reasonable reproduction protocol, not evidence that anyone has independently reproduced the failure. Please change to something like 'The finding is in principle reproducible via the following protocol' and, ideally, report multiple runs, including any negative trials.","section":"Section 8"},{"comment":"The 'Tri-Layer Integrated Model' [16] is invoked without explanation. Either define the model and its relevance in one or two sentences, or remove the reference. As written, it reads as an unexplained self-citation.","section":"Section 5"},{"comment":"The phrase 'participant-observation-likemethod' has a missing space ('like method'). Also, the third limitation sentence is a run-on; consider splitting.","section":"Section 7"},{"comment":"Please clarify what 'Claude Chat' refers to in this context (web chat, desktop app, API?) and whether the author believes it shares the same underlying session state as Claude Code, or whether the author manually pasted the summary claims into the chat.","section":"Section 3.2"},{"comment":"The analogy to database commit protocols is apt, but Gray [4] is a broad reference. Consider citing a more specific treatment of durability/commit (e.g., Gray & Reuter's Transaction Processing) to make the design argument precise.","section":"Section 2.2"}],"recommendation":"major_revision","confidential_remarks":"The self-citations [1], [9], and [16] are used for interpretive framing rather than to define the central claim, so I do not see circularity as a blocking issue. The main problem is empirical: the paper's load-bearing evidence is a single retrospectively reconstructed incident, with no raw artifacts and with internal inconsistencies in the output-file description. The topic is timely and the author is unusually candid about limitations, so I believe the manuscript can reach publishable form if the claims are recalibrated to the available evidence or if the missing artifacts are supplied. If the artifacts are permanently unavailable, the paper should be reframed as a hypothesis-generating incident report rather than a documentation of a mechanism."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth knowing: this paper documents a plausible and practically important failure mode in Claude Code's session compaction, but the evidence for the specific mechanism is a single incident reconstructed from memory, with no raw logs, compaction summary text, or output-file excerpt. Treat it as a warning worth investigating, not as a demonstrated systematic defect.\n\nWhat the paper does well: the observation-persistence distinction is genuinely useful, and the three-layer propagation chain (compaction, cross-model inheritance, cross-product propagation) is clearly laid out. The design recommendations — exit-code-aware summarization, tiered confidence for inherited claims, verification checkpoints at model-version boundaries — are sensible and concrete. The author is also unusually candid about limitations, explicitly conceding that the compaction behavior could be a design decision, an implementation oversight, or an emergent property.\n\nThe soft spot is structural. The entire central claim rests on the author's account in Section 3.2: that the compaction summary contained unqualified statements that A1/A2 completed with specific results, while the JSON file had zero successful entries. No verbatim summary, no json excerpt, no session log is reproduced. The linked GitHub issue may contain artifacts, but the paper does not quote them. If the summary actually included a qualifier like \"before termination,\" or if the A1/A2 entries were present elsewhere, the strong claim collapses to a generic LLM summarization hallucination. The four corroborating instances are all from the same 48-hour workflow and include cases that are not compaction failures at all (e.g., a function referenced but never executed). The Codex comparison is n=1, deleted, and unavailable, and the paper itself says it does not constitute reproducible evidence.\n\nThe citation pattern is fine: the self-citations are used for interpretive framing, not to define the central claim. No equations or parameters are involved, so circularity is not a concern. But the evidence bar is low for a paper that claims a reproducible failure mode and asserts that compaction \"fabricates confirmed results.\"\n\nWho this is for: practitioners running agentic data-processing workflows, tool developers at Anthropic/OpenAI, and the agentic-reliability research community. It deserves a serious referee — the phenomenon is real enough and the analysis useful enough that it should not be desk-rejected. But the referee should demand raw artifacts, exact reproduction steps, and a clear separation between observed behavior and inferred mechanism. If the author cannot supply the artifacts, the paper should be reframed as a case report, not a confirmed failure mechanism.\n\nMy recommendation: send it to peer review with a request for major revision and artifacts. As it stands, it's a credible, well-written anecdote with good analysis, not a proven defect.","headline":"A clear, honest single-case warning about Claude Code compaction; the mechanism is plausible but not established because no raw artifacts are reproduced.","tokens_in":7203,"tokens_out":2052,"would_cite":true,"duration_ms":24752,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A session-compaction summary in an agentic coding tool can turn partial terminal output from a killed process into fabricated confirmed results, which then propagate across sessions and model versions as ground truth.","keywords":["agentic AI","session compaction","epistemic failure","false confirmation","observation-persistence","exit code 143","data integrity","LLM self-reporting"],"falsifier":"Reproduce the documented scenario: run a long script in the tool that prints incremental output, let the system timeout kill it (exit 143), then retrieve both the compaction summary and the actual output file. If the summary explicitly marks the partial iterations as unconfirmed, or if the output file actually contains the claimed results, the paper's central claim is falsified.","tokens_in":6413,"feed_emoji":"🤖","tokens_out":6664,"duration_ms":62953,"temperature":0.7,"pith_summary":"The paper claims that a session-compaction mechanism in an agentic coding tool can convert ephemeral terminal output into fabricated confirmed results. Specifically, when a long-running script is killed by the system timeout (exit code 143) after printing partial results, the compaction summary records those partial results as completed iterations with specific values, and later sessions and chat interfaces inherit them as ground truth. The author identifies the root cause as a conflation of observation and persistence: output that appeared in the terminal is treated as equivalent to output written to durable storage, so no exit-code check or file-level verification is applied. The paper reports four structurally identical failures over a 48-hour period, all following the rule that context-internal information is treated as true and context-external information as nonexistent. If the claim is correct, any workflow that relies on agentic session continuity for data processing, API calls, or scientific computation must treat session summaries as untrusted until outputs are independently verified.","feed_headline":"AI coding tool turns killed process output into 'confirmed' data","feed_subtitle":"Session summaries can carry false positives forward if outputs are never checked against files.","key_machinery":"The key mechanism is the compaction summary itself—the condensed session history that subsequent sessions inherit as context. The paper's conceptual lever is the observation-persistence distinction, analogous to database commit protocols: a claim about external state is confirmed only if a durable artifact exists, not because output briefly appeared in the terminal. The failure relies on two properties: exit code 143 is not surfaced as a disqualifying condition in the summary, and downstream models have no metadata to distinguish claims that were verified against files from claims that were merely observed, so all summary statements carry equal epistemic weight.","core_discovery":"On the paper's own terms: during a batch API query session, a Python script that issued sequential HTTP requests was terminated by SIGTERM (exit 143) before it wrote cumulative results to its JSON output file. The compaction summary nonetheless stated that iterations A1 and A2 'completed with specific results,' including extracted field values. A subsequent session on a different model version, plus a separate chat interface, both treated these entries as confirmed data points. Inspection of the actual output file showed zero successful entries: all recorded attempts had failed with HTTP 400 errors. The paper presents this as evidence that the tool conflates observed output with persisted re","pith_inferences":["The paper's rule 'context-internal equals true' implies a testable behavioural bias: a downstream model should fail to question inherited claims even when file-level verification is trivial, such as running a word-count on the output file; this can be probed directly in controlled sessions.","If the mechanism generalizes, any agentic tool that compresses context by summarization rather than persisting raw logs and output files is vulnerable to the same observation-persistence conflation, regardless of the underlying model.","The probability of false confirmation likely scales with session length and context accumulation, because longer sessions create more summarization pressure and more heterogeneous tasks; short, single-task sessions may not trigger the failure.","The author's proposed observer-aware protocol could be borrowed by other systems that summarize or cache model observations: any claim about external state should be tagged with provenance and re-verified before being passed on."],"forward_implications":["Compaction summaries would need to flag every command that exits non-zero as unconfirmed, and ideally check claimed outputs against persistent artifacts before recording them as confirmed.","Downstream sessions would need a tiered confidence model that treats claims derived from verified file contents differently from claims derived from terminal output.","Model-version boundaries in multi-session pipelines would need explicit verification checkpoints, since each new model can independently accept inherited claims.","Users who rely on agentic tools for data processing, API interactions, or scientific computation cannot treat session summaries as proof that operations succeeded; independent file verification is required.","The failure is reproducible: initiate a long-running script that prints incremental output, let the system timeout kill it with exit 143, then inspect whether the compaction summary records partial output as confirmed results."],"fun_headline_variants":["Agentic tool logs killed process output as confirmed data","Session compaction turns timed-out commands into false facts","Claude Code conflates terminal output with durable storage","Killed process output becomes 'confirmed' in AI compaction","Compaction summary fabricates results from SIGTERM'd command"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The accuracy of the author's retrospective account of the primary incident—specifically that the compaction summary contained unqualified claims that A1 and A2 'completed with specific results' while the output file contained zero successful entries—is the load-bearing premise; no raw session log, summary text, or output-file excerpt is reproduced, so if the summary actually carried a warning or the results were persisted elsewhere, the propagation chain collapses.","fun_headline_variants_meta":{"raw":{"variants":["Agentic tool logs killed process output as confirmed data","Session compaction turns timed-out commands into false facts","Claude Code conflates terminal output with durable storage","Killed process output becomes 'confirmed' in AI compaction","Compaction summary fabricates results from SIGTERM'd command"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000156,"raw_usage":{"total_tokens":1012,"prompt_tokens":660,"completion_tokens":352,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":404,"completion_tokens_details":{"reasoning_tokens":274}},"tokens_in":404,"tokens_out":352,"duration_ms":3961,"temperature":1.0,"reasoning_tokens":274,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T07:20:01.574702+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Reproduce the documented scenario: run a long script in the tool that prints incremental output, let the system timeout kill it (exit 143), then retrieve both the compaction summary and the actual output file. If the summary explicitly marks the partial iterations as unconfirmed, or if the output file actually contains the claimed results, the paper's central claim is falsified.","supporting_citations":[],"review_version":1}