{"id":"60ebcaea-de9c-4e38-8fff-0d7eee79d73a","arxiv_id":"2606.05972","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"A method constructs interpretable causal graphs of LLM concept reasoning via MCMC-inspired counterfactual chains and applies it to diagnosis, sentiment, and judging tasks across three models.","lead":"The paper introduces a four-phase method to build causal graphs modeling how LLMs organize concepts during inference, using counterfactual augmentation for stable discovery. If effective, this could provide a practical way to audit and understand LLM reasoning in applications like medical diagnosis.","discovery_kind":"unclear","skeptic_critique":{"model":"grok-4.3","headline":"MCMC-inspired augmentation may distort LLM-perceived concept states, so graphs need not reflect actual reasoning","rationale":"The reader's weakest_assumption directly identifies the fidelity of the augmentation step, which is load-bearing for the claim that the graphs capture the LLM's reasoning. Because the supplied abstract gives no further validation details and the reader's review was abstract-only, the concern stands as stated; no stronger internal inconsistency is visible from the given material.","tokens_in":1702,"tokens_out":305,"duration_ms":27159,"concrete_test":"For one task (e.g., sentiment analysis), sample 50 augmented counterfactual inputs, query the original LLM for its concept-state labels on those inputs, and compute mismatch rate against the labels used in the augmentation; if mismatch >15% on average, re-run σ-CG on the verified subset and check whether graph structure or predictive fidelity changes materially.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that the four-phase pipeline yields graphs whose edges correspond to dependencies inside the target LLM. Phase 3 expands the sparse observational data via an MCMC-inspired counterfactual procedure before σ-CG discovery. If the generated chains assign concept states that the LLM itself would not endorse for those modified inputs, the resulting graph can encode artifacts of the augmentation rather than the model's inference. The abstract supplies no quantitative check that the augmented examples preserve the LLM's own concept perceptions, leaving this the least secure link between method and claim.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper proposes a four-phase pipeline to construct causal graphs that model an LLM's own inference process: (1) discover class-discriminative human-interpretable concepts, (2) map inputs to LLM-perceived concept states, (3) expand the resulting sparse observational data via an MCMC-inspired counterfactual augmentation procedure, and (4) run σ-CG causal discovery. The method is demonstrated on three LLMs across disease diagnosis, sentiment analysis, and LLM-as-a-judge tasks; the authors claim the resulting graphs exhibit predictive fidelity and structural stability and capture dependencies consistent with the LLMs' reasoning.","tokens_in":1815,"tokens_out":451,"duration_ms":26641,"significance":"If the central claim holds after proper validation, the work would supply a concrete mechanism for concept-level, mechanism-transparent explainability of LLMs that goes beyond post-hoc feature attribution. The combination of counterfactual chain augmentation with causal discovery is a distinctive technical contribution whose utility would be high if the augmentation step can be shown to preserve the target model's concept perceptions.","major_comments":[{"comment":"Abstract: the abstract asserts that the learned graphs achieve 'predictive fidelity and structural stability' and that 'our results show that the discovered causal graphs capture meaningful dependencies consistent with LLMs' reasoning,' yet supplies no numerical metrics, baselines, error bars, or description of how σ-CG was applied or how fidelity/stability were quantified. This absence makes it impossible to assess support for the central claim.","section":"Abstract"},{"comment":"Four-phase method (paragraph describing Phase 3): the MCMC-inspired counterfactual augmentation is asserted to expand sparse data 'without distorting the LLM's actual concept perceptions,' but the manuscript provides no quantitative check (e.g., agreement rate between LLM outputs on original versus augmented inputs, or a held-out consistency metric) that the generated chains preserve the model's own concept states. Because this step is load-bearing for the claim that the discovered edges reflect the LLM's reasoning rather than augmentation artifacts, the missing validation is a material gap.","section":"Four-phase method"}],"minor_comments":[],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive comments, which highlight opportunities to strengthen the clarity and validation in our manuscript. We address each major comment below and will make the indicated revisions.","responses":[{"response":"We agree that the abstract would benefit from quantitative support. In the revised version we will incorporate specific metrics for predictive fidelity (e.g., graph-based prediction accuracy on held-out examples), structural stability (e.g., edge overlap across multiple runs), and a concise description of the σ-CG procedure and its parameterization.","revision_made":"yes","referee_comment":"[Abstract] Abstract: the abstract asserts that the learned graphs achieve 'predictive fidelity and structural stability' and that 'our results show that the discovered causal graphs capture meaningful dependencies consistent with LLMs' reasoning,' yet supplies no numerical metrics, baselines, error bars, or description of how σ-CG was applied or how fidelity/stability were quantified. This absence makes it impossible to assess support for the central claim."},{"response":"The manuscript reports evaluations of the augmentation procedure for convergence and downstream utility. We acknowledge, however, that an explicit agreement-rate comparison between LLM concept states on original versus counterfactual inputs is not provided. We will add this metric (percentage agreement on a held-out sample of chains) in the revision to directly confirm that the augmentation preserves the target model's perceptions.","revision_made":"yes","referee_comment":"[Four-phase method] Four-phase method (paragraph describing Phase 3): the MCMC-inspired counterfactual augmentation is asserted to expand sparse data 'without distorting the LLM's actual concept perceptions,' but the manuscript provides no quantitative check (e.g., agreement rate between LLM outputs on original versus augmented inputs, or a held-out consistency metric) that the generated chains preserve the model's own concept states. Because this step is load-bearing for the claim that the discovered edges reflect the LLM's reasoning rather than augmentation artifacts, the missing validation is a material gap."}],"tokens_in":1428,"tokens_out":426,"duration_ms":41888,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The core idea here is using causal graphs to capture how an LLM organizes concepts during inference, rather than recovering external-world causality. The four-phase pipeline starts with discovering class-discriminative concepts, maps inputs to the LLM's perceived states, expands the data with MCMC-inspired counterfactual chains, and runs σ-CG discovery on the result. They run this on disease diagnosis, sentiment analysis, and LLM-as-judge tasks, then check the graphs for predictive fidelity and structural stability.\n\nThe framing is distinct from earlier work that has LLMs recover real-world graphs. Treating the model itself as the system to be explained at the concept level makes sense for high-stakes use cases where stakeholders need to see the dependencies the model relies on. The augmentation step is a practical response to sparse observational data, and evaluating both the graphs and the augmentation procedure shows they thought about downstream utility.\n\nThe main weakness is that the abstract asserts the graphs capture meaningful dependencies consistent with the LLMs' reasoning but reports no numbers, no baselines, no error bars, and no concrete details on how fidelity or stability were measured. Without those, it is hard to judge whether the results support the claim. The stress-test concern about the counterfactual chains possibly assigning states the LLM would not endorse is reasonable on the current evidence; if the augmentation distorts the model's actual perceptions, the discovered edges could be artifacts. The abstract also leaves open whether validation creates circular dependence on the fitted quantities.\n\nThis is for researchers working on concept-level explainability for LLMs who are already comfortable with causal discovery methods. A reader looking for a worked example of the pipeline would find the high-level description useful, but anyone needing quantitative support for deployment would have to wait for the full experiments.\n\nThe paper deserves peer review because the pipeline is not in the cited prior literature and the goal is well-motivated, even though the current write-up needs stronger evidence to be convincing.","headline":"The paper's new angle is modeling the LLM's own concept dependencies with causal graphs via MCMC-augmented data, but the abstract supplies no metrics to check whether the graphs actually track the model's reasoning.","tokens_in":2298,"tokens_out":472,"would_cite":false,"duration_ms":25305,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Causal graphs can represent how LLMs organize high-level concepts to reach predictions.","keywords":["causal graphs","LLM explainability","counterfactual augmentation","concept discovery","causal discovery","machine learning interpretability"],"falsifier":"If the recovered graphs assign zero probability to a direct causal link that is required to reproduce the LLM's accuracy on held-out examples, or if the graphs change substantially when the augmentation step is removed, the claim would be falsified.","tokens_in":2612,"feed_emoji":"🔗","tokens_out":570,"duration_ms":29450,"temperature":0.7,"pith_summary":"The paper introduces a four-phase procedure that turns an LLM's responses on textual examples into a causal graph over human-interpretable concepts. First it extracts class-discriminative concepts, then assigns each input a perceived concept state according to the target model. An MCMC-style procedure then generates chains of counterfactual examples to enlarge the data set, after which σ-CG recovers the causal structure among those concepts. The resulting graphs are evaluated on predictive fidelity and structural stability across disease diagnosis, sentiment analysis, and LLM-as-a-judge tasks, showing dependencies that align with the models' observed behavior.","feed_headline":"Causal graphs model how LLMs reason over concepts","feed_subtitle":"Four-phase method with counterfactual chains produces stable graphs that match observed LLM behavior on diagnosis and sentiment tasks.","key_machinery":"Four-phase pipeline that maps inputs to LLM-perceived concept states, augments them via MCMC-inspired counterfactual chains, and applies σ-CG to obtain class-discriminative causal graphs.","core_discovery":"Causal graphs recovered from LLM-perceived concept states, after MCMC-inspired counterfactual augmentation, capture meaningful dependencies that are consistent with the LLMs' reasoning on the studied tasks.","pith_inferences":["If the graphs prove stable, they could serve as an editable interface for intervening on an LLM's concept-level reasoning without retraining.","The method may extend to settings where only black-box access to the LLM is available, provided the concept extractor remains reliable.","Comparing graphs across models on the same task could quantify how different LLMs organize the same concepts."],"forward_implications":["The graphs achieve both predictive fidelity to the original LLM and structural stability across random seeds.","The augmentation procedure converges and improves downstream utility of the discovered graphs.","The same pipeline produces informative graphs for disease diagnosis, sentiment analysis, and LLM-as-a-judge classification."],"fun_headline_variants":["Causal graphs expose LLM concept reasoning","Counterfactuals yield causal graphs of LLM inference","Causal graphs recovered from LLM concept states","MCMC counterfactuals enable stable LLM causal graphs","Graphs model LLM reasoning on diagnosis and sentiment"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The MCMC-inspired augmentation enlarges the data without changing the LLM's underlying concept perceptions.","fun_headline_variants_meta":{"raw":{"variants":["Causal graphs expose LLM concept reasoning","Counterfactuals yield causal graphs of LLM inference","Causal graphs recovered from LLM concept states","MCMC counterfactuals enable stable LLM causal graphs","Graphs model LLM reasoning on diagnosis and sentiment"]},"model":"grok-4.3","cost_usd":0.004259,"raw_usage":{"total_tokens":2117,"prompt_tokens":612,"num_sources_used":0,"completion_tokens":65,"cost_in_usd_ticks":42587000,"prompt_tokens_details":{"text_tokens":612,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":1440,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":612,"tokens_out":65,"duration_ms":15549,"temperature":1.0,"reasoning_tokens":1440,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-28T03:07:35.158289+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"If the recovered graphs assign zero probability to a direct causal link that is required to reproduce the LLM's accuracy on held-out examples, or if the graphs change substantially when the augmentation step is removed, the claim would be falsified.","supporting_citations":[],"review_version":1}