Pith. sign in

REVIEW 2 major objections 4 minor 66 references

When Many Answers Are Valid, Voting Fails: Symbolic Verification for Best-of-K Causal Reasoning in LLMs

T0 review · 2 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Verifier beats majority voting on multi-answer causal tasks by 11 points

desk verdict Solid, careful paper: CALVER is a real selection signal for causal reasoning, but the headline CLEAR result is self-graded and needs an exact-match re-scoring before the claim is fully transferable. read the letter →

arxiv 2608.03506 v1 pith:NKS2CUGA submitted 2026-08-04 cs.AI stat.ML

Omatharv Bharat Vaidya , Connor Thomas Jerzak , Zayne Rea Sprague , Fangcong Yin , Nhat Ho This is my paper · ORCID
classification cs.AIstat.ML
keywords causalreasoningsymbolicverificationbest-of-Kselectionself-consistencyanswermultiplicityd-separationbackdooradjustmentLLMinference-timescaling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that when a causal-reasoning question admits several equally valid answers, test-time scaling by majority voting fails: the valid answers split the sampled votes while one familiar invalid answer becomes the largest mode. To fix this, it introduces CALVER, a training-free symbolic verifier that scores each sampled reasoning trace against executable causal criteria—graph-theoretic tests such as d-separation, the backdoor criterion, and intervention surgery—and returns the earliest highest-scoring candidate without ever seeing a reference answer. On a frozen pool of find-one-valid causal queries, CALVER reaches 42.1% selected-answer accuracy where exact plurality, a reward model, an LLM judge, and model confidence all sit near 30%, and the gap grows with the number of samples through K=32. The paper also proves a validity-fragmentation theorem explaining why plurality converges to an invalid answer, proves soundness of a strict average-treatment-effect certificate, and shows the same selection principle transfers to logic puzzles via a truth-table checker.

What carries the argument

The load-bearing object is the trace contract: each sampled reasoning trace is typed into six slots—graph, query, strategy, derivation record, computed result, and answer—and scored by six deterministic bits covering parsing/binding, graphical validity via d-separation or m-separation for mixed graphs, the backdoor adjustment test, provenance and format, numerical recomputation, and answer consistency. Selection is the earliest trace achieving the maximum deployed score of six. Two theorems carry the argument: a validity-fragmentation bound showing that exact plurality's probability of returning a valid answer decays exponentially when one invalid answer outweighs every individual valid answ

What would settle it

Re-grade the same frozen pools and selected indices using exact string match to the dataset's reference answer. If CALVER's advantage over plurality and the reward model disappears under that grading rule, the headline result is an artifact of the graph-validity criterion rather than a property of causal selection.

Watch

Extended reading notes

Core claim

The central claim is that executable causal-validity checking is a stronger target-label-free selection signal for best-of-K causal reasoning than agreement-based or score-based aggregation. When a query asks for any single object satisfying a causal predicate—any valid backdoor adjustment set, any d-separating set, any intervention witness—the set of valid answers is a decidable class, so validity can be checked per candidate rather than inferred from consensus. CALVER implements this by parsing each sampled trace into six typed slots and returning the earliest trace whose six deterministic checks all pass on the verification graph. On the same frozen candidate pools the symbolic selector r

Load-bearing premise

The headline comparison defines an answer as correct when it satisfies the causal-graph predicate, not when it matches the benchmark's listed answer; under exact-match grading the reported 42.1% versus ~30% margins would not transfer.

Editorial extensions

If this is right

  • If CALVER's results hold, best-of-K selection in causal reasoning should use executable validity rather than plurality whenever a causal graph is available or can be built from text.
  • The advantage should continue to widen with sampling budget because validation pools validity across answer strings, while plurality saturates once an invalid answer is the largest mode; the paper observes this through K=32.
  • Generic scorers—reward models, model confidence, and LLM judges—cannot recover the signal by scaling; a 72B judge given the same graph remains statistically tied with plurality.
  • The selection principle generalizes to any domain with a decidable validity predicate, demonstrated on knights-and-knaves puzzles with a truth-table checker.
  • When text-to-graph extraction is reliable, extracting one graph and solving exactly beats candidate-wise verification; the paper gives an empirical crossover rule for choosing between the two routes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the same candidate-wise validity check could apply to other test-time scaling regimes, such as code generation where acceptance tests are the predicate, or theorem proving with a proof checker.
  • Editorial inference: the paper's grading criterion—graph-predicate validity rather than the dataset's listed answer—is the faithful reading of 'find-one-valid', but it means headline accuracies are not comparable to exact-match benchmark scores; re-grading the same pools under exact match would be the decisive test.
  • Editorial inference: a distilled process-reward model trained to imitate the symbolic scores fails to recover the selection gain, suggesting the mechanism depends on exact recomputation rather than any learnable proxy.
  • Editorial inference: the monotone gain with K suggests a testable extension—hybrid routing that estimates extraction reliability and switches between exact solving and candidate-wise verification; the paper's data on correlated misreads predicts such a gate should not rely on edge-set agreement between traces.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper proposes CALVER, a training-free symbolic verifier that scores sampled LLM reasoning traces against executable causal criteria (d-separation, backdoor adjustment, intervention, etc.) and selects the highest-scoring trace for best-of-K causal reasoning. The authors argue that when a query has multiple valid answers, plurality voting fragments valid mass and can select an invalid repeated answer. CALVER instead checks each candidate against a fixed graph predicate. The paper contributes theoretical results: a soundness theorem for a strict ATE certificate, a Hoeffding-style validity-fragmentation theorem showing when plurality converges to an invalid mode, an exact finite-K selection law, and a query-local graph transfer result. Empirically, on a 126-item clean core of CLEAR find-one-valid tasks, CALVER reaches 42.1% accuracy versus 30.5% for the best generic scorer, with gains reproduced across eight policies, ten bnlearn networks, text-to-graph construction settings, an independent do-calculus prover, and a Knights-and-Knaves truth-table task. The paper emphasizes that correctness is defined as satisfying the graph predicate, not as matching the benchmark's listed answer.

Significance. If the headline empirical claim holds, the paper makes a useful contribution: it demonstrates that executable validity checking can outperform answer-frequency, reward-model, and LLM-judge selection in a regime where multiple answers are correct. The theoretical results are standard but correctly presented; the fragmentation bound is a clean formal explanation of why plurality fails under answer multiplicity, and the exact selection law clarifies when larger sampling budgets help. The experimental design is unusually thorough, with structure-only controls, tie-rule ablations, graph-corruption tests, node-renaming invariance checks, independent DoVerifier and truth-table validators, and exact-ground-truth ATE evaluation. The central caveat is that the headline CLEAR comparison is graded by the same executable predicate used by the verifier, rather than by exact match to the benchmark's listed answer. The authors disclose this, but the abstract's headline numbers are reported without the caveat, and the transfer of those numbers to the community-standard exact-match metric is unestablished. The external experiments (ATE, K&K, graph-from-text) use distinct verifier/grader objects and

major comments (2)
  1. [Abstract; Experiments, Figure 2/Table 4; Supplementary 'What the checker sees and what grades it'] The headline claim that CALVER 'reaches 42.1% where plurality, a reward model, an LLM judge, and model confidence remain near 30%' is computed under the paper's own correctness definition: a selected answer is correct if it satisfies the published graph predicate, not if it equals CLEAR's listed answer. The Supplementary states that 'both the checker and the grader evaluate the published graph predicate instead of matching CLEAR's listed example,' and the audited subset shows 11 of 21 CALVER selections differ from the listed answer. Under the standard CLEAR exact-match metric, those 11 selections would be scored as errors, and the reported ranking advantage may not transfer. Please report exact-match accuracy on the same frozen pools for all selectors, along with exact-match candidate coverage. Without these numbers, the abstract's headline comparison is not comparable to conventional CL
  2. [Experiments, 'Where does CALVER repair plurality's errors?' and Supplementary Table 10] The per-pool decomposition reports 197 repairs and 54 new errors, yielding a net gain of 24.8 percentage points, and this is presented as the fragmentation mechanism working as predicted. This decomposition is also computed under the predicate-validity correctness criterion. Under exact-match grading, the sets of 'plurality errors' and 'valid answers' change, and the repair/new-error ratio could be substantially different. Since this decomposition is used to locate the mechanism, it should be repeated under the exact-match criterion, or the interpretation should be explicitly restricted to the multi-validity setting. This is load-bearing because the claimed net gain is a central empirical quantity.
minor comments (4)
  1. [Abstract] The abstract reads 'including -separation, backdoor adjustment, and intervention' where 'd-separation' is intended. The missing 'd' is a typographical error.
  2. [Table 2 and Experimental Setup] The per-policy n values differ (360 to 378) due to a parse-admissibility rule. The paper states that the rule is applied identically across selectors, but it would help to clarify in the caption how many of the 126 problems are lost per policy and whether the common-row comparison (Table 4) is the primary analysis. The current presentation invites over-reading of small absolute differences.
  3. [Figure 2] The candidate-coverage curve is computed from held-out labels after generation and is a diagnostic ceiling. The caption says this, but the main text could more clearly state that coverage is not available to any selector and that the gap to coverage is not a comparison with a competing method.
  4. [Supplementary, 'Exact finite-K selection law'] Theorem 9 (Supplementary numbering) is stated for i.i.d. candidates. The main text claims the widening gap through K=32 is 'consistent with the fragmentation mechanism in Theorem 2.' That is plausible, but the empirical curves in Table 8 mix policy effects (SFT, GRPO) with selection mechanics; a sentence acknowledging that policy changes across rows are not controlled would improve precision.

Circularity Check

0 steps flagged · score 2.0 of 10

No structural circularity in the derivation; the headline CLEAR comparison is scored with the same graph predicate the verifier optimizes, which is an evaluation-alignment caveat rather than a circular derivation.

full rationale

The paper's derivation chain is largely self-contained. Theorem 2 (validity fragmentation) is a Hoeffding bound over answer masses that does not invoke CALVER, and Theorem 1/5 (strict ATE soundness) re-proves the standard backdoor-adjustment guarantee from explicit assumptions (correct DAG, positivity, decision margin); the certificate is defined to exclude the two concrete failure modes, but the content is the identification lemma, not an assumption of the conclusion. No parameter is fitted to correctness labels and then renamed as a prediction: CALVER's six checks are fixed, training-free graph computations, and all same-pool comparisons use frozen traces with target labels removed. The DoVerifier, K&K truth-table, graph-from-text, and exact-ground-truth ATE experiments use verifier and grader objects that are distinct, so those transfer claims are externally grounded. The only definitional point is the supplied-graph CLEAR regime, where the supplement states: 'both the checker and the grader evaluate the published graph predicate instead of matching CLEAR's listed example.' Because the queries are find-one-valid, this is the task's own semantic correctness criterion rather than a redefinition that makes the conclusion true by construction. The reported 42.1% therefore measures ranking under the same predicate used for grading, so it does not by itself establish transfer to exact-match scoring; that is a measurement-validity caveat, not a circular derivation. No load-bearing self-citation or imported uniqueness theorem appears.

Assumptions & free parameters 1 free parameters · 5 assumptions · 0 invented entities

The method introduces no new entities or fitted physical constants; its free parameters are limited to declared tolerances and the fixed six-bit scoring rule. The main load-bearing assumptions are the decidability of the validity predicate, correctness of the supplied causal structure, and the predicate-based correctness standard for evaluation.

free parameters (1)
  • epsilon (ATE numerical tolerance) = 0.02 (declared before evaluation)
    Used in the ATE recomputation bit (bi5) and the strict margin guard; chosen by hand, not fitted to data, but the ATE audit's correctness depends on this tolerance.
assumptions (5)
  • domain assumption Graph-validity is algorithmically decidable for the studied queries (d-separation, backdoor, m-separation, intervention reachability).
    Method section states 'validity is algorithmically decidable through standard graph computations'; the verifier depends on this.
  • domain assumption The supplied DAG and observational distribution are correct and positivity holds.
    Supplementary Assumption 1, required for Theorem 1 (strict ATE certificate soundness).
  • standard math LLM answer traces are i.i.d. draws from a fixed mass function.
    Theorem 7 and Theorem 9 assume i.i.d. answers; the practical selection treats traces as exchangeable.
  • ad hoc to paper Query-local signature invariance extends beyond the backdoor predicate to d-separation, mediation, and intervention checks.
    Theorem 10 proves backdoor-case transfer only; empirical gains on other query types assume analogous transfer without proof.
  • ad hoc to paper Correctness of a find-one-valid answer is its satisfaction of the graph predicate rather than equality with the dataset's listed example.
    Experimental protocol redefines accuracy for supplied-graph CLEAR; the headline numbers depend on this convention.

how reviews work

0 comments
Cite this review

Pith. "Pith review of When Many Answers Are Valid, Voting Fails: Symbolic Verification for Best-of-K Causal Reasoning in LLMs." pith.science (2026). https://pith.science/paper/NKS2CUGA

@misc{pith2026260803506,
  author       = {Pith},
  title        = {Pith review of: When Many Answers Are Valid, Voting Fails: Symbolic Verification for Best-of-K Causal Reasoning in LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NKS2CUGA}},
  note         = {Machine review of arXiv:2608.03506}
}
read the original abstract

Self-consistency assumes the most frequent answer among sampled reasoning traces is the most reliable, but this can fail in causal reasoning: samples often repeat the same confounding error, and votes fragment across multiple valid answers, letting an invalid answer win despite a valid minority trace. We introduce CALVER (Causal Axiom-Level VERification), a training-free symbolic verifier that scores structured traces against Pearl's causal criteria, including -separation, backdoor adjustment, and intervention, and selects the highest-scoring candidate without consulting a reference answer. On CLEAR find-one-valid queries that admit multiple graph-valid answers, CALVER reaches 42.1% where plurality, a reward model, an LLM judge, and model confidence remain near 30% on identical frozen pools. Scaling the judge to 72B does not close the gap. In an audited clean-core subset, 11 of 21 graph-valid CALVER selections differ from the benchmark's listed answer while still satisfying the requested predicate. The advantage widens with the sampling budget and reproduces across ten published Bayesian networks, a second model family, and settings where the model must build the graph from text. CALVER also improves thresholded average-treatment-effect decisions against exact ground truth, generalizes to logic under a truth-table checker, and scores each candidate in milliseconds on CPU. CALVER needs only a causal structure, supplied outright or built from the text; wherever that holds, selection can aggregate via causal validity.

Figures

Figures reproduced from arXiv: 2608.03506 by the authors.

Figure 1
Figure 1. CALVER as a best-of-K selector. (a) In the illustrative graph, every nonempty subset of {Z1, Z2, Z3} satisfies the backdoor criterion for the effect of X on Y . Valid probability mass 0.70 is split across seven strings, while the invalid answer ∅ is the single largest mode. (b) Six deterministic checks score K = 8 sampled traces; plurality returns ∅, whereas CALVER returns the earliest maximum-score valid trace. (c)… view at source ↗
Figure 2
Figure 2. Left: non-oracle selectors on the 1,111 CLEAR units for which every scorer is available. Center: prefix scaling on the same frozen eight-candidate pools. Right: deeper scaling on the Qwen-7B SFT clean core. The dashed gray curve labelled candidate coverage is the fraction of pools with at least one correct candidate, computed from held-out labels after generation. It is a diagnostic ceiling and is unavailable to all… view at source ↗
Figure 3
Figure 3. Exact solving after consensus graph extraction ver [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (5 more)
Figure 1
Figure 1. Figure 1: A CLEAR clean-core problem exactly as the policy receives it, above the rule, and one of the eight traces it returned, [PITH_FULL_IMAGE:figures/full_fig_p016_1.png]
Figure 2
Figure 2. Figure 2: One CLEAR pool, showing the traces the checker [PITH_FULL_IMAGE:figures/full_fig_p016_2.png]
Figure 3
Figure 3. Figure 3: One sachs problem at the three levels of indirec￾tion, quoted from the evaluation corpora. The variables, the question and the source subgraph PKC→PKA→Raf with PKC→Raf are the same at every level, so only the telling changes. The valid adjustment set is {PKC} throughou…
Figure 4
Figure 4. Figure 4: The direct-solver advantage disappears as exact [PITH_FULL_IMAGE:figures/full_fig_p022_4.png]
Figure 5
Figure 5. Figure 5: Lift over exact plurality on the CLEAR core as the [PITH_FULL_IMAGE:figures/full_fig_p023_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

66 extracted references · 44 canonical work pages

  1. [1]

    The Eleventh International Conference on Learning Representations , year=

    Self-Consistency Improves Chain of Thought Reasoning in Language Models , author=. The Eleventh International Conference on Learning Representations , year=

  2. [2]

    Snell, Charlie Victor and Lee, Jaehoon and Xu, Kelvin and Kumar, Aviral , booktitle =. Scaling

    work page

  3. [3]

    2407.21787 , archivePrefix=

    Large Language Monkeys: Scaling Inference Compute with Repeated Sampling , author=. 2407.21787 , archivePrefix=

    arXiv

  4. [4]

    Advances in Neural Information Processing Systems , volume=

    Jin, Zhijing and Chen, Yuen and Leeb, Felix and Gresele, Luigi and Kamal, Ojasv and Lyu, Zhiheng and Blin, Kevin and Gonzalez Adauto, Fernando and Kleiman-Weiner, Max and Sachan, Mrinmaya and Sch. Advances in Neural Information Processing Systems , volume=

    work page

  5. [5]

    The Twelfth International Conference on Learning Representations , year =

    Can Large Language Models Infer Causation from Correlation? , author =. The Twelfth International Conference on Learning Representations , year =

    work page

  6. [6]

    Chen, Sirui and Xu, Mengying and Wang, Kun and Zeng, Xingyu and Zhao, Rui and Zhao, Shengjie and Lu, Chaochao , booktitle=

    work page

  7. [7]

    2009 , publisher=

    Causality: Models, Reasoning, and Inference , author=. 2009 , publisher=

    2009

  8. [8]

    Scandinavian Journal of Statistics , volume=

    Markov Properties for Acyclic Directed Mixed Graphs , author=. Scandinavian Journal of Statistics , volume=. 2003 , doi=

    work page 2003

Show all 66 references
  1. [9]

    Journal of Machine Learning Research , volume=

    Complete Identification Methods for the Causal Hierarchy , author=. Journal of Machine Learning Research , volume=. 2008 , url=

    2008

  2. [10]

    Advances in Neural Information Processing Systems , volume=

    Chain-of-Thought Prompting Elicits Reasoning in Large Language Models , author=. Advances in Neural Information Processing Systems , volume=

  3. [11]

    2023 , publisher=

    Ni, Ansong and Iyer, Srini and Radev, Dragomir and Stoyanov, Veselin and Yih, Wen-Tau and Wang, Sida and Lin, Xi Victoria , booktitle=. 2023 , publisher=

    2023

  4. [12]

    Ye, Xi and Chen, Qiaochu and Dillig, Isil and Durrett, Greg , booktitle=

  5. [13]

    Nature , volume=

    Solving Olympiad Geometry without Human Demonstrations , author=. Nature , volume=

  6. [14]

    2110.14168 , archivePrefix=

    Training Verifiers to Solve Math Word Problems , author=. 2110.14168 , archivePrefix=

    arXiv

  7. [15]

    2211.14275 , archivePrefix=

    Solving Math Word Problems with Process- and Outcome-Based Feedback , author=. 2211.14275 , archivePrefix=

    arXiv

  8. [16]

    The Twelfth International Conference on Learning Representations , year =

    Let's Verify Step by Step , author =. The Twelfth International Conference on Learning Representations , year =

  9. [17]

    Math-Shepherd: Verify and Reinforce

    Wang, Peiyi and Li, Lei and Shao, Zhihong and Xu, Runxin and Dai, Damai and Li, Yifei and Chen, Deli and Wu, Yu and Sui, Zhifang , booktitle=. Math-Shepherd: Verify and Reinforce

  10. [18]

    Uncovering Hidden Correctness in

    He, Paul and Huang, Yinya and Sachan, Mrinmaya and Jin, Zhijing , booktitle=. Uncovering Hidden Correctness in

  11. [19]

    and Shen, Yelong and Wallis, Phillip and Allen-Zhu, Zeyuan and Li, Yuanzhi and Wang, Shean and Wang, Lu and Chen, Weizhu , booktitle=

    Hu, Edward J. and Shen, Yelong and Wallis, Phillip and Allen-Zhu, Zeyuan and Li, Yuanzhi and Wang, Shean and Wang, Lu and Chen, Weizhu , booktitle=

  12. [20]

    2022 , howpublished=

    Bayesian Network Repository , author=. 2022 , howpublished=

    2022

  13. [21]

    Is Best-of-

    Huang, Audrey and Block, Adam and Liu, Qinghua and Jiang, Nan and Krishnamurthy, Akshay and Foster, Dylan J , booktitle=. Is Best-of-

  14. [22]

    The Fourteenth International Conference on Learning Representations , year =

    The Limits of Inference Scaling Through Resampling , author=. The Fourteenth International Conference on Learning Representations , year =

  15. [23]

    2017 , publisher=

    Elements of Causal Inference: Foundations and Learning Algorithms , author=. 2017 , publisher=

    2017

  16. [24]

    Sheth, Ivaxi and Fatemi, Bahare and Fritz, Mario , booktitle=

  17. [25]

    The Majority is not always right:

    Zhao, Wenting and Aggarwal, Pranjal and Saha, Swarnadeep and Celikyilmaz, Asli and Weston, Jason and Kulikov, Ilia , eprint=. The Majority is not always right:

  18. [26]

    2026 , url=

    Liu, Chris Yuhao and Zeng, Liang and Xiao, Yuzhen and He, Jujie and Liu, Jiacai and Wang, Chaojie and Yan, Rui and Shen, Wei and Zhang, Fuxiang and Xu, Jiacheng and Liu, Yang , booktitle =. 2026 , url=

    2026

  19. [27]

    Yang, An and Yang, Baosong and Zhang, Beichen and Hui, Binyuan and Zheng, Bo and Yu, Bowen and Li, Chengyuan and Liu, Dayiheng and Huang, Fei and Wei, Haoran and others , eprint=

  20. [28]

    2024 , howpublished=

    Mistral. 2024 , howpublished=

    2024

  21. [29]

    Dettmers, Tim and Pagnoni, Artidoro and Holtzman, Ari and Zettlemoyer, Luke , booktitle=

  22. [30]

    and Schult, Daniel A

    Hagberg, Aric A. and Schult, Daniel A. and Swart, Pieter J. , booktitle=. Exploring network structure, dynamics, and function using

  23. [31]

    International Conference on Learning Representations , year=

    Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation , author=. International Conference on Learning Representations , year=

  24. [32]

    Nature , volume=

    Detecting hallucinations in large language models using semantic entropy , author=. Nature , volume=

  25. [33]

    2311.17311 , archivePrefix=

    Universal self-consistency for large language model generation , author=. 2311.17311 , archivePrefix=

    arXiv

  26. [34]

    Pan, Liangming and Albalak, Alon and Wang, Xinyi and Wang, William Yang , booktitle=. Logic-

  27. [35]

    Olausson, Theo and Gu, Alex and Lipkin, Ben and Zhang, Cedegao and Solar-Lezama, Armando and Tenenbaum, Joshua and Levy, Roger , booktitle=

  28. [36]

    2023 , publisher=

    Gao, Luyu and Madaan, Aman and Zhou, Shuyan and Alon, Uri and Liu, Pengfei and Yang, Yiming and Callan, Jamie and Neubig, Graham , booktitle=. 2023 , publisher=

    2023

  29. [37]

    On memorization of large language models in logical reasoning , author=. Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics , pages=

  30. [38]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) , pages=

    Soft Self-Consistency Improves Language Model Agents , author=. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) , pages=

  31. [39]

    and Chen, Yatong and Cruz, Andr

    Dorner, Florian E. and Chen, Yatong and Cruz, Andr. The Fourteenth International Conference on Learning Representations , year=

  32. [40]

    The Fourteenth International Conference on Learning Representations , year =

    Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domains , author=. The Fourteenth International Conference on Learning Representations , year =

  33. [41]

    2603.10624 , archivePrefix=

    Reinforcement Learning with Conditional Expectation Reward , author=. 2603.10624 , archivePrefix=

  34. [42]

    Estornell, Andrew and Liu, Yang , booktitle=. Multi-

  35. [43]

    Pearl, Judea , title =

  36. [44]

    Elements of Causal Inference: Foundations and Learning Algorithms , publisher =

    Peters, Jonas and Janzing, Dominik and Sch. Elements of Causal Inference: Foundations and Learning Algorithms , publisher =

  37. [45]

    Scandinavian Journal of Statistics , volume =

    Richardson, Thomas , title =. Scandinavian Journal of Statistics , volume =. 2003 , doi =

    2003

  38. [46]

    Journal of Machine Learning Research , volume =

    Shpitser, Ilya and Pearl, Judea , title =. Journal of Machine Learning Research , volume =

  39. [47]

    Journal of the American Statistical Association , volume =

    Hoeffding, Wassily , title =. Journal of the American Statistical Association , volume =. 1963 , doi =

    1963

  40. [48]

    and Schult, Daniel A

    Hagberg, Aric A. and Schult, Daniel A. and Swart, Pieter J. , title =. Proceedings of the 7th Python in Science Conference , pages =

  41. [49]

    Findings of the Association for Computational Linguistics: EMNLP 2024 , pages =

    Chen, Sirui and Xu, Mengying and Wang, Kun and Zeng, Xingyu and Zhao, Rui and Zhao, Shengjie and Lu, Chaochao , title =. Findings of the Association for Computational Linguistics: EMNLP 2024 , pages =. 2024 , doi =

    2024

  42. [50]

    Journal of Statistical Software , volume =

    Scutari, Marco , title =. Journal of Statistical Software , volume =. 2010 , doi =

    2010

  43. [51]

    2022 , howpublished =

    Scutari, Marco , title =. 2022 , howpublished =

    2022

  44. [52]

    Advanced Intelligent Computing Technology and Applications (ICIC 2025) , series =

    Wang, Chen and Zhao, Dongming and Wang, Bo and He, Ruifang and Hou, Yuexian , title =. Advanced Intelligent Computing Technology and Applications (ICIC 2025) , series =. 2025 , doi =

    2025

  45. [53]

    Findings of the Association for Computational Linguistics: NAACL 2025 , pages =

    Sheth, Ivaxi and Fatemi, Bahare and Fritz, Mario , title =. Findings of the Association for Computational Linguistics: NAACL 2025 , pages =. 2025 , doi =

    2025

  46. [54]

    Xie, Chulin and Huang, Yangsibo and Zhang, Chiyuan and Yu, Da and Chen, Xinyun and Lin, Bill Yuchen and Li, Bo and Ghazi, Badih and Kumar, Ravi , title =. Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-P...

    2025

  47. [55]

    Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers) , pages =

    He, Paul and Huang, Yinya and Sachan, Mrinmaya and Jin, Zhijing , title =. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers) , pages =. 2026 , doi =

    2026

  48. [56]

    Advances in Neural Information Processing Systems , volume =

    Jin, Zhijing and Chen, Yuen and Leeb, Felix and Gresele, Luigi and Kamal, Ojasv and Lyu, Zhiheng and Blin, Kevin and Gonzalez Adauto, Fernando and Kleiman-Weiner, Max and Sachan, Mrinmaya and Sch. Advances in Neural Information Processing Systems , volume =

  49. [57]

    Jin, Zhijing and Liu, Jiarui and Lyu, Zhiheng and Poff, Spencer and Sachan, Mrinmaya and Mihalcea, Rada and Diab, Mona T. and Sch. Can Large Language Models Infer Causation from Correlation? , booktitle =

  50. [58]

    arXiv preprint arXiv:2412.15115 , year =

    Yang, An and Yang, Baosong and Zhang, Beichen and Hui, Binyuan and Zheng, Bo and Yu, Bowen and Li, Chengyuan and Liu, Dayiheng and Huang, Fei and Wei, Haoran and others , title =. arXiv preprint arXiv:2412.15115 , year =

    arXiv

  51. [59]

    2024 , howpublished =

    Mistral. 2024 , howpublished =

    2024

  52. [60]

    and Shen, Yelong and Wallis, Phillip and Allen-Zhu, Zeyuan and Li, Yuanzhi and Wang, Shean and Wang, Lu and Chen, Weizhu , title =

    Hu, Edward J. and Shen, Yelong and Wallis, Phillip and Allen-Zhu, Zeyuan and Li, Yuanzhi and Wang, Shean and Wang, Lu and Chen, Weizhu , title =. The Tenth International Conference on Learning Representations , year =

  53. [61]

    Advances in Neural Information Processing Systems , volume =

    Dettmers, Tim and Pagnoni, Artidoro and Holtzman, Ari and Zettlemoyer, Luke , title =. Advances in Neural Information Processing Systems , volume =

  54. [62]

    Shao, Zhihong and Wang, Peiyi and Zhu, Qihao and Xu, Runxin and Song, Junxiao and Bi, Xiao and Zhang, Haowei and Zhang, Mingchuan and Li, Y. K. and Wu, Y. and Guo, Daya , title =. arXiv preprint arXiv:2402.03300 , year =

    arXiv

  55. [63]

    The Fourteenth International Conference on Learning Representations , year =

    Liu, Chris Yuhao and Zeng, Liang and Xiao, Yuzhen and He, Jujie and Liu, Jiacai and Wang, Chaojie and Yan, Rui and Shen, Wei and Zhang, Fuxiang and Xu, Jiacheng and Liu, Yang , title =. The Fourteenth International Conference on Learning Representations , year =

  56. [64]

    arXiv preprint arXiv:2509.06870 , year =

    Zhao, Wenting and Aggarwal, Pranjal and Saha, Swarnadeep and Celikyilmaz, Asli and Weston, Jason and Kulikov, Ilia , title =. arXiv preprint arXiv:2509.06870 , year =

    arXiv

  57. [65]

    , title =

    Huang, Audrey and Block, Adam and Liu, Qinghua and Jiang, Nan and Krishnamurthy, Akshay and Foster, Dylan J. , title =. Proceedings of the 42nd International Conference on Machine Learning , series =

  58. [66]

    and Chen, Yatong and Cruz, Andr

    Dorner, Florian E. and Chen, Yatong and Cruz, Andr. The Fourteenth International Conference on Learning Representations , year =

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.