Pith. sign in

REVIEW 4 major objections 5 minor 137 references

Position: It's Time to Optimize LLMs for Self-Consistency

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This position paper argues that persistent LLM failures — sycophancy, factual contradiction, unfaithful reasoning, overconfidence — all trace to one modeling assumption: evaluating behavior one output pair at a time.

desk verdict A useful unifying framework for cross-input consistency in LLMs, but the new-capability claims rest on an unproven assumption about LLM judges. read the letter →

arxiv 2608.05188 v1 pith:3M62NNJG submitted 2026-07-31 cs.CL cs.AI

Itamar Pres , Belinda Z. Li , Laura Ruis , Zifan Carl Guo , Keya Hu , Mehul Damani , Isha Puri , Ekdeep Singh Lubana
show 1 more author
Jacob Andreas
This is my paper
classification cs.CLcs.AI
keywords self-consistencyconsistencyoptimizationlanguagemodelalignmentsycophancychain-of-thoughtfaithfulnesscalibrationself-descriptionpositionpaper
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This position paper argues that many stubborn failures of large language models — sycophancy, factual contradiction, unfaithful chain-of-thought, overconfidence — share a single root cause: behavior is specified and evaluated on single output pairs, while the failure only appears when responses across related inputs are compared together. The paper proposes treating self-consistency, the coherence of a model's behavior across related inputs, as a first-class training objective, formalized as maximizing a consistency function over sets of related input–output pairs. It shows that a wide range of existing techniques, from robustness training to calibration to AI-feedback reinforcement learning, are special cases of this one objective, and argues that the same lens specifies emerging capabilities such as models that describe their own behavior and red-team themselves. If right, this gives the field a common language for diagnosing failures and a standard set of optimization tools for fixing them, instead of bespoke patches per failure mode.

What carries the argument

The load-bearing object is the consistency function $\phi(x_{1:n}, y_{1:n})$, a scalar scoring how well a set of model responses satisfies a relational constraint across related inputs, together with the consistency objective $\mathbb{E}_{y_{1:n} \sim p_\theta(\cdot|x_{1:n})}[\phi(x_{1:n}, y_{1:n})]$. The paper instantiates $\phi$ through two pattern families: instance-level constraints of the form $\phi = \mathbb{1}[R(x_{1:n}) \rightarrow S(y_{1:n})] - 1$, where $R$ and $S$ are relations on input and output tuples (invariances like "same question, different user framing" and equivariances like "contradicting facts should get contradicting answers"), and meta-level constraints that tie a model's self-description to its own observed behavior. The objective is optimized by three generic tools — hard constraints, soft regularization, and posterior regularization — so the semantics of consistency are decoupled from the optimization procedure.

What would settle it

Train a model on the consistency objective over a finite set of relations (paraphrase invariance, contradiction propagation, preference invariance) and then probe a held-out relational constraint, such as a novel logical transformation never used in training; if the model fails to respect the held-out relation despite near-perfect scores on the training relations, the claim that optimizing $\phi$ transfers to consistency relations broadly would be refuted. A direct measurement showing an LLM judge's consistency score disagrees with human judgment on a large fraction of candidate response pairs would likewise break the scalable-judge premise.

Watch

Extended reading notes

Core claim

The central claim is a unification: most persistent LLM failure modes and many post-training techniques can be seen as failures or optimizations of a single relational objective. If $R(x_{1:n})$ is a relation on inputs (e.g., two prompts are paraphrases, or two statements contradict each other) and $S(y_{1:n})$ a corresponding relation on outputs, then a consistency constraint $\phi = \mathbb{1}[R(x_{1:n}) \rightarrow S(y_{1:n})] - 1$ penalizes the model whenever related inputs produce outputs that violate the expected output relation. Maximizing $\mathbb{E}_{y_{1:n} \sim p_\theta(\cdot|x_{1:n})}[\phi(x_{1:n}, y_{1:n})]$ over curated sets of related inputs recovers sycophancy reduction, knowledge-update propagation, calibration, RLAIF, and chain-of-thought faithfulness training as special cases. The paper further argues that meta-level consistency — requiring a model's natural-language descriptions of its own behavior to match its actual behavior — unifies introspection, self-critique, and self-improvement, and opens new capabilities such as automated self-red-teaming and hypothesis generation.

Load-bearing premise

The framework works only if, in practice, the consistency function $\phi$ and the sets of related inputs can be specified reliably — typically by LLM-as-a-judge or verifiers — and if evaluating whether two responses are consistent is substantially easier than generating a consistent response; the paper notes but does not resolve that models can converge to degenerate fixed points that trivially satisfy $\phi$ without being helpful.

Editorial extensions

If this is right

  • A single post-training recipe, applied with a generic $\phi$, should simultaneously reduce sycophancy, factual contradiction, prompt-format sensitivity, and unfaithful chain-of-thought, replacing bespoke fixes for each failure mode.
  • Self-description and self-behavior become mutually reinforcing: a model trained to state its own failure modes accurately provides supervision that corrects those failures, yielding scalable loops of self-improvement.
  • New capabilities become formally specifiable as consistency constraints, including self-red-teaming (a model proposing inputs that make it violate a stated constraint) and hypothesis generation (a model articulating executable rules consistent with its own predictions).
  • Because consistency constraints are typically checkable without external ground truth, models can be trained to respect them from unlabeled structured data, easing pressure from data scarcity.
  • Meta-level consistency gives auditors a new lever: a model whose explanations must match its actions is harder to keep deceptively misaligned, since the explanation layer itself becomes testable.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper's instances, the framework suggests a testable taxonomy of LLM failure modes by the type of relation $R$ that is violated — identity (paraphrase, bias), order (logical implication, knowledge update), and group (pluralistic alignment) — each implying a different data-generation strategy for building training pairs.
  • The paper treats $\phi$ as externally specified, but an interesting extension is learning $\phi$ itself as a consistency reward model jointly with the policy; this risks reward hacking, and a concrete stress-test would be measuring whether degenerate fixed points are detectable through adversarial generation of new input sets.
  • If consistent generalization holds, it suggests a scaling phenomenology: consistency violations should decay as the number of distinct relations seen in training grows, which could be measured on existing benchmark suites before committing to the framework.
  • The framework implies an institutional shift in evaluation: benchmarks should move from single-response scoring to paired or set-based scoring, which would change how model leaderboards are built.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper is a position paper arguing that persistent LLM failures—sycophancy, incomplete logical generalization, and confident incorrectness—share a common modeling assumption: behavior is specified and evaluated independently on single input-output pairs. It proposes self-consistency as a unifying framework, formalized in Eq. (4) as maximizing the expectation of a consistency function phi over sets of related inputs. Section 3 casts existing instance-level objectives such as sycophancy reduction and factual consistency as instances of the template phi = 1[R(x)->S(y)]-1, distinguishing invariances from equivariances. Section 4 introduces meta-level self-consistency, covering self-description, chain-of-thought faithfulness, calibration, self-red-teaming, hypothesis generation, and self-alignment including RLAIF. Section 5 and the appendices discuss optimization tools, evaluation via LLM-as-a-judge, and potential safety objections. The paper explicitly acknowledges degenerate fixed points and cites evidence that consistency training can entrench misalignment, but the central capability claims rely on scalable and faithful relational judges.

Significance. If the position is correct, it would supply a common vocabulary and shared optimization toolkit for a wide range of alignment, robustness, and interpretability problems, and it would give a concrete proposal for training models whose self-descriptions are faithful to their behavior. The paper has genuine strengths: the instance/meta-level taxonomy is clear, the case studies make the proposal concrete, and the authors are candid about limitations, including the risk of degenerate fixed points and the possibility that consistency training can entrench misalignment. The paper does not present machine-checked proofs, reproducible code, or empirical validations; its contribution is conceptual. As a position paper this is acceptable in principle, but several load-bearing formalizations need correction before the unification claim is credible.

major comments (4)
  1. [§2, Eq. (7) and §C.2.2, Eq. (30)] The posterior regularization objective in Eq. (7) is written as max_theta [L(theta) + min_{q in Q} KL(q || p_theta)]. Since KL is nonnegative and L is a maximization objective, adding the min-KL term rewards models for which no consistent distribution q is close, i.e., it pushes p_theta away from the constraint set. This contradicts the text's description of projecting the model toward the closest distribution satisfying the consistency constraints. Standard posterior regularization subtracts this term. The same sign issue appears in Eq. (30) in Appendix C.2.2 with the reversed KL. This is a load-bearing error for the claim that Eq. (7) provides a shared optimization tool.
  2. [§4.3 (Case Study 8), Eq. (19)] The RLAIF instantiation is a relabeling rather than a derivation. The proposed consistency function phi_improve = y_meta log p_theta(y^D_1 | x^D_1) is a reward-weighted log-likelihood; it contains no predicate checking that the meta-output y_meta is consistent with, or faithfully evaluates, the behavior y^D_1. Any RLHF-style objective can be written in this form by choosing y_meta as the reward, so this case study does not demonstrate that RLAIF is a special case of self-consistency. To support the unification claim, the authors should either reformulate RLAIF with an explicit consistency predicate (e.g., requiring that the evaluation predicts behavior under counterfactuals) or remove this case study and qualify the claim that existing self-training methods are all captured by Eq. (4).
  3. [§4.1 and §C.4, Eqs. (14)-(15)] The most novel capability claims depend on judges that can evaluate whether 'ymeta explains change between y^D_1 and y^D_2' (Eq. 14) and whether a counterfactual input difference 'is explained by CoT in xmeta' (Eq. 15). These are exactly the open problems of counterfactual simulatability and chain-of-thought faithfulness that the paper cites as difficult. The paper asserts that evaluating consistency is substantially easier than generating consistent responses, and that LLMs can serve as scalable judges, but it provides no evidence for these assertions. Moreover, allowing the judge to share parameters with the evaluated model (§C.4) creates a possible circular objective in which the model maximizes phi by producing vacuous self-consistent-sounding meta-descriptions without faithfully tracking behavior. The authors should either narrow the capability claims or specify a concrete validation protocol, such as measuring judge accuracy against human labels for each phi.
  4. [§5] The paper acknowledges degenerate fixed points—models that trivially avoid contradiction—but the proposed remedies are only qualitative ('training jointly with standard SFT or RLHF objectives' or 'post-training refinement with regularization'). The central optimistic claim that optimizing Eq. (4) can simultaneously reduce sycophancy, factual inconsistency, chain-of-thought unfaithfulness, and overconfidence requires at least a minimal formal condition under which the joint objective does not collapse to trivial constant policies. Without such a condition, or an explicit statement that this is an open assumption, the framework's predictive content is hard to evaluate. I recommend adding a short subsection stating the assumptions needed for the joint objective to have non-degenerate solutions.
minor comments (5)
  1. [Abstract] Page 1: 'proposeself-consistency' should read 'propose self-consistency'.
  2. [§2, Eq. (5)] The hard constraint as written is not well-formed: 's.t. E[phi]' should include an inequality such as E[phi] >= 0 to match the definition of Q in Eq. (7).
  3. [Appendix A.2, Eq. (25)] The output relation S is written as S(x1, x2) in Eq. (25) but should be S(y1, y2), consistent with the template in Eq. (8).
  4. [§5 and Appendix A.2] There are typos: 'graphical mdoels' in Section 5 should be 'graphical models', and 'resonse' in Appendix A.2 should be 'response'.
  5. [§2 and §4.1 (Case Study 5)] The claim that self-consistency constraints 'can often be done without reference to any external source of ground truth' sits in tension with the calibration case study, where the consistency function phi_cal in Eq. (16) requires external correctness labels a_i. The word 'often' provides some cover, but the relationship between self-contained consistency and externally grounded consistency should be clarified.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the unifying objective is definitional by design, no fitted parameters or predictions reduce to their inputs, and self-citations are illustrative rather than load-bearing.

full rationale

This is a position paper, not a derivation with fitted parameters or empirical predictions. The central object, Eq. (4), is proposed as a general objective, and Section 3 explicitly shows how existing techniques can be recovered by choosing φ through Eq. (8); this is an organizing taxonomy, not a prediction forced by construction. The paper states that φ 'may encode equivalence, logical entailment, factual consistency, normative consistency, or any other relational constraint between responses', so the flexibility is acknowledged rather than hidden, and the contribution is the unification itself. No benchmark or empirical prediction is made, so there is no fitted-input-called-prediction pattern. The self-citations (e.g., Li et al. 2025b for φ_interv, Damani et al. 2026 for φ_cal) appear only as examples of prior instantiations; the framework's validity does not rest on those papers, and independent citations (Plunkett et al., Binder et al., Chen et al.) also support the meta-level phenomena. Section 5 explicitly acknowledges the two serious limitations (degenerate fixed points and the assumption that relational evaluation is easier than generation); these are correctness risks, not circular steps. The only mild concern is that a framework defined by an arbitrary consistency function is close to a definitional statement, but the paper claims only a framework and a research program, so this does not constitute circular reasoning.

Assumptions & free parameters 0 free parameters · 3 assumptions · 1 invented entities

The paper introduces no fitted free parameters. Its load-bearing axioms are that language model behavior is fully described by conditional distributions, that consistency functions can be practically specified and evaluated, and that optimizing these objectives will not trivially collapse. The only invented entity is the self-consistency objective itself, which is a formal construct without independent empirical evidence.

assumptions (3)
  • domain assumption Language model behavior is fully captured by conditional distributions p_theta(y|x)
    Eqs. (1)-(4) define all objectives on these distributions; no output-level uncertainty beyond sampling is considered.
  • domain assumption A scalar consistency function phi can encode all desired relational constraints and is computable by LLM judges or verifiers
    Central to the framework; the paper does not prove this for any specific instantiation, and the reliability of LLM judges is assumed.
  • ad hoc to paper Optimizing Eq. (4) jointly with a pointwise loss does not collapse to degenerate solutions
    Section 5 acknowledges degenerate fixed points but provides no mitigation; this is an open assumption of the proposal.
invented entities (1)
  • Self-consistency objective (Eq. 4)
    purpose: Unified objective for training and evaluating LMs across related inputs
    A formal construct introduced by the paper; no empirical validation is provided, and its practical tractability is not demonstrated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Position: It's Time to Optimize LLMs for Self-Consistency." pith.science (2026). https://pith.science/paper/3M62NNJG

@misc{pith2026260805188,
  author       = {Pith},
  title        = {Pith review of: Position: It's Time to Optimize LLMs for Self-Consistency},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3M62NNJG}},
  note         = {Machine review of arXiv:2608.05188}
}
read the original abstract

Despite ever-increasing sophistication in language model (LM) pre- and post-training pipelines, many important failures persist: models overcondition on user framing ("sycophancy"), exhibit incomplete logical generalization, and produce confident but incorrect responses. We argue that these failures arise from a modeling assumption permeating all aspects of the pipeline: that behavior can be specified and evaluated independently on single-output pairs. Many model failures are difficult, if not impossible, to detect without reasoning about relationships between a model's responses across inputs. In this position paper, we propose self-consistency as a framework for understanding these failures. We first observe that a wide variety of techniques designed to improve specific aspects of LM behavior-targeting properties as diverse as adversarial robustness and factual coherence-can be understood as special cases of a common "consistency optimization" procedure and addressed with a standard set of optimization tools. We next outline a set of new model properties that could be achieved by optimizing for consistency, and conclude with a discussion of what it would mean to develop generally consistent LMs, including the capabilities they would enable and the objections they raise.

Figures

Figures reproduced from arXiv: 2608.05188 by the authors.

Figure 1
Figure 1. (A) Standard SFT/RL trains responses independently, which can induce inconsistent outputs. (B) Cross-input optimiza￾tion directly enforces consistency through a relational objective ϕ. et al., 2024), and frontier labs are turning to synthetic data to push the established data-and-model scaling paradigm further (Grattafiori et al., 2024; Zeng et al., 2025, inter alia). Yet a persistent set of failures continues to un… view at source ↗
Figure 2
Figure 2. Framework Overview. In the general self-consistency framework (left), a consistency function ϕ jointly scores sets of input-output pairs. We instantiate two relationship types: instance/instance self-consistency (middle), where a relation R on inputs implies a corresponding relation S on outputs; and meta/instance self-consistency (right), where ϕ scores a meta-level input-output pair for consistency with model beha… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

137 extracted references · 27 canonical work pages

  1. [1]

    Forty-second International Conference on Machine Learning , year =

    Archiki Prasad and Weizhe Yuan and Richard Yuanzhe Pang and Jing Xu and Maryam Fazel-Zarandi and Mohit Bansal and Sainbayar Sukhbaatar and Jason E Weston and Jane Yu , title =. Forty-second International Conference on Machine Learning , year =

  2. [2]

    Lew and Tan Zhi-Xuan and Gabriel Grand and Vikash Mansinghka , title =

    Alexander K. Lew and Tan Zhi-Xuan and Gabriel Grand and Vikash Mansinghka , title =. ICML 2023 Workshop: Sampling and Optimization in Discrete Space , year =

    2023

  3. [3]

    Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , year =

    Kassner, Nora and Tafjord, Oyvind and Sabharwal, Ashish and Richardson, Kyle and Schuetze, Hinrich and Clark, Peter , title =. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , year =. doi:10.18653/v1/2023.emnlp-main.877 , url =

    doi:10.18653/v1/2023.emnlp-main.877 2023

  4. [4]

    ICLR 2026 Workshop on Lifelong Agents: Learning, Aligning, Evolving , year =

    Siyan Zhao and Zhihui Xie and Mengchen Liu and Jing Huang and Guan Pang and Feiyu Chen and Aditya Grover , title =. ICLR 2026 Workshop on Lifelong Agents: Learning, Aligning, Evolving , year =

    2026

  5. [5]

    , journal=

    Scudder, H. , journal=. Probability of error of some adaptive pattern-recognition machines , year=

  6. [6]

    International conference on machine learning , pages=

    Born again neural networks , author=. International conference on machine learning , pages=. 2018 , organization=

    2018

  7. [7]

    International Conference on Learning Representations , year=

    Your classifier is secretly an energy based model and you should treat it like one , author=. International Conference on Learning Representations , year=

  8. [8]

    The Eleventh International Conference on Learning Representations , year=

    Self-Consistency Improves Chain of Thought Reasoning in Language Models , author=. The Eleventh International Conference on Learning Representations , year=

Show all 137 references
  1. [9]

    1805.00899 , archivePrefix=

    Geoffrey Irving and Paul Christiano and Dario Amodei , year=. 1805.00899 , archivePrefix=

    arXiv

  2. [10]

    and Zoph, Barret and Mane, Dandelion and Vasudevan, Vijay and Le, Quoc V

    Cubuk, Ekin D. and Zoph, Barret and Mane, Dandelion and Vasudevan, Vijay and Le, Quoc V. , title =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , month =

  3. [11]

    ImageNet Classification with Deep Convolutional Neural Networks , url =

    Krizhevsky, Alex and Sutskever, Ilya and Hinton, Geoffrey E , booktitle =. ImageNet Classification with Deep Convolutional Neural Networks , url =

  4. [12]

    Adversarial Example Generation with Syntactically Controlled Paraphrase Networks

    Iyyer, Mohit and Wieting, John and Gimpel, Kevin and Zettlemoyer, Luke. Adversarial Example Generation with Syntactically Controlled Paraphrase Networks. Proceedings of the 2018 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human L...

    2018 doi

  5. [13]

    Semantically Equivalent Adversarial Rules for Debugging NLP models

    Ribeiro, Marco Tulio and Singh, Sameer and Guestrin, Carlos. Semantically Equivalent Adversarial Rules for Debugging NLP models. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2018. doi:10.18653/v1/P18-1079

    2018 doi

  6. [14]

    and Wallach, Hanna and Cotterell, Ryan

    Zmigrod, Ran and Mielke, Sabrina J. and Wallach, Hanna and Cotterell, Ryan. Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphology. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019. doi:...

    2019 doi

  7. [15]

    Journal of Machine Learning Research , volume=

    Causal abstraction: A theoretical foundation for mechanistic interpretability , author=. Journal of Machine Learning Research , volume=

  8. [16]

    2021 , eprint=

    How to Train Your Energy-Based Models , author=. 2021 , eprint=

    2021

  9. [17]

    Langley , title =

    P. Langley , title =. Proceedings of the 17th International Conference on Machine Learning (ICML 2000) , address =. 2000 , pages =

    2000

  10. [18]

    The Twelfth International Conference on Learning Representations , year=

    Towards Best Practices of Activation Patching in Language Models: Metrics and Methods , author=. The Twelfth International Conference on Learning Representations , year=

  11. [19]

    T. M. Mitchell. The Need for Biases in Learning Generalizations. 1980

    1980

  12. [20]

    M. J. Kearns , title =

  13. [21]

    Machine Learning: An Artificial Intelligence Approach, Vol. I. 1983

    1983

  14. [22]

    R. O. Duda and P. E. Hart and D. G. Stork. Pattern Classification. 2000

    2000

  15. [23]

    Suppressed for Anonymity , author=

  16. [24]

    Newell and P

    A. Newell and P. S. Rosenbloom. Mechanisms of Skill Acquisition and the Law of Practice. Cognitive Skills and Their Acquisition. 1981

    1981

  17. [25]

    A. L. Samuel. Some Studies in Machine Learning Using the Game of Checkers. IBM Journal of Research and Development. 1959

    1959

  18. [26]

    Aho and Jeffrey D

    Alfred V. Aho and Jeffrey D. Ullman , title =. 1972

    1972

  19. [27]

    Publications Manual , year = "1983", publisher =

    1983

  20. [28]

    Chandra and Dexter C

    Ashok K. Chandra and Dexter C. Kozen and Larry J. Stockmeyer , year = "1981", title =. doi:10.1145/322234.322243

    1981

  21. [29]

    Scalable training of

    Andrew, Galen and Gao, Jianfeng , booktitle=. Scalable training of

  22. [30]

    Dan Gusfield , title =. 1997

    1997

  23. [31]

    NeurIPS , year =

    Kevin Meng and David Bau and Alex Andonian and Yonatan Belinkov , title =. NeurIPS , year =

  24. [32]

    Tetreault , title =

    Mohammad Sadegh Rasooli and Joel R. Tetreault , title =. Computing Research Repository , volume =. 2015 , url =

    2015

  25. [33]

    A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data , Volume =

    Ando, Rie Kubota and Zhang, Tong , Issn =. A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data , Volume =. Journal of Machine Learning Research , Month = dec, Numpages =

  26. [34]

    The Reversal Curse:

    Lukas Berglund and Meg Tong and Maximilian Kaufmann and Mikita Balesni and Asa Cooper Stickland and Tomasz Korbak and Owain Evans , booktitle=. The Reversal Curse:. 2024 , url=

    2024

  27. [35]

    Exploring the Sensitivity of LLM s' Decision-Making Capabilities: Insights from Prompt Variations and Hyperparameters

    Loya, Manikanta and Sinha, Divya and Futrell, Richard. Exploring the Sensitivity of LLM s' Decision-Making Capabilities: Insights from Prompt Variations and Hyperparameters. Findings of the Association for Computational Linguistics: EMNLP 2023. 2023. doi:10.18653/v1/2023.findi...

    2023 doi

  28. [36]

    2023 , eprint=

    Taken out of context: On measuring situational awareness in LLMs , author=. 2023 , eprint=

    2023

  29. [37]

    Emergent Misalignment: Narrow finetuning can produce broadly misaligned

    Jan Betley and Daniel Chee Hian Tan and Niels Warncke and Anna Sztyber-Betley and Xuchan Bao and Mart. Emergent Misalignment: Narrow finetuning can produce broadly misaligned. Forty-second International Conference on Machine Learning , year=

  30. [38]

    Fundamental Problems With Model Editing: How Should Rational Belief Revision Work in

    Peter Hase and Thomas Hofweber and Xiang Zhou and Elias Stengel-Eskin and Mohit Bansal , journal=. Fundamental Problems With Model Editing: How Should Rational Belief Revision Work in. 2024 , url=

    2024

  31. [39]

    Findings of the Association for Computational Linguistics: ACL 2024 , pages=

    Deductive closure training of language models for coherence, accuracy, and updatability , author=. Findings of the Association for Computational Linguistics: ACL 2024 , pages=

    2024

  32. [40]

    Findings of the Association for Computational Linguistics: ACL 2023 , pages=

    Evaluating the factual consistency of large language models through news summarization , author=. Findings of the Association for Computational Linguistics: ACL 2023 , pages=

    2023

  33. [41]

    arXiv preprint arXiv:2506.18781 , year=

    Existing LLMs Are Not Self-Consistent For Simple Tasks , author=. arXiv preprint arXiv:2506.18781 , year=

    arXiv

  34. [42]

    Transactions on Machine Learning Research , year =

    Ahmed M Ahmed and Kevin Klyman and Yi Zeng and Sanmi Koyejo and Percy Liang , title =. Transactions on Machine Learning Research , year =

  35. [43]

    arXiv preprint arXiv:2509.03730 , year=

    The personality illusion: Revealing dissociation between self-reports & behavior in llms , author=. arXiv preprint arXiv:2509.03730 , year=

    arXiv

  36. [44]

    Proceedings of the 30th Conference on Pattern Languages of Programs , pages=

    A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT , author=. Proceedings of the 30th Conference on Pattern Languages of Programs , pages=

  37. [45]

    arXiv preprint arXiv:2512.18311 , year=

    Monitoring Monitorability , author=. arXiv preprint arXiv:2512.18311 , year=

  38. [46]

    2025 , note =

    Chain-of-Thought Is Not Explainability , author =. 2025 , note =

    2025

  39. [47]

    2025 , month = nov, url =

    Claude Opus 4.5 System Card , institution =. 2025 , month = nov, url =

    2025

  40. [48]

    2025 , month = dec, date =

    Update to GPT-5 System Card: GPT-5.2 , institution =. 2025 , month = dec, date =

    2025

  41. [49]

    arXiv preprint arXiv:2306.05499 , year=

    Prompt injection attack against llm-integrated applications , author=. arXiv preprint arXiv:2306.05499 , year=

    arXiv

  42. [50]

    Advances in Neural Information Processing Systems , volume=

    Tree of attacks: Jailbreaking black-box llms automatically , author=. Advances in Neural Information Processing Systems , volume=

  43. [51]

    arXiv preprint arXiv:2402.09154 , year=

    Attacking large language models with projected gradient descent , author=. arXiv preprint arXiv:2402.09154 , year=

    arXiv

  44. [52]

    Same Question, Different Words: A Latent Adversarial Framework for Prompt Robustness

    Fu, Tingchen and Barez, Fazl. Same Question, Different Words: A Latent Adversarial Framework for Prompt Robustness. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025. doi:10.18653/v1/2025.emnlp-main.1595

    2025 doi

  45. [53]

    Findings of the Association for Computational Linguistics: EACL 2024 , pages=

    Prompt perturbation consistency learning for robust language models , author=. Findings of the Association for Computational Linguistics: EACL 2024 , pages=

    2024

  46. [54]

    arXiv preprint arXiv:2511.08579 , year=

    Training Language Models to Explain Their Own Computations , author=. arXiv preprint arXiv:2511.08579 , year=

  47. [55]

    arXiv preprint arXiv:2510.05092 , year=

    Learning to Interpret Weight Differences in Language Models , author=. arXiv preprint arXiv:2510.05092 , year=

  48. [56]

    2025 , eprint=

    Self-Interpretability: LLMs Can Describe Complex Internal Processes that Drive Their Decisions , author=. 2025 , eprint=

    2025

  49. [57]

    arXiv preprint arXiv:2410.13787 , year=

    Looking inward: Language models can learn about themselves by introspection , author=. arXiv preprint arXiv:2410.13787 , year=

    arXiv

  50. [58]

    arXiv preprint arXiv:2406.07882 , year=

    Designing a dashboard for transparency and control of conversational AI , author=. arXiv preprint arXiv:2406.07882 , year=

    arXiv

  51. [59]

    Forty-first International Conference on Machine Learning , year=

    Position: A Roadmap to Pluralistic Alignment , author=. Forty-first International Conference on Machine Learning , year=

  52. [60]

    2021 , eprint=

    Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges , author=. 2021 , eprint=

    2021

  53. [61]

    A Benchmark Dataset for Learning to Intervene in Online Hate Speech

    Qian, Jing and Bethke, Anna and Liu, Yinyin and Belding, Elizabeth and Wang, William Yang. A Benchmark Dataset for Learning to Intervene in Online Hate Speech. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Join...

    2019 doi

  54. [62]

    Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021 , year =

    Garimella, Aparna and Amarnath, Akhash and Kumar, Kiran and Yalla, Akash Pramod and N, Anandhavelu and Chhaya, Niyati and Srinivasan, Balaji Vasan , title =. Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021 , year =. doi:10.18653/v1/2021.findings-acl....

    2021 doi

  55. [63]

    Auto-Debias: Debiasing Masked Language Models with Automated Biased Prompts

    Guo, Yue and Yang, Yi and Abbasi, Ahmed. Auto-Debias: Debiasing Masked Language Models with Automated Biased Prompts. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2022. doi:10.18653/v1/2022.acl-long.72

    2022 doi

  56. [64]

    Logic, language, and security: essays dedicated to Andre Scedrov on the occasion of his 65th birthday , pages=

    Gender bias in neural natural language processing , author=. Logic, language, and security: essays dedicated to Andre Scedrov on the occasion of his 65th birthday , pages=. 2020 , publisher=

    2020

  57. [65]

    arXiv preprint arXiv:2010.06032 , year=

    Measuring and reducing gendered correlations in pre-trained models , author=. arXiv preprint arXiv:2010.06032 , year=

    2010 arXiv

  58. [66]

    Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society , pages =

    Dixon, Lucas and Li, John and Sorensen, Jeffrey and Thain, Nithum and Vasserman, Lucy , title =. Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society , pages =. 2018 , isbn =. doi:10.1145/3278721.3278729 , abstract =

    2018

  59. [67]

    and Rossi, Ryan A

    Gallegos, Isabel O. and Rossi, Ryan A. and Barrow, Joe and Tanjim, Md Mehrab and Kim, Sungchul and Dernoncourt, Franck and Yu, Tong and Zhang, Ruiyi and Ahmed, Nesreen K. Bias and Fairness in Large Language Models: A Survey. Computational Linguistics. 2024. doi:10.1162/coli_a_00524

    2024 doi

  60. [68]

    and Rambachan, Ashesh and Mullainathan, Sendhil , booktitle =

    Vafa, Keyon and Chang, Peter G. and Rambachan, Ashesh and Mullainathan, Sendhil , booktitle =. What Has a Foundation Model Found?. 2025 , editor =

    2025

  61. [69]

    Highly accurate protein structure prediction with

    Jumper, John and Evans, Richard and Pritzel, Alexander and Green, Tim and Figurnov, Michael and Ronneberger, Olaf and Tunyasuvunakool, Kathryn and Bates, Russ and. Highly accurate protein structure prediction with. Nature , year =

  62. [70]

    2025 , month = feb, day =

    Interpreting Evo 2: Arc Institute's Next-Generation Genomic Foundation Model , author =. 2025 , month = feb, day =

    2025

  63. [71]

    2025 , url=

    Over 30 AI models have been trained at the scale of GPT-4 , author=. 2025 , url=

    2025

  64. [72]

    Position: Will we run out of data? Limits of

    Pablo Villalobos and Anson Ho and Jaime Sevilla and Tamay Besiroglu and Lennart Heim and Marius Hobbhahn , booktitle=. Position: Will we run out of data? Limits of. 2024 , url=

    2024

  65. [73]

    2024 , eprint=

    The Llama 3 Herd of Models , author=. 2024 , eprint=

    2024

  66. [74]

    2025 , eprint=

    GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models , author=. 2025 , eprint=

    2025

  67. [75]

    The Twelfth International Conference on Learning Representations , year=

    Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting , author=. The Twelfth International Conference on Learning Representations , year=

  68. [76]

    Miao Xiong and Zhiyuan Hu and Xinyang Lu and YIFEI LI and Jie Fu and Junxian He and Bryan Hooi , booktitle=. Can. 2024 , url=

    2024

  69. [77]

    What Did I Do Wrong? Quantifying LLM s' Sensitivity and Consistency to Prompt Engineering

    Errica, Federico and Sanvito, Davide and Siracusano, Giuseppe and Bifulco, Roberto. What Did I Do Wrong? Quantifying LLM s' Sensitivity and Consistency to Prompt Engineering. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Compu...

    2025 doi

  70. [78]

    Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2 , pages =

    Liu, Xiaoou and Chen, Tiejin and Da, Longchao and Chen, Chacha and Lin, Zhen and Wei, Hua , title =. Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2 , pages =. 2025 , isbn =. doi:10.1145/3711896.3736569 , abstract =

    2025

  71. [79]

    First Workshop on Foundations of Reasoning in Language Models , year=

    On the generalization of language models from in-context learning and finetuning: a controlled study , author=. First Workshop on Foundations of Reasoning in Language Models , year=

  72. [80]

    Bell , booktitle=

    Polina Kirichenko and Mark Ibrahim and Kamalika Chaudhuri and Samuel J. Bell , booktitle=. AbstentionBench: Reasoning. 2025 , url=

    2025

  73. [81]

    Transactions on Machine Learning Research , issn=

    Teaching Models to Express Their Uncertainty in Words , author=. Transactions on Machine Learning Research , issn=. 2022 , url=

    2022

  74. [82]

    Journal of the American statistical Association , volume=

    Strictly proper scoring rules, prediction, and estimation , author=. Journal of the American statistical Association , volume=. 2007 , publisher=

    2007

  75. [83]

    Beyond Binary Rewards: Training

    Mehul Damani and Isha Puri and Stewart Slocum and Idan Shenfeld and Leshem Choshen and Yoon Kim and Jacob Andreas , booktitle=. Beyond Binary Rewards: Training. 2026 , url=

    2026

  76. [84]

    Taming Overconfidence in

    Jixuan Leng and Chengsong Huang and Banghua Zhu and Jiaxin Huang , booktitle=. Taming Overconfidence in. 2025 , url=

    2025

  77. [85]

    Self-Supervised Learning with Data Augmentations Provably Isolates Content from Style , url =

    von K\". Self-Supervised Learning with Data Augmentations Provably Isolates Content from Style , url =. Advances in Neural Information Processing Systems , editor =

  78. [86]

    and Darzi, Ara and Etemadi, Mozziyar and Garcia-Vicente, Florencia and Gilbert, Fiona J

    McKinney, Scott Mayer and Sieniek, Marcin and Godbole, Varun and Godwin, Jonathan and Antropova, Natasha and Ashrafian, Hutan and Back, Trevor and Chesus, Mary and Corrado, Greg S. and Darzi, Ara and Etemadi, Mozziyar and Garcia-Vicente, Florencia and Gilbert, Fiona J. and Hal...

  79. [87]

    International Conference on Learning Representations , year=

    Theoretical Analysis of Self-Training with Deep Networks on Unlabeled Data , author=. International Conference on Learning Representations , year=

  80. [88]

    Guo, Daya and Yang, Dejian and Zhang, Haowei and Song, Junxiao and Wang, Peiyi and Zhu, Qihao and Xu, Runxin and Zhang, Ruoyu and Ma, Shirong and Bi, Xiao and Zhang, Xiaokang and Yu, Xingkai and Wu, Yu and Wu, Z. F. and Gou, Zhibin and Shao, Zhihong and Li, Zhuoshu and Gao, Zi...

  81. [89]

    Training language models to follow instructions with human feedback , url =

    Ouyang, Long and Wu, Jeffrey and Jiang, Xu and Almeida, Diogo and Wainwright, Carroll and Mishkin, Pamela and Zhang, Chong and Agarwal, Sandhini and Slama, Katarina and Ray, Alex and Schulman, John and Hilton, Jacob and Kelton, Fraser and Miller, Luke and Simens, Maddie and As...

  82. [90]

    2025 , eprint=

    Why Language Models Hallucinate , author=. 2025 , eprint=

    2025

  83. [91]

    The Twelfth International Conference on Learning Representations , year=

    Towards Understanding Sycophancy in Language Models , author=. The Twelfth International Conference on Learning Representations , year=

  84. [92]

    Science , year =

    Cheng, Myra and Lee, Cinoo and Khadpe, Pranav and Yu, Sunny and Han, Dyllan and Jurafsky, Dan , title =. Science , year =. doi:10.1126/science.aec8352 , url =

    doi

  85. [93]

    Findings of the Association for Computational Linguistics: EMNLP 2025 , year =

    Hong, Jiseung and Byun, Grace and Kim, Seungone and Shu, Kai , title =. Findings of the Association for Computational Linguistics: EMNLP 2025 , year =. doi:10.18653/v1/2025.findings-emnlp.121 , url =

    2025 doi

  86. [94]

    Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society , author=

    SycEval: Evaluating LLM Sycophancy , volume=. Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society , author=. 2025 , month=. doi:10.1609/aies.v8i1.36598 , abstractNote=

    2025 doi

  87. [95]

    Goodfire , year =

    Aranguri, Santiago and McGrath, Tom , title =. Goodfire , year =

  88. [96]

    The Twelfth International Conference on Learning Representations , year=

    Human Feedback is not Gold Standard , author=. The Twelfth International Conference on Learning Representations , year=

  89. [97]

    2024 , eprint=

    Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Models , author=. 2024 , eprint=

    2024

  90. [98]

    2024 , eprint=

    Alignment faking in large language models , author=. 2024 , eprint=

    2024

  91. [99]

    2025 , eprint=

    Superintelligent Agents Pose Catastrophic Risks: Can Scientist AI Offer a Safer Path? , author=. 2025 , eprint=

    2025

  92. [100]

    The Alignment Problem from a Deep Learning Perspective , booktitle =

    Richard Ngo and Lawrence Chan and S. The Alignment Problem from a Deep Learning Perspective , booktitle =. 2024 , publisher =

    2024

  93. [101]

    2024 , editor =

    Lee, Harrison and Phatale, Samrat and Mansoor, Hassan and Mesnard, Thomas and Ferret, Johan and Lu, Kellie Ren and Bishop, Colton and Hall, Ethan and Carbune, Victor and Rastogi, Abhinav and Prakash, Sushant , booktitle =. 2024 , editor =

    2024

  94. [102]

    2025 , eprint=

    Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety , author=. 2025 , eprint=

    2025

  95. [103]

    arXiv preprint arXiv:2212.08073 , year=

    Constitutional ai: Harmlessness from ai feedback , author=. arXiv preprint arXiv:2212.08073 , year=

    arXiv

  96. [104]

    The Twelfth International Conference on Learning Representations , year=

    Benchmarking and Improving Generator-Validator Consistency of Language Models , author=. The Twelfth International Conference on Learning Representations , year=

  97. [105]

    Advances in Neural Information Processing Systems , volume=

    Self-refine: Iterative refinement with self-feedback , author=. Advances in Neural Information Processing Systems , volume=

  98. [106]

    and Guo, Zifan Carl and Andreas, Jacob , booktitle =

    Li, Belinda Z. and Guo, Zifan Carl and Andreas, Jacob , booktitle =. (. 2025 , editor =

    2025

  99. [107]

    Thirty-seventh Conference on Neural Information Processing Systems , year =

    Shankar Padmanabhan and Yasumasa Onoe and Michael JQ Zhang and Greg Durrett and Eunsol Choi , title =. Thirty-seventh Conference on Neural Information Processing Systems , year =

  100. [108]

    arXiv preprint arXiv:1610.02424 , year=

    Diverse beam search: Decoding diverse solutions from neural sequence models , author=. arXiv preprint arXiv:1610.02424 , year=

    arXiv

  101. [109]

    International Conference on Learning Representations , year=

    The Curious Case of Neural Text Degeneration , author=. International Conference on Learning Representations , year=

  102. [110]

    arXiv preprint arXiv:2601.13566 , year=

    Self-Improvement as Coherence Optimization: A Theoretical Account , author=. arXiv preprint arXiv:2601.13566 , year=

  103. [111]

    Transformer Circuits Thread , year=

    Lindsey, Jack , title=. Transformer Circuits Thread , year=

  104. [112]

    Pseudo-Label : The Simple and Efficient Semi-Supervised Learning Method for Deep Neural Networks , journal =

    Lee, Dong-Hyun , year =. Pseudo-Label : The Simple and Efficient Semi-Supervised Learning Method for Deep Neural Networks , journal =

  105. [113]

    Reprompting: Automated Chain-of-Thought Prompt Inference Through

    Xu, Weijia and Banburski, Andrzej and Jojic, Nebojsa , booktitle =. Reprompting: Automated Chain-of-Thought Prompt Inference Through. 2024 , editor =

    2024

  106. [114]

    ICCV , year =

    Linfeng Zhang and Jiebo Song and Anni Gao and Jingwei Chen and Chenglong Bao and Kaisheng Ma , title =. ICCV , year =

  107. [115]

    The Journal of Machine Learning Research , volume=

    Posterior regularization for structured latent variable models , author=. The Journal of Machine Learning Research , volume=. 2010 , publisher=

    2010

  108. [116]

    arXiv preprint arXiv:2505.05410 , year=

    Reasoning Models Don't Always Say What They Think , author=. arXiv preprint arXiv:2505.05410 , year=

    arXiv

  109. [117]

    arXiv preprint arXiv:2307.13702 , year=

    Measuring faithfulness in chain-of-thought reasoning , author=. arXiv preprint arXiv:2307.13702 , year=

    arXiv

  110. [118]

    Transactions on Machine Learning Research , issn=

    Does equivariance matter at scale? , author=. Transactions on Machine Learning Research , issn=. 2025 , url=

    2025

  111. [119]

    arXiv preprint arXiv:2408.10646 , year=

    Beneath the surface of consistency: Exploring cross-lingual knowledge representation sharing in llms , author=. arXiv preprint arXiv:2408.10646 , year=

    arXiv

  112. [120]

    First Conference on Language Modeling , year=

    Measuring and Controlling Instruction (In)Stability in Language Model Dialogs , author=. First Conference on Language Modeling , year=

  113. [121]

    L ex S ym: Compositionality as Lexical Symmetry

    Akyurek, Ekin and Andreas, Jacob. L ex S ym: Compositionality as Lexical Symmetry. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023. doi:10.18653/v1/2023.acl-long.38

    2023 doi

  114. [122]

    arXiv preprint arXiv:1707.06347 , year=

    Proximal policy optimization algorithms , author=. arXiv preprint arXiv:1707.06347 , year=

    arXiv

  115. [123]

    Advances in neural information processing systems , volume=

    Direct preference optimization: Your language model is secretly a reward model , author=. Advances in neural information processing systems , volume=

  116. [124]

    arXiv preprint arXiv:2402.03300 , year=

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models , author=. arXiv preprint arXiv:2402.03300 , year=

    arXiv

  117. [125]

    McKeown , title =

    Yanda Chen and Ruiqi Zhong and Narutatsu Ri and Chen Zhao and He He and Jacob Steinhardt and Zhou Yu and Kathleen R. McKeown , title =. Forty-first International Conference on Machine Learning,. 2024 , pages =

    2024

  118. [126]

    2026 , eprint=

    Unsupervised Elicitation of Language Models , author=. 2026 , eprint=

    2026

  119. [127]

    The Eleventh International Conference on Learning Representations,

    Collin Burns and Haotian Ye and Dan Klein and Jacob Steinhardt , title =. The Eleventh International Conference on Learning Representations,. 2023 , publisher =

    2023

  120. [128]

    Second Conference on Language Modeling , year =

    Juan Diego Rodriguez and Wenxuan Ding and Katrin Erk and Greg Durrett , title =. Second Conference on Language Modeling , year =

  121. [129]

    2026 , eprint=

    Counterfactual Simulation Training for Chain-of-Thought Faithfulness , author=. 2026 , eprint=

    2026

  122. [130]

    Emerging Properties in Self-Supervised Vision Transformers , booktitle =

    Mathilde Caron and Hugo Touvron and Ishan Misra and Herv. Emerging Properties in Self-Supervised Vision Transformers , booktitle =. 2021 , pages =. doi:10.1109/ICCV48922.2021.00951 , url =

    2021

  123. [131]

    Self-Distillation Enables Continual Learning , booktitle =

    Idan Shenfeld and Mehul Damani and Jonas H. Self-Distillation Enables Continual Learning , booktitle =. 2026 , url =

    2026

  124. [132]

    Forty-second International Conference on Machine Learning Position Paper Track , year=

    Position: Deep Learning is Not So Mysterious or Different , author=. Forty-second International Conference on Machine Learning Position Paper Track , year=

  125. [133]

    2025 , eprint=

    Bias-Augmented Consistency Training Reduces Biased Reasoning in Chain-of-Thought , author=. 2025 , eprint=

    2025

  126. [134]

    2025 , eprint=

    Consistency Training Helps Stop Sycophancy and Jailbreaks , author=. 2025 , eprint=

    2025

  127. [135]

    ICML Workshop: Reliable and Responsible Foundation Models , year =

    Miles Turpin and Andy Arditi and Marvin Li and Joe Benton and Julian Michael , title =. ICML Workshop: Reliable and Responsible Foundation Models , year =

  128. [136]

    2026 , eprint=

    Consistency Training while Mitigating Obfuscation via Rate Matching , author=. 2026 , eprint=

    2026

  129. [137]

    Forty-third International Conference on Machine Learning , year =

    David Demitri Africa and Arathi Mani , title =. Forty-third International Conference on Machine Learning , year =

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.