Pith. sign in

REVIEW 4 major objections 5 minor 34 references

PSSD: Making Large Language Models Self-denial via Human Psyche Structure

T0 review · 4 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that LLMs improve reasoning accuracy by running an internal self-denial loop—an id role that generates attempts, a superego role that critiques them with pre-summarized rules, and an ego role that executes a corrective…

desk verdict A credible three-role prompting method with consistent but modest gains; the rule component is overhyped and under-tested, but the core idea is sound enough for serious review. read the letter →

arxiv 2502.01344 v1 pith:RL4UEZ6M submitted 2025-02-03 cs.AI cs.CLcs.IR

Jinzhi Liao , Zenghua Liao , Xiang Zhao This is my paper · ORCID
classification cs.AIcs.CLcs.IR
keywords largelanguagemodelsself-denialmulti-agentdebatereasoningcorrectionpsychestructurekeypointsscriptgenerationLoRAfine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that the element missing from current LLM correction methods is self-denial: the model should first decide, with confidence, that an earlier output might be wrong, and then repair it along targeted steps, rather than just regenerate, debate, or call a tool. To operationalize this, the paper builds PSSD, a three-role prompting pipeline modelled on the psychoanalytic division of the mind into id, superego, and ego. An id role writes several intuitive reasoning attempts; a superego role, guided by a pre-summarized rule set, returns key points that criticize those attempts; an ego role converts the key points into an executable script and runs it to give a final answer. The authors report consistent accuracy gains across AdvHotpotQA, 2WikiMultiHopQA, GSM8K, and MATH, with fewer LLM calls than the multi-agent debate baseline, and they show a fine-tuned single-model variant, PSSD-SFT, also beats the fine-tuning competitors. If this is right, internal, rule-guided self-correction is a cheap and transferable way to strengthen LLM reasoning.

What carries the argument

The object that carries the argument is the three-role psyche pipeline, together with the pre-summarized rule set that feeds it. The id role is a simple prompted LLM that emits several reasoning attempts; the superego role is the same LLM prompted to judge those attempts using rules $U=\{u_1,\dots,u_{10}\}$, distilling the judgment into key points; the ego role is the same LLM prompted to write a concrete script $S$, execute it as $S'$, and then answer from $S'$ plus all prior material. The rule set is constructed in a contrastive offline stage: for each training question, GPT-4's high-quality key points are compared with a smaller model's suboptimal key points, the resulting patterns are collected, and GPT-4 summarizes them into ten rules for ten question types. In the single-model variant PSSD-SFT, two LoRA adapters trained on the id task and on the superego/ego task are merged into the base weights, so one forward pass sequence runs the whole self-denial loop.

What would settle it

Take a held-out reasoning set whose question types are absent from the four training sets, and compare PSSD against the paper's own no-rule ablation, i.e. superego without $U$. If the full method is not more accurate than the no-rule version, the rule-based guidance is not what generates the gain.

Watch

Extended reading notes

Core claim

The central claim, stated on the paper's own terms, is that multi-agent reasoning methods work only when each agent is able to deny its own earlier answer in a targeted way. PSSD implements that ability by assigning the three psychoanalytic roles to LLMs in sequence: the intuition-based id role produces $l=5$ initial reasoning paths for a question; the rule-driven superego role uses a pre-extracted rule set $U$ to generate key points that single out flaws in those paths; and the script-centric ego role turns the question, attempts, and key points into a step-by-step script, executes the script, and then synthesizes all of it into the final answer $R$. The rule set is built offline: GPT-4 contrasts high-quality key points with suboptimal ones from a smaller model, extracts patterns from training questions, and summarizes ten rule types that the superego then reuses at inference time. The paper reports that on the four benchmarks PSSD beats CoT with self-consistency, ReAct, VE, and Self-Contrast on almost all metrics, with e.g. EM scores of 47.08 on AdvHotpotQA, 41.96 on 2WikiMultiHopQA, 96.80 on GSM8K, and 77.20 on MATH, and that PSSD-SFT, obtained by merging two LoRA adapters into one open-source model, outperforms the fine-tuning baselines on most of the same tasks.

Load-bearing premise

The load-bearing premise is that the ten-type rule set extracted from the four benchmark training sets keeps paying off on new questions; if those rules are tuned to the distribution of these particular benchmarks, the superego's key-point guidance would add nothing on unseen problem types.

Editorial extensions

If this is right

  • A single rule-guided correction loop can replace resource-heavy debate: PSSD uses three agents and five LLM calls per question, versus 7.8 for Self-Contrast, and PSSD-SFT uses one model and two calls.
  • The superego's rule set is a reusable artifact: once the ten rules are summarized from training data, they are applied to every new question, so the per-question inference cost stays low.
  • Tool-based retrieval is not excluded by the method: adding ReAct or VE on top of PSSD raises exact match further, for example PSSD+ReAct reaches 48.05 on AdvHotpotQA, showing self-denial and external verification are complements.
  • Ablation results imply each role earns its place: removing the superego drops EM by 1.92 on AdvHotpotQA and 2.40 on GSM8K, while removing the ego drops it by 1.63 and 1.20.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • One immediate test the paper leaves open is cross-dataset transfer: running PSSD on a fifth benchmark with the same rule set would show whether the ten rules generalize beyond the four training distributions.
  • The reported PM and RM numbers suggest the bottleneck shifts from generation to selection: the correct answer is often already present among the id's attempts, so the crucial skill is picking and repairing, not creating new candidates.
  • Because PSSD-SFT collapses three roles into one model, an extension would be to train the entire loop on a larger base model or to make the number of correction rounds adaptive, using the ego's confidence to stop early.
  • The confidence analysis shows PSSD can be overconfident on wrong samples; pairing the loop with a cheap external verifier, as the compatibility study starts to do, is a natural place to look for the next gain.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes PSSD, a multi-agent prompting framework that operationalizes Freudian id/superego/ego roles to improve LLM reasoning. The id role generates multiple initial reasoning paths, the superego role uses a pre-distilled rule set U to produce key-point guidance, and the ego role builds and executes a script before giving a final answer. The paper also introduces PSSD-SFT, a LoRA fine-tuned variant that merges the three roles into a single model. Experiments are reported on AdvHotpotQA, 2WikiMultiHopQA, GSM8K, and MATH, comparing against fine-tuning, tool-augmented, and multi-agent debate baselines. The central claims are that PSSD achieves state-of-the-art results on nearly all tasks and metrics, that it is generalizable, and that its role design reduces the number of LLM calls relative to comparable debate methods.

Significance. The three-role framework is a plausible and potentially useful way to structure LLM self-correction, and the paper contains several strengths: the ablation in Table 3 demonstrates that each role contributes incrementally, the PSSD-SFT variant addresses resource efficiency by merging roles into a single LoRA model, and Table 5 quantifies API-call reductions relative to Self-Contrast and other debate methods. If the empirical claims were fully supported, the framework would be a practical contribution to tool-free reasoning enhancement. However, the current evidence does not support the strength of the advertised claims: the rule set U is distilled from the same distributions on which the method is evaluated, no statistical uncertainty is reported, and the main results table contradicts parts of the prose. These issues are load-bearing for the paper's central generalization and state-of-the-art claims.

major comments (4)
  1. [Section 4.2, Table 1] The statement that PSSD 'consistently achieves state-of-the-art results across nearly all tasks in all metrics' is contradicted by Table 1. On 2WikiMultiHopQA, ReAct achieves EM 45.24 and VE achieves EM 42.26, both above PSSD's 41.96; on GSM8K, Self-Contrast achieves PM 86.71 versus PSSD's 84.75; and on 2WikiMultiHopQA, ReAct's RM 14.32 is higher than PSSD's 9.48. The prose should be revised to state the exact per-metric rankings and to acknowledge these exceptions explicitly.
  2. [Section 3.3.1, Eq. (3), Section 4.2] The generalizability claim is not supported by the experimental design because the rule set U is distilled from the training splits of the same four datasets used for evaluation, and the hyperparameters m=3 and n=10 are selected based on experimental results (footnote 2). No experiment builds U on one distribution and evaluates on another, and no out-of-distribution question-type benchmark is used. Since U is inserted into the superego prompt at test time and is credited with providing persuasive references for confident denial, the observed gains could in part reflect matching benchmark-specific question types in U rather than a transferable self-denial mechanism. At minimum, the authors should add a cross-dataset transfer experiment for U and report whether the rule set transfers to held-out question types; the marginal contribution of U in Table 3 (+0.64 EM on AdvHotpotQA, +0.60 on GSM8K) does not remove this concern because the paper explicitly credits U with supplying the references needed for the superego's guidance.
  3. [Section 4.2, Table 6] No confidence intervals, error bars, or significance tests are reported, and the test sets are small (308 examples for AdvHotpotQA and 336 for 2WikiMultiHopQA, per Table 6). Several reported advantages are within a few percentage points, such as PSSD's +2.28 EM over ReAct on AdvHotpotQA and PSSD-SFT's +0.33 EM over AugGPT on AdvHotpotQA in Table 2, so the strength of the 'superior performance' claims exceeds what the experiments can support. The authors should provide repeated-run statistics, bootstrap confidence intervals, or significance tests, and temper the prose accordingly.
  4. [Section 3.3.1, Eqs. (2)-(3)] Equations (2) and (3) are not well-formed. In Eq. (2), P_b is written as an arg max over b while the maximization should be over the pattern set T, and the probability P(T|q_t^b, K_h^b, K_s^b) is never defined. In Eq. (3), U is written as an arg max over n, which is not a meaningful optimization; the intended operation is presumably a clustering or selection over question types. These equations are the formal statement of the rule-construction procedure, so they should be corrected or replaced with a precise algorithmic description.
minor comments (5)
  1. [Abstract and throughout] The manuscript contains numerous grammatical errors and typos, including 'The enhance of accuracy', 'errorneous', 'goolge engine', and 'confidential determination' (likely 'confident determination'). A thorough language edit is needed.
  2. [Section 4.6, Figure 2] The consistency analysis is qualitative and based only on kernel density plots. If this analysis is meant to support the claim that PSSD instills greater confidence, the authors should report a quantitative measure of distribution difference or at least describe the density estimation procedure and sample sizes.
  3. [Table 6] The table formatting makes the training and test sizes ambiguous; for each dataset, the Size, Training, and Test values should be clearly separated in the table cells.
  4. [Appendix A.2] The 200-sample comparison between PSSD and Self-Contrast is reported without any measure of uncertainty. If this comparison is retained, it should be accompanied by a binomial test or a bootstrap confidence interval.
  5. [Section 3.5.2, Eq. (9)] The notation d_<i^b is not defined precisely; please clarify that it denotes the response tokens of the structured record before position i, and state the LoRA rank and other hyperparameters used in fine-tuning.

Circularity Check

1 steps flagged · score 3.0 of 10

One fitted hyperparameter selection on the reported metrics; the core multi-role prompting pipeline is not definitionally circular.

  1. fitted input called prediction [Section 3.3.1, footnote 2; Section 4.2, Table 1]
    "Regarding hyper-parameter m and n, we first establish parameter ranges following related studies [32]. We then refine the values slightly and select the optimal parameters (m = 3 and n = 10) according to experimental results."

    The rule-set summarization (Eq. 3) depends on m (top patterns per question) and n (number of rule types). The authors state these were selected 'according to experimental results' on the same benchmarks whose test metrics are later reported in Table 1 as evidence that 'PSSD consistently achieves state-of-the-art results across nearly all tasks in all metrics, demonstrating the superiority and generalizability of our design.' If 'experimental results' includes the reported test metrics, the configuration was chosen to maximize the very numbers used to validate the method; the reported gains are then partly a product of model selection on the evaluation data rather than an independent prediction of generalizability.

full rationale

PSSD is an empirical multi-agent prompting framework, not a mathematical derivation: the final answer R (Eq. 7) is produced by an LLM (ego role) from q', K, S, and S', none of which is a fitted parameter whose value equals R by construction. The superego rules U are distilled from the training splits of the four datasets (Eqs. 2-3), which is standard supervised use of training data rather than circular; no equation reconstructs the test answer from U alone. The only fitted-component concern is footnote 2, where m and n are selected 'according to experimental results'; this is flagged above as a fitted-input issue because the same experimental results are then presented as evidence of state-of-the-art generalization. The central gains, however, come from the id/superego/ego role structure, and the ablation shows the rule component alone contributes only about 0.6 EM, so the core comparison does not reduce to the fitted hyperparameters. There is no load-bearing self-citation; the Freud citation is an external analogy, and no uniqueness claim is imported from the authors' prior work. The missing cross-dataset transfer test for U is a generalizability limitation, not a circularity. Overall: partial circularity confined to hyperparameter selection, score 3.

Assumptions & free parameters 6 free parameters · 4 assumptions · 3 invented entities

The central claim rests on several assumptions that are not independently verified: the Freudian mapping with no mechanistic support, the transferability of the GPT-4-summarized rules, the quality of the contrastive pattern extraction, and the correctness of the human-annotated training records for PSSD-SFT. The free parameters include the number of initial attempts, rule-related hyperparameters that were tuned on experimental results, and unreported LoRA and sampling settings. The id, superego, and ego roles are conceptual entities with no falsifiable external handle.

free parameters (6)
  • l (number of initial attempts) = 5
    Id role samples l reasoning paths; l=5 used in experiments (Section 3.5.2), no sensitivity analysis reported.
  • m (number of extracted patterns per question) = 3
    Selected according to experimental results (footnote 2, Section 3.3.1); affects rule quality.
  • n (number of question types and rule sets) = 10
    Selected according to experimental results (footnote 2); defines the rule set U used by the superego role.
  • Consistency threshold for external tool trigger = 3
    In the compatibility study, ReAct/VE are invoked when consistency values fall below 3 (Section 4.5); no threshold analysis.
  • z (number of few-shot examples) = not reported
    Id role uses z examples from the training set (Section 3.2); z is not specified.
  • LoRA hyperparameters (rank, alpha, dropout, epochs) = not reported
    Only learning rate, batch size, and max length are reported (Section 4.1.4); the LoRA configuration is underdetermined for replication.
assumptions (4)
  • ad hoc to paper Freudian tripartite psyche (id, ego, superego) is a useful model for LLM self-correction
    The entire framework is motivated by Freud's structural model without empirical evidence that this mapping improves reasoning. Section 1.
  • domain assumption Rules U summarized from training data by GPT-4 generalize to the test distribution
    Superego key point generation relies on pre-summarized rules (Eq. 3, Section 3.3.1); no out-of-distribution evaluation is provided.
  • domain assumption GPT-4 contrastive pattern extraction identifies meaningful distinctions between good and bad key points
    Rule development assumes Eq. 2 captures useful patterns; these patterns are then summarized into rules.
  • domain assumption Manually annotated GPT-4 reasoning records are correct and complete training signals
    PSSD-SFT fine-tuning depends on annotations from three graduate students with consensus review (Section 3.5.1); annotation accuracy is not measured.
invented entities (3)
  • id role
    purpose: Generates multiple initial reasoning attempts via LLM sampling
    A prompt role named after Freud's id; it is a design metaphor with no falsifiable prediction outside the paper's benchmarks.
  • superego role
    purpose: Uses pre-summarized rules to produce key points that criticize initial attempts
    A prompt role whose guidance is derived from benchmark training data; effectiveness is only measured within the PSSD pipeline.
  • ego role
    purpose: Generates an executable script, executes it, and produces the final answer
    A prompt role that combines earlier outputs; no independent evidence for its claimed mediating function.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PSSD: Making Large Language Models Self-denial via Human Psyche Structure." pith.science (2026). https://pith.science/paper/RL4UEZ6M

@misc{pith2026250201344,
  author       = {Pith},
  title        = {Pith review of: PSSD: Making Large Language Models Self-denial via Human Psyche Structure},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RL4UEZ6M}},
  note         = {Machine review of arXiv:2502.01344}
}
read the original abstract

The enhance of accuracy in reasoning results of LLMs arouses the community's interests, wherein pioneering studies investigate post-hoc strategies to rectify potential mistakes. Despite extensive efforts, they are all stuck in a state of resource competition demanding significant time and computing expenses. The cause of the situation lies in the failure of identifying the fundamental feature of the solutions in this line, coined as the self-denial of LLMs. In other words, LLMs should confidently determine the potential existence of mistakes and carefully execute the targeted correction. As the whole procedure conducts within LLMs, supporting and persuasive references are hard to acquire, while the absence of specific steps towards refining hidden mistakes persists even when errors are acknowledged. In response to the challenges, we present PSSD, which refers to and implements the human psyche structure such that three distinct and interconnected roles contribute to human reasoning. Specifically, PSSD leverages the recent multi-agent paradigm, and is further enhanced with three innovatively conceived roles: (1) the intuition-based id role that provides initial attempts based on benign LLMs; (2) the rule-driven superego role that summarizes rules to regulate the above attempts, and returns specific key points as guidance; and (3) the script-centric ego role that absorbs all procedural information to generate executable script for the final answer prediction. Extensive experiments demonstrate that the proposed design not only better enhance reasoning capabilities, but also seamlessly integrate with current models, leading to superior performance.

Figures

Figures reproduced from arXiv: 2502.01344 by the authors.

Figure 1
Figure 1. The framework of PSSD. It is mainly constituted of the id role, the superego role, and the ego role. Here we expect that the discussion of the three roles will enlighten LLMs to reason better. agents’ self-denial results in perpetual resource competition. PSSD tries to tackle the identified challenges via the initial attempts, the regular guidance, and detailed executable steps from tailored roles. 3 Proposed Method… view at source ↗
Figure 2
Figure 2. Sketch of the consistency distribution in different [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 4
Figure 4. We compare the results of the PSSD and Self￾Contrast using two pie charts. It shows PSSD is more accurate and stable than direct Self-Contrast. A.3 Answering Ability Analysis 0 0.1 0.2 0.3 0.4 0.5 0.6 Pure Incorrect Answer Missing Answer Correct Answer Pure Incorrect Answer Missing Answer Correct Answer [PITH_FULL_IMAGE:figures/full_fig_p010_4.png] view at source ↗
Figures from the paper (2 more)
Figure 3
Figure 3. Figure 3: The comparison of different paradigms [PITH_FULL_IMAGE:figures/full_fig_p010_3.png]
Figure 5
Figure 5. Figure 5: Sketch of the answering type distribution between [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

34 extracted references · 21 canonical work pages

  1. [1]

    Shengnan An, Zexiong Ma, Zeqi Lin, Nanning Zheng, Jian-Guang Lou, and Weizhu Chen. 2023. Learning From Mistakes Makes LLM Better Reasoner. CoRR abs/2310.20689 (2023)

    arXiv 2023

  2. [2]

    Kai Chen, Chunwei Wang, Kuo Yang, Jianhua Han, Lanqing Hong, Fei Mi, Hang Xu, Zhengying Liu, Wenyong Huang, Zhenguo Li, Dit-Yan Yeung, Lifeng Shang, Xin Jiang, and Qun Liu. 2023. Gaining Wisdom from Setbacks: Aligning Large Language Models via Mistake Analysis. CoRR abs/2310.10477 (2023)

    arXiv 2023

  3. [3]

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training Verifiers to Solve Math Word Problems. CoRR abs/2110.14168 (2021)

    arXiv 2021

  4. [4]

    Roi Cohen, May Hamri, Mor Geva, and Amir Globerson. 2023. LM vs LM: Detect- ing Factual Errors via Cross Examination. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023. Association for Computational Linguistics, 12621–12640

    work page 2023

  5. [5]

    Haixing Dai, Zhengliang Liu, Wenxiong Liao, Xiaoke Huang, Zihao Wu, Lin Zhao, Wei Liu, Ninghao Liu, Sheng Li, Dajiang Zhu, Hongmin Cai, Quanzheng Li, Dinggang Shen, Tianming Liu, and Xiang Li. 2023. ChatAug: Leveraging ChatGPT for Text Data Augmentation. CoRR abs/2302.13007 (2023)

    arXiv 2023

  6. [6]

    Tenenbaum, and Igor Mor- datch

    Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, and Igor Mor- datch. 2023. Improving Factuality and Reasoning in Language Models through Multiagent Debate. CoRR abs/2305.14325 (2023)

    arXiv 2023

  7. [7]

    Sigmund Freud. 1923. Das Ich und das Es . Gesammelte Werke, Vol. XIX. Interna- tionaler Psychoanalytischer Verlag, Leipzig, Vienna and Zurich

    work page 1923

  8. [8]

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring Mathematical Problem Solving With the MATH Dataset. InProceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks 1, NeurIPS Datasets and Benchmarks 2021, December 2021, virtual

    2021

Show all 34 references
  1. [9]

    Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. 2020. Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reason- ing Steps. In Proceedings of the 28th International Conference on Computational Linguistics, COLING 2020, Barcelona, Spain (Online...

    2020

  2. [10]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-Rank Adaptation of Large Language Models. In The Tenth International Conference on Learning Representa- tions, ICLR 2022, Virtual Event, April 25-29, ...

    2022

  3. [11]

    Larsen, N

    Tiffany A Ito, Jeff T. Larsen, N. Kyle Smith, and John T. Cacioppo. 1998. Negative information weighs more heavily on the brain: the negativity bias in evaluative categorizations. Journal of personality and social psychology 75 4 (1998), 887–900

    1998

  4. [12]

    Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yujiu Yang, Zhaopeng Tu, and Shuming Shi. 2023. Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate. CoRR abs/2305.19118 (2023)

    2023 arXiv

  5. [13]

    Ilya Loshchilov and Frank Hutter. 2019. Decoupled Weight Decay Regulariza- tion. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019 . OpenReview.net

    2019

  6. [14]

    OpenAI. 2023. GPT-4 Technical Report. CoRR abs/2303.08774 (2023)

    2023 arXiv

  7. [15]

    Arkil Patel, Satwik Bhattamishra, and Navin Goyal. 2021. Are NLP Models really able to Solve Simple Math Word Problems?. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT...

    2021

  8. [16]

    Chi, Nathanael Schärli, and Denny Zhou

    Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed H. Chi, Nathanael Schärli, and Denny Zhou. 2023. Large Language Models Can Be Easily Distracted by Irrelevant Context. In International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, ...

    2023

  9. [17]

    Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: language agents with verbal reinforcement learning. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, Neu...

    2023

  10. [18]

    Alon Talmor, Ori Yoran, Ronan Le Bras, Chandra Bhagavatula, Yoav Goldberg, Yejin Choi, and Jonathan Berant. 2021. CommonsenseQA 2.0: Exposing the Limits of AI through Gamification. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks 1, ...

    2021

  11. [19]

    Yongqi Tong, Dawei Li, Sizhe Wang, Yujia Wang, Fei Teng, and Jingbo Shang. 2024. Can LLMs Learn from Previous Mistakes? Investigating LLMs’ Errors to Boost for Reasoning. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long...

    2024

  12. [20]

    Danqing Wang and Lei Li. 2023. Learning from Mistakes via Cooperative Study Assistant for Large Language Models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023. Association for Computational...

    2023

  13. [21]

    Le, Ed H

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. Self-Consistency Improves Chain of Thought Reasoning in Language Models. In The Eleventh International Conference on Learning Representations, ICLR 2023, K...

    2023

  14. [22]

    Chi, Quoc V

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. In NeurIPS

    2022

  15. [23]

    Zeyuan Yang, Peng Li, and Yang Liu. 2023. Failures Pave the Way: Enhancing Large Language Models through Tuning-free Rule Accumulation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023 . Assoc...

    2023

  16. [24]

    Cohen, Ruslan Salakhutdinov, and Christopher D

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Langu...

    2018

  17. [25]

    Narasimhan, and Yuan Cao

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R. Narasimhan, and Yuan Cao. 2023. ReAct: Synergizing Reasoning and Acting in Language Models. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023 . Ope...

    2023

  18. [26]

    Xi Ye and Greg Durrett. 2022. The Unreliability of Explanations in Few-shot Prompting for Textual Reasoning. In NeurIPS

    2022

  19. [27]

    Jiahao Ying, Mingbao Lin, Yixin Cao, Wei Tang, Bo Wang, Qianru Sun, Xuanjing Huang, and Shuicheng Yan. 2024. LLMs-as-Instructors: Learning from Errors Toward Automating Model Improvement. CoRR abs/2407.00497 (2024)

    2024 arXiv

  20. [28]

    Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah D. Goodman. 2022. STaR: Boot- strapping Reasoning With Reasoning. In Advances in Neural Information Process- ing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, Novemb...

    2022

  21. [29]

    Wenqi Zhang, Yongliang Shen, Linjuan Wu, Qiuying Peng, Jun Wang, Yueting Zhuang, and Weiming Lu. 2024. Self-Contrast: Better Reflection Through In- consistent Solving Perspectives. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volu...

    2024

  22. [30]

    Ruochen Zhao, Xingxuan Li, Shafiq Joty, Chengwei Qin, and Lidong Bing. 2023. Verify-and-Edit: A Knowledge-Enhanced Chain-of-Thought Framework. In Pro- ceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2023, Toront...

    2023

  23. [31]

    Le, and Ed H

    Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoc V. Le, and Ed H. Chi. 2023. Least-to-Most Prompting Enables Complex Reasoning in Large Language Models. In The Eleventh International Conference on...

    2023

  24. [32]

    Shuyan Zhou, Li Zhang, Yue Yang, Qing Lyu, Pengcheng Yin, Chris Callison- Burch, and Graham Neubig. 2022. Show Me More Details: Discovering Hierar- chies of Procedures from Semi-structured Web Data. In Proceedings of the 60th Annual Meeting of the Association for Computational...

    2022

  25. [33]

    Zico Kolter, and Matt Fredrikson

    Andy Zou, Zifan Wang, J. Zico Kolter, and Matt Fredrikson. 2023. Univer- sal and Transferable Adversarial Attacks on Aligned Language Models. CoRR abs/2307.15043 (2023). A Complementary Experiments A.1 Resource Analysis A.2 PSSD vs. Self-Contrast PSSD inspires LLMs to engage i...

    2023 arXiv

  26. [2021]

    Association for Computational Linguistics, 2080–2094

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.