Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

Few-shot Policy (de)composition in Conversational Question Answering

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Few-shot LLM prompting can decompose policies into logic formulas that determine compliance answers.

desk verdict Solid neuro-symbolic pipeline on ShARC that mostly delivers on its promises, but the abstract oversells the no-fine-tuning claim. read the letter →

arxiv 2501.11335 v1 pith:NCI4DB35 submitted 2025-01-20 cs.CL cs.AI

Kyle Erwin , Guy Axelrod , Maria Chang , Achille Fokoue , Maxwell Crouse , Soham Dan , Tian Gao , Rosario Uceda-Sosa
show 3 more authors
Ndivhuwo Makondo Naweed Khan Alexander Gray
This is my paper · ORCID
classification cs.CLcs.AI MSC 68T5003B50
keywords policycompliancedetectionconversationalmachinereadingfew-shotin-contextlearninglogicformulationthree-valuedShARCdatasetneuro-symbolicreasoningself-consistency
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces Logical Decomposition for Policy Compliance (LDPC), a neuro-symbolic pipeline for conversational policy compliance detection that turns a written policy into explicit logical form before answering. An LLM prompted with 20 in-context examples decomposes the policy into yes/no sub-questions and composes those questions into a propositional formula, and a three-valued logic with Maybe is evaluated over truth values assigned by a natural-language-inference model, so the final yes/no/follow-up answer is computed deterministically rather than generated opaquely. On the ShARC benchmark the pipeline reaches 79.0% micro accuracy on the development set, within 1.5 points of the best trained system, while the decomposition and logic-formulation stages receive no task-specific fine-tuning. The explicit logic graphs also allow the authors to trace errors, and they report that roughly 36% of inspected errors are borderline cases where the reference answer relies on unsupported assumptions.

What carries the argument

The load-bearing object is a propositional formula over question variables, produced by an LLM in two prompted steps: first decomposing the policy into yes/no questions, then combining them into a Python boolean expression. The paper stabilizes formula selection with self-consistency, sampling several formulas, grouping logically equivalent ones, and returning a shortest member of the largest equivalence class. Evaluation uses Kleene's strong three-valued logic on values {False, Maybe, True}, where conjunction is min, disjunction is max, negation maps True to False and False to True and leaves Maybe fixed, and an outcome of Maybe triggers a follow-up question. Each variable's truth value comes either from the chat history or from a RoBERTa-large NLI model fine-tuned on ShARC entailment data, mapping 'entails', 'contradiction', and 'neutral' to True, False, and Maybe.

What would settle it

Run the full pipeline on the ShARC development set with the NLI module fine-tuned only on MNLI and not on ShARC entailment data, and compare end-to-end accuracy; if accuracy stays at roughly the reported 79%, the few-shot decomposition is carrying the result, while a large drop would show the ShARC-specific NLI fine-tuning is the actual source of the competitive numbers.

Watch

Extended reading notes

Core claim

The central claim is that the reasoning needed for policy compliance conversations can be made explicit and still work well: decompose the policy into basic questions, combine those questions into a logical expression, and evaluate the expression with Kleene's strong three-valued logic, where Maybe signals that a follow-up question must be asked. The paper demonstrates this on ShARC, achieving competitive accuracy against systems trained on thousands of examples while using only 20 in-context examples for the decomposition and logic steps. It further claims that the transparent architecture is a diagnostic tool, showing that a large share of its residual errors on ShARC are cases where the dataset's reference answer makes assumptions the policy text does not support.

Load-bearing premise

The load-bearing premise is that the fine-tuned natural-language-inference model assigns correct truth values to the generated sub-questions from the user's scenario; if that judgment is wrong, the deterministic logic evaluation propagates the wrong value into the final answer.

Editorial extensions

If this is right

  • Policy compliance answers become auditable: every final yes/no/follow-up decision can be traced to a logical formula and to the truth value assigned to each sub-question.
  • The framework is portable to new policies without retraining, since the decomposition and logic-formation stages are few-shot prompted rather than fine-tuned on the target dataset.
  • Error localization is direct: a wrong final answer can be attributed to one of four stages, which the paper uses to show that logic formulation and question answering are the dominant failure points.
  • The Maybe outcome provides a principled trigger for asking a follow-up question whenever the formula is undecided under the current information.
  • The ambiguity findings imply that benchmark scores on ShARC should be interpreted cautiously, and that reference answers may encode a hidden closed-world assumption.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension is to move from propositional formulas to deontic or temporal policy logic, which would let the system represent obligations, prohibitions, deadlines, and exceptions that a boolean expression cannot capture.
  • A testable generalization claim the paper does not make: the same few-shot decomposition should work on out-of-domain policies from a different regulatory domain, requiring only new in-context examples.
  • Since the error analysis points to the NLI module as a near-tie bottleneck with logic formulation, replacing it with a stronger or uncertainty-calibrated entailment model is a concrete path to higher end-to-end accuracy.
  • The closed-world versus open-world assumption is the deepest unresolved issue: the system asks a follow-up when evidence is missing, whereas many ShARC reference answers assume missing conditions are satisfied, and any deployment would need to choose one convention explicitly.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes LDPC, a neuro-symbolic pipeline for conversational policy compliance detection on the ShARC benchmark. An LLM, prompted with 20 in-context examples, decomposes a policy into yes/no sub-questions and then composes those questions into propositional logical formulas; a fine-tuned NLI model assigns each sub-question a truth value (True/False/Maybe) from the user scenario; and a deterministic Kleene three-valued logic evaluates the formula to produce Yes, No, Irrelevant, or a follow-up question. The authors report near-state-of-the-art micro accuracy on the ShARC dev set (79.0) and competitive macro accuracy, with a noticeable drop on the test set (70.2 micro). They also present an error analysis attributing failures to the logic formulation, question answering, decomposition, and relevance components, and claim that 36% of inspected errors are ambiguous reference-answer cases. The central advertised claim is that the approach achieves competitive ShARC performance with 'no task-specific finetuning.'

Significance. If the claims were fully supported, the paper would make a useful contribution: it combines few-shot LLM decomposition with a transparent, deterministic logical evaluator, and it explicitly separates the interpretable symbolic decision step from the neural component that assigns truth values. The use of logical equivalence classes for self-consistency, the model-selection protocol on QA4PC, and the component-level error analysis are all strengths. However, the headline 'no task-specific finetuning' claim is not literally supported, because the QA module is fine-tuned on ShARC-derived NLI data and every sub-question truth value flows through that module. The paper also contains a direct contradiction about the provenance of the in-context examples, and the self-consistency sample size is unreported. These issues are fixable with revised claims, clarified experimental setup, and additional analysis, so the work is worth further consideration rather than rejection.

major comments (4)
  1. [Abstract, §4.5, §5.4] The claim of 'no task-specific finetuning' is contradicted by the QA module. Section 4.5 fine-tunes RoBERTa-large on MNLI and then 'further fine-tune[s] on the ShARC NLI data,' and Algorithm 1 uses Q(s, question) to assign the truth value of every generated sub-question. Since the deterministic logic evaluator can only be as correct as these assignments, and §5.4 attributes 28% of inspected errors to this QA step, the headline result depends on ShARC-derived training. The Limitations section's narrower phrasing ('without ShARC-specific fine-tuning for (de)composition') is accurate, but the abstract and contribution 2 should be revised to match it, or the QA module should be replaced or ablated to substantiate the stronger claim.
  2. [§4.2, Appendix Table 5, Table 2] The provenance of the in-context examples is stated inconsistently: §4.2 says they are 'taken from the QA4PC dev set,' while Appendix Table 5 identifies the tree IDs as coming from the ShARC train set. This matters for evaluating the dev result in Table 2: if the QA4PC dev examples overlap with ShARC dev utterances, the 79.0 dev micro accuracy is inflated by leakage, and the drop to 70.2 on the test set would be partly explained. Please clarify the exact provenance of the 20 examples, report whether any overlap exists between those examples and the ShARC dev/test sets, and if necessary report test-only results or exclude overlapping examples.
  3. [§4.3, §5.3, Table 4] The self-consistency sample size k is never specified for the reported ShARC results. Section 4.3 describes sampling 'k alternative logical forms,' but Table 4 only reports behavior for sample size 3. Without k, the equivalence-class grouping procedure, and the tie-breaking rule, the numbers in Tables 2 and 3 are not reproducible. Please report k for all runs, state whether k was tuned, and include a sensitivity analysis over k.
  4. [§5.4] The claim that 36% of inspected errors are 'borderline cases that, although different from the expected answer in the ShARC dataset, could be considered correct' is based on a single pass of manual inspection of 160 errors, with no stated annotation protocol or inter-annotator agreement. Since this is the central evidence for the paper's dataset-ambiguity contribution, the authors should provide the criteria used, have at least two annotators label the sample, and report agreement.
minor comments (6)
  1. [Section 3] The notation in Section 3 is malformed in places, e.g., 'P = {pi}D i=1' and 'C = {(qi, si, hi)}D i=1'; these should be written as indexed sets or cleaned up.
  2. [Table 2, §5.3] The text says macro accuracy on the dev set dropped from 79.9 to 72.8, but Table 2 lists the dev macro score as 79.7; please reconcile the two numbers.
  3. [Table 3] In Table 3, the BERTQA dev row reports BLEU-1 = 47.4 and BLEU-4 = 54.0; it is unusual for BLEU-4 to exceed BLEU-1, so please verify these values and the evaluation script.
  4. [§4.3] The logic formulation prompt asks for a 'python boolean expression,' but the illustrative formulas use symbols such as '¬' and '∧'; please clarify how these are converted into valid Python expressions and whether the LLM is expected to output Python syntax.
  5. [Algorithm 1] Algorithm 1 deletes dictionary entries with 'del questions[ID]' and 'del answers[ID]' while iterating over 'questions'; in Python this would raise a RuntimeError. If this is intended as pseudocode, say so, or show the intended behavior explicitly.
  6. [§4.1 and Algorithm 1] Section 4.1 says the relevance threshold is 'predetermined from a sensitivity analysis,' but Algorithm 1 hardcodes a value of 0.25; the sensitivity analysis should be reported or referenced so the reader can assess how this free parameter was chosen.

Circularity Check

1 steps flagged · score 3.0 of 10

Advertised 'no task-specific finetuning' is undercut by a ShARC-NLI-finetuned QA module through which every final prediction flows; the central decomposition/logic chain is otherwise independent.

  1. fitted input called prediction [Abstract vs. Section 4.5 (Question Answering) and Algorithm 1 (line 11)]
    "We apply this approach to the popular PCD and conversational machine reading benchmark, ShARC, and show competitive performance with no task-specific finetuning. ... We use a RoBERTa-large model fine-tuned on the MNLI corpus and further fine-tune on the ShARC NLI data."

    The abstract's load-bearing claim of 'no task-specific finetuning' is contradicted by the QA component: RoBERTa is fine-tuned on ShARC NLI, i.e., data derived from the benchmark being evaluated. Algorithm 1 assigns every sub-question's truth value via 'answers[ID] ← Q(s,question)', where Q is this fine-tuned model, and the deterministic Kleene-logic evaluation then converts those truth values into the final Yes/No/follow-up prediction. Thus the final ShARC result is not produced by a fully few-shot, untrained pipeline; it depends on a task-fitted model at the step where all sub-question truth values are set.

full rationale

The paper's central derivation—LLM decomposes policy into yes/no questions, LLM forms a logical formula, and a deterministic Kleene-logic evaluator maps truth values to Yes/No/follow-up—is self-contained and not circular: the decomposition and logic-formulation modules are prompted rather than trained on ShARC labels, and the final ShARC test evaluation is an external benchmark. No self-citation chain or imported uniqueness theorem is load-bearing. However, there is one material circularity-adjacent defect. The abstract's headline 'no task-specific finetuning' is contradicted by Section 4.5, where the QA model is 'further fine-tune[d] on the ShARC NLI data.' Since Algorithm 1 assigns every sub-question's truth value via Q(s,question), and the final answer is the deterministic evaluation of those values, the benchmark result is not produced by an untuned pipeline; it inherits a component fitted to data derived from ShARC. This is an overstated-independence issue rather than a by-construction reduction, so the score is moderate. The use of QA4PC-derived in-context examples and QA4PC-based model selection further ties the prompt design to the ShARC family, though this is standard practice and not itself circular. The Limitations section correctly narrows the claim to 'without ShARC-specific fine-tuning for (de)composition,' confirming that the abstract overstates the result.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The approach imports standard semantic machinery (Kleene logic, NLI entailment) and relies on a small set of hand-picked examples and thresholds. The central pipeline adds no new entities, but several free parameters are tuned without full disclosure.

free parameters (5)
  • Relevance threshold = 0.25
    Section 4.1 and Algorithm 1 line 2; 'predetermined from a sensitivity analysis', but the analysis is not shown.
  • Self-consistency sample size = 3 (used in Table 4, not stated in Algorithm 1)
    Section 4.3 samples k logical forms; Table 4 reports sample size 3, but the algorithm leaves sampleSize unspecified.
  • Number of in-context examples = 20
    Section 4.2 uses 20 in-context examples; model choice experiments vary k from 0 to 20, and 20 is chosen for the final pipeline.
  • Question-filter trigger size = 5
    Algorithm 1 line 14 filters generated questions only when |questions| >= 5; no ablation is provided.
  • QA fine-tuning hyperparameters = 5 epochs, batch 32, lr 1e-5, weight decay 0.1
    Section 4.5; training choices for the RoBERTa NLI model, which is the component fitted to ShARC-derived data.
assumptions (5)
  • standard math Kleene strong three-valued logic semantics for conjunction, disjunction, and negation
    Section 4.4 defines the truth tables and uses them for all evaluation; this is a standard formal system.
  • domain assumption LLM-generated Python boolean expressions are syntactically valid and semantically faithful to the policy
    Section 4.3 prompts the model to 'combine the question variables into a python boolean expression'; the paper does not provide a parser or failure-rate analysis for invalid expressions.
  • domain assumption NLI entailment labels (entails, contradiction, neutral) correspond to True, False, and Maybe for generated sub-questions
    Section 4.5 maps RoBERTa-MNLI-ShARC outputs to truth values; if the scenario does not mention a condition, neutral is treated as unknown, which is a modeling choice.
  • domain assumption The 20 in-context examples sampled from QA4PC dev set generalize to unseen ShARC policies
    Section 4.2 selects in-context examples from QA4PC; the paper shows decomposition+logic accuracy of only 40-45% on QA4PC, so generalization to ShARC is not guaranteed.
  • domain assumption ShARC reference answers are correct enough to score the system
    Section 5.4 estimates 36% of inspected errors are borderline because references rely on unstated assumptions; the paper's own analysis partially undermines this assumption.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Few-shot Policy (de)composition in Conversational Question Answering." pith.science (2026). https://pith.science/paper/NCI4DB35

@misc{pith2026250111335,
  author       = {Pith},
  title        = {Pith review of: Few-shot Policy (de)composition in Conversational Question Answering},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NCI4DB35}},
  note         = {Machine review of arXiv:2501.11335}
}
read the original abstract

The task of policy compliance detection (PCD) is to determine if a scenario is in compliance with respect to a set of written policies. In a conversational setting, the results of PCD can indicate if clarifying questions must be asked to determine compliance status. Existing approaches usually claim to have reasoning capabilities that are latent or require a large amount of annotated data. In this work, we propose logical decomposition for policy compliance (LDPC): a neuro-symbolic framework to detect policy compliance using large language models (LLMs) in a few-shot setting. By selecting only a few exemplars alongside recently developed prompting techniques, we demonstrate that our approach soundly reasons about policy compliance conversations by extracting sub-questions to be answered, assigning truth values from contextual information, and explicitly producing a set of logic statements from the given policies. The formulation of explicit logic graphs can in turn help answer PCDrelated questions with increased transparency and explainability. We apply this approach to the popular PCD and conversational machine reading benchmark, ShARC, and show competitive performance with no task-specific finetuning. We also leverage the inherently interpretable architecture of LDPC to understand where errors occur, revealing ambiguities in the ShARC dataset and highlighting the challenges involved with reasoning for conversational question answering.

Figures

Figures reproduced from arXiv: 2501.11335 by the authors.

Figure 1
Figure 1. Policy decomposition and logical formulation [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Logic formulation given questions models to perform. Llama3-8B, llama3-70B and codellama performed similarly around 40% to 45%, with codellama performing slightly better as the number of prompt examples increase. Mixtral per￾formed worse than the other models and performed worse as the number of in-context examples in￾creased [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Question decomposition and logic formulation [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. A Survey of the State-of-the-Art in Conversational Question Answering Systems

    cs.CL 2025-09 conditional novelty 2.0 of 10

    A review that categorizes ConvQA components, techniques, models, and datasets, with no new experimental result.

Reference graph

Works this paper leans on

30 extracted references · 15 canonical work pages · cited by 1 Pith paper

  1. [1]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    https://ai.meta.com/blog/meta-llama-3/

    Introducing Meta Llama 3: The most capable openly available LLM to date. https://ai.meta.com/blog/meta-llama-3/

    work page

  4. [4]

    Merrie Bergmann. 2008. Three-Valued Propositional Logics: Semantics, page 71–99. Cambridge University Press

    work page 2008

  5. [5]

    Yifan Gao, Chien-Sheng Wu, Jingjing Li, Shafiq Joty, Steven CH Hoi, Caiming Xiong, Irwin King, and Michael Lyu. 2020. Discern: Discourse-aware entailment reasoning network for conversational machine reading. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2439--2449

    work page 2020

  6. [6]

    Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12):1--38

    2023

  7. [7]

    Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guillaume Lample, L \'e lio Renard Lavaud, Lucile Saulnier, Marie-Anne Lachaux, Pierre Stock, Sandeep Subramanian, Sophia Yang, Szymon Antoniak, T...

    arXiv doi:10.48550/arxiv.2401.04088 2024

  8. [8]

    S. C. Kleene. 1938. http://www.jstor.org/stable/2267778 On notation for ordinal numbers . The Journal of Symbolic Logic, 3(4):150--155

    work page arXiv 1938

Show all 30 references
  1. [9]

    Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. https://api.semanticscholar.org/CorpusID:249017743 Large language models are zero-shot reasoners . ArXiv, abs/2205.11916

    2022 arXiv

  2. [10]

    Neema Kotonya, Andreas Vlachos, Majid Yazdani, Lambert Mathias, and Marzieh Saeidi. 2022. Policy compliance detection via expression tree inference. arXiv preprint arXiv:2205.12259

    2022 arXiv

  3. [11]

    Carolin Lawrence, Bhushan Kotnis, and Mathias Niepert. 2019. https://doi.org/10.18653/v1/D19-1001 Attending to future tokens for bidirectional sequence generation . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th Internatio...

    2019 doi

  4. [12]

    Arindam Mitra, Luciano Del Corro, Shweti Mahajan, Andres Codas, Clarisse Simoes, Sahaj Agarwal, Xuxi Chen, Anastasia Razdaibiedina, Erik Jones, Kriti Aggarwal, et al. 2023. Orca 2: Teaching small language models how to reason. arXiv preprint arXiv:2311.11045

    2023 arXiv

  5. [13]

    Siru Ouyang, Zhuosheng Zhang, and Hai Zhao. 2020. Dialogue graph modeling for conversational machine reading. arXiv preprint arXiv:2012.14827

    2020 arXiv

  6. [14]

    Liangming Pan, Michael Saxon, Wenda Xu, Deepak Nathani, Xinyi Wang, and William Yang Wang. 2023. Automatically correcting large language models: Surveying the landscape of diverse self-correction strategies. arXiv preprint arXiv:2308.03188

    2023 arXiv

  7. [15]

    Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. Squad: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2383--2392

    2016

  8. [16]

    Baptiste Rozi \`e re, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, J \'e r \'e my Rapin, Artyom Kozhevnikov, Ivan Evtimov, Joanna Bitton, Manish Bhatt, Cristian Canton Ferrer, Aaron Grattafiori, ...

    2024 doi arXiv

  9. [17]

    Marzieh Saeidi, Max Bartolo, Patrick Lewis, Sameer Singh, Tim Rockt \"a schel, Mike Sheldon, Guillaume Bouchard, and Sebastian Riedel. 2018. https://doi.org/10.18653/v1/D18-1233 Interpretation of natural language rules in conversational machine reading . In Proceedings of the ...

    2018 doi

  10. [18]

    Marzieh Saeidi, Majid Yazdani, and Andreas Vlachos. 2021. Cross-policy compliance detection via question answering. arXiv preprint arXiv:2109.03731

    2021 arXiv

  11. [19]

    Haitian Sun, William W Cohen, and Ruslan Salakhutdinov. 2022. Scenario-based question answering with interacting contextual properties. In The Eleventh International Conference on Learning Representations

    2022

  12. [20]

    Miles Turpin, Julian Michael, Ethan Perez, and Samuel Bowman. 2024. Language models don't always say what they think: unfaithful explanations in chain-of-thought prompting. Advances in Neural Information Processing Systems, 36

    2024

  13. [21]

    Karthik Valmeekam, Alberto Olmo, Sarath Sreedharan, and Subbarao Kambhampati. 2022. https://openreview.net/forum?id=wUU-7XTL5XO Large language models still can't plan (a benchmark for LLM s on planning and reasoning about change) . In NeurIPS 2022 Foundation Models for Decisio...

    2022

  14. [22]

    Nikhil Verma, Abhishek Sharma, Dhiraj Madan, Danish Contractor, Harshit Kumar, and Sachindra Joshi. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.589 Neural conversational QA : Learning to reason vs exploiting patterns . In Proceedings of the 2020 Conference on Empirical M...

    2020 doi

  15. [23]

    Lei Wang, Wanyu Xu, Yihuai Lan, Zhiqiang Hu, Yunshi Lan, Roy Ka-Wei Lee, and Ee-Peng Lim. 2023. https://api.semanticscholar.org/CorpusID:258558102 Plan-and-solve prompting: Improving zero-shot chain-of-thought reasoning by large language models . In Annual Meeting of the Assoc...

    2023

  16. [24]

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2022. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations

    2022

  17. [25]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824--24837

    2022

  18. [26]

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2024. Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems, 36

    2024

  19. [27]

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. 2022. React: Synergizing reasoning and acting in language models. In The Eleventh International Conference on Learning Representations

    2022

  20. [28]

    Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah Goodman. 2022. Star: Bootstrapping reasoning with reasoning. Advances in Neural Information Processing Systems, 35:15476--15488

    2022

  21. [29]

    Xiao Zhang, Heyan Huang, Zewen Chi, and Xian-Ling Mao. 2022. Et5: A novel end-to-end framework for conversational machine reading comprehension. arXiv preprint arXiv:2209.11484

    2022 arXiv

  22. [30]

    Victor Zhong and Luke Zettlemoyer. 2019. https://doi.org/10.18653/v1/P19-1223 E 3: Entailment-driven extracting and editing for conversational machine reading . In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2310--2320

    2019 doi

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.