Pith. sign in

REVIEW 4 major objections 6 minor 43 references

Commonsense Generation and Evaluation for Dialogue Systems using Large Language Models

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper argues that instruction-tuned large language models can generate commonsense-aware dialogue turns and rank which commonsense relation each turn was built on, with GPT-4 as evaluator outperforming both ACCENT and GPT-3.5.

desk verdict A useful turn-level extension of SODA/ACCENT with clean prompt templates, but the evaluation measures self-consistency rather than commonsense quality, so the headline claim is overstated. read the letter →

arxiv 2506.19483 v1 pith:EXRAVFR5 submitted 2025-06-24 cs.CL

Marcos Estecha-Garitagoitia , Chen Zhang , Mario Rodríguez-Cantelar , Luis Fernando D'Haro This is my paper · ORCID
classification cs.CL
keywords commonsenseknowledgeeventreasoningnaturallanguagegenerationprompt-basedandevaluationopen-domaindialogueslargemodelsATOMICrelationslistwisereranking
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to show that large language models can both generate commonsense-infused dialogue turns and grade whether a generated turn actually reflects a given commonsense relation. If this works, dialogue systems could augment their training data on demand and automatically check the augmented data without hand annotation. Concretely, the paper expands 1,721 dialogue turns from five open-domain datasets along 12 ATOMIC commonsense relations, then asks GPT-3.5 and GPT-4 to rank which relation each generated turn was meant to express. It reports that GPT-4 ranks the correct relation first in 24% of cases (MRR 0.415), beating ACCENT and GPT-3.5, and that GPT-3.5 and GPT-4 generations are judged similar in quality. The authors frame the results as preliminary.

What carries the argument

The mechanism is a pair of prompt templates. The expansion prompt takes a dialogue context plus twelve human-readable definitions of ATOMIC relations and asks the model to output one next utterance per relation. The evaluation prompt, adapted from a listwise ranking prompt template (reference [31]), presents the same twelve definitions plus the generated turns and asks the model to rank the most fitting definition for each turn; ranking quality is then measured with top-k accuracy and MRR. A secondary mechanism is the cross-confusion matrix, which compares which relations each model assigns to turns generated by itself or by the other model, serving as a proxy for generation quality. The twelve ATOMIC relation definitions are the vocabulary that anchors both prompts and defines the task.

What would settle it

Manually annotate the intended ATOMIC relation for each of the 1,721 augmented turns using the same twelve definitions and measure inter-annotator agreement (e.g., Cohen's kappa). If human annotators agree with each other no more often than GPT-4 agrees with the ground-truth labels (24% of turns), the evaluation task is underdetermined and the reported ranking gaps do not measure commonsense generation quality.

Watch

Extended reading notes

Core claim

The central claim is that, with carefully phrased instruction prompts, an off-the-shelf LLM can perform both halves of commonsense data augmentation: generating a plausible next turn conditioned on a specified ATOMIC relation (such as xIntent, oEffect, or IsAfter), and later identifying which relation a generated turn instantiates. The evidence is a ranking experiment in which twelve relation definitions are listed, the model orders them by fit to each expanded turn, and the correct definition's rank is scored with top-k accuracy and MRR. GPT-4 as evaluator reaches top-1 accuracy 0.24 and MRR 0.415 on GPT-4-generated turns, outperforming ACCENT (top-1 0.08) and GPT-3.5. Cross-confusion matrices indicate that GPT-3.5 and GPT-4 generations are roughly interchangeable in judged quality, while the evaluation diagonal is weaker than one would hope because several relation definitions overlap, a point the paper itself raises.

Load-bearing premise

The ranking experiment assumes the twelve ATOMIC relation definitions are unambiguous and mutually exclusive, so that picking the right one measures how well a generated turn instantiates commonsense; the paper's own confusion matrices indicate the definitions overlap, which would cap the achievable accuracy regardless of model skill.

Editorial extensions

If this is right

  • Turn-level augmentation can multiply a small dialogue corpus by 12 dimensions of commonsense, producing responses that are on average 35% longer than the original turns, with no task-specific training.
  • GPT-4 can serve as an automatic evaluator for commonsense attribute fidelity, outperforming ACCENT on top-1 accuracy and MRR, potentially reducing the need for hand annotation in early pipeline checks.
  • GPT-3.5 is a viable cheaper generator, since cross-confusion matrices show its outputs are judged similarly to GPT-4's, so cost-conscious setups could substitute it for generation.
  • Because the same LLM can both generate and rank, the pipeline can be closed-loop: generate with one model, evaluate with the same or another, without training a separate classifier.
  • As the paper itself notes, overlapping relation definitions cap how much ranking accuracy can tell us about commonsense quality; improving the metric will require cleaner relation definitions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the method generalizes, any taxonomy with mutually exclusive utterance classes—dialogue acts, intents, politeness strategies—could be plugged into the same two-prompt pipeline, making the contribution a general mechanism for instruction-based data augmentation rather than a claim about ATOMIC in particular.
  • The 0.24 top-1 ceiling suggests the bottleneck may be the relation definitions rather than the models; a human-annotation study on the same 1,721 turns would separate definitional ambiguity from model error and could provide a gold standard for calibrating the metric.
  • Because GPT-4 expansions were generated for only half the dialogues, cost per dialogue drives sample size; the cross-confusion matrices hint that a cheaper generation model could be substituted, and running the full comparison at scale would test whether that cost-saving trade-off holds.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes a prompt-based method for turn-level dialogue data augmentation conditioned on 12 ATOMIC commonsense relations, and an automatic evaluation framework that asks GPT-3.5 and GPT-4 to rank these relation definitions according to their fit with a generated turn. Using 200 dialogues from five datasets (1,721 turns, each augmented 12 times), the authors report top-k accuracy and MRR for the reranking task, compare with ACCENT, and present confusion matrices. They conclude that the approach effectively harnesses LLMs for commonsense reasoning and evaluation, with GPT-4 the best evaluator and GPT-3.5 a cost-effective generator.

Significance. If the evaluation were valid, the paper would offer a simple and transparent prompt-based pipeline for commonsense-conditioned dialogue augmentation and automatic quality assessment. The strengths are concrete: full prompt templates are provided in Figure 1, the model outputs are cross-evaluated (GPT-3.5 generations scored by GPT-4 and vice versa), and the data span five dialogue datasets. However, the current evaluation is essentially a self-consistency check: the same relation definitions used in generation are used as the ranking options during evaluation, so the reported numbers measure whether an LLM can recover its own generation condition, not whether the generated turns express correct or contextually appropriate commonsense. The paper acknowledges the lack of external validation in Section 5. Consequently, the central claim is not supported by the evidence presented.

major comments (4)
  1. [Section 3.3, Figure 1] The evaluation is circular with respect to the central claim. The generation prompt includes the same 12 relation definitions from Table 1, and the evaluation prompt asks the model to rank exactly those definitions; the ground-truth label is the relation whose definition was inserted during generation. Thus the evaluation measures whether the evaluator can recover the generation condition, not whether the generated turn reflects real commonsense. The paper itself states in Section 5 that manual annotations are future work, confirming that no external ground truth or human validation was used. Because the abstract's claim is about effective commonsense reasoning and evaluation, this circularity is load-bearing.
  2. [Section 4, Table 2] The comparison with ACCENT is not apples-to-apples. ACCENT is designed to output a continuous event-commonsense alignment score for a response, not a ranking over 12 relational categories. The paper does not explain how ACCENT's output was converted into top-k accuracy and MRR. Without specifying this adaptation, the ACCENT baseline rows (e.g., top-1 0.08) are uninterpretable, and the claimed superiority of the GPT-based rerankers is not established.
  3. [Section 3.2, Table 2] No error bars, confidence intervals, or significance tests are reported. The datasets are small (40 dialogues per dataset; only 20 for GPT-4 expansions), and the best top-1 accuracy of 0.24, while above the chance level of 1/12 ≈ 0.083, is not shown to be statistically different from other configurations (e.g., 0.245 vs. 0.21). The absence of variance estimates makes the ranking of methods unreliable.
  4. [Section 4, Figure 2] The paper acknowledges 'an overlapping issue concerning the definitions of the commonsense relations.' This is a problem for the ground-truth labels themselves: if the Table 1 paraphrases are not mutually exclusive, then a 'correct' label is ambiguous, and both the confusion matrices and the top-k numbers are difficult to interpret. The authors should quantify label reliability (e.g., with human agreement on a sample) or restructure the relation set to reduce ambiguity.
minor comments (6)
  1. [Section 5] The conclusion says the approach is 'powered by open-source large language models,' but GPT-3.5 and GPT-4 are proprietary. Please correct this wording, e.g., 'instruction-following large language models.'
  2. [Section 1] There is a typo: 'Aproaches' should be 'Approaches.'
  3. [Section 2] The sentence 'Various works has already addressed the issue' should use the plural verb 'have.'
  4. [Table 2] The table layout is hard to parse because the 'Top-k' and 'MRR' headers repeat and the row for 'One-Shot GPT-3.5 w/Comet' contains only dashes for the GPT-4 column. Please clarify that these evaluations were not run and separate the column groups more clearly.
  5. [Figure 1] The evaluation prompt contains typos: 'reponse' should be 'response,' and 'for each target reponse' appears twice.
  6. [Section 3.2] The phrase '1,721 turns were augmented a total of 12 times each turn' is awkward; suggest 'each of the 1,721 turns was augmented 12 times.'

Circularity Check

1 steps flagged · score 6.0 of 10

Evaluation loop defined by generation condition: quality scores reduce to self-consistency.

  1. self definitional [Section 3.3, Methods and Prompt Design; Figure 1 (right); Section 4, Table 2 and Figure 2]
    "Taking as ground-truths each relation that identifies the expansion of each turn, the GPT models are proposed to sort the definitions of the relations in the form of an ordered list. Thus, the task is to identify the definitions that best fit the expanded response."

    The 'correct' label for each generated response is the very relation definition that was inserted into the generation prompt (Table 1 / Figure 1 left). The evaluator is then asked to rank the same 12 definitions (Figure 1 right) against the response, and top-k accuracy and MRR are computed against that same label. Hence the score measures whether the LLM can recover its own generation condition, not whether the generated utterance expresses correct, contextually appropriate commonsense. The paper itself defers manual annotation to future work in Section 5, confirming that no external ground truth was used.

full rationale

The paper builds no formal derivation chain, but its central quantitative evidence in Table 2 and Figure 2 is produced by a closed evaluation loop. The generation step (Section 3.3) conditions each response on one of 12 ATOMIC relation definitions, and the evaluation step asks an LLM to rank those same 12 definitions for each response, using the generation relation as the ground-truth label. The reported top-k accuracy and MRR therefore measure recoverability of the generation prompt, i.e., self-consistency, not an externally validated commonsense property. The paper itself states in Section 5 that manual annotations are planned for future work, confirming that no independent labels were used. Cross-evaluation between GPT-3.5 and GPT-4 changes the evaluator model but not the label source, so it does not break the loop. There is no load-bearing self-citation, and the ACCENT comparison is a validity concern rather than a circularity issue. The finding is therefore one central self-definitional step: 'quality' is defined, by construction, as agreement with the generation condition, so the abstract's conclusion that the approach is effective for commonsense reasoning and evaluation exceeds what the self-consistency metric can show.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The paper does not invent new entities. It introduces a new dataset and prompt templates, but the core assumptions are the ATOMIC relation definitions, stochastic LLM behavior, and the comparability of ACCENT. The free parameters are the dialogue counts and the relation definition templates, both chosen by hand.

free parameters (3)
  • Number of dialogues per dataset (40, halved to 20 for GPT-4) = 40 and 20
    Chosen by hand for cost reasons (Section 3.2). This affects the comparison between GPT-3.5 and GPT-4 generations.
  • Relation definition paraphrases for ATOMIC (Table 1) = 12 text templates
    The human-readable definitions in Table 1 are the operational definitions of the classes. The paper notes they overlap, which affects the evaluation ceiling.
  • Number of relations (12) = 12
    Selected from ATOMIC's 16+ relations without justification for dropping others (e.g., xReason). This sets the difficulty of the ranking task.
assumptions (3)
  • domain assumption ATOMIC relation definitions, as paraphrased in Table 1, are valid and sufficient to characterize commonsense aspects of dialogue turns.
    The entire generation and evaluation pipeline is built on these definitions. The paper itself notes overlap issues, weakening this axiom.
  • domain assumption The GPT models' generation and ranking are stable and deterministic enough for the reported measurements to be meaningful.
    The paper does not report temperature, number of runs, variance, or any sampling details. LLM outputs are stochastic, so single-run results may not be reliable.
  • ad hoc to paper ACCENT's output can be adapted to a top-k ranking over 12 relations for a fair comparison.
    ACCENT is a continuous alignment score, not a relation classifier. The adaptation is described only as 'implemented to evaluate in an integrated way', which is insufficient.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Commonsense Generation and Evaluation for Dialogue Systems using Large Language Models." pith.science (2026). https://pith.science/paper/EXRAVFR5

@misc{pith2026250619483,
  author       = {Pith},
  title        = {Pith review of: Commonsense Generation and Evaluation for Dialogue Systems using Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EXRAVFR5}},
  note         = {Machine review of arXiv:2506.19483}
}
read the original abstract

This paper provides preliminary results on exploring the task of performing turn-level data augmentation for dialogue system based on different types of commonsense relationships, and the automatic evaluation of the generated synthetic turns. The proposed methodology takes advantage of the extended knowledge and zero-shot capabilities of pretrained Large Language Models (LLMs) to follow instructions, understand contextual information, and their commonsense reasoning capabilities. The approach draws inspiration from methodologies like Chain-of-Thought (CoT), applied more explicitly to the task of prompt-based generation for dialogue-based data augmentation conditioned on commonsense attributes, and the automatic evaluation of the generated dialogues. To assess the effectiveness of the proposed approach, first we extracted 200 randomly selected partial dialogues, from 5 different well-known dialogue datasets, and generate alternative responses conditioned on different event commonsense attributes. This novel dataset allows us to measure the proficiency of LLMs in generating contextually relevant commonsense knowledge, particularly up to 12 different specific ATOMIC [10] database relations. Secondly, we propose an evaluation framework to automatically detect the quality of the generated dataset inspired by the ACCENT [26] metric, which offers a nuanced approach to assess event commonsense. However, our method does not follow ACCENT's complex eventrelation tuple extraction process. Instead, we propose an instruction-based prompt for each commonsense attribute and use state-of-the-art LLMs to automatically detect the original attributes used when creating each augmented turn in the previous step. Preliminary results suggest that our approach effectively harnesses LLMs capabilities for commonsense reasoning and evaluation in dialogue systems.

Figures

Figures reproduced from arXiv: 2506.19483 by the authors.

Figure 1
Figure 1. Expansion (Left) and Evaluation (Right) prompt templates. [PITH_FULL_IMAGE:figures/full_fig_p010_1.png] view at source ↗
Figure 2
Figure 2. Confusion matrices [PITH_FULL_IMAGE:figures/full_fig_p011_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

43 extracted references · 17 canonical work pages

  1. [1]

    Graph of Thoughts: Solving Elaborate Problems with Large Language Models

    Besta, Maciej, et al. “Graph of Thoughts: Solving Elaborate Problems with Large Language Models.” arXiv.Org, 21 Aug. 2023, arxiv.org/abs/2308.09687

    arXiv 2023

  2. [2]

    Bosselut, A.; Rashkin, H.; Sap, M.; Malaviya, C.; Celikyilmaz, A.; and Choi, Y . 2019. Comet: Commonsense Transformers for Automatic Knowledge Graph Construction. arXiv preprint arXiv:1906.05317

    arXiv 2019

  3. [3]

    Muhao Chen, Hongming Zhang, Qiang Ning, Manling Li, Heng Ji, Kathleen McKeown, and Dan Roth. 2021. Event-centric natural language processing. In Proceedings of the 59th An- nual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: Tutorial Abstracts, pages 6–14, Online. Associ...

    work page 2021

  4. [4]

    Choi, Y . 2022. The Curious Case of Commonsense Intelligence. Daedalus, 151(2): 139–155

    work page 2022

  5. [5]

    Ghosal, D.; Shen, S.; Majumder, N.; Mihalcea, R.; and Poria, S. 2022. CICERO: A Dataset for Contextualized Commonsense Inference in Dialogues. arXiv preprint arXiv:2203.13926

    arXiv 2022

  6. [6]

    Ghosal, D.; Hong, P.; Shen, S.; Majumder, N.; Mihalcea, R.; and Poria, S. 2021. CIDER: Commonsense Inference for Dialogue Explanation and Reasoning. arXiv:2106.00510 [cs]. ArXiv: 2106.00510

    arXiv 2021

  7. [7]

    Hao Sun, Zhexin Zhang, Fei Mi, Yasheng Wang, Wei Liu, Jianwei Cui, Bin Wang, Qun Liu, and Minlie Huang. 2022. Moraldial: A framework to train and evaluate moral dialogue sys- tems via constructing moral discussions. arXiv preprint arXiv:2212.10720

    work page Pith review arXiv 2022

  8. [8]

    Howard, J.; and Ruder, S. 2018. Universal language model fine-tuning for text classification. In ACL

    work page 2018

Show all 43 references
  1. [9]

    Large Language Models Are Zero-Shot Rankers for Recommender Sys- tems

    Hou, Yupeng, Junjie Zhang, Zihan Lin, Hongyu Lu, Ruobing Xie, Julian McAuley, and Wayne Xin Zhao. “Large Language Models Are Zero-Shot Rankers for Recommender Sys- tems.” arXiv.org, May 15, 2023. https://arxiv.org/abs/2305.08845

    2023 arXiv

  2. [10]

    Hwang, Chandra Bhagavatula, Ronan Le Bras, Jeff Da, Keisuke Sakaguchi, Antoine Bosselut, and Yejin Choi

    Jena D. Hwang, Chandra Bhagavatula, Ronan Le Bras, Jeff Da, Keisuke Sakaguchi, Antoine Bosselut, and Yejin Choi. 2021. Comet-atomic 2020: On symbolic and neural commonsense knowledge graphs. In AAAI

    2021

  3. [11]

    Hyunwoo Kim, Youngjae Yu, Liwei Jiang, Ximing Lu, Daniel Khashabi, Gunhee Kim, Yejin Choi, and Maarten Sap. 2022. Prosocialdialog: A prosocial backbone for conversational agents. arXiv preprint arXiv:2205.12688

    2022 arXiv

  4. [12]

    Hyunwoo Kim, Jack Hessel, Liwei Jiang, Ximing Lu, Youngjae Yu, Pei Zhou, Ronan Le Bras, Malihe Alikhani, Gunhee Kim, Maarten Sap, et al. 2022. Soda: Million-scale dialogue distillation with social commonsense contextualization. arXiv preprint arXiv:2212.10465 Commonsense Synth...

    2022 arXiv

  5. [13]

    Jan Koco ´n, Igor Cichecki, Oliwier Kaszyca, Mateusz Kochanek, Dominika Szydło, Joanna Baran, Julita Bielaniewicz, Marcin Gruza, Arkadiusz Janz, Kamil Kanclerz, et al. 2023. Chat- gpt: Jack of all trades, master of none. arXiv preprint arXiv:2302.10724

    2023 arXiv

  6. [14]

    Yann LeCun. 2022. A path towards autonomous machine intelligence version 0.9. 2, 2022- 06-27

    2022

  7. [15]

    Douglas B Lenat. 1995. Cyc: A large-scale investment in knowledge infrastructure. Commu- nications of the ACM, 38(11):33–38

    1995

  8. [16]

    L., Kuncoro, A., Hoffmann, J., de Masson d’Autume, C., Blunsom, P., Nematzadeh, A

    Li, X. L., Kuncoro, A., Hoffmann, J., de Masson d’Autume, C., Blunsom, P., Nematzadeh, A. (2022, December). A systematic investigation of commonsense knowledge in large language models. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (...

    2022

  9. [17]

    Hugo Liu and Push Singh. 2004. Conceptnet—a practical commonsense reasoning tool-kit. BT technology journal, 22(4):211–226

    2004

  10. [18]

    Pre-train,prompt,and predict:A sys- tematic survey of prompting methods in natural language processing

    P.Liu, W.Yuan, J.Fu, Z.Jiang, H.Hayashi, and G.Neubig. Pre-train,prompt,and predict:A sys- tematic survey of prompting methods in natural language processing. 2021

    2021

  11. [19]

    John McCarthy and Patrick J Hayes. 1981. Some philosophical problems from the standpoint of artificial intelligence. In Readings in artificial intelligence, pages 431–450. Elsevier

    1981

  12. [20]

    OpenAI. 2022. Introducing chatgpt. https://openai. com/blog/chatgpt

    2022

  13. [21]

    OpenAI. 2023. Gpt-4 technical report. ArXiv, abs/2303.08774

    2023 arXiv

  14. [22]

    Lianhui Qin, Aditya Gupta, Shyam Upadhyay, Luheng He, Yejin Choi, and Manaal Faruqui. 2021. Timedial: Temporal commonsense reasoning in dialog. arXiv preprint arXiv:2106.04571

    2021 arXiv

  15. [23]

    Zhen Qin, Rolf Jagerman, Kai Hui, Honglei Zhuang, Junru Wu, Jiaming Shen, Tianqi Liu, Jialu Liu, Donald Metzler, Xuanhui Wang, and Michael Bendersky. 2023. Large language models are effective text rankers with pairwise ranking prompting. arXiv:2306.17563

    2023 arXiv

  16. [24]

    ”SYNDICOM: Improving Con- versational Commonsense with Error-Injection and Natural Language Feedback.” arXiv preprint arXiv:2309.10015 (2023)

    Richardson, Christopher, Anirudh Sundar, and Larry Heck. ”SYNDICOM: Improving Con- versational Commonsense with Error-Injection and Natural Language Feedback.” arXiv preprint arXiv:2309.10015 (2023)

    2023 arXiv

  17. [25]

    Commonsense Reasoning for Conversational AI: A Survey of the State of the Art

    Richardson, Christopher, and Larry Heck. Commonsense Reasoning for Conversational AI: A Survey of the State of the Art. arXiv:2302.07926, arXiv, 15 Feb. 2023. arXiv.org, https://doi.org/10.48550/arXiv.2302.07926

    2023 doi arXiv

  18. [26]

    Sarik Ghazarian, Yijia Shao, Rujun Han, Aram Galstyan, and Nanyun Peng. 2023. ACCENT: An automatic event commonsense evaluation metric for open-domain dialogue systems. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (V olume 1: Long ...

    2023

  19. [27]

    X. Wang, J. Wei, D. Schuurmans, Q. Le, E. Chi, and D. Zhou. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171, 2022

    2022 arXiv

  20. [28]

    Xueguang Ma, Xinyu Zhang, Ronak Pradeep, and Jimmy Lin. 2023. Zero-shot listwise doc- ument reranking with a large language model. arXiv:2305.02156

    2023 arXiv

  21. [29]

    ”RankVicuna: Zero-Shot List- wise Document Reranking with Open-Source Large Language Models.” arXiv preprint arXiv:2309.15088 (2023)

    Pradeep, Ronak, Sahel Sharifymoghaddam, and Jimmy Lin. ”RankVicuna: Zero-Shot List- wise Document Reranking with Open-Source Large Language Models.” arXiv preprint arXiv:2309.15088 (2023)

    2023 arXiv

  22. [30]

    Robyn Speer, Joshua Chin, and Catherine Havasi. 2017. Conceptnet 5.5: An open multilingual graph of general knowledge. In Proceedings of the AAAI conference on artificial intelligence, volume 31

    2017

  23. [31]

    Weiwei Sun, Lingyong Yan, Xinyu Ma, Pengjie Ren, Dawei Yin, and Zhaochun Ren

  24. [32]

    Yizhong Wang, Hamish Ivison, Pradeep Dasigi, Jack Hessel, Tushar Khot, Khyathi Raghavi Chandu, David Wadden, Kelsey MacMillan, Noah A Smith, Iz Beltagy, et al. 2023. How far can camels go? exploring the state of instruction tuning on open resources. arXiv preprint arXiv:2306.04751

    2023 arXiv

  25. [33]

    J. Wei, X. Wang, D. Schuurmans, M. Bosma, E. Chi, Q. Le, and D. Zhou. Chain of thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.11903, 2022. 10 Estecha-Garitagoitia et al

    2022 arXiv

  26. [34]

    Y . Xie, K. Kawaguchi, Y . Zhao, X. Zhao, M.-Y . Kan, J. He, and Q. Xie. Decomposition enhances reasoning via self-evaluation guided decoding. arXiv preprint arXiv:2305.00633, 2023

    2023 arXiv

  27. [35]

    S. Yao, D. Yu, J. Zhao, I. Shafran, T. L. Griffiths, Y . Cao, and K. Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. arXiv preprint arXiv:2305.10601, 2023

    2023 arXiv

  28. [36]

    Caleb Ziems, Jane A Yu, Yi-Chia Wang, Alon Halevy, and Diyi Yang. 2022. The moral in- tegrity corpus: A benchmark for ethical dialogue systems. arXiv preprint arXiv:2204.03021

    2022 arXiv

  29. [37]

    and Boureau, Y .L., 2018

    Rashkin, H., Smith, E.M., Li, M. and Boureau, Y .L., 2018. Towards empathetic open-domain conversation models: A new benchmark and dataset. arXiv preprint arXiv:1811.00207

    2018 arXiv

  30. [38]

    and Niu, S., 2017

    Li, Y ., Su, H., Shen, X., Li, W., Cao, Z. and Niu, S., 2017. Dailydialog: A manually labelled multi-turn dialogue dataset. arXiv preprint arXiv:1710.03957

    2017 arXiv

  31. [39]

    and Prabhumoye, S., 2020

    Dinan, E., Logacheva, V ., Malykh, V ., Miller, A., Shuster, K., Urbanek, J., Kiela, D., Szlam, A., Serban, I., Lowe, R. and Prabhumoye, S., 2020. The second conversational intelligence challenge (convai2). In The NeurIPS’18 Competition: From Machine Learning to Intelligent Co...

    2020

  32. [40]

    and Weston, J., 2018

    Zhang, S., Dinan, E., Urbanek, J., Szlam, A., Kiela, D. and Weston, J., 2018. Personalizing dialogue agents: I have a dog, do you have pets too?. arXiv preprint arXiv:1801.07243

    2018 arXiv

  33. [41]

    and Hakkani-Tur, D., 2023

    Gopalakrishnan, K., Hedayatnia, B., Chen, Q., Gottardi, A., Kwatra, S., Venkatesh, A., Gabriel, R. and Hakkani-Tur, D., 2023. Topical-chat: Towards knowledge-grounded open- domain conversations. arXiv preprint arXiv:2308.11995

    2023 arXiv

  34. [42]

    Dinan, E., Roller, S., Shuster, K., Fan, A., Auli, M., and Weston, J. 2018. Wizard of Wikipedia: Knowledge-powered conversational agents. arXiv preprint arXiv:1811.01241. Appendix USER: Given the following dialogue context and the {num_relations} common- sense concepts, genera...

    2018 arXiv

  35. [2023]

    arXiv:2304.09542

    Is ChatGPT good at search? Investigating large language models as re-ranking agent. arXiv:2304.09542

    arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.