The Behavior Gap: Evaluating Zero-shot LLM Agents in Complex Task-Oriented Dialogs

Avinash Baidya; Kamalika Das; Xiang Gao

REVIEW 5 major objections 4 minor 16 references

The Behavior Gap: Evaluating Zero-shot LLM Agents in Complex Task-Oriented Dialogs

T0 review · 5 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Zero-shot LLM agents underperform in complex task-oriented dialogs because of a measurable behavior gap from human experts, and injecting human behavior patterns into prompts improves performance by 24.3 percent on average.

desk verdict A useful diagnostic framework with plausible descriptive findings, but the headline causal claim is unsupported because the intervention never verifies that it actually narrowed the measured behavior gap. read the letter →

arxiv 2506.12266 v1 pith:ISQZZWXX submitted 2025-06-13 cs.CL cs.AIcs.HCcs.LG

Avinash Baidya , Kamalika Das , Xiang Gao This is my paper · ORCID

classification cs.CLcs.AIcs.HCcs.LG

keywords behaviorgapzero-shotLLMagentstask-orienteddialogsystemsactstoolusageexternalknowledgeutilizationteacher-forcingevaluationtaskcomplexity

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to explain why zero-shot LLM agents underperform in task-oriented dialogs, and it attributes the shortfall to a measurable 'behavior gap' between what agents do and what human experts do. The paper reports that on three tasks of increasing complexity, agents diverge from human behavior in dialog acts, tool usage, and use of external knowledge, and that the divergence grows with task complexity (correlation 0.963). The headline result is that closing part of the gap—by injecting known human dialog acts or tool selections into the system prompt—improves turn-level performance by 24.3 percent on average, with the biggest gains on the hardest task. A sympathetic reader would care because it turns a vague complaint about AI agents into a quantifiable, actionable diagnosis: a recipe for evaluating and patching agent behavior without retraining.

What carries the argument

The load-bearing machinery is a teacher-forcing evaluation harness plus a set of GPT-4o-based few-shot classifiers. In teacher forcing, the agent receives the real conversation between a human user and a human expert up to each turn, so the agent's next response can be compared with the expert's actual next response in the identical context. Three classifier families quantify the gap: dialog act classifiers over two taxonomies (the task-specific WOZ acts and the task-independent ISO 24617-2 acts), task-specific tool-usage classifiers, and a response-quality judge scored on coherence, specificity, effectiveness, and satisfaction. A knowledge-usage comparison adds two metrics, ROUGE-1 precision for verbatim copying and a compression ratio for condensation. Task complexity is measured by normalized turn count and dialog act diversity. The pivotal causal step is 'behavior injection'—feeding the agent's system prompt with human dialog acts or tool selections and measuring the performance delta.

What would settle it

Manually label a random sample of PCS turns (say 100) for dialog acts, tools, and copy-paste, then recompute the gap and the 24.3 percent improvement; if cleaner labels shrink the gap to near zero or erase the injection gain, the claim fails—look especially for the PCS tool classifier missing human ScreenShare actions because they are implied rather than announced.

Watch

Extended reading notes

Core claim

The paper's central claim is that the performance gap between zero-shot LLM agents and human experts in task-oriented dialogs is to a significant degree a behavior gap. Measuring the gap as $1-\text{micro-F1}$ alignment between agent and human annotations, they find that even GPT-4o stays far from human patterns on the most complex task—dialog act F1 of 0.464 and tool use F1 of 0.139—and that LLM agents over-call tools and copy retrieved knowledge verbatim rather than condensing it. The same framework shows the gap tracks task complexity with correlation 0.963. The decisive experiment is behavioral intervention: adding known human dialog acts or chosen tools to the system prompt improves response quality by 24.3 percent on average (22.4 percent and 26.3 percent on the complex PCS task for acts and tools respectively), which the paper reads as evidence that the gap is not just correlated with poor performance but limits it.

Load-bearing premise

The entire measurement rests on trusting the GPT-4o classifiers that label dialog acts, tool use, and response quality on the test dialogs, including the private support transcripts where the tool classifier is only 74.8 percent accurate and has no ground truth to check against.

Editorial extensions

If this is right

If the behavior gap is a cause of the performance shortfall, then alignment efforts—prompt injection or training—should yield their largest gains on exactly the most complex tasks, where the gap is widest.
Simple slot-filling benchmarks overstate zero-shot agent readiness; the widening gap with task complexity is a warning for real-world deployment.
Tool-call counts are a poor quality signal on their own: agents invoke tools more often than humans do but with far lower alignment, so tool behavior must be measured against a human benchmark.
External knowledge handling needs a different objective than instruction-following: agents should condense retrieved passages, not copy them, and the paper's metrics provide a way to measure that directly.
Extending the framework to full-dialog evaluation and to newer reasoning models—both explicitly left open by the paper—would test whether the identified gaps persist as agents improve.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the gap is causal rather than merely correlated, fine-tuning an agent on human expert dialogs should recover a large share of the performance deficit; the prompt-injection result makes this a concrete, testable prediction.
Because the PCS tool-use alignment sits near the floor (F1 = 0.139) while the PCS tool classifier reaches only 0.748 micro-F1, part of the measured gap could be annotation noise; a manual audit of sampled PCS turns would show how much of the gap is real.
The framework could serve as a behavior regression test in agent development: track the three gap metrics across model versions to catch dialog-strategy drift before deployment.
The verbatim-copying result suggests that instruction-level prompting (e.g., 'be concise') is too weak a lever; retrieval-augmented agents may need training objectives that explicitly penalize copying content verbatim from retrieved passages.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

You should know two things about this one. First, the evaluation framework is genuinely useful: measuring the gap between zero-shot agents and human experts along dialog acts, tool usage, and knowledge use, in a teacher-forcing setup, is a sensible and well-executed way to diagnose where agents go wrong. The descriptive findings (agents use more dialog acts per turn, over-invoke tools, and copy-paste external knowledge) are plausible and align with prior work. Second, the central causal claim—that reducing the behavior gap improves performance by 24.3%—does not hold up as presented. The intervention in Section 4.2 injects known human acts and tools into the prompt but never checks whether this actually reduced the measured gap. Without that manipulation check, the gain could just as easily come from adding task-relevant instructions or lengthening the prompt. There is no control matched for specificity or length. So the headline result is an unverified bridge.

The correlation of 0.963 is computed from only three task-level points, so it is largely decorative; the turn-level comparison in Figure 6 is more informative, though it is correlational and likely confounded by turn difficulty. The circularity concern is real but not fatal: GPT-4o serves as behavior annotator, performance judge, and dataset filter, yet there is some independent grounding (the evaluator is validated against MultiWOZ success rates, and the dialog act classifiers are checked on benchmark data). Still, the PCS tool classifier at micro-F1 0.748 is weak, and the PCS transcripts themselves are private. The knowledge-use comparison also assumes humans and agents retrieved the same knowledge, which is questionable for the PCS task.

What the paper does well: it proposes a concrete, multi-dimensional behavioral evaluation, uses teacher-forcing to avoid user-simulator artifacts, and is honest about its limitations, including the need for domain-specific classifier validation.

Who is this for? People building TOD or LLM-agent evaluation pipelines will get value from the framework and the descriptive gap analysis. The causal claim should not be cited. The paper deserves a serious referee, but only with the expectation of major revision: either run the intervention with a manipulation check and appropriate controls, or substantially soften the causal interpretation. I would not cite the 24.3% result in its current form.

Referee Report

5 major / 4 minor

Summary. The paper proposes an evaluation framework for quantifying the 'behavior gap' between zero-shot LLM agents and human experts in task-oriented dialogs, measured across three dimensions: dialog acts, tool usage, and external knowledge utilization. The framework is applied to three datasets of increasing complexity (MultiWOZ, SpokenWOZ, and a new private PCS dataset) under a teacher-forcing protocol. The authors report that the behavior gap widens with task complexity (correlation 0.963), that LLM agents overuse tools and copy knowledge verbatim, and that injecting known human dialog acts and tools into system prompts yields a 24.3% average performance improvement, which they attribute to reducing the behavior gap.

Significance. If the paper's claims were established, it would provide a useful diagnostic framework for TODS evaluation and a practical alignment strategy. The authors have put together a substantial evaluation apparatus: a new private real-world dataset (PCS), a teacher-forcing evaluation protocol that controls conversational context, validation of the LLM classifiers against ground-truth benchmarks, and a falsifiable claim that the behavior gap increases with task complexity. However, the headline correlation is based on only three task-level points, the classifiers are applied to an unvalidated private domain, and the causal intervention lacks a manipulation check. These issues currently prevent the paper from supporting its central conclusions.

major comments (5)

[Section 4.1.1, Fig. 2a] The reported 'average correlation: 0.963' is computed over only three task-level data points (one per dataset). With n = 3, this correlation is not statistically reliable, and no confidence interval or significance test is provided. Furthermore, the task complexity metric defined in Section 3.5 includes Dialog Act Diversity, which is measured using the same dialog act dimension as the gap itself, so the correlation may be partly built in by construction rather than reflecting an independent empirical trend.
[Sections 3.3.1, 3.3.2, and 6] The dialog act and tool classifiers are validated on MultiWOZ 2.2 and DialogBank, then applied without further validation to the private PCS transcripts, which the authors themselves describe as significantly different. The PCS tool classifier has micro-F1 0.748, notably lower than the MultiWOZ (0.893) and SpokenWOZ (0.898) tool classifiers. Because PCS is the key dataset supporting the 'widening gap' claim, systematic classifier error on this domain could produce the observed behavior gap as an artifact. The limitation paragraph in Section 6 acknowledges this risk but does not quantify or correct for it.
[Section 4.2, Behavior Intervention] The causal claim that 'reducing such behavior gaps leads to significant performance improvement (24.3% on average)' is not supported by the presented evidence. The intervention injects known human dialog acts and selected tools into the system prompt, but the paper does not measure whether the injected condition actually reduces the measured behavior gap (e.g., micro-F1 alignment in dialog acts or tool usage). The performance gain could equally result from adding task-relevant instructions, increasing prompt length, or providing more specific guidance; none of these is controlled for. A manipulation check and an instruction-matched control condition are required to attribute the improvement to gap reduction.
[Section 3.3.3, External Knowledge Usage] The compression ratio metric assumes that 'human experts retrieved the same external knowledge as LLM agents, so the knowledge length was identical for both.' This assumption is not validated, particularly for the PCS KnowledgeLookup tool, where the retrieval queries and returned knowledge base articles are not controlled between humans and agents. If human experts retrieve different or more focused knowledge, the reported difference in compression ratios would not be interpretable as a behavioral gap.
[Sections 3.3 and 3.4, Fig. 6] The same model family (GPT-4o) is used to annotate dialog acts and tools, to score response quality, and to serve as the strongest agent. The evaluator is validated only against MultiWOZ success rates (Appendix A.6), with no validation for SpokenWOZ or PCS. This shared model family raises the concern that the correlation between behavior gap and performance in Figure 6 partly reflects the evaluator's own priors about what constitutes good behavior, rather than an independent causal relationship.

minor comments (4)

[Section 3.3.1] There is a stray double period after the citation list in the description of the WOZ framework ('...Rastogi et al., 2020)..').
[Appendix A.2] The sentence 'Here we provide details on the the Zero-shot ReAct agent employed in this work' contains a doubled article 'the the'.
[Section 4.2 and Abstract] The text reports PCS improvements of 22.4% and 26.3% for dialog act and tool injection, while the abstract quotes 24.3% on average; the paper should clarify how this average is computed across tasks and metrics.
[Section 3.5] The Normalized Turn Count formula is set as a displayed equation but is not numbered, whereas the Compression Ratio equation in Section 3.3.3 is also displayed without a number; consider numbering equations consistently.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity; central measurements are externally validated and the causal claim rests on an experiment, not a definitional identity.

full rationale

The paper's derivation chain is not circular in the sense defined here. The behavior-gap metrics (dialog-act micro-F1, tool micro-F1, knowledge ROUGE/compression) are measured with GPT-4o-based classifiers, but these classifiers are validated against external ground-truth corpora (MultiWOZ for WOZ acts, DialogBank for ISO acts, and per-task tool annotations), and the performance evaluator is validated against MultiWOZ dialog success rates. Thus, the gap and performance are distinct constructs with independent anchors; they are not defined in terms of each other. The correlation between gap and performance is an empirical association, and the behavior intervention is an experiment that injects human acts/tools into prompts, not a fitting procedure that forces the target result. The main weakness is causal: the paper does not measure whether the prompt injection actually reduced the measured behavior gap, so the 24.3% improvement could be due to added instructions or prompt length. That is a validity limitation, not a circular reduction. The paper itself acknowledges classifier transfer limitations: 'applying them to domains that significantly differ from those benchmarks may require further validation.' Minor self-citations (Zhang et al. 2020, 2022; Xiang Gao is a coauthor) appear only as background support for retrieval/knowledge and dialog evaluation, and are not load-bearing. Overall, no step equates an output to its input by construction.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The framework rests on several unvalidated domain assumptions: classifier transfer, identical knowledge retrieval, and GPT-4o as both annotator and judge. The complexity metric includes hand-chosen constants, and the alignment analysis uses an arbitrary threshold.

free parameters (3)

C in Normalized Turn Count = 1000
Hand-chosen constant in the complexity metric (Section 3.5); changes the relative complexity scores of the three tasks and therefore the reported correlation.
dmax in Dialog Act Diversity = 11
Normalization constant equal to the number of available ISO dialog acts; affects the complexity values used in the correlation.
Alignment threshold F1 >= 0.5 = 0.5
Hand-chosen cutoff for classifying turns as aligned vs misaligned in the performance correlation analysis (Section 4.2).

assumptions (5)

domain assumption Human experts and LLM agents retrieve the same external knowledge in knowledge-gap turns, so knowledge length is identical for both.
Stated in Section 3.3.3 footnote 5; if retrieval differs, the ROUGE and compression comparisons are not apples-to-apples.
domain assumption The GPT-4o dialog act and tool classifiers transfer reliably to the target datasets, including the private PCS transcripts.
Validated on MultiWOZ/DialogBank and LLM traces, but not on the target domains; the PCS tool classifier has micro-F1 only 0.748; acknowledged as a limitation.
domain assumption The GPT-4o response quality evaluator is a valid measure of task performance on all datasets, including PCS where no success-rate ground truth exists.
Validated against MultiWOZ success rates (Figure 8) only; on PCS the same model family supplies both behavior labels and outcome scores.
domain assumption Teacher-forcing, where the agent sees the previous human conversation as context, is a valid controlled setting for behavior comparison.
Explicit methodological choice in Section 3.4; it avoids user simulators but restricts analysis to turn level, as stated in Limitations.
domain assumption The selected dialog act taxonomies (WOZ/ISO) and tool sets are sufficient to capture the relevant behavioral differences.
The taxonomies define what counts as a behavior; unmodeled behavior types are invisible to the gap measurement.

how reviews work

0 comments

Cite this review

Pith. "Pith review of The Behavior Gap: Evaluating Zero-shot LLM Agents in Complex Task-Oriented Dialogs." pith.science (2026). https://pith.science/paper/ISQZZWXX

@misc{pith2026250612266,
  author       = {Pith},
  title        = {Pith review of: The Behavior Gap: Evaluating Zero-shot LLM Agents in Complex Task-Oriented Dialogs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ISQZZWXX}},
  note         = {Machine review of arXiv:2506.12266}
}

read the original abstract

Large Language Model (LLM)-based agents have significantly impacted Task-Oriented Dialog Systems (TODS) but continue to face notable performance challenges, especially in zero-shot scenarios. While prior work has noted this performance gap, the behavioral factors driving the performance gap remain under-explored. This study proposes a comprehensive evaluation framework to quantify the behavior gap between AI agents and human experts, focusing on discrepancies in dialog acts, tool usage, and knowledge utilization. Our findings reveal that this behavior gap is a critical factor negatively impacting the performance of LLM agents. Notably, as task complexity increases, the behavior gap widens (correlation: 0.963), leading to a degradation of agent performance on complex task-oriented dialogs. For the most complex task in our study, even the GPT-4o-based agent exhibits low alignment with human behavior, with low F1 scores for dialog acts (0.464), excessive and often misaligned tool usage with a F1 score of 0.139, and ineffective usage of external knowledge. Reducing such behavior gaps leads to significant performance improvement (24.3% on average). This study highlights the importance of comprehensive behavioral evaluations and improved alignment strategies to enhance the effectiveness of LLM-based TODS in handling complex tasks.

Figures

Figures reproduced from arXiv: 2506.12266 by the authors.

**Figure 1.** Task complexity. Comparison of task complexity across the three tasks measured by dialog act diversity and normalized turn count. Higher values indicate greater task complexity. Error bars denote 95% confidence intervals. 2. Dialog Act Diversity: Measures complexity from the perspective of dialog acts: Dialog Act Diversity = d/dmax, where d is the average number of distinct dialog acts per chat, and dmax is the m… view at source ↗

**Figure 2.** Dialog act gap. Comparing the dialog act selection pattern across the three tasks between the human expert and the three LLM agents. (a) Overall discrepancy, measured as 1 − micro-F1 score, between LLM agents and human experts. Error bars denote 95% confidence intervals. (b) Mean number of dialog acts chosen per turn by agents and human experts. Asterisks denote statistical significance in the differences between… view at source ↗

**Figure 4.** Knowledge usage gap. Comparison of knowledge usage patterns between human experts and three LLM agents across tasks. (a) ROUGE-1 precision shows the extent to which agents copy-paste retrieved knowledge into responses. (b) Compression ratio quantifies how efficiently retrieved knowledge is condensed, with higher values indicating greater compression. Asterisks denote statistical significance in the differences be… view at source ↗

Figures from the paper (4 more)

**Figure 5.** Figure 5: Overall performance. GPT-4o-based agent performance across the four evaluation metrics. Higher score indicated better performance. Error bars denote 95% confidence intervals. some degree of copying may occasionally be appropriate—especially for preserving important de…

**Figure 6.** Figure 6: Correlation of behavior gap with performance. Comparison of performance on turns with aligned (filled bars) versus misaligned (unfilled bars) dialog acts (a) or tool usage (b). The patterned area shows the performance gap. Asterisks denote statistical significance in …

**Figure 7.** Figure 7: Improving performance through behavior injection. Performance improvement (%) when injecting human dialog acts (a) or human-selected tools (b) into system prompts, compared to the no injection baseline. Asterisks denote statistical significance in the performance diff…

**Figure 8.** Figure 8: Relative difference (%) in scores for the four [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

16 extracted references · 13 canonical work pages

[1]

WOZ classifier: For this classifier, we val- idated predictions using the annotated dia- logue acts from the test set of MultiWOZ 2.2 (Budzianowski et al., 2020) dataset

work page 2020
[2]

V ojtˇech Hudeˇcek and Ondˇrej Dušek

Chatgpt for zero-shot dialogue state tracking: A solution or an opportunity? In The 61st Annual Meeting Of The Association For Computational Lin- guistics. V ojtˇech Hudeˇcek and Ondˇrej Dušek. 2023. Are LLMs All You Need for Task-Oriented Dialogue? arXiv preprint. ArXiv:2304.06556 [cs]. Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Rames...

arXiv 2023
[3]

The agent provides information about the restaurant, such as its name, location, cuisine, price, rating, etc

**FindRestaurants**: The agent uses this tool to find restaurants in Cambridge, UK. The agent provides information about the restaurant, such as its name, location, cuisine, price, rating, etc. The agent generally uses this tool to recommend one or more restaurants to the user

work page
[4]

The agent provides information about the booking, such as the restaurant name, reservation date, reservation time, number of guests, price, etc

**BookRestaurant**: The agent uses this tool to book a restaurant in Cambridge, UK. The agent provides information about the booking, such as the restaurant name, reservation date, reservation time, number of guests, price, etc. The agent generally uses this tool to confirm that the booking was successful or not and provide details. ... ... ... ***Few-Sho...

work page
[5]

In Pro- ceedings of the AAAI Conference on Artificial Intelli- gence, volume 38, pages 19368–19376

Zhongjing: Enhancing the chinese medical capabilities of large language model through expert feedback and real-world multi-turn dialogue. In Pro- ceedings of the AAAI Conference on Artificial Intelli- gence, volume 38, pages 19368–19376. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. 2022. React: Synergizing ...

work page 2022
[8]

I recommend

ISO classifier: For this classifier, we val- idated predictions using the DialogBank dataset (Bunt et al., 2019), which contains dialogues from various sources annotated ac- cording to the ISO 24617-2 standards. A.5 Tool Classifier We developed task-specific tool classifiers for each of the three tasks: MultiWOZ, SpokenWOZ, and PCS. Similar to the dialogu...

work page 2021
[9]

The agent provides information about the hotel, such as its name, location, price, rating, etc

**FindHotels**: The agent uses this tool to find hotels in Cambridge, UK. The agent provides information about the hotel, such as its name, location, price, rating, etc. The agent generally uses this tool to recommend one or more hotels to the user

work page
[10]

The agent may provide information about the booking, such as the hotel name, check-in date, check-out date, number of guests, price, etc

**BookHotel**: The agent uses this tool to book a hotel in Cambridge, UK. The agent may provide information about the booking, such as the hotel name, check-in date, check-out date, number of guests, price, etc. The agent generally uses this tool to confirm that the booking was successful or not and provide details

work page

Show all 16 references

[13]

Coherence (Flow and connection with context): Score 1: Completely disconnected from context, ignores previous conversation Score 2: Barely acknowledges context, major inconsistencies Score 3: Basic connection to context, some flow issues Score 4: Good connection to context, mi...
[14]

Specificity (Level of detail and precision): Score 1: Extremely vague, no concrete information Score 2: Minimal details, mostly general statements Score 3: Basic details, some specific information Score 4: Detailed information, most aspects covered Score 5: Comprehensive detai...
[15]

Satisfaction (Likelihood of meeting user needs): Score 1: Completely fails to address user needs Score 2: Minimally addresses user needs, requires multiple follow-ups Score 3: Partially addresses needs, requires some clarification Score 4: Mostly addresses needs, minor clarifi...
[16]

Effectiveness (How well the response addresses the user input): Score 1: Completely misses the point of the input Score 2: Barely addresses the input, major gaps Score 3: Addresses main point but misses details Score 4: Addresses input well with minor omissions Score 5: Perfec...
[2020]

welcome,

MultiWOZ 2.2 : A Dialogue Dataset with Ad- ditional Annotation Corrections and State Tracking Baselines. arXiv preprint. ArXiv:2007.12720 [cs]. Xiaoying Zhang, Baolin Peng, Kun Li, Jingyan Zhou, and Helen Meng. 2023a. Sgp-tod: Building task bots effortlessly via schema-guided ...

2007 arXiv
[2022]

Advances in neural in- formation processing systems, 35:27730–27744

Training language models to follow instruc- tions with human feedback. Advances in neural in- formation processing systems, 35:27730–27744. Chen Qian, Wei Liu, Hongzhang Liu, Nuo Chen, Yufan Dang, Jiahao Li, Cheng Yang, Weize Chen, Yusheng Su, Xin Cong, et al. 2024. Chatdev: C...

2024 arXiv
[2023]

arXiv preprint arXiv:2305.16798

Schema-guided user satisfaction model- ing for task-oriented dialogues. arXiv preprint arXiv:2305.16798. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. 2025. Deepseek-r1: In- centivizing reasoning c...

2025 arXiv
[2024]

Grounding gaps in language model genera- tions. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies (Volume 1: Long Papers), pages 6279–6296. Shuzheng Si, Wentao Ma, Haoyu Gao, Yuch...

2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

WOZ classifier: For this classifier, we val- idated predictions using the annotated dia- logue acts from the test set of MultiWOZ 2.2 (Budzianowski et al., 2020) dataset

work page 2020

[2] [2]

V ojtˇech Hudeˇcek and Ondˇrej Dušek

Chatgpt for zero-shot dialogue state tracking: A solution or an opportunity? In The 61st Annual Meeting Of The Association For Computational Lin- guistics. V ojtˇech Hudeˇcek and Ondˇrej Dušek. 2023. Are LLMs All You Need for Task-Oriented Dialogue? arXiv preprint. ArXiv:2304.06556 [cs]. Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Rames...

arXiv 2023

[3] [3]

The agent provides information about the restaurant, such as its name, location, cuisine, price, rating, etc

**FindRestaurants**: The agent uses this tool to find restaurants in Cambridge, UK. The agent provides information about the restaurant, such as its name, location, cuisine, price, rating, etc. The agent generally uses this tool to recommend one or more restaurants to the user

work page

[4] [4]

The agent provides information about the booking, such as the restaurant name, reservation date, reservation time, number of guests, price, etc

**BookRestaurant**: The agent uses this tool to book a restaurant in Cambridge, UK. The agent provides information about the booking, such as the restaurant name, reservation date, reservation time, number of guests, price, etc. The agent generally uses this tool to confirm that the booking was successful or not and provide details. ... ... ... ***Few-Sho...

work page

[5] [5]

In Pro- ceedings of the AAAI Conference on Artificial Intelli- gence, volume 38, pages 19368–19376

Zhongjing: Enhancing the chinese medical capabilities of large language model through expert feedback and real-world multi-turn dialogue. In Pro- ceedings of the AAAI Conference on Artificial Intelli- gence, volume 38, pages 19368–19376. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. 2022. React: Synergizing ...

work page 2022

[6] [8]

I recommend

ISO classifier: For this classifier, we val- idated predictions using the DialogBank dataset (Bunt et al., 2019), which contains dialogues from various sources annotated ac- cording to the ISO 24617-2 standards. A.5 Tool Classifier We developed task-specific tool classifiers for each of the three tasks: MultiWOZ, SpokenWOZ, and PCS. Similar to the dialogu...

work page 2021

[7] [9]

The agent provides information about the hotel, such as its name, location, price, rating, etc

**FindHotels**: The agent uses this tool to find hotels in Cambridge, UK. The agent provides information about the hotel, such as its name, location, price, rating, etc. The agent generally uses this tool to recommend one or more hotels to the user

work page

[8] [10]

The agent may provide information about the booking, such as the hotel name, check-in date, check-out date, number of guests, price, etc

**BookHotel**: The agent uses this tool to book a hotel in Cambridge, UK. The agent may provide information about the booking, such as the hotel name, check-in date, check-out date, number of guests, price, etc. The agent generally uses this tool to confirm that the booking was successful or not and provide details

work page