Pith. sign in

REVIEW 3 major objections 6 minor 46 references

Few-Shot Query Intent Detection via Relation-Aware Prompt Learning

T0 review · 3 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that pretraining a language model on the relational structure of chat sessions—both how users refine queries and how assistants answer them—as soft prompts, then fine-tuning with intent-specific relation prompts, substanti

desk verdict A real new idea in few-shot intent detection with consistent gains, but the missing decontamination check and absent error bars mean the headline numbers need scrutiny before trusting them. read the letter →

arxiv 2509.05635 v1 pith:J6NG5KTL submitted 2025-09-06 cs.CL cs.IR

Liang Zhang , Yuan Li , Shijie Zhang , Zheng Zhang , Xitong Li This is my paper
classification cs.CLcs.IR
keywords few-shotintentdetectionstructure-awarepretrainingrelation-awaresoftpromptsquery-queryrelationquery-answerpromptlearningquery-adaptiveattentionclassification
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to show that the structure of a conversation—how a user's follow-up queries refine an earlier one, and how the assistant answers each query—carries signal that few-shot intent detectors are throwing away. It introduces SAID, which pretrains a language model on unlabeled chat logs by inserting learnable relation tokens between query-query and query-answer pairs and training with a masked-language objective. At fine-tuning time, the task is reformulated as building a prompt that ties the query to each intent name through intent-specific relation tokens. The claim is that this transfer lets small models like BERT outperform several specialized few-shot intent detectors by double-digit margins and even rival zero-shot LLMs dozens of times larger. A reader should care because the method targets a practical bottleneck: intent detection in production systems where labeled examples are scarce but raw chat logs are abundant.

What carries the argument

The central object is the relation-aware soft prompt: a small set of learnable embedding vectors inserted between two texts to mark a relation, such as query-query, query-answer, or query-intent. In pretraining, these prompts feed a structure-aware masked language modeling objective that learns shared knowledge across relations. In fine-tuning, an intent-specific relation prompt is built by inserting new relation tokens between the query and each intent name. QueryAdapt is the mechanism that generates those intent-specific tokens as a query-dependent combination of the pretrained relation tokens, so the model can explicitly transfer the two relational perspectives into intent classification.

What would settle it

Pretrain SAID on the same chat logs but with sessions shuffled so that query-query pairs are arbitrary, while keeping the query-answer pairs intact; if accuracy over text-only pretraining persists, the relational structure is not the cause of the gains. Complementarily, learn intent-specific relation tokens freely with full supervision and check whether they lie in the span of the query-query and query-answer tokens; if they do not, QueryAdapt's linear span assumption is violated.

Watch

Extended reading notes

Core claim

SAID establishes that conversational structure can be turned into pretraining signal for few-shot intent detection. During pretraining, each query is paired with another query from the same session and with its answer, and learnable relation tokens are inserted between the two texts; the model is trained with masked language modeling on these relation-aware prompts, learning to read a query in light of the query-query and query-answer relations. For downstream few-shot intent detection, the problem is reformulated as a prompt: a query is concatenated with intent-specific relation tokens and the intent name, and the model is fine-tuned to score each intent. The enhanced variant, QueryAdapt, g

Load-bearing premise

Same-session queries and query-answer pairs form coherent relational contexts that resemble the query-intent relation, and the intent-specific relation token can be written as a query-dependent combination of the two pretrained relation tokens.

Editorial extensions

If this is right

  • Unlabeled chat sessions can be used for structure-aware pretraining without requiring any intent labels, turning abundant conversational logs into useful signal for downstream few-shot tasks.
  • The framework is plug-and-play across backbone models: the reported gains hold for BERT, DistilBERT, and ALBERT, and the method lifts even smaller backbones above strong text-only baselines.
  • QueryAdapt's query-dependent weighting yields additional gains in low-shot settings, with longer, noisier queries leaning more on the query-query relation, pointing to a practical rule for when each relation matters.
  • With only a few labeled examples, BERT-sized models equipped with SAID can match or beat zero-shot LLMs that are around 60-70 times larger, suggesting a cost-effective deployment alternative.
  • Ablations show that both structural pretraining and prompt-based fine-tuning contribute: removing either the query-query or query-answer relation, or replacing the prompt reformulation with a standard classification head, degrades performance.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The mechanism suggests a general recipe for other NLU tasks: any weak relational signal between unlabeled instances, such as session membership, thread structure, or user follow-up edits, could be encoded as soft prompts and pretrained on before few-shot fine-tuning.
  • The linear-span assumption in Equation 5 is directly testable: with more supervision, one could learn intent-specific relation tokens freely and check whether they lie in the span of the query-query and query-answer tokens; if not, a richer generator than a weighted sum would be needed.
  • The two real-world datasets are labeled with eight relatively coarse intent categories, so the reported gains may depend on that granularity; fine-grained or domain-specific intent taxonomies would stress the transfer mechanism more.
  • Because relation tokens are shared across all queries, the method may transfer to dialogue state tracking or next-action prediction, where query-query and query-answer structures arise naturally and labels are similarly scarce.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes SAID, a two-stage framework for few-shot query intent detection. In the pretraining stage, unlabeled chat sessions are used to construct relation-aware prompts by inserting learnable relation tokens between pairs of queries (query-query relation) and between a query and its answer (query-answer relation); the PLM is then trained with a structure-aware masked language modeling objective. In the fine-tuning stage, intent detection is reformulated as a relation-aware prompt task in which intent-specific relation tokens are introduced alongside intent names. The enhanced variant SAID (+QueryAdapt) generates these intent-specific relation tokens as a query-dependent weighted sum of the pretrained query-query and query-answer relation tokens (Eq. 5). Experiments on two datasets (IntentChat and WildChat) across 3/5/10/20-shot settings report large gains over strong baselines, with improvements up to 27% in the 3-shot setting, and additional gains up to 21% from QueryAdapt. Ablations, backbone-transfer experiments, efficiency comparisons, and a case study of the learned attention weights are also provided.

Significance. If the empirical claims hold, the paper makes a useful contribution: it identifies a neglected signal (conversational structure) for few-shot intent detection, packages it in a clean soft-prompt mechanism, and demonstrates that the resulting model can outperform much larger zero-shot LLMs on the two constructed benchmarks. The method is falsifiable and the annotated WildChat subset is a useful public resource. However, the significance is currently conditional. The evaluation rests entirely on two self-created datasets, one proprietary and one annotated by GPT-4, with no external benchmark. More importantly, the paper does not report any decontamination between the unlabeled pretraining corpora and the fine-tuning/test queries, which is a load-bearing issue for a method whose pretraining objective reconstructs query text. The lack of variance reporting also makes the size of the claimed gains difficult to assess. These concerns are addressable, but they must be fixed before the central claim can be accepted.

major comments (3)
  1. [§5.1.1] The paper does not state that the evaluation or fine-tuning queries were removed from the pretraining corpus. For IntentChat, the pretraining set is 7,817 sessions / 91,595 q-a pairs drawn from the same internally human-annotated source from which 20,000 evaluation queries are sampled. For WildChat, the pretraining set uses 62,949 sessions / 354,104 q-a pairs, and the evaluation set is described only as 'hold out and annotate 10,000 queries' from those conversations. Because structure-aware MLM pretraining (§4.1.3) masks and reconstructs exact query text and its conversational context, any leakage of evaluation or few-shot validation queries into the pretraining set could inflate the reported gains through memorization rather than relational transfer, and this advantage would not be available to the baselines. The central transfer claim therefore requires an explicit decontamination stat
  2. [§5.1.2 (Tables 3, 7, 8)] All results are averages over five runs, but no standard deviations, confidence intervals, or significance tests are reported. This matters because several QueryAdapt results in Table 8 are negative (e.g., IntentChat/ALBERT 10-shot: -4.90% accuracy; IntentChat/DistilBERT 20-shot: -4.12% accuracy), while the text states that QueryAdapt 'consistently yields significant improvements.' Without a measure of run-to-run variance, the headline improvements of up to 27% and 21% could be within noise. Please report per-run results or standard deviations and perform paired significance tests between SAID and the best baselines and between SAID (+QueryAdapt) and SAID.
  3. [Eq. (5), §4.2.2] The QueryAdapt mechanism assumes that the query-intent relation token z_qi lies in the linear span of the pretrained query-query and query-answer relation tokens, with query-dependent weights. This representational assumption is central to the explicit-transfer claim, but the paper provides no direct evidence for it. The Linear and MLP baselines in Table 7 also operate in the same span, so they only test how the weights are computed, not whether the span itself is adequate. If the query-intent relation is not well represented in that span, the additional gains from QueryAdapt may be dataset-specific or may rely on the fine-tuning of the backbone in ways not analyzed. Please add an analysis of relation-token geometry (e.g., whether random intent-specific tokens can be projected onto the span without losing information) or compare against an architecture that uses query-dependent weighting
minor comments (6)
  1. [§5.1.1] For WildChat, the labels are produced by GPT-4 Turbo and then reviewed by two student experts. Please report inter-annotator agreement or a quantitative measure of label quality; otherwise it is difficult to assess the noise in the evaluation target.
  2. [§5.1.4] No code is released, and the IntentChat dataset is proprietary. The Reproducibility subsection reports hyperparameters, but the lack of public code or data makes the experiments very difficult to reproduce. Consider releasing the WildChat annotation code and, if possible, the SAID implementation.
  3. [Table 2 and §5.1.3] The zero-shot LLM baselines use the eight-category prompt shown in Table 2, while SAID uses intent names. Please clarify whether the same intent names/category descriptions are available to all methods and whether the LLM baselines were given the same label inventory; this is relevant to the fairness of the comparison.
  4. [§5.4] The ablation study reports only the IntentChat dataset. Since one of the two datasets is used for the central claims, reporting ablations on WildChat as well would strengthen the generality of the conclusions.
  5. [§5.5, Figures 3 and 4] The axis labels are not described in the captions; it would help to specify which few-shot setting is shown or whether the curves are averaged over settings.
  6. [Throughout] There are several typographical issues in the LaTeX source (e.g., 'T echnologies', 'T ransfer') and inconsistent use of 'significant' to mean both 'large' and 'statistically significant.' Please revise for clarity.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: QueryAdapt weights and prompt tokens are learned, not defined as the targets they predict.

full rationale

Walking the derivation chain, SAID pretrains a PLM with masked language modeling over relation-aware prompts (Eqs. 1-2) and then fine-tunes with intent-specific relation prompts (Eqs. 3-4). QueryAdapt (Eq. 5) generates intent-specific relation tokens as a query-dependent convex combination of pretrained query-query and query-answer tokens, with attention weights produced by a learned MLP over the PLM encoding of the query. None of these steps defines a target quantity in terms of itself: the attention weights are optimized on the few-shot labeled examples, and the reported gains are measured on separate evaluation queries. The logits in Eq. 4 are the output of a trained classifier, not an algebraic restatement of the pretraining loss in Eq. 2. The paper's self-citations (e.g., Refs. [20] and [45]) support peripheral methodological conventions and do not carry the central transfer claim. The most substantive concern is empirical rather than circular: Section 5.1.1 does not explicitly state that the 20,000 IntentChat or 10,000 WildChat evaluation queries, or the sessions containing them, were excluded from the 7,817- and 62,949-session pretraining corpora, so contamination is a possible validity threat that could inflate few-shot gains via memorization. However, that would be a data-handling flaw, not a circular derivation: the claimed improvements never reduce by construction to the fitted inputs. Therefore the paper shows no significant circularity.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The method is empirical, so no mathematical axioms are introduced. The listed premises are the modeling assumptions the reported gains depend on: MLM transfer, coherent session structure, informative intent names, and the span assumption behind QueryAdapt. None of these are validated against an independent external benchmark in this preprint.

free parameters (5)
  • m, number of relation tokens per relation = 3
    Section 5.1.4 sets the number of relation tokens to 3 without reporting sensitivity across other values.
  • masking ratio for structure-aware MLM = 0.25
    Section 5.5 selects 0.25 as the best masking ratio after inspecting performance on IntentChat, which is also the dataset used for the main results.
  • pretraining epochs = 4
    Section 5.1.4 uses 4 epochs, and Figure 4 shows that more epochs generally helps; early stopping is not described for pretraining.
  • fine-tuning learning rate = 1e-5, 4e-5, or 1e-4
    Section 5.1.4 reports a hyperparameter search over three values with early stopping, but the per-setting selected values are not reported.
  • QueryAdapt weights lambda_qq and lambda_qa = query-dependent outputs of an MLP
    Equation 5 generates intent-specific relation tokens as a weighted sum, with weights learned during fine-tuning on the few-shot target data.
assumptions (4)
  • domain assumption Masked language modeling on relation-prompted sequences produces representations that transfer to intent classification.
    The pretraining objective in Equation 2 is standard BERT-style MLM; no guarantee is given that the relation tokens retain or transfer relational knowledge.
  • domain assumption Queries within the same session are semantically related enough to define a useful query-query relation.
    Section 3 defines r_qq between any two queries in a session, but chat sessions can contain unrelated successive requests, especially in WildChat.
  • domain assumption Intent names are informative text anchors when concatenated with relation tokens.
    Equation 3 treats the intent name y_k as semantic input; if intent names are opaque or uninformative, the prompt mainly relies on the query and random relation tokens.
  • ad hoc to paper The query-intent relation token lies in the span of the pretrained query-query and query-answer relation tokens.
    Equation 5 constructs z_qi as a weighted sum of z_qq and z_qa; this design choice is not derived from any theoretical property of the relations.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Few-Shot Query Intent Detection via Relation-Aware Prompt Learning." pith.science (2026). https://pith.science/paper/J6NG5KTL

@misc{pith2026250905635,
  author       = {Pith},
  title        = {Pith review of: Few-Shot Query Intent Detection via Relation-Aware Prompt Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/J6NG5KTL}},
  note         = {Machine review of arXiv:2509.05635}
}
read the original abstract

Intent detection is a crucial component of modern conversational systems, since accurately identifying user intent at the beginning of a conversation is essential for generating effective responses. Recent efforts have focused on studying this problem under a challenging few-shot scenario. These approaches primarily leverage large-scale unlabeled dialogue text corpora to pretrain language models through various pretext tasks, followed by fine-tuning for intent detection with very limited annotations. Despite the improvements achieved, existing methods have predominantly focused on textual data, neglecting to effectively capture the crucial structural information inherent in conversational systems, such as the query-query relation and query-answer relation. To address this gap, we propose SAID, a novel framework that integrates both textual and relational structure information in a unified manner for model pretraining for the first time. Building on this framework, we further propose a novel mechanism, the query-adaptive attention network (QueryAdapt), which operates at the relation token level by generating intent-specific relation tokens from well-learned query-query and query-answer relations explicitly, enabling more fine-grained knowledge transfer. Extensive experimental results on two real-world datasets demonstrate that SAID significantly outperforms state-of-the-art methods.

Figures

Figures reproduced from arXiv: 2509.05635 by the authors.

Figure 1
Figure 1. The model architecture of SAID. (a) Pretraining stage with relation-aware soft prompts; (b) Fine-tuning and inference [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. The model architecture of SAID (+QueryAdapt). [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Impact of masking ratio on IntentChat. (a) Accuracy (b) F1 [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Impact of pretraining epochs on IntentChat. [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: Query embedding visualizations on IntentChat. [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 6
Figure 6. Figure 6: Comparison of query texts between QQ- and QA [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

46 extracted references · 30 canonical work pages

  1. [1]

    Bert: Pre-training of deep bidirectional transformers for language understanding,

    J. D. M.-W. C. Kenton and L. K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” in Proceedings of naacL-HLT, vol. 1. Minneapolis, Minnesota, 2019, p. 2

    work page 2019

  2. [2]

    Exploring the limits of transfer learning with a unified text-to-text transformer,

    C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P . J. Liu, “Exploring the limits of transfer learning with a unified text-to-text transformer,”Journal of Machine Learning Research, vol. 21, no. 140, pp. 1–67, 2020. [Online]. Available: http://jmlr.org/papers/v21/20-074.html

    work page 2020

  3. [3]

    Language models are few-shot learners,

    T. B. Brown, “Language models are few-shot learners,”arXiv preprint arXiv:2005.14165, 2020

    arXiv 2005

  4. [4]

    Llama: Open and efficient foundation language models,

    H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozi`ere, N. Goyal, E. Hambro, F. Azharet al., “Llama: Open and efficient foundation language models,”arXiv preprint arXiv:2302.13971, 2023

    arXiv 2023

  5. [5]

    Gpt-4 technical report,

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkatet al., “Gpt-4 technical report,”arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  6. [6]

    Gemini: a family of highly capable multimodal models,

    G. Team, R. Anil, S. Borgeaud, Y. Wu, J.-B. Alayrac, J. Yu, R. Soricut, J. Schalkwyk, A. M. Dai, A. Hauthet al., “Gemini: a family of highly capable multimodal models,”arXiv preprint arXiv:2312.11805, 2023

    arXiv 2023

  7. [7]

    Deepseek-r1: Incentivizing reason- ing capability in llms via reinforcement learning,

    D. Guo, D. Yang, H. Zhang, J. Song, R. Zhang, R. Xu, Q. Zhu, S. Ma, P . Wang, X. Biet al., “Deepseek-r1: Incentivizing reason- ing capability in llms via reinforcement learning,”arXiv preprint arXiv:2501.12948, 2025

    arXiv 2025

  8. [8]

    A new dialogue response generation agent for large language models by asking questions to detect user’s intentions,

    S. Wu, X. Shen, and R. Xia, “A new dialogue response generation agent for large language models by asking questions to detect user’s intentions,”arXiv preprint arXiv:2310.03293, 2023

    arXiv 2023

Show all 46 references
  1. [9]

    Tell me more! towards implicit user intention understanding of language model driven agents,

    C. Qian, B. He, Z. Zhuang, J. Deng, Y. Qin, X. Cong, Z. Zhang, J. Zhou, Y. Lin, Z. Liuet al., “Tell me more! towards implicit user intention understanding of language model driven agents,”arXiv preprint arXiv:2402.09205, 2024

    2024 arXiv

  2. [10]

    Effectiveness of pre-training for few-shot intent classi- fication,

    H. Zhang, Y. Zhang, L.-M. Zhan, J. Chen, G. Shi, X.-M. Wu, and A. Y. Lam, “Effectiveness of pre-training for few-shot intent classi- fication,” inFindings of the Association for Computational Linguistics: EMNLP 2021, 2021, pp. 1114–1120

    2021

  3. [11]

    Cluster & tune: Boost cold start performance in text classification,

    E. Shnarch, A. Gera, A. Halfon, L. Dankin, L. Choshen, R. Aharonov, and N. Slonim, “Cluster & tune: Boost cold start performance in text classification,” inProceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2022, pp...

    2022

  4. [12]

    Fine-tuning pre-trained language models for few-shot intent detection: Supervised pre-training and isotropization,

    H. Zhang, H. Liang, Y. Zhang, L.-M. Zhan, X.-M. Wu, X. Lu, and A. Lam, “Fine-tuning pre-trained language models for few-shot intent detection: Supervised pre-training and isotropization,” in Proceedings of the 2022 Conference of the North American Chapter of the Association fo...

    2022

  5. [13]

    Few-shot intent detection via con- trastive pre-training and fine-tuning,

    J. Zhang, T. Bui, S. Yoon, X. Chen, Z. Liu, C. Xia, Q. H. Tran, W. Chang, and S. Y. Philip, “Few-shot intent detection via con- trastive pre-training and fine-tuning,” inProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021, pp. 1906–1912

    2021

  6. [14]

    Roberta: A robustly optimized bert pretraining approach,

    Y. Liu, “Roberta: A robustly optimized bert pretraining approach,” arXiv preprint arXiv:1907.11692, 2019

    1907 arXiv

  7. [15]

    Exploring zero and few-shot techniques for intent classification,

    S. Parikh, M. Tiwari, P . Tumbade, and Q. Vohra, “Exploring zero and few-shot techniques for intent classification,” inProceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 5: Industry T rack), 2023, pp. 744–751

    2023

  8. [16]

    Simcse: Simple contrastive learn- ing of sentence embeddings,

    T. Gao, X. Yao, and D. Chen, “Simcse: Simple contrastive learn- ing of sentence embeddings,” inEMNLP 2021-2021 Conference on Empirical Methods in Natural Language Processing, Proceedings, 2021

    2021

  9. [17]

    Efficient intent detection with dual sentence encoders,

    I. Casanueva, T. Tem ˇcinas, D. Gerz, M. Henderson, and I. Vuli ´c, “Efficient intent detection with dual sentence encoders,” inPro- ceedings of the 2nd Workshop on Natural Language Processing for Conversational AI, 2020, pp. 38–45

    2020

  10. [18]

    Dialoglue: A natural language understanding benchmark for task-oriented dialogue,

    S. Mehri, M. Eric, and D. Hakkani-Tur, “Dialoglue: A natural language understanding benchmark for task-oriented dialogue,” arXiv preprint arXiv:2009.13570, 2020

    2009 arXiv

  11. [19]

    Revisit few-shot intent classification with plms: Direct fine-tuning vs. con- tinual pre-training,

    H. Zhang, H. Liang, L.-M. Zhan, X.-M. Wu, and A. Y. Lam, “Revisit few-shot intent classification with plms: Direct fine-tuning vs. con- tinual pre-training,” inFindings of the Association for Computational Linguistics: ACL 2023, 2023, pp. 11 105–11 121

    2023

  12. [20]

    Region embedding with intra and inter-view contrastive learning,

    L. Zhang, C. Long, and G. Cong, “Region embedding with intra and inter-view contrastive learning,”IEEE T ransactions on Knowl- edge and Data Engineering, vol. 35, no. 9, pp. 9031–9036, 2022

    2022

  13. [21]

    A simple framework for contrastive learning of visual representations,

    T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, “A simple framework for contrastive learning of visual representations,” in International conference on machine learning. PMLR, 2020, pp. 1597– 1607

    2020

  14. [22]

    Discriminative nearest neighbor few- shot intent detection by transferring natural language inference,

    J. Zhang, K. Hashimoto, W. Liu, C.-S. Wu, Y. Wan, S. Y. Philip, R. Socher, and C. Xiong, “Discriminative nearest neighbor few- shot intent detection by transferring natural language inference,” inProceedings of the 2020 Conference on Empirical Methods in Natural Language Proce...

    2020

  15. [23]

    Exploring the limits of transfer learning with a unified text-to-text transformer,

    C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P . J. Liu, “Exploring the limits of transfer learning with a unified text-to-text transformer,”Journal of machine learning research, vol. 21, no. 140, pp. 1–67, 2020

    2020

  16. [24]

    Language models are unsupervised multitask learners,

    A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskeveret al., “Language models are unsupervised multitask learners,”OpenAI blog, vol. 1, no. 8, p. 9, 2019

    2019

  17. [25]

    Commonsense knowl- edge mining from pretrained models,

    J. Davison, J. Feldman, and A. M. Rush, “Commonsense knowl- edge mining from pretrained models,” inProceedings of the 2019 conference on empirical methods in natural language processing and JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 14 the 9th international join...

    2019

  18. [26]

    Ppt: Pre-trained prompt tuning for few-shot learning,

    Y. Gu, X. Han, Z. Liu, and M. Huang, “Ppt: Pre-trained prompt tuning for few-shot learning,” inProceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2022, pp. 8410–8423

    2022

  19. [27]

    Learning to compose soft prompts for compositional zero-shot learning,

    N. V . Nayak, P . Yu, and S. Bach, “Learning to compose soft prompts for compositional zero-shot learning,” inThe Eleventh International Conference on Learning Representations

  20. [28]

    Exploiting cloze-questions for few-shot text classification and natural language inference,

    T. Schick and H. Sch ¨utze, “Exploiting cloze-questions for few-shot text classification and natural language inference,” inProceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, 2021, pp. 255–269

    2021

  21. [29]

    The power of scale for parameter-efficient prompt tuning,

    B. Lester, R. Al-Rfou, and N. Constant, “The power of scale for parameter-efficient prompt tuning,” inProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021, pp. 3045–3059

    2021

  22. [30]

    P-tuning: Prompt tuning can be comparable to fine-tuning across scales and tasks,

    X. Liu, K. Ji, Y. Fu, W. Tam, Z. Du, Z. Yang, and J. Tang, “P-tuning: Prompt tuning can be comparable to fine-tuning across scales and tasks,” inProceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 2022, pp. 61–68

    2022

  23. [31]

    Pre- train, prompt, and predict: A systematic survey of prompting methods in natural language processing,

    P . Liu, W. Yuan, J. Fu, Z. Jiang, H. Hayashi, and G. Neubig, “Pre- train, prompt, and predict: A systematic survey of prompting methods in natural language processing,”ACM Computing Sur- veys, vol. 55, no. 9, pp. 1–35, 2023

    2023

  24. [32]

    A systematic survey of prompt engineering in large language models: Techniques and applications,

    P . Sahoo, A. K. Singh, S. Saha, V . Jain, S. Mondal, and A. Chadha, “A systematic survey of prompt engineering in large language models: Techniques and applications,”arXiv preprint arXiv:2402.07927, 2024

    2024 arXiv

  25. [33]

    Visual prompt tuning,

    M. Jia, L. Tang, B.-C. Chen, C. Cardie, S. Belongie, B. Hariharan, and S.-N. Lim, “Visual prompt tuning,” inEuropean Conference on Computer Vision. Springer, 2022, pp. 709–727

    2022

  26. [34]

    Explor- ing visual prompts for adapting large-scale models,

    H. Bahng, A. Jahanian, S. Sankaranarayanan, and P . Isola, “Explor- ing visual prompts for adapting large-scale models,”arXiv preprint arXiv:2203.17274, 2022

    2022 arXiv

  27. [35]

    Graphprompt: Unifying pre-training and downstream tasks for graph neural networks,

    Z. Liu, X. Yu, Y. Fang, and X. Zhang, “Graphprompt: Unifying pre-training and downstream tasks for graph neural networks,” inProceedings of the ACM Web Conference 2023, 2023, pp. 417–428

    2023

  28. [36]

    Gppt: Graph pre- training and prompt tuning to generalize graph neural networks,

    M. Sun, K. Zhou, X. He, Y. Wang, and X. Wang, “Gppt: Graph pre- training and prompt tuning to generalize graph neural networks,” inProceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2022, pp. 1717–1727

    2022

  29. [37]

    Convert: Efficient and accurate conversational rep- resentations from transformers,

    M. Henderson, I. Casanueva, N. Mrk ˇsi´c, P .-H. Su, T.-H. Wen, and I. Vuli ´c, “Convert: Efficient and accurate conversational rep- resentations from transformers,” inFindings of the Association for Computational Linguistics: EMNLP 2020, 2020, pp. 2161–2174

    2020

  30. [38]

    Dialogpt: Large-scale generative pre- training for conversational response generation,

    Y. Zhang, S. Sun, M. Galley, Y.-C. Chen, C. Brockett, X. Gao, J. Gao, J. Liu, and W. B. Dolan, “Dialogpt: Large-scale generative pre- training for conversational response generation,” inProceedings of the 58th Annual Meeting of the Association for Computational Linguistics: Sy...

    2020

  31. [39]

    Tod-bert: Pre-trained natural language understanding for task-oriented dialogue,

    C.-S. Wu, S. C. Hoi, R. Socher, and C. Xiong, “Tod-bert: Pre-trained natural language understanding for task-oriented dialogue,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020, pp. 917–929

    2020

  32. [40]

    Wildchat: 1m chatgpt interaction logs in the wild,

    W. Zhao, X. Ren, J. Hessel, C. Cardie, Y. Choi, and Y. Deng, “Wildchat: 1m chatgpt interaction logs in the wild,” inThe Twelfth International Conference on Learning Representations

  33. [41]

    Learn to adapt for generalized zero-shot text classification,

    Y. Zhang, C. Yuan, X. Wang, Z. Bai, and Y. Liu, “Learn to adapt for generalized zero-shot text classification,” inProceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2022, pp. 517–527

    2022

  34. [42]

    Distilbert, a distilled version of bert: Smaller, faster, cheaper and lighter,

    V . Sanh, “Distilbert, a distilled version of bert: Smaller, faster, cheaper and lighter,”arXiv preprint arXiv:1910.01108, 2019

    1910 arXiv

  35. [43]

    Albert: A lite bert for self-supervised learning of language representations,

    Z. Lan, “Albert: A lite bert for self-supervised learning of language representations,”arXiv preprint arXiv:1909.11942, 2019

    1909 arXiv

  36. [44]

    Efficient few-shot learning without prompts,

    L. Tunstall, N. Reimers, U. E. S. Jo, L. Bates, D. Korat, M. Wasserblat, and O. Pereg, “Efficient few-shot learning without prompts,”arXiv preprint arXiv:2209.11055, 2022

    2022 arXiv

  37. [45]

    Beyond similarity: Relation-based collaborative filtering,

    G. Liu, L. Zhang, and J. Wu, “Beyond similarity: Relation-based collaborative filtering,”IEEE T ransactions on Knowledge and Data Engineering, vol. 35, no. 1, pp. 128–140, 2021

    2021

  38. [46]

    Visualizing data using t-sne,

    L. Van der Maaten and G. Hinton, “Visualizing data using t-sne,” Journal of Machine Learning Research, vol. 9, no. 2605, pp. 2579–2605, 2008

    2008

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.