Pith. sign in

REVIEW 16 cited by

Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2107.13586 v1 pith:QVMUDD26 submitted 2021-07-28 cs.CL cs.AIcs.LG

Pengfei Liu , Weizhe Yuan , Jinlan Fu , Zhengbao Jiang , Hiroaki Hayashi , Graham Neubig This is my paper
classification cs.CLcs.AIcs.LG
keywords languagemodellearningexistingmodelsprompt-basedfinalinput
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

This paper surveys and organizes research works in a new paradigm in natural language processing, which we dub "prompt-based learning". Unlike traditional supervised learning, which trains a model to take in an input x and predict an output y as P(y|x), prompt-based learning is based on language models that model the probability of text directly. To use these models to perform prediction tasks, the original input x is modified using a template into a textual string prompt x' that has some unfilled slots, and then the language model is used to probabilistically fill the unfilled information to obtain a final string x, from which the final output y can be derived. This framework is powerful and attractive for a number of reasons: it allows the language model to be pre-trained on massive amounts of raw text, and by defining a new prompting function the model is able to perform few-shot or even zero-shot learning, adapting to new scenarios with few or no labeled data. In this paper we introduce the basics of this promising paradigm, describe a unified set of mathematical notations that can cover a wide variety of existing work, and organize existing work along several dimensions, e.g.the choice of pre-trained models, prompts, and tuning strategies. To make the field more accessible to interested beginners, we not only make a systematic review of existing works and a highly structured typology of prompt-based concepts, but also release other resources, e.g., a website http://pretrain.nlpedia.ai/ including constantly-updated survey, and paperlist.

Discussion (0). Sign in to comment.

Forward citations

Cited by 16 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. OpenAlex reports about 518 citations worldwide. Full citation record

  1. Machine Understanding of Scientific Language

    cs.CL 2025-06 conditional novelty 7.0 of 10

    The thesis defines and evaluates tasks and datasets for automatic fact checking, cite-worthiness, exaggeration detection, and information change measurement in science communication, culminating in SPICED, a cross-med...

  2. Aligned but Blind: Alignment Increases Implicit Bias by Reducing Awareness of Race

    cs.CL 2025-05 conditional novelty 7.0 of 10

    Alignment on Llama 3 reduces explicit bias but amplifies implicit bias, because aligned models no longer represent 'black' and 'white' as racial concepts in ambiguous contexts.

  3. LLM-as-a-Coach: Experiential Learning for Non-Verifiable Tasks

    cs.LG 2026-07 conditional novelty 6.0 of 10

    Training a language model by distilling a coach's written experiential knowledge beats training on a scalar rubric score for open-ended tasks, with better out-of-distribution transfer.

  4. Task-Aware Adaptive Modulation: A Replay-Free and Resource-Efficient Approach For Continual Graph Learning

    cs.LG 2025-08 conditional novelty 6.0 of 10

    TAAM achieves replay-free, pre-training-free continual graph learning by routing each task to a small node-attentive modulator inserted into a frozen GNN.

  5. Time Series Foundation Models for Multivariate Financial Time Series Forecasting

    q-fin.GN 2025-07 reject novelty 6.0 of 10

    Pretrained TTM shows large transfer and sample-efficiency gains in three financial forecasting tasks relative to training from scratch, but methodological flaws including possible look-ahead bias weaken the quantitati...

  6. GeNRe: A French Gender-Neutral Rewriting System Using Collective Nouns

    cs.CL 2025-05 conditional novelty 6.0 of 10

    GeNRe is the first French gender-neutral rewriting system to replace masculine plural member nouns with collective nouns, reaching 3.81% WER with its rule-based version.

  7. Contextualized Early Detection of Online Firestorms: A Sequential LLM-Based Approach

    cs.AI 2026-07 conditional novelty 5.0 of 10

    An LLM judging sliding windows of Reddit threads can flag escalating firestorm threads early (recall 0.98 at a mean 8.56 comments) and classify complete threads retrospectively with 0.915 accuracy.

  8. Prompt Engineering for Requirements Engineering: A Literature Review and Roadmap

    cs.SE 2025-07 conditional novelty 5.0 of 10

    The first roadmap-oriented systematic literature review of prompt engineering for requirements engineering analyzes 35 studies and proposes a hybrid taxonomy and research roadmap.

  9. Workflow-Based Evaluation of Music Generation Systems

    eess.AS 2025-06 conditional novelty 5.0 of 10

    A single-producer workflow evaluation of eight music AI tools finds they work as idea and sound generators but not as complete composers, and proposes a reusable framework.

  10. The Maskability Index: Predicting Task-Objective Alignment in Pretrained Language Models

    cs.CL 2026-07 conditional novelty 4.0 of 10

    A five-example DepthRank gap between masked and prefix templates sorts ATOMIC2020 relations into groups that match which prompt style generates better tails.

  11. The Role of Orthographic Consistency in Multilingual Embedding Models for Text Classification in Arabic-Script Languages

    cs.CL 2025-07 reject novelty 4.0 of 10

    Language-specific RoBERTa models for four Arabic-script languages beat multilingual baselines on news classification, though the claimed orthographic-consistency mechanism is not demonstrated.

  12. Breaking Physical and Linguistic Borders: Multilingual Federated Prompt Tuning for Low-Resource Languages

    cs.CL 2025-07 conditional novelty 4.0 of 10

    Federated averaging of prompt embeddings from a frozen multilingual model improves accuracy on some low-resource tasks (XNLI) but not consistently on others (MasakhaNEWS).

  13. Dynamic Context-Aware Prompt Recommendation for Domain-Specific AI Applications

    cs.AI 2025-06 conditional novelty 4.0 of 10

    A dynamic prompt recommendation system for skill-based security copilots combines retrieval, hierarchical skill selection, and telemetry-based ranking, reporting high usefulness in internal evaluations.

  14. Intelligent Assistants for the Semiconductor Failure Analysis with LLM-Based Planning Agents

    cs.AI 2025-06 conditional novelty 4.0 of 10

    An LLM-based planning agent using ReAct, retrieval tools, and RAG can autonomously answer semiconductor failure analysis queries in an industrial setting, with best average quality 3.6/5.

  15. Improving TCM Question Answering through Tree-Organized Self-Reflective Retrieval with LLMs

    cs.CL 2025-02 conditional novelty 4.0 of 10

    A tree-organized, self-reflective retrieval framework over a TCM knowledge base lifts GPT-4 accuracy on a 600-question licensing-exam sample by 19.85 absolute percentage points.

  16. CU-ICU: Customizing Unsupervised Instruction-Finetuned Language Models for ICU Datasets via Text-to-Text Transfer Transformer

    cs.CL 2025-07 reject novelty 2.0 of 10

    CU-ICU applies LoRA, AdaLoRA, and (IA)3 to FLAN-T5 for ICU sepsis detection, mortality prediction, and note generation, claiming efficiency gains that are not backed by reported baselines.

Pith tools