Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Contrastive Prompting Enhances Sentence Embeddings in LLMs through Inference-Time Steering

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that contrasting a sentence's normal prompt embedding with an auxiliary 'irrelevant information' prompt—subtracting the latter's last-token value vector—steers LLM embeddings to core semantics and improves zero-shot…

desk verdict A training-free contrastive steering trick that reliably adds a few points to prompt-based sentence embeddings, but the stated mechanism—per-sentence removal of non-essential information—is unverified and needs a control with a fixed auxiliary sentence. read the letter →

arxiv 2505.12831 v1 pith:V5BUSP5T submitted 2025-05-19 cs.CL

Zifeng Cheng , Zhonghui Wang , Yuchen Fu , Zhiwei Jiang , Yafeng Yin , Cong Wang , Qing Gu This is my paper · ORCID
classification cs.CL
keywords contrastivepromptingsentenceembeddingsinference-timesteeringpromptengineeringlargelanguagemodelssemantictextualsimilarityzero-shot
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Contrastive Prompting (CP) is a training-free, inference-time method for extracting better sentence embeddings from large language models. The paper claims that existing prompt-based embeddings—the hidden state of the final token after a prompt like 'This sentence: [TEXT] means in one word:'—still carry too much non-essential information such as stopwords. CP adds a second auxiliary prompt, 'The irrelevant information of this sentence: [TEXT] means in one word:', encodes both prompts up to an early transformer layer, and subtracts the auxiliary prompt's last-token value vector from the normal prompt's. After rescaling the difference and continuing the forward pass, the resulting embedding is claimed to concentrate on core semantics, yielding consistent gains on semantic-textual-similarity benchmarks, transfer classification tasks, and MTEB subsets across several LLaMA models; the largest reported average gain is +5.24 on STS for PromptEOL with LLaMA2-7B.

What carries the argument

The central object is the last-token contextualized value vector of a multi-head attention layer—the per-head output before the output projection at the position of the final prompt token. CP's machinery is the contrast $\Delta v^\ell = v^{(\ell)}_{\mathrm{nor},N_{\mathrm{nor}}} - v^{(\ell)}_{\mathrm{aux},N_{\mathrm{aux}}}$ between the normal prompt's value vector and the auxiliary 'irrelevant information' prompt's value vector at a chosen early layer ($\ell = 5$ or $7$), followed by norm scaling ($\hat v = \alpha \Delta v^\ell$) or norm recovering ($\hat v = \Delta v^\ell \cdot \|v^{(\ell)}_{\mathrm{nor},N_{\mathrm{nor}}}\|_2 / \|\Delta v^\ell\|_2$). Substituting this adjusted vector into the normal prompt's forward pass steers later layers toward the sentence's core semantics, and the final embeddings are extracted from an intermediate layer rather than the last layer. The auxiliary prompt needs only to propagate to the low intervention layer, which the paper cites as the reason the extra cost is small.

What would settle it

Construct sentence pairs matched for core meaning but differing only in surface wording (stopwords, articles, reordering), and check whether CP raises their embedding cosine similarity above the no-CP baseline; if it does not, the claimed removal of non-essential information is not happening. A second check: replace the auxiliary prompt with a control that asks for core meaning or uses scrambled tokens; if STS gains persist, the improvement is not due to contrasting away irrelevant information.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that the excess non-semantic content in LLM-based sentence embeddings can be removed by a contrast at inference time rather than by better prompt wording alone. Given a normal prompt with last-token contextualized value vector $v^{(\ell)}_{\mathrm{nor},N_{\mathrm{nor}}}$ at layer $\ell$, the method builds an auxiliary prompt that asks for the sentence's 'irrelevant information' and obtains $v^{(\ell)}_{\mathrm{aux},N_{\mathrm{aux}}}$. The steering vector $\Delta v^\ell = v^{(\ell)}_{\mathrm{nor},N_{\mathrm{nor}}} - v^{(\ell)}_{\mathrm{aux},N_{\mathrm{aux}}}$ is then inserted in place of the normal prompt's last-token value vector, optionally scaled by $\alpha$ (norm scaling) or renormalized to the original norm (norm recovering), and the forward pass is continued to an intermediate output layer. The paper reports that this procedure improves PromptEOL, Pretended CoT, Knowledge, and their averaged CK combination across LLaMA2-7B, LLaMA2-13B, and LLaMA3.1-8B on STS tasks, transfer tasks, and MTEB subsets, and that the largest gains occur for the simplest prompt, PromptEOL.

Load-bearing premise

The load-bearing premise is that the auxiliary prompt 'The irrelevant information of this sentence: [TEXT] means in one word:' makes the LLM's last-token vector encode the sentence-specific non-essential content, so subtracting it from the normal prompt's vector removes that content while leaving core semantics intact.

Editorial extensions

If this is right

  • Any existing prompt-based sentence-embedding method can be upgraded at inference time with no extra training data or fine-tuning.
  • The simplest prompts benefit most, so CP narrows the performance gap between prompt variants and makes embedding quality less sensitive to prompt choice.
  • Gains persist across model families and sizes (LLaMA2-7B, LLaMA2-13B, LLaMA3.1-8B) and across STS, transfer-learning, and MTEB classification, clustering, and reranking tasks.
  • The extra computation is modest because the auxiliary prompt only runs through lower layers, and one auxiliary forward pass can serve several normal prompts.
  • Under in-context learning, CP still improves performance, though the gain is smaller than in the zero-shot setting.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the auxiliary vector truly isolates non-essential information, CP is a general way to subtract a 'surface wording' subspace from embeddings, and the same trick could plausibly be applied to fine-tuned embedding models, not only zero-shot prompt-based ones.
  • The near-tie between norm scaling and norm recovering suggests the gain comes from the direction of the steering vector rather than from changing the vector's length; a control that subtracts a random or fixed vector of the same norm should not reproduce the gains.
  • The auxiliary-prompt ablation (sentiment and entity prompts hurt) implies the method depends on the prompt naming 'irrelevant' content; a principled rule for generating the auxiliary prompt for each normal prompt would be a natural extension.
  • Because the intervention happens at layers 5–7 while embeddings are read at layers 27–31, the steering appears to redirect the whole forward trajectory; probing how the semantic/non-semantic distinction evolves across layers could identify where CP works best.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper proposes Contrastive Prompting (CP), a training-free inference-time intervention for extracting sentence embeddings from LLMs. For each input sentence, CP runs two forward passes: a normal prompt (e.g., PromptEOL, Pretended CoT, Knowledge) and an auxiliary prompt ('The irrelevant information of this sentence: "[TEXT]" means in one word: "'). At a chosen layer, the last-token contextualized value vector of the auxiliary prompt is subtracted from that of the normal prompt (Eq. 1), the difference is rescaled by either a scaling factor (NS) or norm renormalization (NR), and the remaining forward pass is completed; the final embedding is taken from an intermediate layer. Experiments on seven STS datasets, SentEval transfer tasks, and MTEB subsets across LLaMA2-7B/13B, LLaMA3.1-8B, and OPT-6.7B report consistent gains over prompt-based baselines, with the largest gains on PromptEOL (e.g., +5.24 average on STS for LLaMA2-7B) and smaller gains on Knowledge and Pretended CoT. The central interpretative claim is that the subtraction removes sentence-specific non-essential information rather than applying a generic steering direction.

Significance. If the mechanism is validated, the contribution is practically useful: a plug-and-play, data-free intervention that improves zero-shot embeddings across backbones and tasks with modest overhead, since the auxiliary prompt only propagates to early layers. The paper's strengths are its broad empirical evaluation, use of multiple backbones, and clear ablations of intervention layer, scaling factor, auxiliary prompt wording, and intervention position. The STS and transfer results are extensive and mostly consistent. The main weakness is that the load-bearing mechanistic interpretation is not directly tested: no experiment separates the sentence-dependent component of the difference vector from the constant prompt-template offset. Because this interpretation is central to the abstract and to Section 4.2, the paper needs an explicit control before the claimed mechanism can be accepted.

major comments (3)
  1. [§4.2, Eq. (1)] The central mechanism claim is that the difference vector in Eq. (1) isolates sentence-specific non-essential information, but the paper never tests whether the sentence content in the auxiliary prompt is doing the work. Table 4 varies only the instruction phrase while keeping [TEXT], and Table 7 is a qualitative case study; neither compares against a fixed placeholder sentence in the same auxiliary template. If the same improvement appears with a constant auxiliary sentence, then CP is applying a sentence-independent steering direction rather than removing the sentence's non-essential information, and the interpretation in the abstract and Section 4.2 is unsupported. The assertion in Section 4.2 that 'the semantic activation vector is sentence-dependent' needs direct evidence, for example the variance of the difference vector across sentences relative to a constant-prompt baseline, and an ablation with a fixed auxiliary sentence. The Limitations section acknowledges that auxiliary prompt design is preliminary, but it does not address this more basic control.
  2. [§5.1, §5.7, Tables 1, 2, 6] Hyperparameter selection and lack of statistical validation weaken the claim of 'significantly improves'. Section 5.1 grid-searches the intervention layer and scaling factor on the STS-B development set and then reports STS-B test results, without stating how selection bias is avoided; transfer and MTEB experiments (Section 5.7 and Appendix D) use different hyperparameters without describing a validation split. Several improvements are small or negative at the dataset level (e.g., Pretended CoT + CP-NS on STS13 in Table 1 is -0.23, and the LLaMA2-13B gains in Table 2 are around +0.5 on average), and no error bars, seed variance, or significance tests are reported anywhere in the paper. I ask for a held-out hyperparameter protocol and either confidence intervals or paired significance tests on the main STS results.
  3. [§5.7, Table 6] The claim that CP-NS 'outperforms the supervised method ST5-Enc' rests on a 0.10-point average difference over seven transfer tasks with no variance estimates. This is a side claim, but the wording should be softened to 'comparable to' unless significance is established.
minor comments (5)
  1. [§5.6] The sentence 'using Knowledge + NP-NS on the STS-B development set' should read 'Knowledge + CP-NS'; NP-NS is not introduced anywhere.
  2. [Table 3] The layer counts are not self-explanatory: PromptEOL without CP is 27 layers and with CP is 31 layers, but 27 + 5 (the stated intervention layer) is 32, and Knowledge with CP is listed as 37 layers, which exceeds the model depth. Please clarify whether the counts include the auxiliary forward pass and how each number is computed.
  3. [Appendix D] The MTEB evaluation is described as a subset of the data, but the selection criteria for the subset are not stated; please specify how the datasets were chosen to avoid selection bias.
  4. [References] There is a typo in the reference to Voorhees and Tice: 'V oorhees' should be 'Voorhees'.
  5. [§5.1] The phrase 'the length of norm scaling is 4' in Section 5.7 should be 'the scaling factor is 4'; length of a scaling factor is not meaningful.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the CP intervention is an empirical construction evaluated on external benchmarks, and no load-bearing step reduces to its own input.

full rationale

The paper's derivation chain is empirical rather than definitional. The central intervention in Eq. (1), Δv^ℓ = v_nor^(ℓ) − v_aux^(ℓ), is a constructed operation on hidden states from two different prompt templates; the claim that this difference removes non-essential information is a hypothesis that is tested against external Semantic Textual Similarity (STS), SentEval transfer, and MTEB benchmarks, not an identity or a fitted target. The auxiliary prompt encodes an intuitive notion of 'irrelevant information', but no equation defines the target metric, the similarity labels, or the core semantics in terms of the auxiliary prompt's output; the auxiliary vector is a distinct hidden state and the method's success is an empirical question. Hyperparameter selection (intervention layer ℓ and scaling factor α on the STS-B development set) is standard model selection and does not constitute fitting the evaluation targets into the method; the reported gains on held-out STS test sets and other task families remain externally evaluated. Self-citations (e.g., Fu et al. 2024, Cheng et al. 2024a,b) appear only in related-work or baseline context and are not load-bearing for the main claim. The paper's Limitations section identifies auxiliary-prompt optimization and language coverage as open issues, which is consistent with a non-circular empirical contribution. Even if the mechanism were under-validated (for example, if Δv contained a large constant prompt-template component rather than sentence-specific non-essential content), that would be a correctness or control concern, not circularity. No load-bearing step in the paper reduces by construction to its own inputs, and no 'prediction' is a renamed fitted parameter.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The method rests on several hand-picked design choices (auxiliary prompt template, intervention layer, scaling factor, output layer) and on the assumption that activation subtraction is semantically meaningful. No new entities are introduced. The free parameters are tuned on a dev set, which is standard but should be disclosed in every claim.

free parameters (4)
  • Scaling factor alpha (NS) = 2 for PromptEOL, 3 for Pretended CoT and Knowledge on STS; 4 for transfer tasks
    Controls the magnitude of the steering vector; grid-searched on STS-B dev set. Sensitivity shown in Figure 3(c).
  • Intervention layer l = 5 for PromptEOL, 7 for Pretended CoT and Knowledge; 7 for transfer tasks
    Layer at which the auxiliary value vector is subtracted; grid-searched on STS-B dev set. Varies by prompt.
  • Output layer for sentence embedding = 27 for PromptEOL and Pretended CoT; penultimate for Knowledge
    Intermediate layer whose hidden state is used as the embedding; chosen via validation set. Affects performance.
  • Auxiliary prompt template = Hand-chosen: "The irrelevant information of this sentence: [TEXT] means in one word:" (variants in Table 4)
    Selected by the authors to elicit non-essential information. Table 4 shows robustness to similar templates, but the choice is ad hoc.
assumptions (5)
  • domain assumption The last-token value vector of the auxiliary prompt contains sentence-specific non-essential information.
    Introduced in Section 4.1; the entire method relies on this premise.
  • domain assumption Subtracting the auxiliary value vector from the normal prompt value vector removes non-essential information while preserving core semantics.
    Used in Eq. (1) and Section 4.2; no theoretical proof, only intuitive motivation.
  • domain assumption The value vectors of the multi-head attention layer are an appropriate intervention target.
    Motivated by information interaction in attention (Section 3), but not justified formally.
  • domain assumption Intermediate-layer embeddings yield better sentence embeddings than the final layer.
    Stated in Section 4.4, citing prior work (Liu et al., 2024b; Jin et al., 2024).
  • domain assumption Hyperparameters tuned on the STS-B development set transfer to other STS datasets and tasks.
    Section 5.1; partially supported by Table 9, but not tested exhaustively.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Contrastive Prompting Enhances Sentence Embeddings in LLMs through Inference-Time Steering." pith.science (2026). https://pith.science/paper/V5BUSP5T

@misc{pith2026250512831,
  author       = {Pith},
  title        = {Pith review of: Contrastive Prompting Enhances Sentence Embeddings in LLMs through Inference-Time Steering},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/V5BUSP5T}},
  note         = {Machine review of arXiv:2505.12831}
}
read the original abstract

Extracting sentence embeddings from large language models (LLMs) is a practical direction, as it requires neither additional data nor fine-tuning. Previous studies usually focus on prompt engineering to guide LLMs to encode the core semantic information of the sentence into the embedding of the last token. However, the last token in these methods still encodes an excess of non-essential information, such as stop words, limiting its encoding capacity. To this end, we propose a Contrastive Prompting (CP) method that introduces an extra auxiliary prompt to elicit better sentence embedding. By contrasting with the auxiliary prompt, CP can steer existing prompts to encode the core semantics of the sentence, rather than non-essential information. CP is a plug-and-play inference-time intervention method that can be combined with various prompt-based methods. Extensive experiments on Semantic Textual Similarity (STS) tasks and downstream classification tasks demonstrate that our method can improve the performance of existing prompt-based methods across different LLMs. Our code will be released at https://github.com/zifengcheng/CP.

Figures

Figures reproduced from arXiv: 2505.12831 by the authors.

Figure 1
Figure 1. The comparison of sentence embeddings ob [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Illustration of the contrastive prompting method. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Effects of intervention layer, scaling factor, and output layer. (a) Effects of the intervention layer on norm [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Token Prepending: A Training-Free Approach for Eliciting Better Sentence Embeddings from LLMs

    cs.CL 2024-12 conditional novelty 6.0 of 10

    Prepending each early layer's decoded sentence embedding to the next layer's input improves prompt-based sentence embeddings from decoder-only LLMs without fine-tuning.

Reference graph

Works this paper leans on

52 extracted references · 15 canonical work pages · cited by 1 Pith paper

  1. [1]

    Cer, Mona T

    Eneko Agirre, Carmen Banea, Claire Cardie, Daniel M. Cer, Mona T. Diab, Aitor Gonzalez - Agirre, Weiwei Guo, I \ n igo Lopez - Gazpio, Montse Maritxalar, Rada Mihalcea, German Rigau, Larraitz Uria, and Janyce Wiebe. 2015. https://doi.org/10.18653/V1/S15-2045 Semeval-2015 task 2: Semantic textual similarity, english, spanish and pilot on interpretability ....

    doi:10.18653/v1/s15-2045 2015

  2. [2]

    Cer, Mona T

    Eneko Agirre, Carmen Banea, Claire Cardie, Daniel M. Cer, Mona T. Diab, Aitor Gonzalez - Agirre, Weiwei Guo, Rada Mihalcea, German Rigau, and Janyce Wiebe. 2014. https://doi.org/10.3115/V1/S14-2010 Semeval-2014 task 10: Multilingual semantic textual similarity . In Proceedings of the 8th International Workshop on Semantic Evaluation, SemEval@COLING 2014, ...

    doi:10.3115/v1/s14-2010 2014

  3. [3]

    Cer, Mona T

    Eneko Agirre, Carmen Banea, Daniel M. Cer, Mona T. Diab, Aitor Gonzalez - Agirre, Rada Mihalcea, German Rigau, and Janyce Wiebe. 2016. https://doi.org/10.18653/V1/S16-1081 Semeval-2016 task 1: Semantic textual similarity, monolingual and cross-lingual evaluation . In Proceedings of the 10th International Workshop on Semantic Evaluation, SemEval@NAACL-HLT ...

    doi:10.18653/v1/s16-1081 2016

  4. [4]

    Cer, Mona T

    Eneko Agirre, Daniel M. Cer, Mona T. Diab, and Aitor Gonzalez - Agirre. 2012. https://aclanthology.org/S12-1051/ Semeval-2012 task 6: A pilot on semantic textual similarity . In Proceedings of the 6th International Workshop on Semantic Evaluation, SemEval@NAACL-HLT 2012, pages 385--393. The Association for Computer Linguistics

    work page 2012

  5. [5]

    Cer, Mona T

    Eneko Agirre, Daniel M. Cer, Mona T. Diab, Aitor Gonzalez - Agirre, and Weiwei Guo. 2013. https://aclanthology.org/S13-1004/ *sem 2013 shared task: Semantic textual similarity . In Proceedings of the Second Joint Conference on Lexical and Computational Semantics, *SEM 2013, pages 32--43. Association for Computational Linguistics

    work page 2013

  6. [6]

    Parishad BehnamGhader, Vaibhav Adlakha, Marius Mosbach, Dzmitry Bahdanau, Nicolas Chapados, and Siva Reddy. 2024. Llm2vec: Large language models are secretly powerful text encoders. arXiv preprint arXiv:2404.05961

    arXiv 2024

  7. [7]

    Cer, Mona T

    Daniel M. Cer, Mona T. Diab, Eneko Agirre, I \ n igo Lopez - Gazpio, and Lucia Specia. 2017. https://arxiv.org/abs/1708.00055 Semeval-2017 task 1: Semantic textual similarity - multilingual and cross-lingual focused evaluation . CoRR, abs/1708.00055

    arXiv 2017

  8. [8]

    Sachin Chanchani and Ruihong Huang. 2023. https://doi.org/10.18653/V1/2023.ACL-LONG.882 Composition-contrastive learning for sentence embeddings . In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15836--15848. Association for Computational Linguistics

    doi:10.18653/v1/2023.acl-long.882 2023

Show all 52 references
  1. [9]

    Zifeng Cheng, Zhaoling Chen, Zhiwei Jiang, Yafeng Yin, Shiping Ge, Yuliang Liu, and Qing Gu. 2024 a . Multi-prompting decoder helps better language understanding. CoRR, abs/2406.06279

    2024 arXiv

  2. [10]

    Zifeng Cheng, Zhiwei Jiang, Yafeng Yin, Zhaoling Chen, Cong Wang, Shiping Ge, Qiguo Huang, and Qing Gu. 2024 b . A debiased nearest neighbors framework for multi-label text classification. CoRR, abs/2408.03202

    2024 arXiv

  3. [11]

    Jacob Devlin, Ming - Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. https://doi.org/10.18653/V1/N19-1423 BERT: pre-training of deep bidirectional transformers for language understanding . In Proceedings of the 2019 Conference of the North American Chapter of the Associat...

    2019 doi

  4. [12]

    Bill Dolan and Chris Brockett. 2005. Automatically constructing a corpus of sentential paraphrases. In Third international workshop on paraphrasing (IWP2005)

    2005

  5. [13]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    2024 arXiv

  6. [14]

    Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dari...

    2021

  7. [15]

    Yuchen Fu, Zifeng Cheng, Zhiwei Jiang, Zhonghui Wang, Yafeng Yin, Zhengliang Li, and Qing Gu. 2024. Token prepending: A training-free approach for eliciting better sentence embeddings from llms. CoRR, abs/2412.11556

    2024 arXiv

  8. [16]

    Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021. Simcse: Simple contrastive learning of sentence embeddings. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6894--6910

    2021

  9. [17]

    Minqing Hu and Bing Liu. 2004. Mining and summarizing customer reviews. In Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 168--177

    2004

  10. [18]

    Ting Jiang, Shaohan Huang, Zhongzhi Luan, Deqing Wang, and Fuzhen Zhuang. 2023. Scaling sentence embeddings with large language models. arXiv preprint arXiv:2307.16645

    2023 arXiv

  11. [19]

    Ting Jiang, Jian Jiao, Shaohan Huang, Zihan Zhang, Deqing Wang, Fuzhen Zhuang, Furu Wei, Haizhen Huang, Denvy Deng, and Qi Zhang. 2022. https://doi.org/10.18653/V1/2022.EMNLP-MAIN.603 Promptbert: Improving BERT sentence embeddings with prompts . In Proceedings of the 2022 Conf...

    2022 doi

  12. [20]

    Mingyu Jin, Qinkai Yu, Jingyuan Huang, Qingcheng Zeng, Zhenting Wang, Wenyue Hua, Haiyan Zhao, Kai Mei, Yanda Meng, Kaize Ding, Fan Yang, Mengnan Du, and Yongfeng Zhang. 2024. https://doi.org/10.48550/ARXIV.2404.07066 Exploring concept depth: How large language models acquire ...

    2024 doi arXiv

  13. [21]

    Chankyu Lee, Rajarshi Roy, Mengyao Xu, Jonathan Raiman, Mohammad Shoeybi, Bryan Catanzaro, and Wei Ping. 2024. https://doi.org/10.48550/ARXIV.2405.17428 Nv-embed: Improved techniques for training llms as generalist embedding models . CoRR, abs/2405.17428

    2024 doi arXiv

  14. [22]

    Yibin Lei, Di Wu, Tianyi Zhou, Tao Shen, Yu Cao, Chongyang Tao, and Andrew Yates. 2024. Meta-task prompting elicits embedding from large language models. arXiv preprint arXiv:2402.18458

    2024 arXiv

  15. [23]

    Chak Tou Leong, Yi Cheng, Jiashuo Wang, Jian Wang, and Wenjie Li. 2023. Self-detoxifying language models via toxification reversal. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023 , pages 4433--4449. Association for Computa...

    2023

  16. [24]

    Vi \' e gas, Hanspeter Pfister, and Martin Wattenberg

    Kenneth Li, Oam Patel, Fernanda B. Vi \' e gas, Hanspeter Pfister, and Martin Wattenberg. 2023. http://papers.nips.cc/paper\_files/paper/2023/hash/81b8390039b7302c909cb769f8b6cd93-Abstract-Conference.html Inference-time intervention: Eliciting truthful answers from a language ...

    2023

  17. [25]

    Xianming Li and Jing Li. 2024. Bellm: Backward dependency enhanced large language model for sentence embeddings. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Pa...

    2024

  18. [26]

    Xianming Li, Zongxi Li, Jing Li, Haoran Xie, and Qing Li. 2025. ESE: espresso sentence embeddings. In The Thirteenth International Conference on Learning Representations, ICLR 2025

    2025

  19. [27]

    Tian Yu Liu, Matthew Trager, Alessandro Achille, Pramuditha Perera, Luca Zancato, and Stefano Soatto. 2024 a . Meaning representations from trajectories in autoregressive models. In The Twelfth International Conference on Learning Representations

    2024

  20. [28]

    Zhu Liu, Cunliang Kong, Ying Liu, and Maosong Sun. 2024 b . https://doi.org/10.48550/ARXIV.2403.01509 Fantastic semantics and where to find them: Investigating which layers of generative llms reflect lexical semantics . CoRR, abs/2403.01509

    2024 doi arXiv

  21. [29]

    Marco Marelli, Stefano Menini, Marco Baroni, Luisa Bentivogli, Raffaella Bernardi, and Roberto Zamparelli. 2014. http://www.lrec-conf.org/proceedings/lrec2014/summaries/363.html A SICK cure for the evaluation of compositional distributional semantic models . In Proceedings of ...

    2014

  22. [30]

    Niklas Muennighoff, Hongjin Su, Liang Wang, Nan Yang, Furu Wei, Tao Yu, Amanpreet Singh, and Douwe Kiela. 2024. https://doi.org/10.48550/ARXIV.2402.09906 Generative representational instruction tuning . CoRR, abs/2402.09906

    2024 doi arXiv

  23. [31]

    Niklas Muennighoff, Nouamane Tazi, Lo \" c Magne, and Nils Reimers. 2022. Mteb: Massive text embedding benchmark. arXiv preprint arXiv:2210.07316

    2022 arXiv

  24. [32]

    Jianmo Ni, Gustavo Hernandez Abrego, Noah Constant, Ji Ma, Keith Hall, Daniel Cer, and Yinfei Yang. 2022 a . Sentence-t5: Scalable sentence encoders from pre-trained text-to-text models. In Findings of the Association for Computational Linguistics: ACL 2022, pages 1864--1874

    2022

  25. [33]

    Hall, Daniel Cer, and Yinfei Yang

    Jianmo Ni, Gustavo Hern \' a ndez \' A brego, Noah Constant, Ji Ma, Keith B. Hall, Daniel Cer, and Yinfei Yang. 2022 b . https://doi.org/10.18653/V1/2022.FINDINGS-ACL.146 Sentence-t5: Scalable sentence encoders from pre-trained text-to-text models . In Findings of the Associat...

    2022 doi

  26. [34]

    Zhijie Nie, Zhangchi Feng, Mingxin Li, Cunwang Zhang, Yanzhao Zhang, Dingkun Long, and Richong Zhang. 2024. When text embedding meets large language model: A comprehensive survey. CoRR, abs/2412.09165

    2024

  27. [35]

    Bo Pang and Lillian Lee. 2004. A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04), pages 271--278

    2004

  28. [36]

    Bo Pang and Lillian Lee. 2005. Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL’05), pages 115--124

    2005

  29. [37]

    Nina Rimsky, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Matt Turner. 2024. https://doi.org/10.18653/V1/2024.ACL-LONG.828 Steering llama 2 via contrastive activation addition . In Proceedings of the 62nd Annual Meeting of the Association for Computatio...

    2024 doi

  30. [38]

    Fei Shen, Xin Jiang, Xin He, Hu Ye, Cong Wang, Xiaoyu Du, Zechao Li, and Jinhui Tang. 2025. Imagdressing-v1: Customizable virtual dressing. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 6795--6804

    2025

  31. [39]

    Fei Shen and Jinhui Tang. 2024. Imagpose: A unified conditional framework for pose-guided person generation. Advances in neural information processing systems, 37:6246--6266

    2024

  32. [40]

    Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language...

    2013

  33. [41]

    Jacob Mitchell Springer, Suhas Kotha, Daniel Fried, Graham Neubig, and Aditi Raghunathan. 2024. Repetition improves language model embeddings. arXiv preprint arXiv:2402.15449

    2024 arXiv

  34. [42]

    Smith, Luke Zettlemoyer, and Tao Yu

    Hongjin Su, Weijia Shi, Jungo Kasai, Yizhong Wang, Yushi Hu, Mari Ostendorf, Wen - tau Yih, Noah A. Smith, Luke Zettlemoyer, and Tao Yu. 2023. https://doi.org/10.18653/V1/2023.FINDINGS-ACL.71 One embedder, any task: Instruction-finetuned text embeddings . In Findings of the As...

    2023 doi

  35. [43]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

    2023 arXiv

  36. [44]

    Gomez, Lukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. https://proceedings.neurips.cc/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html Attention is all you need . In Advances in Neural I...

    2017

  37. [45]

    Ellen M Voorhees and Dawn M Tice. 2000. Building a question answering test collection. In Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, pages 200--207

    2000

  38. [46]

    Cong Wang, Kuan Tian, Yonghang Guan, Fei Shen, Zhiwei Jiang, Qing Gu, and Jun Zhang. 2025. Ensembling diffusion models via adaptive feature aggregation. In The Thirteenth International Conference on Learning Representations, ICLR 2025

    2025

  39. [47]

    Janyce Wiebe, Theresa Wilson, and Claire Cardie. 2005. Annotating expressions of opinions and emotions in language. Language resources and evaluation, 39:165--210

    2005

  40. [48]

    Bowen Zhang, Kehua Chang, and Chunping Li. 2024. Simple techniques for enhancing sentence embeddings in generative language models. arXiv preprint arXiv:2404.03921

    2024 arXiv

  41. [49]

    Shengyao Zhuang, Shuai Wang, Bevan Koopman, and Guido Zuccon. 2024. Starbucks: Improved training for 2d matryoshka embeddings. CoRR, abs/2410.13230

    2024 arXiv

  42. [50]

    Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, J

    Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann - Kathrin Dombrowski, Shashwat Goel, Nathaniel Li, Michael J. Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, J. Zi...

    2023 doi arXiv

  43. [51]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  44. [52]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.