Pith. sign in

REVIEW 3 major objections 4 minor 45 references

SemCSE: Semantic Contrastive Sentence Embeddings Using LLM-Generated Summaries For Scientific Abstracts

T0 review · 3 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SemCSE trains scientific text embeddings by contrasting LLM-generated summaries of the same abstract, rather than relying on citation links, and reports that this captures true semantic content better than citation-based supervision.

desk verdict Plausible method, honest ablations, but the main semantic benchmark is contaminated by train/eval overlap, so the headline superiority claim is unsupported. read the letter →

arxiv 2507.13105 v1 pith:M3DYGVYV submitted 2025-07-17 cs.CL cs.IRcs.LG

Marc Brinner , Sina Zarriess This is my paper
classification cs.CLcs.IRcs.LG
keywords semanticembeddingsscientificabstractscontrastivelearningLLM-generatedsummariestripletlosscitation-basedsupervisionbenchmarkSciRepEval
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SemCSE claims that scientific text embeddings are best trained by contrasting LLM-generated summaries of the same abstract, rather than by relying on citation links as a proxy for semantic similarity. The method is unsupervised, requiring only a corpus of abstracts and a small LLM to produce multiple summaries per abstract. On a new benchmark of title-abstract matching, abstract-segment matching, query matching, and semantic clustering, SemCSE outperforms all evaluated models, including much larger ones; on the SciRepEval benchmark it achieves the second-best overall score and the best classification score. If correct, this shows that direct semantic supervision from summaries produces embeddings that track content better than citation-based supervision, at lower cost than large general-purpose models.

What carries the argument

The load-bearing mechanism is the triplet loss over LLM-generated summary pairs. For each abstract, Llama-3-8B is prompted with five different continuations to produce a set of summarizing sentences, and the model is trained to embed two summaries of the same abstract closer together than a randomly sampled summary, with loss $L(e_a, e_+, e_-) = \max(0, d(e_a, e_+) - d(e_a, e_-) + 1)$ using Euclidean distance. To force the model to relate different surface forms of the same paper, positives are occasionally titles (15%) or sentences sampled from the abstract (35%) instead of a second summary. This contrasts with unsupervised SimCSE, which uses the same sentence as anchor and positive and only obtains variance from dropout; SemCSE's distinct-but-related inputs provide a harder and more informative alignment signal, supported by an ablation showing that the 'Same Input' variant collapses on the semantic benchmark.

What would settle it

Take the title-abstract matching and abstract-segments tasks and generate the test abstracts with a different LLM under a paraphrasing instruction that changes wording but keeps meaning; if SemCSE's average rank rises to the level of citation-trained baselines (SPECTER around 10.25 and 12.23), the claim that it captures semantic content better than citation-based models would be falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is that two LLM-generated summaries of the same scientific abstract form a better contrastive training pair than the dropout-distorted copies used by SimCSE or the citation-linked papers used by SPECTER-type models. Trained with a triplet loss in Euclidean space on top of SciDeBERTa, SemCSE learns to place semantically related summaries close together and unrelated ones apart, and the authors report that this transfers to titles, abstract sentences, and entire abstracts. The strongest evidence is on the proposed semantic benchmark, where SemCSE reaches average ranks of 2.47 for title-abstract matching and 2.68 for abstract-segments matching, beating the 7.9B-parameter generalist NvEmbed model that scores 3.38 and 3.84, and a clustering score of 0.739, the highest among all models. On SciRepEval, SemCSE attains 65.76 overall, second only to NvEmbed (66.19) and ahead of all citation-trained scientific models, with the best classification accuracy (69.52). The authors interpret these results as evidence that summary-contrastive training produces embeddings that reflect genuine semantic content rather than citation-network artifacts.

Load-bearing premise

SemCSE's main evidence rests on the assumption that the new benchmark measures general semantic understanding, while the model was directly trained to align titles, abstract sentences, and summaries of the same paper, so its scores on the matching tasks may reflect the training signal rather than an independent semantic capability.

Editorial extensions

If this is right

  • Scientific text embeddings can be trained without citation networks or human labels, using only abstracts plus an LLM summarizer.
  • Literature search and matching tasks that rely on semantic content, such as finding related papers by topic, should improve with SemCSE-style models.
  • The proposed four-task benchmark offers a way to compare embeddings on semantic content, complementing citation-based benchmarks like SciRepEval.
  • The training scheme is offered as a general recipe beyond science, since it only requires a corpus of short texts and an LLM that can summarize them.
  • SemCSE's gains on classification and clustering suggest that semantically trained embeddings better separate thematic categories of papers.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Going beyond the paper: because titles and abstract sentences are used as positives, the new benchmark's matching tasks partially overlap with the training signal; a harder test would paraphrase articles and see whether the advantage holds.
  • Going beyond the paper: combining SemCSE's summary signal with citation-based supervision might yield embeddings that are both semantically coherent and citation-aware, potentially improving SciRepEval scores further.
  • Going beyond the paper: the same summary-contrastive recipe should transfer to other domains such as legal or medical texts, and a natural next experiment is to test it on a non-scientific corpus.
  • Going beyond the paper: the benchmark's query-matching task uses LLM-generated queries, which may favor models trained on LLM-like text; adding human queries would test retrieval behavior for real users.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces SemCSE, an unsupervised contrastive learning method for scientific text embeddings. The method generates multiple Llama-3-8B summaries of each scientific abstract and trains a SciDeBERTa encoder with a triplet loss, using summaries as anchor-positive pairs and occasionally sampling paper titles or abstract sentences as additional positives (Section 4.2). The authors propose a new four-task 'semantic embedding benchmark' (title-abstract matching, abstract-segments matching, query matching, and semantic clustering) and report state-of-the-art results on it, as well as second-best overall performance on SciRepEval, arguing that semantic supervision is a better training signal than citation-based supervision.

Significance. The core idea is attractive: LLM-generated summaries provide a scalable, unsupervised source of semantically related positive pairs, and the paper ships code, generated training data, and model checkpoints, which is exemplary for reproducibility. If the evaluation were clean, the reported gains over citation-based models on matching tasks would be a valuable contribution. The SciDocs MAG clustering result and the anisotropy analysis provide supporting evidence that is less directly affected by the training/evaluation overlap. However, the paper's central benchmark appears to be contaminated by training data overlap, and the SciRepEval advantage is small, so the overall significance hinges on whether a clean re-evaluation confirms the claims.

major comments (3)
  1. [Appendix A.1, §4.2, §5.2, Table 1, Table 3] The semantic evaluation benchmark is built from 500 random samples per dataset drawn from the train splits of 12 SciRepEval datasets, while SemCSE training samples 350K examples from the train splits of six of those datasets. Five datasets (high_influence_cite, mesh_descriptors, fos, same_author, search) appear in both. The paper does not report any deduplication between training and evaluation, so exact or near-duplicate title-abstract pairs can appear in both. This concern is reinforced by the paper's own ablation: Table 3 shows that removing title/abstract positives from training ('Just Summaries') degrades title-abstract matching from 2.47 to 6.04 and abstract-segments from 2.68 to 3.52, while the full model is trained with exactly those positives (Section 4.2). Thus the two headline tasks of the new benchmark directly measure the training objective, and the reported advantage over citation-based models in Table 1 may reflect memorization rather than general semantic capability. Please re-evaluate on a held-out split that is disjoint from training, ideally using the test splits of the SciRepEval datasets or a newly sampled set with explicit deduplication, and report both the full model and the 'Just Summaries' variant on that disjoint set.
  2. [§6.1, Table 2] The abstract and Section 6.1 claim state-of-the-art performance among models of comparable size on SciRepEval, but the margin over the strongest citation-based baseline is small: SemCSE scores 65.76 versus 65.46 for SPECTER2 base, while NvEmbed-V2 (66.19) scores higher overall. The paper reports neither variance nor significance tests for the aggregated averages, which are computed by averaging over many tasks and metrics (Appendix A.4). A 0.3-point gap could plausibly be within run-to-run noise; please report per-task dispersion, repeated runs, or a significance test to support the SOTA claim, or soften the claim.
  3. [§5.1, Appendix A.3] The generalization experiment in Section 5.1 uses a 'validation dataset comprising 900 title-abstract pairs and corresponding paper summaries,' and Appendix A.3 states that early stopping is performed on this set. The provenance of this validation set is not specified. If these 900 pairs are drawn from the same SciRepEval train splits used for training, then early stopping may select a checkpoint specialized to the training distribution, and the reported ranks (e.g., 1.542) would overstate generalization to unseen abstracts. Please state the source of the validation set and ensure it is disjoint from the training samples.
minor comments (4)
  1. [§3] The text says 'm is a margin hyperparameter' but the triplet loss equation uses a fixed '1' with no m; please clarify whether the margin is tunable or simply set to 1.
  2. [Table 3] The header structure of Table 3 is confusing: 'Dataset' appears to merge model-variant labels (Full, Just Summaries) with dataset sizes; please separate the columns clearly.
  3. [Abstract] The abstract states that the objective 'ensures that the model captures the true semantic content of a text'; this overclaims, since the objective only encourages alignment of generated summaries with their source. A more measured phrasing would be appropriate.
  4. [Figure 1] The t-SNE figure is described as colored by topic labels, but no legend or label list is visible in the text; please add a legend or describe the color mapping in the caption.

Circularity Check

2 steps flagged · score 6.0 of 10

Semantic benchmark re-measures the training objective: title-abstract and abstract-segment matching are the exact document-representation relations trained in Section 4.2, and the benchmark is sampled from the same SciRepEval train splits used for training.

  1. self definitional [Section 4.2 (Model Training) and Section 5.2 (Semantic Embedding Benchmark)]
    "While we only use generated summaries as anchors, we increase variance within the positives by also sampling paper titles or sentences from the abstracts in 15% and 35% of cases, respectively. This forces the model to learn meaningful relationships between different representations of the same document, leading to a deeper semantic understanding. ... The Title–Abstract Matching task measures a model’s ability to match a paper’s title with its corresponding abstract."

    The semantic benchmark's first two tasks, title-abstract matching and abstract-segments consistency, evaluate exactly the document-representation relation that Section 4.2 trains: the model is explicitly trained with title and abstract-sentence positives so that titles, abstract sentences, and summaries of the same paper are embedded close together. Reporting SOTA ranks on these tasks as evidence of capturing 'true semantic content' is circular because the benchmark re-measures the training objective rather than an independent semantic capability. The paper's own ablation confirms this dependence: the 'Just Summaries' variant, which removes title/abstract positives, degrades Title-Abstr.

  2. fitted input called prediction [Section 4.1 (Dataset) and Appendix A.1 (Dataset Creation)]
    "From these corpora, we sample 350K paper titles and abstracts spanning a variety of domains (for details, see Appendix A). ... The dataset used in the semantic evaluation benchmark uses 500 random samples from the train split of each of the datasets used in the SciRepEval benchmark: 'relish', 'high_influence_cite', 'mesh_descriptors', 'biomimicry', 'drsm', 'cite_prediction', 'fos', 'paper_reviewer_matching', 'peer_review_score_hIndex', 'same_author', 'search', 'tweet_mentions'."

    SemCSE's 350K training set is sampled from the train splits of SciRepEval datasets including high_influence_cite, mesh_descriptors, fos, same_author, search, and cite_prediction_new, while the semantic benchmark is constructed by taking 500 random samples from the train splits of the same dataset family. The evaluation is therefore not held out from the training pool: title-abstract and abstract-segment pairs used for scoring can, and for datasets whose train split is fully sampled necessarily do, coincide with training instances. The paper presents the resulting matching ranks as validation of its method, but the prediction is made on inputs that are part of the fitted data, so the reported SOTA scores conflate memorization with semantic generalization.

full rationale

The core training contribution—using LLM-generated summaries as anchors with distinct positive summary pairs—is a non-circular method, and the SciRepEval results (Table 2) and SciDocs clustering (Table 1) provide some external, held-out signal. However, the paper's own newly proposed semantic benchmark, which is the primary evidence for the 'true semantic content' claim, is not independent of the training objective. Section 4.2 explicitly trains the model to pull titles and abstract sentences of the same paper toward summary anchors, and Section 5.2's first two tasks measure exactly that title/abstract alignment. Table 3's 'Just Summaries' ablation shows those benchmark scores collapse when the title/abstract positives are removed (Title-Abstr. 2.47 to 6.04; Abstr.-Segments 2.68 to 3.52). Additionally, Appendix A.1 draws benchmark items from the train split of the same SciRepEval datasets used to build the training set, so the matching scores may include memorized instances. Because two of the four benchmark tasks reduce to the training signal, the paper's headline semantic-separation claim is partially circular; the external SciRepEval and clustering results keep it from being fully so. No load-bearing self-citation was found.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The method rests on the domain assumption that LLM summaries of one abstract are valid semantic positives, and that the new benchmark tasks isolate semantic content rather than the training signal. The evaluation is partly circular because titles and abstract sentences are used as training positives, so the benchmark measures a trained skill. No new physical or linguistic entities are introduced.

free parameters (5)
  • positive mixing ratio = 15% titles, 35% abstract sentences
    Manually set in Section 4.2; the ablation changes this ratio and materially changes title-abstract and abstract-segment ranks, showing the result depends on it.
  • L2 regularization weight = 1/250
    Appendix A.3; affects embedding compactness and anisotropy.
  • triplet margin m = 1
    Section 3; authors state precise value is not important due to scale invariance, but it is still a chosen hyperparameter.
  • cosine temperature tau = 0.07
    Appendix A.5; used in the cosine contrastive ablation.
  • number of summaries per text = 3 continuations per prompt, 5 prompts
    Section 4.1; the amount and type of summary diversity are design choices affecting the training signal.
assumptions (4)
  • standard math Triplet loss and Euclidean distance are valid for embedding training.
    Hoffer and Ailon (2015); a standard, uncontroversial background assumption.
  • domain assumption LLM-generated summaries of the same abstract are valid positive pairs.
    Core to the method (Section 3); the paper argues summarization is easy and systematic errors preserve relatedness (Appendix B.1), but no human evaluation is provided.
  • domain assumption The new benchmark tasks measure semantic understanding.
    Section 5.2; the tasks are constructed by the authors and overlap with the training positives in Section 4.2, so this is a strong assumption.
  • domain assumption SciDocs MAG categories reflect semantic relatedness.
    Section 5.2 clustering task; no validation that MAG topic labels correspond to embedding-space semantics.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SemCSE: Semantic Contrastive Sentence Embeddings Using LLM-Generated Summaries For Scientific Abstracts." pith.science (2026). https://pith.science/paper/M3DYGVYV

@misc{pith2026250713105,
  author       = {Pith},
  title        = {Pith review of: SemCSE: Semantic Contrastive Sentence Embeddings Using LLM-Generated Summaries For Scientific Abstracts},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/M3DYGVYV}},
  note         = {Machine review of arXiv:2507.13105}
}
read the original abstract

We introduce SemCSE, an unsupervised method for learning semantic embeddings of scientific texts. Building on recent advances in contrastive learning for text embeddings, our approach leverages LLM-generated summaries of scientific abstracts to train a model that positions semantically related summaries closer together in the embedding space. This resulting objective ensures that the model captures the true semantic content of a text, in contrast to traditional citation-based approaches that do not necessarily reflect semantic similarity. To validate this, we propose a novel benchmark designed to assess a model's ability to understand and encode the semantic content of scientific texts, demonstrating that our method enforces a stronger semantic separation within the embedding space. Additionally, we evaluate SemCSE on the comprehensive SciRepEval benchmark for scientific text embeddings, where it achieves state-of-the-art performance among models of its size, thus highlighting the benefits of a semantically focused training approach.

Figures

Figures reproduced from arXiv: 2507.13105 by the authors.

Figure 1
Figure 1. A t-SNE visualization of embeddings generated for scientific papers from the SciDocs MAG dataset [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Variance of the SciDocs MAG embedding space explained by the first 50 principal components for the [PITH_FULL_IMAGE:figures/full_fig_p013_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

45 extracted references · 18 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Acuna, Tulakan Ruangrong, and Konrad Kording

    Titipat Achakulvisut, Daniel E. Acuna, Tulakan Ruangrong, and Konrad Kording. 2016. https://doi.org/10.1371/journal.pone.0158423 Science concierge: A fast content-based recommendation system for scientific publications . 11(7):e0158423

    work page doi:10.1371/journal.pone.0158423 2016

  4. [4]

    Iz Beltagy, Kyle Lo, and Arman Cohan. 2019. https://doi.org/10.18653/v1/D19-1371 SciBERT : A pretrained language model for scientific text . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing ( EMNLP - IJCNLP ) , pages 3615--3620. Association f...

    doi:10.18653/v1/d19-1371 2019

  5. [5]

    Chandra Bhagavatula, Sergey Feldman, Russell Power, and Waleed Ammar. 2018. https://doi.org/10.18653/v1/N18-1022 Content-based citation recommendation . In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 238--251. Association for C...

    doi:10.18653/v1/n18-1022 2018

  6. [6]

    Lutz Bornmann, Robin Haunschild, and Rüdiger Mutz. 2021. https://doi.org/10.1057/s41599-021-00903-w Growth rates of modern science: a latent piecewise growth curve approach to model publication numbers from established and new literature databases . 8(1):1--15. Publisher: Palgrave

    doi:10.1057/s41599-021-00903-w 2021

  7. [7]

    Marc Brinner, Tarek Al Mustafa, and Sina Zarrieß. 2025. https://arxiv.org/abs/2503.22006 Enhancing domain-specific encoder models with llm-generated data: How to leverage ontologies, and how to do without them . Preprint, arXiv:2503.22006

    work page arXiv 2025

  8. [8]

    Haonan Chen, Liang Wang, Nan Yang, Yutao Zhu, Ziliang Zhao, Furu Wei, and Zhicheng Dou. 2025. https://doi.org/10.18653/v1/2025.naacl-long.64 Little giants: Synthesizing high-quality embedding data at scale . In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technol...

    work page doi:10.18653/v1/2025.naacl-long.64 2025

Show all 45 references
  1. [10]

    Arman Cohan, Sergey Feldman, Iz Beltagy, Doug Downey, and Daniel Weld. 2020 b . https://doi.org/10.18653/v1/2020.acl-main.207 SPECTER : Document-level representation learning using citation-informed transformers . In Proceedings of the 58th Annual Meeting of the Association fo...

    2020 doi

  2. [11]

    Manuel Frank and Haithem Afli. 2024. https://arxiv.org/abs/2411.04914 Gase: Generatively augmented sentence encoding . Preprint, arXiv:2411.04914

    2024 arXiv

  3. [12]

    Jun Gao, Di He, Xu Tan, Tao Qin, Liwei Wang, and Tie-Yan Liu. 2019. https://arxiv.org/abs/1907.12009 Representation degeneration problem in training natural language generation models . Preprint, arXiv:1907.12009

    2019 arXiv

  4. [13]

    Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021. https://doi.org/10.18653/v1/2021.emnlp-main.552 S im CSE : Simple contrastive learning of sentence embeddings . In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6894--6910, Online ...

    2021 doi

  5. [14]

    Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Meng Wang, and Haofen Wang. 2024. https://arxiv.org/abs/2312.10997 Retrieval-augmented generation for large language models: A survey . Preprint, arXiv:2312.10997

    2024 arXiv

  6. [15]

    Aaron Grattafiori et al. 2024. https://arxiv.org/abs/2407.21783 The llama 3 herd of models . Preprint, arXiv:2407.21783

    2024 arXiv

  7. [16]

    Michael Günther, Jackmin Ong, Isabelle Mohr, Alaeddine Abdessalem, Tanguy Abel, Mohammad Kalim Akram, Susana Guzman, Georgios Mastrapas, Saba Sturua, Bo Wang, Maximilian Werk, Nan Wang, and Han Xiao. 2023. https://arxiv.org/abs/2310.19923 Jina embeddings 2: 8192-token general-...

    2023 arXiv

  8. [17]

    Amir Hadifar, Lucas Sterckx, Thomas Demeester, and Chris Develder. 2019. https://doi.org/10.18653/v1/W19-4322 A self-training approach for short text clustering . In Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019), pages 194--199, Florence, I...

    2019 doi

  9. [18]

    Birger Hjørland and Hanne Albrechtsen. 1995. https://doi.org/10.1002/(SICI)1097-4571(199507)46:6<400::AID-ASI2>3.0.CO;2-Y Toward a new horizon in information science: Domain-analysis . 46(6):400--425

    1995 doi

  10. [19]

    Elad Hoffer and Nir Ailon. 2015. Deep metric learning using triplet network. In Similarity-Based Pattern Recognition, pages 84--92, Cham. Springer International Publishing

    2015

  11. [20]

    Junjie Huang, Duyu Tang, Wanjun Zhong, Shuai Lu, Linjun Shou, Ming Gong, Daxin Jiang, and Nan Duan. 2021. https://doi.org/10.18653/v1/2021.findings-emnlp.23 W hitening BERT : An easy unsupervised sentence embedding approach . In Findings of the Association for Computational Li...

    2021 doi

  12. [21]

    Anshul Kanakia, Zhihong Shen, Darrin Eide, and Kuansan Wang. 2019. https://doi.org/10.1145/3308558.3313700 A scalable hybrid research paper recommender system for microsoft academic . In The World Wide Web Conference, pages 2893--2899

    2019

  13. [22]

    Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.550 Dense passage retrieval for open-domain question answering . In Proceedings of the 2020 Conference on Empiric...

    2020 doi

  14. [23]

    Eunhui Kim, Yuna Jeong, and Myung-Seok Choi. 2023. https://doi.org/10.1109/ACCESS.2023.3341612 Medibiodeberta: Biomedical language model with continuous learning and intermediate fine-tuning . IEEE Access, 11:141036--141044

    2023

  15. [24]

    Chankyu Lee, Rajarshi Roy, Mengyao Xu, Jonathan Raiman, Mohammad Shoeybi, Bryan Catanzaro, and Wei Ping. 2025. https://arxiv.org/abs/2405.17428 Nv-embed: Improved techniques for training llms as generalist embedding models . Preprint, arXiv:2405.17428

    2025 arXiv

  16. [25]

    Jinhyuk Lee, Zhuyun Dai, Xiaoqi Ren, Blair Chen, Daniel Cer, Jeremy R. Cole, Kai Hui, Michael Boratko, Rajvi Kapadia, Wen Ding, Yi Luan, Sai Meher Karthik Duddu, Gustavo Hernandez Abrego, Weiqiang Shi, Nithi Gupta, Aditya Kusupati, Prateek Jain, Siddhartha Reddy Jonnalagadda, ...

    2024 arXiv

  17. [26]

    Bohan Li, Hao Zhou, Junxian He, Mingxuan Wang, Yiming Yang, and Lei Li. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.733 On the sentence embeddings from pre-trained language models . In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing...

    2020 doi

  18. [27]

    Xianming Li and Jing Li. 2024. https://doi.org/10.18653/v1/2024.acl-long.101 A o E : Angle-optimized embeddings for semantic textual similarity . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1825--183...

    2024 doi

  19. [28]

    Hongjiang Lv, Zhibin Niu, Wei Han, and Xiang Li. 2024. https://doi.org/10.1007/s12650-024-01010-z Can GPT embeddings enhance visual exploration of literature datasets? A case study on isostatic pressing research . Journal of Visualization

    2024 doi

  20. [29]

    H. J. Meijer, J. Truong, and R. Karimi. 2021. https://doi.org/10.48550/arXiv.2107.05151 Document embedding for scientific articles: Efficacy of word embeddings vs TFIDF . Preprint, arxiv:2107.05151 [cs]

    2021 doi arXiv

  21. [30]

    Mistral AI . 2025. https://mistral.ai/news/mistral-small-3-1 Mistral Small 3.1 Mistral AI

    2025

  22. [31]

    Sheshera Mysore, Arman Cohan, and Tom Hope. 2022. https://doi.org/10.18653/v1/2022.naacl-main.331 Multi-vector models with textual guidance for fine-grained scientific document similarity . In Proceedings of the 2022 Conference of the North American Chapter of the Association ...

    2022 doi

  23. [32]

    Malte Ostendorff, Nils Rethmeier, Isabelle Augenstein, Bela Gipp, and Georg Rehm. 2022. https://arxiv.org/abs/2202.06671 [cs] Neighborhood contrastive learning for scientific document representations with citation embeddings . Preprint, arxiv:2202.06671 [cs]

    2022 arXiv

  24. [33]

    Simon Pasternack. 1969. https://doi.org/10.1126/science.164.3880.669 The scientific enterprise: Public knowledge. an essay concerning the social dimension of science. j. m. ziman. cambridge university press, new york, 1968. xii + 154 pp. cloth, \ 3.95; paper, \ 1.95. 164(3880)...

    1969 doi

  25. [34]

    Nils Reimers and Iryna Gurevych. 2019. https://doi.org/10.18653/v1/D19-1410 Sentence- BERT : Sentence embeddings using S iamese BERT -networks . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference...

    2019 doi

  26. [35]

    Amanpreet Singh, Mike D'Arcy, Arman Cohan, Doug Downey, and Sergey Feldman. 2023. https://arxiv.org/abs/2211.13308 [cs] SciRepEval : A multi-format benchmark for scientific document representations . Preprint, arxiv:2211.13308 [cs]

    2023 arXiv

  27. [36]

    Saba Sturua, Isabelle Mohr, Mohammad Kalim Akram, Michael Günther, Bo Wang, Markus Krimmel, Feng Wang, Georgios Mastrapas, Andreas Koukounas, Andreas Koukounas, Nan Wang, and Han Xiao. 2024. https://arxiv.org/abs/2409.10173 jina-embeddings-v3: Multilingual embeddings with task...

    2024 arXiv

  28. [37]

    Alvin Subakti, Hendri Murfi, and Nora Hariadi. 2022. https://doi.org/10.1186/s40537-022-00564-9 The performance of BERT as data representation of text clustering . Journal of Big Data, 9(1):15

    2022 doi

  29. [38]

    Shicheng Tan, Tao Zhang, Shu Zhao, and Yanping Zhang. 2023. https://doi.org/10.1007/s11192-023-04782-7 Self-supervised scientific document recommendation based on contrastive learning . 128(9):5027--5049

    2023 doi

  30. [39]

    Hongyin Tang, Xingwu Sun, Beihong Jin, Jingang Wang, Fuzheng Zhang, and Wei Wu. 2021. https://doi.org/10.18653/v1/2021.acl-long.392 Improving document representations by generating pseudo query embeddings for dense retrieval . In Proceedings of the 59th Annual Meeting of the A...

    2021 doi

  31. [40]

    Raghuveer Thirukovalluru and Bhuwan Dhingra. 2025. https://doi.org/10.18653/v1/2025.findings-naacl.122 G en EOL : Harnessing the generative power of LLM s for training-free sentence embeddings . In Findings of the Association for Computational Linguistics: NAACL 2025, pages 22...

    2025 doi

  32. [41]

    Oleg Vasilyev, Randy Sawaya, and John Bohannon. 2025. https://doi.org/10.18653/v1/2025.naacl-short.28 Preserving multilingual quality while tuning query encoder on E nglish only . In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association f...

    2025 doi

  33. [42]

    Jianyou (Andre) Wang, Kaicheng Wang, Xiaoyue Wang, Prudhviraj Naidu, Leon Bergen, and Ramamohan Paturi. 2023. https://proceedings.neurips.cc/paper_files/paper/2023/file/78f9c04bdcb06f1ada3902912d8b64ba-Paper-Datasets_and_Benchmarks.pdf Scientific document retrieval using multi...

    2023

  34. [43]

    Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. 2024. https://doi.org/10.18653/v1/2024.acl-long.642 Improving text embeddings with large language models . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguisti...

    2024 doi

  35. [44]

    Tongzhou Wang and Phillip Isola. 2020. Understanding contrastive representation learning through alignment and uniformity on the hypersphere. In Proceedings of the 37th International Conference on Machine Learning, ICML'20. JMLR.org

    2020

  36. [45]

    Zhuofeng Wu, Sinong Wang, Jiatao Gu, Madian Khabsa, Fei Sun, and Hao Ma. 2020. https://arxiv.org/abs/2012.15466 Clear: Contrastive learning for sentence representation . Preprint, arXiv:2012.15466

    2020 arXiv

  37. [46]

    Borui Xu, Yao Chen, Zeyi Wen, Weiguo Liu, and Bingsheng He. 2025. https://aclanthology.org/2025.naacl-long.253/ Evaluating small language models for news summarization: Implications and factors influencing performance . In Proceedings of the 2025 Conference of the Nations of t...

    2025

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.