REVIEW 49 cited by
Improving Text Embeddings with Large Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
In this paper, we introduce a novel and simple method for obtaining high-quality text embeddings using only synthetic data and less than 1k training steps. Unlike existing methods that often depend on multi-stage intermediate pre-training with billions of weakly-supervised text pairs, followed by fine-tuning with a few labeled datasets, our method does not require building complex training pipelines or relying on manually collected datasets that are often constrained by task diversity and language coverage. We leverage proprietary LLMs to generate diverse synthetic data for hundreds of thousands of text embedding tasks across 93 languages. We then fine-tune open-source decoder-only LLMs on the synthetic data using standard contrastive loss. Experiments demonstrate that our method achieves strong performance on highly competitive text embedding benchmarks without using any labeled data. Furthermore, when fine-tuned with a mixture of synthetic and labeled data, our model sets new state-of-the-art results on the BEIR and MTEB benchmarks.
Forward citations
Cited by 49 Pith papers
-
Vortex: Overcoming Memory Capacity Limitations in GPU-Accelerated Large-Scale Data Analytics
By routing data through idle IO paths of neighboring GPUs, Vortex lets a single GPU run analytics on datasets exceeding its memory at 140GB/s aggregate transfer and beats CPU baselines.
-
UEmbed: Unified Sparse and Dense Multimodal Embeddings
UEmbed uses 16 special tokens over a partitioned vocabulary to make a decoder-only multimodal model emit dense and sparse embeddings in one forward pass; the 9B model scores 71.8 dense / 71.0 sparse on MMEB-v2.
-
Bifrost: Empowering Pretrained Language Model with Fallibility Representation for Log-Based Fault Diagnosis
Bifrost fine-tunes BART-Base on logs with three self-supervised contrastive tasks, beating general PLMs by roughly 10-20% on anomaly detection, root-cause localization, and fault identification.
-
No Thoughts Just AI: Biased LLM Hiring Recommendations Alter Human Decision Making and Limit Human Autonomy
People largely adopt racially biased AI hiring recommendations, selecting the AI-favored group up to 90% of the time; prior IAT exposure may reduce stereotype-congruent choices by about 13%.
-
NER Retriever: Zero-Shot Named Entity Retrieval with Type-Aware Embeddings
Mid-layer LLM value vectors, projected through a contrastively trained MLP, enable zero-shot retrieval of documents by ad-hoc entity type.
-
THEME: Enhancing Thematic Investing with Semantic Stock Representations and Temporal Dynamics
A hierarchical contrastive learning framework that aligns stocks with theme descriptions and refines embeddings with short-term return signals improves thematic retrieval and backtested portfolio metrics.
-
BiXSE: Improving Dense Retrieval via Probabilistic Graded Relevance Distillation
BiXSE trains dense retrievers with binary cross-entropy on LLM-generated graded relevance probabilities, consistently beating InfoNCE and roughly matching pairwise ranking baselines.
-
BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent
BrowseComp-Plus is a fixed, human-verified document corpus with supporting and negative documents that allows controlled, reproducible evaluation of deep-research agents and their retrievers.
-
Learning Robust Negation Text Representations
Finetuning text encoders on taxonomy-guided, LLM-generated negation and hedging triples substantially improves negation benchmark performance while keeping general embedding quality roughly intact.
-
Text-ADBench: Text Anomaly Detection Benchmark based on LLMs Embedding
LLM embeddings improve text anomaly detection, shallow detectors match deep ones only under oracle embedding selection, and AUROC matrices are low-rank enough to support fast model evaluation.
-
Scaling Self-Supervised Representation Learning for Symbolic Piano Performance
Self-supervised pretraining on 60,000 hours of symbolic piano music produces a generative model and contrastive embeddings that beat leading baselines on continuation quality and several MIR classification benchmarks.
-
Maximally-Informative Retrieval for State Space Model Generation
RICO ranks documents by how much they reduce an SSM's question perplexity, using gradient-document inner products, and matches BM25 while often beating E5 on answer quality without finetuning.
-
TongSearch-QR: Reinforced Query Reasoning for Retrieval
TongSearch-QR trains 1.5B and 7B models with GRPO and a frozen-embedding reward to rewrite queries, reaching 27.9 nDCG@10 on BRIGHT with BM25, above GPT-4o's 26.5.
-
Establishing Trustworthy LLM Evaluation via Shortcut Neuron Analysis
Shortcut neuron patching suppresses benchmark-contamination shortcuts in LLMs and yields evaluation scores that strongly correlate with the external MixEval benchmark.
-
Context is Gold to find the Gold Passage: Evaluating and Training Contextual Document Embeddings
A new benchmark (ConTEB) and training method (InSeNT) show that context-aware chunk embeddings greatly improve retrieval on context-dependent queries, with minimal computational overhead.
-
Don't Reinvent the Wheel: Efficient Instruction-Following Text Embedding based on Guided Space Transformation
GSTransform trains a shallow linear transformation on a few thousand LLM-labeled examples to make precomputed text embeddings adapt to user instructions in real time.
-
DeepRTL2: A Versatile Model for RTL-Related Tasks
DeepRTL2 claims state-of-the-art results across RTL generation, understanding, code search, equivalence checking, and performance prediction, but the evidence is weakened by benchmark construction issues and a contrad...
-
Diffusion vs. Autoregressive Language Models: A Text Embedding Perspective
Diffusion-language-model embeddings beat LLM embeddings on long-document and reasoning retrieval benchmarks, but the paper overstates some gains and releases no code or data.
-
Revealing the Numeracy Gap: An Empirical Investigation of Text Embedding Models
Across 13 embedding models and 18 numeric formats, retrieval accuracy on the new EmbedNum-1K benchmark averages 54%, just above chance, showing that embedding models largely fail to encode numeric detail.
-
S2Sent: Nested Selectivity Aware Sentence Representation Learning
A lightweight cross-layer fusion module, S2Sent, improves unsupervised sentence embeddings by gating and DCT frequency selection across Transformer blocks.
-
Semantic Compression for Word and Sentence Embeddings using Discrete Wavelet Transform
Keeping only the low-frequency DWT coefficients of word and sentence embeddings preserves most of their semantic quality at 50 to 93 percent fewer dimensions.
-
Traits Run Deep: Enhancing Personality Assessment via Psychology-Guided LLM Representations and Multimodal Apparent Behaviors
Psychology-guided LLM text embeddings fused with audio and facial cues achieved the lowest MSE in the AVI 2025 personality assessment challenge.
-
Automated Novelty Evaluation of Academic Paper: A Collaborative Approach Integrating Human and Large Language Model Knowledge
Method novelty prediction from peer-review novelty sentences and ChatGPT method summaries improves accuracy on ICLR 2022 data, but the benchmark leaks reviewer opinions into the input.
-
UniConv: Unifying Retrieval and Response Generation for Large Language Models in Conversations
A single LLM jointly fine-tuned for conversational dense retrieval and retrieval-augmented generation beats separate retriever-plus-generator pipelines on most test collections, though its headline benchmark was conta...
-
Towards Building General Purpose Embedding Models for Industry 4.0 Agents
Fine-tuning embedding models on nine ISO-derived industrial tasks yields large reported retrieval gains, but the evaluation is weakened by very small test sets and test-set hyperparameter selection.
-
Position: Text Embeddings Should Capture Implicit Semantics, Not Just Surface Meaning
State-of-the-art text embeddings lag far behind on tasks requiring pragmatic inference, stance detection, and social meaning, relative to their strong performance on surface semantic benchmarks.
-
Towards an Explainable Comparison and Alignment of Feature Embeddings
SPEC finds and aligns the sample clusters that two embedding models capture differently by analyzing the eigenvectors of the difference of their kernel matrices.
-
Politics and polarization on Bluesky
On Bluesky from December 2024 to May 2025, about 13% of posts were political; the most polarized topics, like Israel-Palestine and Russia-Ukraine, had opposing minorities of only 1-2% of users, indicating a politicall...
-
Optimizing Question Semantic Space for Dynamic Retrieval-Augmented Multi-hop Question Answering
Q-DREAM improves multi-hop retrieval-augmented QA by decomposing questions, rewriting dependent subquestions, and retrieving with cluster-specific LoRA embeddings.
-
Principled Content Selection to Generate Diverse and Personalized Multi-Document Summaries
Selecting LLM-extracted key points with a diversity-aware determinantal point process before rewriting improves source coverage in multi-document news summarization.
-
Towards Better Instruction Following Retrieval Models
A new training corpus and embedding model improve instruction-following p-MRR by up to 9 points on FollowIR, MAIR, and Bright benchmarks.
-
Soft Thinking: Unlocking the Reasoning Potential of LLMs in Continuous Concept Space
A training-free method that feeds probability-weighted token embeddings back into LLMs during reasoning, improving accuracy and token efficiency on math and coding tasks.
-
Universal Model Routing for Efficient LLM Inference
UniRoute represents each language model by its error rates on a few prompt clusters, letting a router choose among models it has never seen during training.
-
Exploring Block Anomaly Detection In HDFS Log Data Analysis
An LLM-BiLSTM hybrid model detects anomalous HDFS log blocks with 94.8% F1 on the Loghub HDFS dataset, outperforming DeepLog.
-
LPCAN: Lightweight Pyramid Cross-Attention Network for Rail Surface Defect Detection Using RGB-D Data
A lightweight RGB-D cross-attention network is proposed for rail defect detection, but the SOTA accuracy and generalization claims are internally inconsistent and the implementation is not public.
-
QZhou-Embedding Technical Report
QZhou-Embedding reports state-of-the-art average scores on MTEB and CMTEB as of August 27, 2025, using a two-stage multi-task pipeline with LLM-based data synthesis.
-
Granite Embedding R2 Models
Granite Embedding R2 is an Apache-2.0 family of ModernBERT-based retrieval and reranking models that posts high average scores on several benchmarks but falls short of top code-retrieval and reranking baselines.
-
A Scalable and Efficient Signal Integration System for Job Matching
STAR integrates fine-tuned LLM embeddings as node features into a large-scale GNN, improving job matching metrics across three LinkedIn products.
-
Llama Nemoretriever Colembed: Top-Performing Text-Image Retrieval Model
A new family of text-image retrieval models, built from Eagle2 with bidirectional attention and ColBERT-style late interaction, reports state-of-the-art NDCG@5 scores on ViDoRe V1 (91.0) and V2 (63.5).
-
Maintaining MTEB: Towards Long Term Usability and Reproducibility of Embedding Benchmarks
The MTEB maintainers document their infrastructure for versioning and validating benchmark components, plus a zero-shot score that flags models trained on benchmark tasks.
-
CCRS: A Zero-Shot LLM-as-a-Judge Framework for Comprehensive RAG Evaluation
CCRS is a zero-shot LLM-as-a-judge framework whose five metrics discriminate between RAG systems on BioASQ with comparable or better power than RAGChecker at lower compute.
-
Data Curation Matters: Model Collapse and Spurious Shift Performance Prediction from Training on Uncurated Text Embeddings
Training on LLM text embeddings can cause tabular classifiers to collapse to single-class predictions, which spuriously inflates Accuracy-on-the-Line correlations.
-
Combining Log Data and Collaborative Dialogue Features to Predict Project Quality in Middle School AI Education
Log data best predicts training phrase quantity, dialogue best predicts phrase content richness, and multimodal fusion helps only for some outcomes.
-
QUST_NLP at SemEval-2025 Task 7: A Three-Stage Retrieval Framework for Monolingual and Crosslingual Fact-Checked Claim Retrieval
A three-stage ensemble of retrieval models, rerankers, and weighted voting achieves strong multilingual fact-checked claim retrieval results at SemEval-2025 Task 7.
-
LGAI-EMBEDDING-Preview Technical Report
A Mistral-7B embedding model trained with in-context instructions, soft labels from an in-house retrieval pipeline, and margin-based hard-negative mining reports top-tier MTEB English v2 scores.
-
GATE: General Arabic Text Embedding for Enhanced Semantic Textual Similarity with Matryoshka Representation Learning and Hybrid Loss Training
GATE's Arabic-Triplet-Matryoshka-V2 reports the highest average scores on the MTEB Arabic STS17/STS22/STS22-v2 tasks among the models compared in the paper.
-
LazyVLM: Neuro-Symbolic Approach to Video Analytics
LazyVLM decomposes multi-frame video queries into vector-search entity matching, SQL-style relationship lookup, and lightweight VLM refinement, but provides no experimental evaluation of its claims.
-
Hard Negative Mining for Domain-Specific Retrieval in Enterprise Systems
A reranker fine-tuned on hard negatives selected by two cosine-distance criteria outperforms older negative sampling methods on enterprise and domain-specific retrieval benchmarks.
-
Knowledge-Embedded and Hypernetwork-Guided Few-Shot Substation Meter Defect Image Generation Method
Fine-tuning Stable Diffusion with DreamBooth-style knowledge and hypernetwork-guided crack control maps can synthesize substation meter defect images that boost a YOLOv8 defect detector's mAP when added to the training set.
Discussion (0). Continue with ORCID to comment.