REVIEW 14 cited by
Making Text Embedders Few-Shot Learners
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large language models (LLMs) with decoder-only architectures demonstrate remarkable in-context learning (ICL) capabilities. This feature enables them to effectively handle both familiar and novel tasks by utilizing examples provided within their input context. Recognizing the potential of this capability, we propose leveraging the ICL feature in LLMs to enhance the process of text embedding generation. To this end, we introduce a novel model bge-en-icl, which employs few-shot examples to produce high-quality text embeddings. Our approach integrates task-related examples directly into the query side, resulting in significant improvements across various tasks. Additionally, we have investigated how to effectively utilize LLMs as embedding models, including various attention mechanisms, pooling methods, etc. Our findings suggest that retaining the original framework often yields the best results, underscoring that simplicity is best. Experimental results on the MTEB and AIR-Bench benchmarks demonstrate that our approach sets new state-of-the-art (SOTA) performance. Our model, code and dataset are freely available at https://github.com/FlagOpen/FlagEmbedding .
Forward citations
Cited by 14 Pith papers
-
BitNet Text Embeddings
BITEMBED trains 1.58-bit ternary-weight LLM embedders with contrastive pre-training, supervised distillation, and multi-precision output training, matching FP16 teachers within ~0.6 MMTEB points at ~2x CPU speed.
-
Negative Matters: Multi-Granularity Hard-Negative Synthesis and Anchor-Token-Aware Pooling for Enhanced Text Embeddings
A new MTEB state-of-the-art for text embeddings is reported by combining multi-granularity LLM-generated hard negatives with curriculum training and an anchor-token-aware pooling method.
-
Functional Consistency of LLM Code Embeddings: A Self-Evolving Data Synthesis Framework for Benchmarking
A data synthesis framework generates four syntax/semantics code pair types, and fine-tuning embedding models on the resulting datasets improves code clone detection, functional consistency, and retrieval.
-
Hierarchical Memory Organization for Wikipedia Generation
Generating Wikipedia articles with factoid memory units organized into a hierarchical outline improves informativeness, verifiability, and citation coverage over RAG and STORM baselines.
-
Enhancing Automatic Term Extraction with Large Language Models via Syntactic Retrieval
Syntactic similarity retrieval of demonstrations improves LLM-based automatic term extraction in cross-domain settings, but gains are modest and in-domain lexical retrieval is often competitive or better.
-
Maximally-Informative Retrieval for State Space Model Generation
RICO ranks documents by how much they reduce an SSM's question perplexity, using gradient-document inner products, and matches BM25 while often beating E5 on answer quality without finetuning.
-
SUCEA: Reasoning-Intensive Retrieval for Adversarial Fact-checking through Claim Decomposition and Editing
SUCEA improves adversarial fact-checking by decomposing claims into atomic sub-claims, editing each sub-claim toward retrieved evidence, and re-retrieving before predicting the final label.
-
ExpeTrans: LLMs Are Experiential Transfer Learners
ExpeTrans transfers textual task-solving experience from source tasks to target tasks and reports accuracy gains over zero-shot baselines on 13 NLP datasets.
-
Position: Text Embeddings Should Capture Implicit Semantics, Not Just Surface Meaning
State-of-the-art text embeddings lag far behind on tasks requiring pragmatic inference, stance detection, and social meaning, relative to their strong performance on surface semantic benchmarks.
-
HotelMatch-LLM: Joint Multi-Task Training of Small and Large Language Models for Efficient Multimodal Hotel Retrieval
A hotel retrieval system combining small query encoders with large document encoders, multi-task training, and pooled image features outperforms prior multimodal retrievers on four synthetic test sets.
-
QZhou-Embedding Technical Report
QZhou-Embedding reports state-of-the-art average scores on MTEB and CMTEB as of August 27, 2025, using a two-stage multi-task pipeline with LLM-based data synthesis.
-
MAGMaR Shared Task System Description: Video Retrieval with OmniEmbed
Fine-tuning OmniEmbed on MultiVENT 2.0 with text, audio, and video inputs yields the best public MAGMaR video retrieval results, with non-text signals matching text-only performance.
-
LGAI-EMBEDDING-Preview Technical Report
A Mistral-7B embedding model trained with in-context instructions, soft labels from an in-house retrieval pipeline, and margin-based hard-negative mining reports top-tier MTEB English v2 scores.
-
Preference-based learning for news headline recommendation
On real French news data, a preference-based greedy recommender matched neural Thompson sampling, and English translations performed nearly as well as original French embeddings.
Discussion (0). Continue with ORCID to comment.