REVIEW 25 cited by
Gecko: Versatile Text Embeddings Distilled from Large Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We present Gecko, a compact and versatile text embedding model. Gecko achieves strong retrieval performance by leveraging a key idea: distilling knowledge from large language models (LLMs) into a retriever. Our two-step distillation process begins with generating diverse, synthetic paired data using an LLM. Next, we further refine the data quality by retrieving a set of candidate passages for each query, and relabeling the positive and hard negative passages using the same LLM. The effectiveness of our approach is demonstrated by the compactness of the Gecko. On the Massive Text Embedding Benchmark (MTEB), Gecko with 256 embedding dimensions outperforms all existing entries with 768 embedding size. Gecko with 768 embedding dimensions achieves an average score of 66.31, competing with 7x larger models and 5x higher dimensional embeddings.
Forward citations
Cited by 25 Pith papers
-
mamabench and mamaretrieval: Benchmarks for Evaluating Medical Retrieval-Augmented Generation in Maternal, Neonatal, and Reproductive Health
The authors introduce and release two new benchmarks for maternal-health RAG evaluation, built from expert sources with graded labels and disclosed limitations rather than binary judgments or new question authoring.
-
LLM2Vec-Gen: Generative Embeddings from Large Language Models
LLM2Vec-Gen learns embeddings that encode an LLM's likely response to a query, improving self-supervised MTEB scores, safety, and reasoning-intensive retrieval.
-
AI for Statutory Simplification: A Comprehensive State Legal Corpus and Labor Benchmark
State-of-the-art LLMs with retrieval answer simplified boolean questions about state unemployment insurance law with at best 0.69 F1, well short of reliable end-to-end code simplification.
-
BitNet Text Embeddings
BITEMBED trains 1.58-bit ternary-weight LLM embedders with contrastive pre-training, supervised distillation, and multi-precision output training, matching FP16 teachers within ~0.6 MMTEB points at ~2x CPU speed.
-
Sketching the Readout of Large Language Models for Scalable Data Attribution and Valuation
RISE applies CountSketch to dual lexical and semantic channels derived from output-layer gradient outer products, cutting data attribution storage by up to 112x and enabling retrospective and prospective influence ana...
-
LLMs Meet Isolation Kernel: Lightweight, Learning-free Binary Embeddings for Fast Retrieval
Isolation-kernel binary hashing (IKE) compresses LLM embeddings to a few hundred bytes per point with retrieval accuracy near the original and large speedups in exhaustive and ANN search.
-
LLM-guided Hierarchical Search for End-to-end Reasoning Intensive Retrieval
An LLM directly traverses a hierarchical semantic index of a corpus, using calibrated path-relevance scores to retrieve documents for reasoning-intensive queries.
-
Efficient Item ID Generation for Large-Scale LLM-based Recommendation
LLM-based recommenders can treat item IDs as single direct embeddings and decode in one step, with a two-level softmax for efficiency and quality matching or beating multi-token models.
-
Negative Matters: Multi-Granularity Hard-Negative Synthesis and Anchor-Token-Aware Pooling for Enhanced Text Embeddings
A new MTEB state-of-the-art for text embeddings is reported by combining multi-granularity LLM-generated hard negatives with curriculum training and an anchor-token-aware pooling method.
-
BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent
BrowseComp-Plus is a fixed, human-verified document corpus with supporting and negative documents that allows controlled, reproducible evaluation of deep-research agents and their retrievers.
-
Towards Compute-Optimal Many-Shot In-Context Learning
Hybrid demonstration selection that adds 20 similar examples to a large cached random or k-means set matches or beats similarity-only selection at up to 10x lower estimated inference cost in many-shot ICL.
-
Clustering and Median Aggregation Improve Differentially Private Inference
Clustering seed texts and privately aggregating median token logits improves representativeness and reduces reported privacy cost for DP synthetic text generation.
-
Redundancy, Isotropy, and Intrinsic Dimensionality of Prompt-based Text Embeddings
Prompt-based text embeddings can be truncated to a small fraction of their dimensions with little performance loss on classification and clustering, but retrieval and STS degrade faster; the difference tracks lower in...
-
Aligning Web Query Generation with Ranking Objectives via Direct Preference Optimization
Using direct preference optimization with reranker or GPT-3.5 preferences to align synthetic query generation improves downstream dense retrieval effectiveness on MS MARCO and TREC-DL.
-
RaDeR: Reasoning-aware Dense Retrieval Models
A math-trained dense retriever and reranker, built from MCTS reasoning trajectories and self-reflection, outperforms strong baselines on reasoning-intensive retrieval benchmarks and beats BM25 on chain-of-thought queries.
-
SemCSE: Semantic Contrastive Sentence Embeddings Using LLM-Generated Summaries For Scientific Abstracts
Using multiple LLM-generated summaries of the same abstract as positive pairs trains scientific text embeddings that beat citation-trained baselines on retrieval and clustering, while the new benchmark shares its trai...
-
Towards Better Instruction Following Retrieval Models
A new training corpus and embedding model improve instruction-following p-MRR by up to 9 points on FollowIR, MAIR, and Bright benchmarks.
-
Synthesizing and Adapting Error Correction Data for Mobile Large Language Model Applications
A production mobile-keyboard error-correction system that synthesizes large LLM-generated training data, reweights it with a differentially private federated small LM, and fine-tunes a billion-parameter LLM via LoRA r...
-
MIRB: Mathematical Information Retrieval Benchmark
MIRB, a unified benchmark of four math retrieval tasks across 12 datasets, shows current retrieval models score far lower on premise retrieval than on semantic retrieval, and cross-encoder rerankers often hurt.
-
Generative Data Refinement: Just Ask for Better Data
A pretrained LLM can rewrite individual data samples to strip out PII or toxic content while preserving useful information, creating safer training data.
-
Exploring Reasoning-Infused Text Embedding with Large Language Models for Zero-Shot Dense Retrieval
Reasoning-infused text embedding, which prepends LLM-generated reasoning to queries before embedding, improves zero-shot dense retrieval on BRIGHT.
-
QZhou-Embedding Technical Report
QZhou-Embedding reports state-of-the-art average scores on MTEB and CMTEB as of August 27, 2025, using a two-stage multi-task pipeline with LLM-based data synthesis.
-
LGAI-EMBEDDING-Preview Technical Report
A Mistral-7B embedding model trained with in-context instructions, soft labels from an in-house retrieval pipeline, and margin-based hard-negative mining reports top-tier MTEB English v2 scores.
-
GATE: General Arabic Text Embedding for Enhanced Semantic Textual Similarity with Matryoshka Representation Learning and Hybrid Loss Training
GATE's Arabic-Triplet-Matryoshka-V2 reports the highest average scores on the MTEB Arabic STS17/STS22/STS22-v2 tasks among the models compared in the paper.
-
A Practical Guide for Evaluating LLMs and LLM-Reliant Systems
A guide that organizes LLM evaluation into three pillars (datasets, metrics, and methodology) and introduces a '5 D's' checklist for building evaluation datasets.
Discussion (0). Sign in to comment.