Pith. sign in

REVIEW 26 cited by

NV-Retriever: Improving text embedding models with effective hard-negative mining

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2407.15831 v2 pith:OFLWBQIJ submitted 2024-07-22 cs.IR cs.AI

Gabriel de Souza P. Moreira , Radek Osmulski , Mengyao Xu , Ronay Ak , Benedikt Schifferer , Even Oldridge This is my paper · ORCID
classification cs.IRcs.AI
keywords modelsminingretrievalembeddinghard-negativemethodscontrastiveeffective
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Text embedding models have been popular for information retrieval applications such as semantic search and Question-Answering systems based on Retrieval-Augmented Generation (RAG). Those models are typically Transformer models that are fine-tuned with contrastive learning objectives. One of the challenging aspects of fine-tuning embedding models is the selection of high quality hard-negative passages for contrastive learning. In this paper we introduce a family of positive-aware mining methods that use the positive relevance score as an anchor for effective false negative removal, leading to faster training and more accurate retrieval models. We provide an ablation study on hard-negative mining methods over their configurations, exploring different teacher and base models. We further demonstrate the efficacy of our proposed mining methods at scale with the NV-Retriever-v1 model, which scores 60.9 on MTEB Retrieval (BEIR) benchmark and placed 1st when it was published to the MTEB Retrieval on July, 2024.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 26 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. DenseOn with the LateOn: Fully Open Dense and Late-Interaction Models for Multilingual, Long-Context, and Code Search

    cs.CL 2026-07 conditional novelty 7.0 of 10

    With matched open data and backbones, ColBERT-style late interaction turns English translate-train into multilingual generalization, while dense retrieval stays mostly inside the translated languages.

  2. DocReRank: Single-Page Hard Negative Query Generation for Training Multi-Modal RAG Rerankers

    cs.IR 2025-05 conditional novelty 7.0 of 10

    Training a reranker on VLM-verified hard negative queries, generated per page from LLM rephrasings of positive queries, outperforms training on document-level hard negatives in multimodal RAG retrieval.

  3. Retro-Rank-In: A Ranking-Based Approach for Inorganic Materials Synthesis Planning

    physics.chem-ph 2025-02 conditional novelty 7.0 of 10

    A pairwise ranking model over a shared embedding space outperforms prior fixed-vocabulary classifiers for inorganic retrosynthesis, especially when ranking many candidate precursor sets.

  4. OCR Hinders RAG: Evaluating the Cascading Impact of OCR on Retrieval-Augmented Generation

    cs.CV 2024-12 conditional novelty 7.0 of 10

    The new OHRBench benchmark shows OCR errors cause large performance drops in retrieval-augmented generation, with semantic errors hurting more than formatting noise.

  5. KoVRE: Training an Efficient Embedding Model for Korean Visual Document Retrieval

    cs.AI 2026-08 conditional novelty 6.0 of 10

    A 2B single-vector retriever, KoVRE, trained with hard-negative mining and reranker distillation, achieves top aggregate Korean visual document retrieval scores.

  6. A Comparative Evaluation of Embeddings and LLMs in a Greek Book Publisher Setting - The CUP Dataset

    cs.CL 2026-07 conditional novelty 6.0 of 10

    CUP, a 104-query Greek book retrieval benchmark, shows hybrid lexical-semantic retrieval outperforms both BM25 and dense-only methods in a real publisher catalog.

  7. BiXSE: Improving Dense Retrieval via Probabilistic Graded Relevance Distillation

    cs.IR 2025-08 conditional novelty 6.0 of 10

    BiXSE trains dense retrievers with binary cross-entropy on LLM-generated graded relevance probabilities, consistently beating InfoNCE and roughly matching pairwise ranking baselines.

  8. Redundancy, Isotropy, and Intrinsic Dimensionality of Prompt-based Text Embeddings

    cs.CL 2025-06 conditional novelty 6.0 of 10

    Prompt-based text embeddings can be truncated to a small fraction of their dimensions with little performance loss on classification and clustering, but retrieval and STS degrade faster; the difference tracks lower in...

  9. LLMs as Better Recommenders with Natural Language Collaborative Signals: A Self-Assessing Retrieval Approach

    cs.IR 2025-05 conditional novelty 6.0 of 10

    SCORE retrieves similar users' behaviors, uses an LLM self-assessment to rerank them, and injects the top ones in natural language into an LLM prompt, improving recommendation accuracy on two datasets.

  10. mFollowIR: a Multilingual Benchmark for Instruction Following in Retrieval

    cs.IR 2025-01 conditional novelty 6.0 of 10

    The paper introduces mFollowIR, a multilingual instruction-following retrieval benchmark across Russian, Chinese, and Persian, and finds that English instruction-trained models transfer cross-lingually but struggle in...

  11. DIVE: Diversified Iterative Self-Improvement

    cs.CL 2025-01 conditional novelty 6.0 of 10

    DIVE combines global sample pooling with diversity-aware data selection to counter output diversity collapse in iterative preference learning for LLMs.

  12. CoRNStack: High-Quality Contrastive Data for Better Code Retrieval and Reranking

    cs.CL 2024-12 conditional novelty 6.0 of 10

    CoRNStack, a consistency-filtered 21M-pair contrastive dataset with curriculum hard negatives, yields state-of-the-art code retrievers and the first finetuned LLM listwise code reranker.

  13. CodeXEmbed: A Generalist Embedding Model Family for Multiligual and Multi-task Code Retrieval

    cs.SE 2024-11 conditional novelty 6.0 of 10

    A family of code embedding models (400M to 7B) trained with multi-stage LoRA achieves top CoIR scores and competitive BEIR text scores, improving code RAG.

  14. Learn to Pool: Lightweight Fine-Tuning for Flexible Multi-Vector Compression

    cs.IR 2026-07 conditional novelty 5.0 of 10

    Lightweight pooling-aware fine-tuning with k-means on a single dataset enables up to 83% vector compression in ColBERT models with no retrieval accuracy loss and positive cross-dataset transfer.

  15. Position: Text Embeddings Should Capture Implicit Semantics, Not Just Surface Meaning

    cs.CL 2025-06 conditional novelty 5.0 of 10

    State-of-the-art text embeddings lag far behind on tasks requiring pragmatic inference, stance detection, and social meaning, relative to their strong performance on surface semantic benchmarks.

  16. VRAG-RL: Empower Vision-Perception-Based RAG for Visually Rich Information Understanding via Iterative Reasoning with Reinforcement Learning

    cs.CL 2025-05 conditional novelty 5.0 of 10

    VRAG-RL uses GRPO reinforcement learning with visual cropping actions and a retrieval-aware reward to improve vision-language RAG agents on document benchmarks.

  17. Towards Better Instruction Following Retrieval Models

    cs.CL 2025-05 conditional novelty 5.0 of 10

    A new training corpus and embedding model improve instruction-following p-MRR by up to 9 points on FollowIR, MAIR, and Bright benchmarks.

  18. Conventional Contrastive Learning Often Falls Short: Improving Dense Retrieval with Cross-Encoder Listwise Distillation and Synthetic Data

    cs.IR 2025-05 conditional novelty 5.0 of 10

    Contrastive fine-tuning often degrades strong dense retrievers, while combining cross-encoder listwise distillation with diverse synthetic queries consistently improves them.

  19. Arctic-Embed 2.0: Multilingual Retrieval Without Compromise

    cs.CL 2024-12 conditional novelty 5.0 of 10

    Arctic-Embed 2.0 delivers open multilingual embedding models with competitive MTEB-R and CLEF retrieval scores and strong 256-dimension MRL compression, plus new evidence on cross-lingual transfer.

  20. Boosting Data Utilization for Multilingual Dense Retrieval

    cs.IR 2025-09 conditional novelty 4.0 of 10

    A three-stage data-utilization pipeline for multilingual dense retrieval, combining ensemble hard-negative mining, LLM-based filtering/generation, and monolingual topic-diverse mini-batches, improves MIRACL nDCG@10 by...

  21. QZhou-Embedding Technical Report

    cs.CL 2025-08 conditional novelty 4.0 of 10

    QZhou-Embedding reports state-of-the-art average scores on MTEB and CMTEB as of August 27, 2025, using a two-stage multi-task pipeline with LLM-based data synthesis.

  22. Llama Nemoretriever Colembed: Top-Performing Text-Image Retrieval Model

    cs.CV 2025-07 conditional novelty 4.0 of 10

    A new family of text-image retrieval models, built from Eagle2 with bidirectional attention and ColBERT-style late interaction, reports state-of-the-art NDCG@5 scores on ViDoRe V1 (91.0) and V2 (63.5).

  23. LGAI-EMBEDDING-Preview Technical Report

    cs.CL 2025-06 conditional novelty 4.0 of 10

    A Mistral-7B embedding model trained with in-context instructions, soft labels from an in-house retrieval pipeline, and margin-based hard-negative mining reports top-tier MTEB English v2 scores.

  24. Hard Negative Mining for Domain-Specific Retrieval in Enterprise Systems

    cs.IR 2025-05 conditional novelty 4.0 of 10

    A reranker fine-tuned on hard negatives selected by two cosine-distance criteria outperforms older negative sampling methods on enterprise and domain-specific retrieval benchmarks.

  25. Jasper and Stella: distillation of SOTA embedding models

    cs.IR 2024-12 conditional novelty 4.0 of 10

    A 2B-parameter embedding model distilled from two larger teachers achieves a 71.54 average MTEB score (No.3 as of Dec 2024), matching 7B-parameter models.

  26. LLMs are Also Effective Embedding Models: An In-depth Overview

    cs.CL 2024-12 conditional novelty 2.0 of 10

    A structured survey of using decoder-only LLMs as text embedding models, covering prompting, fine-tuning, data construction, benchmarks, and open problems.

Pith tools