Pith. sign in

REVIEW 13 cited by

mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrieval

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2407.19669 v2 pith:WDXXXGLW submitted 2024-07-29 cs.CL cs.IR

Xin Zhang , Yanzhao Zhang , Dingkun Long , Wen Xie , Ziqi Dai , Jialong Tang , Huan Lin , Baosong Yang
show 5 more authors
Pengjun Xie Fei Huang Meishan Zhang Wenjie Li Min Zhang
This is my paper
classification cs.CLcs.IR
keywords textlong-contextmodelsmultilingualrerankerretrievalefficiencyencoder
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

We present systematic efforts in building long-context multilingual text representation model (TRM) and reranker from scratch for text retrieval. We first introduce a text encoder (base size) enhanced with RoPE and unpadding, pre-trained in a native 8192-token context (longer than 512 of previous multilingual encoders). Then we construct a hybrid TRM and a cross-encoder reranker by contrastive learning. Evaluations show that our text encoder outperforms the same-sized previous state-of-the-art XLM-R. Meanwhile, our TRM and reranker match the performance of large-sized state-of-the-art BGE-M3 models and achieve better results on long-context retrieval benchmarks. Further analysis demonstrate that our proposed models exhibit higher efficiency during both training and inference. We believe their efficiency and effectiveness could benefit various researches and industrial applications.

Discussion (0). Sign in to comment.

Forward citations

Cited by 13 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Bekko Embedding: Parameter-Efficient Multilingual Retrieval with Ultra-Compact Encoders

    cs.IR 2026-07 conditional novelty 7.0 of 10

    Bekko a8m, with 7.7M active parameters, scores 56.2 on MMTEB Multilingual v2 Retrieval, beating mE5 models and BGE-M3, while a25m reaches 57.5, on par with gte-multilingual-base.

  2. IssueExec: A Test-Driven Approach for Localizing Software Engineering Issues

    cs.SE 2026-07 conditional novelty 6.0 of 10

    IssueExec localizes issue-relevant code by retrieving related tests and following their execution traces, beating prior localization methods on SWE-bench Lite.

  3. ImportSnare: Directed "Code Manual" Hijacking in Retrieval-Augmented Code Generation

    cs.CR 2025-09 conditional novelty 6.0 of 10

    Documentation poisoning with hidden ranking and suggestion sequences can make RAG-based code generators confidently recommend malicious dependencies, even at 0.01% poisoning ratios.

  4. Language Models Improve When Pretraining Data Matches Target Tasks

    cs.CL 2025-07 conditional novelty 6.0 of 10

    Ranking pretraining documents by similarity to benchmark training examples (BETR) yields consistent benchmark gains and a 2.1x compute multiplier over DCLM-Baseline.

  5. CACTI: Leveraging Copy Masking and Contextual Information to Improve Tabular Data Imputation

    cs.LG 2025-06 conditional novelty 6.0 of 10

    CACTI combines median-truncated copy masking with language-model column embeddings to improve tabular imputation accuracy across MCAR, MAR, and MNAR missingness.

  6. Uncovering Visual-Semantic Psycholinguistic Properties from the Distributional Structure of Text Embedding Space

    cs.CL 2025-05 conditional novelty 6.0 of 10

    Words whose caption-embedding neighborhoods are stable under nearest-neighbor agreement are rated by humans as more imageable and concrete, and this stability score outperforms prior unsupervised predictors.

  7. Towards Applying Large Language Models to Complement Single-Cell Foundation Models

    cs.LG 2025-07 conditional novelty 5.0 of 10

    A fusion model called scMPT, combining scGPT with an LLM text encoder, improves single-cell cell type classification on most tested datasets, and the paper shows the LLM relies on marker genes and simple expression patterns.

  8. Text-to-LoRA: Instant Transformer Adaption

    cs.LG 2025-06 conditional novelty 5.0 of 10

    A hypernetwork can generate task-specific LoRA adapters from a text description, and when trained with supervised fine-tuning it zero-shot outperforms a multi-task LoRA baseline on ten benchmarks.

  9. Simple and Effective Baselines for Code Summarisation Evaluation

    cs.CL 2025-05 conditional novelty 5.0 of 10

    Asking an LLM to rate a code summary directly correlates with human judgments as well as or better than n-gram, learned, and embedding metrics, even without a reference summary.

  10. Granite Embedding R2 Models

    cs.CL 2025-08 conditional novelty 4.0 of 10

    Granite Embedding R2 is an Apache-2.0 family of ModernBERT-based retrieval and reranking models that posts high average scores on several benchmarks but falls short of top code-retrieval and reranking baselines.

  11. From Domain Documents to Requirements: Retrieval-Augmented Generation in the Space Industry

    cs.SE 2025-07 conditional novelty 4.0 of 10

    A RAG-based pipeline with neural label classification generates draft space requirements from mission documents, shown in a single qualitative case study.

  12. Beyond the Surface: Measuring Self-Preference in LLM Judgments

    cs.CL 2025-06 conditional novelty 4.0 of 10

    The DBG metric measures LLM self-preference bias as the gap between a judge model's own win rate and the win rate assigned by an ensemble of gold judges.

  13. Hard Negative Mining for Domain-Specific Retrieval in Enterprise Systems

    cs.IR 2025-05 conditional novelty 4.0 of 10

    A reranker fine-tuned on hard negatives selected by two cosine-distance criteria outperforms older negative sampling methods on enterprise and domain-specific retrieval benchmarks.

Pith tools