Pith. sign in

REVIEW 11 cited by

Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2004.09813 v2 pith:HDDMEYW6 submitted 2020-04-21 cs.CL

Nils Reimers , Iryna Gurevych This is my paper
classification cs.CL
keywords sentenceembeddingslanguagesmodelsextendmonolingualmultilingualoriginal
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

We present an easy and efficient method to extend existing sentence embedding models to new languages. This allows to create multilingual versions from previously monolingual models. The training is based on the idea that a translated sentence should be mapped to the same location in the vector space as the original sentence. We use the original (monolingual) model to generate sentence embeddings for the source language and then train a new system on translated sentences to mimic the original model. Compared to other methods for training multilingual sentence embeddings, this approach has several advantages: It is easy to extend existing models with relatively few samples to new languages, it is easier to ensure desired properties for the vector space, and the hardware requirements for training is lower. We demonstrate the effectiveness of our approach for 50+ languages from various language families. Code to extend sentence embeddings models to more than 400 languages is publicly available.

Discussion (0). Sign in to comment.

Forward citations

Cited by 11 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Bekko Embedding: Parameter-Efficient Multilingual Retrieval with Ultra-Compact Encoders

    cs.IR 2026-07 conditional novelty 7.0 of 10

    Bekko a8m, with 7.7M active parameters, scores 56.2 on MMTEB Multilingual v2 Retrieval, beating mE5 models and BGE-M3, while a25m reaches 57.5, on par with gte-multilingual-base.

  2. SMCLM: Semantically Meaningful Causal Language Modeling for Autoregressive Paraphrase Generation

    cs.CL 2025-07 conditional novelty 6.0 of 10

    SMCLM prepends a frozen sentence embedding to GPT-2 and trains with causal language modeling, producing paraphrases that the authors find competitive with supervised methods and best among the unsupervised baselines tested.

  3. The Missing Link: Joint Legal Citation Prediction using Heterogeneous Graph Enrichment

    cs.SI 2025-06 accept novelty 6.0 of 10

    A graph neural network that enriches legal citation graphs with categorical metadata nodes predicts case and law citations more accurately than prior GNN baselines, and joint training boosts case citation prediction.

  4. DeMeVa at LeWiDi-2025: Modeling Perspectives with In-Context Learning and Label Distribution Learning

    cs.CL 2025-09 conditional novelty 5.0 of 10

    In-context learning with up to 10 demonstrations per annotator can mimic individual annotator behavior well enough to produce competitive soft labels on four disagreement-rich datasets.

  5. Leveraging Large Language Models for Accurate Sign Language Translation in Low-Resource Scenarios

    cs.CL 2025-08 unverdicted novelty 5.0 of 10

    A prompting method that links signs to short text descriptions lets large language models translate English and Italian into sign language glosses, beating prior models in low-data settings.

  6. CCL-XCoT: An Efficient Cross-Lingual Knowledge Transfer Method for Mitigating Hallucination Generation

    cs.CL 2025-07 conditional novelty 5.0 of 10

    CCL-XCoT combines curriculum-based contrastive pretraining with cross-lingual chain-of-thought fine-tuning, lifting hallucination-free rates in low-resource QA from 1-18% to 55-74%.

  7. Building a Few-Shot Cross-Domain Multilingual NLU Model for Customer Care

    cs.CL 2025-06 conditional novelty 5.0 of 10

    The proposed teacher-student distillation with isotropic regularization improves few-shot multilingual intent detection on proprietary customer care data, though the advertised gains are against baselines not given th...

  8. Private, Verifiable, and Auditable AI Systems

    cs.CR 2025-08 conditional novelty 4.0 of 10

    A thesis demonstrating partial prototypes for zk-verifiable model evaluation and privacy-preserving retrieval, and arguing these pieces can compose into end-to-end auditable AI systems.

  9. AdaDocVQA: Adaptive Framework for Long Document Visual Question Answering in Low-Resource Settings

    cs.CL 2025-08 conditional novelty 4.0 of 10

    A hybrid retrieval, data augmentation, and ensemble inference framework fine-tunes Qwen2.5-VL-72B to 59% on the Japanese LAVA document VQA benchmark.

  10. Never Come Up Empty: Adaptive HyDE Retrieval for Improving LLM Developer Support

    cs.SE 2025-07 conditional novelty 4.0 of 10

    A HyDE retrieval pipeline with full-answer context and adaptive similarity thresholding improves LLM answers to Stack Overflow questions over zero-shot prompting for three of four open-source models.

  11. FullRecall: A Semantic Search-Based Ranking Approach for Maximizing Recall in Patent Retrieval

    cs.IR 2025-07 reject novelty 4.0 of 10

    A three-phase patent retrieval pipeline achieved 100% recall on five examiner-cited test queries, but the score is driven by post hoc cutoff choices and a candidate set that already contains the target patents.

Pith tools