REVIEW 14 cited by
XNLI: Evaluating Cross-lingual Sentence Representations
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
State-of-the-art natural language processing systems rely on supervision in the form of annotated data to learn competent models. These models are generally trained on data in a single language (usually English), and cannot be directly used beyond that language. Since collecting data in every language is not realistic, there has been a growing interest in cross-lingual language understanding (XLU) and low-resource cross-language transfer. In this work, we construct an evaluation set for XLU by extending the development and test sets of the Multi-Genre Natural Language Inference Corpus (MultiNLI) to 15 languages, including low-resource languages such as Swahili and Urdu. We hope that our dataset, dubbed XNLI, will catalyze research in cross-lingual sentence understanding by providing an informative standard evaluation task. In addition, we provide several baselines for multilingual sentence understanding, including two based on machine translation systems, and two that use parallel data to train aligned multilingual bag-of-words and LSTM encoders. We find that XNLI represents a practical and challenging evaluation suite, and that directly translating the test data yields the best performance among available baselines.
Forward citations
Cited by 14 Pith papers
-
ChiKhaPo: A Large-Scale Multilingual Benchmark for Evaluating Lexical Comprehension and Generation in Large Language Models
ChiKhaPo is an 8-subtask benchmark that measures word-level comprehension and generation in 2,700+ languages and shows state-of-the-art models perform poorly on low-resource languages.
-
FLEXITOKENS: Flexible Tokenization for Evolving Language Models
FLEXITOKENS replaces rigid subword tokenizers and fixed-compression auxiliary losses with a simplified boundary-prediction objective in byte-level models, yielding lower over-fragmentation and up to 10-point gains on ...
-
skLEP: A Slovak General Language Understanding Benchmark
A nine-task Slovak-language understanding benchmark with translated and newly curated datasets, plus the first broad fine-tuned model comparison for Slovak.
-
Training-Free Tokenizer Transplantation via Orthogonal Matching Pursuit
OMP sparse coding of donor token embeddings, with coefficients transferred to the base embedding space, preserves LLM performance after tokenizer replacement better than published zero-shot baselines, though simple he...
-
LIMO: Less is More for Reasoning
LIMO achieves 63.3% on AIME24 and 95.6% on MATH500 via supervised fine-tuning on roughly 1% of the data used by prior models, supporting the claim that minimal strategic examples suffice when pre-training has already ...
-
The Falcon Series of Open Language Models
Falcon-180B is a 180B-parameter open decoder-only model trained on 3.5 trillion tokens that approaches PaLM-2-Large performance at lower cost and is released with dataset extracts.
-
Neural FOXP2 -- Language Specific Neuron Steering for Targeted Language Improvement in LLMs
A three-stage SAE-plus-SVD steering recipe claims to make Hindi or Spanish the default language of an LLM at inference time, but the visible manuscript reports only expected, not measured, outcomes.
-
TASE: Token Awareness and Structured Evaluation for Multilingual Language Models
TASE benchmark shows LLMs lag humans on token-level and structural language tasks across Chinese, English, and Korean despite strong high-level performance.
-
Assessing the Role of Data Quality in Training Bilingual Language Models
A quality filter trained only on English labels can select better French, German, and Chinese pretraining data, improving bilingual model performance and cutting the monolingual-bilingual gap to about 1%.
-
LAGO: Few-shot Crosslingual Embedding Inversion Attacks via Language Similarity-Aware Graph Optimization
LAGO shows that constraining alignment matrices of linguistically similar languages to be close improves few-shot cross-lingual embedding inversion accuracy over independent per-language baselines.
-
Language-Specific Sentiment Polarity Biases in Encoder and Large Language Model Classification of Product Reviews
LLMs show negative polarity bias in French and encoder models show positive bias in Japanese when classifying product review sentiment.
-
Cross-lingual Few-shot Learning for Persian Sentiment Analysis with Incremental Adaptation
Combining few-shot fine-tuning with incremental learning and regularization lets XLM-R and mDeBERTa reach about 96% accuracy on Persian sentiment analysis across five domains.
-
Evaluating and Improving Robustness in Large Language Models: A Survey and Future Directions
LLM robustness research is organized into adversarial robustness, out-of-distribution robustness, and evaluation, with an accompanying GitHub collection of papers.
- Prompt, Translate, Fine-Tune, Re-Initialize, or Instruction-Tune? Adapting LLMs for In-Context Learning in Low-Resource Languages
Discussion (0). Sign in to comment.