Pith. sign in

REVIEW 4 major objections 7 minor 42 references

Pipeline Analysis for Developing Instruct LLMs in Low-Resource Languages: A Case Study on Basque

T0 review · 4 major / 7 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A complete instruction-following model for Basque can be built almost entirely from translated English data, via continual pre-training, LoRA instruction tuning, and DPO, gaining more than 12 NLU points and 24 instruction-following points.

desk verdict Solid, useful case study: the NLU gain is credible, but the instruction-following claims need a larger, reliability-checked evaluation before they support stage-by-stage conclusions. read the letter →

arxiv 2412.13922 v1 pith:AKQAD7QQ submitted 2024-12-18 cs.CL cs.AIcs.LG

Ander Corral , Ixak Sarasua , Xabier Saralegi This is my paper · ORCID
classification cs.CLcs.AIcs.LG
keywords low-resourcelanguagesBasquecontinualpre-traininginstructiontuningdirectpreferenceoptimizationmachine-translateddataLLMadaptationnaturallanguageunderstanding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that a complete instruction-following model for a low-resource language can be built almost entirely from translated English data, provided the base model is first adapted to the target language. The authors test this on Basque by continually pre-training Llama-3.1-8B on about 521 million words of clean Basque text mixed 80:20 with English, then instruction-tuning with LoRA on machine-translated versions of the No_Robots and SlimOrca datasets, then applying DPO on a machine-translated version of UltraFeedback. Across twelve Basque NLU benchmarks, the resulting foundational model averages 61.22, up from 48.75 for its English-centric base, and on a 100-item manual Basque instruction test the final model is fully correct on 30% of items versus 6% for the stock instruct model. The authors conclude that each stage contributes, that translated data is a viable substitute for native Basque instruction data, and that the resulting models are the strongest Basque models under 10 billion parameters. If this reasoning transfers to other low-resource languages, it would give a cheap, reproducible recipe for building instruct models without expensively curated native corpora.

What carries the argument

The load-bearing mechanism is a three-stage transfer pipeline built on an 80:20 Basque-English mixture during continual pre-training, with language-matched sequence packing; LoRA (low-rank adaptation) for instruction tuning; and DPO (direct preference optimization) on translated preference pairs. The 80:20 mix is the object that carries the argument: it transfers functional English competences to Basque while preventing catastrophic forgetting. The instruction and preference stages are carried by two machine-translated datasets, SlimOrca_eu for supervised fine-tuning and UltraFeedback_eu for preference alignment, generated with a single machine translation system.

What would settle it

Re-run the same 100-instruction Basque test with three or more independent native-speaker raters and report per-model correct rates with confidence intervals; if the 30% versus 6% advantage of Llama-eus-8B-instruct-DPO over Llama-3.1-8B-instruct collapses or the 30% versus 23% DPO-over-SFT gap reverses, the paper's central instruction-following claim is not stable. A larger automatically scored or human-rated sample would settle the same question with more power.

Watch

Extended reading notes

Core claim

The paper's central claim is that a complete instruction-following model can be developed for a low-resource language by adapting an 8-billion-parameter English-centric model at three successive stages. First, full continual pre-training on a curated 521-million-word Basque corpus mixed 80:20 with English raises average performance on 12 Basque NLU benchmarks from 48.75 to 61.22, beating existing Basque-specific models of comparable size and nearly matching a 70B Basque model. Second, LoRA instruction tuning on machine-translated No_Robots and SlimOrca data makes the model follow Basque instructions: 23% fully correct and 41% partially correct on the manual test, versus 6% and 26% for Llama-3.1-8B-instruct. Third, DPO on a machine-translated UltraFeedback pushes the fully correct rate to 30%. The resulting Llama-eus-8B and Llama-eus-8B-instruct are, the paper claims, the best Basque models under 10B parameters; in English the pre-trained model loses only 1.96 points on average, leaving a 13.28-point Basque-English gap.

Load-bearing premise

The headline instruction-following improvements rest on a single native Basque speaker rating only 100 translated instructions into correct, partially correct, or wrong, with no second annotator to check the ratings.

Editorial extensions

If this is right

  • Languages without native instruction corpora can nevertheless obtain instruction-following models by translating existing English SFT and preference data, as long as a solid target-language foundation model is available.
  • Continual pre-training on a relatively small, clean corpus of about half a billion words is enough to surpass much larger Basque-specific models in NLU, so dataset quality can substitute for dataset scale.
  • Starting preference training from a target-language-adapted instruct model is necessary: DPO on the English-centric Llama-3.1-8B-instruct degraded its Basque correctness from 6% to 2%.
  • Basque still trails English by 13.28 average points in NLU and the instructed model lags far behind Llama-3.1-8B-instruct in English, so the pipeline reduces but does not close the language gap.
  • Larger synthetic instruction data (SlimOrca_eu) beat a smaller curated set (No_Robots_eu) for instruction-following, implying scale matters more than source quality for translated SFT data.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the translated-data results hold for Basque, a similar 80:20 continual pre-training plus translated SFT/DPO recipe is a cheap candidate for other low-resource languages whose base LLM already has some multilingual coverage; the recipe's success likely depends on how much of the target language the tokenizer and base model already encode.
  • The 24-point instruction-following gain is measured on a single native speaker's rating of 100 items; a re-rating with multiple annotators and per-item confidence intervals would either firm up or revise the ranking of DPO versus SFT, and could be done without retraining.
  • Because the English version of the same 100-item test scores 91% correct, the remaining Basque gap is partly a translation-quality and task-complexity problem; improving machine translation quality for the SFT data may yield further gains without more Basque data.
  • The paper's own dataset-comparison appendix suggests adding more, noisier Basque data yields only marginal NLU gains (61.22 to 61.84), so the next bottleneck for Basque is probably instruction-following and generation quality rather than raw understanding.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes and evaluates a three-stage pipeline for adapting Llama-3.1-8B to Basque: (1) continual pre-training on about 521 million words of Basque (ZelaiHandi) mixed 80:20 with English (FineWeb); (2) LoRA instruction tuning on machine-translated No_Robots and SlimOrca datasets; (3) DPO alignment on machine-translated UltraFeedback. Using 12 Basque NLU benchmarks, the authors report a +12.47 average improvement over Llama-3.1-8B (from 48.75 to 61.22). For instruction-following, they rely on a 100-instruction manual test scored by a single native Basque speaker into correct/partially correct/wrong categories and report a 24-point improvement in the correct rate (6% for Llama-3.1-8B-instruct to 30% for Llama-eus-8B-instruct-DPO). The paper introduces four manually translated benchmark subsets and new Basque training corpora (No_Robots_eu, SlimOrca_eu, UltraFeedback_eu).

Significance. If the results hold, the NLU improvement is consistent across all 12 benchmarks and provides a practically valuable sub-10B Basque model, with the released datasets and evaluation subsets being useful resources for the Basque NLP community. The paper also gives a detailed and reproducible account of training configurations, mixing ratios, and carbon footprint. However, the instruction-following claims are built on a small, single-annotator manual evaluation with no reliability or significance testing; several stage-wise incremental gains are within sampling noise. The 'state-of-the-art' and 'first instructed Basque LLM' assertions require a broader comparison and are not yet fully supported. The central NLU contribution is credible, but the alignment-stage conclusions need substantial strengthening or rephrasing.

major comments (4)
  1. [Section 4.2, Tables 3-4] The manual instruction-following evaluation uses 100 instructions from the No_Robots test set scored by one native Basque speaker into correct/partially correct/wrong. The paper reports no inter-annotator agreement, no confidence intervals, and no evidence of blinded scoring; the rubric requires subjective judgments about whether a response fully addressed the task and introduced hallucinations. This makes the 24-point improvement and the 7-point DPO gain non-auditable. The authors should publish the 100 inputs and all model outputs, obtain at least a second annotation on a random subset, and report a reliability measure such as Cohen's kappa. Without this, the headline instruction-following claims cannot be independently verified.
  2. [Section 4.4 and 5.2, Tables 3-4] With n=100, the incremental improvements between consecutive pipeline stages are not statistically supported. The DPO gain (23% to 30% correct) and the benefit of Basque pre-training for instruction tuning (14% to 23%) are not significant (Fisher exact two-tailed p>0.1 and p≈0.14, respectively); the 15% to 23% difference between No_Robots_eu and SlimOrca_eu is also non-significant. The paper labels these effects as 'significant benefits' and states that translated preference data are 'viable' based on these differences. Please report exact p-values or confidence intervals and temper the stage-wise conclusions; only the full-pipeline comparison (6% vs 30%) appears statistically robust.
  3. [Introduction and Section 2] The paper claims that Llama-eus-8B-instruct is 'the first instructed LLM for Basque' and achieves state-of-the-art performance among sub-10B models. The related work discussion only treats Latxa (Etxaniz et al., 2024) as a family of foundational models, but the cited Latxa work also describes instruction-tuned variants. If such models exist, they must be included in the instruction-following comparison and the 'first' claim must be revised. Please clarify whether any instruction-tuned Basque model predates this work and, if so, compare against it.
  4. [Abstract and Section 6] The abstract attributes the '24-point improvement in instruction-following performance' to 'instruction tuning and human preference alignment,' but the 6% baseline is Llama-3.1-8B-instruct and the 30% result also includes the effect of continual pre-training, which alone accounts for a 9-point gain over Llama-3.1-8B+SlimOrca_eu. Thus the 24-point gain is the cumulative effect of the full pipeline, not solely the instruction-tuning and alignment stages. The wording should be corrected to avoid this misattribution.
minor comments (7)
  1. [Section 4.4] There are typos in this section, including 'intruction tunining' and 'intructions datasets'; these should be corrected.
  2. [Section 3.3, Table 1] For the four newly translated subsets (ARC_HT_eu_sample, Winogrande_HT_eu_sample, MMLU_HT_eu_sample, HellaSwag_HT_eu_sample), please report the sample sizes in the table and, if possible, 95% confidence intervals, so that the reader can assess the precision of the 12.47 average gain.
  3. [Section 4.1] The reported machine translation quality (19.3 BLEU and 52.2 chrF++) is not discussed; a short analysis of how translation errors might affect the downstream instruction-tuning results would strengthen the paper.
  4. [Introduction and Conclusions] The unqualified phrase 'state-of-the-art among sub-10B parameter models' should be scoped to 'among the compared baselines' or supported by a broader comparison (e.g., Qwen2, Gemma, or other open sub-10B models).
  5. [Appendix E] The Basque example for the Summarization category contains a likely typo ('nitxoko' instead of 'nitzat' or a similar form); please proofread all Basque examples in the appendix.
  6. [References] The OpenOrca reference has a malformed URL ('https://https://huggingface.co/...') and should be fixed.
  7. [Figure 1] The caption should explicitly explain the lighter-color shading for 'partially correct' answers, and the figure should be checked for legibility in print.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: all headline results are empirical measurements on held-out or external benchmarks; no fitted parameter is renamed as a prediction and no derivation reduces to its inputs.

full rationale

The paper's claims are empirical pipeline measurements, not derived quantities. The 12.47-point NLU gain (Section 3.5, Table 1) is computed on translated and pre-existing Basque benchmarks that are not part of the continual pre-training objective; no parameter is fitted to those scores. The instruction-following results (Section 4.4, Table 3) and DPO results (Section 5.2, Table 4) use a fixed 100-item manual test sampled from the No_Robots test set, while training uses the separate No_Robots/SlimOrca/UltraFeedback splits translated to Basque; the eval set is not used for training or hyperparameter selection, so the 24-point claim is a measurement rather than a construction. The 80:20 Basque/English mixing ratio, LoRA settings, and DPO choice are taken from prior work or preliminary experiments, not fitted to the reported test sets. Self-citations appear (ZelaiHandi dataset; BL2MP and BasqueGLUE benchmarks include current authors), but they are used as data artifacts and baselines, not as load-bearing justifications, and no uniqueness or ansatz argument is imported from them. The Limitations section explicitly hedges the manual 100-item evaluation—'The results of this manual evaluation and the conclusions drawn should be interpreted within the context of the nature and size of this test set'—which is a reliability/validity caveat (single annotator, no CI, small n), not circularity. No equation or claim in the paper reduces to its own input by definition.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

No new theoretical entities are introduced. The load-bearing assumptions are about evaluation validity and the adequacy of machine translation for training data; all quantitative choices are hyperparameters set by hand.

free parameters (4)
  • Basque-English data mixing ratio = 80:20
    Chosen to balance Basque learning against catastrophic forgetting; not optimized in the paper.
  • Continual pretraining epochs = 4
    Number of passes over the corpus; set by the authors based on compute budget, not selection.
  • LoRA rank = 64
    Rank of LoRA adapters for instruction tuning; alpha 16, dropout 0.1, chosen without systematic search.
  • Peak learning rates = 1e-4 (pretraining), 2e-5 (SFT)
    Selected a priori, following common practice; no learning-rate sweep reported.
assumptions (4)
  • domain assumption The manually translated Basque benchmark subsets preserve the difficulty and construct of the original English tests.
    Used to interpret scores on ARC_eu, MMLU_eu, HellaSwag_eu, Winogrande_eu as evidence of NLU improvements.
  • domain assumption Machine-translated instruction and preference data (Elia, BLEU 19.3) are of sufficient quality to train instruction-following in Basque.
    Central to Sections 4 and 5; if the translations are too degraded, the instruction-tuning results would not transfer.
  • domain assumption A random sample of 100 instructions from the No_Robots test set, excluding coding, is representative for evaluating instruct models.
    The 24-point claim depends on this small sample; the paper states it chose 100 examples to evaluate more models.
  • domain assumption The single-annotator manual scoring rubric (correct/partially correct/wrong) produces stable model rankings.
    No inter-annotator agreement or score variation is reported, so the reliability of the manual evaluation is assumed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Pipeline Analysis for Developing Instruct LLMs in Low-Resource Languages: A Case Study on Basque." pith.science (2026). https://pith.science/paper/AKQAD7QQ

@misc{pith2026241213922,
  author       = {Pith},
  title        = {Pith review of: Pipeline Analysis for Developing Instruct LLMs in Low-Resource Languages: A Case Study on Basque},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AKQAD7QQ}},
  note         = {Machine review of arXiv:2412.13922}
}
read the original abstract

Large language models (LLMs) are typically optimized for resource-rich languages like English, exacerbating the gap between high-resource and underrepresented languages. This work presents a detailed analysis of strategies for developing a model capable of following instructions in a low-resource language, specifically Basque, by focusing on three key stages: pre-training, instruction tuning, and alignment with human preferences. Our findings demonstrate that continual pre-training with a high-quality Basque corpus of around 600 million words improves natural language understanding (NLU) of the foundational model by over 12 points. Moreover, instruction tuning and human preference alignment using automatically translated datasets proved highly effective, resulting in a 24-point improvement in instruction-following performance. The resulting models, Llama-eus-8B and Llama-eus-8B-instruct, establish a new state-of-the-art for Basque in the sub-10B parameter category.

Figures

Figures reproduced from arXiv: 2412.13922 by the authors.

Figure 1
Figure 1. Comparison of Basque performance between [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 9 canonical work pages

  1. [1]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  2. [2]

    Lucas Bandarkar, Davis Liang, Benjamin Muller, Mikel Artetxe, Satya Narayan Shukla, Donald Husa, Naman Goyal, Abhinandan Krishnan, Luke Zettlemoyer, and Madian Khabsa. 2024. https://aclanthology.org/2024.acl-long.44 The belebele benchmark: a parallel reading comprehension dataset in 122 language variants . In Proceedings of the 62nd Annual Meeting of the ...

    2024

  3. [3]

    Edward Beeching, Clémentine Fourrier, Nathan Habib, Sheon Han, Nathan Lambert, Nazneen Rajani, Omar Sanseviero, Lewis Tunstall, and Thomas Wolf. 2023. Open llm leaderboard. https://huggingface.co/spaces/open-llm-leaderboard-old/open_llm_leaderboard

    work page 2023

  4. [4]

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv:1803.05457v1

    arXiv 2018

  5. [5]

    Ganqu Cui, Lifan Yuan, Ning Ding, Guanming Yao, Wei Zhu, Yuan Ni, Guotong Xie, Zhiyuan Liu, and Maosong Sun. 2023 a . https://arxiv.org/abs/2310.01377 Ultrafeedback: Boosting language models with high-quality feedback . Preprint, arXiv:2310.01377

    arXiv 2023

  6. [6]

    Yiming Cui, Ziqing Yang, and Xin Yao. 2023 b . Efficient and effective text encoding for chinese llama and alpaca. arXiv preprint arXiv:2304.08177

    arXiv 2023

  7. [7]

    Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher R \'e . 2022. Flashattention: Fast and memory-efficient exact attention with io-awareness. Advances in Neural Information Processing Systems, 35:16344--16359

    2022

  8. [8]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    arXiv 2024

Show all 42 references
  1. [9]

    Julen Etxaniz, Oscar Sainz, Naiara Miguel, Itziar Aldabe, German Rigau, Eneko Agirre, Aitor Ormazabal, Mikel Artetxe, and Aitor Soroa. 2024. https://doi.org/10.18653/v1/2024.acl-long.799 Latxa: An open language model and evaluation suite for B asque . In Proceedings of the 62n...

    2024 doi

  2. [10]

    Kazuki Fujii, Taishi Nakamura, Mengsay Loem, Hiroki Iida, Masanari Ohi, Kakeru Hattori, Hirai Shota, Sakae Mizuki, Rio Yokota, and Naoaki Okazaki. 2024. Continual pre-training for cross-lingual llm adaptation: Enhancing japanese language capabilities. arXiv preprint arXiv:2404.17790

    2024 arXiv

  3. [11]

    Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac'h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang S...

    2024 doi

  4. [12]

    Sylvain Gugger, Lysandre Debut, Thomas Wolf, Philipp Schmid, Zachary Mueller, Sourab Mangrulkar, Marc Sun, and Benjamin Bossan. 2022. Accelerate: Training and inference at scale made simple, efficient and adaptable. https://github.com/huggingface/accelerate

    2022

  5. [13]

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. Measuring massive multitask language understanding. Proceedings of the International Conference on Learning Representations (ICLR)

    2021

  6. [14]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. https://arxiv.org/abs/arXiv:2106.09685 Lora: Low-rank adaptation of large language models

    2021 arXiv

  7. [15]

    Shengyi Costa Huang, Agustín Piqueres, Kashif Rasul, Philipp Schmid, Daniel Vila, and Lewis Tunstall. 2024. Open hermes preferences. https://huggingface.co/datasets/argilla/OpenHermesPreferences

    2024

  8. [16]

    Ayyoob Imani, Peiqin Lin, Amir Hossein Kargaran, Silvia Severini, Masoud Jalili Sabet, Nora Kassner, Chunlan Ma, Helmut Schmid, Andr \'e FT Martins, Fran c ois Yvon, et al. 2023. Glot500: Scaling multilingual corpora and language models to 500 languages. arXiv preprint arXiv:2...

    2023 arXiv

  9. [17]

    Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. 2024. Mixtral of experts. arXiv preprint arXiv:2401.04088

    2024 arXiv

  10. [18]

    Sakaguchi Keisuke, Le Bras Ronan, Bhagavatula Chandra, and Choi Yejin. 2019. Winogrande: An adversarial winograd schema challenge at scale

    2019

  11. [19]

    Hele-Andra Kuulmets, Taido Purason, Agnes Luhtaru, and Mark Fishel. 2024. https://doi.org/10.18653/v1/2024.findings-naacl.210 Teaching llama a new language through cross-lingual knowledge transfer . In Findings of the Association for Computational Linguistics: NAACL 2024, page...

    2024 doi

  12. [20]

    Alexandre Lacoste, Alexandra Luccioni, Victor Schmidt, and Thomas Dandres. 2019. Quantifying the carbon emissions of machine learning. arXiv preprint arXiv:1910.09700

    2019 arXiv

  13. [21]

    Wing Lian, Bleys Goodson, Eugene Pentland, Austin Cook, Chanvichet Vong, and "Teknium". 2023. Openorca: An open dataset of gpt augmented flan reasoning traces. https://https://huggingface.co/Open-Orca/OpenOrca

    2023

  14. [22]

    o rg Tiedemann, Andr \'e FT Martins, and Hinrich Sch \

    Peiqin Lin, Shaoxiong Ji, J \"o rg Tiedemann, Andr \'e FT Martins, and Hinrich Sch \"u tze. 2024. Mala-500: Massive language adaptation of large language models. arXiv preprint arXiv:2401.13303

    2024 arXiv

  15. [23]

    Diab, Veselin Stoyanov, and Xian Li

    Xi Victoria Lin, Todor Mihaylov, Mikel Artetxe, Tianlu Wang, Shuohui Chen, Daniel Simig, Myle Ott, Naman Goyal, Shruti Bhosale, Jingfei Du, Ramakanth Pasunuru, Sam Shleifer, Punit Singh Koura, Vishrav Chaudhary, Brian O'Horo, Jeff Wang, Luke Zettlemoyer, Zornitsa Kozareva, Mon...

    2021 arXiv

  16. [24]

    Guilherme Penedo, Hynek Kydlíček, Loubna Ben allal, Anton Lozhkov, Margaret Mitchell, Colin Raffel, Leandro Von Werra, and Thomas Wolf. 2024. https://arxiv.org/abs/2406.17557 The fineweb datasets: Decanting the web for the finest text data at scale . Preprint, arXiv:2406.17557

    2024 arXiv

  17. [25]

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. https://openreview.net/forum?id=HPuSIXJaa9 Direct preference optimization: Your language model is secretly a reward model . In Thirty-seventh Conference on Neural Infor...

    2023

  18. [26]

    Rush, and Thomas Wolf

    Nazneen Rajani, Lewis Tunstall, Edward Beeching, Nathan Lambert, Alexander M. Rush, and Thomas Wolf. 2023. No robots. https://huggingface.co/datasets/HuggingFaceH4/no_robots

    2023

  19. [27]

    Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. 2020. Zero: Memory optimizations toward training trillion parameter models. In SC20: International Conference for High Performance Computing, Networking, Storage and Analysis, pages 1--16. IEEE

    2020

  20. [28]

    Iñaki San Vicente, Gorka Urbizu, Ander Corral, Zuhaitz Beloki, and Xabier Saralegi. 2024. https://huggingface.co/datasets/orai-nlp/ZelaiHandi Zelaihandi: A large collection of basque texts

    2024

  21. [29]

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347

    2017 arXiv

  22. [30]

    Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, L \'e onard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ram \'e , et al. 2024. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118

    2024 arXiv

  23. [31]

    NLLB Team, Marta R. Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejia Gonzalez, Prang...

    2022

  24. [32]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth \'e e Lacroix, Baptiste Rozi \`e re, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971

    2023 arXiv

  25. [33]

    Gorka Urbizu, Iñaki San Vicente, Xabier Saralegi, Rodrigo Agerri, and Aitor Soroa. 2022. https://aclanthology.org/2022.lrec-1.172 Basqueglue: A natural language understanding benchmark for basque . In Proceedings of the Language Resources and Evaluation Conference, pages 1603-...

    2022

  26. [34]

    Gorka Urbizu, Muitze Zulaika, Xabier Saralegi, and Ander Corral. 2024. https://aclanthology.org/2024.lrec-main.731 How well can BERT learn the grammar of an agglutinative and flexible-order language? the case of B asque. In Proceedings of the 2024 Joint International Conferenc...

    2024

  27. [35]

    Alex Warstadt, Alicia Parrish, Haokun Liu, Anhad Mohananey, Wei Peng, Sheng-Fu Wang, and Samuel R. Bowman. 2020. https://doi.org/10.1162/tacl_a_00321 BL i MP : The benchmark of linguistic minimal pairs for E nglish . Transactions of the Association for Computational Linguistic...

    2020 doi

  28. [36]

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mari...

    2020 doi

  29. [37]

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. 2024. Qwen2 technical report. arXiv preprint arXiv:2407.10671

    2024 arXiv

  30. [38]

    Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. Hellaswag: Can a machine really finish your sentence? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

    2019

  31. [39]

    Jun Zhao, Zhihao Zhang, Qi Zhang, Tao Gui, and Xuanjing Huang. 2024. Llama beyond english: An empirical study on language capability transfer. arXiv preprint arXiv:2401.01055

    2024 arXiv

  32. [40]

    Banghua Zhu, Evan Frick, Tianhao Wu, Hanlin Zhu, and Jiantao Jiao. 2023. Starling-7b: Improving llm helpfulness and harmlessness with rlaif

    2023

  33. [41]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  34. [42]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.