pith. sign in

arxiv: 2410.03730 · v3 · pith:6YPOQIXInew · submitted 2024-09-30 · 💻 cs.CL · cs.AI· cs.LG

Teuken-7B-Base & Teuken-7B-Instruct: Towards European LLMs

Mehdi Ali , Michael Fromm , Klaudia Thellmann , Jan Ebert , Alexander Arno Weber , Richard Rutmann , Charvi Jain , Max L\"ubbering
show 33 more authors
Daniel Steinigen Johannes Leveling Katrin Klug Jasper Schulze Buschhoff Lena Jurkschat Hammam Abdelwahab Benny J\"org Stein Karl-Heinz Sylla Pavel Denisov Nicolo' Brandizzi Qasid Saleem Anirban Bhowmick Lennard Helmer Chelsea John Pedro Ortiz Suarez Malte Ostendorff Alex Jude Lalith Manjunath Samuel Weinbach Carolin Penke Oleg Filatov Fabio Barth Paramita Mirza Lucas Weber Ines Wendler Rafet Sifa Fabian K\"uch Andreas Herten Ren\'e J\"akel Georg Rehm Stefan Kesselheim Joachim K\"ohler Nicolas Flores-Herr
This is my paper
classification 💻 cs.CL cs.AIcs.LG
keywords europeanllmsmodelsmultilingualdatalanguagesperformanceteuken
0
0 comments X
read the original abstract

We present two multilingual LLMs, Teuken 7B-base and Teuken 7B-instruct, designed to embrace Europe's linguistic diversity by supporting all 24 official languages of the European Union. Trained on a dataset comprising around 60% non-English data and utilizing a custom multilingual tokenizer, our models address the limitations of existing LLMs that predominantly focus on English or a few high-resource languages. We detail the models' development principles, i.e., data composition, tokenizer optimization, and training methodologies. The models demonstrate strong performance across multilingual benchmarks, as evidenced by their performance on European versions of ARC, HellaSwag, and TruthfulQA.

This paper has not been read by Pith yet.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. XL-SafetyBench: A Country-Grounded Cross-Cultural Benchmark for LLM Safety and Cultural Sensitivity

    cs.CL 2026-05 unverdicted novelty 7.0

    XL-SafetyBench is a new cross-cultural benchmark showing frontier LLMs decouple jailbreak robustness from cultural sensitivity while local models trade off attack success against neutral-safe rates in a near-linear pa...

  2. TildeOpen LLM: Leveraging Curriculum Learning to Achieve Equitable Language Representation

    cs.CL 2026-03 unverdicted novelty 5.0

    A new 30B open LLM trained with curriculum learning and upsampling outperforms other multilingual models on European languages, especially low-resource ones, with up to 10x fewer linguistic errors in human evaluations.

  3. Training LLMs on HPC Systems: Best Practices from the OpenGPT-X Project

    cs.DC 2025-04 unverdicted novelty 2.0

    Engineering report detailing HPC infrastructure, software choices, and performance measurements for training a 7B LLM using 3D parallelism on JUWELS Booster.