REVIEW 7 cited by
Salamandra Technical Report
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Salamandra Technical Report
read the original abstract
This work introduces Salamandra, a suite of open-source decoder-only large language models available in three different sizes: 2, 7, and 40 billion parameters. The models were trained from scratch on highly multilingual data that comprises text in 35 European languages and code. Our carefully curated corpus is made exclusively from open-access data compiled from a wide variety of sources. Along with the base models, supplementary checkpoints that were fine-tuned on public-domain instruction data are also released for chat applications. Additionally, we also share our preliminary experiments on multimodality, which serve as proof-of-concept to showcase potential applications for the Salamandra family. Our extensive evaluations on multilingual benchmarks reveal that Salamandra has strong capabilities, achieving competitive performance when compared to similarly sized open-source models. We provide comprehensive evaluation results both on standard downstream tasks as well as key aspects related to bias and safety.With this technical report, we intend to promote open science by sharing all the details behind our design choices, data curation strategy and evaluation methodology. In addition to that, we deviate from the usual practice by making our training and evaluation scripts publicly accessible. We release all models under a permissive Apache 2.0 license in order to foster future research and facilitate commercial use, thereby contributing to the open-source ecosystem of large language models.
Forward citations
Cited by 7 Pith papers
-
XL-SafetyBench: A Country-Grounded Cross-Cultural Benchmark for LLM Safety and Cultural Sensitivity
XL-SafetyBench is a new cross-cultural benchmark showing frontier LLMs decouple jailbreak robustness from cultural sensitivity while local models trade off attack success against neutral-safe rates in a near-linear pa...
-
Fairness Pruning: Locating Demographic Bias in GLU-MLP Layers via Differential Activations
Unsigned differential activations locate a few GLU-MLP neurons whose zeroing surgically destabilizes demographic bias while retaining ~99.5% of measured capabilities.
-
A Sovereign, Open-Source Foundation Model for German and English
Soofi S 30B-A3B, a fully documented German–English hybrid Mamba-MoE base model, matches dense 14–27B peers on bilingual aggregates while delivering 8–9× long-context decode throughput.
-
A Sovereign, Open-Source Foundation Model for German and English
A fully documented German–English hybrid MoE base model matches dense 14–27B peers, leads open code scores, and sustains high long-context throughput at 3B active parameters.
-
S-DiverSe: Spanish Diverse Speech
S-DiverSe is a 3.2-hour multi-pathology Spanish pathological-speech benchmark on which heuristic post-processing outperforms fine-tuning for out-of-domain ASR.
-
A Sovereign, Open-Source Foundation Model for German and English
Soofi S 30B-A3B, a hybrid Mamba-MoE model pretrained on ~27T tokens with deliberately up-weighted German, reports the highest English and German aggregate scores among fully open base models in its comparison while ma...
-
TildeOpen LLM: Leveraging Curriculum Learning to Achieve Equitable Language Representation
A new 30B open LLM trained with curriculum learning and upsampling outperforms other multilingual models on European languages, especially low-resource ones, with up to 10x fewer linguistic errors in human evaluations.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.