Pith. sign in

REVIEW 6 cited by

Nemotron-4 15B Technical Report

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2402.16819 v2 pith:D7VCUP55 submitted 2024-02-26 cs.CL cs.AIcs.LG

Jupinder Parmar , Shrimai Prabhumoye , Joseph Jennings , Mostofa Patwary , Sandeep Subramanian , Dan Su , Chen Zhu , Deepak Narayanan
show 19 more authors
Aastha Jhunjhunwala Ayush Dattagupta Vibhu Jawa Jiwei Liu Ameya Mahabaleshwarkar Osvald Nitski Annika Brundyn James Maki Miguel Martinez Jiaxuan You John Kamalu Patrick LeGresley Denys Fridman Jared Casper Ashwath Aithal Oleksii Kuchaiev Mohammad Shoeybi Jonathan Cohen Bryan Catanzaro
This is my paper · ORCID
classification cs.CLcs.AIcs.LG
keywords modelsmultilingualnemotron-4openperformancesimilarly-sizedtasksachieves
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

We introduce Nemotron-4 15B, a 15-billion-parameter large multilingual language model trained on 8 trillion text tokens. Nemotron-4 15B demonstrates strong performance when assessed on English, multilingual, and coding tasks: it outperforms all existing similarly-sized open models on 4 out of 7 downstream evaluation areas and achieves competitive performance to the leading open models in the remaining ones. Specifically, Nemotron-4 15B exhibits the best multilingual capabilities of all similarly-sized models, even outperforming models over four times larger and those explicitly specialized for multilingual tasks.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 6 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Recipes for Pre-training LLMs with MXFP8

    cs.LG 2025-05 conditional novelty 6.0 of 10

    MXFP8-E4M3 with round-up scale computation matches BF16 pre-training accuracy for LLMs up to 8B parameters, enabling 2x faster training on Blackwell.

  2. ZUNA1.1: A more flexible EEG foundation model for Denoising and Super-resolution

    cs.LG 2026-07 conditional novelty 5.0 of 10

    ZUNA1.1, an open-source 380M EEG diffusion autoencoder, reconstructs variable-length, flexibly masked EEG at least as well as its predecessor and far better than spherical spline interpolation.

  3. APT: Improving Specialist LLM Performance with Weakness Case Acquisition and Iterative Preference Training

    cs.CL 2025-06 conditional novelty 5.0 of 10

    APT trains a model on its own wrong answers plus retrieved similar answers using iterative DPO with SFT loss, improving math, code, and instruction-following while keeping general benchmarks about flat.

  4. Scaling Fine-Grained MoE Beyond 50B Parameters: Empirical Evaluation and Practical Insights

    cs.LG 2025-06 conditional novelty 5.0 of 10

    At 56B total parameters, fine-grained MoE with smaller, more numerous experts beats standard Switch and Mixtral-style MoE on validation loss and average downstream accuracy at matched FLOPs.

  5. Salamandra Technical Report

    cs.CL 2025-02 conditional novelty 5.0 of 10

    Salamandra is an open, from-scratch multilingual LLM family with 2B, 7B, and 40B checkpoints, instruction-tuned variants, a vision proof-of-concept, and detailed evaluations across Iberian and European languages.

  6. PARAM-1 BharatGen 2.9B Model

    cs.CL 2025-07 reject novelty 3.0 of 10

    A technical report on a 2.9B English-Hindi model whose headline evaluation numbers are internally inconsistent and whose promoted tokenizer was not used to train the final model.

Pith tools