pith:4F4CSDS3
OLMoE: Open Mixture-of-Experts Language Models
OLMoE shows a 7B-parameter sparse MoE model with 1B active parameters per token can outperform denser models like Llama2-13B.
arxiv:2409.02060 v2 · 2024-09-03 · cs.CL · cs.AI · cs.LG
Add to your LaTeX paper
\usepackage{pith}
\pithnumber{4F4CSDS3JUYT4CXTN6WB3MYQAQ}
Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge
Record completeness
Claims
Our models outperform all available models with similar active parameters, even surpassing larger ones like Llama2-13B-Chat and DeepSeekMoE-16B.
That benchmark comparisons are fair across models trained under different data regimes, token counts, and optimization details, with no post-hoc selection affecting the reported gains.
OLMoE-1B-7B is an open MoE language model activating 1B parameters per token that outperforms models with similar active parameters after pretraining on 5T tokens.
References
Formal links
Cited by
Receipt and verification
| First computed | 2026-05-17T23:38:47.556662Z |
|---|---|
| Builder | pith-number-builder-2026-05-17-v1 |
| Signature | Pith Ed25519
(pith-v1-2026-05) · public key |
| Schema | pith-number/v1.0 |
Canonical hash
e178290e5b4d313e0af36fac1db310041b0ab0879963349fe3d04da5142c5cfd
Aliases
· · · · ·Agent API
Verify this Pith Number yourself
curl -sH 'Accept: application/ld+json' https://pith.science/pith/4F4CSDS3JUYT4CXTN6WB3MYQAQ \
| jq -c '.canonical_record' \
| python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: e178290e5b4d313e0af36fac1db310041b0ab0879963349fe3d04da5142c5cfd
Canonical record JSON
{
"metadata": {
"abstract_canon_sha256": "9c16d9c7d6967a5232dece92619a6ab9873c90da68f13440851db7ecfbd86e8e",
"cross_cats_sorted": [
"cs.AI",
"cs.LG"
],
"license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
"primary_cat": "cs.CL",
"submitted_at": "2024-09-03T17:08:20Z",
"title_canon_sha256": "ea4f149eedbdfb33fbd75aabf8e4f4c5741ff88e5d58565436ddf2c697541b16"
},
"schema_version": "1.0",
"source": {
"id": "2409.02060",
"kind": "arxiv",
"version": 2
}
}