{"record_type":"pith_number_record","schema_url":"https://pith.science/schemas/pith-number/v1.json","pith_number":"pith:2025:XB3B7E5EZFGEUEGNGRF5LRPULS","short_pith_number":"pith:XB3B7E5E","schema_version":"1.0","canonical_sha256":"b8761f93a4c94c4a10cd344bd5c5f45c8c31dc1c09fa8a149b92d27727ad9912","source":{"kind":"arxiv","id":"2510.25741","version":4},"attestation_state":"computed","paper":{"title":"Scaling Latent Reasoning via Looped Language Models","license":"http://creativecommons.org/licenses/by-sa/4.0/","headline":"Looped language models match up to 12B model performance with 1.4B and 2.6B parameters by reasoning iteratively in latent space.","cross_cats":[],"primary_cat":"cs.CL","authors_text":"Andrew Smith, Bohong Wu, Boyi Wei, Chenghua Lin, Enduo Zhao, Fan Yin, Ge Zhang, Haoran Que, He Xing, Hongzhi Huang, Jason Eshraghian, Jiaheng Liu, Jiajun Shi, Jian Yang, Kai Hua, Kaijing Ma, Lu Li, Mude Hui, Qiyang Min, Rui-Jie Zhu, Shanda Li, Taylor Kergan, Tianle Cai, Tianyu Zhang, Wei Ye, Wenhao Huang, Xingwei Qu, Xun Zhou, Yoshua Bengio, Yunfeng Shi, Ziniu Li, Zixin Wen, Zixuan Wang","submitted_at":"2025-10-29T17:45:42Z","abstract_excerpt":"Modern LLMs are trained to \"think\" primarily via explicit text generation, such as chain-of-thought (CoT), which defers reasoning to post-training and under-leverages pre-training data. We present and open-source Ouro, named after the recursive Ouroboros, a family of pre-trained Looped Language Models (LoopLM) that instead build reasoning into the pre-training phase through (i) iterative computation in latent space, (ii) an entropy-regularized objective for learned depth allocation, and (iii) scaling to 7.7T tokens. Ouro 1.4B and 2.6B models enjoy superior performance that match the results of"},"verification_status":{"content_addressed":true,"pith_receipt":true,"author_attested":false,"weak_author_claims":0,"strong_author_claims":0,"externally_anchored":false,"storage_verified":false,"citation_signatures":0,"replication_records":0,"graph_snapshot":true,"references_resolved":true,"formal_links_present":true},"canonical_record":{"source":{"id":"2510.25741","kind":"arxiv","version":4},"metadata":{"license":"http://creativecommons.org/licenses/by-sa/4.0/","primary_cat":"cs.CL","submitted_at":"2025-10-29T17:45:42Z","cross_cats_sorted":[],"title_canon_sha256":"a0df220d6138fbd4b125ce65a3ca729ec8e2284bed4a8b7ef7cd2c9f9cac4707","abstract_canon_sha256":"f105c0d60e4d96858f7d1c29bc54f9cce41f06ad3c95e5c75c1158ee56807267"},"schema_version":"1.0"},"receipt":{"kind":"pith_receipt","key_id":"pith-v1-2026-05","algorithm":"ed25519","signed_at":"2026-05-17T23:38:53.109425Z","signature_b64":"YNn/6oIp8tgiv4WtlJWuhiJyUG17hujjeXkwcxnkUC01cWyNjiD5fzaWcpBxhOtnUwJqbUmV1qaWYc+ZkH6PCw==","signed_message":"canonical_sha256_bytes","builder_version":"pith-number-builder-2026-05-17-v1","receipt_version":"0.3","canonical_sha256":"b8761f93a4c94c4a10cd344bd5c5f45c8c31dc1c09fa8a149b92d27727ad9912","last_reissued_at":"2026-05-17T23:38:53.108771Z","signature_status":"signed_v1","first_computed_at":"2026-05-17T23:38:53.108771Z","public_key_fingerprint":"8d4b5ee74e4693bcd1df2446408b0d54"},"graph_snapshot":{"paper":{"title":"Scaling Latent Reasoning via Looped Language Models","license":"http://creativecommons.org/licenses/by-sa/4.0/","headline":"Looped language models match up to 12B model performance with 1.4B and 2.6B parameters by reasoning iteratively in latent space.","cross_cats":[],"primary_cat":"cs.CL","authors_text":"Andrew Smith, Bohong Wu, Boyi Wei, Chenghua Lin, Enduo Zhao, Fan Yin, Ge Zhang, Haoran Que, He Xing, Hongzhi Huang, Jason Eshraghian, Jiaheng Liu, Jiajun Shi, Jian Yang, Kai Hua, Kaijing Ma, Lu Li, Mude Hui, Qiyang Min, Rui-Jie Zhu, Shanda Li, Taylor Kergan, Tianle Cai, Tianyu Zhang, Wei Ye, Wenhao Huang, Xingwei Qu, Xun Zhou, Yoshua Bengio, Yunfeng Shi, Ziniu Li, Zixin Wen, Zixuan Wang","submitted_at":"2025-10-29T17:45:42Z","abstract_excerpt":"Modern LLMs are trained to \"think\" primarily via explicit text generation, such as chain-of-thought (CoT), which defers reasoning to post-training and under-leverages pre-training data. We present and open-source Ouro, named after the recursive Ouroboros, a family of pre-trained Looped Language Models (LoopLM) that instead build reasoning into the pre-training phase through (i) iterative computation in latent space, (ii) an entropy-regularized objective for learned depth allocation, and (iii) scaling to 7.7T tokens. Ouro 1.4B and 2.6B models enjoy superior performance that match the results of"},"claims":{"count":4,"items":[{"kind":"strongest_claim","text":"Ouro 1.4B and 2.6B models enjoy superior performance that match the results of up to 12B SOTA LLMs across a wide range of benchmarks. This advantage stems not from increased knowledge capacity, but from superior knowledge manipulation capabilities.","source":"verdict.strongest_claim","status":"machine_extracted","claim_id":"C1","attestation":"unclaimed"},{"kind":"weakest_assumption","text":"The observed performance gains are caused by the latent iterative computation and entropy-regularized objective rather than differences in training data volume, optimization details, or other unstated architectural choices.","source":"verdict.weakest_assumption","status":"machine_extracted","claim_id":"C2","attestation":"unclaimed"},{"kind":"one_line_summary","text":"Looped language models with latent iterative computation and entropy-regularized depth allocation achieve performance matching up to 12B standard LLMs through superior knowledge manipulation.","source":"verdict.one_line_summary","status":"machine_extracted","claim_id":"C3","attestation":"unclaimed"},{"kind":"headline","text":"Looped language models match up to 12B model performance with 1.4B and 2.6B parameters by reasoning iteratively in latent space.","source":"verdict.pith_extraction.headline","status":"machine_extracted","claim_id":"C4","attestation":"unclaimed"}],"snapshot_sha256":"f55fdc932cf5db6a55b05b2521ba751b54a8bdf9734fac591f4edb07a32e4bf0"},"source":{"id":"2510.25741","kind":"arxiv","version":4},"verdict":{"id":"080472ca-0be8-41d2-bd74-3eeadf1fef01","model_set":{"reader":"grok-4.3"},"created_at":"2026-05-15T07:37:35.132217Z","strongest_claim":"Ouro 1.4B and 2.6B models enjoy superior performance that match the results of up to 12B SOTA LLMs across a wide range of benchmarks. This advantage stems not from increased knowledge capacity, but from superior knowledge manipulation capabilities.","one_line_summary":"Looped language models with latent iterative computation and entropy-regularized depth allocation achieve performance matching up to 12B standard LLMs through superior knowledge manipulation.","pipeline_version":"pith-pipeline@v0.9.0","weakest_assumption":"The observed performance gains are caused by the latent iterative computation and entropy-regularized objective rather than differences in training data volume, optimization details, or other unstated architectural choices.","pith_extraction_headline":"Looped language models match up to 12B model performance with 1.4B and 2.6B parameters by reasoning iteratively in latent space."},"references":{"count":99,"sample":[{"doi":"","year":1901,"title":"Language models are few-shot learners","work_id":"1109e15b-0e77-4d1f-9248-ed7317a8400c","ref_index":1,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2024,"title":"Qwen2 Technical Report","work_id":"a1857881-ab9b-4b80-9b5f-9ae4b5c2566d","ref_index":2,"cited_arxiv_id":"2407.10671","is_internal_anchor":true},{"doi":"","year":2025,"title":"Qwen3 Technical Report","work_id":"25a4e30c-1232-48e7-9925-02fa12ba7c9e","ref_index":3,"cited_arxiv_id":"2505.09388","is_internal_anchor":true},{"doi":"","year":2025,"title":"Gemma 3 Technical Report","work_id":"f93e08bf-9e96-409b-8ac6-b8385fd17fd7","ref_index":4,"cited_arxiv_id":"2503.19786","is_internal_anchor":true},{"doi":"","year":2024,"title":"The llama 3 herd of models.arXiv e-prints, pages arXiv–2407","work_id":"2dfe07e4-932e-4ce0-ad85-badb06bf579b","ref_index":5,"cited_arxiv_id":"","is_internal_anchor":false}],"resolved_work":99,"snapshot_sha256":"a0d732f5b1970c4e497c71d7020e7febbeaf03307f601595c37ee69a6ed7aea3","internal_anchors":28},"formal_canon":{"evidence_count":3,"snapshot_sha256":"72086cef15d7a042ed898590427d1808a398e9a2b3c3e12022cd606e6a5e1c3a"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"},"aliases":[{"alias_kind":"arxiv","alias_value":"2510.25741","created_at":"2026-05-17T23:38:53.108884+00:00"},{"alias_kind":"arxiv_version","alias_value":"2510.25741v4","created_at":"2026-05-17T23:38:53.108884+00:00"},{"alias_kind":"doi","alias_value":"10.48550/arxiv.2510.25741","created_at":"2026-05-17T23:38:53.108884+00:00"},{"alias_kind":"pith_short_12","alias_value":"XB3B7E5EZFGE","created_at":"2026-05-18T12:33:37.589309+00:00"},{"alias_kind":"pith_short_16","alias_value":"XB3B7E5EZFGEUEGN","created_at":"2026-05-18T12:33:37.589309+00:00"},{"alias_kind":"pith_short_8","alias_value":"XB3B7E5E","created_at":"2026-05-18T12:33:37.589309+00:00"}],"events":[],"event_summary":{},"paper_claims":[],"inbound_citations":{"count":30,"internal_anchor_count":30,"sample":[{"citing_arxiv_id":"2605.23872","citing_title":"Training-Free Looped Transformers","ref_index":102,"is_internal_anchor":true},{"citing_arxiv_id":"2605.22504","citing_title":"LACO: Adaptive Latent Communication for Collaborative Driving","ref_index":44,"is_internal_anchor":true},{"citing_arxiv_id":"2605.20613","citing_title":"HRM-Text: Efficient Pretraining Beyond Scaling","ref_index":73,"is_internal_anchor":true},{"citing_arxiv_id":"2605.21260","citing_title":"On the Cost and Benefit of Chain of Thought: A Learning-Theoretic Perspective","ref_index":118,"is_internal_anchor":true},{"citing_arxiv_id":"2605.18797","citing_title":"Simply Stabilizing the Loop via Fully Looped Transformer","ref_index":18,"is_internal_anchor":true},{"citing_arxiv_id":"2605.07721","citing_title":"Memory-Efficient Looped Transformer: Decoupling Compute from Memory in Looped Language Models","ref_index":1,"is_internal_anchor":true},{"citing_arxiv_id":"2605.16343","citing_title":"LoopQ: Quantization for Recursive Transformers","ref_index":45,"is_internal_anchor":true},{"citing_arxiv_id":"2605.16048","citing_title":"Looped SSMs: Depth-Recurrence and Input Reshaping for Time Series Classification","ref_index":32,"is_internal_anchor":true},{"citing_arxiv_id":"2605.16638","citing_title":"TTE-Flash: Accelerating Reasoning-based Multimodal Representations via Think-Then-Embed Tokens","ref_index":22,"is_internal_anchor":true},{"citing_arxiv_id":"2605.18464","citing_title":"PERL: Parameter Efficient Reasoning in CLIP Latent Space","ref_index":39,"is_internal_anchor":true},{"citing_arxiv_id":"2512.10941","citing_title":"Mull-Tokens: Modality-Agnostic Latent Thinking","ref_index":78,"is_internal_anchor":true},{"citing_arxiv_id":"2605.13190","citing_title":"N-vium: Mixture-of-Exits Transformer for Accelerated Exact Generation","ref_index":63,"is_internal_anchor":true},{"citing_arxiv_id":"2605.11011","citing_title":"LoopUS: Recasting Pretrained LLMs into Looped Latent Refinement Models","ref_index":4,"is_internal_anchor":true},{"citing_arxiv_id":"2605.09630","citing_title":"Scratchpad Patching: Decoupling Compute from Patch Size in Byte-Level Language Models","ref_index":104,"is_internal_anchor":true},{"citing_arxiv_id":"2605.09948","citing_title":"LoopVLA: Learning Sufficiency in Recurrent Refinement for Vision-Language-Action Models","ref_index":28,"is_internal_anchor":true},{"citing_arxiv_id":"2605.09165","citing_title":"Sparse Layers are Critical to Scaling Looped Language Models","ref_index":2,"is_internal_anchor":true},{"citing_arxiv_id":"2604.22951","citing_title":"The Power of Power Law: Asymmetry Enables Compositional Reasoning","ref_index":67,"is_internal_anchor":true},{"citing_arxiv_id":"2605.06510","citing_title":"Is One Layer Enough? Understanding Inference Dynamics in Tabular Foundation Models","ref_index":53,"is_internal_anchor":true},{"citing_arxiv_id":"2605.06322","citing_title":"SMolLM: Small Language Models Learn Small Molecular Grammar","ref_index":104,"is_internal_anchor":true},{"citing_arxiv_id":"2604.21254","citing_title":"Hyperloop Transformers","ref_index":30,"is_internal_anchor":true},{"citing_arxiv_id":"2604.19550","citing_title":"LoopCTR: Unlocking the Loop Scaling Power for Click-Through Rate Prediction","ref_index":29,"is_internal_anchor":true},{"citing_arxiv_id":"2604.12946","citing_title":"Parcae: Scaling Laws For Stable Looped Language Models","ref_index":96,"is_internal_anchor":true},{"citing_arxiv_id":"2604.11791","citing_title":"A Mechanistic Analysis of Looped Reasoning Language Models","ref_index":34,"is_internal_anchor":true},{"citing_arxiv_id":"2604.09870","citing_title":"Relational Preference Encoding in Looped Transformer Internal States","ref_index":10,"is_internal_anchor":true},{"citing_arxiv_id":"2604.07822","citing_title":"Loop, Think, & Generalize: Implicit Reasoning in Recurrent-Depth Transformers","ref_index":3,"is_internal_anchor":true}]},"formal_canon":{"evidence_count":3,"sample":[],"anchors":[]},"links":{"html":"https://pith.science/pith/XB3B7E5EZFGEUEGNGRF5LRPULS","json":"https://pith.science/pith/XB3B7E5EZFGEUEGNGRF5LRPULS.json","graph_json":"https://pith.science/api/pith-number/XB3B7E5EZFGEUEGNGRF5LRPULS/graph.json","events_json":"https://pith.science/api/pith-number/XB3B7E5EZFGEUEGNGRF5LRPULS/events.json","paper":"https://pith.science/paper/XB3B7E5E"},"agent_actions":{"view_html":"https://pith.science/pith/XB3B7E5EZFGEUEGNGRF5LRPULS","download_json":"https://pith.science/pith/XB3B7E5EZFGEUEGNGRF5LRPULS.json","view_paper":"https://pith.science/paper/XB3B7E5E","resolve_alias":"https://pith.science/api/pith-number/resolve?arxiv=2510.25741&json=true","fetch_graph":"https://pith.science/api/pith-number/XB3B7E5EZFGEUEGNGRF5LRPULS/graph.json","fetch_events":"https://pith.science/api/pith-number/XB3B7E5EZFGEUEGNGRF5LRPULS/events.json","actions":{"anchor_timestamp":"https://pith.science/pith/XB3B7E5EZFGEUEGNGRF5LRPULS/action/timestamp_anchor","attest_storage":"https://pith.science/pith/XB3B7E5EZFGEUEGNGRF5LRPULS/action/storage_attestation","attest_author":"https://pith.science/pith/XB3B7E5EZFGEUEGNGRF5LRPULS/action/author_attestation","sign_citation":"https://pith.science/pith/XB3B7E5EZFGEUEGNGRF5LRPULS/action/citation_signature","submit_replication":"https://pith.science/pith/XB3B7E5EZFGEUEGNGRF5LRPULS/action/replication_record"}},"created_at":"2026-05-17T23:38:53.108884+00:00","updated_at":"2026-05-17T23:38:53.108884+00:00"}