Typed states for the displayed outbound observations.
Source: paper_references, paper_reference_links, observed 2026-08-05T12:35:51.957442Z
Paper Citation Record · LEDGER
Benchmarking Optimizers for Large Language Model Pretraining
As of 14 August 2026, this Paper Citation Record lists 100 of 176 outbound references and 28 inbound Pith citation observations for arXiv:2509.01440.
A citation records a reference. It does not transfer a finding from one paper to another.
Coverage vector
One-hop event checks from named stored sources.
Source: scholarly_work_events, retraction_status_cache, observed 2026-08-14T06:32:32.682623+00:00
Pith citing papers itemized under the disclosed page cap.
Source: paper_references, paper_reference_links, observed 2026-08-05T11:59:49.778357Z
A source-named dated measurement, never combined with another source.
Source: arxiv_reference, observed 2026-08-05T02:28:24.338817Z
Reference resolution
100 of 176 outbound references displayed
- verified exact0
- verified fuzzy3
- unresolved97
- parse uncertain0
- malformed identifier0
- metadata mismatch0
External citation measurements
0
arxiv_reference, observed 2026-08-05T02:28:24.338817Z
Outbound references
Observation f1e3d2e4-6b08-434e-ad32-7d90ef11da41 · outbound
This paper cites Dion: A communication-efficient optimizer for large models, 2025.
Benchmarking Optimizers for Large Language Model Pretraining Dion: A communication-efficient optimizer for large models, 2025
Reference 1
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.343838Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 63753a59-3c59-475d-be3d-de886728aeb2 · outbound
This paper cites an unresolved cited work.
Benchmarking Optimizers for Large Language Model Pretraining Unresolved cited work
Reference 2
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.353802Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 2d976c51-744f-4ac0-8423-f981979703e6 · outbound
This paper cites an unresolved cited work.
Benchmarking Optimizers for Large Language Model Pretraining Unresolved cited work
Reference 3
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.376510Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 3500af33-a6ce-4821-8c65-c2c3dc857f52 · outbound
This paper cites ASGO: Adaptive structured gradient optimization, 2025.
Benchmarking Optimizers for Large Language Model Pretraining ASGO: Adaptive structured gradient optimization, 2025
Reference 4
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.385438Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 8ad23d38-276b-4fa0-b950-4e69775ba4f7 · outbound
This paper cites Dissecting adam: The sign, magnitude and variance of stochastic gradients, 2020.
Benchmarking Optimizers for Large Language Model Pretraining Dissecting adam: The sign, magnitude and variance of stochastic gradients, 2020
Reference 5
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.396648Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation d65bb383-339a-4cbf-9f54-5dfddbcd63ef · outbound
This paper cites Bekas, E.
Benchmarking Optimizers for Large Language Model Pretraining Bekas, E
Reference 6
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.411974Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 99653034-b5f6-463c-8be3-ad2118a1de4c · outbound
This paper cites Straight to zero: Why linearly decaying the learning rate to zero works best for llms, 2025.
Benchmarking Optimizers for Large Language Model Pretraining Straight to zero: Why linearly decaying the learning rate to zero works best for llms, 2025
Reference 7
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.430977Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 6cb39b0b-ccc6-4f97-8da4-694e1ceb2c73 · outbound
This paper cites Old optimizer, new norm: An anthology, 2024.
Benchmarking Optimizers for Large Language Model Pretraining Old optimizer, new norm: An anthology, 2024
Reference 8
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.443584Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation b232b8d0-7608-44ad-bd59-65bcca0fa84b · outbound
This paper cites signsgd: Compressed optimisation for non-convex problems, 2018.
Benchmarking Optimizers for Large Language Model Pretraining signsgd: Compressed optimisation for non-convex problems, 2018
Reference 9
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.461957Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 9da88b8d-b881-4c84-ab66-b87e03d91e37 · outbound
This paper cites Pythia: A suite for analyzing large language models across training and scaling, 2023.
Benchmarking Optimizers for Large Language Model Pretraining Pythia: A suite for analyzing large language models across training and scaling, 2023
Reference 10
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.470832Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation ca24bbd4-073f-45d3-8332-faf66c88c7b5 · outbound
This paper cites Prince, Björn Deiseroth, Andres Felipe Cruz-Salinas, Carlo Luschi, Samuel Weinbach, and Douglas Orr.
Benchmarking Optimizers for Large Language Model Pretraining Prince, Björn Deiseroth, Andres Felipe Cruz-Salinas, Carlo Luschi, Samuel Weinbach, and Douglas Orr
Reference 11
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.480601Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation e8acd836-9d96-4b0f-b589-3a95b51d7fe0 · outbound
This paper cites Loss-to- loss prediction: Scaling laws for all datasets, 2024.
Benchmarking Optimizers for Large Language Model Pretraining Loss-to- loss prediction: Scaling laws for all datasets, 2024
Reference 12
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.488446Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 48f85189-d84d-4b0b-8315-30ee869344d0 · outbound
This paper cites an unresolved cited work.
Benchmarking Optimizers for Large Language Model Pretraining Unresolved cited work
Reference 13
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.496417Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation e99a0e4f-801e-4090-8542-f186ea656dfa · outbound
This paper cites How to scale your ema, 2023.
Benchmarking Optimizers for Large Language Model Pretraining How to scale your ema, 2023
Reference 14
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.503429Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation edcf4437-997a-4819-8877-53086dc35559 · outbound
This paper cites Preconditioned spectral descent for deep learning.
Benchmarking Optimizers for Large Language Model Pretraining Preconditioned spectral descent for deep learning
Reference 15
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.517235Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation bfc187f7-6eb5-4361-bf37-c31eec335328 · outbound
This paper cites Communication-efficient language model training scales reliably and robustly: Scaling laws for diloco, 2025.
Benchmarking Optimizers for Large Language Model Pretraining Communication-efficient language model training scales reliably and robustly: Scaling laws for diloco, 2025
Reference 16
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.525690Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation b635442a-42eb-4898-9d20-ebca781a4a6d · outbound
This paper cites an unresolved cited work.
Benchmarking Optimizers for Large Language Model Pretraining Unresolved cited work
Reference 17
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.536046Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 0218fbe8-da25-4d79-9dd4-28bcb6e12e43 · outbound
This paper cites Gradient clipping improves adagrad when the noise is heavy-tailed, 2024.
Benchmarking Optimizers for Large Language Model Pretraining Gradient clipping improves adagrad when the noise is heavy-tailed, 2024
Reference 18
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.545978Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation d15f6470-9b39-4ca2-8dd7-05ff919ba841 · outbound
This paper cites Palm: Scaling language modeling with pathways, 2022.
Benchmarking Optimizers for Large Language Model Pretraining Palm: Scaling language modeling with pathways, 2022
Reference 19
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.555425Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation f8d737d2-f2e1-4aa1-b0dd-5c0eaa05a356 · outbound
This paper cites an unresolved cited work.
Benchmarking Optimizers for Large Language Model Pretraining Unresolved cited work
Reference 20
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.580817Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 692e9966-eb88-4cf5-be99-1027a0932bb4 · outbound
This paper cites Why do we need weight decay in modern deep learning?, 2024.
Benchmarking Optimizers for Large Language Model Pretraining Why do we need weight decay in modern deep learning?, 2024
Reference 21
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.590632Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation aa651e1c-ed7c-4aa4-9046-acbf225be9e2 · outbound
This paper cites Fu, Stefano Ermon, Atri Rudra, and Christopher Ré.
Benchmarking Optimizers for Large Language Model Pretraining Fu, Stefano Ermon, Atri Rudra, and Christopher Ré
Reference 22
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.599608Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 3084708e-ad23-42b8-9fc3-7920679b9b97 · outbound
This paper cites Deepseek llm: Scaling open-source language models with longtermism, 2024.
Benchmarking Optimizers for Large Language Model Pretraining Deepseek llm: Scaling open-source language models with longtermism, 2024
Reference 23
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.606542Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 2d7c4d5d-fbb7-417a-b303-b8b142ed5d9b · outbound
This paper cites Deepseek-v3 technical report, 2024.
Benchmarking Optimizers for Large Language Model Pretraining Deepseek-v3 technical report, 2024
Reference 24
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.618917Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 7fe2fc43-b927-4b77-8a4a-7c57bd258c9f · outbound
This paper cites Why gradients rapidly increase near the end of training, 2025.
Benchmarking Optimizers for Large Language Model Pretraining Why gradients rapidly increase near the end of training, 2025
Reference 25
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.625839Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation e0a9cbd7-3d3d-43b0-ab96-743e009cb0ed · outbound
This paper cites Optimal linear decay learning rate schedules and further refinements, 2024.
Benchmarking Optimizers for Large Language Model Pretraining Optimal linear decay learning rate schedules and further refinements, 2024
Reference 26
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.637129Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 0bd0961f-59d0-4736-8b53-fa3953b2643a · outbound
This paper cites The road less scheduled, 2024.
Benchmarking Optimizers for Large Language Model Pretraining The road less scheduled, 2024
Reference 27
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.646543Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 0e9511fb-9e28-475e-acbb-6f7b128a5800 · outbound
This paper cites Bert: Pre-training of deep bidirectional transformers for language understanding, 2019.
Benchmarking Optimizers for Large Language Model Pretraining Bert: Pre-training of deep bidirectional transformers for language understanding, 2019
Reference 28
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.653793Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation c586caa8-8625-4451-9055-b8918360c51c · outbound
This paper cites The practitioner’s guide to the maximal update parameterization.
Benchmarking Optimizers for Large Language Model Pretraining The practitioner’s guide to the maximal update parameterization
Reference 29
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.660661Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation b33a0093-52e9-41f7-aefe-ae400211107a · outbound
This paper cites Incorporating Nesterov Momentum into Adam, 2016.
Benchmarking Optimizers for Large Language Model Pretraining Incorporating Nesterov Momentum into Adam, 2016
Reference 30
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.667043Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 3268a32e-81bb-4e54-a09f-a83543bb1881 · outbound
This paper cites Understanding emergent abilities of language models from the loss perspective, 2025.
Benchmarking Optimizers for Large Language Model Pretraining Understanding emergent abilities of language models from the loss perspective, 2025
Reference 31
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.675765Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 796c91ff-7051-4894-9d7e-a330a9565ede · outbound
This paper cites Adaptive subgradient methods for online learning and stochastic optimization.
Benchmarking Optimizers for Large Language Model Pretraining Adaptive subgradient methods for online learning and stochastic optimization
Reference 32
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.684220Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation a236d56d-8584-43e9-8502-d6f3fba362c5 · outbound
This paper cites A simple convergence proof of adam and adagrad, 2022.
Benchmarking Optimizers for Large Language Model Pretraining A simple convergence proof of adam and adagrad, 2022
Reference 33
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.692572Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 78e07626-7a34-4416-924a-e3bf94d4a431 · outbound
This paper cites Data movement bottlenecks to large-scale model training: Scaling past 1e28 flop,.
Benchmarking Optimizers for Large Language Model Pretraining Data movement bottlenecks to large-scale model training: Scaling past 1e28 flop,
Reference 34
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.699221Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 59524814-5713-4ec8-b181-b2e0eabbc3b5 · outbound
This paper cites Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity, 2022.
Benchmarking Optimizers for Large Language Model Pretraining Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity, 2022
Reference 35
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.716164Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 2772b691-3c1e-4765-86fb-a844a891ac4e · outbound
This paper cites A stable whitening optimizer for efficient neural network training, 2025.
Benchmarking Optimizers for Large Language Model Pretraining A stable whitening optimizer for efficient neural network training, 2025
Reference 36
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.728738Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 1b9f62ea-dde8-4235-878b-87c7d31b8cbb · outbound
This paper cites Dimakis, Gabriel Ilharco, Pang Wei Koh, Shuran Song, Thomas Kollar, Yair Carmon, Achal Dave, Reinhard Heckel, Niklas Muennighoff, and Ludwig Schmidt.
Benchmarking Optimizers for Large Language Model Pretraining Dimakis, Gabriel Ilharco, Pang Wei Koh, Shuran Song, Thomas Kollar, Yair Carmon, Achal Dave, Reinhard Heckel, Niklas Muennighoff, and Ludwig Schmidt
Reference 37
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.740367Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 835f6f7a-b65f-4488-bf9a-02c82ab72dad · outbound
This paper cites The Pile: An 800GB Dataset of Diverse Text for Language Modeling.
Benchmarking Optimizers for Large Language Model Pretraining The Pile: An 800GB Dataset of Diverse Text for Language Modeling
Reference 38
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.748594Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 80ad567a-7f6b-4aff-99e4-4ac75f04e1de · outbound
This paper cites The pile: An 800gb dataset of diverse text for language modeling, 2020.
Benchmarking Optimizers for Large Language Model Pretraining The pile: An 800gb dataset of diverse text for language modeling, 2020
Reference 39
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.766003Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 8b0d714c-c4f9-47a8-b0b7-36258def0546 · outbound
This paper cites Gemini: A family of highly capable multimodal models, 2024.
Benchmarking Optimizers for Large Language Model Pretraining Gemini: A family of highly capable multimodal models, 2024
Reference 40
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.776544Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 2081e7d5-38c0-4050-b7a7-cfbe7b09e64b · outbound
This paper cites A loss curvature perspective on training instability in deep learning, 2021.
Benchmarking Optimizers for Large Language Model Pretraining A loss curvature perspective on training instability in deep learning, 2021
Reference 41
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.786424Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 4af57d0a-2b59-4138-96a3-72aaddd8365a · outbound
This paper cites A minimalist optimizer design for llm pretraining, 2025.
Benchmarking Optimizers for Large Language Model Pretraining A minimalist optimizer design for llm pretraining, 2025
Reference 42
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.793612Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 0d192b09-2fc4-408a-b2fe-3a1ab03dd7f5 · outbound
This paper cites Generating sequences with recurrent neural networks, 2014.
Benchmarking Optimizers for Large Language Model Pretraining Generating sequences with recurrent neural networks, 2014
Reference 43
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.802389Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 9d8e5927-11b3-4eec-b73c-53b342eb21fe · outbound
This paper cites Accelerating newton-schulz iteration for orthogonalization via chebyshev-type polynomials, 2025.
Benchmarking Optimizers for Large Language Model Pretraining Accelerating newton-schulz iteration for orthogonalization via chebyshev-type polynomials, 2025
Reference 44
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.816714Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 353c38e4-9f8c-496c-83b4-0664eed62531 · outbound
This paper cites AdaPlus: Integrating nesterov momentum and precise stepsize adjustment on adamw basis, 2023.
Benchmarking Optimizers for Large Language Model Pretraining AdaPlus: Integrating nesterov momentum and precise stepsize adjustment on adamw basis, 2023
Reference 45
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.825415Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation abceb8a5-5972-4481-b633-7da75869a1a4 · outbound
This paper cites Shampoo: Preconditioned stochastic tensor optimization, 2018.
Benchmarking Optimizers for Large Language Model Pretraining Shampoo: Preconditioned stochastic tensor optimization, 2018
Reference 46
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.838173Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 9a5d9988-1f52-4382-b27a-8af80cf7ebc2 · outbound
This paper cites Under- standing and minimising outlier features in neural network training, 2024.
Benchmarking Optimizers for Large Language Model Pretraining Under- standing and minimising outlier features in neural network training, 2024
Reference 47
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.847579Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 4e150506-03f9-42ca-a4f1-16b9938798fe · outbound
This paper cites Deep residual learning for image recognition, 2015.
Benchmarking Optimizers for Large Language Model Pretraining Deep residual learning for image recognition, 2015
Reference 48
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.856289Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation fa584f48-d9d3-408a-8ea6-1ac6408545d8 · outbound
This paper cites an unresolved cited work.
Benchmarking Optimizers for Large Language Model Pretraining Unresolved cited work
Reference 49
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.866960Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation e80a794b-b7a4-48cf-a3e8-04fe3aeb54f1 · outbound
This paper cites Neural networks for machine learning, lecture 6e rmsprop: Divide the gradient by a running average of its recent magnitude, 2012.
Benchmarking Optimizers for Large Language Model Pretraining Neural networks for machine learning, lecture 6e rmsprop: Divide the gradient by a running average of its recent magnitude, 2012
Reference 50
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.875428Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation b225316f-3228-4134-873c-f2faf6e4d95b · outbound
This paper cites Rae, Oriol Vinyals, and Laurent Sifre.
Benchmarking Optimizers for Large Language Model Pretraining Rae, Oriol Vinyals, and Laurent Sifre
Reference 51
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.882146Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 73adb6cd-73a3-40a1-9be7-78c6242c9fff · outbound
This paper cites Minicpm: Unveiling the potential of small language models with scalable training strategies, 2024.
Benchmarking Optimizers for Large Language Model Pretraining Minicpm: Unveiling the potential of small language models with scalable training strategies, 2024
Reference 52
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.892741Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 80d4d8db-1738-4952-96b9-346906feed94 · outbound
This paper cites Improving transformer opti- mization through better initialization.
Benchmarking Optimizers for Large Language Model Pretraining Improving transformer opti- mization through better initialization
Reference 53
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.905305Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation a99c433d-c413-4835-b1ba-2d4a94238b8f · outbound
This paper cites Scaling laws and compute-optimal training beyond fixed training durations, 2024.
Benchmarking Optimizers for Large Language Model Pretraining Scaling laws and compute-optimal training beyond fixed training durations, 2024
Reference 54
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.919282Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation f50c55e8-f38c-4482-9a71-0bc3784b1f17 · outbound
This paper cites Averaging weights leads to wider optima and better generalization, 2019.
Benchmarking Optimizers for Large Language Model Pretraining Averaging weights leads to wider optima and better generalization, 2019
Reference 55
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.926941Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 13415e75-749f-4321-949c-d87048939e02 · outbound
This paper cites Intellect-1 technical report, 2024.
Benchmarking Optimizers for Large Language Model Pretraining Intellect-1 technical report, 2024
Reference 56
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.938288Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation f9d73ab2-3f1d-43a7-b8df-4e343381632d · outbound
This paper cites an unresolved cited work.
Benchmarking Optimizers for Large Language Model Pretraining Unresolved cited work
Reference 57
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.946890Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation d820ecaa-a9fa-4067-a3dc-f4840cd3c4de · outbound
This paper cites modded-nanogpt: Speedrunning the nanogpt baseline, 2024.
Benchmarking Optimizers for Large Language Model Pretraining modded-nanogpt: Speedrunning the nanogpt baseline, 2024
Reference 58
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.956877Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 38974e3d-c682-4d85-a70a-786b757ad391 · outbound
This paper cites Muon: An optimizer for hidden layers in neural networks, 2024.
Benchmarking Optimizers for Large Language Model Pretraining Muon: An optimizer for hidden layers in neural networks, 2024
Reference 59
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.965261Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation c16bd359-1f7d-46b2-be05-164a43f14643 · outbound
This paper cites an unresolved cited work.
Benchmarking Optimizers for Large Language Model Pretraining Unresolved cited work
Reference 60
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.981425Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 92f7ebe0-b223-439b-9169-aecb5f374815 · outbound
This paper cites Why warmup the learning rate? underlying mechanisms and improvements, 2024.
Benchmarking Optimizers for Large Language Model Pretraining Why warmup the learning rate? underlying mechanisms and improvements, 2024
Reference 61
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.990386Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation e13ef9ac-1d10-4655-becb-8e4788c544a7 · outbound
This paper cites Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei.
Benchmarking Optimizers for Large Language Model Pretraining Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei
Reference 62
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:50.998917Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 8a7e8f82-4584-47c5-a4db-6e4e0a4007a0 · outbound
This paper cites Error feedback fixes signSGD and other gradient compression schemes.
Benchmarking Optimizers for Large Language Model Pretraining Error feedback fixes signSGD and other gradient compression schemes
Reference 63
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:51.007417Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 337c6213-9588-41a8-a388-309d8927b8cc · outbound
This paper cites an unresolved cited work.
Benchmarking Optimizers for Large Language Model Pretraining Unresolved cited work
Reference 64
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:51.016356Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation b2aa18dd-7536-4971-b794-82731facaa9d · outbound
This paper cites an unresolved cited work.
Benchmarking Optimizers for Large Language Model Pretraining Unresolved cited work
Reference 65
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:51.023163Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 8b0ea370-d994-49a5-8950-e8350bee3995 · outbound
This paper cites Kingma and Jimmy Ba.
Benchmarking Optimizers for Large Language Model Pretraining Kingma and Jimmy Ba
Reference 66
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:51.029750Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation a3881c39-deb2-4d95-8888-0b1319dfe729 · outbound
This paper cites Sign operator for coping with heavy-tailed noise in non-convex optimization: High probability bounds under (l0,l 1)-smoothness, 2025.
Benchmarking Optimizers for Large Language Model Pretraining Sign operator for coping with heavy-tailed noise in non-convex optimization: High probability bounds under (l0,l 1)-smoothness, 2025
Reference 67
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:51.041783Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation cc26a4d6-5197-4e15-b87b-02d26576d30b · outbound
This paper cites Analyzing & reducing the need for learning rate warmup in gpt training, 2024.
Benchmarking Optimizers for Large Language Model Pretraining Analyzing & reducing the need for learning rate warmup in gpt training, 2024
Reference 68
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:51.048805Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 9bdf0811-3ca2-4b48-9288-c3cd48b5710d · outbound
This paper cites Rotational equilibrium: How weight decay balances learning across neural networks, 2024.
Benchmarking Optimizers for Large Language Model Pretraining Rotational equilibrium: How weight decay balances learning across neural networks, 2024
Reference 69
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:51.055398Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 8398ef5d-ff5e-4bcd-8e36-29e4c59cceaf · outbound
This paper cites Understanding gradient orthogonalization for deep learning via non-euclidean trust-region optimization, 2025.
Benchmarking Optimizers for Large Language Model Pretraining Understanding gradient orthogonalization for deep learning via non-euclidean trust-region optimization, 2025
Reference 70
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:51.060600Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation a4b82c0a-b051-4229-ba82-4f198e4c51e0 · outbound
This paper cites Spector, Blake Bordelon, Niklas Muennighoff, Mansheej Paul, Cengiz Pehlevan, Christopher Ré, and Aditi Raghunathan.
Benchmarking Optimizers for Large Language Model Pretraining Spector, Blake Bordelon, Niklas Muennighoff, Mansheej Paul, Cengiz Pehlevan, Christopher Ré, and Aditi Raghunathan
Reference 71
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:51.074620Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 0910f1f0-0f14-4d1d-aba3-0f1bdc01d2a5 · outbound
This paper cites Why do machine learning optimizers that work, work? PhD thesis, University of British Columbia, 2024.
Benchmarking Optimizers for Large Language Model Pretraining Why do machine learning optimizers that work, work? PhD thesis, University of British Columbia, 2024
Reference 72
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:51.085256Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 6fa78a03-397b-4db9-bdb9-d6eac5e83b0a · outbound
This paper cites Noise is not the main factor behind the gap between sgd and adam on transformers, but sign descent might be, 2023.
Benchmarking Optimizers for Large Language Model Pretraining Noise is not the main factor behind the gap between sgd and adam on transformers, but sign descent might be, 2023
Reference 73
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:51.098766Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 20c8f9dd-e67f-48dd-a6b4-ece308824be8 · outbound
This paper cites Heavy- tailed class imbalance and why adam outperforms gradient descent on language models, 2024.
Benchmarking Optimizers for Large Language Model Pretraining Heavy- tailed class imbalance and why adam outperforms gradient descent on language models, 2024
Reference 74
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:51.112981Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation b0fd45ae-a0e1-46aa-b445-6e6e38e53444 · outbound
This paper cites Farseer: A refined scaling law in large language models, 2025.
Benchmarking Optimizers for Large Language Model Pretraining Farseer: A refined scaling law in large language models, 2025
Reference 75
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:51.120086Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 9d666264-eeb7-4454-822e-29c3b35cf984 · outbound
This paper cites Predictable scale: Part i – optimal hyperparameter scaling law in large language model pretraining, 2025.
Benchmarking Optimizers for Large Language Model Pretraining Predictable scale: Part i – optimal hyperparameter scaling law in large language model pretraining, 2025
Reference 76
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:51.139881Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 7299b8a9-a840-4b50-b3df-edbd8a503584 · outbound
This paper cites Datacomp-lm: In search of the next generation of training sets for language models.
Benchmarking Optimizers for Large Language Model Pretraining Datacomp-lm: In search of the next generation of training sets for language models
Reference 77
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:51.152542Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 5da4ae20-002b-4624-b747-5b594098b239 · outbound
This paper cites Pytorch distributed: Experiences on accelerating data parallel training, 2020.
Benchmarking Optimizers for Large Language Model Pretraining Pytorch distributed: Experiences on accelerating data parallel training, 2020
Reference 78
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:51.163892Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 1c2cf729-c7d3-43ec-8d55-bf9798f8b90f · outbound
This paper cites Sophia: A scalable stochastic second-order optimizer for language model pre-training, 2024.
Benchmarking Optimizers for Large Language Model Pretraining Sophia: A scalable stochastic second-order optimizer for language model pre-training, 2024
Reference 79
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:51.172192Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 4061f7f4-ef2f-4a11-80c1-fc94bb050377 · outbound
This paper cites Same pre-training loss, better downstream: Implicit bias matters for language models, 2022.
Benchmarking Optimizers for Large Language Model Pretraining Same pre-training loss, better downstream: Implicit bias matters for language models, 2022
Reference 80
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:51.180465Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation bc4135a7-2505-4875-900b-84de062d0af9 · outbound
This paper cites Muon is scalable for llm training, 2025.
Benchmarking Optimizers for Large Language Model Pretraining Muon is scalable for llm training, 2025
Reference 81
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:51.191462Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation daaa7974-2a32-4a0b-8a88-d7b69a33f651 · outbound
This paper cites The llama 3 herd of models, 2024.
Benchmarking Optimizers for Large Language Model Pretraining The llama 3 herd of models, 2024
Reference 82
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:51.201700Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation a16d0fcb-fc91-4260-9bda-8d80a8687569 · outbound
This paper cites Sgdr: Stochastic gradient descent with warm restarts, 2017.
Benchmarking Optimizers for Large Language Model Pretraining Sgdr: Stochastic gradient descent with warm restarts, 2017
Reference 83
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:51.211642Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 22c52868-f616-4c9e-ab23-bd15809fe719 · outbound
This paper cites Decoupled weight decay regularization, 2019.
Benchmarking Optimizers for Large Language Model Pretraining Decoupled weight decay regularization, 2019
Reference 84
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:51.224611Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation e5d8e998-92e5-4744-aa5a-a4c761ed6385 · outbound
This paper cites SW AN: SGD with normalization and whitening enables stateless llm training, 2025.
Benchmarking Optimizers for Large Language Model Pretraining SW AN: SGD with normalization and whitening enables stateless llm training, 2025
Reference 85
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:51.235207Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 5dcd21a0-44d1-42aa-bd47-0b5060f0111c · outbound
This paper cites Small batch size training for language models: When vanilla sgd works, and why gradient accumulation is wasteful, 2025.
Benchmarking Optimizers for Large Language Model Pretraining Small batch size training for language models: When vanilla sgd works, and why gradient accumulation is wasteful, 2025
Reference 86
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:51.242809Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation b3c415c9-f333-43d9-862f-75f9d37f68fb · outbound
This paper cites New insights and perspectives on the natural gradient method, 2020.
Benchmarking Optimizers for Large Language Model Pretraining New insights and perspectives on the natural gradient method, 2020
Reference 87
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:51.248543Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation c9e0dec5-84c7-4103-aa90-4253ea486bb8 · outbound
This paper cites Effects of parameter norm growth during transformer training: Inductive bias from gradient descent, 2023.
Benchmarking Optimizers for Large Language Model Pretraining Effects of parameter norm growth during transformer training: Inductive bias from gradient descent, 2023
Reference 88
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:51.255443Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 0c7b8a1c-08c2-4457-8bb3-486055d882cc · outbound
This paper cites Mixed precision training, 2018.
Benchmarking Optimizers for Large Language Model Pretraining Mixed precision training, 2018
Reference 89
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:51.261498Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 8f1263d3-4151-4df4-bfb7-35b3d19913d9 · outbound
This paper cites Prodigy: An expeditiously adaptive parameter-free learner, 2024.
Benchmarking Optimizers for Large Language Model Pretraining Prodigy: An expeditiously adaptive parameter-free learner, 2024
Reference 90
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:51.275664Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation a421d4d2-fa90-4870-84d4-980d0a231179 · outbound
This paper cites Connections between schedule-free optimizers, ademamix, and accelerated sgd variants, 2025.
Benchmarking Optimizers for Large Language Model Pretraining Connections between schedule-free optimizers, ademamix, and accelerated sgd variants, 2025
Reference 91
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:51.358568Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 404f1cf9-6ae6-4461-87eb-25729cb3cdb2 · outbound
This paper cites Nemirovskii and Yu.E.
Benchmarking Optimizers for Large Language Model Pretraining Nemirovskii and Yu.E
Reference 92
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:51.419005Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation c51e241d-6516-4e23-b331-d3ca9c0a86d0 · outbound
This paper cites Nesterov and V.
Benchmarking Optimizers for Large Language Model Pretraining Nesterov and V
Reference 93
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:51.472058Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 3137ecfe-fb7e-417d-b06a-36dc41aae6f9 · outbound
This paper cites A method for unconstrained convex minimization problem with the rate of convergenceo(1/k2), 1983.
Benchmarking Optimizers for Large Language Model Pretraining A method for unconstrained convex minimization problem with the rate of convergenceo(1/k2), 1983
Reference 94
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:51.531481Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation fa5b3773-9138-402e-ada3-c120a8190b59 · outbound
This paper cites Gpt-4 technical report, 2024.
Benchmarking Optimizers for Large Language Model Pretraining Gpt-4 technical report, 2024
Reference 95
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:51.607895Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation b5dec2b1-a411-4fd3-8b95-6396e8d915f3 · outbound
This paper cites Neural networks (maybe) evolved to make adam the best optimizer, 2020.
Benchmarking Optimizers for Large Language Model Pretraining Neural networks (maybe) evolved to make adam the best optimizer, 2020
Reference 96
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:51.676668Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation a68501d0-3c31-42a5-a91b-d30c574888f2 · outbound
This paper cites Open problem: Parameter-free and scale-free online algorithms.
Benchmarking Optimizers for Large Language Model Pretraining Open problem: Parameter-free and scale-free online algorithms
Reference 97
- Resolution
- verified fuzzy
- raw_fallback, observed 2026-08-05T12:36:02.794621Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.
Observation 568a1e7d-7e71-4a89-88cc-4b6afea2f98d · outbound
This paper cites In search of adam’s secret sauce, 2025.
Benchmarking Optimizers for Large Language Model Pretraining In search of adam’s secret sauce, 2025
Reference 98
- Resolution
- verified fuzzy
- raw_fallback, observed 2026-08-05T12:36:02.768138Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.
Observation a8f6afa5-eb57-45d5-bda7-46bd96386e7a · outbound
This paper cites The ademamix optimizer: Better, faster, older, 2024.
Benchmarking Optimizers for Large Language Model Pretraining The ademamix optimizer: Better, faster, older, 2024
Reference 99
- Resolution
- verified fuzzy
- raw_fallback, observed 2026-08-05T12:36:02.733539Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.
Observation f12878bb-536d-4b0f-960d-a3a7eb347cb3 · outbound
This paper cites Pytorch: An imperative style, high-performance deep learning library, 2019.
Benchmarking Optimizers for Large Language Model Pretraining Pytorch: An imperative style, high-performance deep learning library, 2019
Reference 100
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T12:35:51.957442Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Pith citing papers
Observation fb443bfe-da23-4701-8a9a-d8ec61521e0f · inbound
Fantastic Pretraining Optimizers and Where to Find Them cites this paper.
Fantastic Pretraining Optimizers and Where to Find Them Benchmarking Optimizers for Large Language Model Pretraining
Reference 2025
- Resolution
- unresolved
- no resolver link, observed 2026-08-05T11:59:49.778357Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 949e6136-8c02-4f11-92d0-9bccef329228 · inbound
Preconditioned Norms: A Unified Framework for Steepest Descent, Quasi-Newton and Adaptive Methods cites this paper.
Preconditioned Norms: A Unified Framework for Steepest Descent, Quasi-Newton and Adaptive Methods Benchmarking Optimizers for Large Language Model Pretraining
Reference 16
- Resolution
- verified exact
- arxiv_id, observed 2026-05-21T21:10:38.756135Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.
Observation c30db491-0a24-427c-8803-312436a37102 · inbound
Optimal Sample Complexity for Single Time-Scale Actor-Critic with Momentum cites this paper.
Optimal Sample Complexity for Single Time-Scale Actor-Critic with Momentum Benchmarking Optimizers for Large Language Model Pretraining
Reference 50
- Resolution
- verified exact
- arxiv_id, observed 2026-05-16T08:17:36.487221Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.
Observation cb893229-810e-4b4b-a8fb-5b78c166f127 · inbound
LoRDO: Distributed Low-Rank Optimization with Infrequent Communication cites this paper.
LoRDO: Distributed Low-Rank Optimization with Infrequent Communication Benchmarking Optimizers for Large Language Model Pretraining
Reference 2014
- Resolution
- unresolved
- no resolver link, observed 2026-08-03T04:44:56.066179Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation bd78ae6a-878b-48cd-961f-806021dbee74 · inbound
HTMuon: Improving Muon via Heavy-Tailed Spectral Correction cites this paper.
HTMuon: Improving Muon via Heavy-Tailed Spectral Correction Benchmarking Optimizers for Large Language Model Pretraining
Reference 21
- Resolution
- verified exact
- arxiv_id, observed 2026-05-25T06:55:26.265757Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.
Observation c9deeb18-9d70-42b9-8189-95b541cf1845 · inbound
Sharp Capacity Scaling of Spectral Optimizers in Learning Associative Memory cites this paper.
Sharp Capacity Scaling of Spectral Optimizers in Learning Associative Memory Benchmarking Optimizers for Large Language Model Pretraining
Reference 48
- Resolution
- verified exact
- arxiv_id, observed 2026-05-14T23:38:16.407545Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.
Observation 2a8e356d-9410-4f35-93ef-67484520e0cb · inbound
Nexus: Same Pretraining Loss, Better Downstream Generalization via Common Minima cites this paper.
Nexus: Same Pretraining Loss, Better Downstream Generalization via Common Minima Benchmarking Optimizers for Large Language Model Pretraining
Reference 35
- Resolution
- verified exact
- arxiv_id, observed 2026-05-11T05:55:57.177884Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.
Observation e4f7eebb-04aa-417e-872d-8d2b80194fb9 · inbound
Evolution of Optimization Methods: Algorithms, Scenarios, and Evaluations cites this paper.
Evolution of Optimization Methods: Algorithms, Scenarios, and Evaluations Benchmarking Optimizers for Large Language Model Pretraining
Reference 25
- Resolution
- verified exact
- arxiv_id, observed 2026-05-11T10:51:20.489496Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.
Observation 32b87b17-45dd-41fe-b844-5e8b4f504424 · inbound
Benchmarking Optimizers for MLPs in Tabular Deep Learning cites this paper.
Benchmarking Optimizers for MLPs in Tabular Deep Learning Benchmarking Optimizers for Large Language Model Pretraining
Reference 11
- Resolution
- metadata mismatch
- arxiv_id, observed 2026-05-10T12:10:22.021599Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.
Observation 47e8c762-ed4f-428b-87fc-c65a0ce4027a · inbound
StoSignSGD: Unbiased Structural Stochasticity Fixes SignSGD for Training Large Language Models cites this paper.
StoSignSGD: Unbiased Structural Stochasticity Fixes SignSGD for Training Large Language Models Benchmarking Optimizers for Large Language Model Pretraining
Reference 36
- Resolution
- verified exact
- arxiv_id, observed 2026-05-10T12:15:22.218722Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.
Observation ab73c9d0-656a-4a0d-a2d1-9c563db4178b · inbound
When and Why SignSGD Outperforms SGD: A Theoretical Study Based on $\ell_1$-norm Lower Bounds cites this paper.
When and Why SignSGD Outperforms SGD: A Theoretical Study Based on $\ell_1$-norm Lower Bounds Benchmarking Optimizers for Large Language Model Pretraining
Reference 29
- Resolution
- verified exact
- arxiv_id, observed 2026-05-11T19:21:07.675545Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.
Observation c241f457-713a-41f8-9d3e-7a495da4d34b · inbound
Optimizer-Model Consistency: Full Finetuning with the Same Optimizer as Pretraining Forgets Less cites this paper.
Optimizer-Model Consistency: Full Finetuning with the Same Optimizer as Pretraining Forgets Less Benchmarking Optimizers for Large Language Model Pretraining
Reference 23
- Resolution
- verified exact
- arxiv_id, observed 2026-05-11T19:26:08.543311Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.
Observation 49fb2eb5-b6a4-4d35-a8c8-2845f077e2c9 · inbound
Navigating LLM Valley: From AdamW to Memory-Efficient and Matrix-Based Optimizers cites this paper.
Navigating LLM Valley: From AdamW to Memory-Efficient and Matrix-Based Optimizers Benchmarking Optimizers for Large Language Model Pretraining
Reference 37
- Resolution
- verified exact
- arxiv_id, observed 2026-05-12T06:41:45.519353Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.
Observation 5883819f-1012-4d65-a32d-7cdeca1deba4 · inbound
Rescaled Asynchronous SGD: Optimal Distributed Optimization under Data and System Heterogeneity cites this paper.
Rescaled Asynchronous SGD: Optimal Distributed Optimization under Data and System Heterogeneity Benchmarking Optimizers for Large Language Model Pretraining
Reference 32
- Resolution
- verified exact
- arxiv_id, observed 2026-05-14T19:32:50.855805Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.
Observation 68c55c15-3ee8-495a-88c3-3a8999dc5df4 · inbound
Symmetry-Compatible Principle for Optimizer Design: Embeddings, LM Heads, SwiGLU MLPs, and MoE Routers cites this paper.
Symmetry-Compatible Principle for Optimizer Design: Embeddings, LM Heads, SwiGLU MLPs, and MoE Routers Benchmarking Optimizers for Large Language Model Pretraining
Reference 134
- Resolution
- verified exact
- arxiv_id, observed 2026-05-20T09:38:11.217214Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.
Observation 1750ef5d-27b2-4566-848b-b9b4cd74497a · inbound
Symmetry-Compatible Principle for Optimizer Design: Embeddings, LM Heads, SwiGLU MLPs, and MoE Routers cites this paper.
Symmetry-Compatible Principle for Optimizer Design: Embeddings, LM Heads, SwiGLU MLPs, and MoE Routers Benchmarking Optimizers for Large Language Model Pretraining
Reference 136
- Resolution
- verified exact
- arxiv_id, observed 2026-06-30T18:45:00.393205Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.
Observation 56fd8440-a43e-42df-ad27-029d1533ea45 · inbound
LionMuon: Alternating Spectral and Sign Descent for Efficient Training cites this paper.
LionMuon: Alternating Spectral and Sign Descent for Efficient Training Benchmarking Optimizers for Large Language Model Pretraining
Reference 23
- Resolution
- verified exact
- arxiv_id, observed 2026-05-20T07:28:06.765568Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.
Observation 8fd65943-b119-450d-a15c-bc72839058d5 · inbound
LionMuon: Alternating Spectral and Sign Descent for Efficient Training cites this paper.
LionMuon: Alternating Spectral and Sign Descent for Efficient Training Benchmarking Optimizers for Large Language Model Pretraining
Reference 23
- Resolution
- verified exact
- arxiv_id, observed 2026-06-30T18:35:00.435900Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.
Observation cef63c76-0d48-4a8f-ab54-be36c37a7463 · inbound
Why SGD is not Brownian Motion: A New Perspective on Stochastic Dynamics cites this paper.
Why SGD is not Brownian Motion: A New Perspective on Stochastic Dynamics Benchmarking Optimizers for Large Language Model Pretraining
Reference 192
- Resolution
- verified exact
- arxiv_id, observed 2026-05-22T08:11:17.313797Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.
Observation 36e17f08-5a44-462f-b932-356011a2b584 · inbound
Reparametrizing Shampoo and SOAP for Subspace Basis Updates and BFloat16 Storage cites this paper.
Reparametrizing Shampoo and SOAP for Subspace Basis Updates and BFloat16 Storage Benchmarking Optimizers for Large Language Model Pretraining
Reference 22
- Resolution
- verified exact
- arxiv_id, observed 2026-06-29T22:24:00.706903Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.
Observation b9670b42-1ecb-444c-bd9d-520367ce8fa4 · inbound
Softsign: Smooth Sign in Your Optimizer For Better Parameter Heterogeneity Handling cites this paper.
Softsign: Smooth Sign in Your Optimizer For Better Parameter Heterogeneity Handling Benchmarking Optimizers for Large Language Model Pretraining
Reference 44
- Resolution
- verified exact
- arxiv_id, observed 2026-06-28T22:52:44.860967Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.
Observation 670a7054-6c7e-4b71-b263-8d575561617c · inbound
Denoise First, Orthogonalize Later: Understanding Momentum in Muon via Spectral Filtering cites this paper.
Denoise First, Orthogonalize Later: Understanding Momentum in Muon via Spectral Filtering Benchmarking Optimizers for Large Language Model Pretraining
Reference 46
- Resolution
- verified exact
- arxiv_id, observed 2026-07-02T01:56:27.815268Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.
Observation 68a78f89-2c6a-40fe-b4de-9242b71381ca · inbound
Open Problem: Is AdamW Effective Under Heavy-Tailed Noise? cites this paper.
Open Problem: Is AdamW Effective Under Heavy-Tailed Noise? Benchmarking Optimizers for Large Language Model Pretraining
Reference 7
- Resolution
- verified exact
- arxiv_id, observed 2026-07-04T10:09:44.602263Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.
Observation 9c5e31cf-cfa2-4bc6-ac80-97675cc31922 · inbound
OmniOpt: Taxonomy, Geometry, and Benchmarking of Modern Optimizers cites this paper.
OmniOpt: Taxonomy, Geometry, and Benchmarking of Modern Optimizers Benchmarking Optimizers for Large Language Model Pretraining
Reference 97
- Resolution
- unresolved
- no resolver link, observed 2026-07-11T22:10:49.683444Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation d3b04c17-064f-4229-9c1d-ccc1dcc9f23c · inbound
(A)iSpy: Parasitic Trojans for Machine Learning Infrastructure cites this paper.
(A)iSpy: Parasitic Trojans for Machine Learning Infrastructure Benchmarking Optimizers for Large Language Model Pretraining
Reference 71
- Resolution
- unresolved
- no resolver link, observed 2026-08-01T17:47:03.790519Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation d3922175-6ba6-404b-b07c-115aab2d1ab1 · inbound
PoLoRA: A Preconditioned Orthogonalized LoRA Optimizer cites this paper.
PoLoRA: A Preconditioned Orthogonalized LoRA Optimizer Benchmarking Optimizers for Large Language Model Pretraining
Reference 48
- Resolution
- unresolved
- no resolver link, observed 2026-08-01T17:32:35.613381Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 970c7596-d80c-40ac-8060-6449687b52f1 · inbound
Scale Weight Decay and Train Better cites this paper.
Scale Weight Decay and Train Better Benchmarking Optimizers for Large Language Model Pretraining
Reference 67
- Resolution
- unresolved
- no resolver link, observed 2026-07-30T12:53:41.174828Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 76c7a6db-6cbd-4f13-a74e-2226997896a9 · inbound
CMuon: Accelerating and Stabilizing Diffusion Transformer Training via Chunked Momentum Orthogonalization cites this paper.
CMuon: Accelerating and Stabilizing Diffusion Transformer Training via Chunked Momentum Orthogonalization Benchmarking Optimizers for Large Language Model Pretraining
Reference 26
- Resolution
- unresolved
- no resolver link, observed 2026-08-04T06:04:04.927838Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.