RoutePack: Expert Placement and Attention-Aware Data Packing for MoE Reinforcement Learning

Gen Li; Xiaowei Zhu; Xudong Han; Yibo Shen; Zhenxuan Pan

REVIEW 4 major objections 4 minor 44 references

RoutePack: Expert Placement and Attention-Aware Data Packing for MoE Reinforcement Learning

T0 review · 4 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read RoutePack uses rollout routing replay to jointly schedule expert placement and data packing in MoE reinforcement-learning training, reporting 8.85% and 14.89% trainer-measured token-throughput gains on two Ling-3.0 checkpoints.

desk verdict Real gains and a new mechanism combination, but the packing objective's 'exact' score contradicts the paper's own synchronized-row execution model, and the evidence base is thin. read the letter →

arxiv 2608.12146 v1 pith:SZHO6TLY submitted 2026-08-12 cs.DC cs.LG

Yibo Shen , Xudong Han , Xiaowei Zhu , Gen Li , Zhenxuan Pan This is my paper · ORCID

classification cs.DCcs.LG

keywords Mixture-of-Expertsreinforcementlearningexpertplacementdatapackingroutingreplayloadbalancingpopulationannealingdistributedtraining

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MoE reinforcement-learning training has two coupled load-balancing problems: the sequence lengths packed into each data-parallel microbatch set the dense attention tail, while token routing sets the sparse expert tail on expert-parallel ranks, and fixing either one alone can expose the other as the new straggler. The paper exploits a signal specific to MoE RL: rollout-time routing replay records every sample's length and per-layer expert demand before the optimizer step runs. RoutePack is a hierarchical planner that first places logical experts on physical ranks from aggregate demand, then packs whole samples into a fixed, minimum-feasible number of token-capped execution rows, scoring layouts by the accumulated joint attention-and-expert cost of the slowest expert-data-parallel (EDP) shard, the group of data-parallel slots that share one expert-parallel communicator. On two Ling-3.0 checkpoints the coordinated controls raise trainer-measured token throughput by 8.85% and 14.89% over the identity-placement, length-only baseline, with packing still contributing after reordering is applied. The case is that coordinated placement plus packing is a practical, semantics-preserving speedup for sparse RL training.

What carries the argument

The carrying mechanism is the packing-invariant lower bound of Equation (4), together with the lexicographic planner built around it. The bound states that, once a layer-wise placement $\pi_l$ is fixed, summed row-local expert peaks cannot fall below the busiest physical rank's optimizer-window total; this is why sample reordering has a hard floor and expert placement must be decided first. Layer-wise longest-processing-time (LPT) scheduling fixes one permutation per MoE layer from the aggregate replayed counts; a fixed-bin feasibility stage certifies, or best-known bounds, the minimum row count $R^*$ under hard constraints (token capacity, nonempty cells, sample bijection, rectangular $R\times D$ shape); and parallel population annealing then searches sample-to-cell assignments with a scalar energy that mirrors the exact lexicographic score. The joint row cost inside the score is the attention proxy $\sum_s \max_{d\in D_g}(\alpha_s \bar{T}_{r,d}+\beta_s \bar{Q}_{r,d})$ over attention stages plus the per-layer busiest-rank expert load $\sum_l \max_p W_{r,g,l,p}$, evaluated per EDP shard so that attention tails stay cell-local while expert tails aggregate across the shard's data-parallel slots. State-consistent materialization is the piece that lets the placement take effect without breaking training semantics: expert state is copied into pre-existing slot tensors at a step boundary, and only the logical-to-physical dispatch changes.

What would settle it

Two observations would settle the claim. First, inject a controlled replay-execution mismatch and execute each optimizer step with routing recorded $k$ policy updates earlier, then measure the throughput gain as $k$ grows; if the gain does not decline systematically, the exact-demand premise is not what carries the result. Second, run a synthetic workload with uniform expert popularity and equal-length responses, where the paper's own analysis predicts little room beyond length-only FFD packing; a large gain there would falsify the skew-rebalancing explanation.

Watch

Extended reading notes

Core claim

RoutePack's central claim is that routing replay turns data layout into a load-balancing control: because the planner knows the exact per-sample, per-layer expert counts $a_{i,l,e}$ before the optimizer step, it can coordinate where experts reside with which samples execute together. Placement and packing are not interchangeable. For a fixed placement, the summed row-local expert peaks are bounded below by the most-loaded physical rank's total over the optimizer window, $\sum_r \sum_g \max_p W_{r,g,l,p}(x;\pi_l) \ge \max_p L^{\mathrm{agg}}_{l,p}(\pi_l)$, so no permutation of samples can remove aggregate expert skew; conversely, a balanced aggregate placement does not stop correlated samples from overloading one rank within a single row. The planner therefore acts hierarchically: layer-wise longest-processing-time placement lowers the aggregate bound, a certified minimum row count $R^*$ is fixed, and fixed-row population annealing searches sample-to-cell assignments against the lexicographic objective (slowest EDP shard's accumulated joint cost, then total cost, then worst row-local tail), where the joint cost combines a window-normalized attention proxy with the busiest physical EP rank's load at each MoE layer. State-consistent materialization commits the placement by remapping logical experts to physical slots while preserving logical top-$k$ routing, capacity, sample coverage, equal microbatch counts, and communicator topology. The measured consequence is trainer-level token-throughput gains of 8.85% on Ling-3.0-Tiny and 14.89% on Ling-3.0-Flash, decomposing into 3.80% and 10.50% from reordering plus 4.86% and 3.98% from routing-aware packing.

Load-bearing premise

Everything rests on treating the recorded routing replay, each sample's sequence length and per-layer expert counts, as exactly the demand the upcoming optimizer step will execute, so if an asynchronous or off-policy runtime reroutes with a different model version, placement and packing are optimized for the wrong demand and the measured gains are not guaranteed.

Editorial extensions

If this is right

Expert reordering and routing-aware packing are complementary controls, not substitutes: each contributes independently on both models, so systems that do only length-based packing or only expert reordering leave measurable throughput on the table.
Aggregate balance and row-local balance are different targets: LPT removes more than 99% of the global EP-rank coefficient of variation, while packing cuts the worst row-local EP peak by roughly 11%, so suppressing both tails requires both controls.
The layout transformation is semantically transparent: because logical top-$k$ routing, sample coverage, capacity, equal microbatch counts, and communicator topology are preserved, any learning rule whose update is invariant to ordering and physical ownership of the sample multiset can consume the same packed layout without changing the objective.
Host-side planning need not extend training admission: when $T_{\mathrm{LPT}}+T_{\mathrm{pack}} \le \max(T_{\mathrm{LPT}},T_{\mathrm{aux}})+T_{\mathrm{actor}}$, CPU packing overlaps placement-aware state materialization and adds no admission straggler.
The payoff is workload-dependent: under uniform routing, near-equal response lengths, or very small planning windows, the paper expects little room beyond standard length-only packing, and the search budget can be turned down or skipped.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The mechanism's sensitivity can be measured directly: record routing from a policy several updates older than the one that trains, run the same packing pipeline, and track how the throughput gain decays with that replay lag; the paper's own Section 6.2 admits plan quality depends on replay matching executed routing, so this decay quantifies how much of the reported gain rests on the exactness assu
Because the joint objective deliberately trades a slightly higher attention proxy for lower expert tails, sweeping the attention weights in the row cost at a fixed row count should trace a Pareto frontier between attention-optimal and expert-optimal packings, a natural extension the current implementation does not explore.
The decomposition suggests a domain-scaling prediction: workloads with stronger routing diversity (coding, STEM, mixed-domain tasks, which the paper lists as future work) should show larger reordering gains, since LPT attacks an aggregate skew bound that grows with rank-popularity concentration, while single-domain math data may understate the effect.
The same exact pre-step demand signal could drive other optimizer-window controls beyond placement and packing, such as choosing which experts share a physical rank for communication locality or ordering expert-state transfers by memory pressure; RoutePack's pattern is the general one of planning from recorded demand, not the specific two controls.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

RoutePack is a genuinely useful systems paper: it identifies the coupling between expert placement and attention-aware packing in MoE RL, and it has a coherent hierarchical planner that keeps the row count fixed and preserves training semantics. The formal invariant in Eq. (4) is a nice touch—packing cannot remove aggregate expert skew—and the decomposition of measured gains into reordering vs. packing is exactly the right experiment. The related-work survey is careful, and ReLibra is positioned honestly as the closest prior art.

The main soft spot is a possible internal inconsistency in the packing objective. The paper defines an execution row as the synchronized scheduling unit (one microbatch per DP slot). Under that model, the step makespan should be sum over rows of the per-row maxima across EDP shards, i.e. sum_r max_g J_{r,g}. But Eq. (15) scores layouts by lexicographic minimization of max_g sum_r J_{r,g} (the slowest shard's total work). These two quantities disagree. A simple two-shard, two-row example shows the score can prefer a layout whose synchronized makespan is 200 over one whose makespan is 120. So the 'exact' planner score is not exact for the paper's stated execution semantics. Either the authors assume no per-row barrier across shards (which contradicts 'synchronized rows') or the packing search is optimizing the wrong quantity. This directly affects the claimed packing increment (4.86% / 3.98%) and the abstract's claim of minimizing the accumulated cost of the slowest shard. I'd want this clarified before trusting the numbers.

The other weaknesses are the usual ones: two proprietary checkpoints, one dataset, 49/96 valid steps, and step-level samples from a single run used in a Mann-Whitney test. The attention proxy coefficients are not reported. None of these are fatal in a systems paper, but they do cap my confidence at 'plausible' until the artifacts are out.

Bottom line: yes, send it to peer review. The idea is solid, the measurements are promising, and the objective question is resolvable. If the authors can fix Eq. (15) or clearly state the barrier assumption, this is a solid paper for a systems or MLSys venue.

Referee Report

4 major / 4 minor

Summary. The paper presents RoutePack, a hierarchical planner for MoE RL training that uses rollout-time routing replay to jointly decide layer-wise expert placement and whole-sample data packing. The planner first applies LPT-based expert placement per MoE layer on aggregate routing demand, then packs samples into a fixed, capacity-feasible number of execution rows and searches their DP layout with a lexicographic objective combining an attention proxy and per-layer physical EP-rank peaks, using diverse seeding and parallel population annealing. The evaluation on two Ling-3.0-family checkpoints reports trainer-measured token throughput gains of 8.85% and 14.89% over baseline, decomposed into gains from expert rerouting and from routing-aware packing. The paper also derives a sufficient condition for CPU-side packing not to extend the training-admission critical path.

Significance. If the reported results hold, RoutePack is a useful contribution to MoE RL systems: it demonstrates a concrete way to coordinate expert placement with attention-aware packing under a shared routing-replay signal, and the invariant in Eq. (4) crisply justifies why packing cannot remove aggregate expert skew. The state-consistent materialization contract is a thoughtful design that preserves logical top-k routing and existing kernels. However, the headline claim depends on load-bearing technical choices that need correction or clarification, and the empirical evidence is thin; the significance is therefore currently conditional on resolving the issues below.

major comments (4)

[Section 3.5, Eq. (15); Section 2.1; Figure 1] The 'exact planner score' is not exact for the execution model stated in Section 2.1. If an execution row is a synchronized scheduling unit with one microbatch per DP slot, all EDP shards in a row must finish before the next row starts, so the optimizer-step makespan is sum_r max_g J_{r,g}, not max_g sum_r J_{r,g} as in Eq. (15). These quantities can disagree: with two shards and two rows, layout A with J=[[100,0],[0,100]] has max_g sum_r = 100 but synchronized makespan 200, while layout B with J=[[60,60],[60,60]] has max_g sum_r = 120 but makespan 120; Eq. (15) prefers A. The packing search therefore may select layouts with worse trainer-measured throughput, and the abstract's claim that the objective 'minimizes the accumulated cost of the slowest EDP shard' is justified only if EDP shards are asynchronous pipelines without a cross-shard row barrier. The authors should either revise the objective to sum_r max_g J_{r,g} under the stated semantics, or explicitly state and defend the asynchronous-shard execution model, which currently contradicts the 'synchronized rows' language.
[Section 5.2, Table 3] The significance tests treat each optimizer-step throughput measurement as an independent sample, but all measurements within a configuration come from a single run and are serially correlated (sequence lengths and system state evolve). With 49 and 96 valid steps, the effective sample size for the Mann–Whitney tests is much smaller than the raw count, so the reported adjusted p-values (largest 0.0110) are not a reliable basis for the claim that every pairwise comparison is significant. The authors should either report run-level replication (e.g., multiple seeds per configuration) or account for autocorrelation with a block bootstrap or a time-series model; at minimum, they should present the autocorrelation of step-wise throughput and temper the significance claim.
[Section 3.4, Eq. (12); Section 5.3, Table 4] The attention proxy coefficients alpha_s and beta_s are free parameters, but the paper never states their values, how they were calibrated, or a sensitivity analysis. Because the joint objective trades attention cost against expert peaks (Table 4 shows attention increasing while joint cost decreases), the reported load-balance decomposition depends on the relative scale of these coefficients. If the coefficients are arbitrary, the 'joint bottleneck' metric is not anchored to measured attention time. Please report the coefficients used for KDA and MLA, their calibration source, and a sensitivity check showing that the ranking of layouts is stable over a plausible range.
[Section 5.2; Section 6.4] It is not stated whether the reported trainer-measured token throughput includes the planner's wall time, state materialization, and data admission. Section 6.4 admits the implementation does not yet realize the Section 4.2 schedule and that process/IPC/state-transfer costs may be exposed. If the planner runs outside the timed region, the headline 8.85%/14.89% gains overestimate the end-to-end speedup. The paper should specify what is included in the throughput measurement and, if planning is excluded, report its measured overhead on the same testbeds.

minor comments (4)

[Section 5.1] State how many optimizer-step records were excluded due to job restarts, and confirm that the means in Table 3 are computed only over the valid steps.
[Section 5.3, Table 4] The EP balance efficiency and Attention/Joint columns are reported without variance or significance; adding per-step standard deviations or confidence intervals would make the decomposition more convincing.
[Section 6.4] Since the exact checkpoints are not public, releasing the routing traces and a reference implementation (or a synthetic trace generator) would materially improve reproducibility.
[Section 3.5] The lexicographic tuple in Eq. (15) is called 'exact', but it is exact only in the sense of evaluating the stated proxy, not in the sense of matching true makespan; consider renaming it 'the planner score' to avoid overclaiming until the semantics in Eq. (15) are resolved.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the throughput claims are trainer-measured and the planner objective is not fitted to them.

full rationale

RoutePack's central claims are empirical. Section 5.2 states 'Table 3 reports trainer-measured token throughput' and the compared variants preserve the same optimizer-step samples and row count, so the 8.85%/14.89% gains are measured outcomes rather than outputs of the planner objective. The packing increment is isolated by design: 'Reorder applies only the layer-wise LPT expert reordering ... RoutePack adds fixed-row, routing-aware packing to the same reordering.' The objective in Eq. (15) is a search surrogate used to choose layouts; it is never used to compute the reported throughput or the statistical significance of the improvements. Eq. (4) is a genuine inequality derived from Eq. (3): summing the row/shard maxima dominates the maximum of the aggregate per-rank totals, so it is not a restatement of the conclusion. The LPT placement is a cited, parameter-free classical algorithm (Graham 1969), and the routing-replay signal is attributed to external systems (ReLibra and R3), not to a self-citation chain. The unreported alpha/beta attention coefficients are a reproducibility limitation, and Section 6.4's admission that the CPU-overlap schedule is not yet realized is an honest scope statement; neither constitutes a circular reduction. No step in the derivation equates a fitted parameter with a predicted outcome, and no load-bearing uniqueness theorem is imported from the authors' prior work.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The planner's input is assumed-exact replay; its objective is a compute proxy with unreported coefficients; the fixed-row constraints and numerical reordering assumptions are plausible but not verified in public artifacts. No new physical entities are introduced.

free parameters (3)

Attention proxy coefficients alpha_K, alpha_M, beta_M = not reported
Used in Equations (2) and (12) to convert sequence statistics into projected attention work. The paper says they may absorb layer counts or kernel calibration factors, but gives no values, so a reimplementation cannot reproduce the exact planner objective.
Annealing and search budgets Ppop, N_T, K, pguide, rho, w = not reported
Population size, temperature levels, SA steps per level, and proposal tuning are deployment knobs. They affect solution quality but not the formulation, and their absence limits exact replication.
Microbatch token capacity C = 8,192
A testbed configuration chosen by hand. It is a design input rather than a fitted constant, but it directly shapes the fixed-row packing feasibility.

assumptions (5)

domain assumption Rollout routing replay exactly predicts training routing demand for the optimizer step.
Section 2.1 introduces replay as exact pre-scheduling demand, and Section 6.2 admits that asynchronous or off-policy rerouting with a different model version requires refreshing the counts. If false, placement and packing are computed on stale demand.
domain assumption The linear-quadratic attention proxy and the per-layer single-rank expert peak capture the actual training bottleneck.
Section 3.4 defines A_{r,g} and E_{r,g}, while Section 6.4 concedes that kernel tile shapes, distribution of load among local experts, communication, and nonlinear operator behavior are not modeled. If the proxy diverges from real wall time, optimized layouts may not transfer to trainer-measured throughput.
domain assumption Sample coverage constraints in Equations (6)-(8) preserve the learning objective because the update is invariant to sample ordering and physical ownership.
Section 3.6 states this invariance requirement. It is reasonable for PPO or GRPO-style updates, but algorithms with order-dependent cross-sample statistics must materialize dependencies before planning.
domain assumption ULP-scale numerical drift from reordered dispatch and GEMM accumulation is acceptable and does not change training semantics.
Section 3.3 notes floating-point addition is not associative and accepts agreement within dtype tolerances. This is standard, but the paper does not test convergence sensitivity over many steps.
domain assumption Fixed-row feasibility, with a rectangular schedule and one nonempty microbatch per DP slot per row, is the right abstraction for training admission.
Used throughout Section 3.2. Pipeline requirements enter only as an additional lower bound on the row count. Runtimes that do not schedule synchronized rows would need a different model.

how reviews work

0 comments

Cite this review

Pith. "Pith review of RoutePack: Expert Placement and Attention-Aware Data Packing for MoE Reinforcement Learning." pith.science (2026). https://pith.science/paper/SZHO6TLY

@misc{pith2026260812146,
  author       = {Pith},
  title        = {Pith review of: RoutePack: Expert Placement and Attention-Aware Data Packing for MoE Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SZHO6TLY}},
  note         = {Machine review of arXiv:2608.12146}
}

read the original abstract

Training Mixture-of-Experts (MoE) models for reinforcement learning (RL) couples two load-balancing problems: sequence composition determines dense attention work in each data-parallel microbatch, while token routing determines sparse expert work on expert-parallel ranks. Optimizing either alone can shift the bottleneck to the other. In MoE RL, rollout-time routing replay exposes every sample's sequence length and layer-wise expert demand before its training step. We present RoutePack, a hierarchical planner that coordinates state-consistent, layer-wise expert rerouting with joint attention- and expert-aware data packing over an optimizer-step window. RoutePack first places experts independently at each MoE layer using aggregate routing demand. It then packs samples into the smallest certified, or best-known feasible, number of token-capped execution rows and optimizes their DP layout with a projected EDP-shard-aware objective. The objective combines a window-normalized linear-quadratic attention proxy with per-layer physical EP-rank peaks and minimizes the accumulated cost of the slowest EDP shard. Parallel population annealing searches fixed-row feasible layouts while preserving sample coverage, capacity, nonempty cells, equal microbatch counts, and communicator topology. State-consistent materialization preserves logical top-k routing and existing MoE kernels without microbatch-level expert replication. Across Ling-3.0-Tiny and Ling-3.0-Flash, expert rerouting improves mean trainer-measured token throughput by 3.80% and 10.50%, while routing-aware packing adds another 4.86% and 3.98%, respectively. Overall, RoutePack improves throughput by 8.85% and 14.89% over the baseline.

Figures

Figures reproduced from arXiv: 2608.12146 by the authors.

**Figure 2.** Routing replay aggregates exact per-sample, [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Layer-wise expert placement lowers the ag [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: State-consistent layer-wise expert place [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

**Figure 5.** Figure 5: Fixed-row Population Annealing. Parallel fixed-temperature chains resample terminal states while [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]

**Figure 6.** Figure 6: Example integration of RoutePack into AReaL. Routing replay drives layer-wise placement and EDP [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]

**Figure 7.** Figure 7: Global EP-rank CV before and after expert reordering. Measured bars show means and onestandard-deviation whiskers; Flash’s post-reordering marker denotes an upper bound imposed by logging resolution. Aggregate balance does not imply balanced execution rows. To quanti…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

44 extracted references · 21 canonical work pages

[1]

Joshua Ainslie, James Lee-Thorp, Michiel de Jong, Yury Zemlyanskiy, Federico Lebrón, and Sumit Sanghai. 2023. GQA: Training General- ized Multi-Query Transformer Models from Multi-Head Checkpoints. arXiv preprint arXiv:2305.13245(2023)

arXiv 2023
[2]

Ant Group. 2026. Ant Group Unveils Ling-3.0-Flash Delivering Top- Tier Performance at a Fraction of the Parameter Scale. Press release. Accessed: 2026-08-11. https://www.businesswire.com/news/home/ 20260726584441/en/

work page 2026
[3]

Ant Ling. 2025. AState: A High-Performance State Data Management System for Reinforcement Learning. Medium. https://ant-ling.medium.com/astate-a-high-performance-state-data- management-system-for-rl-dd1a8f04b8c4

work page 2025
[4]

Ant Ling. 2026. Ling Model Documentation. Official model documen- tation. Accessed: 2026-08-11. https://developer.ant-ling.com/en/docs/ models/ling/

work page 2026
[5]

AReaL Team. 2026. AReaL Data Packing Implementation. https: //github.com/areal-project/AReaL. Accessed: 2026-07-30

work page 2026
[6]

AReaL Team. 2026. Cluster Resource Configuration in AReaL. https:// github.com/inclusionAI/AReaL/blob/main/docs/cli_reference.md. Ac- cessed: 2026-08-10

work page 2026
[7]

James Carpenter, Peter Clifford, and Paul Fearnhead. 1999. An Im- proved Particle Filter for Non-linear Problems.IEE Proceedings—Radar, Sonar and Navigation146, 1 (1999), 2–7. doi:10.1049/ip-rsn:19990255

work page doi:10.1049/ip-rsn:19990255 1999
[8]

Yutian Chen, Cong Li, Yucheng Wang, and Ming Wei. 2026. MoonEP: A Perfectly Balanced Expert Parallelism Library via Dynamic Redundant Experts. https://github.com/MoonshotAI/MoonEP

work page 2026

Show all 44 references

[9]

Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman
[10]

Fu, Stefano Ermon, Atri Rudra, and Christopher Ré

Tri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra, and Christopher Ré
[11]

DeepSeek-AI. 2024. DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model.arXiv preprint arXiv:2405.04434 (2024)

2024 arXiv
[12]

DeepSeek-AI. 2024. DeepSeek-V3 Technical Report.arXiv preprint arXiv:2412.19437(2024)

2024 arXiv
[13]

DeepSeek-AI. 2026. DeepSeek-V4: Towards Highly Efficient Million- Token Context Intelligence.arXiv preprint arXiv:2606.19348(2026)

2026
[14]

Wei Fu, Jiaxuan Gao, Xujie Shen, Chen Zhu, Zhiyu Mei, Chuyi He, Shusheng Xu, Guo Wei, Jun Mei, Jiashu Wang, Tongkai Yang, Binhang Yuan, and Yi Wu. 2025. AReaL: A Large-Scale Asynchronous Rein- forcement Learning System for Language Reasoning.arXiv preprint arXiv:2505.24298(2025)

2025 arXiv
[15]

Ronald L. Graham. 1969. Bounds on Multiprocessing Timing Anom- alies.SIAM J. Appl. Math.17, 2 (1969), 416–429. doi:10.1137/0117039

1969 doi
[16]

Koji Hukushima and Yukito Iba. 2003. Population Annealing and Its Application to a Spin Glass. InAIP Conference Proceedings, Vol. 690. AIP Publishing, Melville, NY, 200–206. doi:10.1063/1.1632130

2003 doi
[17]

InclusionAI. 2026. Ling-3.0-Tiny. Official model card. Accessed: 2026-08-12. https://huggingface.co/inclusionAI/Ling-3.0-tiny

2026
[18]

Chao Jin, Xinming Wei, Yinmin Zhong, Chengxu Yang, Bingyang Wu, Ruidong Zhu, Zili Zhang, Yuliang Liu, and Xin Jin. 2026. Re- Libra: Routing-Replay-Guided Load Balancing for MoE Training in Reinforcement Learning.arXiv preprint arXiv:2605.08639(2026)

2026 arXiv
[19]

Kimi Team. 2025. Kimi Linear: An Expressive, Efficient Attention Architecture.arXiv preprint arXiv:2510.26692(2025)

2025 arXiv
[20]

Daniel Gelatt, and Mario P

Scott Kirkpatrick, C. Daniel Gelatt, and Mario P. Vecchi. 1983. Opti- mization by Simulated Annealing.Science220, 4598 (1983), 671–680. doi:10.1126/science.220.4598.671

1983 doi
[21]

Yan Li, Zhenyu Zhang, Zhengang Wang, Pengfei Chen, and Pengfei Zheng. 2025. Semantic Parallelism: Redefining Efficient MoE Infer- ence via Model–Data Co-Scheduling.arXiv preprint arXiv:2503.04398 (2025)

2025
[22]

Wenhan Ma, Hailin Zhang, Liang Zhao, Yifan Song, Yudong Wang, Zhifang Sui, and Fuli Luo. 2025. Stabilizing MoE Reinforcement Learning by Aligning Training and Inference Routers.arXiv preprint arXiv:2510.11370(2025)

2025
[23]

Deepak Narayanan, Mohammad Shoeybi, Jared Casper, Patrick LeGres- ley, Mostofa Patwary, Vijay Anand Korthikanti, Dmitri Vainbrand, Prethvi Kashinkunti, Julie Bernauer, Bryan Catanzaro, Amar Phan- ishayee, and Matei Zaharia. 2021. Efficient Large-Scale Language Model Training o...

2021
[24]

NVIDIA. 2026. Host Offload and Asynchronous Checkpointing in NeMo RL. https://docs.nvidia.com/nemo/rl/nightly/apidocs/nemo_rl/ nemo_rl.models.policy.workers.megatron_policy_worker.html. Ac- cessed: 2026-08-10

2026
[25]

NVIDIA. 2026. Sequence Packing and Dynamic Batching in NeMo RL. https://docs.nvidia.com/nemo/rl/latest/design-docs/sequence- packing-and-dynamic-batching.html. Accessed: 2026-07-30

2026
[26]

OpenRLHF Team. 2026. OpenRLHF: Sample Packing and Dynamic Batching. https://github.com/OpenRLHF/OpenRLHF. Accessed: 2026- 07-30

2026
[27]

Qwen Team. 2025. Qwen3-Next: Towards Ultimate Training and Inference Efficiency. Qwen Blog. Accessed: 2026-08-10. https://qwen. ai/blog?id=e34c4305036ce60d55a0791b170337c2b70ae51d

2025
[28]

Julien Rabin, Gabriel Peyré, Julie Delon, and Marc Bernot. 2011. Wasserstein Barycenter and Its Application to Texture Mixing. InScale Space and Variational Methods in Computer Vision. Springer, Berlin, Heidelberg, 435–446. doi:10.1007/978-3-642-24785-9_37

2011 doi
[29]

Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo
[30]

Yifan Sun, Zhexiang Zhang, Jiantong Jiang, Gholamreza Haffari, Minxian Xu, Feng Liu, Rajkumar Buyya, and Adel N. Toosi. 2026. Coordinated Scheduling for MoE LLM Serving.arXiv preprint arXiv:2606.15177(2026)

2026
[31]

Gomez, Lukasz Kaiser, and Illia Polosukhin

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention Is All You Need. InAdvances in Neural Information Processing Systems

2017
[32]

verl Team. 2026. CPU Reward Execution in verl. https://verl. readthedocs.io/en/v0.5.x/_modules/verl/trainer/ppo/reward.html. Ac- cessed: 2026-08-10

2026
[33]

verl Team. 2026. Sequence-Length Balancing in verl. https://verl. readthedocs.io/en/latest/_modules/verl/utils/seqlen_balancing.html. Accessed: 2026-07-30

2026
[34]

Yan Wang, Xiulong Yuan, Kaiming Yang, Jiaxuan Peng, Pengju Lu, Mingzhen Li, Zhipeng Zhang, Chang Si, Zhixiang Ruan, et al. 2026. Li- bra: Taming Attention Workload Skew in Long-Context LLM Training with Bounded Sequence Pool.arXiv preprint arXiv:2607.23250(2026)

2026 arXiv
[35]

Xinming Wei, Chao Jin, Tuo Dai, Yinmin Zhong, Shan Yu, Chengxu Yang, Bingyang Wu, Zili Zhang, Jing Mai, Qianchao Zhu, Zhouyang Li, Yuliang Liu, and Guojie Luo. 2026. UltraEP: Unleash MoE Training and Inference on Rack-Scale Nodes with Near-Optimal Load Balancing. arXiv preprin...

2026 arXiv
[36]

Zhenda Xie, Yixuan Wei, Huanqi Cao, Chenggang Zhao, Chengqi Deng, Jiashi Li, Damai Dai, Huazuo Gao, Jiang Chang, Liang Zhao, et al. 2025. mHC: Manifold-Constrained Hyper-Connections.arXiv preprint arXiv:2512.24880(2025)

2025 arXiv
[37]

Chenqi Zhao, Wenfei Wu, Linhai Song, Yuchen Xu, and Yitao Yuan
[38]

Chenggang Zhao, Shangyan Zhou, Liyue Zhang, Chengqi Deng, Zhean Xu, Yuxuan Liu, Kuai Yu, Jiashi Li, and Liang Zhao. 2025. DeepEP: An Efficient Expert-Parallel Communication Library. https://github.com/ deepseek-ai/DeepEP

2025
[39]

Yuming Zhou, Haoyang Li, Sheng Lin, Yanfeng Zhao, Tong Zhao, Xupeng Miao, Jie Jiang, Fangcheng Fu, and Bin Cui. 2026. Harnessing Routing Foresight for Micro-step-level MoE Load Balancing in RL Post-training.arXiv preprint arXiv:2606.11867(2026)

2026 arXiv
[40]

Zilin Zhu, Chengxing Xie, Xin Lv, and slime Contributors. 2026. slime: Data Packing and Dynamic Batching. https://github.com/THUDM/ slime. Accessed: 2026-08-12. 16

2026
[2021]

Training Verifiers to Solve Math Word Problems.arXiv preprint arXiv:2110.14168(2021)

2021 arXiv
[2022]

Work in progress

FlashAttention: Fast and Memory-Efficient Exact Attention with 15 Preprint. Work in progress. IO-Awareness. InAdvances in Neural Information Processing Systems
[2024]

DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models.arXiv preprint arXiv:2402.03300(2024)

2024 arXiv
[2025]

FineMoE: Fine-Grained Load Balancing for Mixture-of-Experts with Token Scheduling.arXiv preprint arXiv:2511.16947(2025)

2025

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.

[1] [1]

Joshua Ainslie, James Lee-Thorp, Michiel de Jong, Yury Zemlyanskiy, Federico Lebrón, and Sumit Sanghai. 2023. GQA: Training General- ized Multi-Query Transformer Models from Multi-Head Checkpoints. arXiv preprint arXiv:2305.13245(2023)

arXiv 2023

[2] [2]

Ant Group. 2026. Ant Group Unveils Ling-3.0-Flash Delivering Top- Tier Performance at a Fraction of the Parameter Scale. Press release. Accessed: 2026-08-11. https://www.businesswire.com/news/home/ 20260726584441/en/

work page 2026

[3] [3]

Ant Ling. 2025. AState: A High-Performance State Data Management System for Reinforcement Learning. Medium. https://ant-ling.medium.com/astate-a-high-performance-state-data- management-system-for-rl-dd1a8f04b8c4

work page 2025

[4] [4]

Ant Ling. 2026. Ling Model Documentation. Official model documen- tation. Accessed: 2026-08-11. https://developer.ant-ling.com/en/docs/ models/ling/

work page 2026

[5] [5]

AReaL Team. 2026. AReaL Data Packing Implementation. https: //github.com/areal-project/AReaL. Accessed: 2026-07-30

work page 2026

[6] [6]

AReaL Team. 2026. Cluster Resource Configuration in AReaL. https:// github.com/inclusionAI/AReaL/blob/main/docs/cli_reference.md. Ac- cessed: 2026-08-10

work page 2026

[7] [7]

James Carpenter, Peter Clifford, and Paul Fearnhead. 1999. An Im- proved Particle Filter for Non-linear Problems.IEE Proceedings—Radar, Sonar and Navigation146, 1 (1999), 2–7. doi:10.1049/ip-rsn:19990255

work page doi:10.1049/ip-rsn:19990255 1999

[8] [8]

Yutian Chen, Cong Li, Yucheng Wang, and Ming Wei. 2026. MoonEP: A Perfectly Balanced Expert Parallelism Library via Dynamic Redundant Experts. https://github.com/MoonshotAI/MoonEP

work page 2026