Pith. sign in

REVIEW 2 major objections 4 minor 47 references

MACRO: Markov Chain Routing of Transformer Layers

T0 review · 2 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read MACRO claims that task-specific layer rerouting of frozen LLMs—via a Markov-chain policy with top-k Viterbi decoding—improves accuracy by 5.0 percentage points on average over the standard layer order.

desk verdict A novel and well-engineered routing method whose headline gains are mostly a byproduct of an artificially weak baseline; the central practical claim needs a re-evaluation under standard inference settings. read the letter →

arxiv 2608.05872 v1 pith:BBZCER55 submitted 2026-08-06 cs.CL cs.AI

Pawe{l} Batorski , Abtin Pourhadi , Akylgali Aitaza , Przemys{l}aw Spurek , Paul Swoboda This is my paper · ORCID
classification cs.CLcs.AI
keywords layerroutingMarkovchainViterbidecodingfrozenlanguagemodelsdynamiccomputationtest-timeadaptationestimation-of-distributionsearch
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that you can make a frozen large language model more accurate on a given task by changing only the order in which its layers run: skipping some, repeating others, and occasionally adding a previous hidden state. Existing dynamic-routing methods need per-instance search, ground-truth labels at test time, or expensive offline label generation. MACRO instead learns one Markov-chain policy per task from training feedback, decodes a small set of high-probability routes with top-k Viterbi, and picks the best on a held-out validation split. If correct, this means accuracy gains are available without any weight updates and with far less route search than prior approaches. The headline numbers are +5.0% average accuracy over the sequential baseline and +7.2% over Dr.LLM, with route-search time cut by 9.4x.

What carries the argument

The central object is the Markov route generator: a masked, context-dependent transition policy $\pi_\theta(a_t | s_t)$ over states $s_t = (\ell, \phi, \delta, o)$ recording the current layer, the computation-budget phase (early, mid, late), the incoming displacement, and the operator context, with three action families: local layer moves within a small radius, add-and-apply of a previous hidden state, and rejoin of the standard suffix. Top-k Viterbi decoding over this augmented state space is the exact decoder that turns the learned transition table into a small set of candidate routing programs without any per-example test-time search.

What would settle it

Re-run the final selection stage with the validation labels randomly permuted, or with two disjoint 100-example validation pools, and compare the selected routes and their test accuracies; if a route selected on shuffled labels still produces the +5% test gain, the improvement is not driven by genuine route quality. A second check is to hold out a second validation set and verify that the validation-best route is also the test-best among the five Viterbi candidates.

Watch

Extended reading notes

Core claim

MACRO models layer routing as a context-dependent Markov chain whose state records the current layer, the remaining budget phase, the incoming displacement, and whether the hidden state came from a plain layer or an add operation; actions are local layer moves, add-and-apply, and rejoin. It learns a single task-level transition table by sampling routes, scoring them on training subsets, and refitting via weighted maximum likelihood, a structured cross-entropy approach. At deployment it decodes the exact top-k highest-probability valid routes with an augmented Viterbi pass, deduplicates them, and selects one on validation data. The claim is that this route, applied uniformly at test time, improves accuracy over the sequential forward pass on thirteen benchmarks across several open-weight LLMs, with the largest gains on weak and distilled models.

Load-bearing premise

The load-bearing premise is that ranking five candidate routes on a 100-example validation split reliably identifies a route that is genuinely better on the test distribution; if the validation sample is too small or noisy, some reported accuracy gains could be a small-sample selection artifact.

Editorial extensions

If this is right

  • On six open-weight LLMs across thirteen benchmarks, MACRO reports an average +5.0 percentage-point accuracy gain over the sequential baseline, with the largest gains on the weakest models.
  • MACRO's route search costs about 1.6 hours per benchmark versus 14.8 hours for Dr.LLM, a 9.4x reduction, while improving accuracy by +7.2 points.
  • The action space itself is a major contributor: Dr.LLM+Ext., which uses the MACRO action space with Dr.LLM's training procedure, outperforms original Dr.LLM on every tested model.
  • Both decoding and validation selection matter: MACRO(noVit.) and MACRO(top-1) are individually weaker than full MACRO, and validation selection often picks a non-rank-1 Viterbi candidate.
  • A single route trained on a pooled set of eight benchmarks transfers out-of-domain, especially to math reasoning tasks, suggesting routes encode reusable computation rather than only benchmark-specific quirks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the validation-selection result generalizes, the method's implicit claim is that a large pretrained model often represents the right answer internally but suppresses it near the output; the logit-lens evidence points to routing as exposing an already-computed answer, which would make readout-pathway interventions a promising test-time-compute direction.
  • A testable extension is input-conditional routing: the paper learns one route per task, but the state space already contains enough structure to condition on a lightweight input embedding, which could push gains on mixed-task streams.
  • The Markov-table formulation could also apply to other frozen computation graphs, such as vision backbones or diffusion denoisers, where a small per-task transition table over a few stages would be even cheaper to learn.
  • The 100-example validation selection is the main fragility; a direct comparison with a much larger validation pool or leave-one-benchmark-out selection would show how much of the reported gain is route quality versus selection luck.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper proposes MACRO, a framework that learns task-specific layer-routing programs for frozen LLMs. Routing is modeled as a context-dependent Markov policy over layer indices, budget phases, displacements, and operator context, updated via an estimation-of-distribution procedure on training feedback and decoded with an exact top-k Viterbi algorithm. The authors report a +5.0% average accuracy improvement over a sequential baseline across six open-weight LLMs and thirteen benchmarks, a +7.2% margin over Dr.LLM, and a 9.4x reduction in route-search time. They also present ablations, a mechanistic interpretability analysis, and route-transfer experiments.

Significance. If the empirical claims hold, MACRO is a well-engineered contribution: it is parameter-free with respect to the LLM (no weight updates), requires no per-instance test-time search, and its top-k Viterbi decoding is argued to be exact over the augmented state space. The paper is commendable for its detailed experimental protocol, including hyperparameter sweeps, statistical significance tests, ablations of each state component, and a public code release. The mechanistic logit-lens analysis is a useful step toward understanding why repeated or revisited layers can help. However, the significance of the headline accuracy gains depends critically on whether the unrouted baseline is a fair, standard inference configuration.

major comments (2)
  1. [Appendix C ('Splits and Decoding') and Table 2] The baseline decoding configuration appears to be non-standard and likely artificially weak. Appendix C specifies deterministic greedy decoding with at most 400 thinking tokens and 128 answer tokens, but it does not mention the chat template, system prompt, few-shot exemplars, or chain-of-thought prompting. Under this configuration, the reported DeepSeek-R1-Distill-Llama-8B baseline on GSM8K is 20.67%, which is far below the model's typical standard-inference performance (often above 80%). If the baseline is set to a non-standard or insufficiently prompted configuration, the headline claim of a +5.0% average improvement over the unrouted baseline is not a meaningful measure of practical gain. The paper's own observation that gains are largest where the baseline is weakest (e.g., Table 2, DeepSeek row) underscores this concern. The authors should either specify and use a standard chat/CoT prompting setup for all methods and baselines, or explicitly scope the claim as an improvement over a constrained inference budget, and report results under both configurations.
  2. [Section 'Main Results' / Table 2 and Table 3] The statistical significance tests and the per-model average improvements are computed against the same weak baseline. For Qwen3-8B, Qwen3-14B, and Llama-3.2-3B, the gains are small (+0.76, +0.78, +0.35 points respectively), while the large apparent gains on Qwen3-1.7B (+12.68) and DeepSeek-R1-Distill-Llama-8B (+13.04) coincide with baselines that are implausibly low. The paper should report results with a properly configured baseline (including chat template and reasoning prompts) and re-run the significance analysis; otherwise the claim that 'every model improves significantly' is only about a weak reference point.
minor comments (4)
  1. [Section 'Final Selection' and Appendix C] The final route is selected among five Viterbi candidates on a 100-example validation split. While MACRO(top-1) largely reproduces the gains, which mitigates the risk, the paper would benefit from an analysis of validation-rank stability across seeds, or a bootstrap estimate of the selection noise, to quantify how much of the final reported accuracy is attributable to selection on a small validation set.
  2. [Table 2, footnote/legend] The table reports mean and standard deviation over three seeds, but for the baseline rows no variance is given. It would be helpful to report the seed-level variance of the baseline as well, since the significance tests treat the baseline as deterministic.
  3. [Appendix B (Table B.1) and Figure 2] Some of the discovered routes are complex (e.g., Qwen3-1.7B on MS-Arith: 'L0-L2 L4 L3-L7 L7 RJ') and the route notation is dense. A short worked example of how to read a route string (beyond the current caption) would improve reproducibility.
  4. [Appendix D] The inference-cost analysis reports average overhead (+16% latency, +4.6% memory) but does not discuss the variance across benchmarks; the text says overhead is 'uneven' but only the average is highlighted. Reporting per-benchmark overhead with the same granularity as Figure D.1 would be more informative.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: MACRO's test accuracy is an external held-out measurement; route fitting and validation selection are standard model selection, not a fitted quantity renamed as a prediction.

full rationale

The paper's central claim is empirical, not a derivation. A Markov route policy is fitted on training feedback, decoded with an exact top-k Viterbi procedure, and the selected route is then evaluated on a held-out test split. The Method section states: 'The test set is held out and used only for final evaluation' and 'The test set is used only once, to report the final accuracy of the selected route r-hat.' Validation is used only to select among a small set of Viterbi-decoded candidates, which is standard model selection rather than a definitional link to the test result. No equation reduces a reported accuracy gain to an input fit: Eq. (1) defines the optimization objective, and the update rule reweights transition counts by training accuracy, while the final reported accuracy is measured on data not used for fitting or selection. There are no load-bearing self-citations; the cited routing baselines (Dr.LLM, CoLA, Neuroanatomy) are external prior works, and the Markov-chain and Viterbi tools are attributed to standard external sources such as Norris (1998). The appendix's 'Connection to CEM' transparently identifies the search as a structured Cross-Entropy Method, so there is no renaming of a known result. Concerns about the 100-example validation split, greedy decoding configuration, or weak baselines are threats to experimental validity or generalization, not circularity. No circular step was found.

Assumptions & free parameters 10 free parameters · 5 assumptions · 0 invented entities

The central contribution is an empirical search method; the free parameters are hyperparameters of the search, not constants derived from theory. The main domain assumptions are that frozen models contain routable redundancy and that a small validation set can select among decoded routes.

free parameters (10)
  • local move radius r = 5
    Controls the set of local layer moves plus or minus r in the action space. Set by the authors; not swept in the main sensitivity analysis.
  • add history window = 6
    Add-and-apply actions can pull a hidden state from up to six steps back. Chosen without a dedicated ablation.
  • add coefficient gamma = 1.0
    Scaling for the added hidden state. Subtraction and bare merge actions are disabled in main runs.
  • initial policy exploration eps_loc = 0.2
    Probability mass for local moves and Rejoin in the initial forward-biased prior.
  • initial policy exploration eps_op = 0.0867
    Probability mass for add-and-apply actions in the initial prior.
  • update smoothing alpha = 0.0397
    Pseudo-count smoothing in the policy update; keeps unvisited states near the initial prior.
  • selection sharpness beta = 1.8262
    Softmax temperature for converting training accuracies into update weights. Swept in Figure 4; accuracy is flat in beta.
  • elite count m = 5
    Number of top candidates used in the update. Swept in Figure 4; m=1 overfits, m=12 dilutes.
  • search iterations and programs per iteration = 10 and 30
    Computational budget for the search. Not swept in the paper.
  • validation split size = 100 (200 in baseline-anchored runs)
    Size of the held-out validation set used to select the final route. This is the load-bearing small sample.
assumptions (5)
  • standard math Viterbi decoding over the augmented state (current layer, budget phase, incoming delta, operator context, previous layer) returns the exact top-k programs under the masked Markov policy.
    The policy is Markovian in the augmented state and the reachable program space is a finite DAG, so retaining the k best continuations per state is exact. Argued in Appendix C, 'Exactness of top-k decoding'.
  • domain assumption A Markov chain over layer indices, budget phases, displacements, and operator context is a sufficiently expressive policy class for finding accuracy-improving routes.
    The paper motivates this by stating Markov chains are lightweight and constrain the decision space; the state-space ablation shows all factors matter.
  • domain assumption Frozen transformer layers contain redundant or reusable computation such that skipping, repeating, and adding hidden states can improve task accuracy.
    This is the core hypothesis of the paper. Evidence is empirical (Table 2, Figure 2).
  • domain assumption Each program step gets its own attention cache slot, so revisited blocks behave like a tied-weight network with separate key/value histories.
    Stated in the Method section under 'Markov Route Generator'. This is an implementation choice that preserves the transformer's per-step attention semantics.
  • ad hoc to paper A 100-example validation split provides a reliable signal for selecting among the five Viterbi candidate routes.
    Chosen by the authors; not justified with an analysis of selection noise. This is the weakest load-bearing premise in the evaluation protocol.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MACRO: Markov Chain Routing of Transformer Layers." pith.science (2026). https://pith.science/paper/BBZCER55

@misc{pith2026260805872,
  author       = {Pith},
  title        = {Pith review of: MACRO: Markov Chain Routing of Transformer Layers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BBZCER55}},
  note         = {Machine review of arXiv:2608.05872}
}
read the original abstract

Standard Large Language Models (LLMs) execute layers sequentially. Dynamic layer routing, i.e. search for a different execution path through layers involving layer repetitions, skips and other moves, can improve performance. Existing routing approaches often require updating model weights, running expensive search loops per test instance, or demand ground-truth labels during inference. In this work, we propose Markov Chain Routing of Transformer Layers (MACRO), a framework that learns task-specific routes over LLM architectures without modifying underlying parameters. MACRO models layer routing as a context-dependent Markov policy conditioned on layer indices, computation budget phases, directional displacements, and operator context, supporting skip, repeat, and residual hidden-state addition operations. The Markov route distribution is updated via feedback on training data and decoded using a top-k Viterbi algorithm to isolate high-probability candidate programs. We evaluate MACRO across diverse reasoning and knowledge benchmarks on multiple open-weight LLMs. MACRO achieves a +5.0% average accuracy improvement over the unrouted baselines, with largest gains on small models. We outperform the best dynamic routing approach Dr. LLM by +7.2%, while reducing route-search time 9.4x (from 14.8 to 1.6 hours). Our code is publicly available at https://github.com/Batorskq/MACRO.

Figures

Figures reproduced from arXiv: 2608.05872 by the authors.

Figure 1
Figure 1. Against Dr.LLM, MACRO improves mean accuracy while reducing route-search time from 14.8 to 1.6 hours per [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. A single MACRO route on Qwen3-1.7B for GSM8K. The baseline executes the 28 transformer blocks once, front to back. The searched route runs L0–L7, rewinds to L3, and then continues to L27, so L3–L7 execute twice and 33 block calls are made in total. The weights are frozen and only the execution order changes. To overcome these limitations, we introduce Markov Chain Routing of Transformer Layers (MACRO). We for￾malize… view at source ↗
Figure 3
Figure 3. The left panel summarizes the complete workflow. [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Accuracy on Qwen3-1.7B when sweeping the se [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 6
Figure 6. Figure 6: Route-transfer heatmap on Qwen3-1.7B, with red [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 5
Figure 5. Figure 5: Logit-lens log-probability of the correct answer at [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 40 canonical work pages

  1. [1]

    Bae, S.; Kim, Y.; Bayat, R.; Kim, S.; Ha, J.; Schuster, T.; Fisch, A.; Harutyunyan, H.; Ji, Z.; Courville, A.; et al. 2026. Mixture-of-recursions: Learning dynamic recursive depths for adaptive token-level computation. Advances in Neural Information Processing Systems, 38: 96572--96617

    work page 2026

  2. [2]

    Chen, Y.; Pan, X.; Li, Y.; Ding, B.; and Zhou, J. 2024. EE - LLM : Large-Scale Training and Inference of Early-Exit Large Language Models with 3 D Parallelism. In International Conference on Machine Learning, 7163--7189

    work page 2024

  3. [3]

    Chen, Y.; Shang, J.; Zhang, Z.; Xie, Y.; Sheng, J.; Liu, T.; Wang, S.; Sun, Y.; Wu, H.; and Wang, H. 2025. Inner thinking transformer: Leveraging dynamic depth scaling to foster adaptive internal thinking. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 28241--28259

    work page 2025

  4. [4]

    Cobbe, K.; Kosaraju, V.; Bavarian, M.; Chen, M.; Jun, H.; Kaiser, L.; Plappert, M.; Tworek, J.; Hilton, J.; Nakano, R.; et al. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168

    arXiv 2021

  5. [5]

    Dehghani, M.; Gouws, S.; Vinyals, O.; Uszkoreit, J.; and Kaiser, . 2018. Universal transformers. arXiv preprint arXiv:1807.03819

    arXiv 2018

  6. [6]

    Elbayad, M.; Gu, J.; Grave, E.; and Auli, M. 2020. Depth-Adaptive Transformer. In International Conference on Learning Representations

    work page 2020

  7. [7]

    Elhoushi, M.; Shrivastava, A.; Liskovich, D.; Hosmer, B.; Wasti, B.; Lai, L.; Mahmoud, A.; Acun, B.; Agarwal, S.; Roman, A.; Aly, A.; Chen, B.; and Wu, C.-J. 2024. L ayer S kip: Enabling Early Exit Inference and Self-Speculative Decoding. In Ku, L.-W.; Martins, A.; and Srikumar, V., eds., Proceedings of the 62nd Annual Meeting of the Association for Compu...

    work page 2024

  8. [8]

    Fan, A.; Grave, E.; and Joulin, A. 2020. Reducing Transformer Depth on Demand with Structured Dropout. In International Conference on Learning Representations

    work page 2020

Show all 47 references
  1. [9]

    Fedus, W.; Zoph, B.; and Shazeer, N. 2022. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. Journal of Machine Learning Research, 23(120): 1--39

    2022

  2. [10]

    Gao, L.; Madaan, A.; Zhou, S.; Alon, U.; Liu, P.; Yang, Y.; Callan, J.; and Neubig, G. 2023. Pal: Program-aided language models. In International conference on machine learning, 10764--10799. PMLR

    2023

  3. [11]

    M.; Jain, N.; Kirchenbauer, J.; Singh, S.; Bartoldson, B

    Geiping, J.; McLeish, S. M.; Jain, N.; Kirchenbauer, J.; Singh, S.; Bartoldson, B. R.; Kailkhura, B.; Bhatele, A.; and Goldstein, T. 2026. Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach. In The Thirty-ninth Annual Conference on Neural Informatio...

    2026

  4. [12]

    D.; and Papailiopoulos, D

    Giannou, A.; Rajput, S.; Sohn, J.-y.; Lee, K.; Lee, J. D.; and Papailiopoulos, D. 2023. Looped transformers as programmable computers. In International Conference on Machine Learning, 11398--11442. PMLR

    2023

  5. [13]

    He, S.; Ge, T.; Sun, G.; Tian, B.; Wang, X.; and Yu, D. 2025. Router-Tuning: A Simple and Effective Approach for Dynamic Depth. In Christodoulopoulos, C.; Chakraborty, T.; Rose, C.; and Peng, V., eds., Proceedings of the 2025 Conference on Empirical Methods in Natural Language...

    2025

  6. [14]

    Heakl, A.; Gubri, M.; Khan, S.; Yun, S.; and Oh, S. J. 2026. Dr. LLM : Dynamic Layer Routing in LLM s. In The Fourteenth International Conference on Learning Representations

    2026

  7. [15]

    Hou, L.; Huang, Z.; Shang, L.; Jiang, X.; Chen, X.; and Liu, Q. 2020. DynaBERT : Dynamic BERT with Adaptive Width and Depth. In Advances in Neural Information Processing Systems, volume 33, 9782--9793

    2020

  8. [16]

    H.; Sakmann, K.; Zhang, D.; Naesseth, C

    Jazbec, M.; Timans, A.; Veljkovi\' c , T. H.; Sakmann, K.; Zhang, D.; Naesseth, C. A.; and Nalisnick, E. 2024. Fast yet Safe: Early-Exiting with Risk Control. In Advances in Neural Information Processing Systems, volume 37, 129825--129854

    2024

  9. [17]

    Jin, D.; Pan, E.; Oufattole, N.; Weng, W.-H.; Fang, H.; and Szolovits, P. 2021. What disease does this patient have? a large-scale open domain question answering dataset from medical exams. Applied Sciences, 11(14): 6421

    2021

  10. [18]

    Koncel-Kedziorski, R.; Roy, S.; Amini, A.; Kushman, N.; and Hajishirzi, H. 2016. MAWPS : A Math Word Problem Repository. In Knight, K.; Nenkova, A.; and Rambow, O., eds., Proceedings of the 2016 Conference of the North A merican Chapter of the Association for Computational Lin...

    2016

  11. [19]

    Lepikhin, D.; Lee, H.; Xu, Y.; Chen, D.; Firat, O.; Huang, Y.; Krikun, M.; Shazeer, N.; and Chen, Z. 2021. GShard : Scaling Giant Models with Conditional Computation and Automatic Sharding. In International Conference on Learning Representations

    2021

  12. [20]

    Li, Q.; Cui, L.; Zhao, X.; Kong, L.; and Bi, W. 2024. Gsm-plus: A comprehensive benchmark for evaluating the robustness of llms as mathematical problem solvers. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ...

    2024

  13. [21]

    Li, Z.; Li, Y.; and Zhou, T. 2025. Skip a Layer or Loop it? Test-Time Depth Adaptation of Pretrained LLMs. arXiv preprint arXiv:2507.07996

    2025 arXiv

  14. [22]

    Lightman, H.; Kosaraju, V.; Burda, Y.; Edwards, H.; Baker, B.; Lee, T.; Leike, J.; Schulman, J.; Sutskever, I.; and Cobbe, K. 2024. Let's Verify Step by Step. In The Twelfth International Conference on Learning Representations

    2024

  15. [23]

    Liu, W.; Zhou, P.; Wang, Z.; Zhao, Z.; Deng, H.; and Ju, Q. 2020. F ast BERT : a Self-distilling BERT with Adaptive Inference Time. In Jurafsky, D.; Chai, J.; Schluter, N.; and Tetreault, J., eds., Proceedings of the 58th Annual Meeting of the Association for Computational Lin...

    2020

  16. [24]

    Liu, Y.; Meng, F.; Zhou, J.; Chen, Y.; and Xu, J. 2021 a . Faster depth-adaptive transformers. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, 13424--13432

    2021

  17. [25]

    Liu, Z.; Li, F.; Li, G.; and Cheng, J. 2021 b . EBERT : Efficient BERT Inference with Dynamic Structured Pruning. In Zong, C.; Xia, F.; Li, W.; and Navigli, R., eds., Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, 4814--4823. Online: Association fo...

    2021

  18. [26]

    Luo, X.; Wang, W.; and Yan, X. 2025. Adaptive Layer-skipping in Pre-trained LLM s. In Second Conference on Language Modeling

    2025

  19. [27]

    Men, X.; Xu, M.; Zhang, Q.; Yuan, Q.; Wang, B.; Lin, H.; Lu, Y.; Han, X.; and Chen, W. 2025. Shortgpt: Layers in large language models are more redundant than you expect. In Findings of the Association for Computational Linguistics: ACL 2025, 20192--20204

    2025

  20. [28]

    Miao, S.-Y.; Liang, C.-C.; and Su, K.-Y. 2020. A diverse corpus for evaluating and developing English math word problem solvers. In Proceedings of the 58th annual meeting of the Association for Computational Linguistics, 975--984

    2020

  21. [29]

    Mihaylov, T.; Clark, P.; Khot, T.; and Sabharwal, A. 2018. Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering. In Riloff, E.; Chiang, D.; Hockenmaier, J.; and Tsujii, J., eds., Proceedings of the 2018 Conference on Empirical Methods in Natu...

    2018

  22. [30]

    Ng, D. N. 2026. LLM Neuroanatomy: How I Topped the LLM Leaderboard Without Changing a Single Weight. Online

    2026

  23. [31]

    Norris, J. R. 1998. Markov chains. 2. Cambridge university press

    1998

  24. [32]

    Patel, A.; Bhattamishra, S.; and Goyal, N. 2021. Are NLP models really able to solve simple math word problems? In Proceedings of the 2021 conference of the North American chapter of the association for computational linguistics: human language technologies, 2080--2094

    2021

  25. [33]

    C.; and Santoro, A

    Raposo, D.; Ritter, S.; Richards, B.; Lillicrap, T.; Humphreys, P. C.; and Santoro, A. 2024. Mixture-of-depths: Dynamically allocating compute in transformer-based language models. arXiv preprint arXiv:2404.02258

    2024 arXiv

  26. [34]

    Schuster, T.; Fisch, A.; Gupta, J.; Dehghani, M.; Bahri, D.; Tran, V.; Tay, Y.; and Metzler, D. 2022. Confident Adaptive Language Modeling. In Advances in Neural Information Processing Systems, volume 35, 17456--17472

    2022

  27. [35]

    Shazeer, N.; Mirhoseini, A.; Maziarz, K.; Davis, A.; Le, Q.; Hinton, G.; and Dean, J. 2017. Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer. In International Conference on Learning Representations

    2017

  28. [36]

    W.; Chowdhery, A.; Le, Q.; Chi, E.; Zhou, D.; et al

    Suzgun, M.; Scales, N.; Sch \"a rli, N.; Gehrmann, S.; Tay, Y.; Chung, H. W.; Chowdhery, A.; Le, Q.; Chi, E.; Zhou, D.; et al. 2023. Challenging big-bench tasks and whether chain-of-thought can solve them. In Findings of the Association for Computational Linguistics: ACL 2023,...

    2023

  29. [37]

    Teerapittayanon, S.; McDanel, B.; and Kung, H.-T. 2016. Branchynet: Fast inference via early exiting from deep neural networks. In 2016 23rd international conference on pattern recognition (ICPR), 2464--2469. IEEE

    2016

  30. [38]

    Wang, X.; Yu, F.; Dou, Z.-Y.; Darrell, T.; and Gonzalez, J. E. 2018. SkipNet: Learning Dynamic Routing in Convolutional Networks. In Proceedings of the European Conference on Computer Vision, 409--424

    2018

  31. [39]

    Wang, Y.; Ma, X.; Zhang, G.; Ni, Y.; Chandra, A.; Guo, S.; Ren, W.; Arulraj, A.; He, X.; Jiang, Z.; et al. 2024. Mmlu-pro: A more robust and challenging multi-task language understanding benchmark. Advances in Neural Information Processing Systems, 37: 95266--95290

    2024

  32. [40]

    F.; and Gardner, M

    Welbl, J.; Liu, N. F.; and Gardner, M. 2017. Crowdsourcing multiple choice science questions. In Proceedings of the 3rd Workshop on Noisy User-generated Text, 94--106

    2017

  33. [41]

    Wu, Q.; Ke, Z.; Zhou, Y.; Sun, X.; and Ji, R. 2025. Routing Experts: Learning to Route Dynamic Experts in Existing Multi-modal Large Language Models. In The Thirteenth International Conference on Learning Representations

    2025

  34. [42]

    S.; Grauman, K.; and Feris, R

    Wu, Z.; Nagarajan, T.; Kumar, A.; Rennie, S.; Davis, L. S.; Grauman, K.; and Feris, R. 2018. BlockDrop: Dynamic Inference Paths in Residual Networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 8817--8826

    2018

  35. [43]

    Xin, J.; Tang, R.; Lee, J.; Yu, Y.; and Lin, J. 2020. D ee BERT : Dynamic Early Exiting for Accelerating BERT Inference. In Jurafsky, D.; Chai, J.; Schluter, N.; and Tetreault, J., eds., Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2...

    2020

  36. [44]

    Yang, L.; Lee, K.; Nowak, R.; and Papailiopoulos, D. 2024. Looped transformers are better at learning learning algorithms. In International conference on learning representations, volume 2024, 42195--42214

    2024

  37. [45]

    Zhao, A.; Ye, F.; Fan, Y.; Tong, J.; Xiong, J.; Fei, Z.; Su, H.; and Shen, X. 2025. S kip GPT : Each Token is One of a Kind. In International Conference on Machine Learning, 77844--77863

    2025

  38. [46]

    Zhou, W.; Xu, C.; Ge, T.; McAuley, J.; Xu, K.; and Wei, F. 2020. Bert loses patience: Fast and robust inference with early exit. Advances in Neural Information Processing Systems, 33: 18330--18341

    2020

  39. [47]

    Zhu, R.-J.; Wang, Z.; Hua, K.; Zhang, T.; Li, Z.; Que, H.; Wei, B.; Wen, Z.; Yin, F.; Xing, H.; et al. 2025. Scaling Latent Reasoning via Looped Language Models. arXiv preprint arXiv:2510.25741

    2025 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.