Pith. sign in

REVIEW 2 major objections 5 minor 59 references

Dynamic parameterization alone does not establish dynamic inference; a frozen-controller audit is needed to separate coefficient variation from functional dependence and from actual compute savings.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 00:30 UTC pith:6RONDF5A

load-bearing objection A genuinely useful diagnostic and a convincing negative result for FeatureGate, with a real but plausibly fixable gap in the replay verification for MUDDPythia. the 2 major comments →

arxiv 2607.26192 v1 pith:6RONDF5A submitted 2026-07-28 cs.LG

Dynamic Parameterization Is Not Dynamic Inference

Zongfei Li , Yuan-yih Shang , Guozhong Luo This is my paper
classification cs.LG
keywords dynamic inferencedynamic parameterizationfrozen-controller auditingconditional computationresidual gatingcross-layer mixinglanguage modelingefficiency measurement
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that three things often bundled together under 'dynamic inference' are separate: whether a controller's coefficients vary across inputs, whether a frozen model's output depends on which coefficient goes with which input, and whether computation is actually skipped. It introduces Frozen-Controller Auditing (FCA), which caches the controller's coefficients on an undisturbed pass, turns the controller off, and replays the frozen model with those coefficients reassigned, shuffled, or replaced by a static profile; any loss change then measures assignment dependence, not controller feedback. Applied to seven 76M and three 504M FeatureGate transformers, FCA finds that a static layerwise profile retains 98.70% and 99.43% of the Correct-to-GlobalMean performance gap and that layer identity explains 87–96% of coefficient variance, while FeatureGate still runs every block and is 30.8% slower than Dense. Applied to the public MUDDPythia-1.4B checkpoint, the same audit finds cross-input reassignment and token shuffling raise NLL by 1.9067 and 2.9637, showing strong dependence on content-conditioned cross-layer assignment—yet MUDDPythia also executes every block. The upshot is that reports on dynamic models should separately document coefficient variation, functional dependence, and actual execution.

Core claim

On its own terms, the paper establishes that input-dependent controller coefficients are neither necessary nor sufficient for dynamic inference. It defines three independent properties—coefficient variation, functional dependence of a frozen model on coefficient assignment, and conditional execution—and shows that evidence for one is not evidence for another. FCA operationalizes the second property by caching the full coefficient tensor along the unperturbed trajectory, disabling the controller, and replaying the frozen model under reassignment, shuffling, and static profiles. The empirical finding is two-sided: FeatureGate's coefficients vary, but a static layerwise profile reproduces nearl

What carries the argument

Frozen-Controller Auditing (FCA): a two-stage intervention protocol. Stage one caches the controller's complete coefficient tensor A(x) (indexed by target layer, token position, computation stream, and source layer) along the unmodified forward pass; stage two disables the controller and replays the frozen model with the cached tensor either unchanged, reassigned across inputs, token-shuffled, or replaced by a static profile estimated from an independent calibration set. Because coefficients are cached before any perturbation, ΔNLL between replay and the correct trajectory isolates dependence on coefficient assignment from feedback through the controller. Supporting machinery includes paired

Load-bearing premise

The audit assumes that the cached coefficient tensor A(x) is the only channel through which the controller influences the frozen model; if the controller also steers skips, masks, or normalization statistics, replay differences would blur those channels into assignment dependence, and exact replay is verified only on the first batch.

What would settle it

Run FCA's exact-replay check on every evaluation window, not just the first batch, and compare NLL and logits between the normal forward pass and the replay of cached coefficients. For the MUDDPythia-1.4B checkpoint, compute the maximum logit deviation across all 512 test windows; if any nonzero deviation appears outside the first batch, the cached-tensor assumption fails and the reported ΔNLL values mix in controller feedback. More broadly, a model that both varies its coefficients and demonstrably skips layers with measured latency savings would be a counterexample to a blanket reading that

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • A controller may vary substantially across inputs and tokens while a static, input-independent depth profile reproduces nearly the same inference function; FeatureGate at 76M and 504M retains 98.70% and 99.43% of the Correct-to-GlobalMean performance gap with a layerwise static profile.
  • Large functional dependence on dynamic cross-layer assignment (MUDDPythia: +1.9067 NLL cross-input, +2.9637 NLL token-shuffle) is compatible with no compute savings; MUDDPythia executes every Transformer block.
  • Dynamic parameterization does not imply conditional execution: soft gating in FeatureGate adds a controller that runs on every token and makes inference 30.8% slower than Dense rather than faster.
  • Claims about dynamic models should separately report coefficient variation, functional dependence of the frozen model, and actual execution (blocks executed, FLOPs, latency, throughput), because no single number can substantiate all three.
  • Input and token disruption effects are not simply additive; negative interaction in both models indicates overlapping damage, so intervention costs should be reported as paired contrasts rather than summed mechanisms.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • FCA's cache-and-replay logic extends beyond residual gating and cross-layer mixing: the same protocol could audit mixture-of-experts routing or early-exit policies, reassigning expert assignments or exit thresholds while holding the frozen weights fixed, to separate routing's functional contribution from its efficiency claim.
  • The static-profile results suggest a cheap post-training test for any soft-gated Transformer: freeze the controller, fit one coefficient per layer on a holdout set, and compare performance; if the layerwise profile retains most of the gap, the content-dependent component may be mostly a training-time regularizer rather than an inference-time decision.
  • Because static conditions match coefficient means, not effective residual updates (which also depend on hidden states and block outputs), the reported retention numbers should be read as upper bounds on how replaceable a controller is, not exact function equivalence.
  • The MUDDPythia result shows FCA is sensitive enough to detect genuinely functional dynamics, so the near-static FeatureGate result is unlikely to be an artifact of an insensitive protocol.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper argues that input-dependent controller coefficients are often conflated with dynamic inference or computational savings, and proposes a three-way distinction: coefficient variation, functional dependence of the frozen model on coefficient assignment, and actual execution. It introduces Frozen-Controller Auditing (FCA), which caches the full coefficient tensor on an unperturbed trajectory, disables the controller, and replays the frozen model under exact replay, cross-input reassignment, token shuffling, and static profiles estimated from an independent calibration set. FCA is applied to seven 76M FeatureGate Transformers, three 504M FeatureGate models, and the public MUDDPythia-1.4B checkpoint. For FeatureGate, assignment interventions incur penalties of about 1e-3 NLL and static layerwise profiles retain 98.70% (76M) and 99.43% (504M) of the Correct-to-GlobalMean gap; for MUDDPythia, cross-input and token-shuffle penalties are 1.9067 and 2.9637 NLL, with 73.33% retention relative to a different structure-removed reference. Both model families execute every Transformer block; FeatureGate is 30.8% slower than Dense. The paper concludes that coefficient variation, functional dependence, and execution are separable properties that require separate evidence.

Significance. The paper makes a valuable conceptual contribution by operationalizing the distinction between dynamic parameterization and dynamic inference. The FCA protocol is carefully designed in several respects: coefficients are cached before any intervention, reassignment conditions preserve the coefficient multiset, static profiles come from an independent calibration set, absolute ΔNLL penalties are reported alongside retention ratios, and the exact-replay check is explicitly included as a validity condition. The empirical contrast between FeatureGate (nearly static function) and MUDDPythia (strong assignment dependence) is compelling and demonstrates that the audit can separate these cases. If the replay intervention is fully validated, the findings would provide a useful template for evaluating dynamic architectures and would strengthen the field's evidentiary standards.

major comments (2)
  1. [Sections 3.2, 4.4] The audit's validity condition—"intervention results are valid only if this replay reproduces the original logits and NLL"—is not verified on the full evaluation set. The only explicit verification reported is "the logits of the first batch" for MUDDPythia (Section 4.4); the scope of verification for FeatureGate is unspecified, despite Algorithm 1 suggesting per-example checks. This is load-bearing because FeatureGate's ΔNLL values are ~1e-3, so even a small replay mismatch could dominate the measured effects. The authors should either verify exact replay for all evaluation windows or provide a deterministic-equivalence argument explaining why first-batch verification suffices for the entire set.
  2. [Sections 3.2–3.3] FCA assumes that the cached coefficient tensor A(x) is the only channel through which the controller affects the frozen model. If the controller also influences other state—e.g., skip decisions, attention masks, or normalization statistics—not represented in A(x), then replaying the frozen model with cached A(x) will not isolate coefficient-assignment dependence. The exact-replay check is the only safeguard for this assumption, but its limited reported scope leaves the assumption unproven for the full evaluation set. The paper should state the architectural guarantee that all controller outputs are contained in A(x) (or cache any additional state) and verify the replay condition on all examples.
minor comments (5)
  1. [Section 4.3 / Figure 3] The variance decomposition method is not described. The claim that "layer identity explains 87% to 96% of coefficient variance" needs an explicit definition of the balanced functional variance analysis, including the formula and how interaction terms are computed.
  2. [Section 4.1] The "exact two-sided sign-flip tests" are mentioned but not defined. Please describe the test procedure or provide a citation.
  3. [Section 4.5] The statement about post-training thresholding of FeatureGate causing "meaningful block skipping only when PPL degrades severely" is presented without supporting details or a figure. Please add the experimental evidence or remove the claim.
  4. [Figure 2] The dual-scale x-axis (linear for small penalties, separate scale for GlobalMean) is clear enough, but the figure caption should clarify that the GlobalMean point is not to scale relative to the other points.
  5. [General] No code availability statement is provided. Since reproducibility is a stated goal, please include a link to released code or explicitly state that code will be released.

Circularity Check

0 steps flagged

No circularity found: FCA derives its conclusions from cached-coefficient interventions, independently calibrated static profiles, and direct execution measurements, not from its own definitions.

full rationale

The paper's central claims are not circular. FCA's intervention penalties (Eqs. 2-4) are defined as the change in NLL when cached coefficients are replayed under reassignment, shuffling, or static replacement. The static profiles are estimated from an independent calibration set (512 WT103 validation windows) and evaluated on nonoverlapping test windows, so the static-profile result is not fitted to the evaluation set. The retention statistic (Eq. 8) is reported alongside the raw ΔNLL values and is not used as the sole evidence; the paper explicitly says retention must be interpreted with absolute penalties and only within the same model. The MUDDPythia results are obtained from a public checkpoint and show large absolute penalties (cross-input +1.9067, token-shuffle +2.9637), which are direct, non-circular measurements of functional dependence. The execution claims are supported by direct measurements: FeatureGate executes all 24 blocks and is 30.8% slower than Dense, and MUDDPythia also executes every Transformer block. The paper does not rely on self-citation for load-bearing claims, and no prediction reduces by construction to a fitted parameter or to the definition of its target. The noted limitation that exact replay is verified only on the first batch for MUDDPythia is a validity/robustness concern about an assumption of the intervention, not a circularity: it does not make the outcome equivalent to the input by definition. Overall, the derivation chain is self-contained and the empirical conclusions are independently testable.

Axiom & Free-Parameter Ledger

0 free parameters · 5 axioms · 0 invented entities

No new physical entities; the method introduces no fitted parameters. The assumptions are about the coverage of the cached coefficient tensor, representativeness of calibration, and deterministic replay.

axioms (5)
  • domain assumption The controller's influence on the frozen model is fully captured by the coefficient tensor A(x).
    FCA caches coefficients and disables the controller; any extra controller output (skip decisions, attention masks, etc.) would be ignored. Invoked in §3.2–3.3.
  • domain assumption The independent calibration set (512 WT103 validation windows) is representative of the evaluation set.
    Static profiles are estimated on this set and applied to test windows; a biased estimate would change all static-replacement penalties (§4.1).
  • domain assumption Exact replay of cached coefficients reproduces the unperturbed model output; verification is performed only on the first batch.
    The audit assumes deterministic inference and that logit agreement on the first batch implies agreement on all windows (§4.1).
  • domain assumption NLL differences across replay conditions are the correct measure of functional dependence.
    All claims use ΔNLL; other metrics (accuracy, downstream tasks) could behave differently.
  • domain assumption The MUDDPythia checkpoint used is representative of the MUDDFormer family.
    Only one public checkpoint is audited; the paper acknowledges it does not generalize to all training runs (§6).

pith-pipeline@v1.3.0-alltime-deepseek · 13192 in / 12153 out tokens · 105958 ms · 2026-08-01T00:30:55.265400+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Dynamic Parameterization Is Not Dynamic Inference." pith.science (2026). https://pith.science/paper/6RONDF5A

@misc{pith2026260726192,
  author       = {Pith},
  title        = {Pith review of: Dynamic Parameterization Is Not Dynamic Inference},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6RONDF5A}},
  note         = {Machine review of arXiv:2607.26192}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Input-dependent controller coefficients are often treated as evidence of dynamic inference or computational savings. This interpretation conflates three properties: coefficient variation, dependence of a frozen model on how coefficients are assigned to inputs, and conditional execution. We focus on the second property and formulate a general principle of frozen-controller auditing. We provide one concrete implementation, Frozen-Controller Auditing (FCA), which caches the complete coefficient tensor along an unperturbed trajectory, disables the controller, and replays the frozen model with cross-input reassignment, token shuffling, and static profiles estimated from an independent calibration set. Because the coefficients are cached before any intervention, performance changes under replay measure assignment dependence without feedback from recomputing the controller on perturbed hidden states. Across seven independently trained 76M FeatureGate Transformers and three 504M models, static layerwise profiles retain 98.70% and 99.43% of the Correct-to-GlobalMean performance gap, respectively. Layer identity explains 87% to 96% of the coefficient variance. FeatureGate nevertheless executes every Transformer block, and its measured inference is 30.8% slower than Dense. On the public MUDDPythia-1.4B checkpoint, cross-input reassignment and token shuffling increase NLL by 1.9067 and 2.9637, respectively. These penalties show that the model depends strongly on content-conditioned cross-layer assignment. MUDDPythia also executes every Transformer block. The results show that dynamic parameterization alone does not establish dynamic inference and that functional dynamics do not establish computational savings. Claims about dynamic models should separately report coefficient variation, functional dependence of the frozen model, and actual execution.

Figures

Figures reproduced from arXiv: 2607.26192 by Guozhong Luo, Yuan-yih Shang, Zongfei Li.

Figure 1
Figure 1. Figure 1: Overview of Frozen-Controller Auditing (FCA). FCA distinguishes coefficient variation, functional dependence, and actual execution. It first caches controller coefficients along the unperturbed trajectory, then disables the controller and replays the frozen model with reassigned, shuffled, or static coefficients. The audit finds that a static depth profile closely reproduces the inference behavior of Featu… view at source ↗
Figure 2
Figure 2. Figure 2: Frozen-Controller Audit results for 76M-parameter FeatureGate models on WT103. Each point shows a mean intervention penalty. Horizontal error bars give 95% confidence intervals across seven independently trained checkpoints. As￾signment and static-profile interventions increase NLL by only about 10−3 . GlobalMean removes the learned depth structure and produces a much larger penalty, so we plot it on a sep… view at source ↗
Figure 4
Figure 4. Figure 4: Static profile retention for FeatureGate and MUDDPythia. FeatureGate LayerMean retains 98.70% and 99.43% of the Correct-to-GlobalMean performance gap at 76M and 504M, respectively. Horizontal bars show 95% confidence intervals across independently trained check￾points. For MUDDPythia, StaticMixMean retains 73.33% of the gap relative to SourceUniformMean. This value is a point estimate from a single checkpo… view at source ↗
Figure 5
Figure 5. Figure 5: Absolute intervention penalties relative to Correct [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

59 extracted references · 6 linked inside Pith

  1. [1]

    Gomez and Lukasz Kaiser and Illia Polosukhin , title =

    Ashish Vaswani and Noam Shazeer and Niki Parmar and Jakob Uszkoreit and Llion Jones and Aidan N. Gomez and Lukasz Kaiser and Illia Polosukhin , title =. Advances in Neural Information Processing Systems , year =

  2. [2]

    arXiv preprint arXiv:1603.08983 , year =

    Alex Graves , title =. arXiv preprint arXiv:1603.08983 , year =

    Pith/arXiv arXiv

  3. [3]

    International Conference on Learning Representations , year =

    Mostafa Dehghani and Stephan Gouws and Oriol Vinyals and Jakob Uszkoreit and Lukasz Kaiser , title =. International Conference on Learning Representations , year =

  4. [4]

    International Conference on Learning Representations , year =

    Maha Elbayad and Jiatao Gu and Edouard Grave and Michael Auli , title =. International Conference on Learning Representations , year =

  5. [5]

    Tran and Yi Tay and Donald Metzler , title =

    Tal Schuster and Adam Fisch and Jai Gupta and Mostafa Dehghani and Dara Bahri and Vinh Q. Tran and Yi Tay and Donald Metzler , title =. Advances in Neural Information Processing Systems , year =

  6. [6]

    arXiv preprint arXiv:2404.02258 , year =

    David Raposo and Sam Ritter and Blake Richards and Timothy Lillicrap and Peter Conway Humphreys and Adam Santoro , title =. arXiv preprint arXiv:2404.02258 , year =

    Pith/arXiv arXiv

  7. [7]

    Weinberger , title =

    Gao Huang and Yu Sun and Zhuang Liu and Daniel Sedra and Kilian Q. Weinberger , title =. European Conference on Computer Vision , year =

  8. [8]

    International Conference on Learning Representations , year =

    Angela Fan and Edouard Grave and Armand Joulin , title =. International Conference on Learning Representations , year =

  9. [9]

    Proceedings of the 37th Conference on Uncertainty in Artificial Intelligence , pages =

    Thomas Bachlechner and Bodhisattwa Prasad Majumder and Henry Mao and Gary Cottrell and Julian McAuley , title =. Proceedings of the 37th Conference on Uncertainty in Artificial Intelligence , pages =. 2021 , url =

    2021

  10. [10]

    Going Deeper with Image Transformers , booktitle =

    Hugo Touvron and Matthieu Cord and Alexandre Sablayrolles and Gabriel Synnaeve and Herv. Going Deeper with Image Transformers , booktitle =. 2021 , url =

    2021

  11. [11]

    arXiv preprint arXiv:2203.00555 , year =

    Hongyu Wang and Shuming Ma and Li Dong and Shaohan Huang and Dongdong Zhang and Furu Wei , title =. arXiv preprint arXiv:2203.00555 , year =

    Pith/arXiv arXiv

  12. [12]

    arXiv preprint arXiv:2107.05407 , year =

    Andrea Banino and Jan Balaguer and Charles Blundell , title =. arXiv preprint arXiv:2107.05407 , year =

    Pith/arXiv arXiv

  13. [13]

    International Conference on Learning Representations , year =

    Stephen Merity and Caiming Xiong and James Bradbury and Richard Socher , title =. International Conference on Learning Representations , year =

  14. [14]

    2019 , note =

    Aaron Gokaslan and Vanya Cohen , title =. 2019 , note =

    2019

  15. [15]

    Proceedings of the 42nd International Conference on Machine Learning , series =

    MUDDFormer: Breaking Residual Bottlenecks in Transformers via Multiway Dynamic Dense Connections , author =. Proceedings of the 42nd International Conference on Machine Learning , series =. 2025 , publisher =

    2025

  16. [16]

    Proceedings of the 40th International Conference on Machine Learning , series =

    Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling , author =. Proceedings of the 40th International Conference on Machine Learning , series =. 2023 , publisher =

    2023

  17. [17]

    arXiv preprint arXiv:2101.00027 , year =

    The Pile: An 800GB Dataset of Diverse Text for Language Modeling , author =. arXiv preprint arXiv:2101.00027 , year =

    Pith/arXiv arXiv

  18. [18]

    IEEE Transactions on Pattern Analysis and Machine Intelligence , volume =

    Yizeng Han and Gao Huang and Shiji Song and Le Yang and Honghui Wang and Yulin Wang , title =. IEEE Transactions on Pattern Analysis and Machine Intelligence , volume =. 2022 , doi =

    2022

  19. [19]

    Advances in Neural Information Processing Systems , year =

    Xu Jia and Bert De Brabandere and Tinne Tuytelaars and Luc Van Gool , title =. Advances in Neural Information Processing Systems , year =

  20. [20]

    Dai and Quoc V

    David Ha and Andrew M. Dai and Quoc V. Le , title =. International Conference on Learning Representations , year =

  21. [21]

    Le and Jiquan Ngiam , title =

    Brandon Yang and Gabriel Bender and Quoc V. Le and Jiquan Ngiam , title =. Advances in Neural Information Processing Systems , year =

  22. [22]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , year =

    Yinpeng Chen and Xiyang Dai and Mengchen Liu and Dongdong Chen and Lu Yuan and Zicheng Liu , title =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , year =

  23. [23]

    Surat Teerapittayanon and Bradley McDanel and H. T. Kung , title =. 23rd International Conference on Pattern Recognition , pages =. 2016 , doi =

    2016

  24. [24]

    Weinberger , title =

    Gao Huang and Danlu Chen and Tianhong Li and Felix Wu and Laurens van der Maaten and Kilian Q. Weinberger , title =. International Conference on Learning Representations , year =

  25. [25]

    Proceedings of the 36th International Conference on Machine Learning , pages =

    Yigitcan Kaya and Sanghyun Hong and Tudor Dumitras , title =. Proceedings of the 36th International Conference on Machine Learning , pages =. 2019 , url =

    2019

  26. [26]

    Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , pages =

    Ji Xin and Raphael Tang and Jaejun Lee and Yaoliang Yu and Jimmy Lin , title =. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , pages =. 2020 , doi =

    2020

  27. [27]

    Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , pages =

    Weijie Liu and Peng Zhou and Zhiruo Wang and Zhe Zhao and Haotang Deng and Qi Ju , title =. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , pages =. 2020 , doi =

    2020

  28. [28]

    Advances in Neural Information Processing Systems , year =

    Wangchunshu Zhou and Canwen Xu and Tao Ge and Julian McAuley and Ke Xu and Furu Wei , title =. Advances in Neural Information Processing Systems , year =

  29. [29]

    Advances in Neural Information Processing Systems , year =

    Lu Hou and Zhiqi Huang and Lifeng Shang and Xin Jiang and Xiao Chen and Qun Liu , title =. Advances in Neural Information Processing Systems , year =

  30. [30]

    Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics , pages =

    Wei Zhu , title =. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics , pages =. 2021 , doi =

    2021

  31. [31]

    Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics , pages =

    Ji Xin and Raphael Tang and Yaoliang Yu and Jimmy Lin , title =. Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics , pages =. 2021 , doi =

    2021

  32. [32]

    Gonzalez , title =

    Xin Wang and Fisher Yu and Zi-Yi Dou and Trevor Darrell and Joseph E. Gonzalez , title =. Proceedings of the European Conference on Computer Vision , pages =. 2018 , url =

    2018

  33. [33]

    Davis and Kristen Grauman and Rogerio Feris , title =

    Zuxuan Wu and Tushar Nagarajan and Abhishek Kumar and Steven Rennie and Larry S. Davis and Kristen Grauman and Rogerio Feris , title =. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages =. 2018 , url =

    2018

  34. [34]

    Le and Geoffrey E

    Noam Shazeer and Azalia Mirhoseini and Krzysztof Maziarz and Andy Davis and Quoc V. Le and Geoffrey E. Hinton and Jeff Dean , title =. International Conference on Learning Representations , year =

  35. [35]

    International Conference on Learning Representations , year =

    Dmitry Lepikhin and HyoukJoong Lee and Yuanzhong Xu and Dehao Chen and Orhan Firat and Yanping Huang and Maxim Krikun and Noam Shazeer and Zhifeng Chen , title =. International Conference on Learning Representations , year =

  36. [36]

    Journal of Machine Learning Research , volume =

    William Fedus and Barret Zoph and Noam Shazeer , title =. Journal of Machine Learning Research , volume =. 2022 , url =

    2022

  37. [37]

    Proceedings of the 38th International Conference on Machine Learning , pages =

    Mike Lewis and Shruti Bhosale and Tim Dettmers and Naman Goyal and Luke Zettlemoyer , title =. Proceedings of the 38th International Conference on Machine Learning , pages =. 2021 , url =

    2021

  38. [38]

    Advances in Neural Information Processing Systems , year =

    Carlos Riquelme and Joan Puigcerver and Basil Mustafa and Maxim Neumann and Rodolphe Jenatton and Andre Susano Pinto and Daniel Keysers and Neil Houlsby , title =. Advances in Neural Information Processing Systems , year =

  39. [39]

    Dai and Simon Tong and Dmitry Lepikhin and Yuanzhong Xu and Maxim Krikun and Yanqi Zhou and Adams Wei Yu and Orhan Firat and Barret Zoph and Liam Fedus and Maarten P

    Nan Du and Yanping Huang and Andrew M. Dai and Simon Tong and Dmitry Lepikhin and Yuanzhong Xu and Maxim Krikun and Yanqi Zhou and Adams Wei Yu and Orhan Firat and Barret Zoph and Liam Fedus and Maarten P. Bosma and Zongwei Zhou and Tao Wang and Emma Wang and Kellie Webster and Marie Pellat and Kevin Robinson and Kathleen Meier-Hellstern and Toju Duke and...

    2022

  40. [40]

    Zhao and Andrew M

    Yanqi Zhou and Tao Lei and Hanxiao Liu and Nan Du and Yanping Huang and Vincent Y. Zhao and Andrew M. Dai and Zhifeng Chen and Quoc V. Le and James Laudon , title =. Advances in Neural Information Processing Systems , year =

  41. [41]

    Advances in Neural Information Processing Systems , year =

    Yongming Rao and Wenliang Zhao and Benlin Liu and Jiwen Lu and Jie Zhou and Cho-Jui Hsieh , title =. Advances in Neural Information Processing Systems , year =

  42. [42]

    International Conference on Learning Representations , year =

    Youwei Liang and Chongjian Ge and Zhan Tong and Yibing Song and Jue Wang and Pengtao Xie , title =. International Conference on Learning Representations , year =

  43. [43]

    Advances in Neural Information Processing Systems , year =

    Yulin Wang and Rui Huang and Shiji Song and Zeyi Huang and Gao Huang , title =. Advances in Neural Information Processing Systems , year =

  44. [44]

    Alvarez and Arun Mallya and Jan Kautz and Pavlo Molchanov , title =

    Hongxu Yin and Arash Vahdat and Jose M. Alvarez and Arun Mallya and Jan Kautz and Pavlo Molchanov , title =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages =. 2022 , url =

    2022

  45. [45]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages =

    Lingchen Meng and Hengduo Li and Bor-Chun Chen and Shiyi Lan and Zuxuan Wu and Yu-Gang Jiang and Ser-Nam Lim , title =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages =. 2022 , url =

    2022

  46. [46]

    European Conference on Computer Vision , year =

    Mohsen Fayyaz and Soroush Abbasi Koohpayegani and Farnoush Rezaei Jafari and Sunando Sengupta and Hamid Reza Vaezi Joze and Eric Sommerlade and Hamed Pirsiavash and Jurgen Gall , title =. European Conference on Computer Vision , year =

  47. [47]

    International Conference on Learning Representations , year =

    Daniel Bolya and Cheng-Yang Fu and Xiaoliang Dai and Peizhao Zhang and Christoph Feichtenhofer and Judy Hoffman , title =. International Conference on Learning Representations , year =

  48. [48]

    Dauphin and Tengyu Ma , title =

    Hongyi Zhang and Yann N. Dauphin and Tengyu Ma , title =. International Conference on Learning Representations , year =

  49. [49]

    Advances in Neural Information Processing Systems , year =

    Soham De and Sam Smith , title =. Advances in Neural Information Processing Systems , year =

  50. [50]

    Weinberger , title =

    Gao Huang and Zhuang Liu and Laurens van der Maaten and Kilian Q. Weinberger , title =. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages =. 2017 , url =

    2017

  51. [51]

    Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing , pages =

    Ankur Bapna and Mia Chen and Orhan Firat and Yuan Cao and Yonghui Wu , title =. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing , pages =. 2018 , doi =

    2018

  52. [52]

    Wong and Lidia S

    Qiang Wang and Bei Li and Tong Xiao and Jingbo Zhu and Changliang Li and Derek F. Wong and Lidia S. Chao , title =. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , pages =. 2019 , doi =

    2019

  53. [53]

    Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021 , pages =

    Ruining He and Anirudh Ravula and Bhargav Kanagal and Joshua Ainslie , title =. Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021 , pages =. 2021 , doi =

    2021

  54. [54]

    Advances in Neural Information Processing Systems , year =

    Matteo Pagliardini and Amirkeivan Mohtashami and Francois Fleuret and Martin Jaggi , title =. Advances in Neural Information Processing Systems , year =

  55. [55]

    International Conference on Learning Representations , year =

    Defa Zhu and Hongzhi Huang and Zihao Huang and Yutao Zeng and Yunyao Mao and Banggu Wu and Qiyang Min and Xun Zhou , title =. International Conference on Learning Representations , year =

  56. [56]

    Advances in Neural Information Processing Systems , year =

    Paul Michel and Omer Levy and Graham Neubig , title =. Advances in Neural Information Processing Systems , year =

  57. [57]

    arXiv preprint arXiv:2004.12265 , year =

    Jesse Vig and Sebastian Gehrmann and Yonatan Belinkov and Sharon Qian and Daniel Nevo and Simas Sakenis and Jason Huang and Yaron Singer and Stuart Shieber , title =. arXiv preprint arXiv:2004.12265 , year =

    Pith/arXiv arXiv 2004

  58. [58]

    Proceedings of the 39th International Conference on Machine Learning , pages =

    Atticus Geiger and Zhengxuan Wu and Hanson Lu and Josh Rozner and Elisa Kreiss and Thomas Icard and Noah Goodman and Christopher Potts , title =. Proceedings of the 39th International Conference on Machine Learning , pages =. 2022 , url =

    2022

  59. [59]

    Advances in Neural Information Processing Systems , year =

    Kevin Meng and David Bau and Alex Andonian and Yonatan Belinkov , title =. Advances in Neural Information Processing Systems , year =