Pith. sign in

REVIEW 4 major objections 5 minor 22 references

Learning to Access Computation: Accessibility Plasticity as a Principle of Adaptive Intelligence

T0 review · 4 major / 5 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper proposes Accessibility Plasticity, claiming adaptive systems can reduce costly weight modification by first reorganizing which existing computations are accessible, with proof-of-concept tests showing comparable accuracy at roughl

desk verdict A clear, honest conceptual paper on accessibility as a distinct adaptive dimension; the proof-of-concept's staged controller encodes the reuse-first order, so the headline reduction in capability cost is partly by construction. read the letter →

arxiv 2607.22748 v1 pith:AX56F3LF submitted 2026-07-23 cs.LG cs.AIcs.NE

Zhaowen Fan This is my paper · ORCID
classification cs.LGcs.AIcs.NE
keywords AccessibilityPlasticityadaptivecomputationcontinuallearningneuralconditionalreuse-firsthierarchycapabilityvsdynamicsystems
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that computational systems have two separate adaptive objects: what computation they can perform (capability) and which existing computations are reachable in the current context (accessibility). It argues that adaptation should follow a reuse-first hierarchy: exploit existing computation, then reshape accessibility, then modify parameters, and only lastly change the computational organization itself. If this is right, continual learning can shift adaptation burden from expensive weight rewriting to cheaper reorganization of access, reducing modification cost and forgetting. Proof-of-concept experiments on sequential MNIST tasks show the approach cuts shared-capability modification cost by roughly 58–64% while losing only a few points of accuracy compared with direct weight-updating baselines. The paper presents this as a principle and proof-of-concept, not as a state-of-the-art benchmark.

What carries the argument

The adaptive computational substrate: a graph whose nodes are computational operators and whose edges carry learned relationship states (a compatibility function and a resistance value). The relationship state induces accessibility a_ij, which gates whether an edge participates in the current computation. The carrying mechanism is the staged reuse-first controller that tries reuse, then updates only accessibility, then updates capability, while costs C_A and C_W are tracked separately, giving the quantitative evidence that accessibility is doing real work.

What would settle it

Run the same substrate with the three adaptation stages applied in shuffled or capability-first order across tasks; if the accumulated capability-modification cost stays as low as in the reuse-first protocol, then the reduction is not caused by prioritizing accessibility but by the staging itself.

Watch

Extended reading notes

Core claim

The central discovery is that accessibility can be treated as a first-class adaptive object, distinct from capability. In the proposed formalization, a substrate is a graph G=(V,E) where each node is a computational operator W_i and each edge carries a relationship state R_ij=(F_ij,d_ij). Accessibility a_ij = g(F_ij(x_i,x_j) - beta*d_ij) determines whether that relationship participates in computation, inducing a dynamically accessible graph G_t=(V,E*_t). The paper argues that adapting accessibility before updating W yields a shallower, cheaper form of plasticity. Staged sequential-learning experiments demonstrate that this reuse-first ordering reduces accumulated shared-capability movement

Load-bearing premise

The experimental case rests on the assumption that the fixed reuse-first staging is a genuine test of the principle, not a protocol that mechanically forces lower weight-update counts regardless of whether accessibility is truly doing the work.

Editorial extensions

If this is right

  • Continual-learning systems can reduce the amount of weight rewriting needed to acquire a sequence of tasks by first reshaping which existing computations are reachable.
  • Accessibility is a distinct adaptive variable: removing it or randomizing it yields materially different accuracy-cost trade-offs, so the observed effect is not simply extra adaptable parameters.
  • Accessibility plasticity alone is insufficient; capability updates remain necessary on harder tasks, supporting the depth hierarchy rather than a claim that rewriting is obsolete.
  • The deepest level of the hierarchy, reorganizing the substrate itself (Δ(D,G)), is not validated by these experiments and remains open for future work.
  • The dynamic-stream result suggests accessibility can evolve continuously in an online setting, with capability changes concentrated at the hardest regime shift.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the principle generalizes, parameter-efficient techniques such as adapters, prefixes, routing, and sparsely-gated experts could be reinterpreted as different realizations of accessibility plasticity, giving a common vocabulary for comparing them.
  • A direct testable extension would apply the staged reuse-first protocol to a pre-trained transformer on a sequence of language tasks, measuring accumulated parameter update norm while freezing the trunk; if the roughly 60%-plus reduction appears there, the principle holds beyond toy vision benchmarks.
  • The staged controller's fixed order may itself be the source of the cost reduction; testing random or simultaneous stage ordering would separate the principle from the scheduling artifact.
  • Reuse-first adaptation suggests a neuro-inspired timescale separation — fast access changes and slow weight changes — which could inform continual learning in embodied agents and online robotics.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes separating computational capability (what computations exist, W) from computational accessibility (which existing computations can participate, A), treating accessibility as a first-class adaptive object. It introduces a reuse-first hierarchy R ≺ ΔA ≺ ΔW ≺ Δ(D,G), and tests one operational realization on Split/Permuted MNIST using a staged controller (Eq. 20) that tries reuse, then accessibility, then capability. The reported experiments show lower capability-modification cost C_W but also 4–5 point lower accuracy than the baselines; the paper interprets this as proof-of-concept that accessibility adaptation can precede and reduce capability modification.

Significance. If the conceptual separation and the reuse-first hierarchy were well supported, the paper would offer a useful unifying perspective on adapters, routing, mixture-of-experts, and continual learning. The manuscript is transparent about its proof-of-concept scope, includes a Random Accessibility Control, and explicitly discusses limitations (single seed, no benchmark campaign). However, the central empirical test is confounded by construction: the staged controller hard-codes the reuse-first order, so the reported reduction in C_W is partly a design artifact. The conceptual claim is also insulated from falsification by the 'protected theoretical statement' in §3.4. The idea is promising, but the evidence as presented does not yet establish that accessibility is a distinct adaptive variable with the claimed precedence property.

major comments (4)
  1. [§4.1, Eq. (20), Algorithm 1] The reuse-first preference is inserted into the controller by design: s* = min{s: Acc(s) ≥ τ} over {reuse, ΔA, ΔW}, and Algorithm 1 freezes W whenever an earlier stage satisfies the threshold. Under this rule, lower C_W is a direct consequence of the decision order, not an emergent property of accessibility as a distinct adaptive variable. The Random Accessibility Control changes the quality of the accessibility update but preserves the same hard-coded order, so it does not rule out 'just ordering.' A test of the core claim needs an unconstrained optimization using Eq. (18) with λ_A ≤ λ_W (no staged freezing), or a controller that chooses stages by expected cost; otherwise the main empirical conclusion is by construction.
  2. [Tables 1–2; §5.2] The abstract and §5.2 describe the accuracy loss as 'comparable' or as a trade-off, but the AP realization achieves 0.9183 vs 0.9583/0.9658 on Split MNIST and 0.9242 vs 0.9700 on Permuted MNIST — a 4–5 point drop. With a single seed (acknowledged in §5.4) there are no confidence intervals, so the drop could be larger or smaller, and 'maintaining comparable task performance' is not a measured claim. Please provide multiple seeds, confidence intervals, or a pre-specified equivalence threshold; otherwise this part of the central claim is unsupported.
  3. [Eq. (19), Tables 1–2] C_W is an accumulated Frobenius norm of parameter movement, but it is computed over different architectures: the baseline and EWC use a 64-unit MLP trunk, while the AP realization uses an 8-expert bank plus shared post layer. Absolute values of C_W are therefore not commensurable across methods, and the reported 'reduction' from 8.4 to 2.9 may reflect a smaller parameter budget or different architecture rather than the adaptive mechanism. The paper needs matched-capacity models or normalized per-parameter cost to support the reduction claim.
  4. [§3.4, Eq. (14)] The manuscript explicitly designates A^{t+1}=P_A(A^t,X^t) as the 'protected theoretical statement' and treats any concrete update as only a possible realization. As written, this makes the principle unfalsifiable: no experiment can disconfirm the existence of a distinct accessibility state because every failure can be attributed to the realization. The proof-of-concept can at most validate one update rule, not the general claim that accessibility is distinct from capability. A falsifiable criterion for distinctness is needed, such as demonstrating that A-adaptation remains preferred under free choice or comparing against an equivalent gating/selection mechanism.
minor comments (5)
  1. [Eq. (15)] The update d^{t+1}_{ij} = d^t_{ij} − η d s^t_{ij} has unclear notation: is η d a learning rate, or is it η multiplied by d? Please clarify.
  2. [Algorithm 1 vs §4.1] Algorithm 1 uses task-loss thresholds ε_r, ε_A, ε_W, while Eq. (20) uses an accuracy threshold τ. The relationship between these thresholds is not specified; this matters for reproducing the staged behavior.
  3. [§1] The sentence 'Biological intelligence suggests that such a separation may be fundamental. [7]' has a citation placed after the period; reposition the citation.
  4. [Figures 3–5] The figure captions are present but the actual figures appear to be missing from the manuscript text; please ensure embedded images are included in the submitted version.
  5. [Eq. (1)] The objective max P(T)·R(S)/C_adapt is stated but R(S) and C_adapt are not subsequently formalized in a way that connects to the experimental metrics; define these terms or remove the equation.

Circularity Check

2 steps flagged · score 6.0 of 10

Staged controller hard-codes the reuse-first order (Eq. 20), so the reported trace and reduced capability cost are partly by construction.

  1. self definitional [§4.1 Eq. (20) and Algorithm 1]
    "The experiments implement the hierarchy using a performance-based controller: at each task, the system attempts the shallowest adaptation depth and avoids deeper modification if the current depth satisfies a fixed training criterion. Operationally, s∗ = min s {s: Acc(s) ≥ τ}, s ∈ {reuse,ΔA,ΔW}."

    The reuse-first hierarchy is encoded directly in the controller's fixed order: reuse is tried first, then ΔA, then ΔW, and Algorithm 1 returns without modifying W whenever an earlier stage satisfies the threshold. The later observation that the system 'follows the trace reuse→accessibility→capability on every subsequent task' is therefore guaranteed by the construction, not discovered. Similarly, the reduced C_W relative to baselines that always update W is partly an artifact of the forced ordering: W is simply not updated when the accessibility stage alone reaches τ. The Random Accessibility Control rules out 'extra parameters' but does not rule out 'just ordering.'

  2. other [§3.4 Eq. (14)]
    "The abstract update A^{t+1}=P_A(A^t,X^t) is therefore the protected theoretical statement; any concrete distance-based update or relationship-state update is only one possible realization, not the definition of Accessibility Plasticity."

    By declaring the abstract update 'protected,' the paper makes the general principle unfalsifiable by any single realization: a successful relationship update counts as evidence, while a failed one is only one possible realization. This definitional insulation, combined with the staged controller, means the empirical support for accessibility as a distinct adaptive variable is not independently testable from the abstract formulation.

full rationale

The conceptual claim that accessibility is a distinct adaptive object is not inherently circular, and the paper is transparent that the experiments are proof-of-concept rather than benchmarks. However, the empirical support for the specific claim that accessibility adaptation can precede and reduce capability modification is substantially constructed. Eq. (20) and Algorithm 1 force the system to attempt reuse first, then accessibility, then capability, freezing W whenever an earlier stage meets the threshold. Thus the observed 'trace reuse→accessibility→capability' and the lower C_W versus always-updating baselines are partly by design: the protocol encodes the very hierarchy it is used to support. The Random Accessibility Control rules out 'extra adaptable state' but not 'forced ordering.' The 'protected theoretical statement' in §3.4 further shields the abstract principle from empirical disconfirmation. These issues amount to partial by-construction circularity of the central empirical demonstration, although the broader conceptual framing and the random/ablation controls retain some independent content.

Assumptions & free parameters 5 free parameters · 5 assumptions · 2 invented entities

The framework rests on a small set of hand-chosen thresholds and cost definitions; the hierarchy is asserted rather than derived, and the general principle is protected by Eq. 14's high-level form. The proof-of-concept adds no new entities beyond the relationship state and resistance.

free parameters (5)
  • performance threshold τ = 0.9 (task-sequential), 0.92 (datastream)
    Controls when deeper adaptation is invoked in Eq. 20; directly shapes the measured C_W/C_A trade-off.
  • regularization weights λ_A, λ_W = unspecified (constrained λ_A ≤ λ_W)
    Eq. 18 assigns costs to accessibility vs capability change; values not reported in the manuscript, despite controlling the trade-off.
  • resistance coefficient β = unspecified
    Eq. 7 modulates the influence of computational resistance d_ij; value not reported.
  • expert bank size = 8 experts, hidden dim 64
    Hand-chosen architecture for the shared capability bank; affects C_W magnitudes and the baseline comparison.
  • training hyperparameters (learning rate, optimizer, steps, seed) = not reported
    Required to reproduce Tables 1–2 and Figure 6; absent from the manuscript.
assumptions (5)
  • ad hoc to paper Reuse-first hierarchy: systems should exhaust cheaper, more reversible modifications (reuse, then accessibility, then capability, then substrate) before more expensive ones
    Asserted as a principle in §2.2 and §3.4, motivated by biological analogy, not derived from the formal model.
  • ad hoc to paper The abstract update A^{t+1}=P_A(A^t,X^t) is the 'protected theoretical statement' and any specific update is a realization
    §3.4. This makes the general principle non-falsifiable by design, insulating it from counterexamples.
  • domain assumption Biological neural reuse and multi-timescale plasticity license the ordering of adaptive depth
    §1, citing [1,7,9,14,19]; used to motivate the hierarchy.
  • ad hoc to paper C_A and C_W as defined (Eq. 19) are meaningful, commensurate proxies for adaptation cost
    §4.1. The main quantitative claim (reduced capability cost) is stated in terms of these self-defined metrics, which aggregate L1/L2 norms across models with different parameter counts.
  • domain assumption Nodes are linear transition operators without nonlinearity or persistent state (Eqs. 3–4)
    §3.1. A modeling simplification for the proof-of-concept, not a general claim.
invented entities (2)
  • Accessibility Plasticity (accessibility state A)
    purpose: A distinct adaptive degree of freedom separating 'what computation exists' (W) from 'what computation is accessible' (A)
    The central postulate; only in-paper proof-of-concept supports it, and the paper defines it so broadly that attention/routing/MoE count as realizations.
  • Relationship state R_ij=(F_ij,d_ij) with computational resistance d_ij
    purpose: Concrete realization: learned compatibility function and resistance that modulate accessibility between computational units
    New model component introduced in §3.1–3.2; no external falsifiable prediction beyond the reported experiments.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Learning to Access Computation: Accessibility Plasticity as a Principle of Adaptive Intelligence." pith.science (2026). https://pith.science/paper/AX56F3LF

@misc{pith2026260722748,
  author       = {Pith},
  title        = {Pith review of: Learning to Access Computation: Accessibility Plasticity as a Principle of Adaptive Intelligence},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AX56F3LF}},
  note         = {Machine review of arXiv:2607.22748}
}
read the original abstract

Modern neural networks primarily adapt through parameter modification within predefined computational structures. While recent methods introduce modularity, conditional computation, and parameter-efficient adaptation, they generally do not distinguish computational capability from computational accessibility as separate adaptive variables. This work introduces Accessibility Plasticity, a principle of adaptive computation in which systems adapt not only by changing what computation exists, but also by reorganizing which existing computations can interact and participate. We formalize Accessibility Plasticity through a relationship-based operational realization and establish a reuse-first hierarchy of adaptation, where accessibility modification precedes more costly capability and structural changes. A proof-of-concept evaluation on sequential learning tasks shows that accessibility adaptation can reduce capability modification while maintaining comparable task performance. These results suggest accessibility as a distinct adaptive dimension and provide a foundation for future dynamic neural systems whose computational relationships evolve with changing environments.

Figures

Figures reproduced from arXiv: 2607.22748 by the authors.

Figure 1
Figure 1. Adaptive systems possess multiple degrees of adaptive freedom. The proposed [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. Operational realization of the adaptive transformation hierarchy. The system [PITH_FULL_IMAGE:figures/full_fig_p012_2.png] view at source ↗
Figure 3
Figure 3. Capability modification cost across methods on Split MNIST and Permuted [PITH_FULL_IMAGE:figures/full_fig_p015_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Final average accuracy on Split MNIST and Permuted MNIST. The purpose of [PITH_FULL_IMAGE:figures/full_fig_p016_4.png]
Figure 5
Figure 5. Figure 5: Accuracy-capability trade-off space for all methods on both datasets. Colors iden [PITH_FULL_IMAGE:figures/full_fig_p017_5.png]
Figure 6
Figure 6. Figure 6: Online adaptation in a gradually shifting datastream. A modest initial state is [PITH_FULL_IMAGE:figures/full_fig_p022_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

22 extracted references · 7 linked inside Pith

  1. [1]

    Neural reuse: A fundamental organizational principle of the brain

    Michael L Anderson. Neural reuse: A fundamental organizational principle of the brain. Behavioral and brain sciences, 33(4):245–266, 2010

    2010

  2. [2]

    Relational inductive biases, deep learning, and graph networks.arXiv preprint arXiv:1806.01261, 2018

    Peter W Battaglia, Jessica B Hamrick, Victor Bapst, Alvaro Sanchez-Gonzalez, Vinicius Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, et al. Relational inductive biases, deep learning, and graph networks.arXiv preprint arXiv:1806.01261, 2018

    arXiv 2018

  3. [3]

    Estimating or propagat- ing gradients through stochastic neurons for conditional computation.arXiv preprint arXiv:1308.3432, 2013

    Yoshua Bengio, Nicholas L´ eonard, and Aaron Courville. Estimating or propagat- ing gradients through stochastic neurons for conditional computation.arXiv preprint arXiv:1308.3432, 2013

    arXiv 2013

  4. [4]

    On the opportunities and risks of foundation models.arXiv preprint arXiv:2108.07258, 2021

    Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. On the opportunities and risks of foundation models.arXiv preprint arXiv:2108.07258, 2021

    arXiv 2021

  5. [5]

    Language models are few-shot learners.Advances in neural information processing sys- tems, 33:1877–1901, 2020

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners.Advances in neural information processing sys- tems, 33:1877–1901, 2020. 19

    1901

  6. [6]

    Parameter-efficient fine-tuning of large-scale pre-trained language models.Nature machine intelligence, 5(3):220–235, 2023

    Ning Ding, Yujia Qin, Guang Yang, Fuchao Wei, Zonghan Yang, Yusheng Su, Shengding Hu, Yulin Chen, Chi-Min Chan, Weize Chen, et al. Parameter-efficient fine-tuning of large-scale pre-trained language models.Nature machine intelligence, 5(3):220–235, 2023

    2023

  7. [7]

    Neuroscience-inspired artificial intelligence.Neuron, 95(2):245–258, 2017

    Demis Hassabis, Dharshan Kumaran, Christopher Summerfield, and Matthew Botvinick. Neuroscience-inspired artificial intelligence.Neuron, 95(2):245–258, 2017

    2017

  8. [8]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016

    2016

Show all 22 references
  1. [9]

    Experience-dependent structural synaptic plas- ticity in the mammalian brain.Nature Reviews Neuroscience, 10(9):647–658, 2009

    Anthony Holtmaat and Karel Svoboda. Experience-dependent structural synaptic plas- ticity in the mammalian brain.Nature Reviews Neuroscience, 10(9):647–658, 2009

    2009

  2. [10]

    Parameter- efficient transfer learning for nlp

    Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter- efficient transfer learning for nlp. InInternational conference on machine learning, pages 2790–2799. PMLR, 2019

    2019

  3. [11]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Liang Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. Iclr, 1(2):3, 2022

    2022

  4. [12]

    Semi-supervised classification with graph convolu- tional networks.arXiv preprint arXiv:1609.02907, 2016

    Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolu- tional networks.arXiv preprint arXiv:1609.02907, 2016

    2016 arXiv

  5. [13]

    Deep learning.nature, 521(7553):436–444, 2015

    Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning.nature, 521(7553):436–444, 2015

    2015

  6. [14]

    Synaptic plasticity forms and functions

    Jeffrey C Magee and Christine Grienberger. Synaptic plasticity forms and functions. Annual review of neuroscience, 43(1):95–117, 2020

    2020

  7. [15]

    Cellular, synaptic and network effects of neuro- modulation.Neural Networks, 15(4-6):479–493, 2002

    Eve Marder and Vatsala Thirumalai. Cellular, synaptic and network effects of neuro- modulation.Neural Networks, 15(4-6):479–493, 2002

    2002

  8. [16]

    Human-level control through deep reinforcement learning.nature, 518(7540):529–533, 2015

    Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al. Human-level control through deep reinforcement learning.nature, 518(7540):529–533, 2015

    2015

  9. [17]

    Continual lifelong learning with neural networks: A review.Neural networks, 113:54–71, 2019

    German I Parisi, Ronald Kemker, Jose L Part, Christopher Kanan, and Stefan Wermter. Continual lifelong learning with neural networks: A review.Neural networks, 113:54–71, 2019

    2019

  10. [18]

    Modular deep learning.arXiv preprint arXiv:2302.11529, 2023

    Jonas Pfeiffer, Sebastian Ruder, Ivan Vuli´ c, and Edoardo Maria Ponti. Modular deep learning.arXiv preprint arXiv:2302.11529, 2023. 20

    2023 arXiv

  11. [19]

    A deep learning framework for neuroscience.Nature neuroscience, 22(11):1761–1770, 2019

    Blake A Richards, Timothy P Lillicrap, Philippe Beaudoin, Yoshua Bengio, Rafal Bo- gacz, Amelia Christensen, Claudia Clopath, Rui Ponte Costa, Archy de Berker, Surya Ganguli, et al. A deep learning framework for neuroscience.Nature neuroscience, 22(11):1761–1770, 2019

    2019

  12. [20]

    Routing networks: Adaptive selection of non-linear functions for multi-task learning.arXiv preprint arXiv:1711.01239, 2017

    Clemens Rosenbaum, Tim Klinger, and Matthew Riemer. Routing networks: Adaptive selection of non-linear functions for multi-task learning.arXiv preprint arXiv:1711.01239, 2017

    2017 arXiv

  13. [21]

    Outrageously large neural networks: The sparsely-gated mixture- of-experts layer.arXiv preprint arXiv:1701.06538, 2017

    Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. Outrageously large neural networks: The sparsely-gated mixture- of-experts layer.arXiv preprint arXiv:1701.06538, 2017

    2017 arXiv

  14. [22]

    Attention is all you need.Advances in neural information processing systems, 30, 2017

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need.Advances in neural information processing systems, 30, 2017. 21 Figure 6: Online adaptation in a gradually shifting datastream...

    2017

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.