REVIEW 4 major objections 4 minor 30 references
Hierarchically Gated Experts for Efficient Online Continual Learning
T0 review · 4 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read This paper claims that online continual learning can detect task switches from training losses alone, and that organizing the resulting experts into a hierarchical tree cuts expert queries per sample while keeping accuracy close to flat…
desk verdict GE is a solid incremental step, but HGE's headline accuracy-efficiency trade-off is not supported by the paper's own experiments. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the per-expert variational autoencoder, whose reconstruction loss doubles as a suitability score for routing samples to experts. Around it sits the task-switch detector, an EWMA of loss mean and deviation plus a Z-test against a small replay buffer that decides whether a sustained loss spike is a new task or training instability. For HGE, the machinery is the routing tree and the promotion procedure: a new expert is inserted as a child of the lowest common ancestor of the traversal paths its training samples took, and backward connections are added for any masked descendants, so future samples can be routed by descending one path instead of scoring every expert.
What would settle it
Run GE on Permuted MNIST, freeze the expert tree at the moment the first expert is promoted, and measure gating accuracy on held-out samples from each task at several later checkpoints; if gating accuracy falls toward chance as the remaining experts keep training, the stability premise behind HGE is false. Alternatively, retrain only the leaves of an already-built HGE tree and show that samples that were correctly routed before the retraining migrate to a different expert afterward.
Extended reading notes
Core claim
GE detects a task switch by maintaining an exponentially weighted moving average of each expert's training loss and deviation, and treating any loss above mean plus epsilon standard deviations as a candidate switch. Since a single high loss can be an outlier, the sample goes into a high-loss buffer; only when the buffer fills and a Z-test shows the buffered losses differ strongly from the expert's replay-buffer losses is a new expert created. HGE then organizes the growing set of experts into a tree: each promoted expert is inserted at the lowest common ancestor of the paths its training samples took through the current tree, with backward connections added for any sibling-descendant experts whose replay samples would otherwise be captured by the newcomer. The paper reports that in controlled tests HGE's gating accuracy stays close to a flat-tree upper bound while querying far fewer experts per sample, and that in full benchmarks GE matches or exceeds state-of-the-art online continual learners on most scenarios, with HGE trading some accuracy for efficiency. The central claim is that task boundaries are readable from training-loss statistics alone and that expert relationships visible through autoencoding losses are stable enough to support hierarchical routing.
Load-bearing premise
The whole approach assumes that the relative ordering of autoencoding losses among experts remains reasonably stable after a new expert is promoted, so a routing tree built once keeps sending samples to the right expert as training continues.
Editorial extensions
If this is right
- GE's Z-test review step yields zero task-switch false positives on all GE scenarios in Table 1, including ResNet-based CIFAR and ImageNet splits where the TAME baseline produced runaway expert creation.
- In controlled comparisons, HGE matches the flat-tree gating accuracy within about one percentage point on most scenarios while querying fewer experts per sample.
- The paper argues that the LCA-insertion and backward-connection scheme could be applied to expert-based continual learning methods beyond GE.
- Because HGE's training-time insertion cost scales with the number of sibling descendants, the paper identifies more efficient tree-building methods as an open problem.
Reading between the lines
- Editorial inference: the hierarchical routing idea transfers to any mixture-of-experts system whose experts can be scored by reconstruction loss, so the same tree could shrink inference cost in large sparse models.
- Editorial inference: periodically rebuilding or rebalancing the tree after experts have shifted might recover most of the accuracy HGE currently loses, since the paper traces the loss to changing expert relationships rather than to the tree concept itself.
- Editorial inference: the extremely high z-scores observed at real task switches (over 20,000) suggest the normality assumption is false but the signal is strong; a simpler margin or rank-based rule might work as well and be easier to set online.
- Editorial inference: making promotion thresholds self-tuning from stream statistics would close the gap between HGE and the stated online paradigm, since the current thresholds are chosen per scenario from end-of-training accuracy.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes Gated Experts (GE), an online continual learning method that detects task switches by tracking a smoothed training loss and confirming candidate switches with a Z-test on autoencoder reconstruction losses, and Hierarchically Gated Experts (HGE), an extension that organizes experts into a tree to reduce the number of experts queried per sample. GE is evaluated on standard continual learning benchmarks and hybrid scenarios, and HGE is evaluated in controlled experiments and the same online benchmarks. The central claims are that GE matches or exceeds prior online continual learning methods and that HGE achieves comparable accuracy to GE with greater efficiency.
Significance. If the GE task-switch detection mechanism holds up, it is a useful contribution to online continual learning: the Z-test confirmation reduces false-positive expert creation relative to TAME, and the paper provides a thoughtful ablation (GE without the review step) and controlled experiments on tree organization. The hierarchical organization idea is novel and the paper is honest in acknowledging its limitations. However, the HGE efficiency-accuracy claim is not currently supported by the evidence: the promotion threshold is tuned with task-specific oracle information, and the online results show large accuracy drops that the authors themselves attribute to changing expert relationships. The GE results are promising, but the paper's headline contribution, HGE, needs substantially more work before the efficiency claim can be credited.
major comments (4)
- [Section 4, Expert Promotion in HGE] The per-scenario promotion threshold εpromotion is set, in the authors' words, "roughly equal to the accuracy achieved at the end of training," which they admit "does break the Online Continual Learning paradigm as we are using task-specific information." Since promotion timing determines the tree structure, all HGE accuracy and efficiency numbers in Tables 4 and 5 are obtained under access to oracle task-specific information. The paper must either use a fixed threshold across all scenarios, derive a threshold from online-available signals, or report a sensitivity analysis over εpromotion to show that the results do not depend on this oracle choice.
- [Section 5.3, Tables 4 and 5; Abstract] The abstract claims HGE achieves results comparable to GE "with little loss in accuracy," but the paper's own numbers contradict this: HGE accuracy is 82.3 vs GE 97.9 on PMNIST (−15.6 points), 88.62 vs 95.26 on MNIST-CIF10, 85.84 vs 90.3 on CIF10-INV, and 94.18 vs 99.38 on MNIST-KMNIST. These are substantial drops, and Section 6 attributes them to "the changing relationships between experts as they are trained, which causes samples to be assigned incorrectly." The efficiency-accuracy trade-off claimed in the title and abstract is therefore not established. To support the claim, the authors need to address the misassignment problem, not merely report it as a known limitation.
- [Section 5.2, Controlled Experiments] The controlled experiments train each expert separately and freeze it before building the tree, so the online dynamics of concurrent expert training and changing expert relationships are never exercised. The high assignment accuracies in Table 2 therefore cannot be transferred to the online setting, and indeed Tables 4 and 5 show much larger HGE accuracy drops than Table 2 would suggest. This means the central evidence for HGE's efficiency-accuracy trade-off is drawn from a setup that abstracts away exactly the phenomenon the paper identifies as the cause of HGE's failure in practice.
- [Section 5.3, Comparisons to Other Methods] The paper states that the comparison with existing online continual learning methods "is not entirely fair due to some differences in experimental setup." This weakens the claim that GE "is able to match or exceed the state-of-the-art in most benchmarks." The authors should either provide results under matched experimental conditions (e.g., using the same network architectures and evaluation protocol) or clearly quantify which differences affect the comparison. Without this, the competitiveness claim rests on possibly incompatible baselines.
minor comments (4)
- [Section 3.1, EWMA definition] The recurrence for σn is written as an EWMA of absolute deviations from the previous mean, but it is called the standard deviation. Since σ is used to compute a Z-score, the authors should clarify whether they intend a running estimate of the standard deviation or simply a smoothed deviation; if the latter, the label "standard deviation" is misleading.
- [Table 1 and Table 5] The scenario name is misspelled as "MNIST-KMIST" in Table 1 and "MNIST-KNIST" in Table 5; it should be "MNIST-KMNIST".
- [Algorithm 1] In the final line, "remove all samples from r" should be "remove all samples from R" to match the notation used throughout the algorithm.
- [Section 4 and Appendix] The description of εpromotion as "roughly equal to the accuracy achieved at the end of training" is inconsistent with the appendix values: for CIF10 the GE accuracy is 89.06% while εpromotion is 0.95, and for ImgNet(10) the accuracy is 31.02% while εpromotion is 0.5. The relationship between the value and the final accuracy should be clarified.
Circularity Check
No significant circularity; the oracle-tuned promotion threshold is a disclosed validity caveat, not a circular derivation.
full rationale
The paper's derivation chain is empirical rather than analytic: GE detects task switches via a Z-test on EWMA-filtered training losses, and HGE approximates GE's flat minimum-autoencoding-loss selection by a tree search. The controlled experiments in Section 5.2 train experts separately and compare tree search against exhaustive search, so the gating-accuracy comparison is an algorithmic consistency check, not a circular fit. The benchmark comparison in Section 5.3 is against independently published numbers, and the paper discloses that the comparison is 'not entirely fair' due to setup differences. No load-bearing step is justified by self-citation; the only prior works cited (Expert Gate, TAME) are starting points and are explicitly credited. The main caveat is Section 4's admission that a separate epsilon_promotion is set per scenario 'roughly equal to the accuracy achieved at the end of training' and that this 'does break the Online Continual Learning paradigm.' This is a real methodological limitation, because it makes the reported HGE accuracies oracle-influenced upper estimates rather than fully task-agnostic results. However, it is not circularity in the derivation sense: epsilon_promotion is a heuristic schedule parameter, and the reported accuracy is not equal to it by construction. The core task-switch detection results (Table 1) and the GE-vs-HGE query-cost reduction do not depend on this fitted value. Overall, the paper is not circular; it is a standard empirical proposal with a disclosed hyperparameter-tuning caveat.
Assumptions & free parameters
free parameters (8)
- epsilon (high-loss threshold) =
4
- epsilon_review (Z-score threshold) =
20
- promotion path threshold =
0.98
- high-loss buffer capacity =
20
- expert replay buffer capacity =
10
- epsilon_promotion =
0.98 (PMNIST), 0.95 (CIF10), 0.75-0.80 (CIF100), 0.5-0.65 (ImgNet)
- alpha (EWMA smoothing factor) =
0.9
- promotion window =
50
assumptions (4)
- domain assumption Hard task boundaries with a constant C: after a boundary, the next C samples belong to the new task.
- domain assumption Training losses are normally distributed because each loss is a batch mean, justifying the Z-test.
- ad hoc to paper The suitability of an expert for a sample can be measured by its autoencoding loss, and similar experts have correlated losses.
- ad hoc to paper The greedy best-child traversal in the expert tree finds the expert with the globally lowest autoencoding loss.
Cite this review
Pith. "Pith review of Hierarchically Gated Experts for Efficient Online Continual Learning." pith.science (2026). https://pith.science/paper/WIX7ZKCV
@misc{pith2026241217188,
author = {Pith},
title = {Pith review of: Hierarchically Gated Experts for Efficient Online Continual Learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/WIX7ZKCV}},
note = {Machine review of arXiv:2412.17188}
}
read the original abstract
Continual Learning models aim to learn a set of tasks under the constraint that the tasks arrive sequentially with no way to access data from previous tasks. The Online Continual Learning framework poses a further challenge where the tasks are unknown and instead the data arrives as a single stream. Building on existing work, we propose a method for identifying these underlying tasks: the Gated Experts (GE) algorithm, where a dynamically growing set of experts allows for new knowledge to be acquired without catastrophic forgetting. Furthermore, we extend GE to Hierarchically Gated Experts (HGE), a method which is able to efficiently select the best expert for each data sample by organising the experts into a hierarchical structure. On standard Continual Learning benchmarks, GE and HGE are able to achieve results comparable with current methods, with HGE doing so more efficiently.
Figures
Reference graph
Works this paper leans on
-
[1]
write newline
" write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence 'output.state := if if FUNCTION not #0 #1 if FUNCTION and 'skip pop #0 if FUNCTIO...
-
[2]
Aljundi, R., Chakravarty, P., and Tuytelaars, T. (2017). Expert gate: Lifelong learning with a network of experts. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 7120--7129
work page 2017
-
[3]
Chaudhry, A., Rohrbach, M., Elhoseiny, M., Ajanthan, T., Dokania, P. K., Torr, P. H. S., and Ranzato, M. (2019). On tiny episodic memories in continual learning
work page 2019
-
[4]
Clanuwat, T., Bober - Irizar, M., Kitamoto, A., Lamb, A., Yamamoto, K., and Ha, D. (2018). Deep learning for classical japanese literature. CoRR , abs/1812.01718
arXiv 2018
-
[5]
He, K., Zhang, X., Ren, S., and Sun, J. (2015). Deep residual learning for image recognition
2015
-
[6]
Hemati, H., Lomonaco, V., Bacciu, D., and Borth, D. (2023). Partial hypernetworks for continual learning
work page 2023
-
[7]
Hihn, H. and Braun, D. A. (2023). Hierarchically structured task-agnostic continual learning. Machine Learning , 112(2):655--686
work page 2023
-
[8]
Kang, H., Mina, R. J. L., et al. (2022). Forget-free continual learning with winning subnetworks. In Proceedings of the 39th International Conference on Machine Learning , pages 10734--10750. PMLR
work page 2022
Show all 30 references
-
[9]
G., and Pascanu, R
Kirichenko, P., Farajtabar, M., Rao, D., Lakshminarayanan, B., Levine, N., Li, A., Hu, H., Wilson, A. G., and Pascanu, R. (2021). Task-agnostic continual learning with hybrid probabilistic models. CoRR , abs/2106.12772
2021 arXiv
-
[10]
Kirkpatrick, J., Pascanu, R., et al. (2017). Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences , 114(13):3521--3526
2017
-
[11]
Krizhevsky, A. (2012). Learning multiple layers of features from tiny images. University of Toronto
2012
-
[12]
Lecun, Y., Bottou, L., Bengio, Y., and Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE , 86(11):2278--2324
1998
-
[13]
Lee, S., Ha, J., Zhang, D., and Kim, G. (2020). A neural dirichlet process mixture model for task-free continual learning. CoRR , abs/2001.00689
2020 arXiv
-
[14]
and Hoiem, D
Li, Z. and Hoiem, D. (2017). Learning without forgetting
2017
-
[15]
Lin, D. (2013). Online learning of nonparametric mixture models via sequential variational approximation. In Advances in Neural Information Processing Systems , volume 26. Curran Associates, Inc
2013
-
[16]
and Ranzato, M
Lopez - Paz, D. and Ranzato, M. (2017). Gradient episodic memory for continuum learning. CoRR , abs/1706.08840
2017 arXiv
-
[17]
V., Li, Y., Bui, T
Nguyen, C. V., Li, Y., Bui, T. D., and Turner, R. E. (2018). Variational continual learning
2018
-
[18]
I., Kemker, R., Part, J
Parisi, G. I., Kemker, R., Part, J. L., Kanan, C., and Wermter, S. (2018). Continual lifelong learning with neural networks: A review. CoRR , abs/1802.07569
2018 arXiv
-
[19]
C., and Fei-Fei, L
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A. C., and Fei-Fei, L. (2015). Imagenet large scale visual recognition challenge
2015
-
[20]
Shazeer, N., Mirhoseini, A., Maziarz, K., Davis, A., Le, Q., Hinton, G., and Dean, J. (2017). Outrageously large neural networks: The sparsely-gated mixture-of-experts layer
2017
-
[21]
K., Kim, J., and Kim, J
Shin, H., Lee, J. K., Kim, J., and Kim, J. (2017). Continual learning with deep generative replay
2017
-
[22]
Wang, L., Zhang, X., Su, H., and Zhu, J. (2024). A comprehensive survey of continual learning: Theory, method and application
2024
-
[23]
Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., and Farhadi, A. (2020). Supermasks in superposition
2020
-
[24]
and Zhu, Z
Xu, J. and Zhu, Z. (2018). Reinforced continual learning
2018
-
[25]
Yan, Q., Gong, D., Liu, Y., van den Hengel, A., and Shi, J. Q. (2022). Learning bayesian sparse networks with full experience replay for continual learning. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 109--118
2022
-
[26]
Yan, S., Xie, J., and He, X. (2021). DER: Dynamically Expandable Representation for Class Incremental Learning . In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 3013--3022, Los Alamitos, CA, USA. IEEE Computer Society
2021
-
[27]
Yoon, J., Yang, E., Lee, J., and Hwang, S. J. (2018). Lifelong learning with dynamically expandable networks. In 6th International Conference on Learning Representations, ICLR 2018
2018
-
[28]
Zenke, F., Poole, B., and Ganguli, S. (2017). Continual learning through synaptic intelligence. In Proceedings of the 34th International Conference on Machine Learning - Volume 70 , ICML'17, page 3987–3995. JMLR.org
2017
-
[29]
Zeno, C., Golan, I., Hoffer, E., and Soudry, D. (2018). Task agnostic continual learning using online variational bayes. arXiv preprint arXiv:1803.10123
2018 arXiv
-
[30]
Zhu, H., Majzoubi, M., Jain, A., and Choromanska, A. (2022). Tame: Task agnostic continual learning using multiple experts
2022
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.