Theory of Mixture-of-Experts for Mobile Edge Computing

Hongbo Li; Lingjie Duan

REVIEW 5 major objections 5 minor 1 cited by

Theory of Mixture-of-Experts for Mobile Edge Computing

T0 review · 5 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that an availability-aware mixture-of-experts router can keep the overall generalization error of continual learning in mobile edge computing bounded by a small constant, whereas nearest-or-most-powerful offloading lets…

desk verdict Genuinely new MoE-for-MEC theory, but the central convergence proof rests on an unstated signal-closeness assumption; fixable but not ready as is. read the letter →

arxiv 2412.15690 v2 pith:XLO5PPOB submitted 2024-12-20 cs.LG

Hongbo Li , Lingjie Duan This is my paper · ORCID

classification cs.LG

keywords mixture-of-expertsmobileedgecomputingcontinuallearninggeneralizationerroradaptivegatingnetworkexpertavailabilityoverparameterizedlinearregressiontaskoffloading

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to show that treating mobile edge servers as mixture-of-experts (MoE) experts, with a gating network that routes tasks only to idle servers, prevents the generalization error of continual learning from growing over time. It proves that once the number of servers exceeds a derived threshold and an exploration phase has passed, each server specializes in one task cluster, and the average error stays bounded by a small constant that depends only on within-cluster similarity. This contrasts with standard nearest-or-most-powerful offloading, for which the paper derives an error that tends to the maximum expected gap between different task clusters as time grows. A surprising implication is that adding more servers, when already enough for convergence, delays convergence and worsens finite-time error.

What carries the argument

The central mechanism is the adaptive gating network (AGN) combined with switch routing and a locality loss. The gate computes linear scores (3), the router picks the idle expert with the highest score plus exploration noise (4), and the locality loss (11) uses each expert's model-update magnitude to push the softmax outputs (5) apart; gradient descent on this loss (13) diversifies the gate parameters. The convergence proof also leans on the availability concentration bound (Proposition 1), which sizes $M$ via a binomial/normal tail so that a specialized idle expert exists with probability at least $1-\delta$, and on the minimum-norm projection update (9) from overparameterized linear regression, which makes expert-model updates contract after correct routing.

What would settle it

Generate tasks with the same cluster structure but no single embedded signal vector in the feature matrix, e.g., all $s$ samples are informative and cluster differences are spread across samples, then run Algorithm 1 and measure the overall error (20) for large $T$; if the error does not approach $O(\sigma_0^2)$, then Lemma 2's separation mechanism—not just the constants—fails.

Watch

Extended reading notes

Core claim

The central claim is that a mixture-of-experts architecture, where each MEC server is an expert, can be made to specialize under online availability constraints. With $M = \Omega(N M_{\mathrm{th}} \ln(1/\delta))$ experts, where $M_{\mathrm{th}}$ is defined in (16), the overall generalization error of Algorithm 1 satisfies (23) and converges to the minimum within-cluster model error $O(\sigma_0^2)$ as the time horizon $T\to\infty$. The router in (4) selects only among idle experts, the gate is trained with a locality loss that rewards routing same-cluster tasks to the same expert, and after an exploration phase of $T_1 = d_u + \lceil \eta^{-1}\sigma_0^{-0.5}M\ln(M/\delta)\rceil$ rounds, the softmax outputs separate across expert sets (Proposition 2), so same-cluster tasks consistently reach the same expert and each expert's model stabilizes to within $O(\sigma_0^2)$ of its cluster (Proposition 3). In contrast, always offloading to the nearest or most powerful server yields an error that, by Proposition 4, approaches the maximum expected gap $\mathbb{E}[\|w_n - w_{n'}\|^2 \mid n,n'\in[N]]$ as $T$ grows.

Load-bearing premise

The whole routing analysis rests on Assumption 1 and Definition 1: every task's dataset contains exactly one sample equal to a type-specific signal vector, all other samples are isotropic Gaussian noise, and ground truths in the same cluster are $O(\sigma_0^2)$-close while different clusters are $\Theta(\sigma_0)$-apart; additionally, the availability bound silently assumes task types arrive uniformly over clusters.

Editorial extensions

If this is right

MEC operators can provision the minimum expert count from (16) and guarantee, with probability at least $1-\delta$, that every arriving task finds a specialized idle server after convergence.
After the exploration phase $T_1$, the adaptive gating network keeps routing correctly without further exploration, so per-task error stabilizes at $O(\sigma_0^2)$ rather than growing with $T$.
If the number of experts already satisfies the convergence bound, adding more experts increases convergence time and worsens the finite-time generalization error (23), so oversizing is counterproductive.
The contrast with Proposition 4 means that standard nearest/most-powerful offloading has no such guarantee: its error approaches the maximum expected gap between distinct task clusters as $T\to\infty$.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Although the paper only analyzes the uniform-arrival case implicitly in Appendix B, the availability bound would need inflation for skewed arrivals: a hot task type arriving more frequently than $1/N$ would exhaust its specialized experts faster, so (16) is a lower bound for uniform traffic, not for adversarial or skewed traffic.
The locality-loss mechanism suggests a general routing principle for distributed continual learning: the update magnitude of a worker is itself a sufficient training signal for routing, so the same AGN design could transfer to federated or edge settings where server availability varies.
One testable extension is replacing the linear gate with a smooth nonlinear scoring function; the softmax separation argument in Lemma 5 and Proposition 2 should survive for Lipschitz activations, but the exponents in $T_1$ and the $\sigma_0$ thresholds would need recomputation.
The 'more experts hurts' result implies that an operator should choose the smallest $M$ satisfying the bound; a larger cluster is a liability in streaming settings unless the extra experts reduce $d_u$ (e.g., shorter training times), which would alter the trade-off.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this is worth taking seriously, but the main theorem is not yet supported. The paper's genuine contributions are the availability-aware gating design, the minimum-expert bound in Prop 1, and the finite-horizon result that more experts can hurt. Those are new relative to the cited empirical MEC-MoE works and to the authors' prior continual-learning MoE paper. The experiments roughly match the qualitative claims, which is a plus.

The soft spot is in the load path from Lemma 2 to Props 2 and 3 to Theorem 1. Lemma 2 says that for two tasks in the same ground-truth cluster, the same expert's gating outputs differ by O(σ0). The proof bounds the noise sum and then writes |θ^T(v_n - v_n')| = O(σ0^2) with no justification. Definition 1 and Assumption 1 constrain the ground truths, not the signals; nothing prevents two signals in the same cluster from being far apart. If they are Θ(1) apart, the lemma is false and Proposition 2 collapses. That is an internal gap, not just a question about whether real tasks have embedded signals.

There are smaller issues: Proposition 3 uses an undefined T2; Appendix B silently assumes i.i.d. uniform task-cluster arrivals, which is not in the Section II model; the Binomial-to-normal approximation has no uniform error bound; and the contradiction step in Appendix C uses the wrong condition on η (they need η = Ω(σ0), not O(σ0^0.5)).

None of this looks unfixable. Adding a cluster-Lipschitz condition on the signals, stating the arrival model explicitly, and rewriting the appendix proofs would likely make the argument go through. As it stands, the theorem is not established by the text.

I'd send this to peer review rather than desk-reject. The idea is specific enough and the claimed effect (more experts can hurt) is interesting enough that a competent referee should be asked to verify the repairs. The authors should be given the chance to add the missing assumptions and tighten the appendix. If the signal-closeness assumption turns out to be very strong, the paper's scope will shrink, but the core story survives.

Referee Report

5 major / 5 minor

Summary. The paper proposes a theory of mixture-of-experts for mobile edge computing. It models each edge server as an expert and introduces an adaptive gating network that routes incoming tasks to idle experts according to gating outputs and server availability. The main results are: (i) a lower bound M = Ω(N M_th log(1/δ)) on the number of experts needed to guarantee that an idle expert of the correct type exists (Prop. 1); (ii) convergence of the gating/router to specialized experts (Prop. 2) and of expert models within clusters (Prop. 3); and (iii) an overall generalization-error bound converging to O(σ0^2) (Theorem 1), contrasted with a benchmark result showing that nearest/most-powerful offloading incurs an error approaching the inter-cluster gap (Prop. 4). Numerical experiments on synthetic linear data and MNIST are reported.

Significance. If the proof chain were made rigorous, this would be a useful first theoretical treatment of MoE in MEC: it explicitly accounts for server unavailability, gives a quantitative condition on the number of experts, and identifies a finite-horizon trade-off in which extra experts slow convergence. The paper also provides an explicit decomposition of the generalization error into exploration and specialization terms, and it tests the claims on real DNN data. The contribution is therefore potentially significant. At present, however, the main theorem rests on several unstated assumptions and proof gaps, so the stated results are not established as written.

major comments (5)

[Lemma 2, Appendix A, Definition 1] Lemma 2 asserts that two tasks with ground truths in the same cluster have gating outputs within O(σ0). This requires control of |θ^T(v_n − v_n')|, and the proof in Appendix A bounds this by O(σ0^2). But Assumption 1 and Definition 1 only constrain the ground-truth vectors w_t, not the feature signals v_t; two tasks in the same cluster may have very different or even orthogonal signals, in which case Lemma 2 is false. Since Proposition 2 and Theorem 1 rely on Lemma 2 through Lemmas 5–6 and term G4, the main bound is unsupported unless the model is strengthened, for example by assuming that all tasks in a cluster share the same feature signal v_n or that signals satisfy a cluster-Lipschitz condition.
[Proposition 1, Appendix B, Section II] Proposition 1 requires a stochastic model of task-type arrivals, but Section II only says that distributions are unknown. Appendix B writes that each task is selected independently with probability 1/B (presumably 1/N) and approximates the Binomial An by a normal distribution without a quantifiable uniform error bound. As a result, the claimed guarantee 'with probability at least 1 − o(1)' does not follow from the stated condition M = Ω(N M_th log(1/δ)). The authors should either add an i.i.d. uniform arrival assumption to Section II and prove a Chernoff-type bound, or state a weaker theorem.
[Proposition 3, Eq. (18)] The proposition contains an undefined time index T2. The statement 'for any task arrival t > T1, each expert m satisfies ∥w(m)_t − w(m)_{T2}∥∞ = O(σ0^2)' cannot be interpreted without a definition of T2. If T2 is meant to be T1 (or T1+1), the proposition should say so; as written, Theorem 1's use of Proposition 3 for the final time T is not justified.
[Appendix C, Lemmas 6 and 8] The proof of Lemma 6 uses |⟨θ_t, v_n − v_n'⟩| = ∥θ_t∥∞ ∥v_n − v_n'∥∞, which is not a valid vector inequality: the correct upper bound is ∥θ_t∥_1 ∥v_n − v_n'∥∞, or ∥θ_t∥_2 ∥v_n − v_n'∥_2. The same error appears in Lemma 8. Since p may be large in the overparameterized regime s < p, the dimension factor cannot be ignored, and the O(σ0^{0.5}) and O(σ0 η^{−0.5}) bounds used in the contradiction argument of Proposition 2 do not follow.
[Proposition 4, Eq. (22); Theorem 1, Eq. (23) and Appendix F] The benchmark result in Proposition 4 and the comparison in Theorem 1 rely on the assumption that the tasks routed to each expert are i.i.d. uniform over the N clusters, but this is not part of the stated system model. Moreover, the expectation E[∥w_n − w_n'∥^2 | n,n' ∈ [N]] as written includes n = n', and the text says the error 'approaches the maximum' while the formula computes an average over pairs; the statement should be made precise. In the proof of Theorem 1 in Appendix F, the step E[∥w(m_t)_T − w_{n_t}∥^2] = E[∥w(m_t)_{T1} − w_{n_t}∥^2] is not justified for tasks arriving after T1, because the model at time T1 has not seen those tasks; a clearer decomposition of the post-T1 projection factors is needed.

minor comments (5)

[Assumption 1] There is a typo: 'ground different truths' should be 'different ground truths'.
[Section III-B and Algorithm 1] The update index for θ is inconsistent: Eq. (13) updates θ(m)_{t+dt+1}, while Algorithm 1 line 8 says 'θ(m)_t as in (13)'. Please align the indices.
[Definition 1 and Section II-B] The text refers to a 'unique feature signal v_t' without clarifying whether signals are unique per task or shared within a cluster; Lemma 2 depends on this distinction, so it should be stated explicitly.
[Eq. (23)] The exponent r^{L(m_t)}_T − L(m_t)_{T1} is typographically ambiguous; adding parentheses or a clearly defined notation would improve readability.
[Section VI] In the MNIST experiment, the sentence 'we verify our Assumption 1 that the variance among different types of tasks is σ0 = 0.1' is not supported by any shown measurement; give details of the verification or rephrase as a calibration procedure.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the main error bound is derived from the stated data-generation assumptions plus a proved routing-convergence argument, not from a fitted parameter or a self-citation chain.

full rationale

Walking the derivation chain: Assumption 1 and Definition 1 define within-cluster ground-truth separation O(σ0^2), unique feature signals, and the one-signal-plus-noise data model. Lemma 2 uses these to bound gating-output differences; Proposition 2 proves expert-set separation at T1; Proposition 3 proves the selected expert's model stays within the same-cluster ground-truth gap; Theorem 1 combines these with Lemma 9 to obtain (23). The O(σ0^2) term G4 in (23) is the same O(σ0^2) as Assumption 1's within-cluster gap, and in that sense the theorem's 'minimum model error' is an entailment of the model assumption rather than an independent empirical prediction. But that is not circularity: the paper does not fit σ0 to the data in the theorem, does not define the error as the assumption, and the nontrivial content is that the exploration error G3 decays to zero and the router provably stabilizes experts within clusters. The more-experts-worsens-error result follows from monotonicity of r^{L_T} in (23), not from an assumed conclusion. The only same-author reference ([19]) is not load-bearing in the proofs and does not appear as an in-text citation in the derivation chain. Two non-circular proof-completeness concerns exist: Appendix A's proof of Lemma 2 bounds |θ^T(v_n − v_{n_t})| as O(σ0^2) without an explicit assumption that feature signals in the same cluster are close (Assumption 1 constrains ground truths w_t, not signals v_t), and Appendix B silently assumes independent/uniform task-type arrivals. These are correctness/completeness gaps, not reductions of the result to its input by construction.

Assumptions & free parameters 5 free parameters · 7 assumptions · 0 invented entities

The paper introduces no new physical entities or forces. It does introduce a strong idealized data-generation model (Definition 1) and a hidden arrival-distribution assumption (Appendix B), both of which are burden-bearing. The free parameters sigma0, du, eta, and delta are input parameters, not fitted constants, but their values are chosen by the authors in experiments.

free parameters (5)

sigma_0 = 0.6 (synthetic), 0.1 (MNIST)
Assumption 1 parameter controlling within-cluster and between-cluster ground-truth separation; all gap claims, including the final O(sigma0^2) bound, depend on it.
du = 10 (synthetic), 4 (MNIST)
Upper bound on transmission plus execution delay in (1); enters the minimum-expert threshold Mth and the convergence time T1.
eta = 0.2 in both experiments
Learning rate in update (13); T1 is stated as Omega(eta^{-1} sigma0^{-0.5} M ln(M/delta)).
delta = o(1), not numerically specified
Failure probability tolerance in Proposition 1 and in T1; drives the M = Omega(N Mth ln(1/delta)) scaling.
beta_t and C = C = O(1), no numeric value
Random scaling of the signal sample in Definition 1; affects the signal-to-noise structure of the data model.

assumptions (7)

domain assumption Assumption 1: ground truths in the same cluster differ by O(sigma0^2) in infinity norm, and ground truths in different clusters differ by Theta(sigma0).
Defines the task-cluster separation that lets the linear gating network identify task types; introduced in Section III-A before Lemma 2.
ad hoc to paper Definition 1: each task dataset contains exactly one signal sample beta_t v_n and s-1 isotropic Gaussian noise samples.
This structure guarantees Lemma 2 and the router convergence proof; it is not derived from a physical MEC model and is unlikely to hold exactly for real workloads.
domain assumption Task-cluster arrivals are independent and uniform over N clusters in each time slot.
Used in Appendix B to approximate the number of busy experts as Binomial(du, 1/N); never stated as an assumption in Section II.
domain assumption Overparameterized regime with s < p and N < M.
Needed for the minimum-norm update rule in Lemma 1 and for the convergence requirement N < M in Section III-A.
domain assumption Each selected expert stays busy for a bounded random delay dt <= du and cannot be selected again until it finishes.
Defines the availability model in (2) and is the key departure from classical MoE; essential for Proposition 1.
standard math Standard tools: Hoeffding and Chernoff inequalities, normal approximation to binomial, non-expansiveness of orthogonal projections, softmax derivative identities.
Invoked throughout Appendices A-F without proof; normal approximation would require additional tail bounds to be rigorous.
standard math Existing continual-learning generalization-error decomposition and minimum-norm GD solution from refs [28]-[30] and [29].
Taken as external results; Lemma 1 and Lemma 9 rely on them.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Theory of Mixture-of-Experts for Mobile Edge Computing." pith.science (2026). https://pith.science/paper/XLO5PPOB

@misc{pith2026241215690,
  author       = {Pith},
  title        = {Pith review of: Theory of Mixture-of-Experts for Mobile Edge Computing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XLO5PPOB}},
  note         = {Machine review of arXiv:2412.15690}
}

read the original abstract

In mobile edge computing (MEC) networks, mobile users generate diverse machine learning tasks dynamically over time. These tasks are typically offloaded to the nearest available edge server, by considering communication and computational efficiency. However, its operation does not ensure that each server specializes in a specific type of tasks and leads to severe overfitting or catastrophic forgetting of previous tasks. To improve the continual learning (CL) performance of online tasks, we are the first to introduce mixture-of-experts (MoE) theory in MEC networks and save MEC operation from the increasing generalization error over time. Our MoE theory treats each MEC server as an expert and dynamically adapts to changes in server availability by considering data transfer and computation time. Unlike existing MoE models designed for offline tasks, ours is tailored for handling continuous streams of tasks in the MEC environment. We introduce an adaptive gating network in MEC to adaptively identify and route newly arrived tasks of unknown data distributions to available experts, enabling each expert to specialize in a specific type of tasks upon convergence. We derived the minimum number of experts required to match each task with a specialized, available expert. Our MoE approach consistently reduces the overall generalization error over time, unlike the traditional MEC approach. Interestingly, when the number of experts is sufficient to ensure convergence, adding more experts delays the convergence time and worsens the generalization error. Finally, we perform extensive experiments on real datasets in deep neural networks (DNNs) to verify our theoretical results.

Figures

Figures reproduced from arXiv: 2412.15690 by the authors.

**Figure 1.** An illustration of MEC networks with M edge servers as experts. At the beginning of time t, a mobile user arrives to request a task-training service from its nearest Base Station (BS) of expert m˜ t (e.g., m˜ t = 1 in this case). Then our adaptive MoE gating network in [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗

**Figure 2.** The MoE structure of the MEC network operator in Fig. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** The dynamics of overall generalization errors under our MoE Algorithm [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: The dynamics of overall generalization errors under our Algorithm [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

MoE$^2$: Optimizing Collaborative Inference for Edge Large Language Models
cs.NI 2025-01 reject novelty 4.0 of 10

MoE2 trains a gating network once on all edge LLMs, then selects a resource-feasible subset and a per-prompt top-k of experts to fuse token probabilities under latency and energy budgets.

Reference graph

Works this paper leans on

33 extracted references · 26 canonical work pages · cited by 1 Pith paper

[1]

Distributed machine learning for multiuser mobile edge computing systems,

Y . Guo, R. Zhao, S. Lai, L. Fan, X. Lei, and G. K. Karagiannidis, “Distributed machine learning for multiuser mobile edge computing systems,” IEEE Journal of Selected Topics in Signal Processing , vol. 16, no. 3, pp. 460–473, 2022

work page 2022
[2]

Federated learning for online resource allocation in mobile edge computing: A deep reinforcement learning approach,

J. Zheng, K. Li, N. Mhaisen, W. Ni, E. Tovar, and M. Guizani, “Federated learning for online resource allocation in mobile edge computing: A deep reinforcement learning approach,” in 2023 IEEE Wireless Communica- tions and Networking Conference (WCNC) . IEEE, 2023, pp. 1–6

work page 2023
[3]

Follow me at the edge: Mobility- aware dynamic service placement for mobile edge computing,

T. Ouyang, Z. Zhou, and X. Chen, “Follow me at the edge: Mobility- aware dynamic service placement for mobile edge computing,” IEEE Journal on Selected Areas in Communications , vol. 36, no. 10, pp. 2333– 2345, 2018

work page 2018
[4]

A survey on the computation offloading approaches in mobile edge computing: A machine learning-based perspective,

A. Shakarami, M. Ghobaei-Arani, and A. Shahidinejad, “A survey on the computation offloading approaches in mobile edge computing: A machine learning-based perspective,” Computer Networks , vol. 182, p. 107496, 2020

work page 2020
[5]

Winning at the starting line: Joint network selection and service placement for mobile edge computing,

B. Gao, Z. Zhou, F. Liu, and F. Xu, “Winning at the starting line: Joint network selection and service placement for mobile edge computing,” in IEEE INFOCOM 2019-IEEE Conference on Computer Communications . IEEE, 2019, pp. 1459–1467

work page 2019
[6]

Pricing-driven service caching and task offloading in mobile edge computing,

J. Yan, S. Bi, L. Duan, and Y .-J. A. Zhang, “Pricing-driven service caching and task offloading in mobile edge computing,” IEEE Transactions on Wireless Communications, vol. 20, no. 7, pp. 4495–4512, 2021

work page 2021
[7]

Catastrophic interference in connection- ist networks: The sequential learning problem,

M. McCloskey and N. J. Cohen, “Catastrophic interference in connection- ist networks: The sequential learning problem,” inPsychology of Learning and Motivation. Elsevier, 1989, vol. 24, pp. 109–165

work page 1989
[8]

Overcoming catastrophic forgetting in neural networks,

J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska et al. , “Overcoming catastrophic forgetting in neural networks,” Proceedings of the National Academy of Sciences, vol. 114, no. 13, pp. 3521–3526, 2017

work page 2017

Show all 33 references

[9]

Beyond not-forgetting: Con- tinual learning with backward knowledge transfer,

S. Lin, L. Yang, D. Fan, and J. Zhang, “Beyond not-forgetting: Con- tinual learning with backward knowledge transfer,” Advances in Neural Information Processing Systems , vol. 35, pp. 16 165–16 177, 2022

2022
[10]

Inference for the generalization error,

C. Nadeau and Y . Bengio, “Inference for the generalization error,” Advances in neural information processing systems , vol. 12, 1999

1999
[11]

Generalization error of ensemble estima- tors,

N. Ueda and R. Nakano, “Generalization error of ensemble estima- tors,” in Proceedings of International Conference on Neural Networks (ICNN’96), vol. 1. IEEE, 1996, pp. 90–95

1996
[12]

Learning factored represen- tations in a deep mixture of experts,

D. Eigen, M. Ranzato, and I. Sutskever, “Learning factored represen- tations in a deep mixture of experts,” arXiv preprint arXiv:1312.4314 , 2013

2013 arXiv
[13]

Outrageously large neural networks: The sparsely-gated mixture-of-experts layer,

N. Shazeer, A. Mirhoseini, K. Maziarz, A. Davis, Q. Le, G. Hinton, and J. Dean, “Outrageously large neural networks: The sparsely-gated mixture-of-experts layer,” in International Conference on Learning Rep- resentations, 2016

2016
[14]

Scaling vision with sparse mixture of experts,

C. Riquelme, J. Puigcerver, B. Mustafa, M. Neumann, R. Jenatton, A. Susano Pinto, D. Keysers, and N. Houlsby, “Scaling vision with sparse mixture of experts,” Advances in Neural Information Processing Systems , vol. 34, pp. 8583–8595, 2021

2021
[15]

Glam: Efficient scaling of language models with mixture-of-experts,

N. Du, Y . Huang, A. M. Dai, S. Tong, D. Lepikhin, Y . Xu, M. Krikun, Y . Zhou, A. W. Yu, O. Firat et al., “Glam: Efficient scaling of language models with mixture-of-experts,” in International Conference on Machine Learning. PMLR, 2022, pp. 5547–5569

2022
[16]

Megablocks: Ef- ficient sparse training with mixture-of-experts,

T. Gale, D. Narayanan, C. Young, and M. Zaharia, “Megablocks: Ef- ficient sparse training with mixture-of-experts,” Proceedings of Machine Learning and Systems , vol. 5, pp. 288–304, 2023

2023
[17]

Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity,

W. Fedus, B. Zoph, and N. Shazeer, “Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity,” Journal of Machine Learning Research , vol. 23, no. 120, pp. 1–39, 2022

2022
[18]

Towards understanding the mixture-of-experts layer in deep learning,

Z. Chen, Y . Deng, Y . Wu, Q. Gu, and Y . Li, “Towards understanding the mixture-of-experts layer in deep learning,” Advances in Neural Information Processing Systems , vol. 35, pp. 23 049–23 062, 2022

2022
[19]

Theory on mixture- of-experts in continual learning,

H. Li, S. Lin, L. Duan, Y . Liang, and N. B. Shroff, “Theory on mixture- of-experts in continual learning,” arXiv preprint arXiv:2406.16437, 2024

2024 arXiv
[20]

Deepspeed-moe: Advancing mixture-of-experts inference and training to power next-generation ai scale,

S. Rajbhandari, C. Li, Z. Yao, M. Zhang, R. Y . Aminabadi, A. A. Awan, J. Rasley, and Y . He, “Deepspeed-moe: Advancing mixture-of-experts inference and training to power next-generation ai scale,” in International Conference on Machine Learning . PMLR, 2022, pp. 18 332–18 346

2022
[21]

A hybrid tensor-expert-data parallelism approach to optimize mixture- of-experts training,

S. Singh, O. Ruwase, A. A. Awan, S. Rajbhandari, Y . He, and A. Bhatele, “A hybrid tensor-expert-data parallelism approach to optimize mixture- of-experts training,” in Proceedings of the 37th International Conference on Supercomputing, 2023, pp. 203–214

2023
[22]

Toward scalable generative ai via mixture of experts in mobile edge networks,

J. Wang, H. Du, D. Niyato, J. Kang, Z. Xiong, D. I. Kim, and K. B. Letaief, “Toward scalable generative ai via mixture of experts in mobile edge networks,” arXiv preprint arXiv:2402.06942 , 2024

2024 arXiv
[23]

Moesys: A distributed and efficient mixture-of-experts training and inference system for internet services,

D. Yu, L. Shen, H. Hao, W. Gong, H. Wu, J. Bian, L. Dai, and H. Xiong, “Moesys: A distributed and efficient mixture-of-experts training and inference system for internet services,” IEEE Transactions on Services Computing, 2024

2024
[24]

Coscl: Cooperation of small continual learners is stronger than a big one,

L. Wang, X. Zhang, Q. Li, J. Zhu, and Y . Zhong, “Coscl: Cooperation of small continual learners is stronger than a big one,” in European Conference on Computer Vision . Springer, 2022, pp. 254–271

2022
[25]

Joint planning of mec and fiber deployment in sparsely populated areas,

C. Anzola-Rojas, R. J. D. Barroso, I. de Miguel, N. Merayo, J. C. Aguado, P. Fern ´andez, R. M. Lorenzo, and E. J. Abril, “Joint planning of mec and fiber deployment in sparsely populated areas,” in 2021 International Conference on Optical Network Design and Modeling (ONDM). I...

2021
[26]

Locmoe: A low-overhead moe for large language model training,

J. Li, Z. Sun, X. He, L. Zeng, Y . Lin, E. Li, B. Zheng, R. Zhao, and X. Chen, “Locmoe: A low-overhead moe for large language model training,” arXiv preprint arXiv:2401.13920 , 2024

2024 arXiv
[27]

A theoretical understanding of shallow vision transformers: Learning, generalization, and sample complexity,

H. Li, M. Wang, S. Liu, and P.-Y . Chen, “A theoretical understanding of shallow vision transformers: Learning, generalization, and sample complexity,” in International Conference on Learning Representations (ICLR 2023), 2023

2023
[28]

How catastrophic can catastrophic forgetting be in linear regression?

I. Evron, E. Moroshko, R. Ward, N. Srebro, and D. Soudry, “How catastrophic can catastrophic forgetting be in linear regression?” in Conference on Learning Theory . PMLR, 2022, pp. 4028–4079

2022
[29]

Theory on forgetting and generalization of continual learning,

S. Lin, P. Ju, Y . Liang, and N. Shroff, “Theory on forgetting and generalization of continual learning,” in International Conference on Machine Learning. PMLR, 2023, pp. 21 078–21 100

2023
[30]

Characterizing implicit bias in terms of optimization geometry,

S. Gunasekar, J. Lee, D. Soudry, and N. Srebro, “Characterizing implicit bias in terms of optimization geometry,” in International Conference on Machine Learning. PMLR, 2018, pp. 1832–1841

2018
[31]

Efficient lifelong learning with a-gem,

A. Chaudhry, M. Ranzato, M. Rohrbach, and M. Elhoseiny, “Efficient lifelong learning with a-gem,” in International Conference on Learning Representations, 2018

2018
[32]

A theoretical analysis of catastrophic forgetting through the ntk overlap ma- trix,

T. Doan, M. A. Bennani, B. Mazoure, G. Rabusseau, and P. Alquier, “A theoretical analysis of catastrophic forgetting through the ntk overlap ma- trix,” in International Conference on Artificial Intelligence and Statistics . PMLR, 2021, pp. 1072–1080

2021
[33]

Handwritten digit recognition with a back-propagation network,

Y . LeCun, B. Boser, J. Denker, D. Henderson, R. Howard, W. Hubbard, and L. Jackel, “Handwritten digit recognition with a back-propagation network,” Advances in Neural Information Processing Systems , vol. 2, 1989

1989

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

Distributed machine learning for multiuser mobile edge computing systems,

Y . Guo, R. Zhao, S. Lai, L. Fan, X. Lei, and G. K. Karagiannidis, “Distributed machine learning for multiuser mobile edge computing systems,” IEEE Journal of Selected Topics in Signal Processing , vol. 16, no. 3, pp. 460–473, 2022

work page 2022

[2] [2]

Federated learning for online resource allocation in mobile edge computing: A deep reinforcement learning approach,

J. Zheng, K. Li, N. Mhaisen, W. Ni, E. Tovar, and M. Guizani, “Federated learning for online resource allocation in mobile edge computing: A deep reinforcement learning approach,” in 2023 IEEE Wireless Communica- tions and Networking Conference (WCNC) . IEEE, 2023, pp. 1–6

work page 2023

[3] [3]

Follow me at the edge: Mobility- aware dynamic service placement for mobile edge computing,

T. Ouyang, Z. Zhou, and X. Chen, “Follow me at the edge: Mobility- aware dynamic service placement for mobile edge computing,” IEEE Journal on Selected Areas in Communications , vol. 36, no. 10, pp. 2333– 2345, 2018

work page 2018

[4] [4]

A survey on the computation offloading approaches in mobile edge computing: A machine learning-based perspective,

A. Shakarami, M. Ghobaei-Arani, and A. Shahidinejad, “A survey on the computation offloading approaches in mobile edge computing: A machine learning-based perspective,” Computer Networks , vol. 182, p. 107496, 2020

work page 2020

[5] [5]

Winning at the starting line: Joint network selection and service placement for mobile edge computing,

B. Gao, Z. Zhou, F. Liu, and F. Xu, “Winning at the starting line: Joint network selection and service placement for mobile edge computing,” in IEEE INFOCOM 2019-IEEE Conference on Computer Communications . IEEE, 2019, pp. 1459–1467

work page 2019

[6] [6]

Pricing-driven service caching and task offloading in mobile edge computing,

J. Yan, S. Bi, L. Duan, and Y .-J. A. Zhang, “Pricing-driven service caching and task offloading in mobile edge computing,” IEEE Transactions on Wireless Communications, vol. 20, no. 7, pp. 4495–4512, 2021

work page 2021

[7] [7]

Catastrophic interference in connection- ist networks: The sequential learning problem,

M. McCloskey and N. J. Cohen, “Catastrophic interference in connection- ist networks: The sequential learning problem,” inPsychology of Learning and Motivation. Elsevier, 1989, vol. 24, pp. 109–165

work page 1989

[8] [8]

Overcoming catastrophic forgetting in neural networks,

J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska et al. , “Overcoming catastrophic forgetting in neural networks,” Proceedings of the National Academy of Sciences, vol. 114, no. 13, pp. 3521–3526, 2017

work page 2017