{"total":108,"items":[{"citing_arxiv_id":"2606.26938","ref_index":4,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"Focusing on What Matters: Saliency-Harnessing Accurate Routing for Diffusion MoE","primary_cat":"cs.CV","submitted_at":"2026-06-25T12:13:40+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":7.0,"formal_verification":"none","one_line_summary":"SharpMoE is a plug-and-play post-training method that uses clean latent features and a trajectory routing loss to enable accurate saliency-based routing in diffusion MoE models for improved visual generation.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2606.25971","ref_index":10,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"Improving Neural Network Training by Decoupling the Magnitude and Direction of Weight Vectors","primary_cat":"cs.LG","submitted_at":"2026-06-24T15:40:26+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"MD Decoupling factorizes weights into fixed-norm directions and learnable per-row/column magnitudes updated at independent rates, improving Adam and Muon training stability and scale transfer without weight decay or warmup.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2606.26183","ref_index":31,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"LiMoDE: Rethinking Lifelong Robot Manipulation from a Mixture-of-Dynamic-Experts Perspective","primary_cat":"cs.RO","submitted_at":"2026-06-24T13:18:43+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":4.0,"formal_verification":"none","one_line_summary":"LiMoDE uses dynamic MoE pre-training on motion cues followed by lifelong expert addition for continuous robot task adaptation.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2606.23739","ref_index":5,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"Systematic Exploration of 4-Expert Heterogeneous Mixture-of-Experts via Automated Pipeline Search","primary_cat":"cs.LG","submitted_at":"2026-06-21T13:43:27+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":5.0,"formal_verification":"none","one_line_summary":"Automated search of 4463 heterogeneous 4-expert MoE models found enumeration bias anchoring the space to AirNet and ranked ShuffleNet/MobileNetV3 as top performers.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2606.21645","ref_index":24,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"Behavioral and Representational Evidence of Binomial Ordering Preferences in Large Language Models","primary_cat":"cs.CL","submitted_at":"2026-06-19T17:56:44+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"LLMs recover dominant binomial orders from corpora but align less closely with exact preference distributions, with preference strength partially encoded in middle-to-late layers and manipulable via steering.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2606.21428","ref_index":4,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"Does Mixture-of-Experts Actually Help Inference on Consumer and Edge Hardware? An Empirical Study","primary_cat":"cs.PF","submitted_at":"2026-06-19T13:45:45+00:00","verdict":"ACCEPT","verdict_confidence":"MODERATE","novelty_score":4.0,"formal_verification":"none","one_line_summary":"Empirical benchmarks show MoE inference cost on edge hardware tracks total parameters rather than active parameters, with OLMoE-1B-7B behind dense baselines especially on the Jetson device.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2606.21228","ref_index":206,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"Sakana Fugu Technical Report","primary_cat":"cs.LG","submitted_at":"2026-06-19T08:47:40+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":5.0,"formal_verification":"none","one_line_summary":"Sakana Fugu trains LLM orchestrators using fine-tuning, evolutionary algorithms, and RL to build query-adaptive multi-agent scaffolds, claiming SOTA results on benchmarks including SWE-Bench Pro and GPQA-Diamond.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2606.20544","ref_index":5,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"Toward Calibrated Mixture-of-Experts Under Distribution Shift","primary_cat":"cs.AI","submitted_at":"2026-06-18T17:55:00+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":7.0,"formal_verification":"none","one_line_summary":"Expert calibration suffices for MoE calibration under distribution shifts in hard-routed models but not soft-routed ones; adversarial reweighting improves the accuracy-calibration tradeoff across models and shifts.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2606.19172","ref_index":14,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"User as Engram: Internalizing Per-User Memory as Local Parametric Edits","primary_cat":"cs.AI","submitted_at":"2026-06-17T15:15:19+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":7.0,"formal_verification":"none","one_line_summary":"User facts are internalized as surgical local edits to a hash-keyed Engram memory table with reasoning skill held in a shared adapter, claimed to match LoRA recall, improve indirect reasoning 5.6x on average, and compose across users with 33,000x smaller footprint than per-user adapters.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2606.15079","ref_index":81,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"Ling and Ring 2.6 Technical Report: Efficient and Instant Agentic Intelligence at Trillion-Parameter Scale","primary_cat":"cs.CL","submitted_at":"2026-06-13T03:21:49+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":4.0,"formal_verification":"none","one_line_summary":"Technical report announcing Ling-2.6 and Ring-2.6 models with hybrid linear attention, evolutionary CoT, and KPop RL for efficient agentic intelligence at scale.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2606.09038","ref_index":201,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"Personalization Meets Safety:Mechanisms,Risks,and Mitigations in Personalized LLMs","primary_cat":"cs.AI","submitted_at":"2026-06-08T05:10:05+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":5.0,"formal_verification":"none","one_line_summary":"A survey that maps safety risks in personalized LLMs, introduces a unified taxonomy, and highlights three structural inadequacies in existing research on user-invariant safety, isolated techniques, and short-term evaluations.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2606.08814","ref_index":5,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"STAR: Rethinking MoE Routing as Structure-Aware Subspace Learning","primary_cat":"cs.AI","submitted_at":"2026-06-07T20:07:24+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"STAR rethinks MoE routing as structure-aware subspace learning by adding a GHA-tracked principal subspace to standard routers, yielding more stable specialization and better performance on synthetic, language, and vision tasks.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2606.06294","ref_index":36,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"Towards One-to-Many Temporal Grounding","primary_cat":"cs.CV","submitted_at":"2026-06-04T15:31:22+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":7.0,"formal_verification":"none","one_line_summary":"Introduces OMTG benchmark with C-Acc and EtF1 metrics, a 56k dataset, and caption/temporal rewards, reaching 43.65% EtF1 SOTA on the new bench.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2606.04378","ref_index":43,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"DLLG: Dynamic Logit-Level Gating of LLM Experts","primary_cat":"cs.CL","submitted_at":"2026-06-03T02:51:56+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"DLLG learns token-level fusion weights for LLM experts from sparse response supervision and outperforms routing, ensembling, and merging baselines on reasoning and code tasks.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2606.04160","ref_index":23,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"Expert-Aware Refusal Steering","primary_cat":"cs.CL","submitted_at":"2026-06-02T19:23:46+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"Refusal steering works on MoE LLMs; expert-aware variants succeed with single-expert outputs and refusal signals differ from routing patterns.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2606.00609","ref_index":44,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"CARE-RL: Capability-Aware Reinforcement Learning for Mitigating Cross-Domain Conflicts","primary_cat":"cs.LG","submitted_at":"2026-05-30T08:18:40+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":5.0,"formal_verification":"none","one_line_summary":"CARE-RL combines PA-GRM for task-adaptive rewards on open-ended tasks and DACSP for modulating RL updates using historical capability directions, reporting higher total average scores than baselines on Qwen models.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2606.00539","ref_index":9,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"GNMR: Runtime Stability Control for Low-Precision Large Language Model Training","primary_cat":"cs.LG","submitted_at":"2026-05-30T05:11:13+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":5.0,"formal_verification":"none","one_line_summary":"GNMR is a gradient-norm-based controller that maps local stability signals to budgeted recovery actions to stabilize low-precision LLM training while preserving quality.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2606.00531","ref_index":5,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"State Machine Guided Multi-Relational Synthetic Data from Logs for Anomaly Detection","primary_cat":"cs.MA","submitted_at":"2026-05-30T04:49:32+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"A framework extracts a latent state machine from logs, induces a multi-table relational schema, and uses it as a generative prior to create synthetic data that augments real logs for better anomaly detection.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2605.31268","ref_index":14,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"Mellum2 Technical Report","primary_cat":"cs.CL","submitted_at":"2026-05-29T13:01:11+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":3.0,"formal_verification":"none","one_line_summary":"Mellum 2 is a 12B MoE model with 2.5B active parameters, trained on 10.6T tokens with MoE, GQA, SWA, and MTP, then post-trained into Instruct and Thinking variants, claimed competitive with 4B-14B models at 2.5B compute.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2605.31010","ref_index":5,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"MoG: Mixture of Experts for Graph-based Retrieval-Augmented Generation","primary_cat":"cs.CL","submitted_at":"2026-05-29T08:43:12+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"MoG uses hub graphs for shared context and sparsely activates expert graphs with a topology-aware router, reporting over 20% relative gains on MuSiQue.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2605.28207","ref_index":5,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"Pruning and Distilling Mixture-of-Experts into Dense Language Models","primary_cat":"cs.CL","submitted_at":"2026-05-27T09:27:36+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"A systematic MoE-to-dense conversion via expert scoring, grouping, and distillation yields +6.3 pp average accuracy over dense-to-dense pruning at matched parameter count on tested models.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2605.27849","ref_index":8,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"FPMoE: A Sparse Mixture-of-Experts Approach to Functional Code Generation","primary_cat":"cs.PL","submitted_at":"2026-05-27T02:06:11+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"FPMoE applies a sparse MoE architecture with per-language routed experts and a shared expert to improve LLM code generation on functional languages, outperforming fine-tuned baselines while matching larger models with 3B active parameters.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2605.27722","ref_index":6,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"NUCLEUS-MoE: Unified Model of Pool Boiling for Liquid Cooling","primary_cat":"cs.LG","submitted_at":"2026-05-26T21:49:16+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"NUCLEUS-MoE is a single neural network architecture that models saturated and subcooled pool boiling for dielectrics, refrigerants, and cryogens with generalization to new fluids.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2605.26717","ref_index":1,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"L2Rec: Towards Dual-View Understanding of LLMs for Personalized Recommendation","primary_cat":"cs.IR","submitted_at":"2026-05-26T08:57:19+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":7.0,"formal_verification":"none","one_line_summary":"L2Rec introduces dual-view personalized low-rank perturbations via DPMoE to let one LLM backbone produce complementary behavioral and semantic adaptations, with cross-view fusion, outperforming baselines on four datasets and in industrial A/B tests.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2605.24869","ref_index":5,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"Lngram: N-gram Conditional Memory in Latent Space","primary_cat":"cs.CL","submitted_at":"2026-05-24T04:59:11+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"Lngram is a latent N-gram conditional memory module that learns discrete symbols from hidden states for N-gram lookup, outperforming baselines in language modeling and multimodal tasks.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2605.23764","ref_index":4,"ref_count":2,"confidence":0.98,"is_internal_anchor":true,"paper_title":"HyperParallel-MoE: Multi-Core Interleaved Scheduling for Fast MoE Training on Ascend NPUs","primary_cat":"cs.DC","submitted_at":"2026-05-22T15:35:23+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"HyperParallel-MoE reduces Dispatch-to-Combine MoE-FFN latency by up to 1.58x on Ascend A3 clusters via tile-level heterogeneous scheduling that overlaps communication, matrix, and vector computation inside a single kernel launch.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2605.23294","ref_index":13,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"NASiC: 3D NAND-based CAM-Selected Multibit CIM Architecture for Efficient On-Device Mixture-of-Experts LLM Inference","primary_cat":"cs.AR","submitted_at":"2026-05-22T07:10:57+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"NASiC fuses CAM-based expert selection and multibit CIM computation in 3D NAND into one cycle for MoE LLM inference, claiming 4-114.8x performance and 3.9-70x energy efficiency gains over prior designs with high accuracy.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2605.23078","ref_index":4,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"GEMQ: Global Expert-Level Mixed-Precision Quantization for MoE LLMs","primary_cat":"cs.LG","submitted_at":"2026-05-21T22:23:38+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"GEMQ applies global LP-based expert importance estimation and router fine-tuning within progressive quantization to cut memory and speed inference in MoE LLMs with little accuracy loss.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2605.21427","ref_index":12,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"PALS: Power-Aware LLM Serving for Mixture-of-Experts Models","primary_cat":"cs.AI","submitted_at":"2026-05-20T17:19:20+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"PALS adds dynamic GPU power capping to LLM serving frameworks like vLLM, jointly tuning it with batch size via offline models and feedback control to improve energy efficiency up to 26.3% and cut QoS violations 4-7x on dense and MoE models.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2605.20982","ref_index":11,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"Diagnosing Overhead in Dispatch Operations: Cross-architecture Observatory","primary_cat":"cs.DC","submitted_at":"2026-05-20T10:14:00+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"DODOCO measurements show MoE routing imbalance is intrinsic to architecture and real text, not correctable by EP scaling or represented by mock tokens, forming two persistent Gini bands.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2605.20891","ref_index":5,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"HDMoE: A Hierarchical Decoupling-Fusion Mixture-of-Experts Framework for Multimodal Cancer Survival Prediction","primary_cat":"cs.CV","submitted_at":"2026-05-20T08:31:09+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"HDMoE uses hierarchical MoE and RFR modules to address redundant information and fine-grained intra/inter-modality relationships in multimodal cancer survival prediction, with positive results on private liver cancer and TCGA datasets.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2605.19762","ref_index":19,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"What Really Improves Mathematical Reasoning: Structured Reasoning Signals Beyond Pure Code","primary_cat":"cs.AI","submitted_at":"2026-05-19T12:37:01+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"Controlled experiments show structured reasoning traces and higher-density math-domain samples improve mathematical reasoning more than pure executable code, with internal routing patterns reflecting these data effects.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2605.15484","ref_index":41,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"When Does Sparse MoE Help in Vision? The Role of Backbone Compute Leverage in Sparse Routing","primary_cat":"cs.CV","submitted_at":"2026-05-15T00:01:11+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":5.0,"formal_verification":"none","one_line_summary":"Sparse MoE vision models show positive accuracy gaps only when routing a substantial compute fraction ρ and using k≥2 experts at large scale; batch-axis dispatch is identified as a key failure mode.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2605.15053","ref_index":63,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"TFGN: Task-Free, Replay-Free Continual Pre-Training Without Catastrophic Forgetting at LLM Scale","primary_cat":"cs.LG","submitted_at":"2026-05-14T16:46:26+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"TFGN is an architectural overlay for transformers enabling task-free, replay-free continual pre-training across heterogeneous domains at LLM scale with near-zero backward transfer and high gradient orthogonality.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2605.14200","ref_index":10,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"How to Scale Mixture-of-Experts: From muP to the Maximally Scale-Stable Parameterization","primary_cat":"cs.LG","submitted_at":"2026-05-13T23:32:00+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":7.0,"formal_verification":"none","one_line_summary":"The authors derive a Maximally Scale-Stable Parameterization (MSSP) for MoE models that achieves robust learning-rate transfer and monotonic performance gains with scale across co-scaling regimes of width, experts, and sparsity.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2605.13421","ref_index":173,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"Combining pre-trained models via localized model averaging","primary_cat":"stat.ME","submitted_at":"2026-05-13T12:16:04+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"Localized model averaging with covariate-dependent weights achieves asymptotic optimality and weight consistency for combining pre-trained models under a general loss framework.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2605.12476","ref_index":11,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"Routers Learn the Geometry of Their Experts: Geometric Coupling in Sparse Mixture-of-Experts","primary_cat":"cs.LG","submitted_at":"2026-05-12T17:55:02+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":7.0,"formal_verification":"none","one_line_summary":"Routers in SMoE models form geometric alignments with their experts through shared gradient directions, enabling effective specialization that auxiliary load-balancing losses tend to disrupt.","context_count":1,"top_context_role":"background","top_context_polarity":"background","context_text":"Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, volume 1, pages 281-297. University of California Press, 1967. [10] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need, 2017. URL https://arxiv.org/abs/1706. 03762. 10 [11] Damai Dai, Chengqi Deng, Chenggang Zhao, R. X. Xu, Huazuo Gao, Deli Chen, Jiashi Li, Wangding Zeng, Xingkai Yu, Y . Wu, Zhenda Xie, Y . K. Li, Panpan Huang, Fuli Luo, Chong Ruan, Zhifang Sui, and Wenfeng Liang. Deepseekmoe: Towards ultimate expert specialization in mixture-of-experts language models, 2024. URLhttps://arxiv.org/abs/2401.06066. [12] Albert Q."},{"citing_arxiv_id":"2605.11408","ref_index":46,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"MaskTab: Scalable Masked Tabular Pretraining with Scaling Laws and Distillation for Industrial Classification","primary_cat":"cs.LG","submitted_at":"2026-05-12T01:56:04+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":5.0,"formal_verification":"none","one_line_summary":"MaskTab is a masked pretraining method for industrial tabular data that delivers measurable gains in classification AUC and KS metrics while enabling effective distillation to smaller models.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2605.11333","ref_index":63,"ref_count":2,"confidence":0.98,"is_internal_anchor":true,"paper_title":"MLCommons Chakra: Advancing Performance Benchmarking and Co-design using Standardized Execution Traces","primary_cat":"cs.DC","submitted_at":"2026-05-11T23:38:10+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"Chakra introduces a standardized graph-based execution trace representation for distributed ML workloads along with supporting tools to enable benchmarking, analysis, generation, and co-design across simulators and hardware.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2605.11277","ref_index":11,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"Sieve: Dynamic Expert-Aware PIM Acceleration for Evolving Mixture-of-Experts Models","primary_cat":"cs.AR","submitted_at":"2026-05-11T22:00:39+00:00","verdict":"CONDITIONAL","verdict_confidence":"LOW","novelty_score":8.0,"formal_verification":"none","one_line_summary":"Sieve dynamically schedules MoE experts across GPU and PIM hardware to handle bimodal token distributions, achieving 1.3x to 1.6x gains in throughput and interactivity over static prior PIM systems on three large models.","context_count":1,"top_context_role":"background","top_context_polarity":"background","context_text":"computer architecture news44, 3 (2016), 367-379. [10] Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multi- modality, long context, and next generation agentic capabilities.arXiv preprint arXiv:2507.06261(2025). [11] Damai Dai, Chengqi Deng, Chenggang Zhao, R. X. Xu, Huazuo Gao, Deli Chen, Jiashi Li, Wangding Zeng, Xingkai Yu, Y. Wu, Zhenda Xie, Y. K. Li, Panpan Huang, Fuli Luo, Chong Ruan, Zhifang Sui, and Wenfeng Liang. 2024. DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models. arXiv:2401.06066 [cs.CL] https://arxiv.org/abs/2401."},{"citing_arxiv_id":"2605.10670","ref_index":1,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"Surviving Partial Rank Failures in Wide Expert-Parallel MoE Inference","primary_cat":"cs.DC","submitted_at":"2026-05-11T14:53:00+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":7.0,"formal_verification":"none","one_line_summary":"EEP makes wide expert-parallel MoE serving survive single-rank failures with an 11s recovery pause, 8s reintegration pause, and throughput restored to 95% of pre-fault level within 52s while staying within 4.4% of a fixed-membership baseline in steady state.","context_count":1,"top_context_role":"background","top_context_polarity":"background","context_text":"ing, fault tolerance, GPUDirect RDMA 1 Introduction Wide expert parallelism (EP) has become the standard strat- egy for serving large Mixture-of-Experts (MoE) models at ∗Correspond to: zhang_mingxing@mail.tsinghua.edu.cn production scale: dozens of GPUs cooperate on every decod- ing step, aggregating expert weights, memory capacity, and memory bandwidth within a single EP instance [1, 2, 4, 7, 26]. At the scale where EP is necessary, however, partial failures are routine. GPU, process, and network faults occur regularly in large clusters - in one production datacenter deployment, GPU and node failures accounted for over 78% of hardware fault events at a rate of 382 events per month [22]. Studies of large-scale ML clusters [11] further confirm that as job"},{"citing_arxiv_id":"2605.09516","ref_index":2,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"Mixture of Layers with Hybrid Attention","primary_cat":"cs.LG","submitted_at":"2026-05-10T12:53:28+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":7.0,"formal_verification":"none","one_line_summary":"Mixture of Layers replaces monolithic transformer blocks with routed thin parallel blocks using hybrid attention that combines a shared softmax block for global context with Gated DeltaNet linear attention in the routed blocks.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2605.11005","ref_index":4,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"DisagMoE: Computation-Communication overlapped MoE Training via Disaggregated AF-Pipe Parallelism","primary_cat":"cs.LG","submitted_at":"2026-05-10T05:57:05+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"DisagMoE achieves up to 1.8x faster MoE training by disaggregating attention and FFN layers into disjoint GPU groups with a multi-stage uni-directional pipeline and roofline-based bandwidth balancing.","context_count":1,"top_context_role":"dataset","top_context_polarity":"use_dataset","context_text":"activating only a small subset of experts for each to- ken, MoE architectures decouple activated parameter count from total parameter count, achieving sublin- ear growth of per-token computation while scaling model capacity [6, 16, 20, 26, 31]. DeepSeek-V4-Pro, for example, scales to 1.6T total parameters while activating only 49B parameters per token [4, 8]. Despite the compute efficiency, large MoE layers still face GPU memory capacity challenges due to the total parameter count. Expert parallelism (EP) ad- dresses the challenge by sharding experts across GPU devices, where dense components such as attention layers are replicated via data parallelism (DP) among the same GPU group. Two all-to-all communications"},{"citing_arxiv_id":"2605.08322","ref_index":11,"ref_count":2,"confidence":0.98,"is_internal_anchor":true,"paper_title":"SDG-MoE: Signed Debate Graph Mixture-of-Experts","primary_cat":"cs.LG","submitted_at":"2026-05-08T16:25:21+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":7.0,"formal_verification":"none","one_line_summary":"SDG-MoE introduces learned signed interaction graphs and disagreement-gated deliberation among experts in MoE architectures, yielding 19.8% better validation perplexity than the strongest baseline.","context_count":1,"top_context_role":"background","top_context_polarity":"background","context_text":"Shared states are projected into message vectors u(t) j =V h (t) j,shr. Each expert receives support and criticism messages m+,(t) i = X j∈S A+,(t) ij u(t) j , m −,(t) i = X j∈S A−,(t) ij u(t) j .(10) The update MLP sees the current shared state, the support message, and the signed contrast: ∆(t) i =U 2ϕ \u0010 U1[h (t) i,shr;m +,(t) i ;m +,(t) i −γm −,(t) i ] +b 1 \u0011 +b 2.(11) Although the update MLP could in principle learn an equivalent linear reparameterization from (m+ i , m− i ), the explicit contrast input biases the module toward treating the negative channel as counter-evidence rather than as a second unsigned message. The shared state is then updated with a gated residual step followed by anchoring to the initial shared opinion:"},{"citing_arxiv_id":"2605.08292","ref_index":3,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"Hierarchical Mixture-of-Experts with Two-Stage Optimization","primary_cat":"cs.LG","submitted_at":"2026-05-08T09:21:46+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"Hi-MoE uses two-level hierarchical routing objectives to enforce group-level balance while promoting within-group specialization, yielding better perplexity and expert utilization than prior MoE baselines in NLP and vision tasks.","context_count":1,"top_context_role":"background","top_context_polarity":"background","context_text":"toward uniformity, simultaneously regularizing which experts are selected and how confidently they are chosen. Subsequent works refine this basic idea rather than replacing it. Techniques such as logit normalization, temperature scaling, and per-layer or even per-token adaptive coefficients forLload make the balancing signal more robust and easier to tune in deep stacks of MoE layers [3, 6, 7, 30, 36, 38, 42]. These advances reduce sensitivity to hyperparameters and improve training stability across layers, tasks, and batch regimes. However, all loss-based approaches share a structural challenge: the router is optimized under a compound objective Ltask + Lload. As analyzed in Wang et al. [35], the gradients fromLload can conflict"},{"citing_arxiv_id":"2605.07363","ref_index":6,"ref_count":1,"confidence":0.9,"is_internal_anchor":true,"paper_title":"MISA: Mixture of Indexer Sparse Attention for Long-Context LLM Inference","primary_cat":"cs.LG","submitted_at":"2026-05-08T07:19:34+00:00","verdict":"CONDITIONAL","verdict_confidence":"MODERATE","novelty_score":7.0,"formal_verification":"none","one_line_summary":"MISA routes to a small subset of indexer heads via block statistics, matching full DSA performance on LongBench with 4-8x fewer heads and 3.82x speedup while recovering over 92% of selected tokens.","context_count":1,"top_context_role":"background","top_context_polarity":"background","context_text":"indexer itself, which can be combined orthogonally with any token-level scheme. Mixture of experts in language models.Conditional computation via mixture of experts (MoE) was introduced by Shazeer et al. [20] and scaled up in GShard [ 14] and Switch Transformer [ 9], where a learned router activates a sparse subset of expert FFNs per token. Modern open-weight LLMs such as Mixtral [12] and DeepSeek-MoE [6] make their FFN layers MoE-based while leaving the attention layers dense.Attention-sideMoE has also been explored: Mixture-of-Attention-Heads [ 29] treats whole multi-head attention modules as experts and routes one module per token; MoH [13] treats each individual attention head as an expert and selects a sparse subset of heads to compute the attention output."},{"citing_arxiv_id":"2605.07256","ref_index":8,"ref_count":1,"confidence":0.9,"is_internal_anchor":true,"paper_title":"TAS-LoRA: Transformer Architecture Search with Mixture-of-LoRA Experts","primary_cat":"cs.CV","submitted_at":"2026-05-08T05:24:51+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"TAS-LoRA attaches a mixture of LoRA experts to a supernet and uses a dynamic router plus group-wise initialization to let different architecture subnets learn distinct features, yielding higher accuracy than prior TAS methods on ImageNet and transfer datasets.","context_count":1,"top_context_role":"background","top_context_polarity":"background","context_text":"This dynamic routing mechanism allows each layer to select LoRA experts adaptively based on its architectural properties, enabling learning specialized features efficiently2. Group-wise router initialization.We have observed that, in the early stages of training, the router tends to assign similar weights to all LoRA experts, which limits expert specialization and causing redundancy. In the works of MoE [8, 14, 28, 41], this issue is typically addressed by in- troducing auxiliary losses, that encourage different instances within a batch to select different experts. This approach is however not applicable to TAS-LoRA, as our router pro- cesses a single subnet only at a time. We instead propose a group-wise router initialization method using architectural"},{"citing_arxiv_id":"2605.16349","ref_index":22,"ref_count":1,"confidence":0.98,"is_internal_anchor":true,"paper_title":"Geometric Asymmetry in MoE Specialization: Functional Decorrelation and Representational Overlap","primary_cat":"cs.LG","submitted_at":"2026-05-08T04:17:10+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":7.0,"formal_verification":"none","one_line_summary":"MoE experts in pretrained Transformers exhibit functional decorrelation with near-zero Jacobian alignment yet occupy partially overlapping representation subspaces, with routing sparsity modulating the geometry.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2605.06665","ref_index":7,"ref_count":1,"confidence":0.9,"is_internal_anchor":true,"paper_title":"UniPool: A Globally Shared Expert Pool for Mixture-of-Experts","primary_cat":"cs.LG","submitted_at":"2026-05-07T17:59:44+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"A shared global expert pool in MoE improves validation loss over per-layer experts and allows sublinear expert-parameter growth with depth.","context_count":1,"top_context_role":"background","top_context_polarity":"background","context_text":"The modern MoE paradigm for language models was established by sparsely gated expert layers [ 43], then scaled through top-1 routing in Switch Transformer [ 11], expert-parallel distributed training in GShard [28], and stability improvements such as ST-MoE's router z-loss [66]. Recent large-scale systems including Mixtral [23] and the DeepSeek series [7, 8, 9] further show that sparse expert capacity is an effective way to scale language models. Complementary work studies expert granularity and scaling laws, finding that a larger number of smaller experts can improve performance when paired with appropriate routing [25], with extreme variants considering up to a million experts [17]. These works largely retain per-layer expert ownership; UNIPOOLinstead"},{"citing_arxiv_id":"2605.06415","ref_index":17,"ref_count":2,"confidence":0.98,"is_internal_anchor":true,"paper_title":"E = T*H/(O+B): A Dimensionless Control Parameter for Mixture-of-Experts Ecology","primary_cat":"cs.LG","submitted_at":"2026-05-07T15:23:35+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":5.0,"formal_verification":"none","one_line_summary":"Experiments across vision and language tasks indicate that the control parameter E = T*H/(O+B) >= 0.5 ensures no dead experts in Mixture-of-Experts models.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null}],"limit":50,"offset":0}