MoEITS is an information-theoretic algorithm for pruning experts in MoE-LLMs that produces models with higher accuracy and greater size reduction than prior state-of-the-art methods on Mixtral 8x7B, Qwen1.5-2.7B, and DeepSeek-V2-Lite.
Jacobs, Michael I
9 Pith papers cite this work. Polarity classification is still indexing.
9
Pith papers citing it
representative citing papers
HI-MoE introduces hierarchical scene-then-instance routing in a Mixture-of-Experts detector, yielding gains over dense DINO and flat MoE variants on COCO, especially for small objects.
ICEdit achieves state-of-the-art instructional image editing in Diffusion Transformers via in-context generation, requiring only 0.1% of prior training data and 1% trainable parameters.
A proportional weight-update rule creates implicit binary evaluation signals that propagate losslessly through hierarchical selectors while preserving algebraic market integrity and admitting unique interior equilibria.
The Transformer is recovered exactly as the forward Euler step of spherical SVFlow, with multi-head attention and MoE/FFN as approximations to its vector field.
LingBot-Video is an open-source MoE video foundation model for embodied intelligence that scales to 120B parameters, integrates robot data, and uses multi-dimensional RL to improve physical plausibility.
ConceptM³oE is a concept-bottleneck multimodal MoE that matches unconstrained models on brain-tumor classification while producing neuropathologist-validated reasoning traces and improving macro-F1 from 56.41% to 66.70% in small-data regimes.
NBSR frames neural inference as conjugate Dirichlet evidence accumulation over a DAG with Gumbel-Softmax routing, yielding monotonic precision growth and uncertainty-aware early exit.
A low-rank mixture of experts model trained on handwriting data delivers strong Alzheimer's diagnosis performance with substantially reduced parameter activation during inference.
citing papers explorer
-
MoEITS: A Green AI approach for simplifying MoE-LLMs
MoEITS is an information-theoretic algorithm for pruning experts in MoE-LLMs that produces models with higher accuracy and greater size reduction than prior state-of-the-art methods on Mixtral 8x7B, Qwen1.5-2.7B, and DeepSeek-V2-Lite.
-
HI-MoE: Hierarchical Instance-Conditioned Mixture-of-Experts for Object Detection
HI-MoE introduces hierarchical scene-then-instance routing in a Mixture-of-Experts detector, yielding gains over dense DINO and flat MoE variants on COCO, especially for small objects.
-
In-Context Edit: Enabling Instructional Image Editing with In-Context Generation in Large Scale Diffusion Transformer
ICEdit achieves state-of-the-art instructional image editing in Diffusion Transformers via in-context generation, requiring only 0.1% of prior training data and 1% trainable parameters.
-
Implicit Evaluation Under Minimal Information: Price Formation in Hierarchical Component Selection
A proportional weight-update rule creates implicit binary evaluation signals that propagate losslessly through hierarchical selectors while preserving algebraic market integrity and admitting unique interior equilibria.
-
Transformer as an Euler Discretization of Score-based Variational Flow
The Transformer is recovered exactly as the forward Euler step of spherical SVFlow, with multi-head attention and MoE/FFN as approximations to its vector field.
-
Scaling Mixture-of-Experts Video Pretraining for Embodied Intelligence
LingBot-Video is an open-source MoE video foundation model for embodied intelligence that scales to 120B parameters, integrates robot data, and uses multi-dimensional RL to improve physical plausibility.
-
ConceptM$^3$oE: Concept-Guided Multimodal Mixture of Experts for Interpretable Computational Pathology
ConceptM³oE is a concept-bottleneck multimodal MoE that matches unconstrained models on brain-tumor classification while producing neuropathologist-validated reasoning traces and improving macro-F1 from 56.41% to 66.70% in small-data regimes.
-
Neural Bayesian Sequential Routing
NBSR frames neural inference as conjugate Dirichlet evidence accumulation over a DAG with Gumbel-Softmax routing, yielding monotonic precision growth and uncertainty-aware early exit.
-
Efficient Handwriting-Based Alzheimer,s Disease Diagnosis Using a Low-Rank Mixture of Experts Deep Learning Framework
A low-rank mixture of experts model trained on handwriting data delivers strong Alzheimer's diagnosis performance with substantially reduced parameter activation during inference.