{"id":"59c528a8-8b9c-4c37-ad36-4075d6359ffe","arxiv_id":"2504.18598","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"BadMoE implants backdoors into dormant experts of MoE LLMs and uses routing-trigger optimization to activate them, achieving high attack success while preserving normal accuracy.","lead":"This paper presents BadMoE, a backdoor attack on mixture-of-experts (MoE) language models that hides malicious behavior in rarely used \"dormant experts\" and activates them with optimized trigger phrases. It shows the attack succeeds on several open-source MoE models and evades common defenses, highlighting a new security risk for MoE-based AI models.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Stage 3 implementation text contradicts the dormant-expert mechanism: Eq. 16 trains only selected experts S_a, but §7.1 says all methods (including BadMoE) use LoRA on attention layers; if the latter is literal, the central MoE-specific claim is untested.","rationale":"The reader's weakest assumption matches my read. I do not see a separate objection that would overturn the paper if the implementation ambiguity is resolved in BadMoE's favor. The theoretical 'dominating expert' proof is not load-bearing for the empirical attack: it assumes unbounded expert weights, so it establishes existence only in an idealized limit, but the experimental claims could still hold. The implementation contradiction, however, is directly load-bearing: unless Stage 3 demonstrably updates S_a, the reported ASR/defense results are compatible with a standard attention-layer backdoor plus an optimized trigger, and the paper's headline contribution (infecting dormant experts) is unsupported. Since the reader already conditioned acceptance on resolving this, my recommendation is unchanged. A released training script or a checkpoint-diff showing that S_a weights changed would settle it; absent that, the conditional verdict should stand.","tokens_in":27432,"tokens_out":4543,"duration_ms":45353,"concrete_test":"Obtain or reconstruct the exact Stage 3 training script/config and inspect the LoRA target_modules: are adapters attached to the selected expert FFN weights (e.g., expert gate/up/down projections in the attacked MoE layer) or to attention projections? If they are attached to attention only, rerun the SST-2 and AGNews experiments on Mixtral and OLMoE (same seeds, poison rate 1%, N_a=2) with LoRA adapters attached only to the selected experts S_a's FFN weights and all other parameters frozen. The concern is settled if (1) ASR and CA match the paper within a few points, and (2) routing scores on S_a for triggered inputs increase substantially versus clean inputs; failure on either point shows the reported mechanism was not what produced the results.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing concern is an internal contradiction about what Stage 3 actually trains. Section 6.3 (Eq. 16) states that all experts except the selected dormant set S_a are frozen and only S_a's parameters are updated; this is the mechanism that makes the attack MoE-specific. However, Section 7.1 Implementation Details says that for all baselines LoRA targets the attention layers (non-expert parameters), deliberately avoiding router and expert parameters, and then says BadMoE 'follows identical training settings as the baselines for a fair comparison.' If the LoRA adapters in the BadMoE experiments were attached to attention projections rather than to the selected experts' FFN weights, then no expert was actually infected; the reported >95% ASR and defense results would only demonstrate an attention-layer backdoor with an optimized routing trigger. The central claim of poisoning dormant experts and making them dominate would be untested. The contradiction is not resolved by the ablation text; Table 3's wording 'keeping the infected experts unchanged' presupposes infection rather than establishing it. Section 8.1's t-SNE analysis is also consistent with attention-layer feature drift, so it does not independently verify expert modification.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes BadMoE, a three-stage backdoor attack against Mixture-of-Experts LLMs. Stage 1 selects the least-used 'dormant' experts from routing statistics; Stage 2 optimizes a short discrete trigger using a routing-aware loss with a perplexity constraint; Stage 3 fine-tunes the selected experts on poisoned data so that triggered inputs are routed to them and their outputs dominate. The paper also presents a theorem on 'dominating experts' and evaluates the attack on Mixtral-8x7B, OLMoE-1B-7B, and Deepseek-moe-16B on six datasets, reporting ASR above 95% in most settings, preserved clean accuracy, stealthiness relative to word or sentence triggers, and resistance to ONION, fine-tuning, and fine-pruning. The authors commit to releasing code and data.","tokens_in":27702,"tokens_out":9001,"duration_ms":91520,"significance":"If the results hold, BadMoE would identify a meaningful new attack surface for MoE architectures, and the combination of routing-trigger optimization with dormant-expert selection is a plausible and interesting mechanism. The evaluation is broad and includes useful ablations, transferability tests, stealth metrics, and an attempted defense analysis, and the paper is generally clearly structured. On the other hand, the verification that the reported results actually exercise the proposed MoE-specific mechanism is missing: the implementation section appears to contradict the method section on which parameters are updated, and the theoretical argument does not cover the bounded-update regime used by the attack. These issues must be resolved before the central claims can be accepted.","major_comments":[{"comment":"Section 7.1 directly contradicts Section 6.3 on what Stage 3 trains. Section 6.3, Eq. (16), says that all experts except the selected dormant set S_a are frozen and only S_a's parameters are updated, which is the mechanism that makes BadMoE MoE-specific. Section 7.1, by contrast, says that for all baselines LoRA targets the attention layers (non-expert parameters), that this 'intentionally avoids modifying router and expert parameters,' and that BadMoE 'follows identical training settings as the baselines for a fair comparison.' If the BadMoE runs used LoRA adapters on attention projections, then no expert was infected, and the reported ASR and defense results demonstrate an attention-layer backdoor with an optimized trigger rather than the dormant-expert domination claimed in the abstract and Section 6. The Table 3 phrase 'keeping the infected experts unchanged' and the t-SNE analysis in Section 8.1 presuppose infection rather than verifying it, since an attention-layer trigger backdoor would also shift hidden states. Please state explicitly which parameter groups the BadMoE LoRA adapters modified; if the experiments were attention-only, rerun the attack with parameter-efficient or full fine-tuning restricted to the selected experts and report whether the ASR and defense results persist.","section":"§7.1 (Implementation Details) vs. §6.3 (Eq. 16)"},{"comment":"The formal result is an idealized existence proof that does not cover the attack's operating regime. The proof requires ||w1||2 to diverge while ||w2||2 stays bounded, whereas the attack in Section 6.3 updates selected experts with LoRA, which keeps parameter changes small and bounded; the theorem therefore shows domination only under conditions the attack never realizes. Relatedly, the argument models each expert as a single weight vector and computes KL divergence between scalar Gaussian projections, which is far from the vector-valued MoE output used in the actual model. Please present the theorem explicitly as idealized motivation and add direct empirical evidence of domination under bounded LoRA updates, for example the contribution of S_a to the MoE layer output before and after training with and without the trigger, or weaken the corresponding claims in the abstract and Section 5.","section":"§5, Eqs. (3)–(9)"},{"comment":"The clean-utility comparison is confounded by fine-tuning. The 'Clean' row in Table 2 is the unmodified pre-trained checkpoint, whereas every attack row includes LoRA fine-tuning on the target task; the statement that backdoored models 'significantly outperform the clean models' in Section 7.2 therefore reflects task adaptation rather than utility preservation. A proper utility baseline is an equally trained benign model, as in the 'Fine-tuning (No attack)' row of Table 3, and this baseline should appear in the main comparison for every model and dataset before the paper claims that utility is preserved. The same confound should be removed in the unrelated-task discussion if its 'Clean' rows are not fine-tuned on the target task.","section":"§7.2, Table 2; §7.3, Table 3"},{"comment":"The defense evaluation is conducted only on OLMoE, as stated in the Table 7 caption, but the abstract and Section 1 assert without qualification that existing defenses are ineffective against BadMoE. Please either extend the defense evaluation to Mixtral and Deepseek or explicitly scope the defense claims to the single model tested. In addition, the fine-tune defense is described only as using the whole clean training dataset, with no learning rate or number of epochs, and the fine-pruning threshold is not specified; these details are needed to reproduce the robustness result.","section":"§8.5, Table 7"}],"minor_comments":[{"comment":"The formula and the surrounding text disagree: Eq. (12) assigns v_i = 1 for selected dormant experts, but the text immediately after says the indices of selected experts are set to 0 and all others are set to 1. Please reconcile, since the routing-aware loss in Eq. (13) depends on this encoding.","section":"§6.1, Eq. (12)"},{"comment":"The optimization is written as an argmin over all θ even though the text says only θ_e is updated; please make the optimization variable θ_e explicit to avoid confusion with the implementation discussion in Section 7.1.","section":"§6.3, Eq. (16)"},{"comment":"There are several typos: 'build' should be 'built' in the DeepSeek-R1 sentence in Section 1, 'bone' in the Table 2 caption should be 'bold', 'parameter-effective' in Section 7.1 should be 'parameter-efficient', and 'the the stealthiness' in Section 8.4 should be 'the stealthiness'.","section":"Throughout"},{"comment":"The stealthiness metrics are reported as single numbers without variance or significance tests; some differences are small, for example the SST2 grammar-error rate of 11.80 for InSent versus 11.83 for BadMoE, so confidence intervals or additional runs would make the comparisons more convincing.","section":"§8.4, Table 6"}],"recommendation":"major_revision","confidential_remarks":"The key risk is the Stage 3 implementation ambiguity. If the experiments were actually attention-only LoRA, the paper needs substantial new experiments before it can claim an MoE-specific attack; if the authors can confirm expert-targeted updates, the other issues are addressable by rewriting. I would also ask the editor to have the authors calibrate the 'first backdoor attack against MoE LLMs' novelty claim against concurrent work during revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nRead BadMoE (arXiv:2504.18598). The headline: it is the first backdoor attack designed specifically for MoE LLMs, and the mechanism—pick dormant experts, optimize a routing trigger to activate them, then train those experts to dominate—is genuinely new relative to the cited literature. The empirical work is extensive: three models (Mixtral, OLMoE, DeepSeek), six tasks, ablations, and defense evals. The attack reportedly gets >95% ASR while preserving clean accuracy and evading ONION, fine-tuning, and fine-pruning. That is an important practical result for AI security if it holds.\n\nWhat the paper does well: the routing-aware trigger optimization with a PPL constraint is a solid idea, and the ablation of trigger type and the comparison to BadFFN help isolate the contribution. The defense analysis is honest about residual ASR rather than assuming the defenses work.\n\nThe soft spot is load-bearing. Section 6.3 says Stage 3 freezes all experts except the selected dormant set S_a and updates only S_a's parameters (Eq. 16). Section 7.1 says that for all baselines, LoRA is applied to attention layers only, deliberately avoiding expert and router parameters, and then says BadMoE 'follows identical training settings as the baselines.' If that sentence is literal, the experiments never actually trained any expert; they trained attention adapters and only used the optimized trigger for routing. That would reduce the attack to an attention-layer backdoor with a routing-tuned trigger, leaving the central MoE-specific claim untested. The paper does not resolve this. Table 3's 'keeping the infected experts unchanged' presupposes infection, and the t-SNE analysis is consistent with attention drift. No code is released, so I cannot check.\n\nOther issues are minor by comparison. The theoretical proof of 'dominating experts' assumes linear experts, Gaussian activations, and unbounded weights; it is a boundedness argument, not a deep theorem, and the extension to multiple dominators is hand-waved. The fine-tuning defense uses the full clean training set, which is a strong defense setting—good for the attacker, but not the toughest test.\n\nWho is this for: anyone working on LLM supply-chain security or MoE robustness. The paper should be sent to serious peer review, conditional on the authors clarifying exactly which parameters were trained in Stage 3 and, ideally, releasing code. If the contradiction is just bad writing, the result is solid. If it is real, the paper needs a redesign and re-evaluation. I would bring it to reading group either way.","headline":"First MoE-specific backdoor attack with broad empirical coverage, but the central claim about infecting dormant experts is undermined by an internal contradiction in the implementation details.","tokens_in":28267,"tokens_out":4056,"would_cite":true,"duration_ms":38827,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"BadMoE shows that an attacker with white-box access to a Mixture-of-Experts LLM can implant a backdoor in a few underused experts and activate it with a short optimized trigger, forcing malicious outputs while preserving normal accuracy","keywords":["Mixture-of-Experts","backdoor attack","dormant experts","dominating experts","routing trigger optimization","AI security","data poisoning","LLM supply chain"],"falsifier":"Inspect the Stage-3 checkpoint of any reported BadMoE run: if the only LoRA adapters present sit on attention projections and no parameters of the selected dormant set $S_a$ changed, the mechanism claim is unsubstantiated. Then rerun the attack with attention layers frozen and only the two chosen dormant experts updatable; the central claim is settled by whether ASR stays above 95% under that constraint.","tokens_in":27227,"feed_emoji":"🚪","tokens_out":12739,"duration_ms":109806,"temperature":0.7,"pith_summary":"This paper tries to establish that Mixture-of-Experts LLMs have a new, architecture-specific backdoor vulnerability: an attacker who can fine-tune a downloaded model can poison a handful of experts that are rarely used on the target task, then learn a short trigger that routes prompted inputs to those experts and makes them dominate the layer's output. The paper proves that a single expert with a sufficiently large output weight can make the whole MoE layer's output arbitrarily close to its own output, and it argues that the least-used dormant experts are safe places to hide this capability. BadMoE is then presented as a three-stage attack: profile expert usage, optimize a routing-aware trigger, and fine-tune only the selected dormant experts on poisoned data. Experiments on three open-source MoE LLMs and six classification and generation tasks report attack success rates above 95%, often 100%, with clean accuracy near or above clean fine-tuned models, and the attack retains high success against ONION, fine-tuning, and fine-pruning defenses. If the results hold, MoE deployment practices inherit a stealthy supply-chain risk that usage-based and pruning-based defenses do not currently catch.","feed_headline":"New attack wakes idle experts to backdoor MoE LLMs at over 95% ASR","feed_subtitle":"Poisoning two underused experts keeps clean accuracy high and slips past ONION, fine-tuning, and fine-pruning.","key_machinery":"The load-bearing object is the dominating expert: in an MoE layer with routing scores $\\alpha_i$ and expert outputs $E_i(q)$, an expert $E_1$ dominates when $D_{KL}(MoE(q), \\alpha_1 E_1(q)) < \\epsilon$, meaning the full layer output is nearly indistinguishable from that one expert's scaled output. The paper proves such experts exist when hidden states are approximately Gaussian and one expert has a sufficiently large weight norm, and it argues that tuned dormant experts can be promoted into this role. The attack is then carried by three mechanisms: expert-usage profiling to select dormant experts (routing-score frequency $r_i$), a routing-aware cross-entropy loss with a perplexity constraint to optimize trigger tokens, and parameter-isolated fine-tuning that updates only the selected experts while freezing all other parameters of the attacked layer.","core_discovery":"BadMoE's central claim is that the dormant experts of an MoE LLM form a hidden compartment that can be weaponized. The attack first measures each expert's usage on clean task data, selects the least-used experts, and builds a binary routing target vector. It then optimizes a two-token trigger by minimizing the cross-entropy between the router's distribution on the trigger tokens and that target vector, with a perplexity penalty to keep the trigger innocuous. Finally, the attacker fine-tunes the model on poisoned data while freezing everything except the selected dormant experts, so those experts learn to drive the model's predictions to the adversary's target label whenever the trigger appears. The supporting theory shows that a few dominating experts, whose output alone suffices to make the MoE layer's output nearly identical in KL divergence, always exist, and that dormant experts can be promoted into that role; this is what lets fewer than 2% of the model's experts carry the full backdoor.","pith_inferences":["If token-to-expert routing is indeed stable and mostly token-ID driven, the same optimized trigger should transfer across tasks and even across fine-tuned copies of the same base model, making the backdoor persist through downstream adaptation; the paper only tests domain transfer, not this broader persistence.","The proof and mechanism suggest the dormant-component attack may extend to any sparse conditional computation whose routing is stable, such as vision mixture models or retrieval routers, not only decoder-only language models; that is an untested extrapolation.","The evasion of fine-pruning may partly reflect the pruning criterion: pruning units by average activation will rarely touch experts that stay dormant on clean inputs, so a pruning rule based on output-norm contribution to the final prediction might succeed where fine-pruning fails; this is a countermeasure the paper does not evaluate.","The hidden-state separability defense the paper sketches works only at high poisoning rates (above 70%), which suggests the attack can be detected by distributional tests only when the attacker is careless; testing expert-usage or output-norm monitoring at the actual 1% poisoning rate is a concrete next step."],"forward_implications":["On all three tested open-source MoE LLMs, BadMoE reaches at least 95% ASR on most of the six classification and generation tasks, with several at 100%, while clean accuracy stays within about one point of clean fine-tuned models.","Dormant-expert selection and trigger optimization both matter: choosing experts at random degrades clean accuracy on the 64-expert model, and replacing the optimized trigger with a rare word such as 'tq' drops ASR by 4 to 10 points.","The backdoor survives input-side and parameter-side defenses: ONION leaves residual ASR above 97% on SST-2 and above 80% on AGNews, and fine-tuning or fine-pruning leaves the attack near 100% ASR.","The attack withstands surface variation: switching prompt format or verbalizer costs less than 2% ASR where existing baselines drop 25 to 89 points, and transferring from SST-2 to IMDB loses less than 2% ASR.","Poisoning all experts in the attacked layer is strictly worse than poisoning two: the all-experts variant either collapses clean accuracy on one architecture or fails to reach high ASR on another, so the selectivity of the dormant-expert strategy is doing real work."],"supporting_citations":[{"why":"Supplies one of the three victim MoE architectures (8 experts, top-2 routing) on which the attack is evaluated.","marker":"[33]"},{"why":"Supplies the fully open 64-expert MoE model used for most ablations, robustness, and defense evaluations.","marker":"[47]"},{"why":"Supplies the third victim architecture with fine-grained and shared experts, testing the attack on a different MoE design.","marker":"[15]"},{"why":"Defines the classical BadNet trigger-insertion baseline and the ASR/CA evaluation convention the paper adopts.","marker":"[22]"},{"why":"Defines the layerwise weight-poisoning baseline LWP and the threat model of releasing a poisoned pre-trained model.","marker":"[36]"},{"why":"Supplies the model-editing backdoor formulation and the fine-tuning defense evaluated against BadMoE in Section 8.5.","marker":"[39]"},{"why":"Supplies the ONION perplexity-based input filter used as the data-level defense in the defense evaluation.","marker":"[51]"},{"why":"Supplies the fine-pruning defense used as the model-level defense in the defense evaluation.","marker":"[43]"},{"why":"Provides the gradient-based discrete token search that the routing-aware trigger optimizer (Algorithm 1) is motivated by.","marker":"[87]"},{"why":"Provides the empirical Gaussianity of LLM hidden states on which the dominating-expert existence proof rests.","marker":"[31]"}],"fun_headline_variants":["Poisoned dormant experts let attackers hijack MoE LLMs","Backdooring MoE LLMs via underused experts and crafted triggers","Routing triggers turn idle experts into LLM backdoors","MoE backdoor attack exploits dormant experts to control output","Hidden experts trigger backdoors in mixture-of-experts LLMs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central claim depends on the fine-tuning stage actually updating the selected dormant expert parameters, yet the implementation section says all methods, BadMoE included, used LoRA on attention layers; if only attention parameters moved, the reported results would not test the dormant-expert mechanism and the attack would reduce to a standard attention-layer backdoor with an optimized trigger.","fun_headline_variants_meta":{"raw":{"variants":["Poisoned dormant experts let attackers hijack MoE LLMs","Backdooring MoE LLMs via underused experts and crafted triggers","Routing triggers turn idle experts into LLM backdoors","MoE backdoor attack exploits dormant experts to control output","Hidden experts trigger backdoors in mixture-of-experts LLMs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000514,"raw_usage":{"total_tokens":2536,"prompt_tokens":1027,"completion_tokens":1509,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":643,"completion_tokens_details":{"reasoning_tokens":1423}},"tokens_in":643,"tokens_out":1509,"duration_ms":9769,"temperature":1.0,"reasoning_tokens":1423,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T10:33:17.146827+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Inspect the Stage-3 checkpoint of any reported BadMoE run: if the only LoRA adapters present sit on attention projections and no parameters of the selected dormant set $S_a$ changed, the mechanism claim is unsubstantiated. Then rerun the attack with attention layers frozen and only the two chosen dormant experts updatable; the central claim is settled by whether ASR stays above 95% under that constraint.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the model-editing backdoor formulation and the fine-tuning defense evaluated against BadMoE in Section 8.5."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the empirical Gaussianity of LLM hidden states on which the dominating-expert existence proof rests."}],"review_version":1}