{"id":"2de20d61-c2c2-4279-aa1a-17fb7f4d77f9","arxiv_id":"2412.04139","paper_version":4,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":8.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Monet composes experts from shared sub-layers to scale to 262,144 experts per layer with parameter growth proportional to the square root of the expert count, yielding more monosemantic and manipulable LLMs.","lead":"Monet is a new transformer design that splits each feedforward layer into hundreds of thousands of tiny, specialized \"experts\" while keeping memory use low. It aims to make large language models more interpretable and easier to control, by letting researchers remove or steer specific knowledge, such as toxicity or a programming language, after training.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"In-sample expert selection inflates reported domain, language, and toxicity manipulation; held-out identification/evaluation split is needed to support the central manipulation claim.","rationale":"The paper's architecture contribution is a product-key composition that reduces expert parameter growth to O(sqrt N); this is well-defined and supported by the complexity derivations in §A.2 and the implementation in Algorithms 1-2. The strongest claim, however, extends to knowledge manipulation without performance loss. The reader correctly identifies that manipulation is evaluated in-sample: experts are selected based on routing behavior on the same benchmarks used to measure the effect. This is not a mere technicality; it means the reported target-domain drops (e.g., ΔTarget -4.7 for Biology in Table 11) could reflect selection on the evaluation data rather than genuine monosemantic structure. A held-out split would settle this. We considered alternative concerns—e.g., whether the composed experts are truly independent given shared bottom/top weights, and whether qualitative monosemanticity is representative—but the in-sample selection is the most load-bearing because it directly undermines the advertised manipulation capability. The qualitative examples on C4/StarCoder provide some evidence of specialization, but they are illustrative, not quantitative, and the paper's own limitations acknowledge the lack of quantitative interpretability evaluation. Therefore we recommend keeping the conditional verdict: the architecture appears sound, but the manipulation claims require out-of-sample validation before full acceptance.","tokens_in":49779,"tokens_out":6178,"duration_ms":63822,"concrete_test":"Split each benchmark into two disjoint halves (stratified by domain/language or by prompt). Use the first half to identify specialized experts: routing skewness ≥2 for domains/languages, Pearson correlation threshold for toxicity. Delete the identified experts and evaluate on the second half (or on a held-out subset of RealToxicityPrompts for toxicity, with the 8 general benchmarks as an out-of-sample control). If the target-domain/language/toxicity drop is of comparable magnitude and cross-domain performance is preserved, the manipulation claim is supported. If the drop largely disappears, the original results were in-sample artifacts. For MMLU, a simple check is to identify experts on half the questions per domain and evaluate on the other half; for RealToxicityPrompts, use one split for correlation and the other for toxicity evaluation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In the manipulation experiments, the same benchmark instances are used for expert identification and for evaluating the effect of deletion. In §5.1, routing probabilities are computed on MMLU to assign each expert to a domain (Appendix D.1), and the same MMLU questions are then used to measure target-domain accuracy drop and cross-domain preservation. Thus an expert is selected precisely because it activates on those evaluation questions; deleting it naturally lowers their accuracy, so the reported ΔTarget is an in-sample artifact. The same protocol is used for MULTIPL-E in §5.2 and for RealToxicityPrompts/ToxiGen in §5.3, where toxic experts are identified by correlation with toxicity scores on the same prompts used for evaluation. The out-of-sample component (helpfulness on 8 benchmarks, Table 16) shows only mild degradation, but the central claim of precise, disentangled knowledge manipulation rests on target-domain drops that are not demonstrated out of sample. The qualitative monosemanticity examples (Figure 2) are collected on C4/StarCoder and are separate, but they are qualitative only and do not quantify mutual exclusivity. The load-bearing premise that routing-score skewness identifies domain-specific or toxicity-specific knowledge therefore remains untested in an out-of-sample setting.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Monet, a Mixture-of-Experts transformer variant in which each of N composed experts is formed as a product-key combination of shared bottom/top (horizontal decomposition) or left/right (vertical decomposition) weight matrices, allowing N = 262,144 experts per layer with expert-parameter growth O(√N). The authors pretrain models at 850M, 1.4B, and 4.1B scale plus a code model and a chat model, evaluate general-domain benchmarks, present qualitative routing examples, and report domain, language, and toxicity manipulation by deleting experts identified through routing statistics. The parameter-complexity derivations (Eq. 12, Eqs. 30-31) are self-consistent, and the paper releases source code and pretrained checkpoints.","tokens_in":50022,"tokens_out":8094,"duration_ms":90232,"significance":"If the manipulation claims survive held-out evaluation, Monet would be a meaningful contribution to interpretable-by-construction MoE architectures: the √N expert-parameter scaling is a clean and useful alternative to PEER-like linear scaling, and the qualitative specialization examples (e.g., Figure 2) are striking. The architectural core is credible and well positioned against existing MoE and SAE baselines. However, the quantitative evidence for precise knowledge manipulation currently rests on expert identification and evaluation on the same benchmark instances, and the monosemanticity claim is not quantified. These issues are fixable but are load-bearing for the paper's headline interpretability and controllability claims.","major_comments":[{"comment":"The manipulation experiments identify the experts to delete on the same benchmark on which the deletion effect is measured. For domain masking, Appendix D.1 assigns experts to MMLU domains using routing probabilities computed on the reorganized MMLU questions, and the same MMLU benchmark is then used to measure target-domain accuracy; the reported ΔTarget in Figure 3 and Tables 11-14 therefore conflates in-sample selection with genuine unlearning. An expert is selected precisely because it activates on the evaluation examples, so removing it lowers their accuracy even if it is not a general domain-knowledge store. The same protocol is used in §5.2/D.2 for MULTIPL-E and in §5.3/D.3 for RealToxicityPrompts/ToxiGen, where Pearson correlations are computed on the same prompts as the evaluation. The out-of-sample helpfulness results (Table 16) show only mild degradation, but the central claim of precise, disentangled manipulation depends on target-domain drops that are not demonstrated out of sample. Please re-run the experiments with a strict held-out split—for example, identifying experts on one half of the benchmark and evaluating on the other half, with folds—and report whether the ΔTarget/ΔOthers contrast survives.","section":"§5.1, §5.2, §5.3; Appendices D.1-D.3"},{"comment":"The paper does not specify how an individual composed expert (i,j) is 'deleted'. Because E_ij is built from shared U_i and V_j, zeroing a router component (g1_i or g2_j) removes an entire row or column of the Cartesian product, while masking a single (i,j) pair cannot be done inside the factorized computation of Eq. (12) without computing that pair's contribution explicitly. With reported masking ratios as high as 14.4% of experts (Table 4), this distinction matters both for the semantics of 'expert removal' and for the claimed O(√N) computation benefit. Please state the exact masking operation (component-level vs. pair-level) and, if pair-level, provide the implementation and its complexity.","section":"§3, Eqs. (8)-(12); §5"},{"comment":"The claim of 'mutual exclusivity of knowledge across experts' is supported only by qualitative routing examples and by the manipulation experiments, whose in-sample status is noted above. The paper's own Limitations section acknowledges that the self-explained expert descriptions are 'demonstrated only qualitatively' and that quantitative evaluation of automated interpretability remains open. Please add a quantitative selectivity or exclusivity metric—for example, overlap of top-activated token sets across experts, or routing-score skewness on a held-out corpus—and report it for the full expert population rather than for hand-picked examples.","section":"§4.3, Fig. 2; §6 Limitations"}],"minor_comments":[{"comment":"The statements that 'total parameters scale proportionally to the square root of the number of experts' are imprecise; Eqs. (30)-(31) establish O(√N md) for expert parameters only. Please rephrase to 'expert parameters' or 'the expert parameter contribution' to avoid overclaiming.","section":"Abstract; §1"},{"comment":"The heatmaps and the dense 14×15 numeric tables are difficult to read at print size; consider reporting only ΔTarget and ΔOthers with a clear color scale and confidence intervals.","section":"Figure 3; Tables 11-14"},{"comment":"The toxicity identification procedure uses Pearson correlation between routing scores and toxicity scores, but the paper does not state whether the same prompts are used for correlation, threshold selection, and evaluation; please make this explicit (this is related to Major Comment 1).","section":"Appendix D.3"}],"recommendation":"major_revision","confidential_remarks":"The architectural contribution is sound and should not be held hostage to the manipulation-experiment issue. The main risk to the central claim is the in-sample identification/evaluation protocol; a revision that adds held-out splits and clarifies the deletion mechanism would make the paper acceptable. I see no novelty or scope problem."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The architecture is the real news here. Product-key composition with horizontal and vertical decomposition gives you O(sqrt N) parameters for N composed experts, which is a genuine step beyond PEER's linear memory. The derivations in Section 3 and Appendix A.2 are clean and self-consistent, the parameter counts check out, and the pretraining results show the models are competitive with dense LLAMA baselines at matched size, if a bit behind. The qualitative monosemanticity examples are suggestive, and the code/checkpoints being public helps.\n\nThe soft spot is exactly where the reader put it: the manipulation experiments identify experts using the same benchmark instances on which success is measured. In Section 5.1, MMLU routing probabilities assign experts to domains, and then the same MMLU questions are used to measure the accuracy drop after deletion. That makes the reported ΔTarget values in-sample selection artifacts. The same pattern holds for MULTIPL-E and for the toxicity experiments where expert–toxicity correlations are computed on the same prompts used for evaluation. The out-of-sample helpfulness numbers in Table 16 are reassuring only in that general performance holds up; they don't demonstrate precise, disentangled knowledge control.\n\nTo be fair, the paper does admit in the Limitations that the expert selection criteria are \"basic and minimal,\" and the authors aren't hiding the protocol. But the body text sells the manipulation results as robust, and the in-sample issue is not acknowledged there. This is a fixable flaw: hold out a portion of MMLU/MULTIPL-E/RealToxicityPrompts for expert identification and evaluate on the rest, or better, identify on one benchmark and evaluate on another. That would turn a suggestive result into a convincing one.\n\nThe architecture work alone justifies sending this to peer review. The manipulation claims need revision before acceptance, but with the identification/evaluation split they could be strong. I'd bring it to reading group to discuss the method, and I'd cite the architecture if I worked on MoE interpretability. Recommendation: accept for peer review, with the expectation that the authors re-run the manipulation experiments with a proper held-out protocol.","headline":"Monet's expert-composition architecture is a real contribution, but the knowledge-manipulation results rest on in-sample expert selection and need a held-out evaluation before they can be trusted.","tokens_in":712,"tokens_out":862,"would_cite":true,"duration_ms":23423,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"With 262,144 experts per layer, deleting the few tuned to a domain, language, or toxicity erases that knowledge while general performance holds.","keywords":["mixture-of-experts","monosemanticity","mechanistic interpretability","sparse dictionary learning","product key composition","knowledge unlearning","toxicity mitigation","parameter-efficient scaling"],"falsifier":"Split each MMLU domain's questions into two disjoint halves, compute expert routing skewness on half A, delete the skewed experts, and measure accuracy on half B; if the target-domain drop shrinks toward the cross-domain drop under this held-out protocol, the reported surgical unlearning (for example Biology at -4.66 points, other domains at -0.42) is an artifact of selecting experts on the evaluation set itself.","tokens_in":49563,"feed_emoji":"🧠","tokens_out":16142,"duration_ms":136287,"temperature":0.7,"pith_summary":"This paper tries to establish that a transformer can be made interpretable by construction: decompose each feedforward expert into two shared parameter pools so that 262,144 experts exist per layer while total parameters grow only as the square root of the expert count. At that granularity, the paper argues, individual experts become monosemantic — each one fires for a single concept, such as chemical compounds, U.S. states, or the stem 'Bay' in either a geographic or a Bayesian sense. The payoff is claimed to be surgical knowledge manipulation: removing the experts assigned to a domain, a programming language, or toxicity drops performance on exactly that target while leaving everything else nearly intact. If correct, this would collapse mechanistic interpretability and model editing into the pretraining objective itself, as a built-in alternative to post-hoc sparse autoencoders, which the paper argues degrade model quality.","feed_headline":"262K experts per layer make transformer knowledge surgical","feed_subtitle":"Deleting domain-, language-, or toxicity-tuned experts leaves the rest of the model's general performance intact.","key_machinery":"The central object is product key composition through expert decomposition. Instead of storing $N$ expert MLPs, the model stores two pools of $\\sqrt{N}$ parameter blocks each — bottom versus top layers in the horizontal variant, left versus right segments in the vertical variant — and treats their Cartesian product as the expert set. Routing scores factorize as $g_{hij} = g^1_{hi}g^2_{hj}$, so retrieval never enumerates the $N$ composed experts; batch-normalization quantile estimates take the place of top-$k$ sorting, and two auxiliary losses (a uniformity loss over routing distributions and an ambiguity loss against low-confidence routing) push the model toward uniform-but-specialized expert usage. The load-bearing identity is the rearranged summation of Equation 12, which turns the composed-expert output into per-pool precomputable operations and is what makes $O(\\sqrt{N})$ parameter scaling compatible with $O(\\sqrt{N})$ retrieval time.","core_discovery":"MONET replaces each standalone expert network with a composed one built from two parameter pools. In the horizontal variant the model stores $\\sqrt{N}$ bottom projections $\\{U_i\\}_{i=1}^{\\sqrt{N}}$ and $\\sqrt{N}$ top projections $\\{V_j\\}_{j=1}^{\\sqrt{N}}$; the $(i,j)$-th expert is $E_{ij}(x) = V_j\\,\\sigma(U_i x + b^1_i) + b^2_j$, so $N$ distinct experts cost only $O(\\sqrt{N})$ parameters. Routing follows the product-key scheme: the hidden state is split in half, independent top-$k$ selections are made on each half, and the composed expert's routing score is the product of its two components' scores; a reordering of the double summation precomputes each pool's contribution, keeping retrieval cost at $O(\\sqrt{N}Hd)$. At 262,144 experts per layer, the paper claims, routing becomes strongly skewed — each token commits to a single expert — and this yields mutually exclusive knowledge: distinct experts fire for 'Bay Area' versus 'Bayesian', and deleting the experts skewed toward a target MMLU domain drops that domain by roughly 1 to 5 accuracy points while other domains shift by a few tenths of a point.","pith_inferences":["The factorization principle is general: any decomposition of expert weights into two or more shared pools — not just bottom/top or left/right splits — converts the parameter cost of $N$ experts from linear to roughly $N^{1/k}$, so tensor or multilinear expert decompositions are a natural next step the paper does not explore.","The manipulation numbers are upper bounds: experts are identified by routing skewness on the same MMLU or MultiPL-E items used to evaluate the deletion, so a held-out version of the protocol would likely show smaller target-domain drops; the paper itself concedes its expert-selection criteria are basic and minimal.","A scaling law for monosemanticity is directly testable: train MONET variants with, say, 4K, 16K, 64K, and 256K experts under matched budgets and measure whether routing entropy falls and expert exclusivity rises with expert count, as the paper's claims imply."],"forward_implications":["Interpretability can be scaled cheaply: because total parameters grow as the square root of the expert count, the 262,144-expert regime is not a one-off but a range in which finer-grained routing is available at modest memory cost.","Knowledge control reduces to a delete operation: purging routing-skewed experts drops the target MMLU domain by up to about 5.5 accuracy points, the target language's pass@100 by up to 31 points, and expected maximum toxicity from 0.795 toward 0.55, while other-domain accuracy moves by well under a point and average helpfulness stays nearly flat.","Post-hoc sparse autoencoders become unnecessary for feature attribution, because the expert dictionary is learned end-to-end with the language-modeling loss and there is no reconstruction error to cascade through the network.","The architecture carries over to code modeling and to vision-language fine-tuning, with the paper reporting monosemantic specialization preserved across those settings."],"supporting_citations":[{"why":"Supplies the product-key retrieval algorithm that MONET adapts to route among 262,144 composed experts without enumerating them.","marker":"Lample et al. (2019)"},{"why":"PEER, the million-expert MoE whose linearly growing expert memory MONET's $\\sqrt{N}$-parameter scaling is designed to beat; it also provides the retrieval-complexity baseline the paper compares against.","marker":"He (2024)"},{"why":"Source of the batch-quantile routing idea MONET uses to replace top-$k$ sorting with batch-normalization statistics.","marker":"Bussmann et al. (2024)"},{"why":"The benchmark whose 57 subjects, regrouped into 14 domains, are used to measure domain-specific knowledge unlearning.","marker":"Hendrycks et al. (2021)"},{"why":"Provides the 14-domain taxonomy that labels experts as domain-specialized and structures the domain-masking evaluation.","marker":"Wang et al. (2024)"},{"why":"The polyglot code benchmark whose pass@100 scores ground the programming-language masking claims.","marker":"Cassano et al. (2023)"},{"why":"The toxicity-prompts benchmark whose PERSPECTIVE-API scores drive toxic-expert identification via Pearson correlation.","marker":"Gehman et al. (2020)"},{"why":"The second toxicity benchmark, scored by a RoBERTa classifier, confirming the detoxification effect of expert purging.","marker":"Hartvigsen et al. (2022)"},{"why":"The standard sparse-MoE baseline that, in the paper's domain-masking comparison, shows no specialists under the routing-skewness criterion.","marker":"Muennighoff et al. (2024)"}],"fun_headline_variants":["Monet: 262K experts per layer, sqrt(N) parameters","Sparse dictionary learning integrated into MoE pretraining","Deleting domain-tuned experts leaves general performance intact","Interpretable transformers: 262K experts per layer with sqrt(N) params","Mutually exclusive experts enable surgical knowledge editing in LLMs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Everything claimed about unlearning rests on the assumption that an expert's tendency to fire on a benchmark's questions reveals where that knowledge actually lives, even though the same questions are used both to choose which experts to delete and to score how well the deletion worked.","fun_headline_variants_meta":{"raw":{"variants":["Monet: 262K experts per layer, sqrt(N) parameters","Sparse dictionary learning integrated into MoE pretraining","Deleting domain-tuned experts leaves general performance intact","Interpretable transformers: 262K experts per layer with sqrt(N) params","Mutually exclusive experts enable surgical knowledge editing in LLMs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000735,"raw_usage":{"total_tokens":3330,"prompt_tokens":1034,"completion_tokens":2296,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":650,"completion_tokens_details":{"reasoning_tokens":2209}},"tokens_in":650,"tokens_out":2296,"duration_ms":13923,"temperature":1.0,"reasoning_tokens":2209,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T21:42:50.824281+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Split each MMLU domain's questions into two disjoint halves, compute expert routing skewness on half A, delete the skewed experts, and measure accuracy on half B; if the target-domain drop shrinks toward the cross-domain drop under this held-out protocol, the reported surgical unlearning (for example Biology at -4.66 points, other domains at -0.42) is an artifact of selecting experts on the evaluation set itself.","supporting_citations":[],"review_version":1}