{"id":"c7a15203-f8df-43c7-b94b-5f510a2abccd","arxiv_id":"2508.02587","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":5.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":3,"one_line_summary":"Routed fine-tuning, where adaptation modules are themselves routed across MoE experts, improves PEFT performance on commonsense and math reasoning tasks.","lead":"The paper proposes giving each expert in a mixture-of-experts language model its own fine-tuning adapter, instead of using one shared adapter, and tests this on two MoE models. If correct, it offers a more parameter-efficient way to adapt large sparse models to specific tasks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Evidence for the central empirical claim is inaccessible because the supplied full text is corrupted; the claim cannot currently be verified.","rationale":"The reader's verdict of UNVERDICTED is based on the corrupted full text, and my review independently finds that the central empirical claim cannot be checked from the supplied material. The reader's weakest_assumption focuses on the router's stability and informativeness; that is a substantive scientific condition, and it is indeed part of what would need to hold for the claim. However, the most immediately load-bearing issue is even more basic: the experimental evidence itself is unreadable, so no comparison, parameter match, or ablation can be evaluated. I therefore agree with the 'unverdictable' conclusion but only partially with the specific weakest-assumption framing. No reason was found to move the reader's verdict in either direction: the idea is not internally inconsistent, and there is no accessible positive evidence to support acceptance. The correct next step is to recover the intact manuscript and run the concrete verification described above.","tokens_in":10147,"tokens_out":1529,"duration_ms":19771,"concrete_test":"Obtain the intact source file (LaTeX/PDF) for arXiv:2508.02587 and inspect the experimental tables. Specifically, check whether the routed PEFT method is compared against non-routed PEFT baselines under a matched trainable-parameter budget with identical initialization and training protocol. Also check whether an ablation replaces the frozen MoE router with random routing; if random routing matches the reported gains, the improvement is not attributable to router-informed adaptation, and the central claim fails. If the intact tables show a clear parameter-matched gain over non-routed baselines and a clear advantage over random routing, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is empirical: routed adaptation modules should outperform existing PEFT strategies on OLMoE-1B-7B and Mixtral-8x7B for commonsense and math reasoning. For an empirical claim of this kind, the load-bearing condition is that the experiments provide a controlled comparison: the routed method must be compared against non-routed PEFT baselines with matched trainable parameter counts, identical base-model initialization, and otherwise identical training setups, so that any observed gain can be attributed to routing rather than to added per-expert capacity. The supplied full text is almost entirely mojibake, so no table, ablation, or experimental detail can be read. Consequently, this condition cannot be checked from the available material. The abstract asserts extensive experiments, but assertion alone does not support the causal attribution. This is not an internal inconsistency in the argument; it is a verification failure. The reader's UNVERDICTED verdict is appropriate because the central claim is plausible but the evidence is currently unavailable in the provided text.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes that parameter-efficient fine-tuning (PEFT) of mixture-of-experts (MoE) language models should route the adaptation modules themselves across experts, rather than using a single shared adapter. The abstract claims that this routed approach outperforms existing PEFT strategies on commonsense and math reasoning tasks when adapting OLMoE-1B-7B and Mixtral-8x7B, and that the paper identifies optimal configurations and provides empirical analyses. The supplied full text, however, is almost entirely corrupted mojibake, so the experimental tables, ablations, and implementation details cannot be read. The central claim is empirical, and the evidence supporting it is currently inaccessible.","tokens_in":10337,"tokens_out":3950,"duration_ms":41954,"significance":"If the claimed improvements are real and the comparisons are fair, the paper would make a useful contribution to PEFT for MoE models. The research question is well motivated: aligning adapter routing with the router's expert specialization is a natural idea, and the choice of a small and a large MoE model is sensible. However, the significance can only be assessed conditionally, because the full text is unreadable and the abstract provides no quantitative results, baselines, error bars, or ablations. The paper supplies no machine-checked proofs, reproducible code, or parameter-free derivations in the material I could read.","major_comments":[{"comment":"The main text is unreadable mojibake, and no table, ablation, or experimental detail is accessible. The abstract's claim of 'extensive experiments' cannot be verified, and this is load-bearing because the paper's central claim is empirical. Please resubmit a readable manuscript with complete tables and experimental descriptions.","section":"Full Text (as supplied)"},{"comment":"The abstract reports no quantitative results, baselines, error bars, or ablations, so even the summary of results cannot be checked. Please include at least summary statistics and a statement of the baseline comparison in the abstract, with full details in the body.","section":"Abstract"},{"comment":"The abstract states that the paper 'identifies the optimal configurations for different scenarios' but does not clarify whether the configuration search was performed on the same evaluation tasks used for the final reported performance. If the selection is in-sample, the reported gains may be optimistically biased. Please describe the configuration selection procedure and, if applicable, use held-out validation.","section":"Abstract"},{"comment":"Because the full text is corrupted, I cannot verify whether the routed adapters are compared against non-routed PEFT baselines with matched trainable parameter counts, identical base-model initialization, and otherwise identical training setups. The paper's causal attribution of gains to routing depends on such controlled comparisons; please provide them explicitly.","section":"Experimental setup (not readable)"}],"minor_comments":[{"comment":"The supplied text contains the line 'arXiv:2508.02588v1 [hep-ph] 4 Aug 2025', which conflicts with the submission number 2508.02587 and the cs.LG category; please correct this metadata.","section":"Full Text (header)"},{"comment":"The sentence 'existing PEFT strategies fail to leverage' the dynamic routing mechanism would benefit from citations to representative PEFT works on MoE models, so that the claimed gap is clearly situated in the literature.","section":"Abstract"},{"comment":"Figure and table captions are not readable in the supplied version; please ensure that all captions are complete and self-contained, and that the main text refers to each figure and table by an unambiguous label.","section":"Full Text (as supplied)"}],"recommendation":"uncertain","confidential_remarks":"The submitted full text is severely corrupted, so I cannot assess the experimental evidence. If the corruption is an artifact of the submission pipeline, the authors should be asked to resubmit a clean PDF. If the clean version contains the controlled comparisons described in my comments, the paper may be viable for this journal; otherwise, the central empirical claim remains unsupported."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the idea is sensible — if you're adapting an MoE model, the adapters should probably be routed the way the experts are — but I can't verify the central empirical claim because the supplied full text is unreadable mojibake with a stray hep-ph arXiv identifier. As it stands, this is not ready for referees.\n\nWhat's potentially good: the paper targets a real gap. Off-the-shelf PEFT like LoRA treats an MoE backbone as a dense network and ignores the router's expert specialization. Routing adapters through the same or a compatible routing mechanism is a plausible way to get more adaptation per trainable parameter. The claimed testbed — OLMoE-1B-7B and Mixtral-8x7B on commonsense and math reasoning — is the right kind of evaluation for a practical PEFT contribution.\n\nSoft spots. The biggest is that I can't check anything: no table, ablation, or baseline is readable in the supplied text, and the abstract gives no numbers. That leaves the central claim as an assertion. Second, the load-bearing assumption is that the frozen MoE router's expert specialization stays informative during adaptation. That's plausible, but it needs a control: match total trainable parameters and compare routed adapters against per-expert adapters with random or uniform routing, so the gain can be attributed to routing rather than to extra capacity. Third, the 'optimal configurations for different scenarios' phrasing makes me worry the configuration search was done on the evaluation tasks; the paper needs to state how those choices were selected. Fourth, I can't evaluate novelty or prior work because the abstract has no citations and the related work section is unreadable; similar MoE-adapter designs may already exist.\n\nBottom line: the direction deserves a serious look, and I'd be glad to referee a clean version with the promised tables and baselines. But this corrupted submission doesn't belong in front of a referee yet. The right move is to ask for a readable version (ideally with code and data), then reassess.","headline":"Sensible idea for routing adapters in MoE fine-tuning, but the supplied full text is corrupt and the abstract has no numbers, so the empirical claim is unverifiable from the submission.","tokens_in":10815,"tokens_out":4254,"would_cite":false,"duration_ms":45338,"reading_group":"no","serious_thinker":"unclear","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Fine-tuning adapters for Mixture-of-Experts language models should be routed by the model's own router rather than shared uniformly across experts.","keywords":["mixture-of-experts","parameter-efficient fine-tuning","routed adaptation","MoE routing","language model fine-tuning","commonsense reasoning","math reasoning"],"falsifier":"Train a shared adapter with exactly the same total number of parameters as all routed adapters combined, and measure whether routed adaptation still wins; if it ties or loses, the claimed benefit is just capacity rather than routing.","tokens_in":10002,"feed_emoji":"🔀","tokens_out":4025,"duration_ms":39182,"temperature":0.7,"pith_summary":"Mixture-of-Experts (MoE) language models route each token to a small set of specialized sub-models (experts), and that routing is usually frozen during fine-tuning. Standard parameter-efficient fine-tuning (PEFT) methods, which train only a few small adapter modules, apply the same adapter regardless of which expert is active. The paper's central claim is that these adapters should themselves be routed: each expert should have its own small trainable module, selected by the model's existing router. In experiments on OLMoE-1B-7B and Mixtral-8x7B across commonsense and math reasoning tasks, this routed adaptation outperforms existing PEFT baselines while keeping the number of trainable parameters comparable. The paper also traces how the router and experts change during fine-tuning and identifies which routing configurations help most.","feed_headline":"MoE models need routed fine-tuning adapters, not shared ones","feed_subtitle":"Routed adapters beat standard PEFT on commonsense and math reasoning for OLMoE-1B-7B and Mixtral-8x7B.","key_machinery":"The load-bearing mechanism is the routed adapter: a set of small trainable adaptation modules attached to expert-specific positions in the MoE layer, with a router deciding which module is active for each token. The key design choice is reusing the frozen MoE router's assignments to route the adapters as well, so the adaptation signal is concentrated on the same specialized experts that already process that token. The paper compares routing granularity and top-k selection strategies, and uses that analysis to explain when routed adaptation helps.","core_discovery":"The central discovery is that aligning adapter specialization with the MoE router's own specialization improves fine-tuning. Rather than inserting one shared adapter into every layer or expert, the proposed approach keeps a separate lightweight module per expert and lets the frozen router pick which module to apply to each token. The reported result is consistent across two MoE models and multiple reasoning benchmarks: routed adaptation beats standard PEFT strategies, and the best configuration depends on available compute and the number of experts. The paper further shows that different components of the MoE block respond differently during adaptation, which is why routing the adapters delivers an efficiency and accuracy gain rather than merely adding parameters.","pith_inferences":["If the benefit comes from matching the frozen router, then models with more diffuse expert specialization should benefit less; measuring expert overlap at initialization would test this.","A natural next step the paper does not take is to fine-tune or adapt the router itself; if that were done jointly, the routed-adapter advantage could change or grow.","Because the comparison keeps trainable parameter counts comparable, a further ablation that exactly matches total parameter budgets could separate routing effects from per-expert capacity effects.","Routed adapters could plausibly combine with low-rank adapter variants, but the paper does not test that combination."],"forward_implications":["PEFT fine-tuning of MoE models should treat the router as part of the adaptation design, not as a fixed obstacle.","Practitioners can get accuracy gains on commonsense and math reasoning without a proportional increase in trainable parameters, by routing small adapters.","The optimal configuration is scenario-dependent, so the paper provides a practical selection rule rather than a single universal recipe.","The method transfers from a smaller MoE model (OLMoE-1B-7B) to a much larger one (Mixtral-8x7B), suggesting the routing benefit is not a small-model artifact."],"supporting_citations":[],"fun_headline_variants":["Route MoE adapters, don't share them for fine-tuning","Routed adapters beat shared PEFT on MoE benchmarks","Align adapter routing with MoE router for better tuning","MoE models: route your adapters, not just experts","Per-expert adapters with routing improve MoE fine-tuning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's central claim depends on the frozen MoE router remaining informative about which expert is best for each token during fine-tuning; if expert assignments shift, or if the gain comes from having more total parameters, the routed adapter's advantage would not be attributable to routing.","fun_headline_variants_meta":{"raw":{"variants":["Route MoE adapters, don't share them for fine-tuning","Routed adapters beat shared PEFT on MoE benchmarks","Align adapter routing with MoE router for better tuning","MoE models: route your adapters, not just experts","Per-expert adapters with routing improve MoE fine-tuning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000224,"raw_usage":{"total_tokens":1392,"prompt_tokens":811,"completion_tokens":581,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":427,"completion_tokens_details":{"reasoning_tokens":510}},"tokens_in":427,"tokens_out":581,"duration_ms":5937,"temperature":1.0,"reasoning_tokens":510,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T17:37:45.327961+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a shared adapter with exactly the same total number of parameters as all routed adapters combined, and measure whether routed adaptation still wins; if it ties or loses, the claimed benefit is just capacity rather than routing.","supporting_citations":[],"review_version":1}