{"id":"1e33b769-f04d-417d-a1a8-6d8598b1c216","arxiv_id":"2508.07738","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":3,"one_line_summary":"TRGE combines task-specific expert groups, inter-group prototype-based routing, MLLM-based task identification, and progressive fusion to improve multi-domain continual learning on CLIP.","lead":"This paper introduces TRGE, a two-level routing mixture-of-experts method that adapts a frozen CLIP model for multi-domain continual learning. It uses expert groups per task, MLLM-generated task identifiers, and dynamic fusion to reduce forgetting while keeping trainable parameters low.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The method's success hinges on MLLM task recognition for unseen samples; without robustness evidence, inter-group routing can select wrong experts and invalidate the collaboration claim.","rationale":"The reader's weakest assumption is the reliance on the MLLM for task identification, and I agree that this is the most load-bearing unverified component. With only the abstract available, no experimental evidence about MLLM accuracy or its effect on routing is accessible. The central claim of superior performance with fewer parameters cannot be accepted or rejected without seeing the full training and evaluation setup. Therefore the verdict remains UNVERDICTED, not because no concern exists, but because the concern is exactly the kind that the missing full text would clarify. I do not move to CONDITIONAL because we cannot even verify that the training procedure is sound; the correct state is unverified.","tokens_in":775,"tokens_out":3605,"duration_ms":39638,"concrete_test":"Run TRGE on a standard MDCL benchmark (e.g., DomainNet or Office-Home) under three inference conditions: (a) oracle task labels, (b) MLLM-inferred labels, and (c) randomly shuffled labels. Compare final average accuracy and forgetting across conditions. Also report per-domain MLLM classification accuracy on the test set. If condition (b) approaches condition (c), the method is not robust to MLLM errors; if (b) matches (a), the MLLM is effectively providing oracle labels and should be evaluated for cost. Separately, compute total trainable parameters at the final task and report whether TRGE remains below all baselines at that point.","verdict_should_be":"UNVERDICTED","load_bearing_attack":"The strongest claim is that TRGE outperforms advanced methods with fewer trainable parameters. This claim is directly contingent on the inter-group routing policy, which uses task identifiers and task prototype distance to select and combine expert groups. During inference, task identifiers are not provided but are produced by an MLLM. If the MLLM misclassifies the domain of an unseen sample, the router will activate irrelevant expert groups, and the outputs will be combined inappropriately, potentially causing both catastrophic and forward forgetting to resurface. The abstract reports no quantitative MLLM accuracy, no ablation that replaces MLLM labels with oracle labels or random labels, and no failure analysis for out-of-domain inputs. Because the entire collaboration mechanism is routed through this component, the MLLM's reliability is load-bearing, not peripheral. Additionally, the claim of 'fewer trainable parameters' is ambiguous: if each new task adds an expert group, total parameters grow with the number of tasks. The comparison must specify whether parameters are measured at a fixed task or after the full sequence and must include all routing components. Without full-text evidence, the central claim is unverified in exactly this respect.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes TRGE, a two-level routing grouped mixture-of-experts method for multi-domain continual learning (MDCL) built on a frozen CLIP backbone. TRGE dynamically expands expert groups per task, uses an intra-group router to limit routing complexity, and uses an inter-group routing policy based on task identifiers and task prototype distances to combine relevant expert groups. Task identifiers are obtained from a Multimodal Large Language Model (MLLM) that generates semantic task descriptions. A dynamic fusion of frozen CLIP and learned adapter outputs is introduced to mitigate forward forgetting. The abstract claims that TRGE outperforms other advanced methods with fewer trainable parameters.","tokens_in":1041,"tokens_out":1797,"duration_ms":21558,"significance":"If substantiated, the proposed method would be a meaningful contribution to parameter-efficient continual learning under both domain and class shifts, especially the combination of grouped MoE with MLLM-based task identification and prototype-based inter-group routing. However, the abstract contains no experimental evidence: no datasets, task sequences, baseline names, metrics, or error bars are reported. The significance therefore remains entirely conditional on the full manuscript providing such evidence.","major_comments":[{"comment":"The central claim 'our method outperforms other advanced methods with fewer trainable parameters' is asserted without any supporting experimental detail. No datasets, domain/class shift protocols, baselines, metrics, or statistical significance information are presented. Because this claim is the paper's headline, it is load-bearing. The full text must contain a complete evaluation; the abstract alone cannot support it.","section":"Abstract"},{"comment":"The parameter-efficiency claim is ambiguous under the described dynamic expansion. If a new expert group is assigned for each task, total parameters grow with the number of tasks. The claim of 'fewer trainable parameters' must specify whether parameters are measured at a fixed task, at the end of the sequence, or per task, and must include all routing components (intra-group, inter-group, and the MLLM if it is trainable or used for inference). The abstract does not define the parameter accounting, making the comparison unverifiable.","section":"Abstract"},{"comment":"Inter-group routing relies on task identifiers generated by an MLLM 'to recognize the correct task identifier.' The reliability of this component is load-bearing: if the MLLM misidentifies the task for an unseen sample, the wrong expert groups will be activated and the claimed collaboration benefit collapses. The abstract reports no MLLM recognition accuracy, no ablation against oracle or random task identifiers, and no failure analysis for out-of-domain inputs. These are needed to assess whether the method degrades gracefully under realistic MLLM error rates.","section":"Abstract"},{"comment":"The dynamic fusion schedule is described only as 'based on training progress' and is claimed to mitigate forward forgetting. No definition of forward forgetting, no measurement protocol, and no description of the fusion schedule (e.g., linear, inverse progress, trainable weights) are given. Without these details, the forward-forgetting claim is not assessable even from the abstract.","section":"Abstract"}],"minor_comments":[{"comment":"Grammar: 'maintains the static experts count within the group' should be 'maintains the static expert count within the group.'","section":"Abstract"},{"comment":"Phrasing: 'MLLMs which own powerful multimodal comprehension capabilities' could be simplified to 'MLLMs that possess strong multimodal comprehension capabilities.'","section":"Abstract"},{"comment":"The term 'forward forgetting' is used without definition. If this is a standard term in the field, a brief clarification or citation would help the reader.","section":"Abstract"}],"recommendation":"uncertain","confidential_remarks":"The provided review material is abstract-only; the full text was not available. Consequently, the central experimental claims cannot be verified. I recommend that the editor obtain the complete manuscript before making a decision. The abstract does not contain enough information to judge soundness, and no load-bearing error can be identified because no evidence is visible."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This is an abstract-only review, so I can’t verify the experiments. What I can say: the two-level routing idea—static intra-group routing plus dynamic inter-group selection based on task prototypes and MLLM-generated descriptions—is a reasonable way to handle expert growth in continual learning, and the progressive fusion with a frozen CLIP backbone is a sensible addition. The combination is not a restatement of a single prior method, so there is some novelty, even if the components are established.\n\nThe main claims—outperforms advanced methods with fewer trainable parameters—are simply asserted in the abstract. That’s normal for an abstract, but it means I can’t judge soundness. The stress-test concern about the MLLM is the right one: the entire inter-group routing depends on recognizing the correct task identifier at inference. If that fails on unseen samples, the collaboration mechanism could hurt performance. The abstract reports no MLLM accuracy, no oracle/runtime ablation, and no failure analysis. That’s a legitimate worry, but it might be addressed in the full text.\n\nThe second concern is the parameter count. If each new task adds an expert group, total parameters grow with the number of tasks. The “fewer trainable parameters” claim needs a precise definition: measured at what point in the sequence, and does it include the router and the MLLM? Without that, the comparison is ambiguous.\n\nI can’t say the paper is flawed—only that the abstract alone doesn’t establish the results. The logic is coherent, the method is plausible, and the potential market (CLIP-based continual learning) is active. So I’d send it to peer review if the full paper includes the usual experimental details: task settings, baselines, standard deviations, and ablations on the router and fusion. If those are missing, rejection is straightforward. For now, I wouldn’t desk reject based on the abstract alone.\n\nWho is this for? Researchers working on parameter-efficient continual learning for vision-language models. It’s not a breakthrough, but it’s a credible engineering contribution if the results hold. My recommendation: send it out for review, with instructions to the referees to scrutinize the MLLM reliance and the parameter-count definition.","headline":"Plausible integration of known MoE ideas, but the abstract's performance claims are unsupported and the MLLM routing dependency is load-bearing; deserves a referee look, not a desk reject.","tokens_in":1496,"tokens_out":1896,"would_cite":false,"duration_ms":24385,"reading_group":"maybe","serious_thinker":"unclear","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a two-level routing grouped mixture-of-experts can make CLIP learn sequential multi-domain tasks while suffering less catastrophic and forward forgetting than prior methods, using fewer trainable parameters.","keywords":["multi-domain continual learning","mixture of experts","parameter-efficient fine-tuning","CLIP","multimodal large language models","catastrophic forgetting","forward forgetting","dynamic fusion"],"falsifier":"Run TRGE on unseen test images but give the MLLM a deliberately wrong task identifier (or corrupt the semantic description). If accuracy drops sharply compared to the oracle-identifier setting, then the reported gains depend on near-perfect task recognition. A simpler check: measure TRGE's accuracy when the MLLM's task classification accuracy falls below some threshold (e.g., 90 percent) on a held-out domain.","tokens_in":711,"feed_emoji":"🔀","tokens_out":3870,"duration_ms":36971,"temperature":0.7,"pith_summary":"TRGE is a parameter-efficient method for teaching a frozen CLIP model new domains and classes one task at a time. It assigns each task its own small group of experts so old knowledge is not overwritten, and uses a second-level router to mix the outputs of the most relevant groups so new tasks can borrow from old ones. To route correctly it relies on a multimodal large language model to name each task and to identify which task a test image belongs to. A final dynamic fusion blends the frozen CLIP output with the adapter output, which the paper says reduces forward forgetting. On standard multi-domain continual learning benchmarks, the method reports better accuracy than other advanced methods with fewer trainable parameters.","feed_headline":"Two-level router beats prior continual learning with fewer parameters","feed_subtitle":"Per-task expert groups plus a second router cut both catastrophic and forward forgetting in CLIP.","key_machinery":"Two-Level Routing Grouped Mixture-of-Experts (TRGE): a dynamically expanding expert architecture built on a frozen CLIP backbone. Each task receives a dedicated expert group; an intra-group router picks the most appropriate expert within a group, and an inter-group router aggregates outputs from several groups selected by task identity and prototype similarity. A training-progress-weighted fusion layer blends the frozen CLIP output with the routed adapter output for unseen samples.","core_discovery":"The central claim is that separation and collaboration can be combined rather than traded off. TRGE separates knowledge by giving each task a dedicated expert group, while the static size of the group keeps routing complexity bounded, and an intra-group router prevents overfitting as the number of experts grows. Collaboration is achieved by an inter-group router that takes the MLLM-recognized task identifier plus task prototype distances to select and weight relevant expert groups. The method further introduces training-progress-based dynamic fusion between the frozen CLIP features and the TRGE adapter output, which they argue mitigates forward forgetting on unseen samples. The paper reports","pith_inferences":["The reliance on an MLLM for task identification is the main external dependency; if the MLLM is removed or fails, TRGE degrades to a prototype-distance-only router, and the paper's own comparison would be a useful ablation to report.","The same two-level routing could be applied beyond vision-language models to other frozen foundation models (e.g., text or speech encoders), since the routing and fusion logic is not CLIP-specific.","An explicit test of the 'collaboration' claim would be to measure whether later tasks actually improve earlier-task accuracy via inter-group routing, rather than only matching single-task baselines.","The training-progress fusion schedule could be tuned per-domain; one could test whether a learned or uncertainty-based fusion weight outperforms the fixed progress schedule."],"forward_implications":["If TRGE is right, a single frozen CLIP can be incrementally specialized to many domains without retraining the backbone, which makes continual learning cheap in parameters and memory.","The inter-group routing means tasks are not learned in isolation: later tasks can actively reuse earlier expert groups, so positive transfer is possible instead of only protecting old tasks.","Because the number of experts grows but the group size stays fixed, the routing decision space does not blow up, so the method scales to long task sequences.","Dynamic fusion with training progress gives a direct lever to control how much pre-trained knowledge to trust at any moment, which addresses forward forgetting on unseen data."],"supporting_citations":[],"fun_headline_variants":["Two-level router splits experts, shares knowledge to stop forgetting","Dual routing in CLIP reduces both catastrophic and forward forgetting","Per-task experts plus inter-group router: continual learning without forgetting","TRGE: separate expert groups, collaborate via router, fewer parameters","Routing mixture-of-experts cuts forgetting in multi-domain learning"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The method works only if a multimodal language model reliably recognizes which task a given test image belongs to, because the inter-group router uses that identifier to select expert groups.","fun_headline_variants_meta":{"raw":{"variants":["Two-level router splits experts, shares knowledge to stop forgetting","Dual routing in CLIP reduces both catastrophic and forward forgetting","Per-task experts plus inter-group router: continual learning without forgetting","TRGE: separate expert groups, collaborate via router, fewer parameters","Routing mixture-of-experts cuts forgetting in multi-domain learning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000539,"raw_usage":{"total_tokens":2428,"prompt_tokens":756,"completion_tokens":1672,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":500,"completion_tokens_details":{"reasoning_tokens":1587}},"tokens_in":500,"tokens_out":1672,"duration_ms":17383,"temperature":1.0,"reasoning_tokens":1587,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T21:52:33.499071+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run TRGE on unseen test images but give the MLLM a deliberately wrong task identifier (or corrupt the semantic description). If accuracy drops sharply compared to the oracle-identifier setting, then the reported gains depend on near-perfect task recognition. A simpler check: measure TRGE's accuracy when the MLLM's task classification accuracy falls below some threshold (e.g., 90 percent) on a held-out domain.","supporting_citations":[],"review_version":1}