{"id":"80c94447-5059-480f-ad16-da85876a6413","arxiv_id":"2507.04631","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"SMoEStereo uses a selective mixture-of-experts with adaptive LoRA ranks and adapter kernels to fine-tune frozen vision foundation models for robust cross-domain stereo matching.","lead":"This paper introduces SMoEStereo, a method that adapts frozen vision foundation models for stereo matching using a mixture-of-experts of low-rank adaptation and convolutional adapters, plus a decision network that skips unnecessary modules to save compute. It reports state-of-the-art cross-domain and joint generalization results across KITTI, Middlebury, ETH3D, and DrivingStereo benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The core results rest on internally consistent ablations and the decision-network mechanism is supported by the random-policy comparison; the main exposure is the absence of error bars, which makes several state-of-the-art claims equal to noise.","rationale":"The reader's weakest assumption focuses on the decision network's class-token routing. I examined that mechanism and found it to be better supported than the reader suggests: Table 7 (ID 5 vs ID 6) shows a real gap between learned and random policies, Fig. 3 shows distinct expert-selection distributions across datasets, and Table 9 shows throughput gains. However, the random-policy comparison is only one point of evidence and a single ablation with no variance estimates; the routing policy's far-OOD behavior remains untested. The more load-bearing issue is that the headline SOTA claims are quantified by very small metric differences with no error bars or seeds. Since the central claim is essentially a ranking claim, the absence of variance information hits the core of the paper, not a peripheral component. I agree with the reader's CONDITIONAL verdict, but for a somewhat different primary reason. The concrete test (multi-seed re-evaluation of the closest baseline comparisons) would settle the most important uncertainty. I also note the paper has genuine strengths: parameter-free ablations, reproducible recipes in the text, and a random-policy control, which is why I am not proposing REJECT.","tokens_in":27017,"tokens_out":1606,"duration_ms":16447,"concrete_test":"Re-run the central evaluations (SMoEStereo-RVC and the best baseline LoS/CREStereo++ in Table 6, and SMoEStereo vs Former-RAFT in Table 1) with at least 3 independent seeds each, using the same training recipe, and report mean and standard deviation. If the SMoEStereo-vs-baseline gaps exceed 2 standard deviations on the majority of datasets, the SOTA claim holds; if gaps are within 1 standard deviation, the claim should be downgraded to 'competitive'.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's central claim is state-of-the-art cross-domain and joint generalization. The ablations (Table 7 ID 5 vs ID 6, and Fig. 5) support the decision network's utility: learned policies beat random policies of similar compute, and the policy visibly changes per dataset (Fig. 3). The weakest load-bearing assumption is therefore not the routing mechanism itself, but the precision of the headline metric gaps. Many 'state-of-the-art' margins are small (e.g., Table 6: SMoEStereo 9.74 vs LoS 9.30 on Middlebury Bad 2.0, and 0.26 vs 0.25 on ETH3D Bad 2.0; Table 1: 4.22 vs 4.27 vs 3.9/4.0). No seeds, confidence intervals, or significance tests are reported anywhere. If the 0.1-0.3 point differences that anchor the SOTA claims are within run-to-run noise, the strongest headline claims are not established. This is an evidential gap rather than an internal inconsistency: a single-seed evaluation can be perfectly valid, but it cannot support fine-grained rank claims. Additionally, the abstract's code link (github.com/cocowy1/SMoE-Stereo) is not verifiable from the manuscript, and the paper flags no limitation section acknowledging the missing error bars or code verification, so the manuscript itself does not pre-empt this concern.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SMoEStereo, a parameter-efficient fine-tuning framework that adapts frozen Vision Foundation Models to stereo matching via two mixture-of-experts modules: MoE-LoRA with experts of different ranks and MoE-Adapter with experts of different kernel sizes, plus a lightweight decision network that predicts binary per-layer usage policies to selectively activate these modules. The method is evaluated for cross-domain generalization (pretraining on SceneFlow, testing on KITTI, Middlebury, ETH3D, DrivingStereo) and joint generalization (RVC setting, one model evaluated on the KITTI 2015, Middlebury, and ETH3D benchmarks). The paper reports state-of-the-art or near-state-of-the-art results, fewer trainable parameters than full fine-tuning, and improved inference efficiency compared with prior VFM-based stereo approaches. Ablations in the main text and supplementary show each component contributes, and the learned routing policy outperforms random policies of similar compute.","tokens_in":27329,"tokens_out":5837,"duration_ms":60631,"significance":"The paper is timely: leveraging VFMs for dense matching with a parameter-efficient, plug-and-play scheme is of broad interest. The heterogeneous MoE design (different LoRA ranks and CNN kernel sizes) with a trainable layer-selection mechanism is a technically interesting contribution, and the reported gains over the strongest VFM-based baseline (Former-RAFT) and over other PEFT methods are notable if they hold. The experimental coverage is extensive: multiple VFM backbones, PEFT baselines, cross-domain and RVC benchmarks, efficiency comparisons, and ablations. The method also ships a public code link, which aids reproducibility. The main fragility is that the headline 'state-of-the-art' claims rely on single-seed results with small margins and without significance tests, and one abstract claim is not supported by the paper's own Table 1.","major_comments":[{"comment":"The abstract claims 'state-of-the-art cross-domain generalization performance on the KITTI, Middlebury, ETH3D, and DrivingStereo datasets.' However, Table 1 shows that Former-RAFT achieves 3.9 on KITTI 2012 Bad 3.0, while SMoEStereo (DAMV2) achieves 4.22, and SMoEStereo (DINOv2) achieves 4.39. Thus the unqualified 'state-of-the-art on KITTI' claim is contradicted by the paper's own numbers. The claim should either be qualified as 'state-of-the-art on most benchmarks' or the paper should explain why KITTI 2012 is an exception and why the overall conclusion still holds.","section":"Abstract and Table 1"},{"comment":"No error bars, multiple seeds, or significance tests are reported anywhere. Several margins that anchor the state-of-the-art claims are very small: in Table 6, the ETH3D AvgErr is 0.14 for both SMoEStereo and LoS, and the Middlebury bad 2.0 is worse for SMoEStereo (9.74) than for LoS (9.30); in Table 1, KITTI 2015 Bad 3.0 differs by 0.03 between SMoEStereo (DAMV2) and Former-RAFT; in Table 2, DrivingStereo average D1 differs by 0.7 from Former-CFNet. With a single seed, these differences could easily be within run-to-run noise. The paper should report at least 3 seeds with means and standard deviations (or equivalent), and for the 'overall rank' claim in Table 6, it should state whether the rank is stable across runs. This is load-bearing because the 'state-of-the-art' and 'significantly outperforming' statements in Sec. 4.2.2 rest on these fine-grained margins.","section":"Tables 1, 2, 6"},{"comment":"The dimensionality of the LoRA expert matrices is inconsistent. The paper writes W_up^i ∈ R^{d×dim} and W_down^i ∈ R^{ri×ri}, but to compose as E_i^L(x_in) = W_up W_down x_in with x_in ∈ R^{N×dim}, W_down must map dim to ri (e.g., R^{dim×ri}) and W_up must map ri back to dim (R^{dim×ri} or R^{d×ri} with d=dim). As written, the product is undefined. This is a reproducibility issue in the core method formulation and should be corrected, along with clarifying the undefined quantity D in 'ri ≪ {D, dim}'.","section":"Sec. 3.2, Eq. (2)"},{"comment":"The decision network's core assumption is that the class token alone carries enough information to predict which MoE layers should be activated for a given input. This is validated only indirectly by the ablations in Table 7 (ID 5 vs. ID 6) and Fig. 5. The paper should provide a more direct analysis: e.g., report the routing policy's behavior on far-out-of-distribution inputs (such as DrivingStereo foggy/rainy subsets in the cross-domain setting), show examples where the policy skips critical layers and degrades accuracy, or quantify the accuracy/efficiency trade-off under mismatched routing. Without this, the efficiency claim and part of the accuracy claim rest on the unverified assumption that the learned policies generalize beyond the training domains.","section":"Sec. 3.2, Decision Network; Tables 7 and 9"}],"minor_comments":[{"comment":"The notation x_cls ∈ R^{N×dim} for the class token is likely a typo; the class token is a single token, so x_cls ∈ R^{1×dim}. The description of using the first element M^l_1 as the binary probability is also confusing and should be clarified.","section":"Sec. 3.2, Eq. (7)"},{"comment":"The text states 'SMoE outperforms Multi-E in cross-dataset performance,' but on KITTI 2015 EPE Multi-E is 0.59 and SMoE is 0.60. This should be acknowledged or the claim qualified, since it is not uniformly true.","section":"Table 9"},{"comment":"The paper claims 'sub-optimal best results in blue,' but the tables are not shown in color in the provided version, making this annotation invisible. Also, Table 5 names the GPU as 'Nvidia 5000 Ada' while the text says 'RTX 5000 Ada'; please unify.","section":"Tables and formatting"},{"comment":"The code link in the abstract is given in red text and is not verifiable from the manuscript. Please ensure the link is operational in the final version and consider adding a reproducibility statement describing how the code can be used to reproduce the main tables.","section":"Code availability"},{"comment":"The row '√ - -' has a stray quote character in the Middlebury column ('7.05), likely a LaTeX artifact. Please proofread the supplementary tables.","section":"Supplementary Table 10"},{"comment":"The paper does not include a limitations section. Given the single-seed evaluation and the unverified code link, adding a short limitations paragraph would improve the manuscript's transparency.","section":"Conclusion"}],"recommendation":"major_revision","confidential_remarks":"The paper fits the journal's scope and the technical idea is sound, but the headline SOTA claims are not yet supported by the reported evidence. The lack of any variance estimate, combined with the KITTI 2012 counterexample in Table 1, makes the current version unsuitable for acceptance without the requested revisions. The authors should also be asked to fix the Eq. (2) dimensionality issue, as it is a reproducibility blocker."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my take. This is a genuinely useful paper for robust stereo matching. It shows how to adapt VFMs with a compact PEFT design: MoE-LoRA with variable ranks, MoE-Adapter with variable kernel sizes, and a learned decision network that skips MoE layers per input. The claim that this is new holds up; prior work used fixed LoRA/adapters or MoE in other settings. The experiments are more thorough than most: cross-domain and joint generalization on four benchmarks, four VFM backbones, several PEFT baselines, ablations for each component, and a random-policy comparison that actually supports the decision network. The supplementary adds compatibility with other stereo backbones and data scaling. This is real work and the core mechanism is plausible.\n\nThe soft spot is not the mechanism but the precision of the headline claims. There are no error bars or multiple seeds anywhere. Several 'state-of-the-art' margins are tiny: SMoEStereo is worse than Former-RAFT on KITTI 2012 (4.22 vs 3.9) and worse than LoS on Middlebury bad 2.0 (9.74 vs 9.30) in the joint table; its overall rank 1 depends on Schulze aggregation across metrics. A 0.1-0.3 point gap can be run-to-run noise. So the paper's own data supports 'very competitive' more than 'state-of-the-art' on every benchmark. The code link in the abstract is not verifiable from the manuscript, and the paper contains no limitation section acknowledging either issue.\n\nThe decision network weakness the skeptic flags is less serious than it looks: the class-token signal is coarse, but the learned policy beats random policies of similar compute in ablation ID 5 vs 6 and Fig. 5, so the mechanism has empirical support. It would still be nice to see routing failure analysis on far OOD inputs, but that is a minor ask.\n\nWho this is for: researchers working on domain-generalized stereo or PEFT for VFMs. It deserves a serious referee. I would send it to review, with a request for multi-seed results and a working code link before acceptance, and a suggestion to soften the SOTA phrasing. This is a solid paper with an addressable evidential gap.","headline":"Solid empirical PEFT-MoE framework for robust stereo; architecture and ablations are convincing, but SOTA claims need error bars and working code.","tokens_in":27896,"tokens_out":2711,"would_cite":true,"duration_ms":28313,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"By inserting rank- and kernel-varying experts into frozen vision foundation models, SMoEStereo claims state-of-the-art zero-shot stereo matching across KITTI, Middlebury, ETH3D, and DrivingStereo with one fixed model.","keywords":["stereo matching","domain generalization","mixture-of-experts","low-rank adaptation","vision foundation models","parameter-efficient fine-tuning","selective routing","zero-shot transfer"],"falsifier":"Train the model, then evaluate it on far out-of-distribution stereo pairs such as heavy rain, night, or specular and transparent surfaces, and record the decision network's per-layer retention masks; if the masks are statistically indistinguishable from random pruning at the same compute budget, the scene-conditional routing claim is falsified.","tokens_in":26805,"feed_emoji":"🧩","tokens_out":4762,"duration_ms":50859,"temperature":0.7,"pith_summary":"SMoEStereo claims that frozen vision foundation models can be turned into robust stereo matchers by inserting two kinds of Mixture-of-Experts modules: LoRA experts with different ranks and CNN adapter experts with different kernel sizes, plus a lightweight decision network that turns MoE layers on or off per input. The paper argues that no single LoRA rank or adapter receptive field suits all scenes, so letting each input pick its own experts gives better cross-domain transfer than fixed parameter-efficient fine-tuning. Under this design, the same model trained on synthetic SceneFlow achieves state-of-the-art zero-shot results on KITTI, Middlebury, ETH3D, and weather-corrupted DrivingStereo, and the same fixed model ranks first among Robust Vision Challenge methods on joint generalization. If true, this offers a parameter- and computation-efficient recipe for reusing vision foundation models in dense geometric vision.","feed_headline":"One fixed stereo model tops four zero-shot benchmarks","feed_subtitle":"Adaptive LoRA and adapter experts cut stereo error up to 30 percent across KITTI, Middlebury, ETH3D, and DrivingStereo.","key_machinery":"The load-bearing objects are the MoE-LoRA layer, whose experts are LoRA matrices with distinct ranks $r_i$ selected by a softmax router, and the MoE-Adapter layer, whose experts are $1\\times1$ to $k\\times k$ CNN blocks with different kernels selected the same way. A decision network reads the class token, uses a Gumbel-softmax relaxation to produce binary masks $\\mathcal{M}^l_L$ and $\\mathcal{M}^l_A$ for each layer, and a usage loss $\\mathcal{L}_{\\text{usage}}$ with budget $\\gamma$ controls how many MoE layers stay active. This machinery converts 'which adaptation subspace fits this scene' into trainable per-input routing decisions and aligns computation with input complexity.","core_discovery":"The paper's central claim is that scene-specific, heterogeneous Mixture-of-Experts adaptation unlocks the robustness of vision foundation models for stereo matching. MoE-LoRA replaces fixed-rank LoRA with multiple rank choices routed per input, MoE-Adapter injects local inductive bias through CNN adapters of different kernel sizes, and a decision network trained with a usage loss learns binary per-layer skip policies. Together these components reduce D1 error by up to thirty percent versus a vanilla VFM-LoRA baseline while using fewer activated parameters at inference. The authors report state-of-the-art cross-domain generalization on KITTI, Middlebury, ETH3D, and DrivingStereo, and state-of-the-art joint generalization on the robust vision benchmarks with a single fixed model.","pith_inferences":["One implication not pursued in the paper: the learned expert-selection distributions across datasets (KITTI, Middlebury, ETH3D) could act as a cheap domain fingerprint, suggesting the router encodes scene type rather than just difficulty.","The class-token-only decision network is a minimal-routing bet; a multi-token or patch-level policy might generalize further, but would trade away the efficiency the paper is after.","The same heterogeneous-expert parameter-efficient fine-tuning recipe could transfer to other dense prediction tasks that suffer from domain shift, such as optical flow or monocular depth, wherever a frozen vision foundation model is the starting point.","The thirty percent D1 reduction relative to vanilla LoRA is measured on real benchmarks; a natural stress test is whether the routing policy holds on far out-of-distribution scenes such as night, snow, or specular surfaces."],"forward_implications":["A frozen vision foundation model backbone can reach top cross-domain stereo accuracy with only a few million trainable parameters, avoiding costly full fine-tuning.","The same trained model can be deployed on indoor, outdoor, and low-resolution benchmarks without dataset-specific adaptation, which is the setting real-world stereo systems face.","Varying the budget $\\gamma$ yields a smooth accuracy-versus-speed trade-off, so one model can serve devices with different compute limits.","Because the MoE blocks are plug-and-play, classic stereo backbones such as PSMNet, CFNet, and IGEV also gain in zero-shot transfer when wrapped in this design.","Larger vision foundation model backbones amplify the gains, so the approach improves as stronger foundation models become available."],"supporting_citations":[{"why":"Supplies the primary frozen vision foundation model backbone whose geometric features the method adapts.","marker":"[74]"},{"why":"Provides the RAFT-Stereo backbone whose iterative disparity refinement remains unchanged.","marker":"[32]"},{"why":"Defines low-rank adaptation, the base mechanism that the MoE-LoRA experts generalize.","marker":"[17]"},{"why":"Is the closest vision-foundation-model stereo fine-tuning baseline that SMoE must beat.","marker":"[79]"},{"why":"Supplies the sparse mixture-of-experts gating and balance loss used for routing and $L_{blc}$.","marker":"[52]"},{"why":"Is the CFNet robust stereo baseline and a Robust Vision Challenge comparison method.","marker":"[53]"},{"why":"Defines the UCFNet and Robust Vision Challenge joint-generalization training setting the paper follows.","marker":"[55]"},{"why":"Is CREStereo++, a robust recurrent stereo baseline and joint-generalization competitor.","marker":"[22]"},{"why":"Is visual prompt tuning, one of the parameter-efficient fine-tuning baselines compared in the experiments.","marker":"[20]"},{"why":"Is AdaptFormer, another parameter-efficient fine-tuning baseline compared in the experiments.","marker":"[6]"}],"fun_headline_variants":["Selective MoE stereo tops four zero-shot benchmarks","Adaptive experts cut stereo error 30% across domains","MoE-LoRA makes stereo robust without per-dataset tuning","Scene-aware routing boosts stereo cross-domain accuracy","One stereo model beats four benchmarks via selective MoE"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The decision network decides which MoE layers to keep using only the class token of the image; if that single vector cannot reliably signal which layers a scene needs, the claimed efficiency gains and part of the accuracy gains would disappear.","fun_headline_variants_meta":{"raw":{"variants":["Selective MoE stereo tops four zero-shot benchmarks","Adaptive experts cut stereo error 30% across domains","MoE-LoRA makes stereo robust without per-dataset tuning","Scene-aware routing boosts stereo cross-domain accuracy","One stereo model beats four benchmarks via selective MoE"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000161,"raw_usage":{"total_tokens":1232,"prompt_tokens":940,"completion_tokens":292,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":556,"completion_tokens_details":{"reasoning_tokens":214}},"tokens_in":556,"tokens_out":292,"duration_ms":3941,"temperature":1.0,"reasoning_tokens":214,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T19:42:45.621665+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the model, then evaluate it on far out-of-distribution stereo pairs such as heavy rain, night, or specular and transparent surfaces, and record the decision network's per-layer retention masks; if the masks are statistically indistinguishable from random pruning at the same compute budget, the scene-conditional routing claim is falsified.","supporting_citations":[{"cited_title":"Iterative geometry encoding volume for stereo matching","cited_arxiv_id":null,"evidence_quote":"Defines low-rank adaptation, the base mechanism that the MoE-LoRA experts generalize."},{"cited_title":"High-frequency stereo match- ing network","cited_arxiv_id":null,"evidence_quote":"Is CREStereo++, a robust recurrent stereo baseline and joint-generalization competitor."},{"cited_title":"Robust synthetic-to-real transfer for stereo matching","cited_arxiv_id":null,"evidence_quote":"Is visual prompt tuning, one of the parameter-efficient fine-tuning baselines compared in the experiments."},{"cited_title":"The open images dataset v4: Unified image classification, object detection, and visual relationship detection at scale","cited_arxiv_id":null,"evidence_quote":"Is AdaptFormer, another parameter-efficient fine-tuning baseline compared in the experiments."}],"review_version":1}