{"id":"501d4bf5-1547-461e-b0a2-fab43111b41c","arxiv_id":"2505.08607","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A training framework that uses monocular depth estimates, diffusion inpainting, pseudo-labels and a DINOv2-based encoder to improve zero-shot stereo matching accuracy.","lead":"This paper trains a stereo depth model on a mix of synthetic images, real-world driving frames, and single photos turned into fake stereo pairs using a monocular depth model plus diffusion inpainting, then tests without fine-tuning on KITTI, ETH3D and Middlebury. It reports large accuracy gains over SceneFlow-only baselines, but the comparisons are not apples-to-apples and the evaluation uses training splits, so the gains need careful validation.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Monocular-teacher error is a hard ceiling for the dominant training signal; the paper never separates teacher error from stereo-model error, so the zero-shot gains in hard regions are not yet established.","rationale":"The reader's weakest assumption identifies exactly the point I would flag: the training signal on all non-synthetic data is dominated by monocular-teacher pseudo-labels. This is load-bearing because the paper's headline numbers are obtained after training on a pipeline whose main novel data sources are generated or supervised by Depth Anything V2. The controlled data-generation comparison in Table 2 is genuine evidence that generated data help, and the ablations in Table 3 attribute gains to data mixing and the hybrid encoder, but those experiments do not hold the teacher fixed. MFS and DiffMFS use different monocular depth sources, so the comparison conflates the generation pipeline with teacher quality. The evaluation on KITTI training splits and the use of ETH3D as a validation set make this harder to rule out, because model selections may be tuned on domains close to the pseudo-label source. The concern does not imply the method is wrong; a teacher-swap experiment or a teacher-versus-model error decomposition could confirm that the stereo model genuinely learns geometry. Until then, the appropriate verdict remains CONDITIONAL: the approach is plausible and partially supported, but the strongest quantitative claim about zero-shot generalization is not yet established at face value.","tokens_in":12495,"tokens_out":6905,"duration_ms":79984,"concrete_test":"On KITTI12/15 and ETH3D, align Depth Anything V2's D_mono to LiDAR GT with a least-squares scale/shift (Eq. 6) and compute the teacher's D1/EPE overall and on masks for textureless, transparent, and reflective regions. Compare those numbers to BooSTer's D1/EPE on the same masks. If the aligned teacher is as accurate as or more accurate than BooSTer in those regions, the reported zero-shot gains are teacher-limited and not evidence of stereo-specific improvement. Additionally, retrain on a 100K-image DiffMFS subset generated from the same left images with a second independent teacher (e.g., Depth Pro or synthetic GT from SceneFlow); if KITTI/ETH3D D1 shifts materially, teacher quality is a load-bearing confound, while a negligible shift would resolve the concern.","verdict_should_be":"UNCHANGED","load_bearing_attack":"All non-SceneFlow supervision originates from Depth Anything V2. DiffMFS right views are forward-warps of D'_mono = alpha D_mono (Eq. 4), and DrivingStereo supervision uses LSDSI to fit a D_pred + b to D_mono (Eqs. 6-9). Wherever the teacher is systematically wrong, such as textureless, transparent, or reflective regions, the generated training pairs and pseudo-labels are wrong in exactly those regions. Section 4.3 and Table 1 claim improvements on such regions, but no experiment decomposes the error into teacher error and stereo-model error. A KITTI12 D1 of 3.04 could reflect a model that has learned a strong monocular prior rather than a model that has resolved geometric ambiguities; in the limit, a model that simply copies the aligned teacher would inherit the teacher's D1. The Table 2 comparison does not remove this confound because the MFS baseline and DiffMFS are not shown to use the same monocular teacher. Thus the central claim that the mixed stereo data boosts zero-shot stereo is not yet distinguished from the weaker claim that BooSTer distills monocular depth into a stereo network.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes BooSTer, a stereo matching framework trained on a mixed dataset of synthetic SceneFlow data, real-world DrivingStereo data, and DiffMFS, a newly generated dataset of stereo pairs synthesized from large-scale single-view images. The synthesis pipeline uses Depth Anything V2 to produce monocular disparity, random scaling to obtain pixel disparity, forward warping to create a right view, and Stable Diffusion inpainting with an edge-aware mask to fill occlusion holes. For real stereo data with sparse LiDAR labels, the authors add a dynamic scale- and shift-invariant (DSSI) loss that aligns the network prediction to Depth Anything V2 pseudo-disparity. The architecture replaces the standard encoder with a hybrid DINOv2-VGG19 encoder. The paper reports zero-shot gains on KITTI 2012, KITTI 2015, ETH3D, and Middlebury, with KITTI12 D1 improving from 4.98 (StereoBase) to 3.04 and ETH3D D1 from 1.28 to 0.70. Ablations in Table 2 compare SceneFlow, MFS, and DiffMFS under the same network architectures, and Table 3 ablates data mixture, inpainting, and encoder choices.","tokens_in":12758,"tokens_out":3212,"duration_ms":34459,"significance":"If the reported gains survive a fully controlled comparison, the paper makes a useful contribution to zero-shot stereo matching: it demonstrates that large-scale single-view data can be converted into stereo training pairs at scale, that real-world pseudo-labels from a monocular teacher can supplement sparse LiDAR, and that a frozen VFM encoder can improve transfer. The cross-domain evaluation on externally acquired LiDAR ground truth means the headline result is not circular. The same-architecture comparison in Table 2 is a genuine strength and provides support for the data-generation claim, as does the ablation structure in Table 3. The main unresolved risk is that all non-SceneFlow supervision derives from a single monocular teacher, so the method's ceiling in hard regions may be set by the teacher and the paper does not quantify that ceiling.","major_comments":[{"comment":"The headline comparison in Table 1 is not controlled: 'Ours' simultaneously changes the training data (mixed dataset vs. SceneFlow-only) and the encoder (DINOv2-VGG19 hybrid vs. the baselines' standard encoders), so the reported gains over StereoBase and IGEV conflate the data-generation contribution with the architecture contribution. The controlled comparison in Table 2 is better evidence, but it does not repair Table 1's headline claim. Please add an experiment that trains StereoBase or IGEV on the full mixed dataset with the baseline encoder, and also report a version of BooSTer trained on SceneFlow only, so the two factors can be separated.","section":"Section 4.3, Table 1"},{"comment":"The paper does not separate monocular-teacher error from stereo-model error. DiffMFS supervision is derived from D'_mono = alpha D_mono (Eq. 4), and DrivingStereo supervision uses an affine fit to the same D_mono (Eqs. 6-9); both use Depth Anything V2. Wherever that teacher is systematically wrong in textureless, transparent, or reflective regions, the generated data and pseudo-labels are wrong in exactly those regions, yet Section 4.3 claims improvements in such regions. A KITTI12 D1 of 3.04 could in principle be approached by a model that copies the aligned teacher's output; the paper does not report the aligned teacher's own D1 on KITTI/ETH3D or provide a per-region error decomposition. Please add an experiment that aligns Depth Anything V2 predictions to the LiDAR ground truth with the same scale/shift procedure and reports its D1, EPE, and per-region errors on the evaluation sets, and compare those numbers against BooSTer's errors.","section":"Sections 3.2-3.3 and 4.3"},{"comment":"The comparison between MFS Dataset and DiffMFS in Table 2 does not isolate the proposed data-generation components because the two pipelines use different monocular depth models and different inpainting strategies. MFS uses the original Mono-for-Stereo pipeline, while DiffMFS uses Depth Anything V2 plus a Stable Diffusion and edge-aware inpainting; the gains could come from the stronger teacher rather than from the edge-aware inpainting or scaling scheme. Please include an ablation that keeps the teacher and inpainting fixed and varies only the proposed component, or at least report the teacher model used for the MFS baseline and quantify its depth accuracy on a common validation set.","section":"Section 4.4, Table 2"}],"minor_comments":[{"comment":"Several hyperparameters that directly control the generated data and loss are not reported, including the sampling range [dmin, dmax] for alpha in Eq. 4, the edge threshold tau in Eq. 5, the DSSI loss weight beta in Eq. 10, the definition of the 'Quintiles operation' in Eq. 8, and the number of DrivingStereo images used with the 5:6:1 sampling ratio. Please state these values for reproducibility.","section":"Section 4.2, Implementation Details"},{"comment":"The conclusion contains a typo: 'scenting an advancement' should be 'cementing an advancement' or 'representing an advancement'.","section":"Conclusion"},{"comment":"The Table 3 header 'Stable-Diffsuion' is a typo for 'Stable-Diffusion'.","section":"Table 3"},{"comment":"The notation for the valid mask is inconsistent: the text defines Mval as the valid mask, but the equation writes Mval ⊙ (D_GT − D_pred) and the surrounding text also uses M without a subscript. Please unify the notation.","section":"Section 3.1, Eq. 2"},{"comment":"The paper does not mention a code or dataset release plan. Given that the contribution is a training data pipeline and architecture, releasing the DiffMFS generation code and the trained checkpoints would substantially strengthen reproducibility.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper's core idea is promising and the ablation structure is better than many submissions in this area, but the main Table 1 claim conflates data and architecture, and the monocular-teacher ceiling is not quantified. These are fixable with additional experiments rather than being fatal flaws. I do not see evidence of misconduct; the main issue is controllability of the comparison and the need to bound the teacher's contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nBooSTer is a training recipe for zero-shot stereo matching that combines three things: a 597k-image synthetic stereo dataset (DiffMFS) generated from single-view images via Depth Anything V2 and Stable Diffusion inpainting, a dynamic scale-and-shift-invariant loss to supervise sparse real LiDAR with monocular pseudo-labels, and a DINOv2+VGG19 hybrid encoder. The value is real. Building a large stereo dataset from single images is a practical contribution, and the edge-aware inpainting addresses the foreground-background blending that naive inpainting produces. The Table 2 comparison—same architectures trained on SceneFlow vs MfS vs DiffMFS—supports the claim that DiffMFS is better than MfS for three of four backbones. The ablation in Table 3 shows each component adds something.\n\nThe soft spots are mostly about evaluation. Table 1 compares Ours (trained on mixed data) against baselines trained only on SceneFlow, so the gains conflate method with data scale and data source. That's fine as a headline, but it doesn't isolate the claimed contributions. More concerning, all evaluations are on the training splits of KITTI12, KITTI15, ETH3D, and Middlebury. Calling that zero-shot is generous; those splits are public and have likely been used for tuning across many papers. No error bars, no code, and no dataset release means the numbers can't be checked.\n\nThe stress-test concern about the monocular teacher is legitimate and should be addressed. Every non-SceneFlow training signal comes from the same Depth Anything V2 teacher. DiffMFS ground truth is a random-scaled forward warp of its depth, and DrivingStereo pseudo-labels are aligned to its depth. So the model is being asked to reproduce mono-depth, and if the teacher is systematically wrong in textureless, transparent, or reflective areas, those errors are baked in. The Table 2 comparison doesn't break this confound because MfS uses a different teacher (MiDaS-like), so DiffMFS's gain over MfS could be just a better teacher. The paper's claim about hard regions in Section 4.3 is qualitative; there is no error decomposition.\n\nThat said, the central idea—that you can bootstrap stereo training from monocular depth at scale—is plausible and the controlled ablation in Table 3 gives some support. The paper deserves peer review, but the authors should be asked to (1) evaluate on held-out test sets (e.g., KITTI test or the recent test splits), (2) add a comparison where the same teacher is used for both MfS-style and DiffMFS-style generation, and (3) release at least the dataset and code. As is, the headline claims overstate the evidence.","headline":"A promising data-generation recipe and a large new dataset, but the zero-shot claims are not yet fairly demonstrated because the comparisons conflate data scale, teacher quality, and architecture.","tokens_in":13290,"tokens_out":2931,"would_cite":false,"duration_ms":28943,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"BooSTer pre-trains stereo matching on synthetic, real, and single-view images, cutting zero-shot D1 error on KITTI and ETH3D by roughly a third to a half.","keywords":["stereo matching","zero-shot generalization","monocular depth estimation","data generation","diffusion inpainting","pseudo-label supervision","vision foundation models","domain transfer"],"falsifier":"Compute zero-shot error on a dense-ground-truth real dataset separately for pixels where the monocular depth teacher is accurate and where it is wrong (e.g., relative error above 10%): if the stereo model's D1 on teacher-wrong pixels stays as high as the teacher's error, the gains are inherited from the teacher rather than from stereo geometry.","tokens_in":12295,"feed_emoji":"🖼️","tokens_out":10615,"duration_ms":93162,"temperature":0.7,"pith_summary":"Stereo matching needs dense pixel-wise labels, but real-world ground truth is sparse and expensive, and models trained on synthetic scenes do not transfer well outdoors. This paper argues that the bottleneck is not the stereo architecture but the training data and features: pre-training on a mixture of synthetic SceneFlow frames, real DrivingStereo frames, and stereo pairs synthesized from ordinary single-view images can close much of that domain gap. The synthesis pipeline uses monocular depth from a foundation model to produce pseudo-disparity, forward-warps the right view, and fills occlusion holes with an edge-aware diffusion inpainter; the real frames are supervised with a dynamic scale- and shift-invariant loss over monocular pseudo-depth. With a hybrid DINOv2-VGG19 encoder, the resulting BooSTer model reports D1 of 3.04 on KITTI12, 3.22 on KITTI15, 0.70 on ETH3D, and 7.50 on Middlebury in zero-shot evaluation, well below prior SceneFlow-trained baselines. If correct, this makes large-scale unlabeled single-view imagery a viable source of stereo supervision.","feed_headline":"Mixed-data pre-training lifts zero-shot stereo accuracy","feed_subtitle":"Adding monocular-depth pairs and real pseudo-labels to SceneFlow cuts D1 errors on KITTI, ETH3D, and Middlebury.","key_machinery":"The machinery is a three-part training recipe plus a hybrid encoder. Monocular depth maps from Depth Anything V2 are randomly scaled ($D'_{\\mathrm{mono}} = \\alpha D_{\\mathrm{mono}}$) and forward-warped to form the right view; an edge-aware inpainting module, which thresholds the horizontal gradient of the disparity map to build an edge mask, then uses Stable Diffusion to fill occlusion holes without blending foreground and background. Real DrivingStereo frames are supervised with a dynamic scale- and shift-invariant (DSSI) loss: a least-squares fit $(a,b)$ aligns the stereo prediction to the monocular pseudo-depth, outliers beyond a quintile threshold are masked, and the fit is recomputed before an MSE loss. The hybrid feature encoder concatenates DINOv2's high-level semantic features at one-thirty-second scale with VGG19's multi-scale pyramid features, feeding a StereoBase-style cost volume and refinement network. Each component is ablated in the paper.","core_discovery":"The paper's central claim is that zero-shot stereo matching can be made substantially more accurate by controlling what data a model pre-trains on and how its encoder is built. BooSTer pre-trains on a 5:6:1 mix of SceneFlow synthetic frames, generated DiffMFS pairs from 597,727 single-view images, and real DrivingStereo frames. DiffMFS is produced by taking monocular relative depth from Depth Anything V2, scaling it randomly, forward-warping the right image, and filling holes with an edge-aware Stable Diffusion inpainter. The real-world branch adds monocular pseudo-labels through a dynamic scale- and shift-invariant loss, and the encoder combines DINOv2 with VGG19. On held-out KITTI12, KITTI15, ETH3D, and Middlebury, the trained model reduces D1 substantially relative to the StereoBase baseline, for example 3.04 vs 4.98 on KITTI12, 3.22 vs 5.47 on KITTI15, and 0.70 vs 1.28 on ETH3D.","pith_inferences":["Implicit in the paper, and testable, is that the monocular teacher's accuracy sets the ceiling for the synthesized and pseudo-labeled data; corrupting the teacher should measurably degrade the stereo model on the affected surface types.","Because the DSSI loss aligns any stereo prediction to monocular depth up to an affine transform, the same mechanism could plausibly extend to unlabeled stereo video or self-training loops with uncertainty weighting.","The 5:6:1 sampling ratio and the random scaling range are choices rather than derived quantities, so per-target-domain tuning of these hyperparameters could yield further gains beyond the reported numbers."],"forward_implications":["Monocular depth models can be repurposed as dense stereo supervisors, reducing dependence on LiDAR ground truth.","Large single-view image collections become usable stereo training data at negligible annotation cost.","Zero-shot accuracy on KITTI and ETH3D improves by roughly a third to a half in D1 relative to SceneFlow-only training, making cross-dataset deployment more practical.","The hybrid VFM-CNN encoder and the edge-aware inpainting each contribute, with the data mixture giving the largest gain according to the paper's ablations."],"supporting_citations":[{"why":"It supplies the monocular depth predictions used to generate DiffMFS and to build pseudo-labels for DrivingStereo.","marker":"[Yang et al., 2024]"},{"why":"It provides the DINOv2 features used as the semantic branch of the hybrid encoder.","marker":"[Oquab et al., 2023]"},{"why":"It provides SceneFlow, the synthetic component of the mixed training set.","marker":"[Mayer et al., 2016]"},{"why":"It provides DrivingStereo, the real-world frames with sparse LiDAR labels for the DSSI-supervised branch.","marker":"[Yang et al., 2019b]"},{"why":"It is the prior single-image stereo generation approach that DiffMFS is compared against.","marker":"[Watson et al., 2020]"},{"why":"It supplies the StereoBase baseline architecture, the disparity refinement backbone, and the main zero-shot comparison.","marker":"[Guo et al., 2023]"},{"why":"It provides the KITTI12 benchmark used for zero-shot evaluation.","marker":"[Geiger et al., 2012]"},{"why":"It provides the KITTI15 benchmark used for zero-shot evaluation.","marker":"[Menze and Geiger, 2015]"},{"why":"It provides the ETH3D benchmark used for zero-shot evaluation.","marker":"[Schops et al., 2017]"},{"why":"It provides the Middlebury benchmark used for zero-shot evaluation.","marker":"[Scharstein et al., 2014]"}],"fun_headline_variants":["BooSTer mixes datasets to sharpen zero-shot stereo","Monocular depth + diffusion data boost stereo zero-shot","Mixed-source pre-training outperforms on zero-shot stereo","Data mix with diffusion pairs cuts D1 on real benchmarks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole approach rests on the assumption that a single-image depth model's relative depth estimates are reliable enough, after random scaling and warping, to act as true geometry for synthetic stereo pairs and as pseudo-labels for real images; where that teacher is wrong, the stereo model will be wrong too.","fun_headline_variants_meta":{"raw":{"variants":["BooSTer mixes datasets to sharpen zero-shot stereo","Monocular depth + diffusion data boost stereo zero-shot","Mixed-source pre-training outperforms on zero-shot stereo","Data mix with diffusion pairs cuts D1 on real benchmarks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000195,"raw_usage":{"total_tokens":1371,"prompt_tokens":976,"completion_tokens":395,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":592,"completion_tokens_details":{"reasoning_tokens":330}},"tokens_in":592,"tokens_out":395,"duration_ms":4113,"temperature":1.0,"reasoning_tokens":330,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:50:14.246975+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute zero-shot error on a dense-ground-truth real dataset separately for pixels where the monocular depth teacher is accurate and where it is wrong (e.g., relative error above 10%): if the stereo model's D1 on teacher-wrong pixels stays as high as the teacher's error, the gains are inherited from the teacher rather than from stereo geometry.","supporting_citations":[{"cited_title":"Depth anything: Unleashing the power of large-scale unlabeled data","cited_arxiv_id":null,"evidence_quote":"It supplies the monocular depth predictions used to generate DiffMFS and to build pseudo-labels for DrivingStereo."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It provides the DINOv2 features used as the semantic branch of the hybrid encoder."},{"cited_title":"A large dataset to train con- volutional networks for disparity, optical flow, and scene flow estimation","cited_arxiv_id":null,"evidence_quote":"It provides SceneFlow, the synthetic component of the mixed training set."},{"cited_title":"Learning stereo from single images","cited_arxiv_id":null,"evidence_quote":"It is the prior single-image stereo generation approach that DiffMFS is compared against."},{"cited_title":"Are we ready for autonomous driving? the kitti vision benchmark suite","cited_arxiv_id":null,"evidence_quote":"It provides the KITTI12 benchmark used for zero-shot evaluation."},{"cited_title":"Object scene flow for autonomous vehicles","cited_arxiv_id":null,"evidence_quote":"It provides the KITTI15 benchmark used for zero-shot evaluation."},{"cited_title":"A multi-view stereo benchmark with high-resolution images and multi-camera videos","cited_arxiv_id":null,"evidence_quote":"It provides the ETH3D benchmark used for zero-shot evaluation."},{"cited_title":"High-resolution stereo datasets with subpixel-accurate ground truth","cited_arxiv_id":null,"evidence_quote":"It provides the Middlebury benchmark used for zero-shot evaluation."}],"review_version":1}