REVIEW 2 major objections 4 minor 42 references
Reviewed by Pith at T0; open to challenge.
T0 means a machine referee read the full paper against a public rubric. The mark states how deep the mechanical check went, never who wrote it. the ladder, T0–T4 →
T0 review · glm-5.2
Depth models hallucinate phantom walls from physically impossible edges
2026-07-09 23:50 UTC pith:GZLYUBJH
load-bearing objection Solid diagnostic with a real but non-fatal confound: the OOD objection lands but doesn't kill the core finding. the 2 major comments →
Geometric Collapse: When Vision Models Fail to Verify Physical Causality
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central object is Geometric Collapse: a global failure mode where a dense depth predictor adopts a visually salient but physically unsupported edge cue, integrates it as a depth discontinuity, and then propagates the resulting geometric inconsistency across the entire scene — beyond the perturbed region and beyond what local repair can fix. The mechanism carrying the argument is the contrast between Scrambled Edges (which violate physical priors) and two matched controls: energy-matched high-pass noise (same frequency content, no edge structure) and edge-shaped noise (same edge structure, no geometric violation). Collapse appears under the first but not the controls, isolating physical-p
What carries the argument
The diagnostic pipeline has four load-bearing components. First, Scrambled Edges: Canny edge segments are warped by random affine transforms (translation up to 25 percent of image dimension, rotation up to 60 degrees) and darkened, placing them in geometrically smooth regions where they violate continuity, illumination, and occlusion priors. Second, two controls isolate the mechanism: energy-matched high-pass noise (same RMS amplitude, no structure) and edge-shaped noise (same edge masks, no relocation). Collapse under Scrambled Edges but not under these controls attributes the failure to physical-prior violation, not frequency content or edge presence. Third, the Collapse Ratio (RMSE under
Load-bearing premise
The diagnostic assumes that relocating and rotating Canny edge segments isolates a physical-prior violation, rather than simply creating an out-of-distribution artifact that models fail on because it is unfamiliar. The energy-matched noise control matches frequency content but not the structural regularity of coherent edge segments, so the attribution of collapse to missing physical verification rests on the assumption that the perturbation is a clean probe of causality.
What would settle it
If models showed equal deviation under Scrambled Edges and under edge-shaped noise placed at original locations with matched intensity — i.e., if mere edge presence without geometric violation produced the same collapse — the claim that models specifically lack physical-causality verification would be undermined, as the failure would reduce to generic edge sensitivity.
If this is right
- If the claim holds, foundation-model depth predictors deployed in robotics or autonomous navigation could hallucinate phantom obstacles or distorted free space when encountering reflections, shadows, or glass — real-world edges that similarly lack depth support.
- Standard depth benchmarks that report only global pixel-wise error would pass models that have this failure, because smoothing under collapse can reduce average RMSE while destroying boundary fidelity.
- The oracle-repair ceiling (47 percent) implies that post-hoc output correction is fundamentally insufficient; physical-consistency checking would need to occur before edge cues are integrated into the geometric prediction.
- The same diagnostic framework could be applied to other dense geometric tasks that rely on boundary cues, such as optical flow or stereo matching near occlusion boundaries.
Where Pith is reading between the lines
- If Scrambled Edges are simply an out-of-distribution artifact rather than a clean isolation of physical-prior violation, the central claim that models lack physical verification would weaken — the models might be failing because the perturbation is unfamiliar, not because they cannot verify causality. The energy-matched noise control addresses frequency content but does not fully control for the s
- The finding that generative depth estimators (diffusion, flow-matching) show attenuated but significant collapse suggests that iterative refinement partially mitigates but does not solve the problem — which would imply the missing mechanism is not iteration per se but explicit support-aware cue selection before geometric integration.
- If the cue-selection failure is as fundamental as the paper argues, one would expect analogous collapse in other modalities where strong local signals are integrated into global structure without physical verification — for instance, audio source separation adopting specular reflections as direct sources.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper introduces 'Scrambled Edges,' a diagnostic perturbation that relocates, rotates, and darkens real Canny edge segments to create visually salient but physically unsupported cues. The authors show that modern dense depth predictors (CNN, ViT, SSL-based) adopt these cues, causing a 'Geometric Collapse' where errors propagate globally beyond the perturbed region. The study uses energy-matched and structure-matched controls to isolate the effect of physical-prior violations from generic high-frequency noise. The paper is well-written, the experimental design is rigorous, and the findings are significant for the field of computer vision and robustness evaluation.
Significance. The paper makes a strong contribution by identifying a specific, global failure mode in dense depth predictors. The use of controlled counterfactuals (energy-matched and structure-matched controls) to isolate physical-prior violations is a methodological strength. The finding that oracle repair is capped at 47% due to global spillover is particularly impactful for safety-critical applications. The paper provides clear, falsifiable predictions and a reproducible protocol.
major comments (2)
- The central claim that models fail to verify physical causality relies on the assumption that Scrambled Edges isolate physical-prior violations. However, as noted in the stress-test, Scrambled Edges also introduce an out-of-distribution (OOD) pattern: coherent edge segments placed in smooth regions. The existing controls (Edge-Shaped, Mask-Matched) do not fully separate physical-prior violation from OOD-ness because no control is both OOD and physically supported. This confound weakens the mechanism ladder (Table 2) and the 'negative emergence' interpretation. The authors should address this by either (a) designing a control that is OOD but physically supported, or (b) explicitly acknowledging this limitation and discussing why the OOD interpretation does not fully explain the observed global spillover and oracle repair ceiling.
- The oracle spillover analysis (§4.3, Fig. 3) is a key load-bearing result for the claim that errors propagate globally. However, the definition of 'Output Oracle' repair (Eq. 7) simply replaces the masked region of the scrambled prediction with the clean prediction. This measures spillover but does not test whether a more sophisticated repair (e.g., re-running the model with the masked region inpainted in the input space) could recover more. The paper mentions 'Input Inpaint (Oracle Mask)' in Appendix Table 13 (24% recovery), but this is surprisingly low compared to the Output Oracle (47%). The authors should clarify why input-level inpainting performs worse than output-level replacement and whether this supports or contradicts the global propagation claim.
minor comments (4)
- Table 2: The 'Mask-Matched' control shows collapse ratios < 1.0 (e.g., 0.58x for MiDaS v2.1). The text explains this is due to lower global RMS energy, but this could be confusing. Consider adding a brief note in the table caption or normalizing the energy for this control as well.
- §3.1, Definition 3.1: The perturbation intensity alpha is set to 0.8, yielding ~36% mask coverage. It would be helpful to show sensitivity results for alpha (e.g., a plot of collapse ratio vs. alpha) in the main text, not just in Appendix Table 10, to demonstrate that the effect is not an artifact of extreme perturbation strength.
- Figure 2: The pipeline diagram is informative but dense. Consider simplifying or splitting into two figures for clarity.
- Appendix D.1: The P-Score and O-Score definitions are somewhat ad-hoc (e.g., thresholds like Delta L > 30). The authors should briefly justify these choices or show sensitivity to alternative thresholds.
Circularity Check
No circularity found: diagnostic perturbation, physical-prior proxies, and collapse metrics are all defined independently of model outputs
full rationale
The paper is an empirical diagnostic study with no derivation chain that reduces to its own inputs. (1) Scrambled Edges (Definition 3.1) are defined via Canny extraction + affine transforms + darkening, entirely independent of any model output. (2) Collapse Ratio = RMSE_Δ,scram / RMSE_Δ,noise is a behavioral measurement comparing two perturbation conditions, not a fitted parameter. (3) The physical-prior proxies (G-Score, P-Score, O-Score in Appendix D) are validated against ground-truth depth, chromatic signatures, and T-junction analysis—not against model predictions—so they provide independent evidence that the perturbation lacks physical support. (4) The oracle repair ceiling (§3.4, 47% recovery) is a direct measurement using the known perturbation mask, not a quantity derived from a model that was fit to the same data. (5) The spillover repair bound (Appendix B.5, Proposition 3) is a straightforward logical argument: if D1(x) ≠ D0(x) for any x ∉ M, then no repair operator restricted to M can recover D0. This is a mathematical fact, not a self-referential claim. (6) The mechanism ladder (Table 2) ablates prior violations using independently defined conditions; collapse ratios are measured, not predicted from fitted parameters. (7) No load-bearing self-citations appear: all referenced models and methods (MiDaS, DINOv2, DepthAnything, Marigold, DepthFM) are external. The paper is self-contained against external benchmarks and its central claim—that models adopt unsupported edge cues without physical verification—is supported by behavioral measurements that are not forced by construction.
Axiom & Free-Parameter Ledger
free parameters (7)
- K (number of edge segments) =
15
- alpha (perturbation intensity) =
0.8
- Canny thresholds =
thr_low=50, thr_high=150
- Rotation range =
+/-60 degrees
- Translation range =
+/-25% of image dimension
- tau (geometric smoothness threshold) =
median of |grad(D_gt)| over NYU
- sigma (Gaussian blur for high-pass filter) =
5 pixels
axioms (5)
- domain assumption Surface continuity: depth and normals vary smoothly on a single surface.
- domain assumption Illumination coherence: shading and shadow edges should be compatible with plausible lighting.
- domain assumption Occlusion causality: boundary evidence should admit consistent depth ordering.
- ad hoc to paper Deviation from clean prediction measures whether the model rejects unsupported cues.
- ad hoc to paper Scrambled Edges are physically unsupported by construction.
invented entities (3)
-
Geometric Collapse
independent evidence
-
Scrambled Edges
independent evidence
-
Negative emergence
independent evidence
read the original abstract
Recent progress in large-scale self-supervised learning has improved dense geometric prediction, but it remains unclear whether such scaling yields inference-time physical plausibility checks. We propose Scrambled Edges, a controlled counterfactual that injects salient edge-like cues while violating surface continuity, illumination coherence, and occlusion ordering. With energy-matched and structure-matched controls, we isolate the effect of unsupported edge evidence from high-frequency energy and edge sparsity. Across CNN/ViT/SSL depth predictors on NYU Depth v2 and KITTI, Scrambled Edges induce up to 3.2x larger deviation from clean predictions than energy-matched noise; additional diffusion and flow-matching depth estimators show attenuated but still significant collapse. The resulting Geometric Collapse propagates globally: even with oracle knowledge of the corrupted region, output-level repair recovers only 47%, with substantial error outside the mask. These findings provide controlled behavioral evidence that current dense predictors lack reliable mechanisms to quarantine physically unsupported edge cues, motivating explicit plausibility scoring and selective cue integration.
Figures
Reference graph
Works this paper leans on
-
[1]
ImageNet-trained CNNs are biased towards textures; increasing shape bias improves accuracy and robustness , author=. arXiv preprint arXiv:1811.12231 , year=
work page internal anchor Pith review Pith/arXiv arXiv
-
[2]
ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depth
ZoeDepth: Zero-shot transfer by combining relative and metric depth , author=. arXiv preprint arXiv:2302.12288 , year=
work page internal anchor Pith review Pith/arXiv arXiv
-
[3]
Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data
Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data , author=. arXiv preprint arXiv:2401.10891 , year=
work page internal anchor Pith review Pith/arXiv arXiv
-
[4]
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
-
[5]
DepthFM: Fast Monocular Depth Estimation with Flow Matching , author=. 2024 , eprint=
work page 2024
- [6]
-
[7]
Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer , author=. 2020 , eprint=
work page 2020
-
[8]
DINOv2: Learning Robust Visual Features without Supervision , author=. 2024 , eprint=
work page 2024
-
[9]
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale , author=. ICLR , year=
-
[10]
Benchmarking Neural Network Robustness to Common Corruptions and Perturbations
Benchmarking neural network robustness to common corruptions and perturbations , author=. arXiv preprint arXiv:1903.12261 , year=
work page internal anchor Pith review Pith/arXiv arXiv 1903
-
[11]
Indoor segmentation and support inference from rgbd images , author=. ECCV , pages=
-
[12]
Segment anything , author=. arXiv preprint arXiv:2304.02643 , year=
work page internal anchor Pith review Pith/arXiv arXiv
-
[13]
NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis , author=. 2020 , eprint=
work page 2020
-
[14]
ACM Transactions on Graphics , volume=
3D Gaussian Splatting for Real-Time Radiance Field Rendering , author=. ACM Transactions on Graphics , volume=
-
[15]
Masked Autoencoders Are Scalable Vision Learners , author=. 2021 , eprint=
work page 2021
- [16]
-
[17]
Vision: A Computational Investigation into the Human Representation and Processing of Visual Information , author=. 1982 , publisher=
work page 1982
-
[18]
The Ecological Approach to Visual Perception , author=. 1979 , publisher=
work page 1979
-
[19]
Ken Nakayama and Shinsuke Shimojo , title =. Science , volume =. 1992 , doi =
work page 1992
-
[20]
Unsupervised monocular depth estimation with left-right consistency , author=. CVPR , year=
-
[21]
Annual Review of Psychology , volume=
Object Perception as Bayesian Inference , author=. Annual Review of Psychology , volume=. 2004 , doi=
work page 2004
-
[22]
A Computational Approach To Edge Detection , volume =
Canny, John , year =. A Computational Approach To Edge Detection , volume =. Pattern Analysis and Machine Intelligence, IEEE Transactions on , doi =
-
[23]
Rethinking Atrous Convolution for Semantic Image Segmentation
Rethinking atrous convolution for semantic image segmentation , author=. arXiv preprint arXiv:1706.05587 , year=
work page internal anchor Pith review Pith/arXiv arXiv
-
[24]
Journal of Graphics Tools , volume=
An image inpainting technique based on the fast marching method , author=. Journal of Graphics Tools , volume=
-
[25]
Alvey Vision Conference , volume=
A combined corner and edge detector , author=. Alvey Vision Conference , volume=
-
[26]
Are We Ready for Autonomous Driving? The
Geiger, Andreas and Lenz, Philip and Urtasun, Raquel , booktitle=. Are We Ready for Autonomous Driving? The. 2012 , pages=
work page 2012
-
[27]
Lin, Tsung-Yi and Maire, Michael and Belongie, Serge and Bourdev, Lubomir and Girshick, Ross and Hays, James and Perona, Pietro and Ramanan, Deva and Zitnick, C. Lawrence and Doll. Microsoft. European Conference on Computer Vision (ECCV) , year=
-
[28]
Geirhos, Robert and Jacobsen, Jörn-Henrik and Michaelis, Claudio and Zemel, Richard and Brendel, Wieland and Bethge, Matthias and Wichmann, Felix A. , year=. Shortcut learning in deep neural networks , volume=. Nature Machine Intelligence , publisher=. doi:10.1038/s42256-020-00257-z , number=
-
[29]
High-frequency component helps explain the generalization of convolutional neural networks , author=. CVPR , pages=
-
[30]
Why do deep convolutional networks generalize so poorly to small image transformations? , author=. JMLR , year=
-
[31]
Evaluating Robustness of Monocular Depth Estimation with Procedural Scene Perturbations , author=. 2025 , eprint=
work page 2025
-
[32]
A Fourier perspective on model robustness in computer vision , author=. NeurIPS , year=
-
[33]
P3Depth: Monocular Depth Estimation with a Piecewise Planarity Prior , author=. 2022 , eprint=
work page 2022
-
[34]
Image to Image Translation for Domain Adaptation , author=. 2017 , eprint=
work page 2017
-
[35]
Revisiting Single Image Depth Estimation: Toward Higher Resolution Maps with Accurate Object Boundaries , author=. 2018 , eprint=
work page 2018
-
[36]
Advances in Neural Information Processing Systems (NeurIPS) , year=
Overparameterization Improves Robustness to Covariate Shift in High Dimensions , author=. Advances in Neural Information Processing Systems (NeurIPS) , year=
-
[37]
Physics-as-Inverse-Graphics: Unsupervised Physical Parameter Estimation from Video , author=. 2020 , eprint=
work page 2020
-
[38]
Detecting Out-of-Distribution Inputs in Deep Neural Networks Using an Early-Layer Output , author=. 2019 , eprint=
work page 2019
-
[39]
Towards Adversarial Robustness via Transductive Learning , author=. 2021 , eprint=
work page 2021
- [40]
- [41]
-
[42]
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages=
Robustness of 3D Deep Learning in an Adversarial Setting , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages=
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.