{"id":"3bc176da-bbb7-44ef-86fa-3c4d1404c186","arxiv_id":"1908.01238","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A guided convolutional network with factorized, content-dependent spatially-variant kernels achieves state-of-the-art depth completion on KITTI and NYUv2.","lead":"This paper proposes a depth completion network that uses an RGB image to generate spatially-variant convolution kernels, which are then applied to extract features from sparse LiDAR depth. The method ranks first on the KITTI depth completion benchmark at the time of submission and outperforms prior methods on NYUv2.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"NYUv2 benchmark superiority may reflect fitting colorization-inpainted labels rather than true depth; ranking under raw Kinect valid-pixel evaluation is unverified.","rationale":"I read the paper as making a benchmark-performance claim, not a theoretical claim. The architecture is coherent and the memory-reduction calculation in Section III-C is correct; the ablation studies support the fusion design internally. The KITTI result is from the official test server and is credible. The soft spot is the NYUv2 evaluation reference. The paper itself discloses the colorization inpainting in Section IV-A, and because the method's contribution is RGB-guided filtering, the disclosed protocol creates a specific channel by which the reported advantage could be an artifact of the labels rather than of true depth accuracy. The reader's verdict was already CONDITIONAL; this concern reinforces that conditionality. I do not see grounds for REJECT, since the disclosed protocol is standard in the subfield and the KITTI result is independent, so UNCHANGED is appropriate. A single focused re-evaluation on raw-valid pixels would settle whether the NYUv2 superiority is real.","tokens_in":17994,"tokens_out":6434,"duration_ms":72388,"concrete_test":"Evaluate the authors' trained NYUv2 model and the two strongest baselines (DeepLiDAR, CSPN) using the exact same 200/500 sparse sampling masks, but compute RMSE/REL/delta metrics only on pixels where the original raw Kinect depth is valid, masking out colorization-inpainted regions. If the authors' model remains best on these raw-valid-pixel metrics, the concern is resolved; if the ranking changes or the margin shrinks, the NYUv2 part of the central claim is not established. This requires code or predictions from the authors and the baselines; absent that, the claim should remain conditional.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline claim that the method 'outperforms state-of-the-art methods on the NYUv2 dataset' rests on comparisons computed against depth maps inpainted with the official colorization toolbox (Section IV-A). After inpainting, the NYUv2 ground truth is dense and image-aligned, and the training loss in Eq. (9) treats those interpolated values as valid targets. This is more than generic label noise for this architecture: the paper's central mechanism is RGB-guided, spatially-variant kernel generation, which is precisely the class of models that can learn to reproduce smooth, colorization-derived depth structure. If a substantial part of the reported 0.101 RMSE gain over DeepLiDAR/CSPN comes from fitting colorization artifacts rather than Kinect-measured surface geometry, the NYUv2 portion of the central claim does not establish physical depth-completion superiority. The KITTI comparison is less exposed because its reference depth is temporally registered LiDAR verified by stereo, but the NYUv2 claim is load-bearing in the abstract and Table III. No internal inconsistency in the convolution factorization is apparent; the uncertainty is empirical validity of the evaluation reference.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a depth-completion network in which a \"GuideNet\" generates content-dependent, spatially-variant convolution kernels from an RGB image, and these kernels are applied to sparse depth features inside a \"DepthNet.\" To make the scheme practical, the guided convolution is factorized into a spatially-variant channel-wise stage and a spatially-invariant cross-channel 1x1 stage, with a memory-reduction analysis in Eq. (8). The network fuses GuideNet decoder features into DepthNet encoder features at multiple scales. Experiments report rank 1 on the KITTI depth-completion leaderboard at the time of submission, state-of-the-art results on NYUv2, plus ablations and generalization tests under different point densities, lighting/weather conditions, and cross-dataset settings.","tokens_in":18210,"tokens_out":7934,"duration_ms":84805,"significance":"If the empirical claims hold, this is a solid contribution to depth completion and to multi-modal feature fusion more broadly. The factorization analysis in Eq. (8) is mathematically correct and the memory saving is substantial. The KITTI claim is especially strong because it is evaluated by the external test server, and the paper includes careful ablations of fusion position and multi-stage guidance. The generalization experiments across point density, lighting/weather, and datasets are a useful addition. The main weaknesses are empirical: the NYUv2 evaluation uses colorization-inpainted ground truth, no uncertainty quantification is provided for the small performance gaps, and some comparison rows are taken from prior papers while others are re-run by the authors.","major_comments":[{"comment":"The NYUv2 ground truth is depth in-painted with the official colorization toolbox, and the loss in Eq. (9) treats these interpolated values as valid targets. Since the central mechanism of the paper is RGB-guided, spatially-variant kernel generation, the reported NYUv2 advantage over DeepLiDAR and CSPN could partly reflect fitting smooth colorization structure rather than measured Kinect geometry. Please add an evaluation on raw valid Kinect pixels (or on the measured depth before inpainting) and explicitly quantify how much of the improvement survives under that more conservative reference.","section":"Section IV-A, Eq. (9), Table III"},{"comment":"All reported numbers are from a single training run, and several key gaps are small (e.g., NYUv2 RMSE 0.101 vs. 0.115 for DeepLiDAR and 0.117 for CSPN in Table III). Without error bars or multiple-seed runs, the claim of outperforming prior methods on NYUv2 is not statistically grounded. Please report mean and standard deviation over at least three runs for the main comparisons and ablations, or justify why a single run is sufficient for these conclusions.","section":"Section IV-D and Table III"},{"comment":"Table III mixes numbers quoted from previous papers with numbers the authors obtained by running released implementations. To make the comparison meaningful, the paper must state explicitly that identical preprocessing, sparse-sample generation, cropping, padding, and evaluation masks are used for all rows, and must list which rows were re-run and which were transcribed from prior publications. This is especially important for the 200-sample setting, where missing rows for CSPN and DeepLiDAR make the sentence \"outperforms all other methods in both settings\" unsupported as written.","section":"Section IV-E and Table III"},{"comment":"The statement that the factorization reduces the computational complexity of D_{p,n} from O(M*K^2) to O(M+K^2) is only valid if the channel-wise stage of Eq. (6) is computed once and amortized across all N output channels. In a from-scratch computation of a single output element, Eq. (6) still costs O(M*K^2). Please state the amortization assumption explicitly and report the actual total FLOPs ratio, since the paper's abstract claims a reduction in computation as well as memory.","section":"Section III-C, Eqs. (5)-(8)"}],"minor_comments":[{"comment":"There are several typographical errors, including \"naïvely\" in the abstract, \"Equqation (2)\" in Section III-C, \"datset\" in Section IV-A, and \"futher\" in Section IV-A; these should be corrected.","section":"Throughout"},{"comment":"The naming is inconsistent between \"GuideNet\" and \"GuidedNet\" across the text and figure legend; please unify the terminology.","section":"Figure 1 and Section III"},{"comment":"The kernel visualization in Figure 4 is acknowledged as \"extremely rough,\" but the accompanying claim that it \"confirms\" the kernels are content-dependent is stronger than the evidence warrants; please soften this to \"is consistent with\" or add a quantitative evaluation of kernel alignment with image boundaries.","section":"Section IV-C"},{"comment":"The 200-sample block of Table III does not include CSPN or DeepLiDAR, even though the text says the method outperforms all other methods under both settings; please add those results or qualify the claim.","section":"Section IV-E and Table III"},{"comment":"The claim \"ranks 1st on the KITTI depth completion benchmark\" is time-dependent; please state the submission date explicitly in the paper and note that leaderboard rankings can change.","section":"Abstract and Table I"}],"recommendation":"major_revision","confidential_remarks":"I see no evidence of circularity or misconduct; the KITTI leaderboard result is externally verified and the technical core is sound. My recommendation of major revision is driven by the NYUv2 label-quality concern and the lack of uncertainty quantification, both of which are addressable within the scope of a revision. If the authors add the requested raw-pixel evaluation and error bars, I would expect the paper to be acceptable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Pete,\n\nThis is a solid depth completion paper. The genuinely new piece is the factored guided convolution: split a spatially-variant convolution into a channel-wise spatially-variant stage plus a 1x1 cross-channel stage. That cuts memory by roughly 128x in their configuration, and makes it practical to apply dynamic, image-conditioned kernels at multiple scales in an encoder-decoder. The math in Eq. (8) is right, and the memory reduction is the real contribution. The KITTI result is credible—official test server, ranked first when submitted—and the ablations are properly done: guided convolution beats addition and concatenation, the decoder-to-encoder fusion helps, and multi-stage fusion helps over single-stage.\n\nSoft spots. The main one is the NYUv2 evaluation. Like most prior work, they use the official colorization-inpainted ground truth, and the method's RGB-conditioned kernels are exactly the sort of model that could exploit the smooth structure of inpainted depths. The stress-test note is fair: an evaluation against raw valid Kinect pixels would be needed to confirm that the 0.101 RMSE edge over DeepLiDAR is real geometry and not a fit to inpainting artifacts. That said, the same caveat applies to all the competing methods on NYUv2, and the KITTI benchmark is unaffected. Minor issues: no code at submission (they promise it) and no repeated-run statistics. Neither is unusual for this field.\n\nOverall: the central KITTI claim holds up, the NYUv2 claim is standard but worth a pointed question to the authors, and the factorization is a neat technique that others might reuse. It deserves a serious referee round and a conditional acceptance is reasonable.","headline":"Solid, benchmark-driven depth completion paper with a genuinely useful memory-saving guided convolution; KITTI claims credible, NYUv2 numbers rest on the field's questionable inpainted ground truth.","tokens_in":18709,"tokens_out":3082,"would_cite":true,"duration_ms":31941,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Learning image-conditioned, spatially-variant kernels yields state-of-the-art depth completion on NYUv2 and KITTI.","keywords":["depth completion","guided image filtering","spatially-variant kernels","multi-modal fusion","encoder-decoder network","LiDAR","KITTI benchmark","NYUv2"],"falsifier":"Measure the method's edge-region error separately using high-quality dense depth from a different sensor (e.g., active stereo or structured-light scan) on a subset of KITTI/NYUv2 scenes. If the guided kernels' advantage over concatenation disappears or reverses on those true edge pixels, the reported gains are largely an artifact of smoothed or sparse supervision.","tokens_in":1601,"feed_emoji":"🚗","tokens_out":6471,"duration_ms":89325,"temperature":0.7,"pith_summary":"The paper proposes a depth completion network that fuses a sparse LiDAR depth map with a synchronized RGB image by learning to generate convolution kernels from the image, rather than simply concatenating or adding the two modalities. These kernels are content-dependent and spatially-variant, so different image regions receive different filters. To make this computationally feasible, the convolution is factorized into a channel-wise spatially-variant stage and a $1 \\times 1$ cross-channel stage, cutting GPU memory by roughly two orders of magnitude. The authors report that this design outperforms state-of-the-art methods on the NYUv2 indoor dataset and ranked 1st on the KITTI depth completion benchmark at the time of submission.","feed_headline":"Learned guided kernels rank first on KITTI depth completion","feed_subtitle":"Image-dependent filters replace simple fusion, beating prior depth completion on indoor and outdoor benchmarks.","key_machinery":"The guided convolution module: a kernel-generating layer (KGL) produces a full set of spatially-variant weights from RGB features, then these weights are applied to depth features. To control memory, the paper factorizes the operation into a channel-wise spatially-variant convolution (per-channel $K \\times K$ kernels) followed by a $1 \\times 1$ cross-channel convolution whose weights are generated from a globally pooled image feature. This reduces per-pixel complexity from $O(M \\cdot K^2)$ to $O(M + K^2)$ and memory by a factor close to $\\frac{1}{N} + \\frac{1}{K^2 \\cdot H \\cdot B}$, enabling fusion at multiple encoder stages.","core_discovery":"The central claim is that feature fusion for depth completion should be guided by spatially-variant, image-dependent kernels rather than naive concatenation or addition. The paper argues that a network-generated kernel, conditioned on the RGB guidance image, can align depth features with object boundaries and scene content, and that the proposed factorization makes multi-stage fusion practical. The evidence is benchmark results: lower RMSE than existing published methods on the KITTI test set (736.24 mm versus 758.38 for DeepLiDAR) and the best RMSE on NYUv2 with both 200 and 500 samples.","pith_inferences":["An untested consequence is that the same factorization should transfer to other tasks with heterogeneous inputs, such as RGB-guided semantic segmentation of sparse point clouds or image-guided upsampling of low-resolution depth, where spatially-variant fusion is currently done by concatenation.","Because the generated kernels are conditioned purely on RGB features, the method implicitly learns to associate depth discontinuities with image edges; a small image-domain adversarial perturbation could therefore flip the kernels and corrupt the completed depth, a concern for safety-critical deployment.","The reported memory reduction suggests the guided convolution could be applied at higher resolutions or with more fusion stages than the paper explores; testing on higher-resolution inputs is a natural extension.","The method's robustness to point density implies it could adapt to cheaper, sparser sensors such as 16-line LiDAR with modest performance drop, but this is not directly tested on real 16-line data."],"forward_implications":["On the KITTI test set, the method achieves RMSE 736.24 mm, better than DeepLiDAR (758.38), CSPN (1019.64), Sparse-to-Dense (814.73), DDP (832.94), NConv-CNN (829.98), and RGB-certainty (772.87).","On NYUv2, it reaches RMSE 0.101 with 500 samples and 0.142 with 200 samples, improving over all compared methods on all five metrics.","Ablation studies show that replacing guided convolution with addition raises RMSE by 31.59 mm on the KITTI validation set, and concatenation raises it by 24.35 mm, so the guided module itself, not the network backbone, drives the gain.","Fusing GuideNet decoder features into DepthNet encoder features in multiple stages outperforms early or late fusion, and keeping only the first or last guided stage underperforms multi-stage guidance.","The model generalizes across LiDAR densities, lighting and weather conditions in Virtual KITTI, and from NYUv2 to SUN RGBD datasets with a different sensor."],"supporting_citations":[{"why":"Guided image filtering: supplies the inspiration of generating weights from the guidance image.","marker":"[13]"},{"why":"Dynamic Filtering Network: antecedent for content-dependent, spatially-variant kernels.","marker":"[45]"},{"why":"MobileNets: source of the depthwise-factorization idea used to split the convolution.","marker":"[18]"},{"why":"Uhrig et al.: introduces the KITTI depth completion dataset and benchmark used for evaluation.","marker":"[5]"},{"why":"DeepLiDAR: main outdoor baseline using surface-normal guidance, outperformed on KITTI RMSE.","marker":"[6]"},{"why":"Self-supervised sparse-to-dense: baseline method compared on KITTI and Virtual KITTI.","marker":"[7]"},{"why":"NConv-CNN: baseline with confidence propagation, compared on KITTI, NYUv2, and SUN RGBD.","marker":"[33]"},{"why":"CSPN: affinity-based propagation baseline, compared on KITTI and NYUv2.","marker":"[35]"}],"fun_headline_variants":["Guided kernels rank first on KITTI depth completion","Spatially-variant kernels improve depth completion","Image-guided fusion sets depth completion record","Learned guided filters top KITTI and NYUv2","Depth completion with guided kernels beats previous best"],"cache_read_input_tokens":20992,"weakest_assumption_plain":"The evaluation assumes the ground-truth depth used for training and testing is accurate and representative, but the NYUv2 depth maps are produced by colorization inpainting that smooths edges, and the KITTI ground truth is semi-dense registered LiDAR; if these labels are biased, the reported gains may not transfer to true dense depth.","fun_headline_variants_meta":{"raw":{"variants":["Guided kernels rank first on KITTI depth completion","Spatially-variant kernels improve depth completion","Image-guided fusion sets depth completion record","Learned guided filters top KITTI and NYUv2","Depth completion with guided kernels beats previous best"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000338,"raw_usage":{"total_tokens":1850,"prompt_tokens":911,"completion_tokens":939,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":527,"completion_tokens_details":{"reasoning_tokens":865}},"tokens_in":527,"tokens_out":939,"duration_ms":8841,"temperature":1.0,"reasoning_tokens":865,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T15:18:54.672911+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the method's edge-region error separately using high-quality dense depth from a different sensor (e.g., active stereo or structured-light scan) on a subset of KITTI/NYUv2 scenes. If the guided kernels' advantage over concatenation disappears or reverses on those true edge pixels, the reported gains are largely an artifact of smoothed or sparse supervision.","supporting_citations":[{"cited_title":"Guided image filtering,","cited_arxiv_id":null,"evidence_quote":"Guided image filtering: supplies the inspiration of generating weights from the guidance image."},{"cited_title":"Dynamic filter networks,","cited_arxiv_id":null,"evidence_quote":"Dynamic Filtering Network: antecedent for content-dependent, spatially-variant kernels."},{"cited_title":"Sparsity invariant cnns,","cited_arxiv_id":null,"evidence_quote":"Uhrig et al.: introduces the KITTI depth completion dataset and benchmark used for evaluation."},{"cited_title":"DeepLiDAR: Deep Surface Normal Guided Depth Prediction for Outdoor Scene from Sparse LiDAR Data and Single Color Image","cited_arxiv_id":"1812.00488","evidence_quote":"DeepLiDAR: main outdoor baseline using surface-normal guidance, outperformed on KITTI RMSE."},{"cited_title":"Depth estimation via affinity learned with convolutional spatial propagation network,","cited_arxiv_id":null,"evidence_quote":"CSPN: affinity-based propagation baseline, compared on KITTI and NYUv2."}],"review_version":1}