{"id":"0f3107e4-7053-4032-a3d0-7dc2e4a092f9","arxiv_id":"2506.04758","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"An additive SSIM loss, with weights selected on the KITTI test split, yields small depth-error improvements over the multiplicative SSIM baseline in unsupervised monocular depth estimation.","lead":"This paper proposes replacing the multiplicative SSIM term in the photometric loss for unsupervised monocular depth estimation with an additive combination of its luminance, contrast and structure components, and swapping nearest-neighbor depth upsampling for sub-pixel convolution. The authors report that on KITTI the new loss improves squared-relative error and RMSE slightly, though the stated 'remarkable' gain is actually small and not consistent across all metrics.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Weight selection on the Eigen test split makes the reported gains unverifiable; without a validation split or repeated seeds, the claimed improvement may be selection bias.","rationale":"The stress-test identifies the same load-bearing concern as the reader: the loss weights are selected directly on the test split that is later used to report the headline results. This is the most threatening issue because the paper's contribution is entirely empirical — the theoretical motivation ('smoother gradients') is qualitative, and the claimed quantitative gains are small and not uniform across metrics. The grid search over a large set of hyperparameters, with only one seed and no validation set, means the reported 'best' result is expected to be optimistic even if the additive SSIM loss has no real benefit. The paper's own limitation section concedes that the improvements are small, which amplifies the risk that the observed differences come from selection noise rather than a genuine effect. A held-out validation split and repeated-seed evaluation, as proposed in the concrete test, would directly resolve whether the central claim survives. Because the reader's verdict already calls for exactly this evidence, no change to the verdict is needed; the concern reinforces the conditional status rather than overturning it.","tokens_in":9741,"tokens_out":4554,"duration_ms":54977,"concrete_test":"Split the KITTI Eigen test split into a validation portion (e.g., 20% of the sequences) and a held-out test portion (the remaining 80%). Run the same grid search as Table 2 using the validation portion to select the best weights, then evaluate the selected model on the held-out test portion. If the selected weights do not improve over the standard Monodepth2 baseline on the held-out portion, or if the optimal weights differ substantially from those reported, the central claim is undermined. Additionally, repeat the chosen configuration with 3 random seeds to assess whether the reported metric differences exceed run-to-run variance.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim depends on the weights reported in Table 5 (w1=0.4, wl=0.5, wc=0.5, ws=0.7). These weights are chosen as the best configuration from Table 2, whose metrics are computed on the Eigen test split — the same split later used to report the final results in Tables 3, 4, and 5. The grid search in Table 2 covers roughly 54 combinations, so even with no true overfitting, the best-of-many test metric is optimistically biased because the test set is used for model selection. The paper provides no validation split, no repeated trials, and no error bars; the reported differences between configurations are small (e.g., AbsRel 0.114 vs 0.116, SqRel 0.816 vs 0.865), close to the noise floor of a single training run with fixed random seed. The paper's own Limitation section admits the gains are small: RMS improves by only 0.7% and SqRel by 6.4% relative to the baseline, and Tables 3/4 show that sub-pixel convolution improves SqRel and RMSlog but worsens AbsRel and δ1. Therefore the assertion that the additive SSIM loss 'remarkably outperform[s]' the baseline is not supported by a sound evaluation protocol; the load-bearing assumption is that the weights selected on the test split transfer to unseen data, and that assumption is currently untested.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a modified SSIM loss for unsupervised monocular depth estimation. Instead of the standard multiplicative combination of luminance, contrast, and structure terms, Eq. (9) combines them additively, and Eq. (11) fuses this additive SSIM with an MAE term using weights w1, wl, wc, ws. The paper also replaces nearest-neighbor upsampling with sub-pixel convolution in the multi-scale decoder. Experiments on the KITTI Eigen split with Monodepth2 and GVO report improvements on SqRel, RMS, and RMSlog, while AbsRel and δ1 are often unchanged or worse. The central claim is that the additive SSIM leads to smoother gradients and higher performance, and that the optimized loss plus sub-pixel convolution 'remarkably outperform[s]' the baseline.","tokens_in":10039,"tokens_out":5032,"duration_ms":56058,"significance":"If validated, the proposed additive SSIM is a simple, inexpensive modification that could transfer to many self-supervised depth pipelines. The paper has several strengths: it conducts an extensive ablation over many weight combinations in Tables 1 and 2, tests two different backbone methods (Monodepth2 and GVO), reports results at multiple decoder resolutions in Table 4, and includes an explicit limitation section. However, the current evidence is not yet convincing because the final reported weights are selected on the same test split used for evaluation, no error bars or repeated runs are provided, and the paper's own numbers show only small and partly mixed improvements. The central empirical claim therefore needs stronger support before the conclusion can be accepted.","major_comments":[{"comment":"The evaluation protocol is circular in the load-bearing step: the weights w1=0.4, wl=0.5, wc=0.5, ws=0.7 used in Table 5 are the best configuration found by the grid search in Table 2, and Table 2 reports metrics on the same Eigen test split that is later used for the final comparison. Selecting the best of roughly twenty configurations on the test set introduces optimistic bias that is not accounted for. The paper should either perform weight selection on a held-out validation split and then report test results, or show that the selected weights transfer across multiple seeds and, ideally, to a different dataset or split.","section":"§4.1, Tables 2 and 5"},{"comment":"All experiments use a single fixed random seed, and no error bars or repeated runs are reported. The differences between many configurations in Table 2 are extremely small (e.g., AbsRel 0.114 vs 0.115, RMS 4.822 vs 4.823, RMSlog 0.191 vs 0.192). Without an estimate of training run-to-run variance, the ranking of configurations and even the claimed improvement over the baseline cannot be distinguished from noise. At minimum, three or more seeds per configuration and mean±std reporting are needed for the principal comparisons.","section":"§4, Implementation details; Tables 2 and 3"},{"comment":"The abstract's claim that the method can 'remarkably outperform' the baseline is not supported by the paper's own numbers. The Limitation section reports improvements of only 0.7% in RMS (4.856 to 4.822) and 6.4% in SqRel (0.868 to 0.816), and Table 5 shows that Monodepth2 + SSC leaves AbsRel unchanged at 0.115 while worsening δ1 (0.863 vs 0.877) and δ2 (0.957 vs 0.961) relative to Monodepth2. The paper should temper the wording to 'improves selected error metrics' and discuss the accuracy metrics that are not improved.","section":"§5, Limitation paragraph; Abstract; Table 5"},{"comment":"The central motivation that additive combination yields 'smoother gradients' and avoids convergence problems is not directly tested. Figure 1 illustrates the functional forms, and Eq. (9) gives constant per-component gradients, but the paper provides no gradient statistics, loss curves, or convergence diagnostics during training. The empirical results could still support the proposal, but the gradient-smoothness mechanism remains an unverified hypothesis rather than a demonstrated property of the training process.","section":"§3.2 and Figure 1"}],"minor_comments":[{"comment":"The manuscript contains several typos and grammatical errors, including 'comparied' (Section 1), 'Comparision' (Section 4), 'the the weights' (Section 4.2), 'futher' (Section 5), and 'Entensive' (Section 6). A careful proofreading pass is needed.","section":"Throughout"},{"comment":"The notation for the structure component is confusing: S is stated to lie in [−1,1], but Eq. (9) writes (1 − 1/2(1+S)), which equals (1−S)/2. Defining a normalized structure term explicitly would make the relationship between Eqs. (8) and (9) clearer.","section":"Eqs. (8) and (9)"},{"comment":"The reported numbers for Monodepth2 [7] differ between Table 1 and Table 5 (e.g., δ2 0.959 vs 0.961, δ3 0.981 vs 0.982). The authors should clarify whether these are different runs or different published sources, and use consistent baseline values throughout.","section":"Tables 1 and 5"},{"comment":"The text refers to 'our model' when describing GVO + SSC, but GVO is the method of [3]; the contribution here is limited to the loss and the upsampling module. Please clarify the scope of the proposed changes for each backbone.","section":"§4.2"}],"recommendation":"major_revision","confidential_remarks":"The core idea is incremental but potentially useful; the main barrier is the evaluation protocol. If the authors can redo the weight selection on a held-out validation split, add repeated-seed error bars, and temper the abstract's overclaim, a revision could become acceptable. The manuscript currently reads more like a workshop or short paper than a journal article, so the authors should also consider whether the contribution is substantial enough for the target venue."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the paper proposes an additive form of SSIM loss—combining luminance, contrast, and structure terms by addition rather than multiplication—plus a sub-pixel convolution upsampling module for unsupervised monocular depth. The idea is simple, easy to graft onto existing pipelines, and the paper honestly reports that the gains are small. The problem is the evaluation. The loss weights that produce the headline numbers were chosen by grid-searching on the KITTI Eigen test split itself, and the final comparison is made on that same split. There is no validation set, no repeated runs, no error bars. The differences between the best and runner-up configurations are often tiny (AbsRel 0.114 vs 0.116, SqRel 0.816 vs 0.826), so without knowing run-to-run variance, the 'best' configuration is partly a product of noise selection. That makes the claim of improvement unverifiable as-is.\n\nWhat's good: the additive SSIM is a genuinely plausible alternative to the standard multiplicative form, and the weight sensitivity study (Tables 1-2) is a useful empirical observation even if flawed. The paper is clearly written, and the limitation section is candid—RMS improves 0.7%, SqRel 6.4%. The sub-pixel convolution swap is a known trick, but it's applied sensibly.\n\nWhere it's soft: besides the selection-bias issue, the abstract's 'remarkably outperform' is not supported by the paper's own numbers. Monodepth2+SSC matches or barely improves AbsRel (0.115) while delta_1 accuracy drops from 0.877 to 0.863; the accuracy metrics get worse on multiple rows. The claim in the limitation section that the proposed SSIM is better 'in every indicator' is contradicted by those same tables. The gradient smoothing argument is illustrated with toy examples but not formalized—fine for a heuristic, but not a proof. No code or data are released.\n\nBottom line: the central idea has enough plausibility to warrant a deeper look, but the current evidence is not persuasive on its own. The right fix is a clean validation split for weight selection, multiple seeds, and an evaluation on at least one more dataset. I'd recommend sending it to peer review—the idea deserves referee time—but I'd expect major revision and re-analysis before acceptance. I wouldn't cite it for the numbers; I might cite it as an example of an alternative SSIM loss if I were surveying loss functions.","headline":"Plausible additive-SSIM loss for unsupervised depth, undermined by weight selection on the test split and small, inconsistent gains.","tokens_in":10558,"tokens_out":3089,"would_cite":false,"duration_ms":35371,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Replacing multiplicative SSIM with an additive form smooths gradients and improves unsupervised monocular depth estimation on KITTI.","keywords":["monocular depth estimation","unsupervised learning","SSIM loss","photometric consistency","sub-pixel convolution","KITTI dataset","gradient smoothness","self-supervised depth"],"falsifier":"Select the loss weights on a validation split disjoint from the KITTI test set, then evaluate on that test set: if additive SSIM no longer outperforms the multiplicative baseline by a comparable margin, the published gains are an artifact of choosing weights on the test set. Optionally, measure gradient-norm variance of the two loss forms at fixed random weights to test the smoother-gradient mechanism directly.","tokens_in":9514,"feed_emoji":"📏","tokens_out":10218,"duration_ms":94414,"temperature":0.7,"pith_summary":"The paper seeks to establish that the conventional way of building an SSIM-based photometric loss — multiplying the luminance, contrast, and structure terms together — is a poor training signal for unsupervised monocular depth estimation, and that adding the same three terms with tuned weights instead produces smoother gradients and better depth maps. The proposed additive SSIM is combined with mean absolute error into a single loss, and used to replace the photometric loss inside two existing models, Monodepth2 and GVO, trained on KITTI. With weights $w_1=0.4$, $w_l=0.5$, $w_c=0.5$, $w_s=0.7$ and sub-pixel convolution for upsampling, both models improve on squared relative error and RMSE while keeping absolute relative error roughly unchanged. If this holds, it means part of the gap between supervised and unsupervised depth learning can be closed by reshaping the loss rather than by adding network modules or geometric constraints.","feed_headline":"Additive SSIM loss beats multiplicative baseline on KITTI","feed_subtitle":"Weighted sums of luminance, contrast, and structure give smoother gradients and better KITTI depth scores","key_machinery":"The load-bearing mechanism is the additive SSIM loss, Eq. (9), and its MAE combination, Eq. (11). Where the standard SSIM multiplies luminance, contrast, and structure, so that the gradient shrinks wherever any factor saturates, the additive form is a weighted sum of distances-to-one for the three components, giving each a contribution to the gradient that is independent of the others' values. This is what the authors mean by smoother gradients: the loss surface no longer has flat plateaus that stall training. The second component is sub-pixel convolution, which rearranges channel dimensions into spatial dimensions to upsample the depth map, replacing nearest interpolation at each decoder scale; the paper reports that this gives consistent performance even at 1/16 resolution.","core_discovery":"The central discovery, stated on the paper's own terms, is that SSIM's three components — luminance $L$, contrast $C$, and structure $S$ — should be combined by addition rather than multiplication when SSIM is used as a training loss. The multiplicative form $1 - L^\\alpha C^\\beta S^\\gamma$ produces gradients whose size depends on the product's curvature, so large flat regions of the loss carry almost no learning signal; the additive form $SSIM_a = w_l(1-L) + w_c(1-C) + w_s(1 - \\tfrac{1}{2}(1+S))$ gives each component a gradient contribution that does not vanish when the others are near one. Combined with MAE as $L_a = w_1 MAE + w_l(1-L) + w_c(1-C) + w_s(1 - \\tfrac{1}{2}(1+S))$, the loss is optimized with grid-selected weights $w_1=0.4$, $w_l=0.5$, $w_c=0.5$, $w_s=0.7$. Plugging this loss, and sub-pixel convolution in place of nearest-neighbor upsampling, into Monodepth2 improves SqRel from 0.903 to 0.770 and RMS from 4.863 to 4.813 on the KITTI test split; the same recipe improves GVO from SqRel 0.787 to 0.747 and RMS 4.488 to 4.416. The authors explicitly note the gains are modest and that the $\\delta_1<1.25$ accuracy slightly drops with sub-pixel upsampling because the smoother depth lacks edge precision.","pith_inferences":["The loss weights were chosen by grid search on the same KITTI test split whose metrics are reported; a fairer evaluation would select weights on a separate validation split, and the current margins might shrink under that protocol.","The smoother-gradient claim is testable directly: at matched random initializations, the additive loss should show lower variance in gradient norms across pixels than the multiplicative loss; the paper does not report such a measurement.","Because sub-pixel upsampling blurs edges, a hybrid decoder — sub-pixel in smooth regions, nearest or learned upsampling near depth discontinuities — could recover the lost $\\delta_1$ accuracy while keeping the SqRel gains.","The same additive-SSIM idea could apply to other self-supervised photometric tasks (optical flow, video prediction), but the required weights may differ and the paper does not test them."],"forward_implications":["Substituting Eq. (9) into the photometric loss of an existing unsupervised depth model should improve SqRel and RMS without changing the network; the paper demonstrates this on Monodepth2 and GVO.","The weight ranking ($w_c=0.5$, $w_s=0.7$ versus $w_l=0.5$) indicates contrast and structure are the informative parts of SSIM for depth training, so future loss designs should weight them at least as heavily as luminance.","Sub-pixel convolution helps most on error metrics sensitive to large depth mistakes and slightly hurts the $\\delta_1$ accuracy; applications that need edge precision should weigh this trade-off.","The authors claim the additive loss is compatible with occlusion masking and pose refinement, so it can be dropped into other self-supervised SIDE pipelines without violating their assumptions."],"supporting_citations":[{"why":"Monodepth2 baseline whose photometric loss this paper modifies; the main comparison point.","marker":"[7]"},{"why":"Original SSIM definition whose multiplicative combination of luminance, contrast, and structure is replaced.","marker":"[24]"},{"why":"Source of the standard MAE+SSIM photometric loss form and the edge-aware smoothness term.","marker":"[13]"},{"why":"Sub-pixel convolution technique borrowed from super-resolution and used for depth upsampling.","marker":"[17]"},{"why":"GVO baseline; the second model the modified loss is plugged into.","marker":"[3]"},{"why":"Pioneering unsupervised depth-and-pose pipeline that established the photometric-loss paradigm.","marker":"[23]"},{"why":"The standard test split used for all depth evaluations, defining the reported metrics.","marker":"[5]"},{"why":"KITTI dataset from which training and evaluation data come.","marker":"[6]"}],"fun_headline_variants":["Summing SSIM parts beats multiplying for depth loss","Additive SSIM loss improves depth on KITTI","New SSIM loss: add luminance, contrast, structure","Smoother gradients from additive SSIM improve depth"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported gains come from weights selected by grid search on the same KITTI test split whose metrics are later reported as the results, and the assumption is that these test-chosen weights improve depth on new scenes and datasets rather than merely fitting that evaluation split.","fun_headline_variants_meta":{"raw":{"variants":["Summing SSIM parts beats multiplying for depth loss","Additive SSIM loss improves depth on KITTI","New SSIM loss: add luminance, contrast, structure","Smoother gradients from additive SSIM improve depth"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000392,"raw_usage":{"total_tokens":2119,"prompt_tokens":1059,"completion_tokens":1060,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":675,"completion_tokens_details":{"reasoning_tokens":995}},"tokens_in":675,"tokens_out":1060,"duration_ms":12383,"temperature":1.0,"reasoning_tokens":995,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T10:33:49.471943+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Select the loss weights on a validation split disjoint from the KITTI test set, then evaluate on that test set: if additive SSIM no longer outperforms the multiplicative baseline by a comparable margin, the published gains are an artifact of choosing weights on the test set. Optionally, measure gradient-norm variance of the two loss forms at fixed random weights to test the smoother-gradient mechanism directly.","supporting_citations":[{"cited_title":"IJRR (2013)","cited_arxiv_id":null,"evidence_quote":"KITTI dataset from which training and evaluation data come."},{"cited_title":"In: ICCV (2019) 12 Y","cited_arxiv_id":null,"evidence_quote":"Monodepth2 baseline whose photometric loss this paper modifies; the main comparison point."},{"cited_title":"IEEE T-PAMI42(10), 2624–2641 (2020)","cited_arxiv_id":null,"evidence_quote":"Source of the standard MAE+SSIM photometric loss form and the edge-aware smoothness term."},{"cited_title":"Pattern Recognition136, 109262 (2023)","cited_arxiv_id":null,"evidence_quote":"GVO baseline; the second model the modified loss is plugged into."},{"cited_title":"In: CVPR (2017)","cited_arxiv_id":null,"evidence_quote":"Pioneering unsupervised depth-and-pose pipeline that established the photometric-loss paradigm."}],"review_version":1}