REVIEW 3 major objections 5 minor 39 references
When Proxy Prediction Becomes Equation Reconstruction: Diagnostics and Residual Learning for Factor-Derived Proxy Supervision
T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read When proxy targets come from known equations, keeping the formula as an anchor beats direct prediction.
desk verdict Useful diagnostic protocol, but the headline result rests on an information-mismatched baseline; fix that and this deserves a serious referee. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
RASPL combines a fixed formula anchor and an adaptive gate in log space: the predicted log value is clip(y_formula + $\alpha$ * delta, -5, 5), where delta is a contextual residual proposal and $\alpha$ = 1 - $\sigma$(logit) is a learned gate value conditioned on a reliability vector that encodes factor degradation. A residual penalty, q * $delta^{2}$, discourages large corrections when the gate favors retaining the formula. The formula anchor is the degraded RUSLE estimate A = R * K_tilde * LS * C, kept as a reference rather than treated as an ordinary input feature.
What would settle it
Compare RASPL against direct prediction using independently measured field soil-loss values under degraded K inputs; if direct prediction matches or exceeds RASPL's accuracy on observed measurements, the claim that formula preservation is the central design principle for robust proxy learning is falsified.
Extended reading notes
Core claim
The central claim is that in factor-derived proxy supervision, a model that predicts the proxy directly from the same factors can appear excellent while merely reconstructing the generating equation, which is not robustness. The paper's RASPL framework addresses this by defining the prediction as the degraded formula estimate plus an adaptively gated residual learned from context, in log space. Matched experiments on a RUSLE soil-loss proxy show RASPL-MLP-STATS reaches $R^{2}$_all 0.8343 versus 0.1017 for direct MLP, with the formula-feature baseline at 0.8188; the CNN variant gives the best Tail95 MAE and degradation robustness. The paper states that these results establish formula preservation as the central design principle for robust learning from factor-derived proxy targets.
Load-bearing premise
The evaluation assumes the target raster is exactly the RUSLE product of the four factors and that degrading only K is the right test of model quality; the paper explicitly does not assess validity against independently observed soil-loss measurements.
Editorial extensions
If this is right
- When the supervision target is generated by a known equation with possibly degraded inputs, models should anchor on the formula estimate rather than predict the target from scratch.
- Adding the formula as a standard input feature captures most of the accuracy gain, but explicit preservation further improves tail robustness and degradation robustness.
- Neighborhood statistics suffice for average accuracy, while convolutional context improves tail robustness, suggesting different encoder choices for different error regimes.
- Larger convolutional windows do not help; a 3x3 window is sufficient, so additional computational cost is not justified by accuracy gains.
Reading between the lines
- The same reconstruction-versus-robustness ambiguity likely applies to other factor-derived proxies such as evapotranspiration formulas, carbon-cycle models, or climate indices, so the diagnostic and residual-anchor design could transfer.
- Because the paper does not validate against independently observed soil-loss measurements, a natural testable extension is to run the matched RASPL-versus-direct comparison against field-measured erosion data.
- The K-missing stress test shows that even with strong contextual correction, complete factor absence leaves a high underprediction rate; imputing K from neighboring factors instead of a constant mean fallback could be a follow-up.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies factor-derived proxy supervision, where the learning target is constructed from a known scientific equation rather than from independent observations. Using a RUSLE-derived soil-loss proxy, the authors introduce a diagnostic framework with degraded-formula references, tree-based baselines, matched direct and formula-feature predictors, contextual ablations, tail metrics, and a degradation robustness score (DRS). They then propose RASPL, a residual method that keeps the degraded formula estimate as a prediction anchor and learns an adaptively gated contextual correction. The central empirical claim is that formula preservation substantially outperforms matched direct prediction and provides a better degradation--tail tradeoff than treating the formula estimate as an ordinary input feature, thereby establishing formula preservation as a design principle. The paper includes controlled degradation regimes (noise, coarsening, masking, and complete K removal), three seeds, matched comparisons, and a separate missing-K stress test, and it explicitly discloses that no validation against independent soil-loss measurements is performed.
Significance. The paper addresses a real and underappreciated failure mode: high accuracy on factor-derived proxy targets may simply reconstruct the generating equation. The proposed diagnostic framework and the residual anchored prediction idea are potentially useful to the scientific-ML community. The authors should be credited for the controlled degradation protocol, the matched formula-feature and direct baselines, the separate treatment of the missing-K regime, the pre-specified visualization tile selection rule, and the explicit disclosure that validity against independent soil-loss measurements is not assessed. However, the central empirical comparison is undermined by an information-access asymmetry in the statistical-encoder baseline, and several claims rely on small differences across only three seeds without reported variance. Because the core contribution is the formula-preservation principle, these issues are load-bearing and require revision before the results can be accepted as stated.
major comments (3)
- [Matched Direct and Formula-Feature Baselines; Table 2; Table 3] The headline comparison of RASPL-MLP-STATS against Direct-MLP-STATS is not information-matched. In the section on matched baselines, direct models are said to retain the same contextual encoders and regime-matched inputs as their RASPL counterparts, but for MLP-STATS the contextual encoder input is the reliability vector z described in Degraded-Factor Regimes. That vector contains K-derived statistics, a missingness indicator, a one-hot degradation tag, and the center-to-neighborhood-mean difference; it does not contain R, LS, or C. RASPL's formula branch injects R times K-tilde times LS times C, and the formula-feature baseline receives log(1 + A_formula), so both have access to the product that dominates target variation. Direct-MLP-STATS cannot see R, LS, or C, which makes its R2_all close to 0.10 in Table 2 an expected information floor rather than a measure of direct prediction quality. The abstract's claim that RASPL 'substantially outperforms matched direct prediction' therefore rests on an unfair baseline in the MLP comparison. A direct MLP baseline that receives the center-pixel R, LS, C, and degraded K channels (or the same four-factor window) must be added and compared; until then the 0.733 R2 gain in Table 3 cannot be attributed to formula preservation.
- [Results and Analysis, Formula Preservation; Tables 2 and 3] The key differences between RASPL and the formula-feature baseline are small: mean R2_all improvement of 0.015, Tail95 MAE improvement of 0.061, and only 8/9 cells improved in each metric, with Tail95 underprediction improved in only 5/9 cells. All results are macro-averaged over three seeds with no standard deviations, per-seed values, or significance tests reported. Given that the central design-principle claim hinges on these small advantages, the manuscript needs per-seed or interval estimates to show that the advantage is not within seed noise. This is particularly important because DRS is a normalized composite whose values depend on the comparison pool.
- [Diagnostic Criteria, Eq. (6); Tables 2 and 4] The degradation robustness score is a weighted, min-max normalized composite with weights 0.35, 0.25, 0.25, and 0.15, and the authors correctly note that DRS values from different normalization pools are not directly comparable. However, the paper's broader wording, such as 'stronger degradation robustness' and the ranking statements in the abstract, does not always carry this caveat. Since the component metrics are also reported, the ranking claims should be tied explicitly to those components, and ideally the sensitivity of the ranking to the chosen DRS weights should be examined, because the arbitrary weighting scheme could change the ordering of MLP-STATS versus CNN-RAW+STATS.
minor comments (5)
- [Problem Setup and Diagnostics, Eq. (4)] Under K full the formula reference attains R2_all = 1.0000 by construction, as the authors acknowledge; this is a sensible reconstruction diagnostic, but the wording in Table 1's discussion could more clearly separate this identity check from model-based performance.
- [Experimental Setup, Evaluation Regimes] The manuscript reports results only for three seeds and does not state whether a single data split is used across seeds or whether the split itself is reseeded; reporting the split construction would clarify the matched comparisons.
- [Results and Analysis, Missing-Factor Stress Test; Figure 2] The missing-K case shows that the Tail95 underprediction rate remains 0.9732 even for the best model, which is a strong caveat to the overall robustness claim; this caveat is disclosed, but it deserves a sentence in the abstract or conclusions so that the reader does not overgeneralize the robustness result.
- [Throughout] The paper would benefit from a data and code availability statement, since the reproducibility of the three-seed comparisons and the pre-specified visualization rule is otherwise hard to verify.
- [References] Some reference entries have inconsistent formatting, such as 'V .M., P.' and spacing in author initials; a final copyediting pass is needed.
Circularity Check
The central RASPL-vs-direct superiority claim reduces to an information-access difference: Direct-MLP-STATS is constructed without R, LS, or C, while RASPL's anchor injects their product.
-
self definitional
[Matched Direct and Formula-Feature Baselines; Formula Preservation results (Table 2)]
"The three models use the same neighborhood-statistics representation but differ in how they incorporate the degraded formula estimate: the direct model does not receive it ... The reliability vector z contains a mask or missingness indicator, a one-hot degradation tag, seven K-neighborhood statistics when K is observed, and the absolute center-to-neighborhood-mean difference |K̃_center − K̃_mean|."
Direct-MLP-STATS is defined to receive only z, which contains K-derived statistics and no R, LS, or C, whereas RASPL's anchor injects the product R × K̃ × LS × C. The reported mean ΔR² = 0.733 and Tail95 reduction 0.919 therefore measure the presence versus absence of the generating factors, not the benefit of formula preservation. The paper calls this comparison 'matched,' but the matching is only in encoder family; the input spaces differ by the entire formula product. Thus the headline result is forced by the way the baseline is defined.
full rationale
The paper's own decomposition is mostly honest: the formula reference attaining R2=1.0000 under K full is explicitly labeled as a reconstruction benchmark and is not presented as a learned result. The RASPL-versus-formula-feature comparison is information-matched and shows a small, mostly consistent improvement, so formula preservation has some independent content. However, the headline 'RASPL substantially outperforms matched direct prediction' rests on Table 2's MLP comparison, where the direct baseline is constructed without R, LS, or C while RASPL's anchor supplies their product. That gain is an input-access artifact, making the central superiority claim reduce by construction. The CNN comparison does provide all four factor channels, but the abstract's 0.733 R2 gain is taken from the MLP table. No load-bearing self-citation chain or imported uniqueness theorem appears in the paper.
Assumptions & free parameters
free parameters (4)
- Loss weights lambda_pos, lambda_tail, lambda_res =
0.5, 1.0, 0.01
- DRS component weights =
0.35, 0.25, 0.25, 0.15
- Tail underprediction threshold =
0.8 (A < 0.8 A)
- K-degradation severities =
noise std 0.20, block 8x8, mask rate 50%
assumptions (4)
- domain assumption The target raster A exactly equals the product R*K*LS*C computed from the factor maps.
- domain assumption Only the K factor is degraded; R, LS, and C remain accurate at all pixels.
- domain assumption The reliability vector z is sufficient for gating and residual correction.
- domain assumption Log-space mean-squared-error training with weighted tails is an appropriate objective for this proxy.
invented entities (1)
-
Degradation robustness score (DRS)
Cite this review
Pith. "Pith review of When Proxy Prediction Becomes Equation Reconstruction: Diagnostics and Residual Learning for Factor-Derived Proxy Supervision." pith.science (2026). https://pith.science/paper/2V75BJSZ
@misc{pith2026260804393,
author = {Pith},
title = {Pith review of: When Proxy Prediction Becomes Equation Reconstruction: Diagnostics and Residual Learning for Factor-Derived Proxy Supervision},
year = {2026},
howpublished = {\url{https://pith.science/paper/2V75BJSZ}},
note = {Machine review of arXiv:2608.04393}
}
abstract
Scientific machine learning often relies on proxy targets computed from known domain factors when direct observations are limited. When those same factors are used as model inputs, however, high predictive accuracy may reflect reconstruction of the proxy-generating equation rather than robustness to degraded factor information. We study this problem in RUSLE-derived soil-loss proxy prediction under controlled degradation of the soil-erodibility factor $K$. We introduce a diagnostic framework that combines degraded-formula references, classical tree-based baselines, matched direct and formula-feature predictors, contextual ablations, tail-error analysis, and degradation robustness scoring. We then propose RASPL, a formula-preserving residual framework that retains the degraded formula estimate as the prediction anchor and learns an adaptively gated contextual correction. RASPL substantially outperforms matched direct prediction and provides stronger degradation and tail robustness than treating the formula estimate as an ordinary input feature. Within RASPL, a compact statistical encoder achieves the highest macro-averaged $R^2$ and lowest computational cost, whereas a convolutional encoder achieves the strongest degradation robustness and lowest Tail95 mean absolute error (MAE). These results establish formula preservation as the central design principle for robust learning from factor-derived proxy targets.
Figures
Reference graph
Works this paper leans on
-
[1]
Benavidez, R. and Jackson, B. and Maxwell, D. and Norton, K. , title =. Hydrology and Earth System Sciences , volume =. 2018 , doi =
work page 2018
-
[2]
Environmental Science & Policy , volume =
The new assessment of soil loss by water erosion in Europe , author =. Environmental Science & Policy , volume =. 2015 , doi =
work page 2015
-
[3]
International Journal of Digital Earth , year =
Li, Pingheng and Tariq, Aqil and Li, Qingting and Ghaffar, Bushra and Farhan, Muhammad and Jamil, Ahsan and Soufan, Walid and El Sabagh, Ayman and Freeshah, Mohamed , title =. International Journal of Digital Earth , year =
-
[4]
Yuankai Ge and Longlong Zhao and Jinsong Chen and Xiaoli Li and Hongzhong Li and Zhengxin Wang and Yanni Ren , title =. Land , volume =
-
[5]
and Kalopesa, Eleni and Zalidis, George C
Samarinas, Nikiforos and Tsakiridis, Nikolaos L. and Kalopesa, Eleni and Zalidis, George C. , TITLE =. Land , VOLUME =. 2024 , NUMBER =
work page 2024
-
[6]
Hlal, Mohammed and El Monhim, Bilal and Chenal, Jérôme and Munyaka, Jean-Claude Baraka and Azmi, Rida and Sbai, Abdelkader and Cwick, Gary and Hichou, Badr Ben , TITLE =. Water , VOLUME =. 2025 , NUMBER =
work page 2025
-
[7]
Environmental Systems Research , year =
Abiye, Wudu and Dengiz, Orhan , title =. Environmental Systems Research , year =
-
[8]
Kantoush and Tetsuya Sumi , title =
Asma Bouamrane and Hamouda Boutaghane and Ali Bouamrane and Noura Dahri and Habib Abida and Mohamed Saber and Sameh A. Kantoush and Tetsuya Sumi , title =. International Journal of Sediment Research , volume =
Show all 39 references
-
[9]
Frontiers in Environmental Science , VOLUME=
Tadele Bedo Gelete and Pernaidu Pasala and Nigus Gebremedhn Abay and Gezahegn Weldu Woldemariam and Kalid Hassen Yasin and Erana Kebede and Ibsa Aliyi , TITLE=. Frontiers in Environmental Science , VOLUME=
-
[10]
Land Degradation & Development , volume =
Momeni Damaneh, Javad and Safdari, Ali Akbar and Azarnejad, Nazanin and Ghorbani, Majid and Panahi, Fatemeh and Afzali, Sayed Fakhreddin and Loppi, Stefano , title =. Land Degradation & Development , volume =. 2025 , publisher =
2025
-
[11]
, title =
Tkeshelashvili, N. , title =. Journal of Geography, Environment and Earth Science International , volume =. 2024 , publisher =
2024
-
[12]
Journal of Water and Climate Change , volume =
Olii, Muhammad Ramdhan and Zailani Olii, Abdul Kadir and Olii, Aleks and Djau, Rahman Abdul and Mokoagow, Muhamad Alfaikar and Kironoto, Bambang Agus and Bachtiar, Bachtiar and Olii, Rizky Selly Nazarina and Pakaya, Ririn , title =. Journal of Water and Climate Change , volume...
2025
-
[13]
and Ghadah Aldehim and Noha Negm and S
Priyadharshini V.M. and Ghadah Aldehim and Noha Negm and S. Subathradevi , title =. Journal of South American Earth Sciences , volume =. 2025 , publisher =
2025
-
[14]
Brian and Aslam, Rana Waqar and Rebouh, Nazih Y
Islam, Fakhrul and Bibi, Tahmina and Rehman, Nazir Ur and Davis, J. Brian and Aslam, Rana Waqar and Rebouh, Nazih Y. and Elmannai, Hela and Tariq, Aqil , title =. Earth Surface Processes and Landforms , volume =. 2025 , publisher =
2025
-
[15]
Sensors , volume =
Zhao, Shengyu and Tu, Kaiwen and Ye, Shutong and Tang, Hao and Hu, Yaocong and Xie, Chao , title =. Sensors , volume =. 2023 , number =
2023
-
[16]
and Mathis, Mark and Brumby, Steven P
Karra, Krishna and Kontgis, Caitlin and Statman-Weil, Zoe and Mazzariello, Joseph C. and Mathis, Mark and Brumby, Steven P. , booktitle=. Global land use / land cover with Sentinel 2 and deep learning , year=
-
[17]
and Ermon, Stefano , title =
Cong, Yezhen and Khanna, Samar and Meng, Chenlin and Liu, Patrick and Rozi, Erik and He, Yutong and Burke, Marshall and Lobell, David B. and Ermon, Stefano , title =. 2022 , booktitle =
2022
-
[18]
, title =
Fuller, Anthony and Millard, Koreen and Green, James R. , title =. 2023 , booktitle =
2023
-
[19]
2023 , booktitle =
Cepeda, Vicente Vivanco and Nayak, Gaurav Kumar and Shah, Mubarak , title =. 2023 , booktitle =
2023
-
[20]
2019 , issn =
Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations , journal =. 2019 , issn =. doi:https://doi.org/10.1016/j.jcp.2018.10.045 , url =
2019 doi
-
[21]
2022 , issue_date =
Willard, Jared and Jia, Xiaowei and Xu, Shaoming and Steinbach, Michael and Kumar, Vipin , title =. 2022 , issue_date =. doi:10.1145/3514228 , journal =
2022 doi
-
[22]
Nature Reviews Physics , volume =
Physics-informed machine learning , author =. Nature Reviews Physics , volume =. 2021 , doi =
2021
-
[23]
2018 , volume =
Yoon, Jinsung and Jordon, James and van der Schaar, Mihaela , booktitle =. 2018 , volume =
2018
-
[24]
International Conference on Learning Representations , year =
Benchmarking Neural Network Robustness to Common Corruptions and Perturbations , author =. International Conference on Learning Representations , year =
-
[25]
Journal of Big Data , volume =
A survey on Image Data Augmentation for Deep Learning , author =. Journal of Big Data , volume =. 2019 , doi =
2019
-
[26]
Huber , title =
Peter J. Huber , title =. The Annals of Mathematical Statistics , number =. 1964 , doi =
1964
-
[27]
Nature Communications , volume =
An assessment of the global impact of 21st century land use change on soil erosion , author =. Nature Communications , volume =. 2017 , doi =
2017
-
[28]
Proceedings of the 38th International Conference on Machine Learning , pages =
Delving into Deep Imbalanced Regression , author =. Proceedings of the 38th International Conference on Machine Learning , pages =. 2021 , editor =
2021
-
[29]
Machine Learning , volume =
Imbalanced regression and extreme value prediction , author =. Machine Learning , volume =. 2020 , publisher =. doi:10.1007/s10994-020-05900-9 , url =
2020 doi
-
[30]
Rivista Italiana di Economia Demografia e Statistica , volume =
Methods for constructing composite indices: One for all or all for one? , author =. Rivista Italiana di Economia Demografia e Statistica , volume =
-
[31]
A knowledge-based approach to the statistical mapping of climate , journal =
Daly, Christopher and Gibson, Wayne and Taylor, George and Johnson, Gary and Pasteris, Paul , year =. A knowledge-based approach to the statistical mapping of climate , journal =
-
[32]
Soil Survey Geographic (SSURGO) Database , year =
-
[33]
2022 , howpublished =
2022
-
[34]
2021 , howpublished =
2021
-
[35]
Shortcut Learning in Deep Neural Networks , journal =
Geirhos, Robert and Jacobsen, J. Shortcut Learning in Deep Neural Networks , journal =. 2020 , doi =
2020
-
[36]
arXiv preprint arXiv:2309.15284 , year =
Long, Keke and Sheng, Zihao and Shi, Haotian and Li, Xiaopeng and Chen, Sikai and Ahn, Sue , title =. arXiv preprint arXiv:2309.15284 , year =. 2309.15284 , archivePrefix =
-
[37]
and Keller, S
Bock, Frederic E. and Keller, S. Hybrid Modelling by Machine Learning Corrections of Analytical Model Predictions towards High-Fidelity Simulation Solutions , journal =. 2021 , doi =
2021
-
[38]
Frontiers in Environmental Science , VOLUME=
İpek, Ahmet Faruk and Kahya, Ercan , TITLE=. Frontiers in Environmental Science , VOLUME=. 2026 , DOI=
2026
-
[39]
Earth Science Informatics , year =
Zeghmar, Amer and Mokhtari, Elhadj and Marouf, Nadir , title =. Earth Science Informatics , year =. doi:10.1007/s12145-024-01305-7 , issn =
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.