REVIEW 2 major objections 5 minor 21 references
Deep Evidential Regression for Sparse Forest Height Estimation from Multimodal Satellite Imagery
T0 review · 2 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read Deep evidential regression matches a deterministic U-Net's forest-height accuracy while adding calibrated predictive uncertainty in a single forward pass.
desk verdict A sensible, honest adaptation of Deep Evidential Regression to a sparse forest-height mapping problem; the calibration claim has a loose end about interval construction that needs checking but does not sink the paper. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the Normal-Inverse-Gamma (NIG) evidential distribution, a conjugate prior over the mean and variance of the Gaussian likelihood assumed for the $\log(1+y)$-transformed target. The network outputs four parameters $(\gamma, \nu, \alpha, \beta)$ per pixel; $\gamma$ gives the predicted height, while $\beta/(\alpha-1)$ and $\beta/[\nu(\alpha-1)]$ give aleatoric and epistemic variance. The masked evidential loss restricts the NIG negative log-likelihood and its evidence regularizer to pixels with valid inventory labels. This object carries the whole argument: it turns one forward pass into both a point prediction and a Bayesian-style uncertainty estimate.
What would settle it
Retrain the same U-Net on the same TreeUQ split with a heavier-tailed error distribution in place of the bell-shaped one, and recompute the calibration curve. If the expected calibration error stays near 0.025 on the tallest-forest pixels, the Gaussian assumption is not decisive; if it changes sharply, that assumption is the load-bearing piece.
Extended reading notes
Core claim
The paper's central claim is that the evidential regression formulation—a Normal-Inverse-Gamma prior over the mean and variance of a Gaussian likelihood for the $\log(1+y)$-transformed tree height—can be trained on sparse, patchy inventory labels and still deliver usable predictive uncertainty at the pixel level. With the proposed masked evidential loss, only pixels carrying valid annotations contribute to training; the remaining pixels are ignored rather than treated as zeros. On the TreeUQ benchmark, the resulting DER U-Net achieves RMSE 5.91 ± 0.15 m versus 5.75 ± 0.09 m for the identical deterministic U-Net, with a calibration curve close to the diagonal and ECE 0.025. The paper also reports that predicted uncertainties, aggregated to 128-pixel patches, correlate with inventory-derived within-pixel tree-height variance, most strongly for epistemic uncertainty (r = 0.74), and that uncertainty is elevated near forest/non-forest transitions.
Load-bearing premise
The calibration result rests on the assumption that, after a log transform, the remaining errors in predicted tree height are bell-shaped and symmetric around the prediction; if real errors are skewed or heavy-tailed, the reported uncertainty calibration and the aleatoric/epistemic split may not hold.
Editorial extensions
If this is right
- Uncertainty-aware 10 m forest height maps for large regions can be produced in a single forward pass, since the evidential head adds no sampling or ensembling at inference time.
- The masked loss makes evidential training feasible at extreme label sparsity, so sparse field inventories can be used without imputing labels for unobserved pixels.
- The reported calibration (ECE 0.025) means users can treat prediction intervals as approximately reliable confidence statements about per-pixel height.
- Because uncertainty aligns with within-pixel tree-height variance (epistemic r = 0.74) and forest/non-forest boundaries, the maps carry signal about structural heterogeneity, not just noise.
- Predictive accuracy stays essentially on par with the deterministic baseline (RMSE 5.91 vs 5.75 m), so the uncertainty estimates are not bought with a large accuracy drop.
Reading between the lines
- A natural stress test is to replace the bell-shaped error assumption with a heavier-tailed evidential likelihood; if calibration on tall-forest pixels improves, the Gaussian assumption is the main residual limitation.
- The epistemic channel could be repurposed for active learning: acquire new field inventory in high-epistemic-uncertainty patches, which are also the patches with high within-pixel variance.
- The spurious background predictions the paper flags suggest a two-component model—forest/non-forest classifier plus height regressor—would likely remove most of the background error without changing the evidential machinery.
- Cross-region generalization is untested beyond Bavaria; if the same masked evidential training transfers to another Sentinel-1/Sentinel-2 geography, the single-pass uncertainty claim would be materially stronger.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper adapts Deep Evidential Regression (DER) to sparse forest height estimation from multimodal Sentinel-1/Sentinel-2 imagery on the TreeUQ benchmark. The authors introduce a masked evidential loss that restricts the evidential objective to pixels with valid tree inventory labels, train a U-Net with a ResNet-50 encoder and an evidential output head, and compare it against a deterministic U-Net trained with the same backbone and preprocessing. The main empirical claims are that the DER U-Net achieves accuracy comparable to the deterministic U-Net (RMSE 5.91±0.15 m vs. 5.75±0.09 m) while additionally providing well-calibrated predictive uncertainty, as summarized by an ECE of 0.025. The paper also reports patch-level correlations between predicted aleatoric/epistemic uncertainties and inventory-derived tree-height variance, and shows spatial uncertainty maps for a representative test patch.
Significance. If the uncertainty-calibration claim is supported, the paper provides a useful demonstration that uncertainty-aware forest height mapping can be obtained in a single forward pass from satellite imagery, which is relevant for large-scale ecological monitoring. The work is strengthened by the controlled comparison between deterministic and evidential U-Nets under identical preprocessing, masking, and optimization settings, by the use of multiple random seeds, and by the public release of code. The masked evidential loss is a simple but sensible adaptation to the sparse-supervision setting. The main weakness is that the calibration experiment, which is the central piece of evidence for the headline claim, is not specified precisely enough to determine whether the reported ECE actually validates the Student-t predictive distribution stated in Section 3.2. The absence of alternative uncertainty baselines (MC dropout, deep ensembles) limits the contextual framing but does not by itself invalidate the reported within-model calibration.
major comments (2)
- [Section 5, calibration paragraph] The calibration evaluation is the direct support for the paper's headline claim of well-calibrated uncertainty, but the manuscript does not specify the quantiles used to construct the prediction intervals. Section 3.2 defines the predictive distribution as Student-t with density St(y; γ, β(1+ν)/(αν), 2α), and for this distribution the quantiles are not determined by the predictive mean and total predictive uncertainty alone; they also require the degrees of freedom 2α and the scale parameter. If the intervals are computed as γ ± z_p √u²_pred with Gaussian quantiles z_p, then the reported ECE of 0.025 characterizes a Gaussian approximation rather than the model's stated predictive distribution. Please state the exact interval formula, including whether Student-t quantiles with df = 2α and scale √(β(1+ν)/(αν)) were used, and if the released code contains the computation, cite the relevant code location. If Gaussian quantiles were used, the calibration experiment needs to be rerun with the proper Student-t intervals before the uncertainty claim can be accepted.
- [Section 3.2, Eq. (1)] Equation (1) uses L_DER without defining its full form, in particular the weight of the evidence regularizer term from Amini et al. In the original DER objective the regularizer is multiplied by a scalar coefficient λ, and this coefficient controls the trade-off between likelihood fit and penalization of overconfident errors, directly influencing the resulting uncertainty calibration. The manuscript states in Section 4 that all hyperparameters are fixed but does not report λ anywhere. Please provide the explicit L_DER formula with the coefficient and the value used; otherwise the calibration results and the training procedure are not reproducible.
minor comments (5)
- [Section 5, Figure 2] The correlations between inventory-derived variance and predicted aleatoric uncertainty (r=0.53), epistemic uncertainty (r=0.74), and prediction error (r=0.58) are reported without confidence intervals, and the number of patches used in the aggregation is not stated. Given the spatial nature of the data, a 95% confidence interval or a test that accounts for spatial autocorrelation would make these descriptive associations more informative.
- [Section 5, Table 1] The claim that the deterministic and evidential U-Nets have comparable performance is based on overlapping standard deviations across seeds. Reporting a paired or unpaired significance test, or at least the per-seed differences, would strengthen the conclusion that including the evidential head does not substantially degrade accuracy.
- [Section 5, calibration paragraph] The paper reports an ECE of 0.025 but does not define how the ECE is computed for regression, including the number of confidence levels used, the binning strategy, or whether the intervals are evaluated in the log-transformed or original space. Please specify these details, as they affect the interpretation of the calibration curve.
- [Section 3.1, notation] The notation is confusing: bold x denotes a patch, then the text states that x and y refer to individual pixels. Use explicit pixel indices (e.g., x_ij and y_ij) throughout the problem formulation to avoid ambiguity, especially since the masked loss in Eq. (1) already uses such indices.
- [Section 6, limitations] The paper responsibly acknowledges that the Gaussian likelihood assumption may not fully capture the distribution of forest heights. A brief sensitivity analysis, such as comparing the reported calibration with an alternative heavy-tailed likelihood or a post-hoc recalibration, would help quantify how much the calibration result depends on this assumption.
Circularity Check
No significant circularity: the paper's uncertainty evaluation is empirical, and its eventive model is from an external framework (Amini et al.), not derived from the paper's own claims.
full rationale
The paper's central claim—that DER achieves predictive performance comparable to a deterministic U-Net while providing well-calibrated uncertainty—rests on an empirical comparison of two models on the TreeUQ benchmark. The evidential framework, including the NIG prior and the evidential loss, is adopted from Amini et al. (NeurIPS 2020) and cited as an external method. The paper's own contribution, the masked evidential loss, is a straightforward modification that averages the existing DER loss only over valid pixels; it does not redefine any output or target in terms of the prediction itself. Calibration is evaluated by constructing prediction intervals from the predictive mean and total predictive uncertainty and then comparing empirical coverage to nominal levels on the held-out test set. This is a direct empirical probe, not a tautology. The only self-citation is reference [1] to the TreeUQ benchmark, which is a dataset contribution by the same group; using one's own benchmark for evaluation is standard and does not make the derivation circular, because the benchmark labels are external measurements and the train/test split is geographically disjoint. The paper's Section 6 acknowledges limitations, including the Gaussian likelihood assumption and the heuristic nature of the aleatoric/epistemic split; these are honest caveats rather than attempts to force a conclusion. The skeptic's concern about Gaussian versus Student-t intervals in the calibration computation is a potential experimental-validity issue (correctness risk), but it is not circularity: it does not reduce the reported ECE to the model's own definition or to a fitted parameter. Therefore, no circular step can be identified with specific reduction.
Assumptions & free parameters
free parameters (1)
- DER evidence regularizer weight =
not reported
assumptions (4)
- standard math Normal-Inverse-Gamma conjugate prior and DER loss from Amini et al. [2] are adopted without modification.
- domain assumption Tree heights follow a Gaussian distribution after log(1+y) transformation.
- domain assumption Labels missing according to the validity mask can be ignored during training without introducing bias.
- domain assumption TreeUQ inventory-derived mean heights and within-pixel variances are reliable ground truth.
Cite this review
Pith. "Pith review of Deep Evidential Regression for Sparse Forest Height Estimation from Multimodal Satellite Imagery." pith.science (2026). https://pith.science/paper/GJ2QBDXR
@misc{pith2026260806406,
author = {Pith},
title = {Pith review of: Deep Evidential Regression for Sparse Forest Height Estimation from Multimodal Satellite Imagery},
year = {2026},
howpublished = {\url{https://pith.science/paper/GJ2QBDXR}},
note = {Machine review of arXiv:2608.06406}
}
read the original abstract
Accurate estimation of forest height from satellite imagery is essential for applications such as carbon accounting, biodiversity monitoring, and ecosystem management. While recent deep learning approaches provide accurate predictions, they typically do not quantify predictive uncertainty. This limitation is particularly relevant in geospatial settings characterized by sparse supervision and geographic distribution shift. In this work, we investigate Deep Evidential Regression (DER) for forest height estimation on the TreeUQ benchmark, a large-scale dataset designed for the joint estimation of tree count and average tree height at 10 m resolution, based on Sentinel-1/-2 data as well as tree inventory data over the federal state of Bavaria. To account for the extreme label sparsity of the tree inventory data, we introduce a masked evidential loss for dense geospatial prediction. Using a U-Net architecture with multimodal Sentinel-1 and Sentinel-2 inputs, the proposed approach jointly predicts tree height and associated uncertainty estimates in a single forward pass. Experimental results show that DER achieves predictive performance comparable to a deterministic U-Net while additionally providing well-calibrated uncertainty estimates. These findings demonstrate the potential of evidential learning as an efficient framework for uncertainty-aware forest structure estimation from Earth observation data.
Figures
Reference graph
Works this paper leans on
-
[1]
Ahmed, M.A., Bader, L., Kauermann, G., Zhu, X.X.: Bavaria tree benchmark (2026),https://huggingface.co/datasets/mammmarahmed/TreeUQ
work page 2026
-
[2]
In: Advances in Neural Information Processing Systems (2020)
Amini, A., Schwarting, W., Soleimany, A., Rus, D.: Deep Evidential Regression. In: Advances in Neural Information Processing Systems (2020)
work page 2020
-
[3]
Chen, M., Dong, W., Yu, H., Woodhouse, I.H., Ryan, C.M., Liu, H., Georgiou, S., Mitchard, E.T.A.: Multimodal deep learning enables forest height mapping from patchy spaceborne LiDAR using SAR and passive optical satellite data. Inter- national Journal of Applied Earth Observation and Geoinformation143, 104814 (2025).https://doi.org/https://doi.org/10.1016...
-
[4]
In: Proceedings of the 33rd International Conference on Machine Learning (2016)
Gal, Y., Ghahramani, Z.: Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning. In: Proceedings of the 33rd International Conference on Machine Learning (2016)
work page 2016
-
[5]
Gawlikowski, J., Tassi, C.R.N., Ali, M., Lee, J., Humt, M., Feng, J., Kruspe, A., Triebel, R., Jung, P., Roscher, R., Shahzad, M., Yang, W., Bamler, R., Zhu, X.X.: A survey of uncertainty in deep neural networks. Artificial Intelligence Review 56(S1), 1513–1589 (Oct 2023).https://doi.org/10.1007/s10462-023-10562-9
-
[6]
https://doi.org/10.48550/arXiv.2305.16703
Gruber, C., Schenk, P.O., Schierholz, M., Kreuter, F., Kauermann, G.: Sources of Uncertainty in Supervised Machine Learning – A Statisticians’ View (Jan 2025). https://doi.org/10.48550/arXiv.2305.16703
-
[7]
In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
He, K., Zhang, X., Ren, S., Sun, J.: Deep Residual Learning for Image Recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp.770–778.IEEE,LasVegas,NV,USA(2016).https://doi.org/10.1109/CVPR. 2016.90
doi:10.1109/cvpr 2016
-
[8]
Machine Learning110(3), 457–506 (Mar 2021).https://doi.org/10.1007/s10994-021-05946-3 10 L
Hüllermeier, E., Waegeman, W.: Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods. Machine Learning110(3), 457–506 (Mar 2021).https://doi.org/10.1007/s10994-021-05946-3 10 L. Bader et al
Show all 21 references
-
[9]
Remote Sensing15(8), 1969 (Jan 2023).https://doi.org/10
Kacic, P., Thonfeld, F., Gessner, U., Kuenzer, C.: Forest Structure Characteriza- tion in Germany: Novel Products and Analysis Based on GEDI, Sentinel-1 and Sentinel-2 Data. Remote Sensing15(8), 1969 (Jan 2023).https://doi.org/10. 3390/rs15081969
1969
-
[10]
Kendall, A., Gal, Y.: What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? In: Advances in Neural Information Processing Systems. vol. 30 (2017)
2017
-
[11]
Ambio52(11), 1804–1818 (Nov 2023).https://doi.org/10.1007/s13280-023-01908-2
Kujala, H., Minunno, F., Junttila, V., Mikkonen, N., Mäkelä, A., Virkkala, R., Akujärvi, A., Leikola, N., Heikkinen, R.K.: Role of data uncertainty when identi- fying important areas for biodiversity and carbon in boreal forests. Ambio52(11), 1804–1818 (Nov 2023).https://doi.o...
2023 doi
-
[12]
In: Proceedings of the 35th International Conference on Machine Learning
Kuleshov, V., Fenner, N., Ermon, S.: Accurate Uncertainties for Deep Learning Using Calibrated Regression. In: Proceedings of the 35th International Conference on Machine Learning. pp. 2796–2804. PMLR (2018)
2018
-
[13]
Lakshminarayanan, B., Pritzel, A., Blundell, C.: Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles (2017)
2017
-
[14]
Nature Ecology & Evolution7(11), 1778–1789 (Nov 2023)
Lang, N., Jetz, W., Schindler, K., Wegner, J.D.: A high-resolution canopy height model of the Earth. Nature Ecology & Evolution7(11), 1778–1789 (Nov 2023). https://doi.org/10.1038/s41559-023-02206-6
2023 doi
-
[15]
Remote Sensing of Environment268, 112760 (Jan 2022).https://doi.org/10.1016/j.rse.2021.112760
Lang, N., Kalischek, N., Armston, J., Schindler, K., Dubayah, R., Wegner, J.D.: Global canopy height regression and uncertainty estimation from GEDI LIDAR waveforms with deep ensembles. Remote Sensing of Environment268, 112760 (Jan 2022).https://doi.org/10.1016/j.rse.2021.112760
2022
-
[16]
Proceedings of the AAAI Conference on Artificial Intelli- gence37(8), 9134–9142 (2023).https://doi.org/10.1609/aaai.v37i8.26096
Meinert, N., Gawlikowski, J., Lavin, A.: The Unreasonable Effectiveness of Deep Evidential Regression. Proceedings of the AAAI Conference on Artificial Intelli- gence37(8), 9134–9142 (2023).https://doi.org/10.1609/aaai.v37i8.26096
2023 doi
-
[17]
Remote Sensing of Environment253, 112165 (Feb 2021).https://doi.org/10.1016/j.rse.2020.112165
Potapov, P., Li, X., Hernandez-Serna, A., Tyukavina, A., Hansen, M.C., Kom- mareddy, A., Pickens, A., Turubanova, S., Tang, H., Silva, C.E., Armston, J., Dubayah, R., Blair, J.B., Hofton, M.: Mapping global forest canopy height through integration of GEDI and Landsat data. Rem...
2021
-
[18]
In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F
Ronneberger, O., Fischer, P., Brox, T.: U-Net: Convolutional Networks for Biomed- ical Image Segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. pp. 234–241. Springer Internation...
2015 doi
-
[19]
In- ternational Journal of Applied Earth Observation and Geoinformation128, 103711 (Apr 2024).https://doi.org/10.1016/j.jag.2024.103711
Schwartz, M., Ciais, P., Ottlé, C., Truchis, A.D., Vega, C., Fayad, I., Brandt, M., Fensholt, R., Baghdadi, N., Morneau, F., Morin, D., Guyon, D., Dayau, S., Wigneron, J.P.: High-resolution canopy height map in the Landes forest (France) based on GEDI, Sentinel-1, and Sentinel...
2024
-
[20]
In: Advances in Neural Information Processing Systems
Sensoy, M., Kaplan, L., Kandemir, M.: Evidential Deep Learning to Quantify Clas- sification Uncertainty. In: Advances in Neural Information Processing Systems. vol. 31. Curran Associates, Inc. (2018)
2018
-
[21]
Scientific Data12(1), 678 (Apr 2025).https: //doi.org/10.1038/s41597-025-05021-9
Su, Y., Schwartz, M., Fayad, I., García, M., Zavala, M.A., Tijerín-Triviño, J., Astigarraga, J., Cruz-Alonso, V., Liu, S., Zhang, X., Chen, S., Ritter, F., Besic, N., d’Aspremont, A., Ciais, P.: Canopy height and biomass distribution across the forests of Iberian Peninsula. Sc...
2025 doi
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.