{"id":"88b6ef6a-3fd6-4f3c-a70f-eec71f5f8aa1","arxiv_id":"2506.00498","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"UNSURF, a discrepancy-based uncertainty measure for cortical surface reconstruction, correlates with surface errors and improves quality control and Alzheimer's classification when used for filtering.","lead":"UNSURF is a new way to rate how sure we can be about brain surface reconstructions from lower-quality clinical MRI scans, by comparing the model's predicted distance maps to the distance maps of the final fitted surfaces. It is relevant because it could let hospitals automatically flag unreliable measurements in large-scale brain studies.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The QC and downstream-AD claims rest on filtering curves with no random-filtering baseline or error bars; the reported improvements may be artifacts of data removal rather than evidence that UNSURF identifies unreliable measurements.","rationale":"The reader's stated weakest assumption is the FreeSurfer-based thickness-error proxy, but their rationale also flags the missing random-filtering baseline and error bars. I focus on the latter as the single most load-bearing concern because it directly supports two of the paper's headline claims: effective QC and improved downstream AD classification. The filtering curves in Fig. 3(d/e) and Fig. 4 would look qualitatively similar if U were only weakly related to error, because removing high-error cases reduces mean error by construction; the absence of a random control and confidence intervals makes the magnitude and significance of the claimed benefit unverifiable. The proposed test is cheap and decisive: if U-filtering does not significantly beat random filtering, the applications claimed in the abstract are unsupported even though the Section 3.2 correlations might remain. I keep the reader's CONDITIONAL verdict rather than moving to REJECT because the correlation evidence and the plausibility of the method support conditional acceptance pending the missing baseline analysis.","tokens_in":8165,"tokens_out":13153,"duration_ms":143199,"concrete_test":"Recompute Fig. 3(d/e) and Fig. 4 with a random-filtering baseline: at each retention level (100%, 90%, ..., 50%), sample 1000 random subsets of parcels/nodes/subjects and plot the 95% bootstrap CI of mean thickness error and Cohen's d. Also report the AD/CN counts retained at each U_s threshold and rerun the GLM with diagnosis-stratified filtering. If the U-based curve lies inside or barely below the random-filtering CI, or if the diagnostic ratio shifts materially, the QC/downstream claims should be downgraded to correlation-only evidence.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract claims UNSURF enables effective QC and improves downstream AD classification. The evidence for these claims is Fig. 3(d/e) and Fig. 4, which show that removing high-U parcels, nodes, or subjects reduces thickness error or increases Cohen's d. No random-filtering control, confidence interval, or p-value is reported. This matters because any selection rule that preferentially removes high-error cases will produce a downward error curve, and effect sizes computed on shrinking subsamples are biased and noisy; Cohen's d is known to be biased upward at small n. In Fig. 4, removing high-U_s subjects may also preferentially drop one diagnostic group (e.g., AD subjects whose atrophy makes reconstruction harder), changing the AD/CN balance and the adjusted effect size for reasons unrelated to measurement quality. Section 3.2's correlation results are useful, but they do not by themselves establish that U-based filtering outperforms random or simple quality-based filtering. Without a baseline, the claimed QC and downstream benefits are not statistically grounded.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces UNSURF, a per-voxel uncertainty measure for learning-based cortical surface reconstruction, defined as the squared difference between the signed distance function (SDF) predicted by a CNN and the SDF recomputed from the geometrically processed surface mesh. The measure is aggregated to node, parcel, and subject levels and compared with ensemble-dropout variance on two clinical-style datasets. The authors report correlations between UNSURF and thickness errors (with FreeSurfer-derived surfaces as ground truth), show that filtering high-uncertainty parcels/nodes reduces thickness errors, and report that subject-level filtering increases Cohen's d in an AD versus CN classification analysis.","tokens_in":8325,"tokens_out":3886,"duration_ms":38697,"significance":"If the claims are substantiated, UNSURF would be a valuable and low-cost uncertainty signal: it requires no additional training, it is deterministic, and it explicitly reflects the geometric corrections that variance-based measures miss. The paper's strengths are the use of real heterogeneous clinical data, the head-to-head comparison with ensemble dropout variance, and the multi-level aggregation scheme. I also agree that UNSURF is not circular in a parameter-fitting sense; it is an internal consistency measure between the raw network output and the final surface. The main gaps are statistical: the filtering experiments lack random baselines and confidence intervals, and the correlation evidence is reported without uncertainty bounds, so the central QC and downstream-benefit claims are not yet fully supported.","major_comments":[{"comment":"The QC and downstream-benefit claims are not supported by the reported filtering experiments because there is no baseline that removes the same fraction of data at random, or according to an independent quality measure. Any selection rule correlated with thickness error will mechanically reduce mean error as data are removed, and Cohen's d computed on shrinking subsamples is biased and increasingly noisy. Please add random-filtering control curves with confidence intervals (for example, repeated random subsampling at each kept fraction), and report the number of subjects and the AD/CN group sizes retained at each point in Figure 4.","section":"Section 3.3, Figure 3(d)-(e); Section 3.4, Figure 4"},{"comment":"Subject-level filtering by U^s may change the AD/CN composition of the sample: if high-uncertainty subjects are disproportionately from one diagnostic group, the effect-size increase may reflect altered group balance rather than improved measurement quality. The paper should report the AD/CN counts and the age/gender distributions after each filtering step, and should ideally demonstrate that the effect-size trend persists after adjusting for these factors, or when using within-subject parcel- or node-level filtering only.","section":"Section 3.4, Figure 4"},{"comment":"The central correlation evidence lacks uncertainty quantification: SCC/PCC values are presented without confidence intervals, p-values, or correction for the large number of parcels in Figure 3(a). Moreover, the ground-truth thickness errors are computed against FreeSurfer surfaces derived from 1 mm T1 scans, which are themselves an imperfect proxy for true surface placement error; this should be explicitly acknowledged and ideally bounded by a repeatability or inter-method analysis. Without these, it is difficult to judge whether the reported differences between variance and UNSURF are statistically meaningful.","section":"Section 3.2, Figure 2; Section 3.3, Figure 3(a)-(c)"}],"minor_comments":[{"comment":"The text says \"Person correlation coefficient\" but the standard term is \"Pearson correlation coefficient.\"","section":"Figure 2 caption"},{"comment":"The word \"wihtout\" should be \"without\" in the sentence describing the Euler characteristic.","section":"Section 2.1"},{"comment":"The caption says \"At 100, all subjects are kept\" but the filtering in panels (d) and (e) is applied to parcels and nodes; please clarify the exact denominator and the data unit being kept in each panel.","section":"Section 3.3, Figure 3 caption"},{"comment":"The proposed measure is called \"uncertainty\" but is defined as a deterministic discrepancy; the paper would benefit from a brief discussion of what sources of error this discrepancy captures (geometric correction, smoothing, topological changes) and what it cannot capture (systematic bias shared by the prediction and the geometry-processing step).","section":"Section 2.2"},{"comment":"The red dashed line for Recon-All is not clearly defined; please specify whether this is FreeSurfer applied to the original 1 mm T1 or to a native low-resolution scan, and how the mean effect size is computed across parcels.","section":"Section 3.4, Figure 4"}],"recommendation":"major_revision","confidential_remarks":"The core idea is simple and potentially useful, and the authors have chosen appropriate real-world datasets. The main revision needed is statistical: adding random-filtering baselines, confidence intervals, group-count reporting, and correlation uncertainty would make the QC and downstream claims credible. I would be willing to review a revised version."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe one thing to know: UNSURF is a genuinely simple and well-motivated uncertainty measure—the squared difference between the network's predicted SDF and the SDF of the final geometry-processed surface—and it does correlate with thickness errors on clinical scans. That's a solid contribution. But the paper's QC and downstream-AD claims rest on filtering curves with no random-filtering control and no error bars, so those parts need more work.\n\nWhat's new: using the consistency between the raw prediction and the post-processing output as an uncertainty proxy. It's cheap (no M×2 hours of geometry), it naturally accounts for topological correction, and it doesn't require any new parameters. The comparison with ensemble dropout variance is fair: variance is essentially blind to geometry, so it performs poorly, while U captures the mismatch that matters. The correlation numbers (e.g., SCC 0.627 subject-level on clinical data) are believable, and the parcel-level maps make sense.\n\nSoft spots, in order of importance. First, the QC claims in Fig. 3d/e and the AD effect-size curves in Fig. 4 have no random-filtering baseline. Any rule that removes high-error cases will push the curves down; with no control group, you can't tell whether U is doing better than random removal, let alone than simple quality proxies like FreeSurfer's own QC metrics. Second, Cohen's d on a shrinking subsample is biased upward, and removing high-uncertainty subjects may also tilt the AD/CN balance—the paper doesn't report the group counts after filtering. Third, the \"ground truth\" is FreeSurfer on 1mm T1, which is itself not ground truth; the correlations are against a proxy, and the paper doesn't acknowledge this as a limitation. Fourth, there are no confidence intervals on any of the reported correlations.\n\nThe central idea holds up; this is not a circular or fitted measure. But the paper should not be accepted as-is. The core contribution is worth publishing and deserves a serious referee, with the request that the authors add a random-filtering baseline, report CIs, and discuss the FreeSurfer-proxy caveat.\n\nFor your reading group: it's a useful example of a practical uncertainty measure that works despite being simple. I'd cite it if I worked on surface reconstruction.","headline":"UNSURF is a simple, sensible uncertainty measure with solid correlation evidence, but the QC and downstream-AD claims lack the baseline controls needed to back them up.","tokens_in":8858,"tokens_out":2578,"would_cite":true,"duration_ms":24589,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper proposes UNSURF, a surface-placement uncertainty measure based on the squared discrepancy between predicted and fitted signed distance functions, and reports that it tracks thickness errors, enables multi-level quality control…","keywords":["uncertainty quantification","cortical surface reconstruction","clinical MRI","signed distance function","quality control","cortical thickness","Alzheimer's disease classification"],"falsifier":"Take a set of clinical scans for which expert raters manually mark where the reconstructed white-matter or pial surface is misplaced, and compute UNSURF maps on the same scans: if voxels or mesh nodes with the largest $(\\hat{D}-\\tilde{D})^2$ do not preferentially coincide with the expert-marked misplacements, the measure is not tracking surface placement error. Alternatively, substitute manually edited surfaces for the FreeSurfer-based ground truth in the thickness-error correlation and check whether the reported correlations survive.","tokens_in":7960,"feed_emoji":"🧠","tokens_out":6751,"duration_ms":62975,"temperature":0.7,"pith_summary":"UNSURF is proposed as a way to quantify how much a deep-learning cortical surface reconstruction is to be trusted when the input is a clinical MRI of arbitrary orientation, resolution, and contrast. The measure is defined at every voxel as the squared difference between the signed distance function predicted by the network and the signed distance function of the surface that the geometry-processing stage actually fits. The paper reports that this quantity correlates with measured cortical thickness errors, while classical voxel-wise Monte Carlo variance does not, because the discrepancy captures the geometric corrections that variance ignores. Because of this correlation, UNSURF supports automated quality control at the subject, parcel, and mesh-node level and improves Alzheimer's disease versus control classification when high-uncertainty measurements are removed. If the claim holds, large uncurated clinical datasets become more usable for group studies of cortical thickness.","feed_headline":"Squared distance gap flags bad brain-surface fits on clinical MRI","feed_subtitle":"New uncertainty measure tracks thickness errors and sharpens Alzheimer's vs. control separation in hospital scans.","key_machinery":"The object that carries the argument is the signed distance function (SDF), the map from every voxel to its signed distance from a cortical surface, and the fitted-surface SDF $\\tilde{D}$ computed after the geometry-processing stage. The UNSURF measure is the squared difference between the network's raw SDF prediction $\\hat{D}$ and the SDF of the topologically corrected, deformed surface $\\tilde{D}$. Large discrepancies mark voxels where the network's implicit surface disagrees with the final explicit mesh, indicating regions where more than one surface placement is plausible after geometry constraints are enforced.","core_discovery":"On the paper's own terms, the central discovery is that uncertainty in cortical surface placement should be measured after the geometry step, not before it. Given predicted signed distance functions $\\hat{D}^v_i$ from a neural network and signed distance functions $\\tilde{D}^v_i$ computed from the surfaces extracted by topological correction and mesh deformation, the voxel-wise squared discrepancy $U_i^v = (\\hat{D}^v_i - \\tilde{D}^v_i)^2$ acts as a surface-placement uncertainty estimate. Across the evaluated clinical and downsampled scans, $U$ shows positive Spearman and Pearson correlations with thickness error, whereas ensemble-dropout variance of the predicted SDFs is weakly or even negatively correlated with the same errors. Filtering out high-$U$ parcels and nodes reduces thickness errors, and removing high-$U$ subjects increases the Cohen's d effect size separating Alzheimer's patients from controls.","pith_inferences":["UNSURF could be computed in any implicit-surface reconstruction pipeline that has a deterministic post-processing step, not just cortical surfaces; the same squared-discrepancy logic may transfer to other anatomical structures whose surfaces are extracted from predicted distance maps.","Because UNSURF requires no extra network passes at test time, it is cheaper than ensemble dropout or deep ensembles, so a natural next experiment is to measure the computational cost difference directly on the same geometry pipeline.","The lower correlation at 2 mm and 3 mm slice spacings suggests the measure's practical value is highest for heavily downsampled clinical acquisitions; testing UNSURF on near-isotropic but artifact-heavy scans would delimit this boundary.","If post-processing were made stochastic, for example by randomizing mesh initialization or deformation steps, part of the discrepancy $U$ would capture algorithmic variability rather than data ambiguity, so separating those two sources would clarify what $U$ actually means."],"forward_implications":["Automated quality control of cortical surface reconstructions can be performed at the subject, parcel, and mesh-node level on clinical scans without manual review.","Removing high-uncertainty parcels lowers parcel-level cortical thickness error in the evaluated clinical scans.","Node-level filtering within already uncertain parcels further reduces thickness errors relative to parcel-level filtering alone.","Uncertainty-based subject filtering increases the effect size of cortical thickness differences between Alzheimer's and control groups.","Ensemble-dropout variance of predicted SDFs is not a reliable proxy for cortical surface placement uncertainty and should not be used for this purpose."],"supporting_citations":[{"why":"This is the surface-placement pipeline—SDF prediction followed by geometry processing—that UNSURF's discrepancy is computed against.","marker":"[15]"},{"why":"These FreeSurfer surfaces supply the ground-truth thickness errors used in every correlation experiment and the geometry-processing tools used by the pipeline.","marker":"[9]"},{"why":"This is the ensemble-dropout baseline whose voxel-wise variance UNSURF is compared with and reported to outperform.","marker":"[34]"},{"why":"This is the synthetic-image generation strategy that makes the SDF predictor contrast- and resolution-agnostic for clinical scans.","marker":"[4]"},{"why":"This is the domain-randomization approach used to train the reconstruction network on heterogeneous resolutions and contrasts.","marker":"[21]"},{"why":"These segmentations provide the anatomy used to generate the synthetic training images for the SDF network.","marker":"[29]"}],"fun_headline_variants":["Squared SDF gap beats MC variance for brain-surface errors","UNSURF: post-geometry uncertainty predicts cortical thickness error","Discrepancy measure sharpens Alzheimer's separation on clinical MRI","New uncertainty metric flags bad brain surfaces and boosts AD diagnosis","After fitting: computing uncertainty from SDF gap improves QC and AD"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that thickness error computed against FreeSurfer-derived surfaces from high-resolution scans is a valid proxy for true surface placement error in clinical scans, and that the geometry-processing stage is deterministic, so the measured discrepancy reflects only model and data ambiguity rather than algorithmic variability.","fun_headline_variants_meta":{"raw":{"variants":["Squared SDF gap beats MC variance for brain-surface errors","UNSURF: post-geometry uncertainty predicts cortical thickness error","Discrepancy measure sharpens Alzheimer's separation on clinical MRI","New uncertainty metric flags bad brain surfaces and boosts AD diagnosis","After fitting: computing uncertainty from SDF gap improves QC and AD"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000172,"raw_usage":{"total_tokens":1228,"prompt_tokens":851,"completion_tokens":377,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":467,"completion_tokens_details":{"reasoning_tokens":290}},"tokens_in":467,"tokens_out":377,"duration_ms":4141,"temperature":1.0,"reasoning_tokens":290,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T12:02:21.360714+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a set of clinical scans for which expert raters manually mark where the reconstructed white-matter or pial surface is misplaced, and compute UNSURF maps on the same scans: if voxels or mesh nodes with the largest $(\\hat{D}-\\tilde{D})^2$ do not preferentially coincide with the expert-marked misplacements, the measure is not tracking surface placement error. Alternatively, substitute manually edited surfaces for the FreeSurfer-based ground truth in the thickness-error correlation and check whether the reported correlations survive.","supporting_citations":[{"cited_title":"Recon-all-clinical: Cortical surface reconstruction and analysis of heterogeneous clinical brain MRI","cited_arxiv_id":"2409.03889","evidence_quote":"This is the surface-placement pipeline—SDF prediction followed by geometry processing—that UNSURF's discrepancy is computed against."},{"cited_title":"Neuroimage9(2), 195–207 (1999)","cited_arxiv_id":null,"evidence_quote":"These FreeSurfer surfaces supply the ground-truth thickness errors used in every correlation experiment and the geometry-processing tools used by the pipeline."},{"cited_title":"Medical image analysis86 (2023)","cited_arxiv_id":null,"evidence_quote":"This is the synthetic-image generation strategy that makes the SDF predictor contrast- and resolution-agnostic for clinical scans."},{"cited_title":"Science advances9(5), eadd3607 (2023) UNSURF: Uncertainty Quantification for Cortical Surface Reconstruction 11","cited_arxiv_id":null,"evidence_quote":"This is the domain-randomization approach used to train the reconstruction network on heterogeneous resolutions and contrasts."},{"cited_title":"NeuroImage143 (2016)","cited_arxiv_id":null,"evidence_quote":"These segmentations provide the anatomy used to generate the synthetic training images for the SDF network."}],"review_version":1}