{"id":"07494923-9151-4f73-8015-f9f6061ba0b4","arxiv_id":"2411.15513","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"SPA presents users with four representative segmentation candidates, and a learned mixture-of-Gaussians preference distribution updates from the user's selection to converge to their preferred boundary in a few interactions.","lead":"This paper introduces SPA, a medical image segmentation system that shows doctors a few distinct plausible segmentations and adapts to their preference from a single choice, cutting the number of clicks and time needed versus existing interactive tools. It matters because clinical users disagree on ambiguous tumor boundaries, and current systems either show too many similar outputs or demand tedious pixel-level corrections.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Algorithm 1 trains SPA on self-confirming feedback (r_u = argmin ||r_k - yfin_app||), so the generator never learns to respond to a user's genuinely different choice; the central preference-adaptation claim lacks direct evidence.","rationale":"The paper proposes a useful interaction paradigm and reports strong Dice, efficiency, a human study, and released code; those are real assets. I read the central claim as: the preference-distribution generator maps a user's choice among K candidates into an updated p_theta(z), and iterating this converges to that user's preference. For that claim, the generator must be trained on inputs of the same type it receives at test time: a selection that can disagree with the model's current output. Algorithm 1 defines the simulated user as the candidate closest to yfin_app, so during training the generator only sees 'stay near current output' feedback. The segmentation losses supervise the final mask against the fused ground truth, but they do not teach the generator to associate a different candidate mask with a different latent distribution. The appendix proof is for an idealized GMM posterior over latent z samples, not for the actual mask-selection-to-prompt-encoder-to-generator pathway, and Section 8 does not document the test-time simulation of user choice used for automated evaluation. These gaps make the headline numbers compatible with the alternative explanation that the gains come from the segmentation supervision and from the multi-choice interface itself, rather than from learned adaptation. This is exactly the reader's weakest assumption, so I agree. The appropriate remedy is a controlled retraining experiment with preference-grounded simulated selections; until that is run, the strong adaptation claim is not established, and the conditional verdict should stand.","tokens_in":19332,"tokens_out":6985,"duration_ms":67904,"concrete_test":"Retrain SPA with exactly the same losses, architecture, and hyperparameters, but replace the simulated selection in Algorithm 1 with a preference-grounded one: r_u = arg min_{r_k} ||r_k - y_target||_2, where y_target is the fused annotation of one held-out clinician or the stochastic ground-truth y used for supervision, while keeping the rest of the training loop fixed. Then run the same six-iteration evaluation and the leave-one-user-out alignment protocol. If the per-iteration Dice gains and the Table 2 included/excluded-clinician divergence are reproduced, the original self-confirming simulation was sufficient and the concern is resolved. If the gains vanish or reverse, the reported adaptation is an artifact of the self-confirming training signal.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The single load-bearing weakness is the training-time simulation of the user in Algorithm 1. After each inner-loop update the next interaction is defined as r_u = arg min_{r_k} ||r_k - yfin_app||_2, i.e. the simulated user selects the representative candidate closest to the model's own current aggregated prediction. This feedback is self-confirming: it never presents the Preference Distribution Generator with a choice that emanates from a different annotator preference. The only supervision tied to the target mask y is the segmentation loss LCE on yapp; the selection signal e_u = E_I(r_u) is not tied to y or to any held-out rater. Consequently the generator is trained to keep/refine the current latent distribution in response to 'no-change' feedback, and there is no training signal for the operation that the method's central claim depends on: shifting p_theta(z) toward a user's preferred alternative when the user picks a candidate that disagrees with yfin_app. The supplementary convergence proof (Section 7) does not repair this: it assumes interactions r_u are i.i.d. samples from N(mu_u, sigma_u^2), whereas the implemented interaction is a segmentation mask selected from K-means candidates and encoded by a SAM prompt encoder; the proof is for an idealized GMM posterior, not for the amortized neural generator or the actual feedback channel. The automated evaluation protocol (Section 8) also does not state how the user's choice is simulated during testing (only that ground-truth combinations are randomly generated), so the reported alignment in Table 2 cannot be attributed to learned preference adaptation rather than to the supervision or to the candidate-selection mechanism. This is a training-distribution mismatch internal to the algorithm, not a disagreement with community consensus.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"SPA proposes a test-time preference-alignment framework for medical image segmentation. The method models annotator-specific segmentation preferences as a mixture-of-Gaussians latent distribution pθ(z), generates N stochastic predictions per iteration, aggregates them into a recommended mask, and presents K representative candidates via K-means clustering. The user selects one candidate; the selection is encoded and fed to a Preference Distribution Generator that updates pθ(z), and the process repeats. The authors claim that SPA adapts to individual users with fewer interactions than click/box-based interactive segmentation, with state-of-the-art Dice scores across REFUGE2, LIDC-IDRI, and QUBIQ tasks. The paper includes ablations, a human user study, an efficiency analysis, and a supplementary convergence argument for the preference distribution.","tokens_in":48,"tokens_out":6095,"duration_ms":118312,"significance":"If the central claim were validated, SPA would offer a practically valuable interface: replacing pixel-level corrections with a small number of multiple-choice selections is a genuinely attractive way to handle inter-annotator variability in clinical segmentation. The paper has noteworthy strengths: public code, evaluation across seven tasks and three imaging modalities, a human user study, and ablations of the main components. However, the core preference-adaptation mechanism is not established by the evidence as presented. The training-time user simulation in Algorithm 1 is self-confirming, the supplementary convergence proof is for an idealized model rather than the implemented network, and the automated evaluation does not specify how test-time user choices are simulated. These issues are load-bearing for the paper's central claim that the model learns to shift its latent distribution toward a real user's preferred alternative.","major_comments":[{"comment":"The simulated user is implemented as ru = arg min_{rk} ||rk - yfin_app||2. This is self-confirming: in every training interaction the feedback is the representative candidate closest to the model's own aggregated output, so the Preference Distribution Generator never observes a selection that disagrees with the current prediction. The only supervision tied to the multi-rater target y is the segmentation loss LCE on yapp; the interaction embedding e_u = E_I(r_u) carries no information about a user preference different from the model's current behavior. As a result, the training signal for the central claim, namely shifting pθ(z) toward a user's preferred alternative, is absent. The paper needs either a user simulation that selects the candidate closest to a held-out rater or rater-combination, or direct evidence (e.g., real clinician selections at test time) that the generator responds to non-confirming choices.","section":"Algorithm 1 (training loop)"},{"comment":"The convergence proof does not model the implemented system. It assumes the interactions ru^(j) are i.i.d. scalar samples from N(µu, σu^2), whereas in the paper ru is a K-means segmentation mask selected from K candidates and then encoded by a SAM prompt encoder; pθ(z) is updated by an amortized MLP, not by exact posterior computation. The proof establishes only that an ideal GMM posterior concentrates on the true component as J→∞, not that the trained Preference Distribution Generator and the actual feedback channel converge. This proof therefore cannot support the claim that the preference distribution pθ(z) can adapt to specific user preferences based on new interactions.","section":"Supplementary Section 7 (Eqs. 2-7)"},{"comment":"Table 1 reports single Dice scores with no error bars or confidence intervals. Several test sets are very small (e.g., QUBIQ brain tumor has 4 testing cases and brain growth has 5), and several reported gaps between SPA and the next-best method are under a point (e.g., REFUGE2 3-Iter: 85.42 vs 83.08 for SAM, or 88.56 vs 84.10 in the first block). The SOTA claim in Section 4.3.1 is load-bearing and needs repeated-run statistics, paired significance tests, or per-case variability. Please also specify how many runs or random seeds the reported numbers are based on, especially given the stochastic sampling in the method.","section":"Section 4.3.1 / Table 1"},{"comment":"The automated test protocol describes how ground-truth combinations are generated but never states how the simulated user's choice ru is made at test time. This matters because Table 2's alignment analysis (predictions moving toward included clinicians and away from excluded ones) is exactly the evidence needed to distinguish genuine preference adaptation from optimization toward the weighted ensemble used as the training target. Without specifying whether the user selection is oracle-based (closest to the consensus or held-out target) or self-confirming (closest to yfin_app), the alignment results in Section 4.3.3 are uninterpretable.","section":"Supplementary Section 8 and Section 4.3.3"}],"minor_comments":[{"comment":"The notation '10.5' in the aggregation formulas (e.g., yapp = 10.5 (1/N Σ yn)) is ambiguous: it may be an indicator threshold at 0.5 or a scaling constant. Please typeset it explicitly (e.g., 1[mean ≥ 0.5]) and explain what operation is intended.","section":"Section 3.3 and Algorithm 1"},{"comment":"The '1-Iter' and '3-Iter' columns are applied to all method categories, but deterministic and uncertainty-aware methods are not interactive. Please clarify how these columns were computed for non-interactive baselines, or use separate columns for them.","section":"Table 1"},{"comment":"The header for Table 6 shows two numbers per method without labeled subcolumns; please add explicit 'Dice 75%' and 'Dice 84%' subheaders so the reader can parse the results.","section":"Supplementary Table 6"},{"comment":"The human user study reports only five participants and no statistical test, session-order randomization, or inter-annotator variability. A significance test or per-case breakdown would strengthen the efficiency claim, which is otherwise anecdotal.","section":"Section 4.3.2 / Human User Study"}],"recommendation":"major_revision","confidential_remarks":"The main issue I would ask the editor to weigh is the circularity of the training-time user simulation in Algorithm 1. If the authors can revise the paper to train with a non-self-confirming user simulation and specify the oracle used to emulate user choices at test time, the central claim becomes testable. Without such a revision, I would not be able to support acceptance, because the reported alignment results would be consistent with the model simply refining its prediction toward the training-time weighted ensemble rather than adapting to an individual user's preference."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth knowing: SPA replaces click-based interactive segmentation with a multiple-choice interface over K-means representative candidates and models the latent preference as an amortized GMM. That interaction design is new and genuinely appealing. The evaluation is also broad: seven tasks, three modalities, leave-one-user-out experiments, a small human study, and released code. I came away convinced that the interface can save clinicians time and effort.\n\nThe problem is the central adaptation claim. Algorithm 1 simulates the user's next selection as r_u = arg min ||r_k - yfin_app||_2, i.e., the candidate closest to the model's own current aggregate. That is a self-confirming channel: the preference distribution generator is never trained on a choice that disagrees with the model's prediction, so it is never taught the operation the paper says it learns, shifting p_theta(z) toward a user's preferred alternative. The convergence proof in the appendix does not repair this; it assumes interactions are i.i.d. Gaussian samples in the latent space, not the actual K-means candidate masks encoded by the SAM prompt encoder. The test-time protocol in Section 8 also does not state how the user's choice is simulated during automated evaluation, which matters because the alignment shown in Table 2 could simply reflect training toward the blended ground truth rather than genuine preference adaptation.\n\nSmaller issues: Table 1 reports single Dice numbers with no error bars, and several improvements over the strongest interactive baselines are modest. The human study is a real strength, but it measures time-to-satisfaction, so it supports the usability of the interface, not the preference-alignment mechanism.\n\nTo be clear about proportion: the paper is not sloppy. It ships code, the empirical comparison is extensive, and the core idea is worth pursuing. If the authors change the simulated user to select the candidate closest to the ground-truth preference and rerun the alignment experiments, the framework could become credible. As presented, the evidence for the main claim is missing, not merely weak.\n\nFor peer review: send it out. A serious referee should ask for the corrected simulation and an explicit description of the test-time user simulation. That is a clear path to revision, not a desk rejection.","headline":"A genuinely useful multiple-choice interaction scheme and broad evaluation, but the main preference-adaptation claim is not supported because training simulates a user who always picks the candidate closest to the model's own output.","tokens_in":663,"tokens_out":1106,"would_cite":true,"duration_ms":49658,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Four-option picks teach a segmentation model your preference","keywords":["user preference alignment","medical image segmentation","uncertainty modeling","mixture of Gaussians","interactive segmentation","multiple-choice interaction","multi-rater annotations"],"falsifier":"Run SPA with a scripted user who always picks the representative candidate least similar to the current aggregation; if Dice against that user's held-out annotations does not improve relative to a no-feedback baseline, the adaptation shown in the paper is an artifact of self-confirming simulation rather than genuine preference alignment.","tokens_in":19041,"feed_emoji":"🩻","tokens_out":6453,"duration_ms":51740,"temperature":0.7,"pith_summary":"This paper proposes SPA, a medical image segmentation framework that learns a clinician's preferred way to draw ambiguous boundaries from a small number of multiple-choice selections. Instead of asking for pixel-level clicks or boxes, SPA shows one recommended mask and four representative alternatives; the user chooses the one closest to their intent, and the model updates a latent mixture-of-Gaussians preference distribution before offering a new set. The authors claim that within a few iterations this converges to the user's personalized segmentation, and they report an average Dice score of 89.68% after three iterations across fundus, CT, and MRI tasks, beating deterministic, uncertainty-aware, and interactive baselines. A human study with five medical professionals reports markedly less time and fewer interactions than the interactive baseline MedSAM, and the authors state that a user can segment 35% more images with 39% fewer iterations.","feed_headline":"Four-option picks teach a segmentation model your preference","feed_subtitle":"By turning boundary decisions into a 4-option choice, SPA hits 89.68% Dice after three interactions.","key_machinery":"The load-bearing object is the preference distribution $p_{\\theta}(z) = \\sum_{m=1}^{M} \\pi_m \\mathcal{N}(z \\mid \\mu_m, \\sigma_m^2)$, whose latent samples condition the segmentation decoder. A neural Preference Distribution Generator predicts the mixture parameters from the user's chosen candidate; K-means clustering of $N$ stochastic predictions yields the $K$ candidate masks shown to the user; and a SAM-style prompt encoder turns the selected mask into the feedback embedding. Together these convert a user's discrete choice into a changed distribution over segmentation outputs.","core_discovery":"The central claim is that the variability in how clinicians segment ambiguous pixels can be captured by a parameterized preference distribution $p_{\\theta}(z)$, a mixture of $M$ Gaussians, and that this distribution can be adapted at test time from simple preference feedback. SPA samples $N$ latent variables from $p_{\\theta}(z)$, conditions the segmentation network on each, clusters the $N$ predictions into $K$ representative candidates, and asks the user to pick one; the chosen candidate is encoded and fed to a Preference Distribution Generator that re-estimates the mixture parameters. The appendix argues that with enough interactions the distribution converges to the single Gaussian $\\mathcal{N}(\\mu_u, \\sigma_u^2)$ of the target user, and experiments show predictions moving toward included clinicians and away from excluded ones over iterations. On the paper's terms, the result is that uncertain boundary decisions become a multiple-choice task rather than a pixel-refinement task, and the model can align to a new user's preference in roughly three interactions.","pith_inferences":["The paper's training protocol simulates the user's choice as the candidate closest to the model's own aggregated prediction; since that choice never disagrees strongly with the model, the reported adaptation may understate how the system behaves for a user who picks an outlier candidate. A scripted adversarial-user test would separate learned preference adaptation from self-confirmation.","The multiple-choice interface suggests that other high-stakes annotation settings, such as pathology outlines or radiotherapy target volumes, could use the same preference-distribution update with a domain-specific candidate generator.","One testable extension is to let the model choose which $K$ candidates to show so that the selected option carries maximal information about the user's preference, rather than using unsupervised K-means centroids.","The convergence proof assumes the user's selections are i.i.d. from one Gaussian component; real users may switch preference mid-session, which would require the distribution to forget old evidence."],"forward_implications":["If the claim holds, interactive segmentation can be reduced to choosing among a few distinct candidates, sparing clinicians from pixel-level clicking.","Uncertainty-aware models gain a feedback channel, so the many-similar-predictions problem of conventional probabilistic segmentation disappears.","The same user's adapted distribution transfers to new images, so personalization carries across a session.","The reported gains imply fewer failure cases at fixed Dice thresholds, which matters when annotation time is constrained.","Any segmentation task with multi-rater annotations could be fitted to the same preference-alignment loop."],"supporting_citations":[{"why":"Supplies the REFUGE2 fundus dataset with annotations from seven ophthalmologists used for optic cup segmentation evaluation.","marker":"[10]"},{"why":"Supplies the LIDC-IDRI lung CT dataset with multi-radiologist annotations used for lung lesion segmentation.","marker":"[1, 7]"},{"why":"Supplies the QUBIQ benchmark with brain, prostate, and kidney tasks annotated by multiple clinicians.","marker":"[23]"},{"why":"Provides the SAM architecture that SPA reuses for its mask decoder and prompt encoder.","marker":"[18]"},{"why":"Probabilistic U-Net is the main uncertainty-aware baseline that SPA must beat.","marker":"[20]"},{"why":"MRNet multi-rater agreement modeling motivates the latent preference distribution and serves as a comparison method.","marker":"[16]"},{"why":"SAM-U is the uncertainty-interactive comparison that combines SAM with multiple prompts.","marker":"[9]"}],"fun_headline_variants":["Three human picks teach a medical AI your segmentation style","Uncertain pixels become a 4-option quiz that updates the model","SPA turns boundary confusion into a quick multiple-choice","Align a segmentation model to your doctor's eye in 3 picks","Four distinct choices let a model learn your boundary judgment"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Training feeds the model its own closest candidate as the simulated user's choice, so the model may never learn to respond to a user who selects something genuinely different from what the model predicts.","fun_headline_variants_meta":{"raw":{"variants":["Three human picks teach a medical AI your segmentation style","Uncertain pixels become a 4-option quiz that updates the model","SPA turns boundary confusion into a quick multiple-choice","Align a segmentation model to your doctor's eye in 3 picks","Four distinct choices let a model learn your boundary judgment"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000453,"raw_usage":{"total_tokens":2330,"prompt_tokens":1046,"completion_tokens":1284,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":662,"completion_tokens_details":{"reasoning_tokens":1201}},"tokens_in":662,"tokens_out":1284,"duration_ms":9970,"temperature":1.0,"reasoning_tokens":1201,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T14:12:23.993303+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run SPA with a scripted user who always picks the representative candidate least similar to the current aggregation; if Dice against that user's held-out annotations does not improve relative to a no-feedback baseline, the adaptation shown in the paper is an artifact of self-confirming simulation rather than genuine preference alignment.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Probabilistic U-Net is the main uncertainty-aware baseline that SPA must beat."},{"cited_title":"Learning Calibrated Medical Image Segmentation via Multi- rater Agreement Modeling","cited_arxiv_id":null,"evidence_quote":"MRNet multi-rater agreement modeling motivates the latent preference distribution and serves as a comparison method."}],"review_version":1}