{"id":"a4cef5f9-c345-4e0a-87fb-41c609ab703a","arxiv_id":"2501.15610","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"RISE-MAR combines a radiologist-aligned image quality assessor with teacher-student self-training to improve CT metal artifact reduction on real clinical scans.","lead":"This paper introduces a training framework that uses a learned radiologist-aligned quality score to filter which AI-generated CT images are trusted as training targets, then retrains iteratively. It reports better metal artifact reduction on real CT scans than comparison methods, but its clinical evaluation relies heavily on the same quality scorer that guides training.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Clinical superiority rests on CQA as both training filter and test metric; no blinded radiologist validation of final outputs is provided, so the claimed clinical generalization is not yet established.","rationale":"The reader's conditional verdict is appropriate. Simulated-domain results on CTPelvic1K provide genuine out-of-domain evidence with PSNR/SSIM, so the framework has independent support. However, the paper's own framing emphasizes clinical generalization, and the clinical evidence is entirely CQA-based. The entanglement is not merely 'same metric used twice'; it is that the training procedure explicitly enriches the training set with images the CQA rates highly, so a CQA blind spot becomes a training signal, not just an evaluation artifact. The authors' own limitation statement about structural distortions makes this concrete. A blinded radiologist study is the natural settlement because it tests the actual claim of clinical utility rather than proxy agreement. I therefore keep the reader's CONDITIONAL verdict and recommend the external validation be a condition of acceptance.","tokens_in":16649,"tokens_out":3383,"duration_ms":32978,"concrete_test":"Perform a blinded reader study on the clinical test slices: randomly sample 100 slices per anatomy from the DeepLesion and Dental clinical test sets, present RISE-MAR and the three strongest baselines in randomized order to two radiologists who did not annotate CQA training data, and collect per-image quality scores and pairwise preferences. Then compute (i) SRCC between mean radiologist scores and CQA scores on the same outputs, and (ii) a paired test for RISE-MAR vs. the best baseline. If SRCC < 0.7 or RISE-MAR is not significantly preferred (p < 0.05), the clinical generalization claim should be downgraded. A cheaper auxiliary check: recompute the clinical rows of Tables II and III with a second independently trained CQA or a no-reference MAR-specific metric; if RISE-MAR's rank changes, the metric choice is driving the result.","verdict_should_be":"UNCHANGED","load_bearing_attack":"To support the central claim that RISE-MAR progressively improves pseudo-GT quality and generalizes better on real clinical CT, the paper relies on Tables II and III, where the clinical rows contain only CQA quality scores. The same CQA model is used in Eq. (7) as a filter (Q=[7,10]) to admit pseudo-GTs into training, and in Sec. III-E as the evaluation metric. Since the student is trained on CQA-approved pseudo-GTs, the model is explicitly shaped to satisfy the CQA's notion of quality. If the CQA were a perfect surrogate for radiologists, this would be desirable; but the paper does not show that CQA scores on final MAR outputs track radiologist judgments. The Discussion concedes the CQA 'may not adequately detect structural distortions when artifacts are not prominent.' This is precisely the failure mode that would let a method inflate CQA scores while producing clinically misleading images. No independent radiologist assessment of the final outputs is reported, so the clinical-superiority claim rests on a single learned metric that is entangled with the training objective.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes RISE-MAR, a semi-supervised CT metal artifact reduction framework that combines a radiologist-aligned Clinical Quality Assessor (CQA) with teacher-student self-training. A teacher network pretrained on simulated data generates pseudo ground-truths for unpaired clinical images; the CQA filters these candidates to a predefined quality range (Q=[7,10]), and the student is trained jointly on simulated pairs and CQA-approved clinical pseudo pairs. The teacher is updated by EMA, which the authors argue progressively improves the quality and quantity of pseudo ground-truths. Experiments compare RISE-MAR with supervised, unsupervised, and semi-supervised baselines on simulated in-domain, simulated out-of-domain, and real clinical test sets. On the simulated out-of-domain CTPelvic1K set, RISE-MAR attains the best PSNR, SSIM, and CQA quality; on the clinical DeepLesion and Dental sets, RISE-MAR reports the best CQA quality scores. The paper includes ablations of the CQA training strategy, the quality threshold Q, and the self-training components.","tokens_in":16878,"tokens_out":3418,"duration_ms":30823,"significance":"If the clinical claim is accepted, RISE-MAR addresses a real problem: semi-supervised MAR methods with fixed priors suffer from confirmation bias and limited pseudo-label diversity. The use of a learned quality assessor to filter pseudo ground-truths is a reasonable and potentially impactful idea, and the proposed DQAug strategy for training the assessor is a useful contribution. The simulated out-of-domain experiments on CTPelvic1K provide credible independent evidence of generalization: RISE-MAR improves PSNR from 46.42 (SemiMAR) to 47.88 and SSIM from 99.10% to 99.25%, and the qualitative error maps support the claim. The code is publicly available, the paper reports ablations for the main design choices, and the evaluation uses multiple datasets with different anatomies. However, the headline clinical-superiority claim depends entirely on the CQA metric, and the same CQA model is entangled with the training objective. Because the manuscript itself concedes that CQA may miss structural distortions in low-artifact images, the clinical conclusion requires independent radiologist validation or a substantially weakened claim.","major_comments":[{"comment":"The clinical-superiority claim is evaluated exclusively with CQA quality scores on the clinical test sets, but the same CQA model is used in Eqs. (6)-(7) to filter the pseudo ground-truths that train RISE-MAR (Q=[7,10]). The student network is therefore explicitly shaped to satisfy this particular quality scorer, so the clinical rows of Tables II and III may reflect self-reference rather than improved clinical generalization. The Discussion (Sec. IV) concedes that CQA 'may not adequately detect structural distortions when artifacts are not prominent,' which is exactly the failure mode that would let a method inflate CQA scores while producing clinically misleading images. To support the abstract's claim of enhanced generalization on real clinical CT images, the authors need an independent evaluation of the final outputs—for example, blinded radiologist ratings on a holdout set of clinical images—rather than only the CQA scores produced by the model used in training.","section":"Sec. III-E, Tables II and III, Eq. (6)-(7)"},{"comment":"The quality annotation protocol for paired images uses a piecewise rule based on absolute pixel errors in the region of interest, and the Discussion argues that CQA prioritizes clinical quality beyond PSNR/SSIM. If the radiologists' ratings are effectively derived from pixel-error thresholds, then the CQA may be approximating a distortion measure rather than a clinical acceptability measure, and the claimed distinction in Sec. IV is not established. The paper should report inter-rater agreement, the exact piecewise rule, and an independent radiologist validation of CQA scores on MAR outputs, especially given the acknowledged limitation that CQA may not detect structural distortions when artifacts are not prominent.","section":"Sec. II-A.1, Sec. III-A.4, Sec. III-E"},{"comment":"The quality threshold Q=[7,10] is selected on the simulated CTPelvic1K set using CQA quality, PSNR, and SSIM, and then the same CQA is used as the primary clinical metric. This introduces a selection-bias interaction: the clinical comparison is not a neutral test of the method. At minimum, the authors should report the clinical CQA scores for the ablated configurations (e.g., Q=[1,10], Q=[9,10]) on the clinical test sets, and provide confidence intervals or significance tests for the differences in Tables II and III. Without such information, the magnitude of the clinical gains (e.g., 8.554 vs. 8.154 on DeepLesion) cannot be assessed.","section":"Sec. III-F.3, Table IV, Sec. III-E"}],"minor_comments":[{"comment":"The phrase 'Self-T raining' in the title and in the section header contains an erroneous space; please fix.","section":"Title and Sec. II heading"},{"comment":"Using p for both the one-hot quality label and the probability vector pψ in Eq. (2) is confusing; rename the label (e.g., y) to avoid ambiguity.","section":"Eq. (2)"},{"comment":"The rows labeled with checkmarks, crosses, and λ values are not self-explanatory; please clarify the configuration of each row and the meaning of the color-coded bars in the caption.","section":"Fig. 6"},{"comment":"The caption refers to RISE-MAR, but the figure legend is not labeled; ensure the curve colors are defined in the caption so that the reader can map each curve to a method.","section":"Fig. 4"},{"comment":"The 'Supervised [13]' baseline appears to cite the β-CycleGAN paper; if the supervised model is taken from that work its architecture should be made explicit, otherwise a separate citation for the supervised baseline is needed.","section":"Sec. III-D"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is within the journal's scope and the code release is appreciated. The main risk is that the clinical claim outruns the evidence: the clinical evaluation is entirely mediated by the same CQA model used to select training data, and the Discussion concedes a relevant failure mode. I would like the editor to ensure that the requested independent radiologist evaluation (or a clearly weakened clinical claim) is a condition of revision. I do not see a novelty-disclosure issue, but the relationship to the authors' own prior work on the CQA encoder (Ref. [25]) should be clarified if any architectural overlap exists."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my take. The genuinely new thing in this paper is using a radiologist-trained quality model to filter pseudo-labels in a teacher-student self-training loop for MAR, and the authors show the moving parts work. The out-of-domain simulated results on CTPelvic1K are the most credible evidence in the paper: RISE-MAR beats the baselines on PSNR and SSIM there, and those metrics don't touch the CQA. The ablations are thorough and each component earns its place. I also give them credit for building and validating the CQA against real radiologist annotations (SRCC/PLCC), and for stating the limitations plainly.\n\nThe soft spot is exactly where the stress-test note lands. The clinical comparison on DeepLesion and Dental is scored with the same CQA model that was used to filter the pseudo ground-truths during training (Eq. 7 and Sec. III-E). So the model is shaped to please the CQA, and then the CQA is used to declare victory. The authors concede the CQA may miss structural distortions when artifacts are not prominent, which is the failure mode that could inflate scores while hiding clinically important mistakes. Without a blinded radiologist reading of the final outputs, the \"superior generalization on real clinical CT\" claim is not established. That's a load-bearing gap for the headline, not a nitpick.\n\nTwo smaller things. The dental dataset is private and the split details for the clinical quality assessment dataset are thin (9:1 overall, but no patient-level separation), so reproducibility is limited. And the \"radiologist-in-the-loop\" label is a bit generous—radiologists annotate once, then the CQA replaces them. That's fine, but the title oversells the loop.\n\nIf I were refereeing, I would ask for independent clinical evaluation, either a small reader study or at minimum a separate radiologist-scored test set that wasn't used to train the CQA. The simulated out-of-domain experiments can stay as the quantitative backbone.\n\nThis is a solid methods paper with a real idea, and the core mechanism is supported by independent simulated results. The clinical claim needs more work, but the paper deserves serious review and is worth building on.","headline":"A solid semi-supervised MAR method with a genuinely useful quality-filtering idea, but the clinical superiority claim rests on a circular CQA evaluation and needs independent radiologist confirmation.","tokens_in":17416,"tokens_out":2426,"would_cite":true,"duration_ms":21650,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that putting a radiologist-aligned quality assessor inside a self-training loop lets CT metal artifact reduction generalize to real clinical images, with the assessor doubling as a no-ground-truth evaluation metric.","keywords":["CT metal artifact reduction","semi-supervised learning","self-training","clinical quality assessment","pseudo ground-truth selection","domain generalization","radiologist-in-the-loop","medical image quality assessment"],"falsifier":"Take a fresh set of real-metal-artifact CTs that were not used in CQA training, run RISE-MAR, SemiMAR, UDAMAR, and β-CycleGAN on them, and have independent radiologists blindly rank the outputs by diagnostic quality. If the CQA scores do not match the radiologists' rankings on these unseen outputs, the load-bearing claim that CQA provides radiologist-aligned generalization would be refuted.","tokens_in":16450,"feed_emoji":"🩻","tokens_out":9190,"duration_ms":75038,"temperature":0.7,"pith_summary":"RISE-MAR aims to solve a specific failure: MAR networks trained on simulated CT artifacts lose accuracy on real clinical scans because no paired ground truth exists there. The paper's central claim is that this domain gap can be closed by putting a radiologist-aligned clinical quality assessor (CQA) inside a self-training loop. The CQA, trained on radiologists' ten-point quality ratings, decides which pseudo ground-truths produced by a teacher MAR network are good enough to train a student network, and the student then refreshes the teacher by exponential moving average, producing more high-quality pseudo ground-truths over time. On clinical test sets from DeepLesion and a private dental dataset, RISE-MAR obtains the best CQA quality scores among the compared methods, and the paper argues this indicates better generalization to real metal artifacts. A sympathetic reader would care because the same CQA could serve as an automated evaluation metric where ground truth is unavailable.","feed_headline":"Radiologist-aligned filter boosts clinical CT metal artifact reduction","feed_subtitle":"A radiologist-aligned score picks high-quality pseudo labels in a self-training loop, lifting clinical CT scores.","key_machinery":"The load-bearing object is the clinical quality assessor (CQA), a transformer encoder with a quality head that maps a CT image to a ten-point clinical quality score, trained to match two radiologists' consensus ratings. The CQA is the gate in Eqs. (6)–(7): a pseudo ground-truth $\\tilde{Y}_{\\mathrm{cli}}$ from the teacher is kept only when its predicted quality $q_{\\tilde{Y}}$ falls in $Q=[7,10]$; otherwise the sample is discarded to avoid confirmation bias. The second half of the mechanism is self-training: the student learns from verified pseudo pairs plus simulated pairs, and the teacher's parameters are updated by exponential moving average, so the teacher's clinical knowledge improves and more of its future predictions pass the CQA gate. This two-part loop—quality filter plus expanding pseudo ground-truth pool—is what transfers simulated-domain MAR knowledge to real clinical CT images.","core_discovery":"On its own terms, the paper's central claim is that combining quality assurance and quantity assurance is what makes semi-supervised MAR generalize. Quality assurance is the CQA: a transformer-based multi-scale encoder with a quality head that outputs a quality score $\\hat{q}=\\sum_{k=1}^{10} k p^{(k)}$ on a ten-point Likert scale, trained with cross-entropy and supervised contrastive loss plus Diverse Quality Augmentation (MixUp across low-, moderate-, and high-quality images). Quantity assurance is the teacher–student loop: teacher $f_\\phi$ pretrained on simulated pairs proposes pseudo ground-truths for unpaired clinical CTs; CQA accepts only those with quality in $Q=[7,10]$; the student $f_\\theta$ trains on the resulting pseudo pairs together with simulated pairs; and EMA with decay $\\eta=0.999$ updates the teacher. The reported outcome is that RISE-MAR achieves the highest CQA quality on clinical DeepLesion (8.55) and clinical Dental (7.30), with visual results showing streak removal and restoration of bony and soft-tissue structures. The paper concludes that radiologist feedback distilled into CQA, rather than a fixed prior network, is what prevents confirmation bias and expands the clinical training pool.","pith_inferences":["We infer that the reported clinical quality gap is partly self-consistent by construction: the same CQA that filters training pseudo ground-truths also scores the final test outputs, so an independent blinded radiologist re-read of RISE-MAR and comparison outputs would be a stronger test of clinical benefit.","We infer that the CQA's blindness to subtle structural distortions in artifact-light regions, acknowledged in the paper's limitations, means the quality gate may admit clinically misleading pseudo ground-truths even when the numeric score is high.","We infer that a natural extension is to use the CQA score as a continuous reward (for example via reinforcement learning) rather than a binary gate, which the paper names as future work and which would let quality feedback shape the MAR network's updates directly.","We infer that the fixed quality range $Q=[7,10]$ may not transfer to new clinical domains: the optimal threshold depends on the distribution of pseudo ground-truth qualities, and the ablation shows both stricter and looser ranges hurt performance."],"forward_implications":["Semi-supervised MAR no longer needs a frozen pretrained prior: the teacher network's clinical knowledge is updated during training, so pseudo ground-truths improve in quality as training proceeds.","CQA provides a quantitative evaluation route for clinical MAR outputs where PSNR and SSIM cannot be computed, since no artifact-free ground truth exists for real metal artifact cases.","The ablation results imply the quality threshold matters: too loose a range admits low-quality pseudo ground-truths and degrades out-of-domain performance, while too strict a range starves the training pool.","Because the MAR backbone can be any effective learning-based network, the same radiologist-in-the-loop recipe can be attached to other MAR architectures, including those that consume linear-interpolation backprojected images.","The CQA quality scores on clinical test sets are the paper's headline evidence for generalization across anatomies (torso and dental)."],"supporting_citations":[{"why":"Supplies the unsupervised domain-adaptation baseline and the protocol for selecting artifact-free DeepLesion images.","marker":"[20]"},{"why":"Introduced the fixed-prior pseudo ground-truth approach that RISE-MAR argues is limited by confirmation bias.","marker":"[19]"},{"why":"Semi-supervised MAR baseline that uses a fixed prior network for pseudo ground-truths and is a direct comparison in Tables II and III.","marker":"[21]"},{"why":"Provides the attention U-Net architecture used as the MAR backbone and the unsupervised β-CycleGAN baseline.","marker":"[13]"},{"why":"Establishes the confirmation-bias risk in pseudo-label semi-supervised learning that motivates the CQA quality gate.","marker":"[23]"},{"why":"Supplies MixUp augmentation used in the Diverse Quality Augmentation strategy for training CQA.","marker":"[24]"},{"why":"Provides the metal artifact simulation algorithm used to create paired simulated training and test data.","marker":"[30]"},{"why":"Supplies the DeepLesion clinical CT images and simulated dataset used for training and clinical evaluation.","marker":"[28]"},{"why":"Supplies the CTPelvic1K dataset used for out-of-domain simulated generalization comparison.","marker":"[29]"}],"fun_headline_variants":["Radiologist-in-loop self-training cuts CT metal artifacts on real scans","Radiologist feedback steers pseudo-label quality for CT metal artifact reduction","Self-training with radiologist QA improves real CT metal artifact reduction","RISE-MAR: iterative radiologist-guided self-training for CT artifact removal"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The framework assumes the CQA's learned ten-point quality score faithfully reproduces radiologist judgment on clinical images outside CQA's training distribution, and that using the same CQA both to select training pseudo ground-truths and to score final clinical test outputs does not bias the comparison.","fun_headline_variants_meta":{"raw":{"variants":["Radiologist-in-loop self-training cuts CT metal artifacts on real scans","Radiologist feedback steers pseudo-label quality for CT metal artifact reduction","Self-training with radiologist QA improves real CT metal artifact reduction","RISE-MAR: iterative radiologist-guided self-training for CT artifact removal"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000505,"raw_usage":{"total_tokens":2524,"prompt_tokens":1066,"completion_tokens":1458,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":682,"completion_tokens_details":{"reasoning_tokens":1380}},"tokens_in":682,"tokens_out":1458,"duration_ms":10528,"temperature":1.0,"reasoning_tokens":1380,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T14:06:19.444350+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a fresh set of real-metal-artifact CTs that were not used in CQA training, run RISE-MAR, SemiMAR, UDAMAR, and β-CycleGAN on them, and have independent radiologists blindly rank the outputs by diagnostic quality. If the CQA scores do not match the radiologists' rankings on these unseen outputs, the load-bearing claim that CQA provides radiologist-aligned generalization would be refuted.","supporting_citations":[{"cited_title":"Deep- learning-based metal artefact reduction with unsupervised domain adap- tation regularization for practical CT images,","cited_arxiv_id":null,"evidence_quote":"Supplies the unsupervised domain-adaptation baseline and the protocol for selecting artifact-free DeepLesion images."},{"cited_title":"U-DuDoNet: Unpaired dual- domain network for CT metal artifact reduction,","cited_arxiv_id":null,"evidence_quote":"Introduced the fixed-prior pseudo ground-truth approach that RISE-MAR argues is limited by confirmation bias."},{"cited_title":"SemiMAR: Semi-supervised learning for CT metal artifact reduction,","cited_arxiv_id":null,"evidence_quote":"Semi-supervised MAR baseline that uses a fixed prior network for pseudo ground-truths and is a direct comparison in Tables II and III."},{"cited_title":"Unsupervised CT metal artifact learning us- ing attention-guided β-CycleGAN,","cited_arxiv_id":null,"evidence_quote":"Provides the attention U-Net architecture used as the MAR backbone and the unsupervised β-CycleGAN baseline."},{"cited_title":"Pseudo-labeling and confirmation bias in deep semi-supervised learn- ing,","cited_arxiv_id":null,"evidence_quote":"Establishes the confirmation-bias risk in pseudo-label semi-supervised learning that motivates the CQA quality gate."},{"cited_title":"mixup: Beyond empirical risk minimization,","cited_arxiv_id":null,"evidence_quote":"Supplies MixUp augmentation used in the Diverse Quality Augmentation strategy for training CQA."},{"cited_title":"Convolutional neural network based metal artifact reduction in x-ray computed tomography,","cited_arxiv_id":null,"evidence_quote":"Provides the metal artifact simulation algorithm used to create paired simulated training and test data."},{"cited_title":"DeepLesion: automated mining of large-scale lesion annotations and universal lesion detection with deep learning,","cited_arxiv_id":null,"evidence_quote":"Supplies the DeepLesion clinical CT images and simulated dataset used for training and clinical evaluation."},{"cited_title":"Deep learning to segment pelvic bones: large-scale CT datasets and baseline models,","cited_arxiv_id":null,"evidence_quote":"Supplies the CTPelvic1K dataset used for out-of-domain simulated generalization comparison."}],"review_version":1}