{"id":"803e6cd3-62cb-477e-b57d-4956ba02baf1","arxiv_id":"2501.10733","paper_version":2,"verdict":"REJECT","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"A CNN-Transformer trained on longitudinal 3D MRIs claims high accuracy for predicting next-scan hepatocellular carcinoma, but its time-aware positional encoding reveals the future diagnosis date to the model.","lead":"This paper presents HCCNet, a neural network that combines 3D MRI scans taken over time to predict whether a cirrhosis patient will develop liver cancer at the next screening. The model is meant to support personalized surveillance, but the reported accuracy is compromised because the positional encoding uses the date of the future cancer diagnosis as an input, leaking the answer into the model.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (4) leaks the HCC diagnosis date into the positional encoding, so the reported AUROC/AUPRC do not measure prospective predictive ability; the central claim is unsupported.","rationale":"I read the paper in good faith: it proposes a reasonable CNN-Transformer architecture for longitudinal 3D MRI, includes code, and attempts two-stage pre-training. The central claim, however, is that HCCNet provides a robust tool for personalized HCC surveillance by predicting HCC at the next screening. For that claim to hold, the evaluation must measure genuine prospective prediction. Equation (4) makes that impossible: for positive patients the positional encoding is anchored to the future diagnosis date t_HCC, and for negative patients it is anchored to the last observed MRI. This creates a systematic, class-discriminative feature at test time, independent of the MRI content. The internal asymmetry between the final interval for positive patients (positive time to diagnosis) and for negative patients (zero) is a concrete mechanism by which the model can separate classes trivially. The reader's weakest-assumption analysis identified exactly this leakage, and my independent reading agrees. Since the reported metrics and the baseline comparison are contaminated by the same leak, the central claim is unsupported as submitted. The concern is not about disagreement with current consensus or about stylistic choices; it is an internal flaw in the evaluation. I therefore see no reason to change the reader's REJECT verdict, and I recommend no adjustment beyond the already-stated rejection.","tokens_in":18448,"tokens_out":4179,"duration_ms":46972,"concrete_test":"Rerun the fine-tuning and evaluation on the same development/test split with a non-leaky positional encoding that uses only observed screening dates for both classes, e.g. replace the positive-patient branch of Eq. (4) with the same backward-looking interval rule used for negative patients, or use ordinal sequence positions. If the DW-MRI AUROC falls from about 0.93 toward the randomly initialized baseline of about 0.71, the reported gain is attributable to the leaked future endpoint. A cheaper confirmatory check is to train a logistic regression on the final positional-encoding value alone (or on the full positional-encoding sequence) with no image features; if it achieves high AUROC on the test split, the leakage is demonstrated directly.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The decisive problem is Equation (4). For patients who develop HCC, every positional interval is computed relative to the future diagnosis date t_HCC, while for patients who do not develop HCC every interval is computed relative to the last registered MRI. Concretely, the final element of the positional-encoding sequence is sqrt(t_HCC - t_last) > 0 for every positive patient, but is 0 for every negative patient, because the negative reference point is the final screening itself. Since the positional encoding is added to the image embeddings at both training and test time, a model can separate the two classes from the positional encoding alone, without using any image content. The test set is encoded identically, using the retrospectively known diagnosis date, so the reported average AUROC of 0.928 and AUPRC of 0.687 do not measure the ability to predict HCC at the next screening. The baseline comparison in Table 2 shares the same leak, so it cannot support the claim that pre-training improves predictive accuracy. The paper's own statement that the method 'predicts whether the patient will be diagnosed with HCC at the next examination' is also not matched by the label in Eq. (1), which marks any patient with t_HCC != None as positive regardless of how many examinations remain; however, the positional-encoding leak alone invalidates the central claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes HCCNet, a hybrid 3D CNN-Transformer architecture for predicting hepatocellular carcinoma (HCC) from longitudinal MRI data. The model uses a 3D ConvNeXt backbone with DINO-style pretraining and a Transformer encoder pretrained via sequence-order prediction, and it encodes irregular screening intervals through time-based positional encodings. The authors evaluate on a retrospective cohort of cirrhosis patients, reporting average AUROC 0.928 and AUPRC 0.687 for DW-MRI, and claim that pre-training improves performance and calibration over a randomly initialized baseline.","tokens_in":18677,"tokens_out":5586,"duration_ms":54363,"significance":"If the reported results were valid, the work would be a useful contribution to longitudinal medical imaging: it addresses a realistic clinical task, handles irregular timing, and provides a reproducible codebase. The pre-training strategy for 3D MRI is a reasonable extension of existing self-supervised methods. However, the central evaluation is compromised by a label-leaking positional encoding, so the claimed predictive performance is not supported. The paper's contribution therefore reduces to an architectural proposal whose empirical validation remains to be demonstrated.","major_comments":[{"comment":"The positional encoding in Eq. (4) uses the date of HCC diagnosis tp_HCC to compute time intervals for patients who develop HCC, while for patients without HCC it uses the last registered MRI. Since tp_HCC is exactly the outcome the model is meant to predict, the input contains the label at training and test time. Concretely, for a positive patient the final positional-encoding element is sqrt(t_HCC - t_last) > 0, whereas for a negative patient the analogous element is 0 (the distance from the last MRI to itself). Because the positional encoding is added to the image embeddings in Eq. (6), a model can discriminate the two classes from the positional features alone, without any image content. All reported performance figures (Tables 1 and 2, Figures 2 and 3) are therefore uninformative about genuine predictive ability. The baseline comparisons in Table 2 share the same leak, so the claimed benefit of pre-training is also not established.","section":"Section 3.2, Eq. (4)"},{"comment":"The task stated in the abstract and Section 1 is to predict whether the patient will be diagnosed with HCC at the next examination. However, Eq. (1) assigns yp = 1 to any patient with tp_HCC != None, regardless of how many screenings separate the last observed MRI from the diagnosis. A positive patient whose last record is several years before diagnosis is scored as a success even though the 'next examination' after the last observation may not have occurred or may not have led to a diagnosis. The evaluation should be aligned with the stated clinical scenario, e.g., by defining positives based on the next screening result or by using time-to-event modeling.","section":"Section 3.1, Eq. (1)"}],"minor_comments":[{"comment":"The sentence 'we additionally exclude all patients without a definitive diagnosis of HCC yielding a total of 243 patients of which 37 develop HCC' is ambiguous; please clarify the exclusion criterion and report the number of excluded patients.","section":"Section 4.1"},{"comment":"The definition of Δtp_i is hard to parse: the index i appears on the left, while the right-hand side mixes tp_{i+1} and tp_HCC. Please state explicitly how the vector is constructed for the last time point and how it is aligned with the sequence of image embeddings.","section":"Section 3.2, Eq. (4)"},{"comment":"The note states that AUROC ranges from 0.50 to 1; this is not a general property of AUROC and should be revised, or the observed range should be reported separately.","section":"Table 1 note"}],"recommendation":"reject","confidential_remarks":"The label leakage in Eq. (4) is a fundamental methodological flaw that invalidates all reported experimental results. Although the code and architecture may be of interest, a revision would require redefining the positional encoding to use only information available at the time of prediction, re-running the entire evaluation, and potentially reframing the clinical claim. This goes beyond a standard revision; the current manuscript should not be accepted. If the authors can redo the experiments and show a meaningful effect, resubmission could be considered."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing you should know: the architecture is legitimate and the pre-training scheme is a real novelty, but the main result is not trustworthy. Eq. (4) leaks the future diagnosis date into the positional encoding for positive patients, so the reported AUROC/AUPRC reflect the model reading the answer from the input, not predicting HCC.\n\nWhat is genuinely new: a 3D ConvNeXt backbone feeding a Transformer encoder for longitudinal volumetric MRI, with DINO-style SSL for the backbone (treating MRI sequences as augmentations), a sequence-order-prediction task for the Transformer, and a square-root time-interval positional encoding. The paper is clearly written, the experiments cover four model sizes and three MRI modalities, and the code is available. That is a solid engineering contribution.\n\nThe soft spot is decisive. In Eq. (4), every interval for a positive patient is measured relative to t_HCC, the diagnosis date, while for a negative patient it is measured relative to the last registered MRI. The final positional-encoding element is therefore sqrt(t_HCC - t_last) > 0 for every positive and exactly 0 for every negative. Since the positional encoding is added to the image embeddings at both training and test time, a trivial classifier can separate the classes from the positional encoding alone. The test set is encoded the same way, so the AUROC of 0.928 is not a measure of prospective prediction. The baseline in Table 2 shares the leak, so the pre-training benefit claims are also unsupported. There is also a mismatch between the stated task (predict HCC at the next examination) and the label definition in Eq. (1), which marks a patient positive if t_HCC != None regardless of the prediction horizon. I would mention both to the authors, but the positional-encoding leak alone invalidates the central claim.\n\nOn the positive side, the authors acknowledge the need for external validation and discuss the variability across modalities, which is honest. But they never acknowledge the leak, and the abstract's claim that the model provides a robust tool for personalized surveillance is not supported by the evidence as presented.\n\nI would not publish this version. I would, however, send it to peer review rather than desk reject: the core ideas are worth testing, and the leak is fixable by using only past time points in the positional encoding (e.g., intervals to the last observation or to a fixed reference date). A properly re-evaluated version could be a useful contribution.\n\nMy recommendation to the editor: send it out, with a reviewer who knows temporal reasoning pitfalls. The current version should be rejected, but a resubmission with a non-leaky encoding and, ideally, external validation deserves a fair shot.","headline":"A legitimate architectural contribution is undermined by a label leak in the positional encoding: the reported AUROC/AUPRC do not measure predictive ability, but the ideas are worth testing properly.","tokens_in":19251,"tokens_out":2460,"would_cite":false,"duration_ms":24583,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a 3D CNN-Transformer using a patient's longitudinal diffusion-weighted MRI history predicts hepatocellular carcinoma at the next screening, with an average AUROC of 0.928 and AUPRC of 0.687.","keywords":["hepatocellular carcinoma prediction","longitudinal 3D MRI","CNN-Transformer hybrid","convolutional backbone","self-supervised pre-training","time-interval positional encoding","diffusion-weighted MRI","liver cirrhosis surveillance"],"falsifier":"Recompute the same test-set evaluation using positional encodings anchored only to each patient's most recent available scan, with no reference to the future diagnosis date, and compare the AUROC and AUPRC to the reported 0.928 and 0.687; a large drop would show that the diagnosis-date anchor, not genuine temporal prediction, is doing the work.","tokens_in":18221,"feed_emoji":"🩻","tokens_out":8659,"duration_ms":82490,"temperature":0.7,"pith_summary":"This paper seeks to show that a patient's full MRI history, rather than a single scan, can be used to predict whether hepatocellular carcinoma will be found at the next surveillance examination. The proposed model, HCCNet, combines a 3D convolutional backbone with a Transformer encoder and is pre-trained in two stages before being fine-tuned on a small cohort of cirrhosis patients. On diffusion-weighted MRI the fine-tuned models reach an average AUROC of 0.928 and an average AUPRC of 0.687, with pre-training raising AUPRC by up to 156 percent over a from-scratch baseline. If these results hold, the same architecture could support personalized surveillance intervals and transfer to other chronic diseases monitored by repeated imaging.","feed_headline":"Liver cancer risk predicted from serial MRI with 0.93 AUROC","feed_subtitle":"A CNN-transformer on repeated 3D scans flags which cirrhosis patients will develop HCC by the next screening.","key_machinery":"The load-bearing mechanism is the HCCNet architecture itself: a 3D adaptation of ConvNeXt (a modern convolutional network) whose depthwise kernels are inflated to three dimensions and shortened, paired with a Transformer encoder with pre-layer normalization. Temporal position is encoded with fixed sinusoids of the square root of the time interval in months between each scan and an anchor date (the HCC diagnosis date for patients who develop HCC, the last registered screening otherwise), which is what lets the model handle irregular and variable-length patient histories. Before fine-tuning, the backbone is pre-trained with a teacher-student self-supervised objective in which different MRI sequences of the same patient act as natural augmentations, and the Transformer is pre-trained to tell shuffled sequences of image embeddings from intact ones. The [cls] token's final hidden state is passed through a small pooling head to produce the risk probability.","core_discovery":"The paper's central claim is that temporal structure in repeated whole-volume MRIs carries usable signal for future HCC diagnosis, and that a 3D convolutional-plus-sequence model can extract it. HCCNet embeds every 3D scan with a 3D ConvNeXt backbone, orders the embeddings by time, adds sinusoidal positional encodings derived from time intervals, and lets a Transformer encoder read the resulting sequence; the [cls] token's final hidden state is classified as 'HCC at the next examination' or not. According to the paper's retrospective evaluation, DW-MRI variants achieve an average AUROC of 0.928 and AUPRC of 0.687, outperform the same architecture trained from scratch by roughly 30 percent in AUROC and up to 156 percent in AUPRC, and produce better-calibrated confidence scores even when discrimination gains are modest.","pith_inferences":["The paper's evaluation is retrospective: because the positional encoding uses the diagnosis date, the reported 0.928 AUROC is an upper bound on what a clinician would see at decision time, not a prospective estimate.","An ablation that replaces the diagnosis-date anchor with last-scan date would separate 'time until diagnosis' from 'time since last surveillance', and would clarify whether the model learns true risk or proximity to diagnosis.","Treating MRI sequences (e.g., different b-values) as natural augmentations is a cheap pre-training trick that other multi-sequence imaging studies could reuse without hand-designed color augmentations.","If the modality-dependent pretraining gains are real, a single universal longitudinal-MRI checkpoint may be the wrong target; modality-specific pretraining is the likelier route."],"forward_implications":["A working DW-MRI predictor could let surveillance programs rank cirrhosis patients by next-screening risk and tailor scan intervals instead of using one fixed schedule.","The two-stage pre-training appears to carry most of the benefit on DW-MRI, so the recipe may transfer to other small longitudinal-imaging datasets with sparse outcome labels.","Because the model accepts variable-length records and irregular gaps, it can serve as a template for monitoring chronic diseases beyond HCC.","Across the four model sizes, imaging modality matters far more than parameter count, suggesting a small model is enough for deployment if the right sequence type is used."],"supporting_citations":[{"why":"Supplies the ConvNeXt architecture that the paper inflates to 3D for its CNN backbone.","marker":"Liu et al. (2022)"},{"why":"Supplies the Transformer encoder used to model the sequence of image embeddings.","marker":"Vaswani et al. (2023)"},{"why":"Provides the teacher-student self-supervised pre-training objective that the backbone pre-training adapts to 3D MRI.","marker":"Caron et al. (2021)"},{"why":"Provides the sequence-order-prediction idea used to pre-train the Transformer on shuffled patient records.","marker":"Lan et al. (2020)"},{"why":"Motivates time-aware pre-training for healthcare sequences and supports the use of temporal structure in the encoder.","marker":"Ren et al. (2021)"},{"why":"Prior HCC risk prediction from longitudinal ultrasound (AUC 0.775) that HCCNet is positioned against as the closest longitudinal benchmark.","marker":"Zhang et al. (2022b)"},{"why":"Gives the Pre-LN Transformer implementation used as the encoder base.","marker":"Xiong et al. (2020)"},{"why":"Provides the clinical rationale for early HCC detection and surveillance that frames the task.","marker":"Parikh et al. (2020)"}],"fun_headline_variants":["Serial MRI plus AI flags future liver cancer risk","CNN-transformer uses time in MRIs to predict HCC","Temporal MRI patterns boost liver cancer prediction","HCCNet: AI reads scan history to forecast HCC","Longitudinal MRI and transformer up HCC prediction"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"For patients who develop HCC, the time intervals used in the positional encoding are computed backward from the HCC diagnosis date, so the model receives the very outcome it is supposed to predict during training and evaluation.","fun_headline_variants_meta":{"raw":{"variants":["Serial MRI plus AI flags future liver cancer risk","CNN-transformer uses time in MRIs to predict HCC","Temporal MRI patterns boost liver cancer prediction","HCCNet: AI reads scan history to forecast HCC","Longitudinal MRI and transformer up HCC prediction"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000707,"raw_usage":{"total_tokens":3208,"prompt_tokens":992,"completion_tokens":2216,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":608,"completion_tokens_details":{"reasoning_tokens":2143}},"tokens_in":608,"tokens_out":2216,"duration_ms":15329,"temperature":1.0,"reasoning_tokens":2143,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T19:01:33.881969+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute the same test-set evaluation using positional encodings anchored only to each patient's most recent available scan, with no reference to the future diagnosis date, and compare the AUROC and AUPRC to the reported 0.928 and 0.687; a large drop would show that the diagnosis-date anchor, not genuine temporal prediction, is doing the work.","supporting_citations":[{"cited_title":", author Mehta, A.S","cited_arxiv_id":null,"evidence_quote":"Provides the clinical rationale for early HCC detection and surveillance that frames the task."}],"review_version":1}