{"id":"d85ab1d2-a4e4-44f3-aa88-11b5f5000c96","arxiv_id":"2502.06134","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"A joint sequence-image framework with contrastive and clustering losses improves classification of irregular medical time series with heavy missingness.","lead":"This paper combines a recurrent imputation network with image-based analysis of medical time series, then uses contrastive and clustering losses to fuse the two views. On three clinical datasets it reports higher classification accuracy and robustness to missing data than seven published baselines.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Image construction is underspecified: transforms may use raw sparse series or imputed X'. At 94.9% missingness, GAF/MTF/recurrence plots require some fill, so the image-only ablation and fused gains may depend on an unstated leak from the sequence branch.","rationale":"The paper's central claim is empirical: the fused sequence+image representation with three SSL losses is the best classifier on PAM/P12/P19 under the 8:1:1 split. The condition that must hold for this claim to be interpretable is that the image branch receives a well-defined input. The most vulnerable point is how missing values are handled when constructing the six image transforms. With missing rates up to 94.9%, pairwise transforms such as GAF, MTF, and recurrence plots cannot be computed on the raw sparse tensor unless some fill or imputation is applied; the only in-paper mechanism producing a dense series is the generator's X'. If images are made from X', the two views are not independent, the image-only ablation has no well-defined construction, and the final classification loss can inject label information into images through G. The text points to Appendix A for details, but Appendix A is absent from the arXiv v1, so this is not a resolvable omission. This matches the reader's weakest_assumption, so I agree. I am not recommending rejection: the result may survive, but acceptance must remain conditional on specifying and verifying the image construction and demonstrating that the image-only ablation does not depend on a task-trained imputer. Secondary issues such as the medoid-like cluster update in Eq. 12, the supervised nature of Eq. 9, and the unclear P19 label description are real but do not displace this central concern.","tokens_in":12957,"tokens_out":9133,"duration_ms":90462,"concrete_test":"Run the linked repository's data pipeline for one P19 batch with a debug print of the tensor passed to the Swin encoder, recording whether each of the six images is built from raw X (with NaNs masked) or from X' after imputation. Then rerun the Table 4 image-only ablation using images built from raw X with a fixed linear/mean fill and with a generator that has not been exposed to Lclf; if the image-only F1 drops materially, or if the code path cannot produce images without a task-trained X', the unstated dependence on the sequence branch is confirmed.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is the Table 2 result that the fused sequence+image model with three SSL losses beats seven SOTA baselines on PAM/P12/P19. That claim requires the image branch to be a well-defined input stream. The Approach section describes six image transforms (line graph, frequency spectrum, GAF/MTF, recurrence plot) but never states whether missing entries are dropped, zero-filled, interpolated, or replaced by X' from the generator G. Table 1 lists missing rates up to 94.9%; GAF, MTF, and recurrence plots require pairwise computations on dense series, so a raw tensor with mostly NaNs cannot feed a fixed ImageNet-pretrained Swin Transformer. If the images are rendered from X', then (i) the sequence and image views are not independent, (ii) the joint representation double-counts G's imputation, (iii) the image-only row in Table 4 (F1 95.4) is ill-defined because no image construction is specified without G, and (iv) since the final objective includes Lclf, task-specific information can flow into X' and then into the images, making the claimed self-supervised image signal partially label-leaking. This does not make the engineering result impossible, but it makes the reported gains and ablation non-reproducible as stated. The text defers this to Appendix A, which is absent from the arXiv v1, so the omission cannot be resolved from the paper alone.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a joint learning framework for irregular medical time series that combines a sequence branch (a generator-discriminator imputation model with a BiRNN) and an image branch (six time-series-to-image transforms processed by a pretrained Swin Transformer). Three additional losses are introduced: an inter-sequence NT-Xent reconstruction loss, a sequence-image contrastive loss with a margin, and a clustering loss on the fused joint representations. The authors report state-of-the-art classification results on PAM, P12, and P19, robustness gains under leave-sensors-out and leave-samples-out missingness on PAM, an ablation study, and a clinical Turing test. The paper currently appears as an arXiv v1 that refers to appendices A-D, which are not present in the manuscript text provided, leaving several load-bearing details unspecified.","tokens_in":13286,"tokens_out":5268,"duration_ms":49561,"significance":"If the reported results are reproducible and the three losses are correctly characterized, the paper would make a strong empirical contribution: it demonstrates consistent gains over seven baselines on three clinical datasets, with five-run statistics and a robustness analysis under two missingness scenarios. The code link and the clinical Turing test are additional positive elements. However, the current manuscript leaves essential implementation details in absent appendices, and one of the three 'self-supervised' losses requires labels, so the significance can only be assessed after those gaps are closed. The core idea of fusing sequence and image representations for irregular time series is plausible and worth pursuing, but the paper as written does not yet provide a fully checkable account of its own method.","major_comments":[{"comment":"The image construction is underspecified in a load-bearing way. The text states that six image transforms are used and defers details to Appendix A, but Appendix A is absent from the arXiv v1. With missing rates up to 94.9% (P19), the Gramian Angular Field, Markov Transition Field, and recurrence plot transforms require dense series to compute pairwise quantities, yet the paper never states whether the images are built from the raw sparse X, from a zero-filled/interpolated version, or from the imputed sequence X' produced by the generator G. If images are built from X', then the image branch is not independent of the sequence branch, the image-only ablation (Table 4, F1 95.4) is not well-defined without G, and because the final objective includes the classification loss Lclf, label information can flow into X' and then into the images, making the claimed self-supervised image signal partially label-leaking. The authors must specify the exact image construction pipeline, including missing-value handling, and state clearly whether G is updated by Lclf.","section":"Imaging Time Series / Table 1"},{"comment":"The sequence-image contrastive loss is described as 'self-supervised,' but the definition uses P(i), the set of all sample indices in the same category as sample i. This requires ground-truth labels, making it a supervised contrastive loss, not a self-supervised loss. This is not merely a terminological issue: it means that one of the three claimed SSL strategies uses labels during representation learning, and if those labels are the same classification labels used in Lclf, the representation may benefit from task supervision beyond the final classifier. Please relabel the loss and clarify whether the labels used to form P(i) are the training labels from the classification task.","section":"Eq. (9)"},{"comment":"Equation (12) defines the cluster center update as C_k = argmin_{u in S_k} sum_{u' in S_k} ||u - u'||^2, which selects the medoid of the cluster, not the K-means mean. The text repeatedly calls the procedure K-means and Eq. (10) is a standard K-means assignment step, so the algorithm as written is inconsistent. If the medoid update is intentional, the name should be changed; if K-means is intended, Eq. (12) should be the arithmetic mean of the members of S_k. This matters because the clustering loss in Eq. (11) is defined with respect to these centers.","section":"Eq. (12)"},{"comment":"The clustering loss computes cluster assignments and centers from the same fused features u that are being optimized by L_cluster. This self-referential loop can lead to trivial or degenerate solutions (for example, all points assigned to one cluster, or centers collapsing to a single point) unless safeguards are specified. The paper does not state how k is chosen, whether cluster centers are reinitialized, whether empty clusters are handled, or what convergence threshold tau_c is used. The final classification is evaluated on held-out test data, so the central claim is not circular, but the clustering ablation's reported gain is not trustworthy without a demonstration that the clustering procedure avoids collapse.","section":"Eqs. (10)-(12)"},{"comment":"Several parts essential for reproducibility are said to be in appendices, but Appendix A (image transform details), Appendix B (hyperparameters and learning rates), Appendix C (baseline implementations), and Appendix D (additional experimental results) are not included in the arXiv v1 text provided. The Implementation and Training paragraph and the comparison section refer to these absent appendices for details such as learning rates and baseline settings. As a result, the exact configuration used to produce Table 2 cannot be reconstructed from the paper alone. The authors should include these appendices in the manuscript or in a supplementary document.","section":"Appendix availability / Implementation and Training"}],"minor_comments":[{"comment":"The third contribution bullet contains a grammatical error: 'We also simulates two classic types of missingness' should be 'We also simulate two classic types of missingness.'","section":"Contributions bullet"},{"comment":"The notation x_t- is used for 'the closest observation prior to the current missing value,' but the indexing is not formally defined; please clarify how x_t- is selected for each missing position.","section":"Eq. (3)"},{"comment":"The function 'Drop' is not defined; if it denotes dropout, please state the dropout rate or say that it is fixed in the appendix.","section":"Eq. (4)"},{"comment":"The NT-Xent formulation says 'Given 2B pairs (zi,zj)' but does not explicitly define which pairs are positive; in Eqs. (7) the intended pairs are clear from context, but the definition should be written out.","section":"Eq. (6)"},{"comment":"The description of the 'real measured samples' is confusing: 'half imputed using linear interpolation as real measured samples' seems to mean that linearly interpolated signals are labeled as real, but this should be stated explicitly, and the sample size of 60 is quite small for a Turing-test claim.","section":"Clinical Turing tests"},{"comment":"The introduction cites an 80% missing rate for PhysioNet2012, while Table 1 reports 88.4% for P12 and 94.9% for P19; please reconcile these numbers or clarify that they refer to different datasets or preprocessing settings.","section":"Table 1 / Introduction"}],"recommendation":"major_revision","confidential_remarks":"The paper appears to be a camera-ready-style submission (AAAI format) but has been posted as an arXiv v1 without appendices, which is a significant reproducibility issue. I also note that the code link is present, but the conclusion says code and data 'will be made publicly available later,' so it is unclear whether the linked repository is currently accessible. The label-based contrastive loss and the possible label leakage through X' into the image branch are the main technical concerns that need to be resolved before the empirical claims can be fully trusted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The real contribution here is the first combination of a sequence imputation branch with a pretrained Swin Transformer on image transforms of irregular medical time series, plus three losses to align the two views. The headline results are strong: the method beats seven baselines on PAM, P12, and P19, and the ablations show each piece contributes. That is a useful engineering result and a plausible new baseline for this subfield.\n\nThe biggest gap is the image construction. Table 1 shows missing rates up to 94.9%. GAF, MTF, and recurrence plots require dense series, but the paper never states whether the images are built from the raw sparse X or from the imputed X'. If it is X', then the 'image-only' ablation in Table 4 is not well-defined, and the sequence and image views are not independent. The appendices with details are absent from this v1, so the question cannot be resolved from the paper alone. The authors need to specify exactly how missing entries are handled before the results can be fully trusted.\n\nTwo smaller technical issues: Eq. 9 uses labels to define same-category pairs, so it is supervised contrastive learning, not self-supervised; and Eq. 12 defines a medoid update, not a K-means mean. Both are easy fixes but should be corrected. The clinical Turing test is also over-interpreted: chance-level discrimination by five experts on 60 samples does not demonstrate realism; it is a weak side experiment.\n\nTo be fair, the central classification claim is not circular. The clustering targets come from the same training features, but the final evaluation is on held-out data. The paper is honest about missing appendices and delayed code release, though the GitHub URL without code makes current reproducibility moderate at best.\n\nIf I were the editor, I would send this to peer review with a request for major revision: clarify image construction, fix the equations, reclassify the supervised contrastive loss, and release the code. The empirical result is strong enough to deserve referee time.","headline":"A promising joint sequence-image framework for irregular clinical time series with strong empirical results, but the image construction pipeline is underspecified and two of the 'self-supervised' losses lean on labels.","tokens_in":13820,"tokens_out":1813,"would_cite":true,"duration_ms":18156,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A dual-view model that reads irregular medical time series as both sequences and images outperforms seven comparison methods on three clinical datasets.","keywords":["irregular medical time series","self-supervised learning","sequence and image modeling","joint representation","time series imputation","contrastive learning","clustering loss","clinical classification"],"falsifier":"Re-run the image-only and fused ablations with a stated rule for missing entries (zero-fill or drop) using images built from the raw sparse series; if the fused model's F1 gain over the best single view falls below the reported 2.6 points, the fusion advantage over the image branch is falsified.","tokens_in":12731,"feed_emoji":"🩺","tokens_out":8306,"duration_ms":64051,"temperature":0.7,"pith_summary":"The paper claims that irregular medical time series with heavy missing values are better classified by a model that reads the data twice: once as a sequence and once as an image. The sequence branch imputes missing entries with a generator–discriminator pair, while the image branch turns the series into six visual encodings (line graph, frequency spectrum, Gramian angular fields, Markov transition fields, recurrence plot) and processes them with a pretrained Swin Transformer. Three self-supervised losses — an inter-sequence contrastive loss, a margin-based sequence-image contrastive loss, and a clustering loss — align the two views into one joint representation. On the PAM, P12, and P19 clinical datasets, the fused model beats seven comparison methods, with the largest margin on the most heavily missing dataset (P19, 94.9% missing). The practical claim is that joint sequence-image learning is a robust recipe for clinical classification when data are sparse.","feed_headline":"Fusing sequence and image views wins on clinical time series","feed_subtitle":"Three self-supervised losses align the two views and beat seven baselines on three medical datasets.","key_machinery":"The key machinery is a two-branch architecture with three alignment losses. The sequence branch is a generator–discriminator pair: a bidirectional RNN generator with a time-decay term imputes missing values, and the discriminator tries to tell real from imputed entries. Reconstruction is trained with an NT-Xent loss rather than MSE, which the paper argues prevents the generator from collapsing under severe missingness. The image branch converts each series into six RGB images — line graph, frequency spectrum, Gramian angular summation/difference fields, Markov transition field, and recurrence plot — and encodes them with a pretrained Swin Transformer. A joint projection maps sequence and image representations into a shared space, where a margin-based sequence-image contrastive loss and a K-means clustering loss pull the two views together and separate classes. The fused feature is classified with cross-entropy loss on PAM and focal loss on P12 and P19.","core_discovery":"The central discovery, on the paper's own terms, is that the joint sequence-image representation trained with the three self-supervised losses outperforms all seven comparison methods on all three datasets: on PAM it raises accuracy, precision, recall, and F1 by 2.3–3.1 points over the second-best method, and on P19 it raises AUPRC by 5.8 points and AUROC by 2.3 points. The paper further reports that under simulated missingness on PAM, the fused model degrades less than the baselines, staying above 80% on all metrics even when 50% of sensors are masked. These results are presented as evidence that the two views carry complementary information and that the SSL losses are what fuse them effectively.","pith_inferences":["The paper never states whether the six images are built from the raw sparse series or from the imputed sequence X'; if the latter, the image branch is not an independent view and the reported fusion gain may largely inherit the imputation quality.","Because the image encoder is a fixed ImageNet-pretrained Swin Transformer, the method's success may depend on visual features that transfer well from natural images to physiological plots; this transferability is untested on non-clinical irregular series.","The robustness experiments are only run on PAM at a base 60% missing rate; whether the fusion advantage persists at P12/P19's 88–95% baseline missingness remains an open extension.","A direct ablation that varies the number of image transforms (from one to six) would show whether the gain comes from the fusion itself or from the sheer variety of encodings."],"forward_implications":["On the PAM, P12, and P19 benchmarks under the 8:1:1 split, the fused model becomes the reference point that future methods must beat.","Each of the three SSL losses contributes independently: removing the contrastive step or the clustering step lowers PAM F1 by about one point relative to the full model.","Under leave-sensors-out missingness at 50%, the method keeps all metrics above 80% and outscores the second-best method by 6.1% accuracy, indicating the representation is robust to whole sensors disappearing.","The clinical Turing test on P19 suggests that the imputed signals are not distinguishable from linear-interpolation-imputed signals by experienced ICU clinicians, which supports the clinical plausibility of the imputed values."],"supporting_citations":[{"why":"Defines the GRU-D baseline and contributes the time-lag matrix and decay mechanism used in the sequence branch.","marker":"Che et al. 2018"},{"why":"Supplies the adversarial imputation framework and the RNN discriminator architecture adopted by the sequence branch.","marker":"Ma et al. 2019"},{"why":"Provides the adversarial semi-supervised imputation strategy that the sequence branch builds on.","marker":"Miao et al. 2021"},{"why":"Introduces the ViTST image-branch baseline, the line-graph image construction, and the 8:1:1 data split used in all experiments.","marker":"Li, Li, and Yan 2024"},{"why":"Provides the pretrained Swin Transformer V2 used as the image encoder.","marker":"Liu et al. 2022"},{"why":"Defines the NT-Xent loss used both as the reconstruction loss and as the basis for the contrastive alignment losses.","marker":"Chen et al. 2020"},{"why":"Supplies the Gramian angular field and Markov transition field image transforms used in the image branch.","marker":"Wang and Oates 2015"},{"why":"Supplies the recurrence plot image transform used in the image branch.","marker":"Hatami, Gavet, and Debayle 2018"},{"why":"Provides the Raindrop baseline and the leave-sensors-out / leave-samples-out missingness protocol used in the robustness experiments.","marker":"Zhang et al. 2022"}],"fun_headline_variants":["Sequence + image fusion tops seven models on irregular clinical data","Self-supervised fusion of views beats baselines on messy medical series","Two views, one model: beating missingness in medical time series","Joint seq-image SSL robust to sensor and sample gaps","Fused sequence-image self-supervision beats 7 baselines on 3 datasets"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The image branch assumes that six image transforms built from series with up to 94.9% missing values remain informative to a pretrained Swin Transformer, and the paper never states how missing entries are handled during image construction or whether the images come from the raw sparse series or from the imputed sequence.","fun_headline_variants_meta":{"raw":{"variants":["Sequence + image fusion tops seven models on irregular clinical data","Self-supervised fusion of views beats baselines on messy medical series","Two views, one model: beating missingness in medical time series","Joint seq-image SSL robust to sensor and sample gaps","Fused sequence-image self-supervision beats 7 baselines on 3 datasets"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000734,"raw_usage":{"total_tokens":3225,"prompt_tokens":832,"completion_tokens":2393,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":448,"completion_tokens_details":{"reasoning_tokens":2303}},"tokens_in":448,"tokens_out":2393,"duration_ms":15720,"temperature":1.0,"reasoning_tokens":2303,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T16:39:08.920963+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the image-only and fused ablations with a stated rule for missing entries (zero-fill or drop) using images built from the raw sparse series; if the fused model's F1 gain over the best single view falls below the reported 2.6 points, the fusion advantage over the image branch is falsified.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the adversarial semi-supervised imputation strategy that the sequence branch builds on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the pretrained Swin Transformer V2 used as the image encoder."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the NT-Xent loss used both as the reconstruction loss and as the basis for the contrastive alignment losses."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Gramian angular field and Markov transition field image transforms used in the image branch."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the recurrence plot image transform used in the image branch."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the Raindrop baseline and the leave-sensors-out / leave-samples-out missingness protocol used in the robustness experiments."}],"review_version":1}