{"id":"803fb3af-333d-4909-bfd8-e2c1fde82919","arxiv_id":"2411.13428","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"SynEHRgy tokenizes mixed-type MIMIC-III records into one sequence and trains a small decoder-only transformer to generate new synthetic patient records.","lead":"Researchers trained a small GPT-style transformer to generate synthetic patient records from the MIMIC-III database by converting demographics, diagnosis codes, and irregularly sampled vital signs into a single token stream. The generated data scores close to real data on prediction tasks and outperforms several earlier synthetic EHR generators on fidelity tests.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The mixed-type claim is not actually tested: fidelity is evaluated separately per data type, leaving cross-type dependencies between ICD codes and time series unverified.","rationale":"I read the paper as an empirical claim that one autoregressive model can jointly model all data modalities. The strongest evidence is the per-type fidelity tables, the utility experiments, and the privacy analysis. The weakest point is the evaluation design: the only cross-modal outcome is TSTR for phenotype/mortality labels, which does not cover ICD-time-series dependencies, the most clinically important joint structure. The reader's quantization concern is legitimate, because bin details are absent and within-bin uniform sampling is an approximation, but its expected effect is bounded if bins are narrow and it does not address the joint-distribution gap. I credit the paper for reporting PRDC with standard errors, an explicit ablation, and a membership-inference analysis, and I do not see an internal inconsistency. The missing mixed-type evaluation is addressable and does not require rejecting the method, so the verdict remains CONDITIONAL rather than ACCEPT.","tokens_in":14604,"tokens_out":9533,"duration_ms":111275,"concrete_test":"Compute a joint fidelity check on the generated sample: for each real and synthetic patient, concatenate the first-48h time-series statistics used in Section 4.2 with a bag-of-ICD-codes vector for that visit, then compute PRDC between the train and synthetic joint-embedding distributions. If the joint PRDC is substantially lower than the marginal PRDC in Table 2, the model is not preserving the cross-type correlations required by the mixed-type claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that a single decoder-only transformer generates high-quality synthetic EHRs for mixed-type data across visits. However, Section 4.2 evaluates fidelity separately: n-gram correlations for ICD codes, and PRDC/MSE_corr/co-occurrence for time series, while Section 4.3 explicitly compares per data type because baselines do not generate multiple types. No metric measures whether dependencies across data types are preserved, such as the association between ICD codes and contemporaneous vital signs or labs, or between covariates and time series. The only joint signal is the TSTR utility task in Tables 3-4, where time-series embeddings predict phenotype/mortality labels; it does not involve ICD codes and is a single predictive summary rather than a fidelity check of the joint distribution. A model could match all marginal statistics in Tables 1-2 while generating clinically inconsistent records, such as normal vitals paired with septic-shock codes, so the headline mixed-type, high-quality claim is under-supported. This is not a claim that the method is wrong; it is a missing verification of exactly the capability the paper claims.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents SynEHRgy, a method for generating synthetic longitudinal electronic health records with a decoder-only transformer. It tokenizes demographics, labels, ICD codes, irregularly sampled time series, and time intervals, trains a small GPT-2-style model on token sequences from MIMIC-III, and generates new patient records autoregressively. Evaluation covers per-type fidelity (ICD n-gram correlations; PRDC, correlation MSE, and missingness co-occurrence for time series), utility (TSTR and data-augmentation AUROC for phenotype and mortality prediction), and privacy (membership-inference distances). The central claim is that a single GPT-like decoder-only transformer with a tailored tokenization strategy can generate high-quality mixed-type structured EHR data across multiple visits.","tokens_in":14742,"tokens_out":4955,"duration_ms":53359,"significance":"If the empirical claims hold, SynEHRgy is a useful demonstration that a generic causal language model can match or beat specialized generative models for structured EHR data, especially irregularly sampled time series. Strengths include the use of a public benchmark, comparison against several relevant baselines, and evaluation along fidelity, utility, and privacy axes; the TSTR AUROC is close to the validation-split reference. However, the reported evidence is mostly per-data-type and does not directly verify the paper's central 'mixed-type' claim. The tokenization details are also not fully disclosed, which limits reproducibility and makes the main mechanism difficult to audit.","major_comments":[{"comment":"The abstract and introduction claim generation of mixed-type structured EHRs, but the evaluation never checks dependencies across data types. Section 4.2 evaluates ICD fidelity with n-gram correlations and time-series fidelity with PRDC/MSE_corr separately, and the utility tasks in Tables 3 and 4 use time-series embeddings only and do not involve ICD codes. A model could match all reported per-type statistics while generating clinically inconsistent joint records (e.g., normal vitals paired with septic-shock codes). This is a missing verification of the central claim. Please add a joint fidelity test, such as comparing the conditional distribution of time-series statistics given ICD codes or phenotypes between real and synthetic data, or a discriminative test on concatenated representations.","section":"Section 4.2, Tables 1-4"},{"comment":"The method's numerical tokenization relies on uniform quantization, but the paper reports neither the number nor the width of bins per variable. De-tokenization uses uniform sampling inside each bin, so the generated continuous values cannot reproduce the within-bin distribution unless the bins are very narrow. This choice is an unauditable free parameter that affects every fidelity, utility, and privacy result. Please report the full binning configuration and add a sensitivity analysis (e.g., coarse versus fine grids) to show that the method is robust to this choice.","section":"Sections 3.2 and 3.4"},{"comment":"The main empirical claims are not accompanied by error bars in Tables 1, 3, 4, and 5, and the standard deviations in Table 2 are not explained (over random seeds, train splits, or bootstrap samples). The checklist explicitly acknowledges that k-fold or multi-seed evaluation was not performed. Without uncertainty estimates, the reported differences that are small, such as utility AUROCs and MIA metrics, cannot be distinguished from noise. Please provide confidence intervals or multiple-run variability for the key comparisons, at least for the central fidelity and TSTR claims.","section":"Section 5.1 and NeurIPS checklist item 7"}],"minor_comments":[{"comment":"The phrase 'context length of size 4098' appears to be a typo; presumably 4096 was intended.","section":"Section 4.4"},{"comment":"Please state explicitly what the parenthetical values are (standard deviation over which source of randomness, such as seeds or bootstrap samples).","section":"Table 2"},{"comment":"The missingness co-occurrence is only shown visually; a numerical summary, such as the MSE between the co-occurrence matrices of real and synthetic data, would make the claim quantitative.","section":"Figure 3"},{"comment":"CEHR-GPT is described as a GPT-based longitudinal EHR generator; consider including it as a baseline or explaining why a comparison is omitted.","section":"Related Work"},{"comment":"The license question is answered as NA although the paper uses MIMIC-III; the dataset's license and terms of use should be stated.","section":"Checklist item 12"},{"comment":"The statement that uniform sampling 'ensures that the generated values are representative' is too strong, since uniform sampling within a bin ignores the distribution of values inside that bin.","section":"Section 3.4"}],"recommendation":"major_revision","confidential_remarks":"This is an empirical workshop-length paper with a simple and plausible core idea. The time-series fidelity results are encouraging, but the 'mixed-type' claim currently lacks a joint-distribution test, and the binning configuration is not disclosed. The authors' own checklist admits to missing error bars and to releasing code only upon acceptance. I would support a revision rather than rejection; adding a joint fidelity evaluation, uncertainty quantification, and full tokenization details would make the central claim defensible."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a reasonable empirical paper on using a small decoder-only transformer with uniform-quantization tokenization to generate MIMIC-III data (covariates, ICD codes, irregularly sampled time series). The per-type results look decent: SynEHRgy beats the baselines on bigram/trigram ICD correlations and on PRDC and correlation fidelity for the time series, and the TSTR numbers are close to the validation split. The ablation removing quantization (SynEHRgy0) is a useful sanity check and shows the tokenization is doing real work.\n\nThe main soft spot is exactly what the stress-test flags: the 'mixed-type' claim isn't actually tested as a joint distribution. Section 4.2 says the baselines don't generate multiple types, so fidelity is assessed separately for ICD codes and time series. The only joint signal is the TSTR utility task, which uses time series embeddings only (no ICD codes). Nothing checks whether the generated ICD codes match the generated vitals/labs in a clinically consistent way. You could have a model that nails every marginal in Tables 1–2 but pairs septic shock codes with perfectly normal vitals. That doesn't mean the method is wrong—the autoregressive ordering might well capture cross-type dependencies—but the paper simply doesn't demonstrate it. That's the biggest gap between the headline and the evidence.\n\nOther weaknesses are more addressable: no code shipped (the checklist says 'upon acceptance'), no error bars on the utility or privacy tables, the quantization bin grid is never specified (number of bins per variable), and the baseline list skips the mixed-type diffusion approaches the paper itself cites. These are fixable with a revision, not fatal.\n\nI don't buy the privacy analysis as more than a shallow check; the MIA setup is standard but the paper's own caveat about distance metrics is appropriate.\n\nNet: this is a workshop-quality empirical paper with one clear contribution (the tokenization + ablation) and one clear unverified claim (joint fidelity). It deserves a proper peer-review round, not a desk rejection, but the authors should be pushed to either add a joint fidelity metric or soften the mixed-type claim. I'd send it to a serious reviewer; I just wouldn't cite it yet in its current form.","headline":"Per-type fidelity and a useful quantization ablation, but the paper's headline mixed-type claim is never actually tested on joint dependencies.","tokens_in":15335,"tokens_out":2637,"would_cite":false,"duration_ms":28313,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A small decoder-only transformer can generate synthetic electronic health records spanning demographics, ICD codes, and irregularly sampled time series.","keywords":["synthetic electronic health records","decoder-only transformer","tokenization","irregularly sampled time series","ICD codes","MIMIC-III","membership inference attack","data augmentation"],"falsifier":"Generate synthetic time series with the published pipeline and compare real and generated values within each bin, for example with a Kolmogorov-Smirnov test on heart-rate values inside a single bin; if within-bin values are systematically off, or if changing the number of bins from coarse to fine swings the reported fidelity and utility scores by a large margin, the central claim that binning preserves the distribution is false.","tokens_in":14331,"feed_emoji":"🩺","tokens_out":5601,"duration_ms":59335,"temperature":0.7,"pith_summary":"This paper tries to show that one small GPT-style decoder-only transformer can generate entire synthetic patient records that mix demographics, ICD diagnosis and procedure codes, and irregularly sampled vital-sign and lab time series across multiple hospital visits. The key trick is a tokenization scheme that turns every continuous value into one of a fixed set of bin tokens and recovers a number by sampling uniformly inside the bin. On the MIMIC-III critical care data, the generated records score competitively with, and on several n-gram and time-series fidelity metrics better than, specialized GAN and transformer baselines, and they support train-on-synthetic/test-on-real predictions nearly as well as real training data. If this holds, a single autoregressive language model can stand in for a collection of task-specific generative models for structured EHRs.","feed_headline":"One small transformer generates whole synthetic patient records","feed_subtitle":"Bin-tokenized ICD codes, irregular vitals, and visits rival specialized GANs on MIMIC-III fidelity and utility.","key_machinery":"The load-bearing mechanism is the tokenization strategy for mixed-type records. Every continuous value is uniformly quantized into equal-width bins, each bin getting its own token; time intervals between irregular measurements are similarly discretized; ICD codes and categorical variables get one token each; and special tokens (<s>, </s>, </covars>, </labels>, </ts>, </visit>, <PAD>) mark record structure. A small GPT-2-style decoder-only transformer is trained with the causal language-modeling objective to predict the next token, and generation proceeds autoregressively from <s> until </s>; numerical values are recovered by uniform sampling inside the predicted bin. This machinery lets one model handle discrete and continuous data in a single sequence while keeping token counts small.","core_discovery":"The central claim is that discretizing structured EHR data into tokens and learning their next-token distribution lets a decoder-only transformer capture the joint distribution over covariates, ICD codes, and irregular time series well enough for practical synthetic-data uses. The authors demonstrate this on MIMIC-III with a small GPT-2-sized model, measuring fidelity through n-gram correlations for codes and precision/recall/density/coverage plus correlation-matrix error for time-series embeddings, utility through phenotype and mortality prediction with varying amounts of synthetic augmentation, and privacy through membership-inference distance statistics. Their reported results show the method ahead of the baselines on bigram and trigram code fidelity and on time-series precision, recall, and coverage, and closest to the real-data ceiling in train-on-synthetic/test-on-real utility. The paper frames the numerical-tokenization step as the load-bearing innovation: without it, the ablated text-based model performs markedly worse.","pith_inferences":["The bin width and number of bins are not reported; if treated as a tunable hyperparameter, the same pipeline could plausibly be pushed toward higher fidelity with a finer grid or stronger privacy with a coarser grid, so the reported balance is not a fixed property of the method.","Because the model sees only bin identities, it cannot generate values outside the observed range or capture sub-bin structure; downstream tasks that need exact lab values, such as medication dosing rules, would require a different de-tokenization or a second-stage model.","The membership-inference evaluation uses distance distributions that can miss some memorization; a stronger attack targeting exact bin-token sequences might reveal more leakage than the reported near-chance AUROC.","The same tokenization recipe could transfer to non-medical mixed-type longitudinal records, such as sensor logs or administrative event streams, where irregular timing and categorical-numerical mixtures are common."],"forward_implications":["A single decoder-only transformer can jointly model discrete codes and irregular continuous measurements, so mixed-type EHR generation does not require separate GAN or diffusion components per data type.","Irregularly sampled time series with informative missingness can be represented faithfully as token sequences, as shown by the co-occurrence and correlation-matrix results.","Synthetic records from this method can replace real training data for mortality and phenotype prediction with only a small AUROC drop, and can augment small training sets.","Because new variables only require adding tokens to the dictionary, the tokenization extends to new data sources without rearchitecting the model.","The approach inherits the context-length limit: very long patient histories may exceed the 1024-token window used here, a limitation the paper states in its conclusion."],"supporting_citations":[{"why":"Supplies the MIMIC-III dataset that all experiments and baseline comparisons use.","marker":"[17]"},{"why":"Supplies the preprocessing pipeline that defines patient and visit structure, labels, and the 70-15-15 split.","marker":"[42]"},{"why":"Supplies the uniform-quantization binning idea for numerical values that the proposed tokenization adapts.","marker":"[39]"},{"why":"Is the main longitudinal mixed-type baseline and the source of the n-gram fidelity evaluation for ICD codes.","marker":"[12]"},{"why":"Is the irregular time-series GAN baseline used for time-series fidelity and utility comparisons.","marker":"[9]"},{"why":"Supplies the PRDC embedding fidelity metrics used to evaluate generated time series.","marker":"[43]"},{"why":"Supplies the membership-inference-attack protocol used for the privacy evaluation.","marker":"[31]"}],"fun_headline_variants":["GPT-style model generates synthetic patient records from tokens","Decoder-only transformer beats GANs at synthetic EHR fidelity","Token trick helps small GPT rival specialized EHR generators","Synthetic EHRs: mixed-type tokens, one transformer, high fidelity","New tokenization for mixed-type EHRs: small GPT delivers"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that chopping every continuous measurement into a fixed set of value bins and then drawing a random value inside the bin preserves the clinically meaningful distribution of the original measurements.","fun_headline_variants_meta":{"raw":{"variants":["GPT-style model generates synthetic patient records from tokens","Decoder-only transformer beats GANs at synthetic EHR fidelity","Token trick helps small GPT rival specialized EHR generators","Synthetic EHRs: mixed-type tokens, one transformer, high fidelity","New tokenization for mixed-type EHRs: small GPT delivers"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000268,"raw_usage":{"total_tokens":1557,"prompt_tokens":826,"completion_tokens":731,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":442,"completion_tokens_details":{"reasoning_tokens":650}},"tokens_in":442,"tokens_out":731,"duration_ms":7397,"temperature":1.0,"reasoning_tokens":650,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T16:25:39.013305+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Generate synthetic time series with the published pipeline and compare real and generated values within each bin, for example with a Kolmogorov-Smirnov test on heart-rate values inside a single bin; if within-bin values are systematically off, or if changing the number of bins from coarse to fine swings the reported fidelity and utility scores by a large margin, the central claim that binning preserves the distribution is false.","supporting_citations":[{"cited_title":"EHR-Safe: Generating High-Fidelity and Privacy-Preserving Synthetic Electronic Health Records","cited_arxiv_id":null,"evidence_quote":"Supplies the membership-inference-attack protocol used for the privacy evaluation."}],"review_version":1}