{"id":"59508aca-d525-402e-8a50-57cedd504b56","arxiv_id":"2607.03715","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.5,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Weighting entropy minimization by consistency with estimated pathology co-occurrence improves multi-label chest X-ray test-time adaptation under domain shift.","lead":"CoWA adapts chest X-ray models at test time by weighting samples according to how well their predicted disease combinations match co-occurrence patterns estimated from the target data itself. This can make multi-label medical AI more reliable when hospitals use different scanners and patient mixes without sharing training data.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"The co-occurrence matrix M is estimated from the same source-biased predictions that are being adapted, so early errors can lock in a distorted reliability signal that reweighting then amplifies.","rationale":"The reader's weakest_assumption correctly isolates the most load-bearing condition: that M estimated from the model's own initially source-biased predictions is a sufficiently faithful reliability signal. My concern sharpens the same point by emphasizing the temporal circularity (early M is pure source bias) rather than merely the final alignment shown in Fig. 3. The proposed freeze/randomize-M ablation is a direct, low-cost check that would settle whether that circularity actually distorts adaptation. Because the paper already shows progressive alignment and modest but consistent gains, and because code and public data exist, the concern does not overturn the claim; it simply keeps the verdict at CONDITIONAL pending that (or multi-seed) evidence. No stronger internal inconsistency or experimental red flag is present.","tokens_in":9103,"tokens_out":550,"duration_ms":5050,"concrete_test":"On CheXpert\to MIMIC (and one other pair), freeze M after the first 1-2 batches (or replace it with a random positive-semidefinite matrix of matching sparsity) and continue weighted entropy minimization for the remaining stream; report final mean AUROC and per-class drops versus the unfrozen CoWA baseline. If performance collapses toward or below TENT/No-Adapt, the early-M circularity is load-bearing; if gains persist, the concern is largely mitigated by later accumulation.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim rests on M (Eqs. 1-2), accumulated from the model's own binarized predictions on unlabeled target data, serving as a faithful reliability signal for true target label structure. Because M is built from the same source-pretrained f_\theta that is being adapted, early batches inherit source co-occurrence bias (explicitly noted as varying across domains in Fig. 1). The progressive-alignment evidence in Fig. 3 only compares against a fully supervised target proxy after adaptation has already begun; it does not show that the early, source-contaminated M correctly ranks sample reliability before the model has moved. If the initial M systematically mis-ranks samples (downweighting true target patterns or upweighting source-like errors), the weighted entropy objective (Eq. 4) can reinforce the wrong structure rather than correct it. The paper's safeguard (accumulation + w_min) dampens but does not eliminate this circularity, and no experiment freezes or randomizes the early M to isolate the effect.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper proposes CoWA, a source-free test-time adaptation method for multi-label chest X-ray classification. It estimates a pathology co-occurrence matrix M from the model's own soft-then-binarized target predictions (Eqs. 1–2), scores each sample by Frobenius consistency of its outer-product pattern with M (Eq. 3), and performs weighted entropy minimization over BN affine parameters (Eq. 4). The claim is that this structure-aware reweighting yields more reliable adaptation than uniform entropy methods designed for single-label natural images. Evaluation covers six source–target pairs among MIMIC-CXR, CheXpert, VinDr-CXR, and NIH using DenseNet-121 models from TorchXRayVision, reporting AUROC on six shared pathologies against AdaBN, TENT, CoTTA, EATA, and RoTTA. Supporting analyses include progressive alignment of M to a supervised target proxy (Fig. 3), correlation of weights with AUPRC (Fig. 4), and a norm ablation (Table 2).","tokens_in":9403,"tokens_out":1418,"duration_ms":16307,"significance":"Domain shift across clinical sites is a genuine barrier to deploying chest X-ray models, and source-free TTA is a practically relevant setting. The paper correctly identifies that standard entropy-based TTA ignores multi-label co-occurrence structure, which is a real property of thoracic disease. If the gains hold under stricter protocols, CoWA is a simple, architecture-preserving add-on that improves worst-case robustness on low-prevalence classes—clinically more important than mean AUROC alone. Strengths include public code, multiple real domain-shift pairs, and analyses that go beyond leaderboard tables (matrix evolution, weight–reliability correlation, gradient-variance check). The contribution is incremental rather than foundational, but it is a clear and usable idea for multi-label medical TTA.","major_comments":[{"comment":"Section 3 (Co-occurrence Matrix Estimation) and Fig. 3: M is accumulated from the same source-pretrained predictions that are being adapted, so early batches inherit source co-occurrence bias (Fig. 1). The progressive-alignment evidence compares M only to a fully supervised target proxy after adaptation has started; it does not isolate whether the early, source-contaminated M correctly ranks sample reliability. A load-bearing control is missing: freeze M after the first few batches (or replace early M with a random/source matrix) and show that the weighted objective still helps rather than locks in distorted structure. Without this, the central reliability-signal claim remains only partially supported.","section":"Section 3, Eqs. 1–4; Fig. 3"},{"comment":"Section 4.1 Implementation Details: learning rate, τ, and the binarization threshold are selected from small grids, but the selection protocol is not stated. If any selection used target labels or target validation AUROC, the comparison is not pure TTA and overstates gains relative to baselines run with original hyperparameters. Please state explicitly whether hyperparameters were chosen without target labels (e.g., fixed defaults, source-only proxy, or first-batch unsupervised criteria) and report sensitivity across the full grid for all six shifts.","section":"Section 4.1"},{"comment":"Table 1: results appear to be single-run point estimates with no standard deviation, no multiple seeds, and no statistical tests. Several CoWA margins over the second-best method are small (e.g., 0.3–0.8 AUROC points). Given BN-only adaptation and stochastic batch order, variance can be material. Report mean±std over at least 3 seeds (or batch-order shuffles) and mark whether gains remain significant; otherwise the “consistent improvements” claim is overstated.","section":"Table 1"},{"comment":"Section 4.2 / Table 1 class-wise discussion: the strongest clinical claim—that CoWA is the only method avoiding severe drops below the unadapted baseline on all shifts, especially low-prevalence classes such as Pneumothorax—is important but only narrated. Please add a compact worst-case or min-class ΔAUROC summary (or per-class drop table) so this claim is directly verifiable rather than inferred from selected cells.","section":"Section 4.2, Table 1"}],"minor_comments":[{"comment":"Eq. (2): the normalization M_jk = P_jk / sqrt(P_jj P_kk + ε) is a cosine-style co-occurrence; briefly justify why this form is preferred over raw joint probabilities or PMI, especially under class imbalance.","section":"Eq. (2)"},{"comment":"Fig. 1 is referenced for cross-domain co-occurrence variation but is not quantitatively described in the text; a short numeric summary (e.g., Frobenius distance between domain matrices) would help.","section":"Fig. 1"},{"comment":"VinDr results omit Edema (marked “–”); state whether the pathology is absent from labels or excluded for prevalence reasons so the mean is comparable across methods.","section":"Table 1"},{"comment":"Related Work could briefly note multi-label TTA or medical TTA efforts beyond natural-image methods (even if limited) to better position novelty.","section":"Section 2"},{"comment":"Typographical inconsistencies: “CoW A” vs “CoWA” in figure captions and tables; “T arget” spacing artifacts in Table 1 headers.","section":"Table 1, Fig. 2"},{"comment":"w_min = 0.01 is fixed without sensitivity; a one-line ablation would strengthen the “early-stage safeguard” claim.","section":"Eq. (4)"}],"recommendation":"major_revision","confidential_remarks":"The circularity concern raised by the skeptic is real but not fatal; many accepted TTA methods are similarly self-referential. The paper is suitable for a medical imaging / MICCAI-style venue after the hyperparameter protocol and early-M control are clarified. Novelty is moderate—structure-aware reweighting of entropy—but the multi-label clinical framing is useful. I would not reject on novelty alone if the experimental gaps are closed."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The one thing worth knowing: CoWA is a clean, source-free fix for the fact that standard entropy TTA (TENT and friends) treats multi-label chest X-ray pathologies as independent. They accumulate a running co-occurrence matrix from binarized target predictions, score each sample by Frobenius distance of its outer product to that matrix, and reweight the entropy loss. That combination is not in the single-label TTA literature they cite, and the six-shift table shows it is the only method that never tanks below the unadapted baseline on low-prevalence classes.\n\nWhat they do well is straightforward. The method is simple (BN affine only, public TorchXRayVision backbones, code promised), the analyses in Figs. 3–4 are honest (matrix drifts toward a supervised target proxy; higher weights track higher AUPRC), and the ablation on the norm choice is useful. Gains are consistent if modest; worst-case robustness is the real selling point for clinical deployment. Citations look appropriate; no inventing of prior art.\n\nSoft spots, in proportion. The stress-test concern is real but not fatal: M is built from the same initially source-biased predictions that are being adapted, so early batches can lock in a distorted reliability signal. Accumulation plus w_min dampens this; Fig. 3 shows progressive alignment, but they never freeze or randomize the early matrix to isolate the effect. Hyperparameters (τ, binarization threshold, LR) are selected per setting with no multi-seed error bars. These are standard empirical-TTA caveats, not load-bearing contradictions. The circularity is mild because final metrics are against held-out ground truth.\n\nThis is for people who ship multi-label medical models under site shift and already know TENT/EATA. It is not a new principle, but it is a reproducible engineering step that fills a clear gap. I would send it to peer review; a referee can demand the multi-seed numbers and the early-M ablation without killing the paper. Worth engaging if you work in this lane.","headline":"Solid, usable multi-label TTA for chest X-ray that actually respects co-occurrence; modest gains, real circularity risk, still worth a referee.","tokens_in":10008,"tokens_out":524,"would_cite":true,"duration_ms":4520,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"CoWA reweights test-time adaptation by how well each chest X-ray prediction matches target disease co-occurrence patterns, improving multi-label diagnosis under domain shift.","keywords":["test-time adaptation","chest X-ray","multi-label classification","disease co-occurrence","domain shift","entropy minimization","sample reweighting"],"falsifier":"On a held-out domain-shift pair, replace CoWA’s estimated matrix with a deliberately scrambled or source-only co-occurrence matrix and check whether the AUROC gains and the “no severe drop” property disappear.","tokens_in":9997,"feed_emoji":"🫁","tokens_out":615,"duration_ms":5008,"temperature":0.7,"pith_summary":"Chest X-ray classifiers often fail when moved to a new hospital because scanners, protocols, and patient mixes differ. Test-time adaptation tries to fix this by updating the model on unlabeled images from the new site, but standard methods treat every prediction as equally trustworthy and ignore the fact that diseases tend to appear together in structured ways. CoWA estimates those co-occurrence patterns on the fly from the model’s own soft predictions, scores each sample by how well its predicted label set matches the emerging structure, and uses the scores to reweight entropy minimization. Samples that look clinically plausible drive the update; samples that violate expected combinations are downweighted. Across six public domain-shift pairs the method produces higher mean AUROC than established baselines and is the only one that never falls sharply below the unadapted model on any pathology.","feed_headline":"Disease co-occurrence reweights test-time X-ray adaptation","feed_subtitle":"CoWA uses predicted pathology patterns to downweight noisy samples and lift AUROC under hospital shifts","key_machinery":"Co-occurrence Weighted Adaptation (CoWA): the running co-occurrence matrix M estimated from soft-then-binarized predictions, the sample weight w_i = exp(-∥m_i - M∥_F^{2}/τ), and the weighted entropy loss that multiplies each sample’s contribution by max(w_i, w_min).","core_discovery":"By accumulating a normalized co-occurrence matrix from binarized model predictions on unlabeled target images and converting each sample’s local co-occurrence pattern into a Frobenius-norm consistency weight, CoWA turns entropy minimization into a structure-aware update that improves multi-label chest X-ray performance under domain shift without source data or extra labels.","pith_inferences":[],"forward_implications":[],"fun_headline_variants":["Co-occurrence reweights TTA for multi-label chest X-rays","Pathology patterns filter noise in X-ray test-time adaptation","CoWA turns disease co-occurrence into sample reliability weights","Structure-aware entropy min via co-occurrence on unlabeled X-rays","Predicted co-occurrence consistency guides domain-shift TTA"],"cache_read_input_tokens":128,"weakest_assumption_plain":"The co-occurrence matrix built from the model’s own early, source-biased predictions on unlabeled target data is already a reliable enough map of true target disease relationships that downweighting mismatches helps rather than hurts adaptation.","fun_headline_variants_meta":{"raw":{"variants":["Co-occurrence reweights TTA for multi-label chest X-rays","Pathology patterns filter noise in X-ray test-time adaptation","CoWA turns disease co-occurrence into sample reliability weights","Structure-aware entropy min via co-occurrence on unlabeled X-rays","Predicted co-occurrence consistency guides domain-shift TTA"]},"model":"grok-4.5","effort":"low","cost_usd":0.007974,"raw_usage":{"total_tokens":1837,"prompt_tokens":714,"num_sources_used":0,"completion_tokens":91,"cost_in_usd_ticks":79740000,"prompt_tokens_details":{"text_tokens":714,"audio_tokens":0,"image_tokens":0,"cached_tokens":128},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":1032,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":714,"tokens_out":91,"duration_ms":7673,"temperature":1.0,"reasoning_tokens":1032,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-12T00:26:12.658101+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"On a held-out domain-shift pair, replace CoWA’s estimated matrix with a deliberately scrambled or source-only co-occurrence matrix and check whether the AUROC gains and the “no severe drop” property disappear.","supporting_citations":[],"review_version":1}