{"id":"05543790-0c74-4a30-a349-1c413c6acf78","arxiv_id":"2412.08479","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"CAT applies FreeMatch-style adaptive thresholds plus contrastive label refinement to semi-supervised domain generalization, reporting improved accuracy on four benchmarks with internally contradictory tables.","lead":"This paper presents CAT, a method for semi-supervised domain generalization that combines adaptive confidence thresholds with contrastive learning to generate cleaner pseudo-labels from unlabeled images. It claims better accuracy than existing SSDG methods on four standard benchmarks, but the paper's own tables contradict each other, so the results are not currently reliable.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 2 is internally inconsistent with Tables 3-5: CAT OfficeHome is 75.23 vs 65.04 and the printed averages are not the arithmetic means, so the headline SOTA claim is unsupported.","rationale":"The stress-test focus differs from the reader's stated weakest_assumption: the reader emphasizes untested pseudo-label accuracy under domain shift, whereas I identify the internally inconsistent numerical evidence in Table 2 as the most load-bearing problem. The reader's rationale does mention Table 2 contradictions, so there is partial agreement on the overall rejection. I chose the table inconsistency because it is decisive and self-contained: it does not require any external claims about calibration or pseudo-label noise. The reported numbers themselves fail basic consistency checks, so the central empirical claim cannot be verified as stated. Fixed tables might change the verdict, which is why the concrete test asks for recomputation and raw logs. Secondary concerns, such as the claimed domain-aware thresholding not appearing in any equation (Eqs. 4-8 contain no domain index) and missing code/hyperparameters, further support the rejection but are not the primary basis for this assessment.","tokens_in":15770,"tokens_out":5257,"duration_ms":51012,"concrete_test":"Recompute every entry and average in Table 2 from the per-dataset rows in Tables 3-5, and ask the authors for the raw per-domain accuracy logs for the disputed OfficeHome and VLCS results. If the OfficeHome 10-label CAT result cannot be reconciled between 75.23 (Table 2) and 65.04 (Table 4), or if the recomputed averages differ from the printed Table 2 averages by more than 0.1 points, the central claim of state-of-the-art performance cannot be accepted from the submitted tables.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is empirical: CAT achieves state-of-the-art SSDG accuracy and even approaches full-label DG performance. That claim is quantified in Table 2. Table 2 is not internally consistent with the paper's own detailed results. For the 10-label setting, Table 2 lists CAT OfficeHome as 75.23%, while Table 4 reports 65.04% for the same setting; for 5-label, the entries are 75.50% and 61.71%. These are not rounding differences. The printed averages also fail arithmetic checks: Table 2's CAT 10-label entries (82.95, 75.23, 93.43, 80.10) average to 82.93, not the reported 82.00; the Full-Labels entries (79.50, 64.70, 95.96, 69.20) average to 77.34, not the reported 79.50. Since the abstract and Section 5.1 use Table 2's averages to claim superiority over StyleMatch and MultiMatch, an unresolved discrepancy of this magnitude directly undermines the quantitative evidence. The paper also provides no code, error bars, or raw per-domain logs, so the reader cannot determine whether the discrepancy is a typographical error or a substantive reporting problem. This concern is independent of pseudo-label calibration: even if the thresholding and refinement modules behave exactly as intended, the submitted tables do not establish the claimed state-of-the-art result.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CAT, a semi-supervised domain generalization method that combines class-adaptive thresholding (extending FreeMatch to a multi-domain setting) with a noisy pseudo-label refinement module and supervised contrastive learning. The method is evaluated on PACS, OfficeHome, VLCS, and miniDomainNet with 10 and 5 labels per class, and the paper claims that CAT outperforms existing SSDG methods (StyleMatch, MultiMatch) and approaches fully supervised DG performance. The main result is reported in Table 2, with per-dataset details in Tables 3-5, plus ablations on backbones, label counts, and number of source domains.","tokens_in":16048,"tokens_out":4206,"duration_ms":42540,"significance":"If the reported results were correct, CAT would be a simple and practical contribution to low-label domain generalization, combining known ingredients (adaptive thresholds, contrastive learning) in a straightforward way. The method is plausible and the per-dataset tables (3, 4, 5) are internally coherent. However, the quantitative evidence in the main summary table is internally inconsistent and does not support the claimed state-of-the-art performance. The paper also provides no code, no error bars, and no validation of the core pseudo-label quality assumption, so the significance of the contribution is not established by the submission.","major_comments":[{"comment":"Table 2 is internally inconsistent with the detailed results. For example, CAT on OfficeHome with 10 labels per class is listed as 75.23% in Table 2 but 65.04% in Table 4; CAT on miniDomainNet with 10 labels per class is listed as 80.10% in Table 2 but 67.71% in Table 5. The printed averages in Table 2 also fail arithmetic checks: the CAT 10-label entries (82.95, 75.23, 93.43, 80.10) average to 82.93, not the reported 82.00, and the Full-Labels entries (79.50, 64.70, 95.96, 69.20) average to 77.34, not 79.50. Additionally, Table 3's Full-Labels row lists 95.96 and 69.20 under PACS columns, which are actually the VLCS and miniDomainNet values from Table 2, not PACS per-domain accuracies. Since the abstract and Section 5.1 use Table 2's averages to claim superiority over StyleMatch and MultiMatch, these discrepancies invalidate the central quantitative claim. No code or raw per-domain logs are provided to resolve which table is correct.","section":"Table 2 vs Tables 3-5"},{"comment":"The paper's core mechanism is not validated. The method assumes that pseudo-labels exceeding the adaptive threshold are accurate enough to serve as supervision in supervised contrastive learning, and that the refinement module improves noisy labels. However, no measurement of pseudo-label accuracy, no calibration analysis, and no ablation separating the thresholding filter from the refinement module are reported. In particular, the paper does not quantify the fraction of pseudo-labels retained or the agreement rate used in the α-fractile step. Without these measurements, the claim in Section 1 that adaptive thresholding 'generates high-quality pseudo-labels' and that refinement 'improves reliability' is unsupported, especially under the domain shift that SSDG is designed to handle.","section":"Section 3.3 / Eq. (10)"},{"comment":"The method description is underspecified. The text states that CAT incorporates 'both class and domain dependencies' in pseudo-label selection, but Equations (5)-(8) contain no domain index and no domain-specific quantity. The global threshold τg is computed as an EMA over all unlabeled data, and τg(c) is a class-specific scaling of τg; there is no formulation of a domain-aware threshold. If domain-aware thresholding is an intended contribution, it is not defined in the equations, and the implementation could be no more than a direct application of FreeMatch's thresholds to the multi-domain setting.","section":"Section 3.2, Eqs. (5)-(8)"}],"minor_comments":[{"comment":"The caption says the table summarizes PACS, OfficeHome, VLCS, and miniDomainNet, but the table only contains rows for PACS, OfficeHome, and miniDomainNet; a VLCS row is missing or the caption is incorrect.","section":"Table 1"},{"comment":"'PCAS' is a typo and should be 'PACS'.","section":"Section 4.1"},{"comment":"Equation (6) has the same functional form as Equation (5), both averaging max confidence, so it does not implement a class-specific expectation as the text claims; the per-class averaging needs to be spelled out.","section":"Section 3.2, Eq. (6)"},{"comment":"FreeMatch is cited as '[ ?]' in the related work section; this broken citation should be replaced with the full reference.","section":"Section 2"},{"comment":"The description of Figure 1 is qualitative and the sentence 'our method can improve performance by 1.5% than MultiMatch, which is better 1.5% better than StyleMatch' is unclear; reporting exact numerical gains or error bars would improve clarity.","section":"Section 6, Figure 1"},{"comment":"The table caption lists domains as 'Clipart, Infograph, Painting, and Real', but the columns are labeled C, P, R, S and the text describes them as 'Clipart, Painting, Real, Sketch'; the caption and column labels should be made consistent.","section":"Table 5"}],"recommendation":"reject","confidential_remarks":"The internal inconsistency in Table 2 is severe enough that the paper's central empirical claim cannot be accepted on the current evidence. The discrepancy is not a simple rounding issue, and the arithmetic errors in the printed averages compound the problem. Even if the authors later provide corrected tables, the submitted manuscript would require a major re-analysis and likely new experiments, rather than a minor revision. I would recommend that the editor seek the raw per-domain results or code before any resubmission is considered."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Let me cut to it. The method is a straightforward combination of FreeMatch's adaptive thresholding (Eqs. 5-8 are FreeMatch verbatim) and the selective supervised contrastive refinement from [29]. The 'domain-dependent' thresholding promised in the title and Section 3.2 never appears in an equation; there is no domain index anywhere. And the headline SOTA claim is not supported by the paper's own tables: Table 2 lists CAT at 75.23 on OfficeHome and 80.10 on miniDomainNet in the 10-label setting, while Tables 4 and 5 report 65.04 and 67.71 for the same settings. The averages in Table 2 are also arithmetically wrong (Full-Labels averages 77.34, not 79.50). This is not rounding; it is an unreconciled discrepancy.\n\nWhat is genuinely good: the per-dataset tables (3-5) are internally consistent and show CAT ahead of StyleMatch and MultiMatch on PACS, OfficeHome, and miniDomainNet by a few points. The baselines appear to be reimplemented (Table 4 labels StyleMatch as 'ours'), so the comparison is fair on its face. The refinement idea—using neighbor agreement to filter noisy pseudo-labels before supervised contrastive learning—is sensible and well motivated. If the numbers hold, this is a useful empirical data point for SSDG, which is a real but niche problem.\n\nSoft spots, in order. (1) The Table 2 vs Tables 3-5 conflict is load-bearing. The abstract and Section 5.1 lean on Table 2 to claim superiority, so until the authors reconcile the aggregates and provide per-domain logs, the central claim is unverified. (2) VLCS appears only in Table 2; there is no per-domain VLCS table, so those numbers cannot be checked. (3) No code, no error bars, and alpha, K, and tau are underspecified. (4) The novelty is the combination, not the parts; the paper should say that plainly.\n\nFor whom? Someone tracking low-label DG who wants a quick comparison point. I would not cite it for the SOTA claim as it stands. But the per-dataset results and the sensible design mean a serious referee could sort out the tables and missing details. I would not desk reject this; I would send it to review with a strong request for corrected tables, VLCS details, code, and hyperparameters. The potential is there; the current manuscript just does not meet the bar for the claims it makes.","headline":"The method is a sensible FreeMatch-plus-contrastive-refinement combination, but its headline SOTA claim is unsupported by the paper's own internally inconsistent tables.","tokens_in":16626,"tokens_out":4331,"would_cite":false,"duration_ms":39607,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CAT, a class-aware adaptive thresholding method with noisy-label refinement, reports state-of-the-art semi-supervised domain generalization using only 5 or 10 labels per class.","keywords":["semi-supervised domain generalization","adaptive thresholding","pseudo-labeling","noisy label refinement","supervised contrastive learning","low-label regime","domain shift"],"falsifier":"Measure the precision of above-threshold pseudo-labels on a held-out source domain during CAT training, for example on PACS with 10 labels per class. If precision falls below the level needed for self-training to converge, or if removing the noisy-label refinement module changes final accuracy by less than the reported margin, then the claim that adaptive thresholding plus refinement drives the gains would be falsified. A direct ablation of the threshold and refinement components would settle which part carries the improvement.","tokens_in":15505,"feed_emoji":"🎯","tokens_out":4477,"duration_ms":49602,"temperature":0.7,"pith_summary":"This paper claims that semi-supervised domain generalization, where a model must transfer to unseen domains using only a handful of labeled examples per class, can be made practical by replacing fixed pseudo-label thresholds with thresholds that adapt to both class and domain. The proposed method, CAT, combines this adaptive thresholding with a noisy-label refinement step and supervised contrastive learning. Across PACS, OfficeHome, VLCS, and miniDomainNet, CAT is reported to outperform existing semi-supervised domain generalization methods and also standalone domain generalization and semi-supervised learning baselines. If the claims hold, the result matters because it lowers the annotation cost of building domain-generalizable models while keeping accuracy close to fully supervised training.","feed_headline":"Adaptive thresholds top semi-supervised domain generalization","feed_subtitle":"Class- and domain-aware thresholds plus pseudo-label refinement reach 82.95% on PACS with only 10 labeled examples per class.","key_machinery":"The central machinery is a two-part pseudo-labeling pipeline. First, class-domain aware thresholding maintains a global confidence threshold updated by exponential moving average of model confidence, then scales it per class via max-normalization so that the unsupervised loss only accepts pseudo-labels whose confidence exceeds the class-specific threshold. Second, the noisy-label refinement module finds, for each pseudo-labeled sample, its top-K nearest neighbors in representation space, measures agreement between the pseudo-label and the neighbors' labels, and uses a per-class alpha-fractile to mark samples as clean or noisy; clean samples are trained with a supervised contrastive loss and the rest with an unsupervised contrastive loss. The final objective sums the supervised cross-entropy loss, the thresholded unsupervised loss, and the supervised contrastive loss.","core_discovery":"On its own terms, the paper's central discovery is that fixed-confidence pseudo-labeling, as used by FixMatch and StyleMatch, wastes too many usable unlabeled samples in multi-domain settings, and that class- and domain-aware adaptive thresholds recover those samples while a neighbor-agreement refinement step suppresses the noisy pseudo-labels that adaptive thresholding lets through. The resulting pseudo-labels feed a supervised contrastive loss that pulls representations from different source domains closer when they share a refined pseudo-label, producing domain-invariant features. The paper reports that this combination yields, for example, 82.95% average accuracy on PACS and 93.43% on VLCS with 10 labels per class, surpassing the compared SSDG methods and many fully supervised DG baselines.","pith_inferences":["The paper leaves implicit which component contributes the reported gains: no ablation separates the adaptive threshold filter from the noisy-label refinement module, so a fair test would retrain CAT with each component removed.","A testable extension is to measure pseudo-label precision and calibration under domain shift; if above-threshold pseudo-labels are frequently wrong in a new domain, the self-training loop could amplify errors rather than correct them.","The method appears transferable to other low-label multi-domain settings such as medical imaging across hospitals or autonomous driving across weather conditions, but those settings would need explicit evaluation because their label noise and domain gaps differ from PACS and OfficeHome.","Connecting CAT to class-imbalanced SSDG is a natural next step, since the per-class thresholds and alpha-fractile refinement are sensitive to class frequency and could be adapted to rebalance pseudo-label selection."],"forward_implications":["With only 5 or 10 labeled examples per class per source domain, CAT is reported to approach or exceed the fully supervised baseline on PACS and miniDomainNet, suggesting that low-label domain generalization is feasible without large annotation budgets.","Because the adaptive threshold is class- and domain-aware, the method should retain more high-confidence unlabeled samples than fixed-threshold baselines, which directly addresses the main limitation identified for StyleMatch and FixMatch.","The noisy-label refinement is what allows the contrastive loss to use pseudo-labels without being derailed by mislabeled samples, so the method's gains should persist as the number of source domains grows.","The reported improvements grow with the number of source domains, implying CAT makes better use of multi-domain structure rather than just adding more unlabeled data.","CAT is a loss-level modification rather than a new architecture, so it can be layered onto existing SSDG or SSL training pipelines with standard backbones like ResNet."],"supporting_citations":[{"why":"Provides the FixMatch fixed-threshold pseudo-labeling baseline and the unsupervised loss formulation that CAT modifies.","marker":"[41]"},{"why":"Supplies the global and local adaptive thresholding scheme that CAT extends to be class- and domain-aware.","marker":"[48]"},{"why":"Provides the StyleMatch method, the main fixed-threshold SSDG baseline and the source of the evaluation protocol.","marker":"[59]"},{"why":"Provides the MultiMatch baseline that CAT compares against in the SSDG setting.","marker":"[38]"},{"why":"Supplies the UDA weak/strong augmentation strategy used to generate the perturbed views for pseudo-labeling.","marker":"[50]"},{"why":"Supplies the selective-supervised contrastive learning approach with noisy labels that the refinement module is based on.","marker":"[29]"},{"why":"Supplies the supervised contrastive learning objective used to train on the refined pseudo-labeled samples.","marker":"[23]"},{"why":"Supplies the leave-one-domain-out evaluation protocol used for all benchmark experiments.","marker":"[27]"}],"fun_headline_variants":["Adaptive thresholds yield 82.95% on PACS with 10 labels per class","Class-aware adaptive thresholds beat fixed ones in SSDG","Adaptive pseudo-label refinement lifts SSDG accuracy","Noisy pseudo labels refined by adaptive thresholds in SSDG","Adaptive thresholding and refinement key to few-label SSDG"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a pseudo-label whose confidence clears the adaptive threshold is accurate enough to be safely treated as a ground-truth label in supervised contrastive learning; the paper reports no measurement of pseudo-label accuracy or calibration to support this assumption.","fun_headline_variants_meta":{"raw":{"variants":["Adaptive thresholds yield 82.95% on PACS with 10 labels per class","Class-aware adaptive thresholds beat fixed ones in SSDG","Adaptive pseudo-label refinement lifts SSDG accuracy","Noisy pseudo labels refined by adaptive thresholds in SSDG","Adaptive thresholding and refinement key to few-label SSDG"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000914,"raw_usage":{"total_tokens":3907,"prompt_tokens":906,"completion_tokens":3001,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":522,"completion_tokens_details":{"reasoning_tokens":2915}},"tokens_in":522,"tokens_out":3001,"duration_ms":24137,"temperature":1.0,"reasoning_tokens":2915,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T17:44:47.096475+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the precision of above-threshold pseudo-labels on a held-out source domain during CAT training, for example on PACS with 10 labels per class. If precision falls below the level needed for self-training to converge, or if removing the noisy-label refinement module changes final accuracy by less than the reported margin, then the claim that adaptive thresholding plus refinement drives the gains would be falsified. A direct ablation of the threshold and refinement components would settle which part carries the improvement.","supporting_citations":[{"cited_title":"Fixmatch: Simplifying semi-supervised learning with consistency and confidence","cited_arxiv_id":null,"evidence_quote":"Provides the FixMatch fixed-threshold pseudo-labeling baseline and the unsupervised loss formulation that CAT modifies."},{"cited_title":"Semi-supervised domain generalization with stochastic stylematch","cited_arxiv_id":null,"evidence_quote":"Provides the StyleMatch method, the main fixed-threshold SSDG baseline and the source of the evaluation protocol."},{"cited_title":"Mul- timatch: Multi-task learning for semi-supervised domain generalization","cited_arxiv_id":null,"evidence_quote":"Provides the MultiMatch baseline that CAT compares against in the SSDG setting."},{"cited_title":"Unsupervised data augmentation for consistency training","cited_arxiv_id":null,"evidence_quote":"Supplies the UDA weak/strong augmentation strategy used to generate the perturbed views for pseudo-labeling."},{"cited_title":"Selective-supervised contrastive learning with noisy labels","cited_arxiv_id":null,"evidence_quote":"Supplies the selective-supervised contrastive learning approach with noisy labels that the refinement module is based on."},{"cited_title":"Deeper, broader and artier domain generaliza- tion","cited_arxiv_id":null,"evidence_quote":"Supplies the leave-one-domain-out evaluation protocol used for all benchmark experiments."}],"review_version":1}