{"id":"3809777c-b575-4da7-8820-fe0da0e21e31","arxiv_id":"1908.01925","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Semantic Categorical Alignment and Semantic Contrastive Mapping improve open set domain adaptation by separating known and unknown classes in feature space.","lead":"This paper proposes two training losses for open set domain adaptation, where target images may include classes never seen in the source data. One loss aligns the centroids of shared classes across domains, and the other pushes unknown images away from all known class clusters. If the reported results hold, the method provides a small but consistent accuracy gain over prior open set domain adaptation baselines.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'reliable sample selection' safeguard in §3.4 is vacuous: threshold 1/(N+1) equals the guaranteed minimum of the max softmax probability, so almost every target sample is admitted and pseudo-label noise propagates into the SCA/SCM losses on which the central claim depends.","rationale":"The reader flagged pseudo-label reliability as the weakest assumption; I agree but sharpen it to a concrete mathematical point. The threshold 1/(N+1) is not a confidence filter because softmax normalization guarantees max probability ≥ 1/(N+1). The paper's own text in §3.4 introduces reliable selection as the remedy for incorrect pseudo-labels, and the manuscript itself says 'pseudo labels ... are not totally correct'; this admission, combined with the vacuous threshold, means the central mechanism operates on largely uncurated labels. The absence of any pseudo-label accuracy measurement or label-noise sensitivity experiment means the paper's robustness assertion is not evidenced. This does not refute the empirical results; the ablations suggest the modules do something, but they do not establish that the method is robust in the way the central claim needs. I recommend keeping the verdict conditional: the paper should report selected-set statistics and a label-noise stress test, and ideally release code. My concern does not move the verdict because the reader already made acceptance conditional; it adds a precise, testable condition.","tokens_in":12836,"tokens_out":8467,"duration_ms":87790,"concrete_test":"On SVHN→MNIST and Office-31 A→D, instrument the published training loop to log (i) the fraction of target samples passing the §3.4 threshold > 1/(N+1) and (ii) oracle-vs-pseudo label agreement among selected samples at each epoch. If the selected fraction is near 1.0 throughout (as softmax normalization predicts) and/or a control experiment that randomly flips 10% of the selected pseudo-labels before the Eq. (7) centroid update causes a large OS/OS* drop, the robust-to-noise assumption is unsupported; if the selected fraction is substantially below 1 and the flip test is flat, the concern is settled.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires target pseudo-labels to be good enough for the centroids in Eqs. (5)-(7) and for the contrastive mapping in Eq. (9). The paper's only stated safeguard is 'reliable sample selection' (§3.4) with threshold 1/(N+1), but for a softmax over N+1 classes the maximum probability is always ≥ 1/(N+1). The threshold therefore excludes only exact ties and filters essentially no samples; it cannot remove confident errors, which are the dangerous ones. At initialization, Eq. (5) builds target centroids from a source-only model, so early pseudo-label error is largest exactly when the loop begins. The cosine reweighting in Eqs. (6)-(7) and (10)-(11) is also blind to the worst error mode: a sample from class j confidently predicted as class k is close to the source centroid of k, giving high ρ, so the update pulls the target centroid of k toward class j and reinforces the mistake. Section 1 asserts that SCA and SCM are 'robust to such noisy labels' but no bound, experiment, or pseudo-label-accuracy measurement is provided. The Office-31 ablations (Table 2) further show each module alone already beats OSDA+BP by large margins, so the incremental semantic-structure gain is a fragile basis for the headline SOTA claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an open-set domain adaptation (OSDA) method in which only source labels are available and the target domain contains an unknown class not present in the source. The method combines adversarial domain adaptation with two semantic-structure losses: Semantic Categorical Alignment (SCA), which applies a contrastive-center loss on source features and aligns source/target class centroids, and Semantic Contrastive Mapping (SCM), which pulls target samples predicted as known toward the corresponding source centroids and pushes predicted-unknown samples away from all known centroids using an adaptive margin. Target pseudo-labels are used to estimate centroids and to select 'reliable' samples. Experiments on Digit datasets (SVHN, MNIST, USPS) and Office-31 (AlexNet/VGG) report state-of-the-art or competitive OS/OS* accuracy, with ablations showing that removing either module generally degrades performance.","tokens_in":13156,"tokens_out":7557,"duration_ms":77114,"significance":"If the reported results are reproducible, the paper makes a useful empirical contribution: it demonstrates that categorical centroid alignment and contrastive mapping, combined with a standard adversarial OSDA baseline, improve separation of known and unknown classes, and the ablations support that both modules contribute. The paper also provides visualizations and sensitivity analyses for the margin and re-weighting parameters. The main weakness is that the target-side supervision rests on pseudo-labels whose reliability is asserted but not demonstrated; the proposed 'reliable sample selection' is, as written, vacuous. Because the method's central mechanism depends on these pseudo-labels, the empirical claims need to be re-supported by either a corrected selection criterion or an explicit noise-robustness analysis.","major_comments":[{"comment":"The reliable sample selection is vacuous. With a softmax classifier over N+1 classes, every sample's maximum predicted probability is at least 1/(N+1), so the threshold 1/(N+1) in Section 3.4 excludes only exact uniform ties and admits essentially all target samples. The pseudo-label errors therefore flow into the target centroids in Eqs. (5)-(7) and into the contrastive loss in Eq. (9), including exactly the confident errors that are most damaging. The claim in Section 1 that SCA/SCM are 'robust to such noisy labels' is unsupported: no noise bound, no pseudo-label accuracy measurement, and no experiment with corrupted pseudo-labels is provided. This is load-bearing because the target-side terms are the paper's main novelty. Please either use a threshold that actually removes low-confidence samples, or add an explicit robustness analysis and experiment demonstrating that the method tolerates pseudo-label noise.","section":"Section 3.4, Eqs. (9)-(12)"},{"comment":"The hyper-parameters are selected on the evaluation benchmarks without a held-out validation set. The text fixes lambda_s=0.02, lambda_c=0.005, lambda_t=1e-4 and reports a sweep over omega in Fig. 4(a) (the scenario is not identified) that selects omega=0.5, the value used for all reported results. Since the improvements over the state of the art are small (e.g., 1.9% average on Digits), the absence of error bars (Tables 1-2 report only means over three runs) and the absence of a validation protocol make it difficult to assess whether the gain is genuine or a consequence of test-set tuning. Please provide standard deviations and either a validation split or a full sensitivity analysis showing that the ranking of methods is stable across the hyper-parameter range.","section":"Section 4.1, Fig. 4(a)"},{"comment":"The adaptive margin M^k in Eq. (12) inherits pseudo-label bias because it is computed from target centroids c_j^t obtained from the model's own predictions. The cosine re-weighting in Eqs. (6)-(7) and (10)-(11) does not address the main failure mode: a sample from class j that is confidently but incorrectly predicted as class k is close to the source centroid of k, receives a high rho, and therefore updates c_k^t toward class j. This can reinforce the error in both the centroid alignment and the SCM margin. Please add a discussion of this failure mode and, ideally, a controlled experiment (e.g., injecting known pseudo-label corruption and measuring the effect on OS/OS*) to support the robustness claim.","section":"Section 3.4, Eq. (12)"}],"minor_comments":[{"comment":"The text says 'delta = 1 is set to be 10^{-6}'. This is contradictory; it should read 'delta is set to 10^{-6}'.","section":"Section 3.3, Eq. (4)"},{"comment":"Equation (1) has a typo: the closing parenthesis in '- log(D∘G(xs))ys )' is misplaced; the subscript should be on the probability, not outside the parentheses.","section":"Section 3.2, Eq. (1)"},{"comment":"The min/max structure of Eq. (3) is ambiguous: L_adv appears with a minus sign under min_G and a plus sign under min_D, but the overall equality with a sum of two minima does not clearly specify how the gradient-reversal layer interacts with the two terms. Please write the objective as a single min-max expression for clarity.","section":"Section 3.2, Eq. (3)"},{"comment":"The loop structure is unclear: the outer loop increments e while the inner loop runs to max_iter, but e is never used in the updates. Please clarify whether the outer loop is intended to repeat over epochs and where the centroids are re-initialized.","section":"Algorithm 1"},{"comment":"Reference [4] is incomplete: 'Q. Ce and S. Fei' should be expanded to the actual author names and full publication details; the current form makes it hard to verify the contrastive-center loss.","section":"References"},{"comment":"In Table 1, for USPS→MNIST the variant 'w/o SCA' yields a higher OS score than the full method (93.6 vs 93.1). The text says removing SCA decreases performance, which is not uniformly true across all metrics and tasks. Please qualify the ablation claim accordingly.","section":"Table 1"}],"recommendation":"major_revision","confidential_remarks":"The paper is an empirical contribution with moderate novelty. The vacuous-threshold issue is serious but fixable within the scope of a revision; I would not reject on the basis of the current evidence, but I would require the robustness analysis and error bars before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know. First, this is a useful combination of known ingredients: contrastive-center loss for source compactness, centroid alignment across domains, and a contrastive loss with an adaptive margin to push unknown target samples away. On Digits and Office-31 it beats OSDA+BP by 2–3 points on average, and the ablations show each module contributes. Second, the paper's only stated defense against pseudo-label noise is a threshold that literally does nothing. The threshold is set to 1/(N+1); for a softmax over N+1 classes the max probability is always at least that value, so reliable-sample selection excludes only exact ties. Every confident but wrong sample sails through. The centroids in Eqs. (5)–(7) and the contrastive mapping in Eq. (9) are built from those pseudo-labels, so the central mechanism is fed by unvetted noise. The paper asserts in Section 1 that 'SCA and SCM in itself are robust to such noisy labels,' but gives no bound, no pseudo-label accuracy measurement, and no experiment with oracle labels.\n\nWhat's genuinely good: the adaptive margin in Eq. (12) is a sensible idea—using the average distance to other source centroids as a per-class radius. The cosine reweighting is also reasonable. The evaluation is fairly thorough for the time: three digit tasks, Office-31 with two backbones, ablations, and a sensitivity study for omega. They report averages over three runs, though no error bars.\n\nWhere it's soft: beyond the threshold, the hyperparameters (lambda_s, lambda_c, lambda_t, omega) are selected on the test benchmarks; the omega grid search is on A->D and then applied everywhere. No code is released. And the Office-31 ablations show that w/o SCA or w/o SCM already beats OSDA+BP by large margins (e.g., A->D 87.8 vs 76.6 and 89.8 vs 76.6), so the incremental gain of the full method over its own components is small and could be within run-to-run variation. The paper doesn't report significance.\n\nNet: is the central claim true? It might be, but the current evidence doesn't establish the mechanism. A skeptical reader could argue the gains come from the combination of adversarial training plus any of the extra losses, not from robust pseudo-label exploitation. The paper deserves a serious referee—the problem is real, the method is clearly described, and the results are positive—but it needs major revision. The easiest fix is to show pseudo-label accuracy during training, compare the vacuous threshold against stricter ones, and add an oracle-label upper bound. Without that, the robustness claim is just an assertion.\n\nWho this is for: people doing open set domain adaptation or pseudo-label-based domain adaptation. I'd bring it to a reading group as a cautionary case in threshold design. My recommendation: engage with it, but push the authors to make the pseudo-label quality story concrete.","headline":"A plausible open-set domain adaptation method whose only pseudo-label safeguard is vacuous; the empirical gains are real but the mechanism is unproven.","tokens_in":13666,"tokens_out":3246,"would_cite":false,"duration_ms":71251,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Exploiting semantic structure—aligning target class centroids to source centroids and pushing unknown samples beyond an adaptive margin—improves open set domain adaptation, reaching 84.3% on Digits and 89.7% OS on Office-31.","keywords":["open set domain adaptation","semantic categorical alignment","semantic contrastive mapping","adaptive margin","contrastive-center loss","unknown class detection","centroid alignment","adversarial domain adaptation"],"falsifier":"Train the model while randomly corrupting the pseudo-labels of target known samples at a fixed rate, then compare final OS/OS* accuracy and the distance between estimated and true target centroids. If SCA/SCM still improve over the adversarial baseline under strong corruption, the losses are robust; if performance collapses, the reported gains depend on pseudo-label accuracy.","tokens_in":12656,"feed_emoji":"🧲","tokens_out":6822,"duration_ms":66052,"temperature":0.7,"pith_summary":"Open set domain adaptation asks a model trained on labeled source classes to classify target data that includes both shared classes and an unknown class never seen in the source. The paper tries to establish that the semantic structure of the open set data is the missing ingredient: known classes should be compact and aligned across domains, while unknown samples should be pushed away from all known-class centroids. It proposes two losses, Semantic Categorical Alignment (SCA) and Semantic Contrastive Mapping (SCM), built on top of adversarial domain adaptation, and reports that they improve average accuracy on Digit benchmarks to 84.3% and OS accuracy on Office-31 with AlexNet to 89.7%. A sympathetic reader would take the central claim to be that explicitly enlarging two margins—between known classes and between unknown and known classes—makes the open set decision boundary more reliable.","feed_headline":"Two semantic losses lift open-set domain adaptation to 89.7%","feed_subtitle":"Aligning class centroids and pushing unknown samples past an adaptive margin beats prior Digits and Office-31 results.","key_machinery":"The load-bearing objects are class centroids $c^k_s$ and $c^k_t$ in the feature space, updated per iteration with a cosine-similarity reweighting that down-weights faraway mini-batch centroids. Three losses run together: adversarial domain adaptation aligns source and target distributions while treating unknown as class $N+1$; the contrastive-center loss $L_{cct}$ compacts source samples; and the categorical center alignment loss $L_{cca}$ ties each target centroid to its source counterpart. The Semantic Contrastive Mapping uses distance functions from energy-based contrastive learning: known target samples are pulled to source centroids, unknown samples are penalized when they fall inside the adaptive margin $M^k$ of a known class. The adaptive margin is what lets the model grow the unknown/known separation as alignment proceeds.","core_discovery":"The central claim is that the open set problem is best handled by making the margin structure explicit. SCA computes class centroids in source and target feature spaces, compacts source samples around their own centroid with a contrastive-center loss, and aligns each target centroid to its corresponding source centroid through a cosine-similarity reweighted update. SCM then selects target samples whose predicted probability passes a threshold, attracts known samples to their source centroid, and repels unknown samples from every known centroid using a class-dependent adaptive margin $M^k$ given by the average distance from the target centroid of class $k$ to the other source centroids. The paper argues that this combination produces more discriminative features, better cross-domain centroid alignment, and a clearer unknown/known boundary, and supports the claim with ablations, feature visualizations, and accuracy gains over the OSDA+BP baseline.","pith_inferences":["The paper leaves implicit that the pseudo-label threshold could be made adaptive to per-class confidence; a testable extension is to replace the fixed $1/(N+1)$ cutoff with a calibrated uncertainty estimate.","Because the mechanism only needs class centroids and an adaptive margin, the same attract/distract idea could be lifted to open-set recognition without a labeled source, treating learned prototypes as centroids.","A direct stress test would vary the amount of pseudo-label noise on target known classes and check whether SCA/SCM still beats the adversarial baseline; the paper asserts robustness but does not bound the noise level at which the gains disappear."],"forward_implications":["On the Digit benchmarks, the full method reaches 84.3% average OS accuracy, 1.9% higher than the prior best, and raises unknown-class accuracy to 87.9%.","On Office-31 with AlexNet, the method reaches 89.7% OS and 90.7% OS*, improving over the OSDA+BP baseline by 2% and 10.5% respectively.","Ablation studies show that removing either SCA or SCM lowers accuracy, so the two losses work jointly rather than one carrying the whole improvement.","Experiments with a static margin degrade accuracy, indicating the unknown/known margin must expand as cross-domain alignment shrinks inter-class distances."],"supporting_citations":[{"why":"Defines the open set domain adaptation setting where unknown samples appear only in the target and supplies the OSDA+BP adversarial baseline that this method builds on and compares against.","marker":"[22]"},{"why":"Supplies the contrastive-center loss used in Semantic Categorical Alignment to compact same-class source representations.","marker":"[4]"},{"why":"Supplies the gradient reversal technique used by the Adversarial Domain Adaptation module.","marker":"[9]"},{"why":"Raises the open set domain adaptation task and establishes the OS/OS* evaluation protocol used in the experiments.","marker":"[3]"},{"why":"Provides the energy-based contrastive distance functions that Semantic Contrastive Mapping adapts for attracting known and repelling unknown samples.","marker":"[13]"},{"why":"Provides the Office-31 benchmark used for the domain adaptation experiments.","marker":"[31]"}],"fun_headline_variants":["Open-set adaptation gains from semantic structure","Centroid alignment and contrastive mapping push OSDA","Adaptive margins sharpen open-set domain adaptation","Semantic losses lift open-set domain adaptation","Cluster centroids and repulse unknowns: new OSDA"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes the model's own guesses about target class labels are trustworthy enough for estimating centroids and selecting reliable samples; when those guesses are systematically wrong, the alignment and distraction losses can reinforce the error instead of correcting it.","fun_headline_variants_meta":{"raw":{"variants":["Open-set adaptation gains from semantic structure","Centroid alignment and contrastive mapping push OSDA","Adaptive margins sharpen open-set domain adaptation","Semantic losses lift open-set domain adaptation","Cluster centroids and repulse unknowns: new OSDA"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00052,"raw_usage":{"total_tokens":2491,"prompt_tokens":888,"completion_tokens":1603,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":504,"completion_tokens_details":{"reasoning_tokens":1533}},"tokens_in":504,"tokens_out":1603,"duration_ms":12622,"temperature":1.0,"reasoning_tokens":1533,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T14:59:03.837083+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the model while randomly corrupting the pseudo-labels of target known samples at a fixed rate, then compare final OS/OS* accuracy and the distance between estimated and true target centroids. If SCA/SCM still improve over the adversarial baseline under strong corruption, the losses are robust; if performance collapses, the reported gains depend on pseudo-label accuracy.","supporting_citations":[{"cited_title":"Kuniaki, Y","cited_arxiv_id":null,"evidence_quote":"Defines the open set domain adaptation setting where unknown samples appear only in the target and supplies the OSDA+BP adversarial baseline that this method builds on and compares against."},{"cited_title":"Ce and S","cited_arxiv_id":null,"evidence_quote":"Supplies the contrastive-center loss used in Semantic Categorical Alignment to compact same-class source representations."},{"cited_title":"Ganin and V","cited_arxiv_id":null,"evidence_quote":"Supplies the gradient reversal technique used by the Adversarial Domain Adaptation module."},{"cited_title":"Busto and J","cited_arxiv_id":null,"evidence_quote":"Raises the open set domain adaptation task and establishes the OS/OS* evaluation protocol used in the experiments."},{"cited_title":"Hadsell, S","cited_arxiv_id":null,"evidence_quote":"Provides the energy-based contrastive distance functions that Semantic Contrastive Mapping adapts for attracting known and repelling unknown samples."},{"cited_title":"Saenko, B","cited_arxiv_id":null,"evidence_quote":"Provides the Office-31 benchmark used for the domain adaptation experiments."}],"review_version":1}