A Step Towards Robust Unsupervised Domain Adaptation via Fine-Tuning and Reinforcement Learning

Ashutosh Kumar Sinha; Rahul Mishra; Sushant Dagaji Desale

REVIEW 3 major objections 6 minor 24 references

A Step Towards Robust Unsupervised Domain Adaptation via Fine-Tuning and Reinforcement Learning

T0 review · 3 major / 6 minor · reviewed 2026-07-12 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A two-stage CLIP fine-tuning and confidence-guided self-training method raises both clean accuracy and adversarial robustness in unsupervised domain adaptation.

desk verdict Practical CLIP robust-UDA recipe that recombines known pieces and posts real gains on three benchmarks; the 'RL' label is loose and pseudo-label quality under attack is never checked. read the letter →

arxiv 2607.03600 v1 pith:6AWI6SWL submitted 2026-07-03 cs.CV cs.AI

Sushant Dagaji Desale , Rahul Mishra , Ashutosh Kumar Sinha This is my paper · ORCID

classification cs.CVcs.AI

keywords unsuperviseddomainadaptationadversarialrobustnessCLIPpseudo-labelingPGDsupervisedfine-tuningconfidencethresholdvision-languagemodels

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Unsupervised domain adaptation usually fails when the target domain is also under adversarial attack, because noisy pseudo-labels and the shift between clean source and perturbed target features destroy accuracy and robustness at once. This paper claims that a simple two-stage procedure on top of a frozen CLIP visual encoder solves the problem: first adversarially fine-tune only a linear head and the projection layer on labeled source data, then refine the model on unlabeled target data by accepting only high-confidence pseudo-labels whose threshold slowly decays. The resulting classifier is trained on mixed clean and PGD-perturbed batches drawn from both domains. On OfficeHome, PACS and VisDA the method lifts clean accuracy by roughly ten points and adversarial robustness by roughly sixteen points over prior robust UDA baselines, showing that partial unfreezing plus a curriculum of filtered pseudo-labels is enough to keep CLIP’s semantic priors while hardening the model against attacks.

What carries the argument

SFT+RL: the two-stage pipeline that first hardens a partially-unfrozen CLIP projection plus linear head with on-the-fly PGD on labeled source data, then repeatedly filters unlabeled target samples by a decaying confidence threshold τ and retrains the same head on the composite clean-plus-adversarial set.

What would settle it

On any OfficeHome or PACS domain pair, measure the true accuracy of the filtered pseudo-labels under PGD-20 at the moment they are accepted; if that accuracy falls below roughly 70 percent while the reported robust accuracy still rises, the curriculum claim is false.

Watch

Extended reading notes

Core claim

SFT+RL establishes that adversarially supervised fine-tuning of CLIP’s projection layer on source data, followed by reinforcement-style self-training that admits only high-confidence target pseudo-labels under a decaying threshold, yields a single linear classifier that simultaneously improves clean accuracy and PGD robustness across three standard domain-adaptation benchmarks.

Load-bearing premise

High-confidence pseudo-labels produced by the partially adapted CLIP head stay accurate enough under the same PGD attack used at test time that the decaying-threshold curriculum does not inject systematic label noise the later adversarial training cannot correct.

Editorial extensions

If this is right

CLIP-style vision-language models can be made adversarially robust for domain shift without full backbone fine-tuning or labeled target data.
A simple decaying confidence filter is sufficient to turn noisy target predictions into useful training signal under attack.
Partial unfreezing of only the projection layer preserves zero-shot semantic priors while still allowing adaptation to adversarial noise.
The same two-stage recipe can be applied to other frozen foundation encoders that already possess strong cross-domain features.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The same curriculum could be extended to larger multimodal models by replacing the linear head with a lightweight adapter, reducing compute while retaining the robustness gains.
If the confidence threshold were itself learned by a small meta-network rather than hand-scheduled, the method might adapt automatically to harder domain gaps or stronger attacks.
Failure modes on sketch or synthetic domains suggest the approach still relies on CLIP’s pre-training distribution covering the target style; testing on truly out-of-distribution medical or satellite imagery would expose this limit.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

This paper gives a clean two-stage pipeline for making CLIP more robust under domain shift and PGD: first adversarially fine-tune a linear head plus the projection layer on labeled source, then run confidence-thresholded self-training on target mixed with source, again under adversarial training, with a decaying threshold. The numbers are the main thing you should know—average +10% clean and +16% robust accuracy across OfficeHome, PACS and VisDA versus DART, SRoUDA, ARTUDA and the usual UDA+AT/TRADES/MART baselines. Tables 1–3 and the ablation in Table 4 look consistent; both stages contribute.

What is actually new is the particular packaging on a frozen CLIP ViT-B/32 backbone with partial unfreezing and the curriculum. None of the ingredients (PGD, confidence filtering, source–target mixing) is novel, but the combination is a usable engineering recipe that keeps CLIP’s priors while hardening the head. The algorithm is written clearly enough to re-implement, and they include a small sensitivity check on the threshold.

Soft spots are real but proportionate. Calling the second stage “reinforcement learning” is just wrong; there is no policy or reward, only self-training. They claim three runs yet never show standard deviations. Most importantly, they never measure the true accuracy of the filtered pseudo-labels on clean or PGD-perturbed target images, so we cannot tell how much of the robustness gain is simply the model learning to be confident about wrong labels. That is the weakest link, and the stress-test note is right to flag it. Attack evaluation is also narrow (mostly one PGD budget). The code link is a shortener without a commit hash.

Still, the central empirical claim holds relative to the baselines they chose, and the work is honest about its scope. This is for people who actually ship robust foundation-model adapters or need a strong baseline on these three datasets. It is not a conceptual breakthrough, but it is solid enough that a serious editor should send it to referees rather than desk-reject. I would bring it to reading group only if we are currently working on CLIP robustness; otherwise maybe. I would not cite it myself in the next year unless I need the exact numbers.

Referee Report

3 major / 6 minor

Summary. The paper proposes SFT+RL, a two-stage framework for adversarially robust unsupervised domain adaptation on top of a frozen CLIP ViT-B/32 visual encoder. Stage 1 (SFT) adversarially fine-tunes a linear classifier and partially unfreezes CLIP’s visual projection layer on labeled source data using 20-step PGD. Stage 2 (called “Reinforcement Learning” but implemented as confidence-driven self-training) generates pseudo-labels on unlabeled target data, retains only those whose max softmax exceeds a decaying threshold τ (starting at 0.85, floor 0.7), forms a composite source+high-confidence-target set, and continues adversarial training on clean/adversarial mixed batches. Evaluations on OfficeHome, PACS and VisDA under PGD-20 (ε=2/255 or 8/255) report average gains of roughly 10% clean accuracy and 15% robust accuracy over DANN, UDA+AT/TRADES/MART, ARTUDA, SRoUDA and DART; an ablation (Table 4) and sensitivity study (Table 5) support the contribution of both stages and of the decaying threshold.

Significance. If the reported gains hold under stronger scrutiny, the work supplies a practical, architecture-compatible recipe for hardening CLIP-style VLMs against simultaneous domain shift and white-box adversarial attack—an important and still under-explored intersection. The partial-unfreezing schedule, the explicit composite-batch adversarial objective, and the decaying-confidence curriculum are concrete, reproducible design choices that other groups can adopt or ablate. The paper already supplies an algorithmic listing (Algorithm 1), a public codebase link, and multi-dataset tables that make the empirical claim falsifiable. These engineering contributions are valuable even if the “RL” nomenclature is loose and even if some statistical reporting remains incomplete.

major comments (3)

Section 3.3 and Algorithm 1 (lines 14–21): the central claim that the RL stage improves robustness rests on the assumption that high-confidence pseudo-labels remain accurate under the same PGD threat model later used at test time. The manuscript never reports the true accuracy (or class-wise error rate) of the filtered set D_conf on clean or adversarially perturbed target images, nor how that accuracy evolves across the three RL cycles. Without this diagnostic, it is impossible to rule out that the subsequent adversarial training simply hardens the model to systematically incorrect targets. A short table of pseudo-label precision/recall (or accuracy under PGD) per cycle would directly test the weakest assumption of the method.
Tables 1–3 and the abstract’s headline averages: the paper states that all numbers are “averaged over three runs” yet never reports standard deviations or confidence intervals. Given that several domain pairs show large absolute jumps (e.g., OfficeHome Pr→Ar clean 43.7→62.1, PACS Sk→Ar clean 71.9→82.4), the absence of variance makes it impossible to judge whether the claimed 10.2 % / 15.8 % average improvements are statistically reliable or driven by a few high-variance transfers. Adding ±std or a simple paired significance test is a load-bearing reporting requirement for the central empirical claim.
Section 3.3 title and surrounding prose repeatedly call the second stage “Reinforcement Learning,” yet the algorithm contains no policy, reward, value function or policy-gradient update—only confidence-thresholded self-training with a decaying τ. This nomenclature is misleading and risks conflating the method with genuine RL-based domain adaptation. Either rename the stage (e.g., “Confidence-Guided Self-Training”) or supply an explicit RL formulation that matches the implemented procedure.

minor comments (6)

Abstract and §1 claim “average improvements of 10.2 % clean / 15.8 % robust”; the precise baseline against which these percentages are computed is never stated (DANN? best prior? mean of all priors?). Clarify the arithmetic.
Figure 1 caption and t-SNE axes are hard to read; the three clusters (source / clean target / adversarial target) are not quantitatively separated (e.g., by silhouette or MMD). A short quantitative measure would strengthen the motivation.
Inconsistent notation: Algorithm 1 uses “SFT+RF” while the rest of the paper uses “SFT+RL”; ε is set to 8/255 in the SFT generation equations but 2/255 in the experimental protocol. Harmonize.
Table 3 (VisDA) uses ε=8/255 while Tables 1–2 use ε=2/255; a brief justification for the different threat models would help readers compare across datasets.
Several citations appear truncated or mistyped (e.g., “tomm-ude” for OfficeHome, missing page numbers). Clean the bibliography.
The claim of “cross-modal curriculum” and “updating the text encoder” (Introduction and contributions) is not reflected in Algorithm 1 or Figure 2, where the text encoder and CTP remain frozen. Either implement the claimed text-side update or remove the claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: empirical method paper whose accuracy/robustness claims are measured on external public benchmarks against independent baselines, not derived by construction from fitted inputs or self-citation.

full rationale

SFT+RL is a two-stage engineering procedure (PGD adversarial fine-tuning of a linear head + CLIP projection on labeled source, followed by confidence-thresholded self-training on unlabeled target with decaying au and mixed clean/adversarial batches). The load-bearing claims are purely empirical averages (+10.2 % clean, +15.8 % robust accuracy) reported in Tables 1–3 and Figure 4 against public datasets (OfficeHome, PACS, VisDA) and external baselines (DANN, UDA+AT/TRADES/MART, ARTUDA, SRoUDA, DART). No equation equates a reported accuracy to a free parameter by definition; the decaying-threshold curriculum (Alg. 1, Eq. 11) and composite loss (Eq. 10) are design choices whose quality is tested by ablation (Table 4) rather than assumed true by construction. There are no uniqueness theorems, no self-citations that close a logical loop, and no renaming of a known closed-form result presented as a derivation. The skeptic’s concern about unmeasured pseudo-label accuracy under PGD is a validity/assumption risk, not circularity. The derivation chain is therefore self-contained and non-circular.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The method rests on standard adversarial-training and self-training assumptions plus a handful of hand-chosen hyper-parameters that control the curriculum and the attack strength. No new physical entities are postulated; the free parameters are ordinary ML knobs.

free parameters (3)

initial confidence threshold τ = 0.85 → 0.7
Set by hand to 0.85 and decayed by 0.9 each cycle down to a floor of 0.7; directly controls which pseudo-labels enter training.
PGD attack budget (ε, α, steps) = ε=2/255 (or 8/255), 20 steps
Fixed to ε=2/255 or 8/255, α=0.5/255 or 2/255, 20 steps for both training and evaluation; chosen rather than derived.
learning-rate schedule and unfreezing epoch = 1e-3 / 1e-5, unfreeze after epoch 2
Classifier lr=1e-3 then 1e-5, projection unfrozen after epoch 2; pure design choices.

assumptions (3)

domain assumption CLIP ViT-B/32 visual embeddings remain a useful semantic prior after only the final projection layer is lightly adapted.
Invoked throughout Section 3.1–3.2; never proved, only empirically observed.
domain assumption High-confidence softmax predictions on unlabeled target data are sufficiently correct to serve as training labels under the same threat model.
Core premise of the RL stage (Section 3.3).
standard math ℓ∞-bounded PGD is an adequate proxy for the adversarial threat of interest.
Standard in the adversarial-robustness literature; used for both training and evaluation.

how reviews work

0 comments

Cite this review

Pith. "Pith review of A Step Towards Robust Unsupervised Domain Adaptation via Fine-Tuning and Reinforcement Learning." pith.science (2026). https://pith.science/paper/6AWI6SWL

@misc{pith2026260703600,
  author       = {Pith},
  title        = {Pith review of: A Step Towards Robust Unsupervised Domain Adaptation via Fine-Tuning and Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6AWI6SWL}},
  note         = {Machine review of arXiv:2607.03600}
}

read the original abstract

Adversarial robustness in Unsupervised Domain Adaptation (UDA) remains a significant challenge due to noisy pseudo labels and inherent distributional shifts between the clean source and adversarially perturbed target domains. Existing approaches often fail to achieve an optimal trade-off between robustness and accuracy, as pseudo-labels generated by domain-adapted models tend to introduce classification errors under adversarial attacks. In this work, we propose \textbf{SFT+RL}, a two-stage robust UDA framework that integrates Supervised Fine Tuning (SFT) and Reinforcement Learning (RL) on top of CLIP's pre-trained visual encoder. In the SFT stage, we adversarially fine-tune a linear classifier using PGD-based perturbations over the labelled source domain while partially unfreezing CLIP's projection layer. It allows adaptation to adversarial noise while preserving CLIP's rich semantic priors. We introduce a confidence-guided pseudo-labeling strategy in the RL stage to annotate unlabeled target samples progressively. Pseudo labels are filtered using a decaying confidence threshold to balance quality and coverage, and the model is trained on a composite dataset formed by combining clean source samples with high-confidence target samples. Adversarial training is applied to mixed batches of clean and adversarial examples to enhance cross-domain robustness. Comprehensive evaluations on three benchmark datasets OfficeHome~\cite{tomm-ude}, PACS~\cite{pacs}, and VisDA~\cite{visda} demonstrate the effectiveness of our approach. Notably, \textbf{SFT+RL} achieves average improvements of \textbf{10.2\%} in clean accuracy and \textbf{15.8\%} in adversarial robustness across all three datasets, outperforming existing state-of-the-art methods.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

24 extracted references · 4 linked inside Pith

[1]

Bhagwatkar, S

R. Bhagwatkar, S. Nayak, P . Bashivan, and I. Rish. Improving adversar- ial robustness in vision-language models with architecture and prompt design. In Y . Al-Onaizan, M. Bansal, and Y .-N. Chen, editors,Findings of the Association for Computational Linguistics: EMNLP 2024 , pages 17003–17020, 2024

2024
[2]

Ganin, E

Y . Ganin, E. Ustinova, H. Ajakan, P . Germain, H. Larochelle, F. Lavi- olette, M. March, and V . Lempitsky. Domain-adversarial training of neural networks. Journal of Machine Learning Research , 17(59):1–35, 2016

2016
[3]

T. Gao, A. Fisch, and D. Chen. Making pre-trained language models robust to spelling errors. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages 2735–2750, 2021

2021
[4]

Huang, Y

H. Huang, Y . Wang, S. Erfani, Q. Gu, J. Bailey, and X. Ma. Explor- ing architectural ingredients of adversarially robust deep neural net- works. In M. Ranzato, A. Beygelzimer, Y . Dauphin, P . Liang, and J. W. V aughan, editors,Advances in Neural Information Processing Systems , volume 34, pages 5545–5559, 2021

2021
[5]

G. Kang, L. Jiang, Y . Y ang, and A. G. Hauptmann. Contrastive adap- tation network for unsupervised domain adaptation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 4893–4902, 2019

2019
[6]

P . W. Koh, S. Sagawa, H. Marklund, S. M. Xie, M. Zhang, A. Balsub- ramani, W. Hu, M. Y asunaga, R. L. Phillips, I. Gao, et al. Wilds: A benchmark of in-the-wild distribution shifts. In International confer- ence on machine learning , pages 5637–5664. PMLR, 2021

2021
[7]

D. Li, Y . Y ang, Y .-Z. Song, and T. M. Hospedales. Deeper, Broader and Artier Domain Generalization. In Proceedings of the IEEE In- ternational Conference on Computer Vision (ICCV) , pages 5542–5550. IEEE, 2017

2017
[8]

D. Li, Y . Y ang, Y .-Z. Song, and T. Hospedales. Learning to general- ize: Meta-learning for domain generalization. Proceedings of the AAAI Conference on Artiﬁcial Intelligence , 32(1), 2018

2018

Show all 24 references

[9]

Liang, T

P . Liang, T. Wang, and Y . Zhang. We need more training data: Eval- uating domain adaptation in nlp. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL) , 2020

2020
[10]

Lo and V

S.-Y . Lo and V . Patel. Exploring adversarially robust training for un- supervised domain adaptation. In Proceedings of the Asian Conference on Computer Vision, pages 4093–4109, 2022

2022
[11]

Madry, A

A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083, 2017

2017 arXiv
[12]

Madry, A

A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations (ICLR) , pages 1–23, 2018

2018
[13]

X. Peng, B. Usman, N. Kaushik, J. Hoffman, D. Wang, and K. Saenko. Visda: The visual domain adaptation challenge. arXiv preprint arXiv:1710.06924, 2017

2017 arXiv
[14]

Radford, J

A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P . Mishkin, J. Clark, et al. Learning transferable visual models from natural language supervision. In International Con- ference on Machine Learning (ICML) , pages 8748–8763, 2021

2021
[15]

Saito, D

K. Saito, D. Kim, S. Sclaroff, T. Darrell, and K. Saenko. Semi- supervised domain adaptation via minimax entropy. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , pages 8050–8058, 2019

2019
[16]

Schlarmann, N

C. Schlarmann, N. D. Singh, F. Croce, and M. Hein. Robust clip: Unsu- pervised adversarial ﬁne-tuning of vision embeddings for robust large vision-language models. arXiv preprint arXiv:2402.12336, 2024

2024 arXiv
[17]

Tanwisuth, Y

K. Tanwisuth, Y . Aytar, and A. Zisserman. Textual redundancy in vision-language models. In Proceedings of the 40th International Con- ference on Machine Learning (ICML) , 2023

2023
[18]

V enkateswara, J

H. V enkateswara, J. Eusebio, S. Chakraborty, and S. Panchanathan. Deep hashing network for unsupervised domain adaptation. In Proceed- ings of the IEEE Conference on Computer Vision and Pattern Recogni- tion, pages 5018–5027, 2017

2017
[19]

H. Wang, G. Wang, Y . Li, D. Zhang, and L. Lin. Transferable, control- lable, and inconspicuous adversarial attacks on person re-identiﬁcation with deep mis-ranking. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 342–351, 2020

2020
[20]

Wang and W

M. Wang and W. Deng. Deep visual domain adaptation: A survey. Neu- rocomputing, 312:135–153, 2018

2018
[21]

Y . Wang, H. Hazimeh, N. Ponomareva, A. Kurakin, I. Hammoud, and R. Arora. Dart: A principled approach to adversarially robust unsuper- vised domain adaptation. arXiv preprint arXiv:2402.11120, 2024

2024 arXiv
[22]

Y ang, R

J. Y ang, R. Xu, R. Li, X. Qi, X. Shen, G. Li, and L. Lin. An adversarial perturbation oriented domain adaptation approach for semantic segmen- tation. Proceedings of the AAAI Conference on Artiﬁcial Intelligence , 34(07):12613–12620, 2020

2020
[23]

Zhang, Y

H. Zhang, Y . Y u, J. Jiao, E. Xing, L. E. Ghaoui, and M. Jordan. Theo- retically principled trade-off between robustness and accuracy. In Inter- national Conference on Machine Learning (ICML) , pages 7472–7482, 2019

2019
[24]

Zhu, J.-L

W. Zhu, J.-L. Yin, B.-H. Chen, and X. Liu. SRoUDA: Meta Self- Training for Robust Unsupervised Domain Adaptation. In Proceedings of the AAAI Conference on Artiﬁcial Intelligence , pages 3852–3860, 2023

2023

Pith tools

Reviewed July 12, 2026 · model on record in the stance chip above.

[1] [1]

Bhagwatkar, S

R. Bhagwatkar, S. Nayak, P . Bashivan, and I. Rish. Improving adversar- ial robustness in vision-language models with architecture and prompt design. In Y . Al-Onaizan, M. Bansal, and Y .-N. Chen, editors,Findings of the Association for Computational Linguistics: EMNLP 2024 , pages 17003–17020, 2024

2024

[2] [2]

Ganin, E

Y . Ganin, E. Ustinova, H. Ajakan, P . Germain, H. Larochelle, F. Lavi- olette, M. March, and V . Lempitsky. Domain-adversarial training of neural networks. Journal of Machine Learning Research , 17(59):1–35, 2016

2016

[3] [3]

T. Gao, A. Fisch, and D. Chen. Making pre-trained language models robust to spelling errors. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages 2735–2750, 2021

2021

[4] [4]

Huang, Y

H. Huang, Y . Wang, S. Erfani, Q. Gu, J. Bailey, and X. Ma. Explor- ing architectural ingredients of adversarially robust deep neural net- works. In M. Ranzato, A. Beygelzimer, Y . Dauphin, P . Liang, and J. W. V aughan, editors,Advances in Neural Information Processing Systems , volume 34, pages 5545–5559, 2021

2021

[5] [5]

G. Kang, L. Jiang, Y . Y ang, and A. G. Hauptmann. Contrastive adap- tation network for unsupervised domain adaptation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 4893–4902, 2019

2019

[6] [6]

P . W. Koh, S. Sagawa, H. Marklund, S. M. Xie, M. Zhang, A. Balsub- ramani, W. Hu, M. Y asunaga, R. L. Phillips, I. Gao, et al. Wilds: A benchmark of in-the-wild distribution shifts. In International confer- ence on machine learning , pages 5637–5664. PMLR, 2021

2021

[7] [7]

D. Li, Y . Y ang, Y .-Z. Song, and T. M. Hospedales. Deeper, Broader and Artier Domain Generalization. In Proceedings of the IEEE In- ternational Conference on Computer Vision (ICCV) , pages 5542–5550. IEEE, 2017

2017

[8] [8]

D. Li, Y . Y ang, Y .-Z. Song, and T. Hospedales. Learning to general- ize: Meta-learning for domain generalization. Proceedings of the AAAI Conference on Artiﬁcial Intelligence , 32(1), 2018

2018