Pith. sign in

REVIEW 4 major objections 6 minor 62 references

TAPS : Frustratingly Simple Test Time Active Learning for VLMs

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read TAPS shows that a vision-language model can keep adapting on a single-sample data stream by querying an oracle only for its most uncertain images, beating state-of-the-art test-time prompt tuning on 10 cross-dataset transfer benchmarks…

desk verdict Useful first step on streaming test-time active learning, but the headline gains are confounded by the 5% oracle labels and need a same-budget random baseline before I'd trust them. read the letter →

arxiv 2507.20028 v1 pith:QEG7T5XP submitted 2025-07-26 cs.CV cs.AI

Dhruv Sarkar , Aprameyo Chakrabartty , Bibhudatta Bhanja This is my paper
classification cs.CVcs.AI
keywords test-timeadaptationactivelearningvision-languagemodelsprompttuningsingle-samplestreamentropythresholdclass-balancedbufferdistributionalignment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

TAPS proposes that a vision-language model can keep adapting at test time even when test data arrives one sample at a time and only a tiny fraction of the stream can be labeled. Its recipe is to query the oracle whenever a sample's entropy is above a threshold that is re-estimated from the stream, store the returned labels in a small buffer kept class-balanced by evicting the least-informative samples, and use those labels together with the usual unsupervised entropy loss to update the prompts. On 10 cross-dataset transfer benchmarks the method averages 67.40% top-1 accuracy versus 66.92% for the strongest prior baseline, and on 4 domain-generalization sets it averages 63.64% versus 63.42%, at roughly 0.63 seconds per sample. If these numbers hold, the framework would give safety-critical deployments a principled way to consult an expert exactly when the model is unsure, without waiting for batches.

What carries the argument

The load-bearing object is the dynamic entropy threshold $\tau_t = \hat{\mu}_t + z \hat{\sigma}_t$, which converts the streaming active-learning problem into an online quantile rule: query exactly those samples whose marginal entropy lies in the adaptively estimated tail of the current distribution. Two supporting mechanisms carry the rest of the argument: a class-balanced replacement policy that evicts the sample with the lowest cross-entropy loss from the most over-represented buffer class (tie-broken by the class with smallest average loss), and, on adaptation tasks, a class-aware distribution alignment loss that matches the features of each actively labeled sample to the per-class statistics of the source dataset rather than to global statistics. Together they define the composite objective $L = L_{\text{entropy}} + \alpha L_{\text{cross-entropy}} + \beta L_{\text{coarse}} + \gamma L_{\text{fine}}$, where the supervised and fine-grained terms are activated only when the corresponding oracle information is available. The theoretical scaffolding is two propositions: the buffer reaches asymptotic class balance with probability one as the annotation budget grows, and the empirical query ratio converges in probability to the target 5% under a normality assumption on the stream's cross-entropy values.

What would settle it

Run TAPS on a deliberately non-stationary or class-skewed test stream, such as one class presented for a long block followed by another, and measure the cumulative query ratio and the per-class composition of the buffer over time; if the query ratio deviates substantially from 5% (or the budget is exhausted early) or if the buffer's class counts stay far from $L/K$, then the convergence and equilibrium claims are falsified. A second check: record the empirical distribution of per-sample cross-entropy values across the stream; if they are clearly non-normal or strongly serially correlated, the normality assumption used in the proof of Proposition 3.2 is the point of failure.

Watch

Extended reading notes

Core claim

The paper's central claim is that active learning at test time is both feasible and beneficial for vision-language prompt tuning under a strict single-sample streaming assumption. TAPS decides on the spot whether to ask an oracle for the label of the current sample by comparing the entropy of its averaged logits to an adaptive threshold $\tau_t = \hat{\mu}_t + z \hat{\sigma}_t$, with the multiplier $z$ switching to a stricter value if the query budget is being spent too quickly. Queried labels enter a fixed-capacity buffer that is kept class-balanced by evicting the lowest-cross-entropy sample from the most over-represented class, and on domain-generalization tasks the labeled samples enable a fine-grained, per-class feature distribution alignment instead of a single global one. Under this scheme the paper reports average top-1 accuracy of 67.40% (cross-dataset transfer, 10 datasets) and 63.64% (domain generalization, 4 datasets), each above the compared state of the art, with per-sample latency around 0.63 seconds when the buffer is full.

Load-bearing premise

The guarantee that the method stays within its 5% query budget rests on the assumption that per-sample cross-entropy values are independent and identically distributed with a normal distribution, even though every prompt update changes those very values, so the budget and buffer-balance guarantees may not hold on real, non-stationary streams.

Editorial extensions

If this is right

  • If the reported gains replicate, single-sample streaming active learning becomes a practical option for test-time VLM adaptation, removing the batching requirement that prior ATTA methods impose.
  • The dynamic threshold mechanism means the annotation budget is spent on the most uncertain samples without being exhausted early, a prerequisite for any oracle-based deployment.
  • The fair-evaluation protocol—updating on the unsupervised loss first, then querying the oracle only after the sample has been scored—shows that the reported accuracy gains are not an artifact of label leakage.
  • On distribution-shift tasks, knowing the class of a few actively labeled samples improves feature alignment enough to beat coarse, class-agnostic alignment, suggesting that oracle labels are useful beyond the labeled samples themselves.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the theoretical budget guarantee is fragile because the i.i.d. normal assumption on cross-entropy values is almost certainly violated in practice—each prompt update shifts the distribution of entropies—so the 5% budget should be verified empirically on non-stationary streams rather than assumed from Proposition 3.2.
  • Editorial inference: the reported average gains over PromptAlign are small (0.48 and 0.22 percentage points), so the practical value may lie less in raw accuracy and more in enabling expert feedback in high-stakes settings where a wrong prediction is costly.
  • Editorial inference: the class-balance proof requires that under-represented classes be queried sufficiently often; on long-tailed or adversarially ordered streams the buffer may never balance, and one could test this by running TAPS on a class-imbalanced stream and tracking $|D_c| - L/K$ over time.
  • Editorial inference: because the method only tunes prompts, the same query-buffer-loss recipe could transfer to other parameter-efficient test-time adaptation techniques, such as adapter or LoRA updates, which would be a direct test of the framework's claimed generality.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes TAPS, a test-time active prompt learning method for VLMs in a single-sample streaming setting. It selects uncertain samples via an adaptively adjusted entropy threshold, stores oracle labels in a class-balanced bounded buffer, and optimizes a composite loss combining marginal entropy, cross-entropy on queried samples, and (for domain generalization) coarse and class-aware distribution alignment. The authors report average top-1 accuracy gains over PromptAlign of +0.48% across 10 cross-dataset transfer benchmarks and +0.22% across 4 domain-generalization datasets, with roughly 0.63s per-sample latency, and provide theoretical arguments for class-balanced buffer equilibrium and adaptive query convergence to a 5% budget.

Significance. If the active-selection benefit were cleanly isolated, the paper would make a useful contribution to an underexplored problem: streaming test-time active learning with real-time query decisions, low latency, and bounded memory. The method is simple, and the authors provide ablations for query percentage, buffer size, replacement policy, and alignment granularity. However, the central empirical claim is not currently supported because the comparison gives TAPS oracle labels that baselines do not receive, and the theoretical guarantees rest on unverified i.i.d. assumptions and a mismatch between the proof and the implemented switching rule. With a same-budget random-query control in the main tables and corrected theoretical claims, the contribution could be publishable.

major comments (4)
  1. [Sec. 4.1, Tables 2 and 3] The central comparison is confounded by the oracle-label budget: TAPS receives 5% true labels and uses them in the cross-entropy loss, the buffer, and the fine-grained alignment, whereas CLIP+TPT and PromptAlign receive no labels. The average gains of +0.48% (transfer) and +0.22% (domain generalization) therefore do not isolate the active-selection design. The only random-query control, Appendix Fig. 6, is run on four datasets and reports no variance or seed information, so it cannot support the headline claim. Add a same-budget random-query baseline and report the active-selection-only gain in the main tables, together with standard deviations over seeds. Note also that TAPS is worse than PromptAlign on Stanford Cars (67.84 vs 68.50) and ImageNet-Sketch (50.14 vs 50.23), so the abstract's phrase "consistent improvements" needs qualification.
  2. [Sec. 9, Table 3] The domain-generalization comparison is further biased by per-dataset loss coefficients selected after seeing the target data: the text reports alpha=1 for ImageNet-R and ImageNet-V2, alpha=0.15 for ImageNet-A, and alpha=0.5 for ImageNet-Sketch, with beta=1 and gamma=alpha. This is test-set selection and can account for part of the DG gains. The authors should either use a fixed hyperparameter rule for all DG datasets or show sensitivity of the results to these coefficient choices.
  3. [Prop. 3.2, Appendix 11.2] The adaptive query convergence proof assumes the per-image cross-entropy values Xi are i.i.d. N(mu, sigma^2), but the prompt is updated after every sample, so the entropy distribution evolves over time and the assumption is neither stated as an approximation nor verified. More importantly, the proof is a controller-tracking argument: zi is chosen so that the per-image query probability is 0.05 or 0.025, and the concentration bound then shows that the empirical query ratio tracks the designed probability. This does not establish a 5% budget guarantee for the actual non-stationary stream. In addition, the proof's switching threshold is 7.5%, whereas Algorithm 1 switches at N_queried/t >= alpha with alpha=0.05, so the theorem does not describe the implemented policy.
  4. [Prop. 3.1, Appendix 11.1] The class-balance proof requires that the per-step probability pi of querying a class with at least L/m samples is strictly less than 1, and asserts that pi=1 only at cyclic equilibrium. This is false: a skewed stream can keep querying the over-represented class indefinitely without the buffer reaching balance, so f_balance need not decrease and the failure-probability bound does not follow. The statement that "the probability of querying a class with at least L/m samples is strictly less than 1 until equilibrium is reached" is an unjustified assumption, not a consequence of Algorithm 2.
minor comments (6)
  1. [Throughout] The manuscript contains numerous typographical errors and malformed text, including "omiting", "disrepancy", "asymtotically", and the Table 2 header "Caltech101OxfordPets" running into the caption.
  2. [Tables 2 and 3] No standard deviations, confidence intervals, or numbers of seeds are reported for any of the main results; this is especially important given the small average margins between methods.
  3. [Sec. 4.2] The latency figure of 0.63s per sample versus PromptAlign's 0.41s is reported without a measurement protocol; specify hardware, batch size, number of runs, and whether the buffer is at maximum capacity during the measurement.
  4. [Eq. (8)] The symbol alpha is used both for the annotation budget in Algorithm 1 and for the cross-entropy loss coefficient in Eq. (8); this overloaded notation is confusing and should be disambiguated.
  5. [Appendix 10.4] The random-selection ablation in Fig. 6 does not state whether the same number of labels, the same loss coefficients, and the same buffer size were used for the random policy, so the comparison cannot be fully interpreted.
  6. [References] Several reference entries are incomplete or malformed, such as [1] and [10], whose page ranges and venue formatting appear misplaced; the reference list should be cleaned throughout.

Circularity Check

2 steps flagged · score 5.0 of 10

The adaptive-budget and class-balance 'guarantees' are largely programmed into the chosen z-scores and stream assumptions; the empirical benchmark comparison itself is not circular.

  1. self definitional [Section 3, Proposition 3.2; Appendix 11.2.1]
    "The threshold scaling factors zi are chosen adaptively via the switching rule: ... so that the ideal per-image query probability is set to α. ... In particular, if the running query ratio up to time i−1 is below 7.5%, then zi = z0.05 so that the ideal per-image query probability is 0.05, and if it is at least 7.5%, then zi = z0.025 so that the ideal probability is 0.025."

    The target budget α is inserted as the definition of the z-score: zselection is chosen as the Normal tail quantile for which P(X > μ + zσ) = α. Proposition 3.2 then proves that the empirical query ratio RN converges to α by showing that the online mean/variance estimates and the switching rule keep RN near p*, where p* was explicitly set to α (or 0.025 in over-query mode). The 'budget guarantee' is therefore a consistency statement about a controller tracking its own setpoint, not an independently derived property of the entropy-selection policy. If the cross-entropy stream is not i.i.d. Normal, the same z-choice no longer implies a 5% query rate, and the proof offers no other mechanism enforcing the budget. The convergence claim reduces by construction to the choice of tail quantile.

  2. other [Appendix 11.1, 'Probability of Failure to reach equilibrium']
    "Now it is to be noted that pi < 1 strictly since pi = 1 indicates that the system has already reached the cyclic equilibrium so it can not fail. Now it will fail if during the whole process of querying B-L images the classes with < L/m images at a particular instant are not queried at least fbalance(D0 l)/2 times at those instants."

    The proposition that the buffer becomes class-balanced assumes that under-represented classes are queried often enough to drive the imbalance measure down. The proof encodes this as pi < 1 and vmax < 1, but whether scarce classes receive queries is a property of the data stream and the entropy threshold, not of Algorithm 2's eviction rule. The eviction rule only decreases fbalance by 2 when a queried image belongs to a class with fewer than L/m images; it does nothing to make such images arise. Thus the asymptotic balance conclusion imports the stream behavior needed for the conclusion.

full rationale

The empirical core of the paper is not circular: the reported accuracies are obtained by running a fixed algorithm against external baselines on standard benchmarks, and the evaluation protocol labels after evaluation, which is a legitimate check against label leakage. No load-bearing self-citation or imported uniqueness theorem appears. However, the two theoretical justifications that support the practical claims are partly built from the algorithm's own definitions or assumptions. Prop. 3.2's 5% budget guarantee is a tracking result for a threshold whose z-score is defined to make the per-image query probability equal to the target, so the convergence conclusion restates the setpoint rather than deriving it from independent properties of the stream. The class-balance proof requires that under-represented classes actually get queried, which is precisely the external condition needed for balance; for skewed streams the proof's vmax < 1 assumption fails. These are circularity-adjacent reductions in the theoretical scaffolding. Independent benchmark concerns—the absence of a same-budget random-query baseline in the main tables, the per-dataset α choices in Table 3, and the no-variance random-selection ablation in Fig. 6—are real threats to the empirical claim but are not circular derivations, so they do not raise the circularity score further.

Assumptions & free parameters 11 free parameters · 4 assumptions · 0 invented entities

TAPS introduces no new physical entities. Its claims rest on a set of hand-chosen thresholds, z-scores, buffer sizes, and loss weights, and on distributional assumptions about test entropies. The theoretical results analyze the algorithm's own mechanics, namely budget tracking and buffer balancing, rather than demonstrating that the selected samples are the most valuable, which is supported only by ablations.

free parameters (11)
  • z_selection = z-score for top 5% (about 1.645)
    Hand-chosen scaling for the threshold; defines the target query probability of 0.05 and hence the budget.
  • z_high = z-score for top 2.5% (about 1.96)
    Hand-chosen stricter scaling used when over-querying.
  • tau0 = 2
    Static entropy threshold for the first 30 steps, chosen empirically.
  • t_tilde = 30
    Number of initial static-threshold steps, chosen empirically.
  • target_budget_alpha = 0.05
    Query budget fraction; 5% found optimal in ablations among 1%, 5%, 10%.
  • buffer_size = 150 (75 for ImageNet-V2 and ImageNet-Sketch)
    Memory cap; larger buffers help per ablations, 75 forced by memory constraints.
  • loss_alpha_eq8 = 1 for transfer; 1, 0.15, 0.5 for the four DG datasets
    Weight of the supervised cross-entropy loss in Eq. 8; tuned per dataset in Table 3.
  • loss_beta_gamma = beta=1, gamma=alpha for DG; beta=gamma=0 for transfer
    Weights for coarse and fine distribution alignment, used only on domain generalization tasks.
  • learning_rate = 5e-4 or 0.004 depending on dataset
    Per-dataset learning rate inherited from TPT and PromptAlign conventions.
  • num_augmentations = 63
    Number of augmented views per test sample, inherited from TPT.
  • rho = 10%
    Fraction of retained low-entropy augmentations, inherited from TPT.
assumptions (4)
  • domain assumption Cross-entropy values of test samples are i.i.d. N(mu, sigma^2)
    Used in Appendix 11.2 to prove query-rate convergence; not verified and likely violated because prompt updates shift entropies over time.
  • domain assumption Source dataset statistics (feature mean and variance) computed on ImageNet are a valid reference for distribution alignment
    Used in Eqs. 2-5, inherited from PromptAlign; the choice of ImageNet as source is taken as given.
  • standard math The class-imbalance function is non-increasing under the eviction rule except at zero
    The case analysis in Eq. 12 supports this for balanced-capacity buffers, though the appendix does not cover all edge cases.
  • ad hoc to paper The probability of querying a class with at least L/m samples is strictly less than 1 until equilibrium is reached
    Used to bound the failure probability in Appendix 11.1; the claim that pi=1 implies equilibrium is not generally true for skewed class distributions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TAPS : Frustratingly Simple Test Time Active Learning for VLMs." pith.science (2026). https://pith.science/paper/QEG7T5XP

@misc{pith2026250720028,
  author       = {Pith},
  title        = {Pith review of: TAPS : Frustratingly Simple Test Time Active Learning for VLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QEG7T5XP}},
  note         = {Machine review of arXiv:2507.20028}
}
read the original abstract

Test-Time Optimization enables models to adapt to new data during inference by updating parameters on-the-fly. Recent advances in Vision-Language Models (VLMs) have explored learning prompts at test time to improve performance in downstream tasks. In this work, we extend this idea by addressing a more general and practical challenge: Can we effectively utilize an oracle in a continuous data stream where only one sample is available at a time, requiring an immediate query decision while respecting latency and memory constraints? To tackle this, we propose a novel Test-Time Active Learning (TTAL) framework that adaptively queries uncertain samples and updates prompts dynamically. Unlike prior methods that assume batched data or multiple gradient updates, our approach operates in a real-time streaming scenario with a single test sample per step. We introduce a dynamically adjusted entropy threshold for active querying, a class-balanced replacement strategy for memory efficiency, and a class-aware distribution alignment technique to enhance adaptation. The design choices are justified using careful theoretical analysis. Extensive experiments across 10 cross-dataset transfer benchmarks and 4 domain generalization datasets demonstrate consistent improvements over state-of-the-art methods while maintaining reasonable latency and memory overhead. Our framework provides a practical and effective solution for real-world deployment in safety-critical applications such as autonomous systems and medical diagnostics.

Figures

Figures reproduced from arXiv: 2507.20028 by the authors.

Figure 1
Figure 1. Overview of our method. We query uncertain samples at test time and put them in a buffer of limited size. We decide to query a [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. (a). Results of varying active samples query percentage, [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Varying active samples query percentage, 5% active [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Changing the loss coefficient in loss function, the opti [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]
Figure 5
Figure 5. Figure 5: Varying the size of the buffer, increasing the buffer size [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]
Figure 6
Figure 6. Figure 6: Selecting random images vs our selection policy, our [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

62 extracted references · 52 canonical work pages

  1. [1]

    Align your prompts: Test-time prompting with distribution alignment for zero-shot generalization

    Jameel Abdul Samadh, Mohammad Hanan Gani, Noor Hussein, Muhammad Uzair Khattak, Muham- mad Muzammal Naseer, Fahad Shahbaz Khan, and Salman H Khan. Align your prompts: Test-time prompting with distribution alignment for zero-shot generalization. Advances in Neural Information Pro- cessing Systems, 36, 2024. 4, 7, 2, 3

    work page 2024

  2. [2]

    Active prompt learning in vision language models

    Jihwan Bang, Sumyeong Ahn, and Jae-Gil Lee. Active prompt learning in vision language models. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 27004–27014,

    work page

  3. [3]

    Food-101–mining discriminative compo- nents with random forests

    Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. Food-101–mining discriminative compo- nents with random forests. In Computer vision–ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceedings, part VI 13, pages 446–461. Springer, 2014. 2

    work page 2014

  4. [4]

    Describing textures in the wild

    Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. Describing textures in the wild. In Proceedings of the IEEE con- ference on computer vision and pattern recognition , pages 3606–3613, 2014. 2

    work page 2014

  5. [5]

    Imagenet: A large-scale hierarchi- cal image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchi- cal image database. In 2009 IEEE conference on com- puter vision and pattern recognition , pages 248–255. Ieee, 2009. 2

    work page 2009

  6. [6]

    Bert: Pre-training of deep bidirec- tional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirec- tional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018. 1

    arXiv 2018

  7. [7]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020. 1

    arXiv 2010

  8. [8]

    Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 ob- ject categories

    Li Fei-Fei, Rob Fergus, and Pietro Perona. Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 ob- ject categories. In 2004 conference on computer vi- sion and pattern recognition workshop , pages 178–

    work page 2004

Show all 62 references
  1. [9]

    Improved base- lines for vision-language pre-training

    Enrico Fini, Pietro Astolfi, Adriana Romero-Soriano, Jakob Verbeek, and Michal Drozdzal. Improved base- lines for vision-language pre-training. arXiv preprint arXiv:2305.08675, 2023. 1

    2023 arXiv

  2. [10]

    Active test- time adaptation: Theoretical analyses and an algo- rithm

    Shurui Gui, Xiner Li, and Shuiwang Ji. Active test- time adaptation: Theoretical analyses and an algo- rithm. arXiv preprint arXiv:2404.05094, 2024. 2, 7, 8

    2024 arXiv

  3. [11]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 1

    2016

  4. [12]

    Eurosat: A novel dataset and deep learning benchmark for land use and land cover clas- sification

    Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. Eurosat: A novel dataset and deep learning benchmark for land use and land cover clas- sification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12(7):2217– 2226, 2019. 2

    2019

  5. [13]

    The many faces of robustness: A critical analysis of out- of-distribution generalization

    Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, et al. The many faces of robustness: A critical analysis of out- of-distribution generalization. In Proceedings of the IEEE/CVF international ...

    2021

  6. [14]

    Natural adversarial ex- amples

    Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adversarial ex- amples. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 15262–15271, 2021. 2

    2021

  7. [15]

    Entropy-based active learning for object recognition

    Alex Holub, Pietro Perona, and Michael C Burl. Entropy-based active learning for object recognition. In 2008 IEEE Computer Society Conference on Com- puter Vision and Pattern Recognition Workshops , pages 1–8. IEEE, 2008. 1

    2008

  8. [16]

    Maple: Multi-modal prompt learning

    Muhammad Uzair Khattak, Hanoona Rasheed, Muhammad Maaz, Salman Khan, and Fahad Shahbaz Khan. Maple: Multi-modal prompt learning. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19113–19122,

  9. [17]

    Salad: Source-free active label-agnostic do- main adaptation for classification, segmentation and detection, 2022

    Divya Kothandaraman, Sumit Shekhar, Abhilasha Sancheti, Manoj Ghuhan, Tripti Shukla, and Dinesh Manocha. Salad: Source-free active label-agnostic do- main adaptation for classification, segmentation and detection, 2022. 1

    2022

  10. [18]

    3d object representations for fine-grained cate- gorization

    Jonathan Krause, Michael Stark, Jia Deng, and Li Fei- Fei. 3d object representations for fine-grained cate- gorization. In Proceedings of the IEEE international conference on computer vision workshops, pages 554– 561, 2013. 2

    2013

  11. [19]

    Heterogeneous un- certainty sampling for supervised learning

    David D Lewis and Jason Catlett. Heterogeneous un- certainty sampling for supervised learning. In Ma- chine learning proceedings 1994, pages 148–156. El- sevier, 1994. 1

    1994

  12. [20]

    Blip: Bootstrapping language-image pre-training 9 for unified vision-language understanding and genera- tion

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre-training 9 for unified vision-language understanding and genera- tion. In International conference on machine learning, pages 12888–12900. PMLR, 2022. 1

    2022

  13. [21]

    A comprehen- sive survey on test-time adaptation under distribution shifts

    Jian Liang, Ran He, and Tieniu Tan. A comprehen- sive survey on test-time adaptation under distribution shifts. arXiv preprint arXiv:2303.15361, 2023. 2

    2023 arXiv

  14. [22]

    A compre- hensive survey on test-time adaptation under distribu- tion shifts

    Jian Liang, Ran He, and Tieniu Tan. A compre- hensive survey on test-time adaptation under distribu- tion shifts. International Journal of Computer Vision, pages 1–34, 2024. 1

    2024

  15. [23]

    Fine-grained vi- sual classification of aircraft

    Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew Blaschko, and Andrea Vedaldi. Fine-grained vi- sual classification of aircraft. arXiv preprint arXiv:1306.5151, 2013. 2

    2013 arXiv

  16. [24]

    Evaluating prediction-time batch normal- ization for robustness under covariate shift

    Zachary Nado, Shreyas Padhy, D Sculley, Alexan- der D’Amour, Balaji Lakshminarayanan, and Jasper Snoek. Evaluating prediction-time batch normal- ization for robustness under covariate shift. arXiv preprint arXiv:2006.10963, 2020. 2

    2006 arXiv

  17. [25]

    Au- tomated flower classification over a large number of classes

    Maria-Elena Nilsback and Andrew Zisserman. Au- tomated flower classification over a large number of classes. In 2008 Sixth Indian conference on computer vision, graphics & image processing, pages 722–729. IEEE, 2008. 2

    2008

  18. [26]

    Cats and dogs

    Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and CV Jawahar. Cats and dogs. In 2012 IEEE con- ference on computer vision and pattern recognition , pages 3498–3505. IEEE, 2012. 2

    2012

  19. [27]

    Active learning by feature mixing

    Amin Parvaneh, Ehsan Abbasnejad, Damien Teney, Gholamreza Reza Haffari, Anton Van Den Hengel, and Javen Qinfeng Shi. Active learning by feature mixing. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 12237–12246, 2022. 1

    2022

  20. [28]

    Active domain adaptation via clustering uncertainty-weighted embeddings

    Viraj Prabhu, Arjun Chandrasekaran, Kate Saenko, and Judy Hoffman. Active domain adaptation via clustering uncertainty-weighted embeddings. In Pro- ceedings of the IEEE/CVF international conference on computer vision, pages 8505–8514, 2021. 1

    2021

  21. [29]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sas- try, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, ...

    2021

  22. [30]

    Do imagenet classifiers gen- eralize to imagenet? In International conference on machine learning, pages 5389–5400

    Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do imagenet classifiers gen- eralize to imagenet? In International conference on machine learning, pages 5389–5400. PMLR, 2019. 2

    2019

  23. [31]

    A survey of deep active learning

    Pengzhen Ren, Yun Xiao, Xiaojun Chang, Po-Yao Huang, Zhihui Li, Brij B Gupta, Xiaojiang Chen, and Xin Wang. A survey of deep active learning. ACM computing surveys (CSUR), 54(9):1–40, 2021. 1

    2021

  24. [32]

    Margin-based active learning for structured output spaces

    Dan Roth and Kevin Small. Margin-based active learning for structured output spaces. In Machine Learning: ECML 2006: 17th European Conference on Machine Learning Berlin, Germany, September 18- 22, 2006 Proceedings 17 , pages 413–424. Springer,

    2006

  25. [33]

    Akanksha Saran, Safoora Yousefi, Akshay Krishna- murthy, John Langford, and Jordan T. Ash. Streaming active learning with deep neural networks. InProceed- ings of the 40th International Conference on Machine Learning, pages 30005–30021. PMLR, 2023. 1

    2023

  26. [34]

    Improving robustness against common corruptions by covariate shift adaptation

    Steffen Schneider, Evgenia Rusak, Luisa Eck, Oliver Bringmann, Wieland Brendel, and Matthias Bethge. Improving robustness against common corruptions by covariate shift adaptation. Advances in neural infor- mation processing systems, 33:11539–11551, 2020. 2

    2020

  27. [35]

    Active learning for convolutional neural networks: A core-set approach

    Ozan Sener and Silvio Savarese. Active learning for convolutional neural networks: A core-set approach. arXiv preprint arXiv:1708.00489, 2017. 1

    2017 arXiv

  28. [36]

    Test-time prompt tuning for zero-shot general- ization in vision-language models

    Manli Shu, Weili Nie, De-An Huang, Zhiding Yu, Tom Goldstein, Anima Anandkumar, and Chaowei Xiao. Test-time prompt tuning for zero-shot general- ization in vision-language models. Advances in Neu- ral Information Processing Systems, 35:14274–14289,

  29. [37]

    Ucf101: A dataset of 101 human actions classes from videos in the wild

    Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah. Ucf101: A dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv:1212.0402, 2012. 2

    2012 arXiv

  30. [38]

    Deep coral: Correla- tion alignment for deep domain adaptation, 2016

    Baochen Sun and Kate Saenko. Deep coral: Correla- tion alignment for deep domain adaptation, 2016. 4

    2016

  31. [39]

    Vpa: Fully test-time visual prompt adaptation

    Jiachen Sun, Mark Ibrahim, Melissa Hall, Ivan Ev- timov, Z Morley Mao, Cristian Canton Ferrer, and Caner Hazirbas. Vpa: Fully test-time visual prompt adaptation. In Proceedings of the 31st ACM Interna- tional Conference on Multimedia , pages 5796–5806,

  32. [40]

    Test-time training with self-supervision for generalization under distri- bution shifts

    Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei Efros, and Moritz Hardt. Test-time training with self-supervision for generalization under distri- bution shifts. In International conference on machine learning, pages 9229–9248. PMLR, 2020. 2

    2020

  33. [41]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems , 30, 2017. 1

    2017

  34. [42]

    Tent: Fully test-time adaptation by entropy minimization

    Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. arXiv preprint arXiv:2006.10726, 2020. 1, 2 10

    2006 arXiv

  35. [43]

    Active source free domain adaptation, 2022

    Fan Wang, Zhongyi Han, Zhiyan Zhang, and Yilong Yin. Active source free domain adaptation, 2022. 1

    2022

  36. [44]

    Learning robust global representations by penalizing local predictive power.Advances in Neural Information Processing Systems, 32, 2019

    Haohan Wang, Songwei Ge, Zachary Lipton, and Eric P Xing. Learning robust global representations by penalizing local predictive power.Advances in Neural Information Processing Systems, 32, 2019. 2

    2019

  37. [45]

    Position-guided text prompt for vision-language pre-training

    Jinpeng Wang, Pan Zhou, Mike Zheng Shou, and Shuicheng Yan. Position-guided text prompt for vision-language pre-training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 23242–23251, 2023. 1

    2023

  38. [46]

    Active learning us- ing uncertainty information

    Yazhou Yang and Marco Loog. Active learning us- ing uncertainty information. In 2016 23rd Inter- national Conference on Pattern Recognition (ICPR) , pages 2646–2651. IEEE, 2016. 1

    2016

  39. [47]

    Robust test-time adaptation in dynamic scenarios

    Longhui Yuan, Binhui Xie, and Shuang Li. Robust test-time adaptation in dynamic scenarios. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15922–15932, 2023. 1

    2023

  40. [48]

    Central moment discrepancy (cmd) for domain-invariant representation learning, 2019

    Werner Zellinger, Thomas Grubinger, Edwin Lughofer, Thomas Natschl ¨ager, and Susanne Saminger-Platz. Central moment discrepancy (cmd) for domain-invariant representation learning, 2019. 4

    2019

  41. [49]

    A comparative survey of deep active learning

    Xueying Zhan, Qingzhong Wang, Kuan-hao Huang, Haoyi Xiong, Dejing Dou, and Antoni B Chan. A comparative survey of deep active learning. arXiv preprint arXiv:2203.13450, 2022. 1

    2022 arXiv

  42. [50]

    Memo: Test time robustness via adaptation and aug- mentation

    Marvin Zhang, Sergey Levine, and Chelsea Finn. Memo: Test time robustness via adaptation and aug- mentation. Advances in neural information processing systems, 35:38629–38642, 2022. 2

    2022

  43. [51]

    Conditional prompt learning for vision- language models

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Conditional prompt learning for vision- language models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recogni- tion, pages 16816–16825, 2022. 1, 2, 3

    2022

  44. [52]

    Learning to prompt for vision-language models

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision-language models. International Journal of Computer Vision , 130(9):2337–2348, 2022. 1, 2, 3 11 TAPS : Frustratingly Simple Test Time Active Learning for VLMs Supplementary Material

    2022

  45. [53]

    Active Learning In active learning, we have an unlabelled dataset Du

    Preliminaries 7.1. Active Learning In active learning, we have an unlabelled dataset Du. The typical setting is that of multi-class classification, havingK classes. The training happens in an iterative way where, in each iteration, the model selects some samples from Du, which...

  46. [54]

    Active Learning Active Learning promotes label efficiency by imposing a label budget

    Related Work 8.1. Active Learning Active Learning promotes label efficiency by imposing a label budget. It can be used in a variety of settings to gain knowledge about some aspect that is not already known and the model is uncertain about via an oracle. The queried samples are...

  47. [55]

    In domain generalization, we evaluate on four out-of-distribution (OOD) variants of ImageNet [5]; ImageNet-Sketch [44],ImageNet-A [14], ImageNet-V2

    Experimental Setup Datasets. In domain generalization, we evaluate on four out-of-distribution (OOD) variants of ImageNet [5]; ImageNet-Sketch [44],ImageNet-A [14], ImageNet-V2

  48. [56]

    For cross-dataset transfer, we try on 10 image classification datasets which cover a wide variety of visual recognition tasks

    and ImageNet-R [13]. For cross-dataset transfer, we try on 10 image classification datasets which cover a wide variety of visual recognition tasks. Among these Caltech101 [8]; five datasets which are fine-grained Stan- fordCars [18],Flowers102 [25],OxfordPets [26],Food101

  49. [57]

    and FGVC-Aircraft [23], which contain images of transportation, flowers and animals; and four datasets of textures, satellite imagery, scenes and human actions which are DTD [4], EUROSAT [12], SUN397 [40] and UCF101

  50. [58]

    Implementation Details

    respectively. Implementation Details. Following PromptAlign [1], using a single test sample we optimize the prompts on both the text and vision branches. Our models were implemented on a single NVIDIA A40 48GB GPU using the PyTorch framework. Refer to section 7.3, we take n = ...

  51. [59]

    MaPLe [16] is a multi-modal prompt learn- ing baseline, which adapts CLIP by learning deep prompts on both the text and vision branches

    method. MaPLe [16] is a multi-modal prompt learn- ing baseline, which adapts CLIP by learning deep prompts on both the text and vision branches. TPT is a test-time prompt tuning method that tunes the prompt at test time per input sample, which achieved strong performance in pr...

  52. [60]

    Ablation studies 10.1. Active samples queried percentage: Increasing the number of samples actively queried increases the model’s robustness and hence helps improve its perfor- mance, especially for more challenging datasets. But that comes at a cost of the annotation budget, ...

  53. [61]

    Class Balance in Buffer Our policy is to first fill the buffer, not caring about the class of the image added

    Theoretical Discussion 11.1. Class Balance in Buffer Our policy is to first fill the buffer, not caring about the class of the image added. Once the buffer is full, then we remove images from the the class which has the maximum number of images for each newly queried image. If...

  54. [62]

    5% regime

    So asymtotically S tends to 0. So in conclusion P(f ailure) ≈ 0 asymtotically. Lemma 11.1. Let L ∈ N be fixed and c >1 be a constant. Then, lim B→∞ BL cB = 0. Proof. For B ≥ 1, define aB = BL cB . We show that for all sufficiently largeB, there exists a con- stant ρ with 0 < ρ...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.