{"id":"934779a7-4974-427d-b294-8d31579749ad","arxiv_id":"2506.23856","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Using class-name text embeddings, rather than image features, to condition prompts reduces the base-new accuracy tradeoff in vision-language prompt tuning.","lead":"This paper finds that conditioning learned prompts on class names rather than image features better balances accuracy on training and new classes for CLIP-style models. It proposes a lightweight plugin, CaPT, that improves five existing prompt tuning methods on 11 datasets.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Sec. 3.2 does not isolate 'textual' from 'class-level' conditioning: CoCoOp-FRN is task-level, CoCoOp-TCI is per-class, so the claim that TCI is the key is not established without a per-class random-vector control.","rationale":"The reader's transfer concern (a Meta-Net trained on base-class TCI may not produce useful offsets for unseen class-name embeddings) is legitimate but secondary: on these benchmark splits the new class-name embeddings are drawn from the same vocabulary and the aggregate results suggest the trained Meta-Net does transfer reasonably. The more load-bearing problem is internal to the key empirical argument. To establish that TCI is 'the key', the paper needs conditions that differ only in information content, not in granularity. As written, the FRN/TCI comparison conflates per-task versus per-class conditioning with noise versus text; a per-class random-vector control cleanly separates the two. The paper has independent value: CaPT is simple, the code is released, and the reported computation overhead is small. Those positives do not rescue the causal claim, but they justify a conditional rather than a reject judgment: require the control experiment (and error bars) before accepting the conceptual conclusion. The impossible H=98.98 in Table 2 and the test-set selection of epsilon in Fig. 5 are additional reasons to keep the verdict conditional rather than accept as-is.","tokens_in":22136,"tokens_out":12058,"duration_ms":139058,"concrete_test":"Run the Sec. 3.2 suite with an additional CoCoOp variant conditioned on per-class fixed random vectors (random per-class noise, RPN): assign each class a random vector before training, use it as the condition for that class at both train and inference, with no text or image input; use 5 seeds and report mean and standard deviation of Base/New/H over the same 11 datasets. If RPN's average H is within about 0.3 points of CoCoOp-TCI (76.91), the claim that textual content is key is not supported, and the paper should be revised to claim only that per-class conditioning helps. Also recompute the ProDA Food101 H=98.98 entry in Table 2 from its Base=90.37 and New=89.59 values as a sanity check.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that textual class information (TCI), not visual image information (VII), is the key to overcoming the BNT problem (Sec. 3.2, Fig. 2d). The evidence does not isolate this. The four variants change two axes simultaneously: condition source (fixed noise, image features, text embeddings) and conditioning granularity (one vector for the whole task in FRN/VTI versus one vector per class in VCI/TCI). CoCoOp-FRN uses one fixed noise vector shared by all classes, while CoCoOp-TCI uses a different text vector per class. The H gap from 76.20 (FRN) to 76.91 (TCI) can therefore be explained by class-level conditioning alone, with no semantic text content; CoCoOp-VCI, also per-class, is close to TCI for the same reason. The missing control is a per-class fixed random-vector variant. Without it the phrase 'TCI is the key' overclaims. This is compounded by the absence of error bars despite '3 runs' (Sec. 4.1), by the paper's own admission of failure cases (Sec. 4.2, OxfordPets) while claiming consistent gains, and by an impossible table entry (Table 2, ProDA Food101 H=98.98 exceeds both Base=90.37 and New=89.59), all of which weaken the numeric support for the claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper studies conditional prompt tuning for CLIP. It reports a diagnostic study comparing CoCoOp variants conditioned on fixed random noise, visual task information, visual class information, and textual class information, concluding that textual class information (TCI) is the key to overcoming the base-new tradeoff (BNT). It then proposes Class-adaptive Prompt Tuning (CaPT), which uses a Word2Vec/CLIP-text-encoder to extract per-class embeddings, feeds them through a lightweight Meta-Net to produce prompt offsets, and trains with a margin-adjusted ITM loss. CaPT is evaluated as a plug-in on five unconditional baselines across 11 datasets and three transfer settings, reporting average harmonic-mean improvements of about 2.6 points at negligible computational cost. Combining CaPT with the authors' DePT yields DeCaPT, reported to outperform Bayes by 3.49 average H.","tokens_in":22451,"tokens_out":7128,"duration_ms":74432,"significance":"If the claims hold, CaPT would be a simple, cheap and broadly applicable plugin for improving base-to-new generalization in prompt tuning, and the diagnostic finding about VII versus TCI would redirect the design of conditional prompting. The paper's strengths are its clear exposition, the breadth of experiments (base-to-new, cross-dataset, cross-domain), the low induced overhead, and the public code release. However, the central causal claim is currently under-supported because the comparison confounds conditioning source with per-class granularity, and the quantitative evidence is weakened by test-set hyperparameter selection, missing variance estimates, and a few internally inconsistent table entries.","major_comments":[{"comment":"The experimental design does not isolate the effect of textual semantics from per-class conditioning. CoCoOp-FRN uses one fixed random vector shared by all classes, while CoCoOp-TCI uses a distinct text embedding per class; CoCoOp-VCI also uses a distinct vector per class and reaches H=77.32, close to CoCoOp-TCI's 76.91. A per-class fixed random-vector variant is required to establish that TCI is the key; without it, class-level granularity alone is a plausible explanation. Please add such a control (e.g., CoCoOp-FCN) and report per-variant standard deviations, since all differences are within about one H point.","section":"Sec. 3.2, Fig. 2(d)"},{"comment":"The scaling weight epsilon and the number of training epochs are selected by sweeping on the same 11 evaluation datasets and picking the best average (epsilon=0.31, epochs=10). This is test-set model selection; the reported average gains may be optimistic. Please use a per-dataset validation split or report results with a pre-registered fixed epsilon and epoch count, and quantify sensitivity to these choices.","section":"Sec. 4.3.1, Fig. 5"},{"comment":"The H=98.98 entry for ProDA on Food101 is arithmetically impossible given Base=90.37 and New=89.59, since the harmonic mean always lies between the two accuracies. This suggests a data entry error and undermines confidence in the SOTA comparison table; all entries in Tables 1, 2, A1, and A2 should be rechecked.","section":"Table 2, Food101 ProDA row"},{"comment":"The abstract says CaPT consistently improves all baselines, but the paper admits failure cases (OxfordPets), and Table 1 contains per-dataset drops (e.g., KgCoOp ImageNet H 73.25 to 72.95; DePT OxfordPets H 95.78 to 95.68). Because no standard deviations are reported despite three runs, the 2.6% average gain cannot be assessed for significance. Please report per-run results or confidence intervals and qualify consistent as on average.","section":"Sec. 4.2, Table 1"},{"comment":"The fast adaptation to new classes relies on the assumption that the Meta-Net trained on base-class TCI produces useful offsets for unseen class-name embeddings. This is not guaranteed by the training objective; if new-class TCI lies outside the base-class embedding manifold, the prompt offset becomes an uninformative perturbation. The paper does not characterize this. Please report the distribution of cosine similarities between base and new TCI, or show the per-dataset relationship between H gain and TCI distance, and discuss failure cases like OxfordPets in that light.","section":"Eq. (16)"}],"minor_comments":[{"comment":"The equation writes v_l(µ_base) = v_l + h_α(µ_base) but should refer to p_i^base, not µ_base; this is a notation typo that obscures the definition.","section":"Sec. 3.2, Eq. (9)"},{"comment":"The text contains the typo 'µbaase' and inconsistently uses v_l versus v for the context vectors; please standardize the notation.","section":"Sec. 3.3, Eq. (13)"},{"comment":"The word 'methdos' appears in the first paragraph of Section 2; it should be 'methods'.","section":"Related Work"},{"comment":"The phrase 'an margin-adjusted' should be 'a margin-adjusted'.","section":"Sec. 3.3"},{"comment":"All rows in the '+CaPT' part of Table 6 are labeled '+DePT', which appears to be a copy-paste error from the DePT paper; the labels should read '+CaPT'.","section":"Table 6"},{"comment":"The sentence 'DePT establishes the best results ... when ϵ=0.31' should refer to CaPT, not DePT, since this paragraph is ablating CaPT.","section":"Sec. 4.3.1, Impact of the Scaling Weight"},{"comment":"References [7] and [45] are duplicate entries for the same paper (Zhou et al., IJCV 2022); please consolidate.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper leans heavily on the authors' own DePT; the 3.49-point gain over Bayes is more attributable to the DePT backbone than to CaPT alone (+0.78 H over DePT). The '+DePT' label in Table 6 and other copy-paste artifacts suggest the manuscript was assembled quickly. The 'consistently improves' language in the abstract is stronger than the data support; the editors may want to insist on a more measured claim. I see a publishable contribution after the diagnostic control, hyperparameter selection, and reporting issues are fixed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth a look. The empirical surprise is real: fixed random noise conditioning beats CoCoOp's image conditioning in their setup, and the CaPT plugin is simple, cheap, and gives an average +2.6 H across five baselines. I believe the average gain is real, not a fluke. The paper is clearly written and the code is public, which counts for a lot.\n\nBut the headline claim overreaches. The Sec. 3.2 ablation does not isolate 'textual' from 'per-class.' FRN and VTI are task-level conditions; VCI and TCI are per-class. Without a per-class random-vector control, the improvement from 76.20 (FRN) to 76.91 (TCI) could just be per-class conditioning, with text content doing little. Worse, their own oracle VCI (per-class visual) beats TCI (77.32 vs 76.91), undercutting the 'text beats vision' narrative. The honest claim is that per-class conditioning helps, and TCI is a practical non-oracle way to get it.\n\nThere are also reporting problems. No standard deviations anywhere. 'Consistently improves' is contradicted by per-dataset drops in Table 1. Table 2 has an impossible entry: ProDA Food101 H=98.98 with Base=90.37 and New=89.59; harmonic mean is ~89.98, so that's a typo. The hyperparameter epsilon is tuned on the same 11 datasets used for evaluation (Fig. 5), which is a form of test-set selection. And the DeCaPT comparison leans on partly unpublished baseline numbers plus their own DePT; the 3.49% over Bayes is less clean than advertised.\n\nNone of this kills the paper. The transfer assumption (Meta-Net trained on base TCI, applied to unseen class names) is not stress-tested, but the aggregate evidence across 11 datasets suggests it works. For prompt-tuning researchers, this is a useful incremental contribution with a nice diagnostic. It deserves a real referee round: ask for the per-class random control, error bars, fixed tables, and a held-out hyperparameter protocol. I'd engage with the revision.","headline":"A cheap, plausible prompt-tuning plugin with a genuinely surprising random-noise result, but the central 'text is the key' claim is confounded by per-class versus task-level conditioning and needs a missing control.","tokens_in":23014,"tokens_out":2567,"would_cite":true,"duration_ms":29807,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper argues that prompt tuning for vision-language models should condition dynamic prompts on textual class names rather than visual image information, and that this choice raises base-to-new harmonic accuracy by about 2.6 points on…","keywords":["Prompt tuning","Conditional prompt tuning","Vision-language models","Few-shot learning","Base-new tradeoff","Class-adaptive prompts","Textual class information","CLIP"],"falsifier":"Train CaPT on a set of base classes, then evaluate on new classes whose name embeddings have been deliberately pushed far from every base-class embedding, for example by adding large random perturbations to the class-name vectors. If harmonic accuracy collapses to the level of the same prompt without CaPT, the claim that textual class conditioning drives the gain is falsified. A cheaper correlational check is to plot each dataset's CaPT gain against the average cosine distance between that dataset's new-class embeddings and the base-class embedding centroid; the paper's mechanism predicts a clear negative relationship.","tokens_in":21929,"feed_emoji":"🏷️","tokens_out":7318,"duration_ms":80280,"temperature":0.7,"pith_summary":"Prompt tuning adapts CLIP-style vision-language models to a task by learning a few prompt vectors, but tuned prompts tend to overfit base classes and lose accuracy on new classes. The paper identifies this base-new tradeoff and questions the usual remedy of conditioning prompts on per-image visual features. It reports that prompts conditioned on fixed random noise outperform the image-conditioned version, while prompts conditioned on class-name text embeddings match the best class-level visual oracle. The proposed CaPT method learns class-adaptive prompt offsets from frozen textual class embeddings through a small Meta-Net, plus a margin-adjusted loss that stops similar base classes from being pushed too far apart. Plugged into five existing unconditional prompt-tuning methods, CaPT raises average harmonic base-to-new accuracy by about 2.6 points with roughly 0.9 per-mille extra training time, and combining it with DePT sets the best reported harmonic mean on the 11 datasets.","feed_headline":"Class names, not images, should condition vision-language prompts","feed_subtitle":"Conditioning prompts on class-name text lifts base-to-new harmonic accuracy by about 2.6 points on 11 datasets.","key_machinery":"The load-bearing object is the class-conditional prompt $c_i(e_i) = \\{v_1 + h_\\alpha(e_i), \\ldots, v_l + h_\\alpha(e_i), [CLS]_i\\}$, where $e_i = \\varphi(\\text{class name})$ is a frozen textual class embedding, $h_\\alpha$ is a two-layer MLP Meta-Net, and the $v$'s are the learned context vectors. The offset $h_\\alpha(e_i)$ lets a prompt trained on base classes generate modified prompts for unseen new classes at inference, because class names are always available. The margin-ITM loss uses pairwise cosine similarities among base-class text embeddings to subtract a class-similarity margin from the classification logits, keeping similar classes from being pushed too far apart in the learned feature space.","core_discovery":"The paper's controlled comparison replaces the image-conditioning in CoCoOp with four alternatives: fixed random noise, visual task prototypes, visual class prototypes, and textual class embeddings. It finds that image-conditioning is not the source of the new-class generalization gain; even random noise does no worse, class-level visual information helps more, and textual class information matches the oracle class-level visual variant. The central claim is therefore that dynamic prompts conditioned on Textual Class Information (TCI) are the key to overcoming the Base-New Tradeoff in prompt tuning. CaPT operationalizes this by passing frozen class-name embeddings through a lightweight MLP Meta-Net that outputs class-adaptive offsets to learned prompt vectors, and a margin-adjusted Image-Text Matching loss that prevents over-separation of semantically similar base classes.","pith_inferences":["If class-name text is what drives generalization, per-image conditioning in existing conditional prompt-tuning methods could likely be dropped entirely at inference, saving compute without hurting accuracy; the paper does not make this claim directly.","The mechanism predicts a testable correlation: CaPT's gain on a new class should shrink as that class's name embedding moves farther from the base-class embedding distribution, a prediction the paper does not examine.","The margin-ITM principle, using semantic similarity to avoid over-separation of classes, may transfer beyond prompts to other parameter-efficient tuning techniques such as adapters or low-rank updates, though the paper only tests it for prompt tuning.","Because the Meta-Net is trained only on base-class name embeddings, the method's behavior on class vocabularies far outside that distribution, such as specialized medical or technical terms, remains an open untested boundary."],"forward_implications":["Adding CaPT to CoOp raises average harmonic base-to-new accuracy from 74.84 to 77.44, and the same plugin lifts four other unconditional baselines by between 0.78 and 3.89 H points.","Because new-task class names are available at inference, the same learned Meta-Net can adapt a base-trained prompt to unseen classes without seeing any of their images.","Integrating CaPT with DePT yields DeCaPT, which reaches 79.63 average H on 11 datasets, 3.49 points above the Bayes conditional prompt-tuning scheme.","CaPT's gains persist across 1, 2, 4, 8 and 16 shots, and in the 1-shot setting it is the only method tested that stays above CLIP's zero-shot average accuracy.","The plugin adds only about 0.9 per-mille extra training time when applied to the five tested baselines, making the improvement nearly computation-free."],"supporting_citations":[{"why":"DePT is the unconditional prompt-tuning framework that CaPT is plugged into and whose combination with CaPT, DeCaPT, produces the paper's best results.","marker":"[1]"},{"why":"CLIP is the frozen vision-language backbone, and its text encoder is used as the TCI extractor defining the feature space in which prompts act.","marker":"[2]"},{"why":"Bayes is the state-of-the-art conditional prompt-tuning baseline that DeCaPT is compared against and outperforms by 3.49 average H points.","marker":"[4]"},{"why":"CoCoOp supplies the conditional prompt-tuning baseline whose image condition is replaced by random noise, visual task, visual class, and textual class conditions in the diagnostic motivating CaPT.","marker":"[6]"},{"why":"CoOp is the canonical unconditional baseline that CaPT is first applied to, lifting average H from 74.84 to 77.44.","marker":"[45]"}],"fun_headline_variants":["Text-conditioned prompts beat image-conditioned for VLM tuning","For prompt tuning, class names outperform image conditioning","CaPT: class-adaptive prompts lift base-new harmonic accuracy by 2.6 points","Condition prompts on class text, not images, to fix base-new tradeoff","Image-conditioned prompts suboptimal; use textual class info instead"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The Meta-Net is assumed to turn embeddings of class names it has never seen into useful prompt adjustments; if a new class's name embedding sits far outside the range of the base classes, the conditioning signal becomes indistinguishable from noise.","fun_headline_variants_meta":{"raw":{"variants":["Text-conditioned prompts beat image-conditioned for VLM tuning","For prompt tuning, class names outperform image conditioning","CaPT: class-adaptive prompts lift base-new harmonic accuracy by 2.6 points","Condition prompts on class text, not images, to fix base-new tradeoff","Image-conditioned prompts suboptimal; use textual class info instead"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000439,"raw_usage":{"total_tokens":2261,"prompt_tokens":1011,"completion_tokens":1250,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":627,"completion_tokens_details":{"reasoning_tokens":1158}},"tokens_in":627,"tokens_out":1250,"duration_ms":12123,"temperature":1.0,"reasoning_tokens":1158,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T21:30:42.319640+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train CaPT on a set of base classes, then evaluate on new classes whose name embeddings have been deliberately pushed far from every base-class embedding, for example by adding large random perturbations to the class-name vectors. If harmonic accuracy collapses to the level of the same prompt without CaPT, the claim that textual class conditioning drives the gain is falsified. A cheaper correlational check is to plot each dataset's CaPT gain against the average cosine distance between that dataset's new-class embeddings and the base-class embedding centroid; the paper's mechanism predicts a clear negative relationship.","supporting_citations":[{"cited_title":"Dept: Decoupled prompt tuning","cited_arxiv_id":null,"evidence_quote":"DePT is the unconditional prompt-tuning framework that CaPT is plugged into and whose combination with CaPT, DeCaPT, produces the paper's best results."},{"cited_title":"Learning trans- ferable visual models from natural language supervision","cited_arxiv_id":null,"evidence_quote":"CLIP is the frozen vision-language backbone, and its text encoder is used as the TCI extractor defining the feature space in which prompts act."},{"cited_title":"Bayesian prompt learning for image- language model generalization","cited_arxiv_id":null,"evidence_quote":"Bayes is the state-of-the-art conditional prompt-tuning baseline that DeCaPT is compared against and outperforms by 3.49 average H points."},{"cited_title":"Conditional prompt learning for vision-language models","cited_arxiv_id":null,"evidence_quote":"CoCoOp supplies the conditional prompt-tuning baseline whose image condition is replaced by random noise, visual task, visual class, and textual class conditions in the diagnostic motivating CaPT."},{"cited_title":"Learn- ing to prompt for vision-language models","cited_arxiv_id":null,"evidence_quote":"CoOp is the canonical unconditional baseline that CaPT is first applied to, lifting average H from 74.84 to 77.44."}],"review_version":1}