{"id":"ae2324e2-1e1b-448e-a631-fb9e995e0a79","arxiv_id":"2412.17306","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A consistency-guided test-time prompt adaptation method improves CLAP zero-shot audio classification by 4.41% relative on average over DA CLAP across 12 datasets.","lead":"This paper proposes a test-time adaptation method for audio-language models that tunes text prompts using only unlabeled audio, combining augmented-view consistency with cross-sample contrastive learning. On 12 audio classification benchmarks it reports an average 4.41% relative accuracy gain over the prior DA CLAP baseline.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Internal inconsistency in reported per-dataset gains: the SESA/RAVD numbers in Section IV.B do not match Table I, so the 4.41% average is not yet supported by a self-consistent record.","rationale":"The reader's weakest assumption about entropy minimization reinforcing confident errors is a real theoretical limitation of unsupervised TTA, and the paper does not diagnose confident-but-wrong predictions. However, I do not think that is the single most load-bearing issue here, because the method's contrastive loss and the reported ablations partially mitigate the collapse concern, and the empirical question is whether the numbers are right. The more immediate problem is that the paper's own text and Table I disagree on the very examples used to illustrate improvement. SESA is listed as starting from 73.40 in the text but 74.35 in Table I; RAVD is claimed to jump to 43.71 in the text but Table I reports only 33.77. One of these must be a transcription or logging error, and without code or raw outputs there is no way to know which. Since the average 68.83 is computed from Table I, and Table I is internally coherent, the average could still be exactly as reported; but the text/table mismatch undermines confidence in the whole evaluation pipeline. The correct response is to keep the conditional verdict and require the authors to supply a self-consistent per-dataset result table, ideally with code or logs. My recommendation is therefore UNCHANGED relative to the reader's conditional verdict: the central claim is plausible but not yet robustly supported, and the specific SESA/RAVD inconsistency is a concrete, checkable defect that should be fixed before acceptance.","tokens_in":9938,"tokens_out":10841,"duration_ms":102899,"concrete_test":"Recompute the per-dataset test accuracies for RAVD and SESA using the exact protocol described in Section IV.A (CLAP with HTSAT audio encoder and modified GPT2 text encoder, 50 augmentations per raw audio, AdamW lr=1e-6, batch size 5, final loss L_consistency + lambda_contrastive * L_contrastive) and compare the outputs against both Section IV.B and Table I. If RAVD yields 33.77 rather than 43.71, the narrative is incorrect and the 68.83 average should be recomputed from corrected logs; if it yields 43.71, Table I and the 4.41% claim need revision. The same check should be run for SESA to resolve the 73.40/74.35 discrepancy.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is the average improvement from 65.92 (DA CLAP five) to 68.83 (ours) in Table I, which drives the abstract's 4.41% relative gain. But the paper's own narrative contradicts its results table for the two datasets it highlights. Section IV.B says SESA 'improved from 73.40% (DA five) to 76.19%', while Table I lists DA five as 74.35 for SESA. It also says RAVD improved 'from 34.45% to 43.71%', while Table I lists DA five as 33.19 and ours as 33.77 for RAVD. These cannot both be correct. If Table I is the source of the average, the text examples are wrong; if the text values are correct, Table I and the computed 68.83 average are wrong. No code, raw logs, or error bars are provided, so an external reader cannot determine which set of numbers is trustworthy. This is load-bearing because the headline improvement is an arithmetic summary of exactly those per-dataset accuracies; an uncorrected inconsistency in the evidence leaves the central claim unsupported rather than merely imperfectly presented.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a test-time adaptation method for contrastive audio-language models (CLAP) that tunes learnable prompts on unlabeled test audio. Two lightweight MLP networks generate context-aware and domain-aware prompt tokens conditioned on the audio embedding; the prompts are optimized by minimizing the self-entropy of the averaged class distribution over multiple augmented views, plus a contrastive loss that penalizes overly similar predicted distributions across different test samples. The method is evaluated on 12 audio classification datasets, reporting an average accuracy of 68.83% versus 65.92% for the DA CLAP (five) baseline, which corresponds to the 4.41% relative improvement cited in the abstract. Ablations show that both prompt networks and both loss terms contribute to the final accuracy.","tokens_in":10233,"tokens_out":3668,"duration_ms":36211,"significance":"If the reported results are reliable, the paper makes a useful incremental contribution to test-time prompt adaptation for audio-language models, extending the DA CLAP framework with additional consistency and contrastive guidance. The ablations (Table III) showing that both loss terms and both prompt networks contribute are a strength, as is the cross-domain generalization experiment in Table II. However, the central quantitative claim is currently undermined by internal numerical inconsistencies between the text and Table I, and the lack of repeated-seed experiments or error bars makes it impossible to assess whether the reported gains are statistically meaningful. The method is simple and potentially reproducible, which is a positive feature, but the evidence as presented is not yet convincing enough to support the headline improvement.","major_comments":[{"comment":"The text in Section IV.B gives per-dataset gains that do not match Table I. It states that SESA improved from 73.40% (DA five) to 76.19%, and RAVD from 34.45% to 43.71%, but Table I lists DA five for SESA as 74.35 and for RAVD as 33.19, with the proposed method achieving 76.19% on SESA and only 33.77% on RAVD. These two records cannot both be correct. Since the abstract's 4.41% average improvement is an arithmetic summary of the Table I per-dataset accuracies, this inconsistency is load-bearing: the current paper leaves the reader unable to determine which numbers are trustworthy, and the central claim is therefore unsupported until corrected.","section":"Section IV.B and Table I"},{"comment":"No repeated-seed experiments, error bars, or statistical significance tests are reported. Many of the claimed gains over DA CLAP are small relative to typical run-to-run variance on such benchmarks (e.g., ESC50 decreases from 95.05 to 94.65; D17T4 decreases from 52.30 to 51.69; US8K improves by 1.39 percentage points). Without multiple runs, the central claim that the method improves over the state of the art across 12 datasets is not statistically substantiated.","section":"Section IV.A, experimental setup"},{"comment":"The paper does not describe a validation split or a procedure for selecting hyperparameters such as lambda_contrastive, the learning rate, the number of augmented views M, or the number of MLP layers. The reported test-set accuracies appear to be the result of choosing these hyperparameters on the same test sets used for evaluation. This introduces a selection-bias risk, especially because the ablations in Table III show sensitivity to architecture choices (e.g., 1-layer vs. 4-layer MLP) and loss weighting is not ablated at all.","section":"Section IV.A.2 and Appendix, Table III"},{"comment":"The paper motivates the method by stating that previous test-time adaptation methods 'tend to be stuck in incorrect model predictions,' but the proposed L_consistency is exactly a self-entropy minimization loss on the averaged distribution. If the model is confidently wrong for a test sample, that loss can reinforce the wrong prediction. The paper provides no diagnostic, analysis, or comparison case study showing how the added contrastive loss prevents this failure mode. As the main unsupervised signal, this assumption is load-bearing for the claimed advantage over prior methods and should be examined empirically.","section":"Section III.C.2 and Introduction"}],"minor_comments":[{"comment":"The phrase 'end-end' appears repeatedly (e.g., in the abstract and Section III.C) and should be 'end-to-end'.","section":"Throughout"},{"comment":"The sentence 'our method achieves better (+9.38%) zero-shot classification performance than directly testing the time domain prompt tuning (+5.99%), with a batch size of 5 (+4.41%)' is ambiguous and the percentages do not align with Table I; please clarify which baselines and which formula (relative vs. absolute) are being used.","section":"Section IV.B"},{"comment":"Figure 1 lists four augmentations (Time Masking, Frequency Masking, Time & Frequency Masking, Time Reorder), while Section IV.A.1 states that 'We augment each raw audio 50 times.' Please clarify how the four augmentation types are expanded to 50 augmented views.","section":"Figure 1 and Section III.C.1"},{"comment":"Table II is described as training on one target domain and testing on others, but the proposed method is test-time adaptation without training on a target domain. Please clarify the protocol used to produce Table II.","section":"Section IV.D, Table II"},{"comment":"Several column headers contain stray spaces ('RA VD', 'V ocal', 'TU17'); these should be cleaned. Also, 'DA CLAP (five [19])' and similar labels are inconsistent in formatting.","section":"Table I"},{"comment":"The paper compares only against DA CLAP [19] and zero-shot CLAP; no comparison is made to other test-time prompt tuning methods for audio or to the vision-language TPT method [18] that the approach builds on, which would help position the contribution.","section":"References and related work"}],"recommendation":"major_revision","confidential_remarks":"The internal inconsistency between Section IV.B and Table I is the main blocker; it is the kind of issue that a careful reader will immediately spot and that undermines confidence in the reported averages. The paper also lacks any uncertainty quantification in a regime where many reported differences are small. These issues are fixable with corrected numbers, repeated runs, and a proper validation protocol, so I do not recommend rejection, but the current version is not publishable as is. I would also suggest the editor ask for code or at least detailed per-run logs to resolve which numbers are authoritative."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper reports a 4.41% relative average gain over DA CLAP on 12 audio classification datasets, but its own narrative contradicts the table the average is computed from, so the headline number is not yet self-consistently supported.\n\nWhat is new: the authors add a conditional context-aware prompt network and a cross-sample MSE contrastive loss to DA CLAP's consistency-based test-time prompt tuning. Both additions are modest extensions of existing ideas, but the combination is not in the cited prior work. The ablations are a genuine strength: removing either prompt network or either loss degrades the average accuracy in a way that matches the story, and the cross-domain generalization table gives a fuller picture than most TTA papers bother to provide. The writing is clear and the experimental setup is mostly standard.\n\nThe soft spots, in order. First, the internal inconsistency: Section IV.B says SESA improved from 73.40% (DA five) to 76.19% (ours), while Table I lists DA five at 74.35 for SESA; it says RAVD improved from 34.45% to 43.71%, while Table I lists DA five at 33.19 and ours at 33.77. These cannot both be correct. Since the abstract's 4.41% is an arithmetic summary of Table I, the evidence for the central claim is not self-consistent. Second, there are no error bars or repeated seeds, no code, and the hyperparameter lambda_contrastive is never specified, so an independent reader cannot check or reproduce the results. Third, the main unsupervised signal is self-entropy minimization of the averaged distribution over augmented views; the paper doesn't provide a diagnostic for confident-but-wrong predictions, which is the failure mode it attributes to earlier methods. That concern is real but secondary, because the empirical average still ends up positive.\n\nWho should read this: anyone working on test-time adaptation for audio-language models, especially CLAP-based systems. The paper deserves a serious referee: the idea is relevant, the ablations are useful, and the inconsistency looks fixable rather than fatal. My recommendation is to send it to review with a request for corrected numbers, error bars or seeds, and the missing hyperparameter values. If those come back clean, the 4.41% claim would be credible.\n\nThat's my take.","headline":"Incremental but well-ablated TTA method for CLAP; the reported gains are undermined by an internal inconsistency in the highlighted per-dataset numbers.","tokens_in":10713,"tokens_out":3215,"would_cite":false,"duration_ms":28771,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Test-time prompt adaptation with no labels lifts zero-shot audio classification accuracy by 4.41% on average across 12 datasets.","keywords":["test-time adaptation","contrastive audio-language models","prompt learning","self-entropy minimization","consistency regularization","contrastive learning","zero-shot audio classification","SpecAugment"],"falsifier":"On a set of test clips where the frozen CLAP model is known to be confidently wrong (for example, heavily corrupted or adversarial audio), run the adaptation and compare per-sample confidence before and after; if confidence rises while accuracy stays flat or falls, the consistency signal is reinforcing wrong predictions rather than correcting them.","tokens_in":9794,"feed_emoji":"🎧","tokens_out":6327,"duration_ms":53459,"temperature":0.7,"pith_summary":"Contrastive audio-language models such as CLAP can classify sounds zero-shot, but their accuracy depends on the text prompt and drops on unfamiliar domains. This paper claims that the model can adapt to unlabeled test audio at inference time by learning, for each test clip, two sets of prompt tokens—context tokens inserted into the class prompt and domain tokens prepended to it—driven only by unsupervised signals. The signals are minimum self-entropy of the model's averaged prediction across multiple augmented versions of the clip, plus a contrastive penalty that keeps different test clips' predictions distinct. On 12 audio classification benchmarks the method reports an average accuracy of 68.83%, up from 65.92% for the best previous test-time adapter and 62.93% for zero-shot CLAP. If the result holds, deployed audio classifiers can be adapted to new conditions without collecting labels.","feed_headline":"Unlabeled audio alone lifts zero-shot audio accuracy 4.41%","feed_subtitle":"Consistency-guided prompt tuning on 12 benchmarks beats the prior label-free adapter.","key_machinery":"The central mechanism is the conditional test-time prompt, split into a context-aware branch and a domain-aware branch. Each branch is a three-layer MLP ('c-net' and 'd-net') that takes the audio embedding of augmented views and outputs learnable tokens; the tokens are inserted into the CLAP text prompt (context appended, domain prepended) before the frozen text encoder produces class features. The adaptation signal is the final loss $L_{final}=L_{consistency}+\\lambda_{contrastive}L_{contrastive}$: the consistency term is the self-entropy of the averaged softmax distribution over augmented views, and the contrastive term is the negative sum of MSE between distributions of different test samples, encouraging diversity. Augmented views come from SpecAugment-style time masking, frequency masking, combined masking, and a time-reorder operation applied to the mel spectrogram. The average distribution $g_{avg}$ is the bridge that lets one unlabeled clip over 50 augmentations train both prompt networks in an end-to-end fashion.","core_discovery":"The paper's central claim is that two complementary prompt perturbations, together with a consistency objective computed over augmented views, let a frozen contrastive audio-language model adapt to a test-time domain shift without any labeled audio. The context-aware prompt ($p_{context}$) is generated by a lightweight network conditioned on the audio embedding and appended into the text prompt, while the domain-aware prompt ($p_{domain}$) is generated by a second network and prepended; both are updated per test sample. The unsupervised objective combines the self-entropy of the average class distribution over $M$ augmented views, $L_{consistency}=-g_{avg}\\log g_{avg}$, with a contrastive loss $L_{contrastive}=-\\sum_{k_1\\neq k_2}\\mathrm{MSE}(g_{k_1},g_{k_2})$ that penalizes different test samples from producing the same distribution. The reported result is a 4.41% average accuracy improvement (maximum 7.50%) over the strongest prior adapter across 12 datasets, with ablations showing that both losses and both prompt networks contribute to the gain.","pith_inferences":["A natural extension, not tested in the paper, is to use the same consistency-plus-diversity objective for other contrastive audio-language tasks such as retrieval or captioning, where the prompt conditions the output format rather than a class list.","The success of the method depends on augmentations that preserve the semantic content of the audio; if time reorder or masking ever destroys the distinguishing event, the averaged distribution could become confidently wrong, so a future diagnostic could measure augmentation sensitivity per dataset.","An abstention or confidence-threshold mechanism could combine with this adapter to flag test clips where the consistency signal is reinforcing an initial wrong prediction; the paper does not evaluate such a safeguard.","The contrastive term currently operates on a batch of test samples; in a streaming deployment the same effect could be achieved with a small memory bank of recent predictions, which the paper does not discuss."],"forward_implications":["On the 12 datasets evaluated, the method raises average zero-shot accuracy from 65.92% (DA CLAP with five test samples) to 68.83% using no labels.","Both loss terms matter: removing the contrastive loss drops the average to 64.42%, and removing the self-entropy term drops it to 67.75%, so the consistency and the diversity signals are complementary rather than redundant.","The two prompt branches are complementary: the full method reaches 68.83%, while the domain-only and context-only variants reach 67.51% and 66.96% respectively.","Cross-domain experiments show the adapted prompts transfer to other datasets better than the DA CLAP baseline under most train/test combinations and beat zero-shot CLAP on average for every training domain.","The gain is not an artifact of added parameters: doubling the MLP width of either branch produces lower accuracy than combining the two branches."],"supporting_citations":[{"why":"Supplies the pretrained contrastive audio-language model (CLAP) whose zero-shot performance is the starting point for adaptation.","marker":"[1]"},{"why":"Defines the domain-adaptation baseline (DA CLAP), the test-time prompt-tuning setting, the augmentation scheme, and the comparison numbers.","marker":"[19]"},{"why":"Provides the SpecAugment masking operations used to generate the multiple augmented views of each test clip.","marker":"[20]"},{"why":"Source of the contrastive learning loss that penalizes similar distributions across different test samples.","marker":"[21]"},{"why":"Introduces context optimization (CoOp), the basis for the context-aware prompt branch.","marker":"[8]"},{"why":"Introduces conditional prompt tuning (CoCoOp) that conditions prompts on input features, the basis for the c-net.","marker":"[9]"},{"why":"Establishes test-time prompt tuning by entropy minimization, the baseline mechanism the paper extends.","marker":"[18]"}],"fun_headline_variants":["Consistency-tuned prompts boost zero-shot audio by 4.41%","Zero-shot audio: 4.41% gain from unlabeled consistency tuning","Test-time prompt consistency lifts audio zero-shot by 4.41%","No labels, 4.41% more accuracy: consistency-guided audio TTA","On-the-fly audio adaptation: +4.41% zero-shot, no labels"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that minimizing the self-entropy of the averaged distribution over augmented views of an unlabeled test clip pushes the prompts toward correct predictions; if the model starts out confidently wrong, that same pressure can lock the error in.","fun_headline_variants_meta":{"raw":{"variants":["Consistency-tuned prompts boost zero-shot audio by 4.41%","Zero-shot audio: 4.41% gain from unlabeled consistency tuning","Test-time prompt consistency lifts audio zero-shot by 4.41%","No labels, 4.41% more accuracy: consistency-guided audio TTA","On-the-fly audio adaptation: +4.41% zero-shot, no labels"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000874,"raw_usage":{"total_tokens":3777,"prompt_tokens":935,"completion_tokens":2842,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":551,"completion_tokens_details":{"reasoning_tokens":2739}},"tokens_in":551,"tokens_out":2842,"duration_ms":17071,"temperature":1.0,"reasoning_tokens":2739,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T05:36:00.692989+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a set of test clips where the frozen CLAP model is known to be confidently wrong (for example, heavily corrupted or adversarial audio), run the adaptation and compare per-sample confidence before and after; if confidence rises while accuracy stays flat or falls, the consistency signal is reinforcing wrong predictions rather than correcting them.","supporting_citations":[{"cited_title":"Specaugment: A simple data augmenta- tion method for automatic speech recognition,","cited_arxiv_id":null,"evidence_quote":"Provides the SpecAugment masking operations used to generate the multiple augmented views of each test clip."},{"cited_title":"Mixstyle-based contrastive test- time adaptation: Pathway to domain generalization,","cited_arxiv_id":null,"evidence_quote":"Source of the contrastive learning loss that penalizes similar distributions across different test samples."},{"cited_title":"Learning to prompt for vision-language models,","cited_arxiv_id":null,"evidence_quote":"Introduces context optimization (CoOp), the basis for the context-aware prompt branch."},{"cited_title":"Conditional prompt learning for vision-language models,","cited_arxiv_id":null,"evidence_quote":"Introduces conditional prompt tuning (CoCoOp) that conditions prompts on input features, the basis for the c-net."},{"cited_title":"Test-time prompt tuning for zero-shot gen- eralization in vision-language models,","cited_arxiv_id":null,"evidence_quote":"Establishes test-time prompt tuning by entropy minimization, the baseline mechanism the paper extends."}],"review_version":1}