{"id":"3a916b49-d4e6-47a1-9c32-b5c867bd8e03","arxiv_id":"2412.09718","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Applying variational Bayesian inference to the CLAP linear probe adapter improves calibration and high-confidence coverage in CLIP few-shot classification, with a modest accuracy trade-off.","lead":"This paper proposes BayesAdapter, a Bayesian variant of a CLIP adapter that estimates a distribution over the adapter weights instead of a single point. It reports better confidence calibration and selective classification coverage across 11 datasets, at a small accuracy cost.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 2's headline coverage gain for BayesAdapter (13.423% at 99% confidence) is inconsistent with Table 4's 9.213% for the same configuration, making the central coverage claim internally unsupported.","rationale":"The paper's central contribution is that BayesAdapter improves uncertainty estimates, specifically calibration (ECE) and selective classification coverage at high confidence. The calibration numbers are consistent: Table 1 shows ECE 4.151 vs CLAP 6.592, and Table 4 for the same configuration shows ECE 4.168, close enough given rounding and seed variation. Coverage, however, is internally inconsistent: Table 2 reports 13.423% at 99% confidence for the main configuration, while Table 4 reports 9.213% for the same configuration (prior std = 0.01, sMC = 3), both described as aggregated over 11 datasets, 6 shots, and 3 seeds on ResNet-50. This is a discrepancy of 4.2 percentage points, larger than the reported gain over CLAP's 7.103%. Reconstructing the aggregate from the per-dataset table does not yield 13.423 under standard rules, suggesting an error in the table or a non-standard aggregation that is not documented. This is more load-bearing than the reader's prior-std concern because it does not require re-running experiments; it can be checked analytically from the paper's own numbers. If the true coverage is ~9.2, the coverage gain over CLAP is only ~2.1 points, weakening the headline claim of a '6-9% gain' in the abstract. The verdict remains conditional: the method may still work, but the authors must correct the coverage aggregation, clarify the averaging rule, and release code to verify the results. I also note the reader's concern about prior-std selection is legitimate, but the internal inconsistency is the first issue to resolve.","tokens_in":85547,"tokens_out":13510,"duration_ms":118896,"concrete_test":"Recompute the aggregated 99%-confidence coverage for BayesAdapter and CLAP from the per-dataset/per-shot entries in Table 11, using a clearly stated rule (e.g., treat ✗ as 0; average only over reliable entries; or average per-shot means). Check whether any single rule reproduces Table 2's 13.423% for BayesAdapter and 7.103% for CLAP, and whether the same rule applied to Table 4's configuration reproduces 9.213%. If no rule reproduces both, the headline coverage gain is unsupported by the paper's own data.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.2.2 reports in Table 2 that BayesAdapter achieves 13.423% test-set coverage at 99% confidence on ResNet-50 (aggregated over 11 datasets, 6 shot levels, 3 seeds), versus 7.103% for CLAP, a ~6.3-point gain highlighted in the abstract. However, Table 4, which is described as aggregated over the same datasets, shots, and seeds, reports Cov@99% = 9.213 for the exact configuration used in the main experiments (prior std = 0.01, sMC = 3). These two values cannot both be correct under the same aggregation rule. Attempting to reproduce 13.423 from the per-dataset numbers in Table 11 (treating ✗ as 0, or averaging only reliable entries, or per-shot means) yields values around 11.6 or 16.0, not 13.423. If the true coverage is instead ~9.2, the coverage advantage over CLAP shrinks to ~2.1 points, materially weakening the paper's central claim. The reader's concern about prior-standard-deviation selection is also real, but the internal inconsistency is more directly load-bearing because it affects the reported effect size without any re-running.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies uncertainty estimation in few-shot adaptation of CLIP. It observes that CLAP, a state-of-the-art deterministic adapter, can be viewed as MAP inference in a probabilistic model with a Gaussian prior and a multinomial likelihood, and proposes BayesAdapter, a variational Bayes extension that learns a Gaussian posterior over the adapter weights instead of a point estimate. The empirical evaluation covers 11 datasets, two backbones, six shot levels, and three seeds, and reports that BayesAdapter improves calibration and selective-classification coverage over CLAP and six other adapters while remaining competitive in accuracy. The supplementary material includes full per-dataset tables with standard errors, ablations, and training-time comparisons.","tokens_in":85775,"tokens_out":9089,"duration_ms":83142,"significance":"BayesAdapter is conceptually simple and plausible: replacing a point estimate with a variational posterior for a linear adapter is a natural step, and Proposition 1, once the covariance convention in Eq. (3) is corrected, gives a clean probabilistic interpretation of CLAP. The paper's strongest assets are the breadth of the evaluation, the availability of detailed per-dataset tables, and the inclusion of standard errors and ablations. The calibration gains (ECE 4.151 vs 6.592 for CLAP on ResNet-50) are supported by the reported numbers. However, the headline selective-coverage advantage is internally inconsistent across Table 2, Table 4, and Appendix Table 11, and the claimed validation-free protocol is compromised by choosing the prior standard deviation based on the evaluation datasets. These issues are load-bearing for the paper's central claims and must be resolved before the empirical conclusions can be taken at face value.","major_comments":[{"comment":"Table 2 reports 13.423% test-set coverage at 99% confidence for BayesAdapter on ResNet-50, but Table 4 reports Cov@99%=9.213 for the same configuration (prior std=0.01, sMC=3), with both tables described as aggregated over the same datasets, shot levels, and seeds. Recomputing from Appendix Table 11 with ✗ treated as zero gives approximately 11.6%, which matches neither value. The corresponding CLAP entries are also inconsistent: Table 11 averages to about 6.6%, not the 7.103% reported in Table 2. Since the 6.3-point coverage gain over CLAP is a central contribution, the averaging rule must be clarified and the tables corrected; if the correct value is around 9.2%, the advantage over CLAP shrinks to roughly 2.1 points, materially weakening the claim.","section":"§4.2.2, Tables 2 and 4"},{"comment":"The text states that BayesAdapter 'is reliable at all confidence levels,' but Table 11 contains many ✗ entries for BayesAdapter, for example EuroSAT, ImageNet, and SUN397 at 1 shot. Either the statement refers to an aggregate notion of reliability that is not defined, or it is contradicted by the per-configuration table. This ambiguity also affects how the numbers in Table 2 should be interpreted, and a precise definition of how ✗ entries enter any average is needed.","section":"§4.2.2 and Appendix Table 11"},{"comment":"The prior standard deviation is chosen 'based on the average empirical performance of the ZS classification' on the same 11 benchmark datasets and then fixed. This constitutes model selection on the evaluation data and is at odds with the stated validation-free protocol in Section 4.1, which says no validation or test samples are accessible. Since Table 4 shows that coverage at 99% confidence ranges from 3.062% to 12.043% across prior standard deviations, the reported gains are not robustly decoupled from this hyperparameter choice. I ask the authors to select σ_p without reference to test performance, for example from the few-shot support set or from a fixed a-priori value, or to explicitly acknowledge and quantify the resulting optimism.","section":"§4.1 and §4.2.4, Table 4"},{"comment":"The prior covariance written in Eq. (3) is Λ=2·diag(λ_1^{-1},...,λ_C^{-1}) with each entry repeated D times. With this convention, log p(W) equals -Σ_c (λ_c/4)||w_c-t_c||^2, not the CLAP regularizer -Σ_c λ_c||w_c-t_c||^2. The proof in Appendix A.1 uses Λ=0.5·diag(...), which is the correct expression for the stated equivalence. Eq. (3) should be corrected to match the appendix, or the notation should be changed to an inverse-covariance parameterization; otherwise Proposition 1 does not follow from the model as defined.","section":"Eq. (3) and Proposition 1"}],"minor_comments":[{"comment":"There are several typos, including 'apdaters' in Section 4.1, 'probablity' in Proposition 1, and 'Worskshops' in reference [11].","section":"Throughout"},{"comment":"Figure 3 states that CLAP obtains 0.00% coverage on EuroSAT with 4 shots, while Table 11 reports 0.03±0.02 for CLAP at that configuration; please clarify whether the figure is from a single seed and state which seed.","section":"Figure 3 and Table 11"},{"comment":"The caption says results are 'similar to Table 1', but the numbers for the common configuration differ slightly from Table 1, for example accuracy 69.437 vs 69.476; please specify whether Table 4 comes from separate runs and report standard errors.","section":"Table 4 caption"},{"comment":"The linear KL annealing schedule is mentioned but its setting is not detailed, and its sensitivity is not studied; given the paper's emphasis on validation-free adaptation, a sentence on how the schedule was chosen would help.","section":"§4.1"},{"comment":"The abstract's '∼6-9% gain in test set coverage at 99% confidence' is not directly traceable to a single table because the ResNet-50 and ViT-16 values differ; consider reporting the range explicitly in the abstract.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":"The discrepancy between Table 2 and Table 4/Table 11 is serious and cannot be resolved from the text; the authors should be asked to provide the exact aggregation code or corrected tables. The prior-variance selection is also a benchmark-leakage concern. I am sympathetic to the conceptual contribution and would be willing to re-review once these issues are addressed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the thing to know before reading: the abstract's headline gain in selective coverage (13.423% vs 7.103% at 99% confidence, ResNet-50) doesn't match the paper's own ablation table. Table 4 reports 9.213% for the identical configuration (sMC=3, prior std=0.01, same aggregation rule). That's a six-point swing in the central claim, and no caption or footnote reconciles it. If the correct number is ~9.2, the coverage advantage over CLAP shrinks from ~6.3 points to ~2.1 points. The calibration results and the method itself still stand, but the paper's own evidence doesn't support the headline effect size.\n\nWhat's genuinely new: the observation that CLAP is MAP inference of a Gaussian-prior linear model is a clean recasting, and swapping the point estimate for a variational Gaussian posterior over adapter weights is a sensible, parameter-efficient idea that I haven't seen applied to CLIP adapters before. The empirical study is substantial: 11 datasets, two backbones, six shot levels, three seeds, with standard errors in the appendix. The finding that accuracy ranking and calibration ranking diverge among adapters is useful and worth knowing. The method preserves black-box adaptation, since it doesn't backprop through CLIP.\n\nSoft spots: the coverage inconsistency is load-bearing. It directly affects the abstract and the paper's main selling point. Also, Section 4.1 says the prior standard deviation was chosen 'based on the average empirical performance of the ZS classification' on the same 11 datasets. That contradicts the claimed validation-free protocol. It's one hyperparameter, so arguably a minor leak rather than a fatal one, but it needs disclosure. The definition of Λ in Eq. (3) is off by a factor of four from the appendix derivation (main text says 2·diag, appendix uses 0.5·diag); Proposition 1 is technically incorrect as stated, though the appendix derivation fixes it. No code is provided, which makes re-checking the numbers harder.\n\nWho this is for: people working on CLIP few-shot adaptation, calibration, or selective classification will find the method and the empirical comparison informative. It deserves a serious referee: the idea is sound, the experiments are broad, and the flaws are fixable, but the reported coverage numbers need to be reconciled or corrected before the paper can be trusted. My recommendation: send it to peer review, and require the authors to resolve the Table 2/Table 4 discrepancy, fix Eq. (3), and either disclose the prior selection or run a fixed-prior sensitivity analysis.","headline":"A useful recasting of CLAP as MAP inference with a sensible variational extension, but the abstract's coverage claim is internally inconsistent with the paper's own ablation table.","tokens_in":86327,"tokens_out":3273,"would_cite":false,"duration_ms":33169,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A Bayesian posterior over adapter weights produces better-calibrated CLIP few-shot predictions.","keywords":["CLIP","few-shot adaptation","vision-language models","Bayesian deep learning","variational inference","calibration","selective classification","uncertainty estimation"],"falsifier":"Run BayesAdapter on a held-out dataset whose zero-shot accuracy differs clearly from the average of the 11 datasets used to set the prior, keep the prior standard deviation fixed at 0.01 without consulting that dataset's test curves, and measure ECE and coverage at 99% confidence against CLAP. The ablation in Table 4 shows the prior governs the reported gains (coverage moves from 3.1% to 12.0% as the standard deviation goes from 0.0001 to 1.0), so a dataset where the fixed prior is miscalibrated relative to its zero-shot level should shrink the ~2.4-point ECE gap or erase the coverage advantage; finding no such dataset across several held-out tasks would instead support the validation-free claim.","tokens_in":85348,"feed_emoji":"🎯","tokens_out":12858,"duration_ms":111532,"temperature":0.7,"pith_summary":"The paper claims that the best-accuracy CLIP adapters are not the best at knowing when they are right, and that this gap can be closed by treating the adapter weights as a probability distribution instead of a single fitted matrix. Its starting observation is that CLAP, a state-of-the-art few-shot adapter, is exactly maximum-a-posteriori inference in a probabilistic model whose Gaussian prior is centered on the zero-shot text prototypes and whose likelihood is a multinomial softmax. BayesAdapter replaces that point estimate with a variational Gaussian posterior over the weights and averages predictions over it. Across 11 datasets, six shot levels, and two visual backbones, the paper reports consistently better calibration (expected calibration error 4.151 versus 6.592 on ResNet-50) and higher reliable test coverage (13.423% versus 7.103% at 99% confidence), while staying within about 0.7% in accuracy; at 32 shots the accuracy gap disappears and BayesAdapter leads on all metrics.","feed_headline":"Bayesian adapter cuts CLIP calibration error by a third","feed_subtitle":"A posterior over adapter weights lifts 99%-confidence coverage from 7.1% to 13.4% within 0.7% of CLAP's accuracy.","key_machinery":"The object that carries the argument is the variational Gaussian posterior $q(W)=\\mathcal{N}(W|\\Omega,\\Sigma)$ over the adapter weight matrix $W$, a block-diagonal covariance matrix with a separate learned variance for each class. Its prior $p(W)=\\mathcal{N}(W|T,\\Lambda)$, centered on the zero-shot class prototypes $T$, is chosen so that MAP inference in the same probabilistic model reproduces CLAP exactly (Proposition 1); the variational step then replaces the MAP mode with a distribution whose per-class spread is learned from the few-shot data. Training minimizes the negative evidence lower bound, whose data term is the expectation of the cross-entropy under $q$ estimated with $s_{\\mathrm{MC}}=3$ Monte Carlo samples via the reparameterization trick, and whose regularizer is the closed-form KL divergence between two Gaussians — the probabilistic analogue of CLAP's cross-entropy plus $\\ell^2$ penalty. At test time, softmax probabilities are averaged over the posterior by Monte Carlo integration, which is the step that converts the learned parameter distribution into calibrated confidence scores.","core_discovery":"The central claim is that uncertainty in CLIP few-shot adaptation improves when the adapter is fitted by variational Bayes rather than by MAP inference. Proposition 1 shows that the CLAP objective — cross-entropy plus per-class $\\ell^2$ penalties pulling the linear layer $W$ toward the zero-shot prototypes $T$ — is exactly the negative log posterior of the model $p(W)=\\mathcal{N}(W|T,\\Lambda)$ with a multinomial softmax likelihood, so CLAP keeps only the mode of the posterior and discards all other information about parameter uncertainty. BayesAdapter instead fits $q(W)=\\mathcal{N}(W|\\Omega,\\Sigma)$, a Gaussian posterior with a block-diagonal covariance carrying one variance per class, by minimizing the negative evidence lower bound using the reparameterization trick, and integrates over the posterior at test time via Monte Carlo sampling. The empirical claim is that this parameter-level uncertainty propagates into the predictions: BayesAdapter has the best expected and adaptive calibration error on both ResNet-50 and ViT-B/16, is reliable at every confidence level from 99% down to 80% while covering more test samples and more classes than all seven baselines, and pays roughly a 0.5–0.7% accuracy penalty in the low-shot regime that becomes an accuracy advantage by 32 shots.","pith_inferences":["The prior variance is a single global knob chosen using the evaluation benchmarks themselves; on a genuinely new dataset the fixed value 0.01 could be off, and the learned per-class variances suggest a natural repair — estimating the prior from the support set's zero-shot confidence — that the paper does not test.","The per-class variances $\\sigma_1^2,\\dots,\\sigma_C^2$ learned by the posterior are themselves a reliability signal for each category, which could be reused for out-of-distribution detection or for choosing which images to label next in an active-learning loop; the paper only uses them to shape the predictive distribution.","Any adapter whose loss is cross-entropy plus an $\\ell^2$ pull toward a reference point admits the same variational treatment, so the argument should extend to adapters beyond CLAP even though the paper demonstrates it only for that one.","The EuroSAT 4-shot example — CLAP and LP++ abstain completely at 99% confidence while BayesAdapter covers 10.5% — suggests the benefit is largest precisely where deterministic adapters become over-conservative, so datasets with low-confidence support sets are where the method is most likely to matter in deployment."],"forward_implications":["Averaged over the 11 datasets and all shot levels, BayesAdapter is the only compared method that is reliable at every confidence level from 99% down to 80%, and at 99% confidence it covers 13.4% of the ResNet-50 test set versus 7.1% for CLAP, letting a safety-critical system act automatically on roughly twice as many predictions.","The accuracy gap shrinks with more data: the paper reports that with 32 shots BayesAdapter is the best method in accuracy as well as in calibration and coverage, so the probabilistic formulation does not trade away discriminative power once the posterior variance can be estimated.","High-confidence selections are more class-diverse under BayesAdapter — 61.1% of classes represented versus 38.5% for CLAP at 99% confidence on ResNet-50 — so the abstention behavior does not quietly concentrate on particular categories.","Because $s_{\\mathrm{MC}}=3$ Monte Carlo samples (or even 1) suffice during training, the better uncertainty costs little: on Caltech101 the training time is 42.4 seconds versus 32.3 seconds for CLAP at 32 shots, a cheaper route to calibrated confidence than ensembling.","The paper suggests that the number of available shots can serve as the practical switch: keep the deterministic CLAP when data are extremely scarce (around 1 shot per class) and use BayesAdapter when there are enough shots to learn the posterior variance."],"supporting_citations":[{"why":"CLAP, the deterministic adapter that BayesAdapter generalizes; its cross-entropy plus quadratic prototype-anchoring objective is shown to be MAP inference, and it is the primary accuracy and coverage baseline.","marker":"[47]"},{"why":"CLIP, whose frozen visual and text encoders produce the zero-shot class prototypes that anchor the Gaussian prior and initialize the posterior mean.","marker":"[44]"},{"why":"The reparameterization trick, which makes gradient-based variational training of the Gaussian posterior with Monte Carlo samples possible.","marker":"[28]"},{"why":"Supplies the variational-inference / negative-ELBO framework that BayesAdapter's training objective is built on.","marker":"[1]"},{"why":"Defines the Expected Calibration Error that measures the paper's main calibration claim.","marker":"[16]"},{"why":"LP++, a 2024 baseline that is strong in accuracy but unreliable and poor in coverage at high confidence, the contrast that motivates the work.","marker":"[22]"},{"why":"CrossModal, the closest competitor on high-confidence coverage, which BayesAdapter must beat in selective classification.","marker":"[31]"},{"why":"The Bayesian deep learning position that motivates replacing point estimates with posterior distributions over parameters.","marker":"[42]"}],"fun_headline_variants":["BayesAdapter: full posterior beats MAP for CLIP uncertainty","Variational Bayes adapter sharpens CLIP confidence estimates","CLIP adapter with Bayesian inference: better calibration, slight accuracy cost","BayesAdapter trades 0.5% accuracy for far better confidence","Uncertainty-aware CLIP adapter: select with trust, not just accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a prior standard deviation of 0.01, fixed once from the average zero-shot accuracy over the same 11 benchmark datasets, transfers to new tasks without per-task validation; the paper's own ablation shows that this single hyperparameter swings accuracy between 58.7 and 69.4 and swings 99%-confidence coverage between 3.1% and 12.0%.","fun_headline_variants_meta":{"raw":{"variants":["BayesAdapter: full posterior beats MAP for CLIP uncertainty","Variational Bayes adapter sharpens CLIP confidence estimates","CLIP adapter with Bayesian inference: better calibration, slight accuracy cost","BayesAdapter trades 0.5% accuracy for far better confidence","Uncertainty-aware CLIP adapter: select with trust, not just accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000258,"raw_usage":{"total_tokens":1632,"prompt_tokens":1044,"completion_tokens":588,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":660,"completion_tokens_details":{"reasoning_tokens":498}},"tokens_in":660,"tokens_out":588,"duration_ms":6631,"temperature":1.0,"reasoning_tokens":498,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T16:47:25.909766+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run BayesAdapter on a held-out dataset whose zero-shot accuracy differs clearly from the average of the 11 datasets used to set the prior, keep the prior standard deviation fixed at 0.01 without consulting that dataset's test curves, and measure ECE and coverage at 99% confidence against CLAP. The ablation in Table 4 shows the prior governs the reported gains (coverage moves from 3.1% to 12.0% as the standard deviation goes from 0.0001 to 1.0), so a dataset where the fixed prior is miscalibrated relative to its zero-shot level should shrink the ~2.4-point ECE gap or erase the coverage advantage; finding no such dataset across several held-out tasks would instead support the validation-free claim.","supporting_citations":[{"cited_title":"Weinberger","cited_arxiv_id":null,"evidence_quote":"Defines the Expected Calibration Error that measures the paper's main calibration claim."},{"cited_title":"A closer look at the few-shot adaptation of large vision-language models","cited_arxiv_id":null,"evidence_quote":"CLAP, the deterministic adapter that BayesAdapter generalizes; its cross-entropy plus quadratic prototype-anchoring objective is shown to be MAP inference, and it is the primary accuracy and coverage baseline."},{"cited_title":"Learn- ing transferable visual models from natural language super- vision","cited_arxiv_id":null,"evidence_quote":"CLIP, whose frozen visual and text encoders produce the zero-shot class prototypes that anchor the Gaussian prior and initialize the posterior mean."},{"cited_title":"Vari- ational inference: A review for statisticians","cited_arxiv_id":null,"evidence_quote":"Supplies the variational-inference / negative-ELBO framework that BayesAdapter's training objective is built on."},{"cited_title":"Lp++: A surprisingly strong linear probe for few-shot clip","cited_arxiv_id":null,"evidence_quote":"LP++, a 2024 baseline that is strong in accuracy but unreliable and poor in coverage at high confidence, the contrast that motivates the work."},{"cited_title":"Multimodality helps unimodality: Cross- modal few-shot learning with multimodal models","cited_arxiv_id":null,"evidence_quote":"CrossModal, the closest competitor on high-confidence coverage, which BayesAdapter must beat in selective classification."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The Bayesian deep learning position that motivates replacing point estimates with posterior distributions over parameters."}],"review_version":1}