Pith. sign in

REVIEW 5 major objections 5 minor 24 references

Calibrated and Robust Foundation Models for Vision-Language and Medical Image Tasks Under Distribution Shift

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read StaRFM claims that two penalty terms—a Fisher information penalty and a confidence misalignment penalty—jointly fix distribution shift and confidence misalignment in CLIP-style vision-language models and SAM-style 3D medical segmentation…

desk verdict Extends the authors' CalShift to 3D medical segmentation, but the headline claims are contradicted by the paper's own tables and the theory is weak. read the letter →

arxiv 2507.09222 v2 pith:V4IJRWU6 submitted 2025-07-12 cs.CV cs.LG

Behraj Khan , Tahir Qasim Syed , Nouman M. Durrani , Bilal Naseem , Shabir Ahmad , Rizwan Qureshi This is my paper
classification cs.CVcs.LG
keywords foundationmodelsdistributionshiftconfidencecalibrationFisherinformationpenaltyCLIPSegmentAnythingModelmedicalimagesegmentationPAC-Bayesbounds
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to prove that the two failure modes of foundation models under distribution shift—embedding shift in vision-language models and 3D domain shift plus voxel-wise overconfidence in medical segmentation—can be cured by the same two loss terms. StaRFM adds a Fisher information penalty (FIP), extended to 3D via patch-wise gradients, and a confidence misalignment penalty (CMP), reformulated at voxel level, to the standard CLIP and SAM losses. The reported effect is a +3.5% accuracy gain and 28% lower expected calibration error across 19 vision datasets, plus a +4.2% Dice gain over fine-tuned SAM and 4.8 mm HD95 on brain MRI benchmarks. The authors also derive PAC-Bayes bounds intended to show that FIP controls generalization through the Fisher-Rao norm while CMP minimizes Brier score. A reader should care because the method is a plug-and-play regularizer that targets both robustness and calibration at training time without architectural changes.

What carries the argument

The load-bearing mechanism is the fusion of two regularizers. FIP is the negative Hessian of the log-likelihood, a curvature measure the paper uses to stabilize feature geometry under covariate shift; its 3D extension averages gradients over $16\times16\times16$ patches. CMP is a penalty that grows when an incorrect class receives more probability than the true class, and its voxel-level form is $p/(1-p)$ averaged over voxels, pushing overconfident errors down. The second load-bearing piece is the PAC-Bayes argument: Proposition 3.1 bounds target risk by source risk plus a term involving the Fisher information matrix and the shift covariance, and Theorem 3.1 bounds ECE by the square root of the average CMP plus the Brier score. These bounds are what turn the penalties from heuristics into claimed guarantees.

What would settle it

Test Theorem 3.1 on a two-class segmentation model with known confidence $p$: if $\mathrm{CMP}_{3D}=p/(1-p)$ exceeds 1 while the Brier score is at most 1, the inequality $ECE \le \sqrt{\mathrm{CMP}_{3D}/n + \mathrm{Brier}}$ is vacuously loose. Separately, construct a diagonal covariate shift and numerically check whether Proposition 3.1's $\sqrt{(I(\theta)\Sigma_{\mathrm{shift}})}$ term actually bounds the target risk in the stated inequality.

Watch

Extended reading notes

Core claim

StaRFM's central claim is that two scalar penalties, added to the loss of any CLIP-style or SAM-style foundation model, are sufficient to correct the two intertwined defects caused by covariate shift: drift in the image-text embedding geometry and overconfident wrong predictions. The Fisher information penalty $\mathcal{L}_{\mathrm{FIP}} = -\mathbb{E}_{x\sim\mathcal{D}_{\mathrm{tgt}}}[\nabla^2_\theta \log p(x;\theta)]$ is computed globally for CLIP embeddings and patch-wise over $16\times16\times16$ volume patches for SAM; the confidence misalignment penalty penalizes the probability mass an incorrect class receives when it exceeds the true class, and is averaged over voxels in the 3D case. The final losses become $\mathcal{L}_{\mathrm{StaRFM}} = \mathcal{L}_{\mathrm{CLIP}} + \lambda_1 I(\theta) + \lambda_2 \mathrm{CMP}$ and $\mathcal{L}_{\mathrm{med}} = \mathcal{L}_{\mathrm{SAM}} + \lambda_1 I_{3D}(\theta) + \lambda_2 \mathrm{CMP}_{3D}$. The paper reports that this pair raises few-shot accuracy by up to 3.5 points, cuts ECE by up to 28%, improves Dice by 4.2 points over SAM-FT on BraTS, and lowers HD95 to 4.8 mm on ATLAS, while reducing cross-domain gaps by up to 20%.

Load-bearing premise

The load-bearing premise is that the paper's PAC-Bayes bounds are valid and non-vacuous, in particular the covariance inequality in Proposition 3.1 and the claim that the voxel-wise CMP loss bounds the Brier score; if either fails, the theoretical guarantee collapses and only the empirical tables remain.

Editorial extensions

If this is right

  • If the empirical numbers hold, adding StaRFM's two terms to any CLIP-style text encoder should improve both accuracy and calibration in low-shot classification, with the largest gains in the 0-4 shot regime.
  • If the 3D FIP extension works as described, the same patch-wise regularization should reduce scanner-induced domain gaps for other volumetric encoders without architectural changes.
  • If the voxel-level CMP is effective, appending it to any Dice plus BCE segmentation loss should lower overconfidence at lesion boundaries while preserving Dice.
  • If the cross-domain gap reductions replicate, StaRFM could serve as a drop-in training-time component for multi-center medical deployment, at roughly 18% extra memory on volumetric data.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the same two penalties could be tested with entropy or focal misalignment losses to see whether the calibration gains come from CMP's specific form or from any overconfidence penalty.
  • Editorial inference: because the paper reports per-dataset losses (StanfordCars -8.2% accuracy, Flowers102 ECE +6.76%), a per-domain choice of $\lambda_1$ and $\lambda_2$ instead of fixed values is a natural next experiment.
  • Editorial inference: if Theorem 3.1's bound is vacuous, the calibration claim rests on the empirical ECE tables, so a non-trivial finite-sample bound would be the decisive theoretical test.
  • Editorial inference: extending StaRFM to label shift, which the paper leaves open, could be done by combining it with label-distribution matching in multi-center medical data.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The manuscript proposes StaRFM, a training-time regularization framework that adds a Fisher Information Penalty (FIP) and a Confidence Misalignment Penalty (CMP) to CLIP-based vision-language few-shot learning and to SAM-based 3D medical segmentation. The paper claims to derive PAC-Bayes bounds showing that FIP controls the generalization gap under covariate shift and that CMP controls calibration error, and it reports gains of +3.5% accuracy and 28% lower ECE across 19 vision datasets, plus improved DSC and HD95 on BraTS and ATLAS. The work is presented as an extension of the authors' prior CalShift framework via patch-wise 3D FIP and voxel-level CMP.

Significance. If the empirical and theoretical claims were correct, StaRFM would be a low-cost, plug-and-play module with a unified treatment of robustness and calibration across 2D VLM classification and 3D medical segmentation; the high-level idea of extending Fisher-information and confidence penalties to voxel-level medical tasks is sensible, and the code release is a positive step. However, the significance is not currently established: the headline numbers cannot be reproduced from the paper's own tables, the main and ablation tables contradict each other, the central theory is either mislabeled or vacuous, and the direct predecessor baseline is missing. Because these defects affect the paper's main claims and not just presentation, the contribution is not ready for publication.

major comments (5)
  1. [Section 3.2, Proposition 3.1 (Eq. 11)] The bound in Eq. (11) is not derived. The proof asserts Cov <= sqrt(V[P_tgt/P_src] V[l]) <= 1/2 sqrt(D_KL * (I(theta) Sigma_l)) without relating V[l] to the Fisher information product, and Eq. (12) claims that minimizing ||I(theta)|| implies (I(theta) Sigma_shift) <= epsilon without proof. The object I(theta) Sigma_shift is a matrix, so the square root notation is undefined unless a trace or norm is specified. The proof is described as PAC-Bayes but uses importance weighting and Cauchy-Schwarz, with no prior/posterior or finite-sample PAC-Bayes argument, and the O(n^{-1/2}) term is asserted without derivation. Finally, the method in Eq. (10) penalizes a scalar FIP trace, not the nuclear norm ||I(theta)|| used in the proof, so the theory does not match the algorithm.
  2. [Section 3.2, Theorem 3.1] The ECE bound is vacuous and does not support CMP as a calibration penalty. In the binary segmentation setting used in the experiments, CMP(v) = P(y'_v|x_v)/(1-P(y'_v|x_v)) is at least 1 for every voxel because the argmax probability is at least 1/2, while the voxel Brier score B_v is at most 1; hence the asserted inequality B_v <= CMP(v) is trivially true regardless of model behavior, and the resulting bound is looser than the standard ECE <= sqrt(Brier). The theorem states a bound holding with probability at least 1-delta, but the right-hand side contains no delta-dependent term, so Hoeffding's inequality cannot produce the stated result. Minimizing CMP appears directly on the right-hand side, making the theoretical justification self-referential and failing to establish that lowering CMP reduces calibration error.
  3. [Section 5.1, Tables 4 and 5, Abstract] No table reports a combined 19-dataset aggregate corresponding to the abstract's +3.5% accuracy and 28% lower ECE. Table 4 (11 datasets) averages +3.2% accuracy and 5.7% ECE reduction, while Table 5 (8 datasets) averages roughly 1.8% accuracy and 4.4% ECE reduction; the only 28% relative ECE reduction in the paper is Table 9's BraTS result (8.6 to 6.2), a medical result, yet the abstract and conclusion attribute 28% lower ECE to the 19 vision datasets. The same configurations also disagree across tables: Table 4 reports CoOp+FIP UCF101 accuracy of 84.3, while Table 7 reports 82.3 for the same FIP-only setting, and Table 4 reports CoOp+CMP Food101 ECE of 3.02, while Table 8 reports 3.42. The main-table and ablation results are therefore mutually inconsistent, so the central empirical claim is not supported by the reported numbers.
  4. [Section 5.2, Tables 6 and 9, Abstract] The medical segmentation results are contradictory. Table 6 gives BraTS StaRFM DSC 84.5, HD95 4.9 mm, ECE 8.1%, and ATLAS HD95 6.0 mm, ECE 9.0%, while Table 9 gives BraTS DSC 84.7, HD95 4.5 mm, ECE 6.2%, and ATLAS HD95 4.8 mm, ECE 5.9%. The abstract's '+4.2% DSC over SAM-FT' and '4.8 mm HD95' match neither Table 6 nor the arithmetic in the text, since the text's claimed gains of 2.5% and 4.2% over SAM-FT do not match the 2.1 and 3.2 percentage-point differences in Table 6. With such discrepancies, the medical segmentation headline is unverifiable.
  5. [Section 4.2, Baselines] CalShift is listed as a vision-language baseline in Section 4.2 but does not appear in any results table. Because StaRFM is explicitly an extension of CalShift (Section 1 and Table 1), the absence of CalShift results means the claimed improvement over the direct predecessor cannot be checked. The same applies to the 'CalSAM' baseline, which is named but never defined or evaluated.
minor comments (5)
  1. [Section 4.2, Implementation Details] The final hyperparameter configuration is ambiguous: Implementation Details state lambda2=0.5 for vision, while Section 5.4.1 concludes lambda2=0.4, and no validation split is described for the lambda selection in Tables 10 and 11.
  2. [Section 4.1, Medical datasets] The contributions text says '3 medical benchmarks (BraTS, ATLAS)', but only BraTS and ATLAS are evaluated; the third benchmark is never identified.
  3. [References] Several references appear mismatched: the citation for Office-Home ('Venkataraman et al., 2016') and for VLCS ('Fang et al., 2013') do not correspond to those datasets, and the 'CalSAM' baseline is undefined.
  4. [Section 5.4.2] The text refers to 'Fig. S3 in supplement', but no supplementary material is included in the manuscript.
  5. [Section 4.3, Evaluation Metrics] The paper defines Cross-Site Variance as a metric but never reports it in any results table.

Circularity Check

3 steps flagged · score 5.0 of 10

The advertised first-principles bounds for FIP and CMP are constructed by placing the penalized terms directly inside the bounds (Eq. 12, Theorem 3.1), and part of the vision headline reuses datasets whose hyperparameters were tuned to produce the reported numbers; partial circularity.

  1. self definitional [Section 3.2, Proposition 3.1, Eq. (12)]
    "Regularization via ‖𝐼(𝜃)‖ minimizes this bound: min𝜃 src(𝜃)+ 𝜆‖𝐼(𝜃)‖ ⇒ (𝐼(𝜃)Σshift) ≤𝜖 (12), which guarantees that the target risk remains within 𝜖 of the source risk. The nuclear norm penalization ensures control over all singular values of the Fisher matrix."

    Eq. (11)'s bound is R_src(θ) + 1/2√(I(θ)Σ_shift) + O(n^-1/2), where I(θ) is exactly the Fisher information term that FIP adds to the loss in Eqs. (10) and (16). The proof's move from 'minimize ||I(θ)||' to '(I(θ)Σ_shift) ≤ ε' is asserted without a spectral or operator-norm argument relating the nuclear norm of I to the product IΣ_shift. Consequently the claimed 'FIP controls generalization' guarantee is constructed by placing the penalized quantity inside the bound and then minimizing that same quantity: the theorem's content is the training objective itself, not an independent bound on generalization.

  2. self definitional [Section 3.2, Theorem 3.1 proof]
    "Under confidence misalignment, overconfident predictions dominate the calibration gap, particularly when 𝑃(𝑦′𝑣|𝑥𝑣)≫𝑃(𝑦𝑣|𝑥𝑣). Thus, we bound: 𝐵𝑣 ≤ CMP(𝑣)3𝐷 = 𝑃(𝑦′𝑣|𝑥𝑣)/(1−𝑃(𝑦′𝑣|𝑥𝑣))."

    To reach the advertised conclusion that CMP reduces calibration error, the proof must connect the voxel Brier score to CMP3D. It does so by simply asserting B_v ≤ CMP3D_v under a heuristic 'overconfident predictions dominate' clause. Since CMP3D is the term being minimized in Eq. (16), the resulting ECE bound is minimized by construction once this asserted inequality is granted. The calibration guarantee is thus assumed into the definition of the penalty rather than derived; the 'Brier score minimization' claim is a restatement of the assertion B_v ≤ P/(1-P).

1 more flagged steps
  1. fitted input called prediction [Section 5.4.1, Tables 10-11 vs Table 4]
    "Table 10 reports peak accuracy at 𝜆1 = 0.4 across all datasets (Flowers102: 85.5%, Food101: 98.7%, UCF101: 84.3%, DTD: 55.1%) ... These results support our final configuration of 𝜆1=0.4 and 𝜆2=0.4 in STARFM."

    The four datasets listed in Table 10/11 (Flowers102, Food101, UCF101, DTD) are the same datasets whose numbers appear in Table 4's headline rows, and the reported values match the tuned-λ maxima exactly (e.g., CoOp+FIP UCF101 = 84.3 in both Table 4 and Table 10 at λ1=0.4). Hyperparameters were selected on these datasets, so the 'improvements' on them are in-sample selections rather than held-out predictions. The abstract aggregates these tuned numbers into its '+3.5% accuracy / 28% ECE' claim, making part of the empirical headline a fitted input presented as evaluation.

full rationale

The paper is not a pure self-citation chain: it evaluates on external benchmarks (11 vision datasets, 8 domain-shift datasets, BraTS and ATLAS), and the FIP/CMP components are tested on these data. However, the two theoretical results advertised as first-principles support are self-referential in structure. Proposition 3.1 bounds target risk in terms of the Fisher Information Matrix and then asserts that minimizing the Fisher penalty controls that bound; the key inequality (IΣ_shift) ≤ ε is asserted rather than derived, so the bound is effectively constructed to contain the penalized term. Theorem 3.1 connects CMP to calibration by asserting B_v ≤ CMP3D_v, which makes the calibration guarantee an assumption built into the penalty's definition rather than a derived result. Separately, the vision headline numbers include datasets whose hyperparameters were tuned to produce exactly the reported values (Tables 10/11 vs Table 4), so part of the empirical claim is in-sample fitting presented as evaluation. The self-citations to CalShift and C3-CLIP are used to justify the components, but the new empirical results are externally checkable, so I do not treat the self-citation chain itself as the main circularity. Overall, the derivation chain is partially circular: the theoretical guarantees reduce by construction to the penalties being minimized, and the headline gains partly reuse tuned hyperparameters. Score 5.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The method's hyperparameters are tuned on the evaluation datasets, the theory relies on unproven or vacuous inequalities, and the covariate-shift assumption is acknowledged as unaddressed for label shift. No new physical entities are introduced.

free parameters (3)
  • lambda1 (FIP weight) = 0.4 (vision), 0.3 (medical)
    Tuned on Flowers102/Food101/UCF101/DTD and BraTS/ATLAS (Tables 9-11); no held-out validation described.
  • lambda2 (CMP weight) = 0.5 (main text), 0.4 (ablation text)
    Tuned on the same datasets; the paper states different final values in different sections (Section 4.2 vs 5.4.1).
  • FIP patch size = 16x16x16
    Chosen for 3D patches; no sensitivity analysis for patch size is provided.
assumptions (5)
  • domain assumption P_src(y|x) = P_tgt(y|x) under covariate shift
    Assumed in Prop 3.1 and throughout; the paper acknowledges label shift is unaddressed in Limitations.
  • domain assumption D_KL(P_tgt || P_src) is finite
    Stated in Limitations as a bounded-shift assumption.
  • standard math The importance-weighting first-order expansion of the target risk is valid
    Used in Prop 3.1 proof, following Shimodaira (2000); the expansion is heuristic and not rigorously justified for neural networks.
  • ad hoc to paper B_v <= CMP(v)_3D for each voxel
    Asserted in Theorem 3.1 proof; trivially true since CMP>=1 and B_v<=1, making the bound vacuous.
  • standard math Hoeffding's inequality applies to the ECE estimate
    Invoked in Theorem 3.1 proof without specifying the independent variables; ECE is a bin-based average, not a simple sum.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Calibrated and Robust Foundation Models for Vision-Language and Medical Image Tasks Under Distribution Shift." pith.science (2026). https://pith.science/paper/V4IJRWU6

@misc{pith2026250709222,
  author       = {Pith},
  title        = {Pith review of: Calibrated and Robust Foundation Models for Vision-Language and Medical Image Tasks Under Distribution Shift},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/V4IJRWU6}},
  note         = {Machine review of arXiv:2507.09222}
}
read the original abstract

Foundation models like CLIP and SAM have advanced computer vision and medical imaging via low-shot transfer learning, aiding CADD with limited data. However, their deployment faces two key challenges. \textit{distribution shift} where pre-training and post-training data distributions differ (e.g., due to inter-center image acquisition) and \textit{confidence misalignment}, which leads to overconfident errors. These issues surface differently, vision-language models (e.g., CLIP) suffer from 2D embedding shift (image-text misalignment), while medical models (e.g., SAM) encounter 3D domain shifts (e.g., scanner variation) and voxel-wise calibration need. Existing solutions are domain-specific. We propose \textbf{StaRFM}, a fusion of Fisher information penalty (FIP) and confidence misalignment penalty (CMP) tackling both challenges. It applies FIP, extended to 3D via patch-wise regularization, to reduce embedding shift, and CMP, reformulated for voxel-level predictions, to calibrate segmentation uncertainty. We derive PAC-Bayes bounds. FIP controls generalization via the Fisher-Rao norm, and CMP reduces calibration error via Brier score minimization. StaRFM surpasses baselines by \texttt{+}3.5\% accuracy and 28\% lower ECE on 19 vision datasets (e.g., ImageNet, Office-Home), achieves +4.2\% DSC over SAM-FT and 4.8mm HD95 on medical benchmarks (e.g., BraTS, ATLAS), and reduces cross-domain gaps by up to 20\%. The framework is plug-and-play, requiring minimal architectural changes. Code and models are available at: \href{https://anonymous.4open.science/r/StaRFM-C0CD/}{\textcolor{blue}{\underline{StaRFM}}}

Figures

Figures reproduced from arXiv: 2507.09222 by the authors.

Figure 1
Figure 1. Workflow of the proposed StaRFM framework: The sub-figure (a) illustrates the confidence misalignment problem caused by covariate shift. The (bottom right) part of the sub-figure (a) show misaligned predictions. The subfigure (b) middle section represents the two components as recipe in method: Fisher information penalty (𝐼(𝜃)) for covariate shift correction and confidence misalignment penalty (CMP) for calibration.… view at source ↗
Figure 2
Figure 2. Workflow of the proposed StaRFM framework for medical segmentation tasks. Both FIP and CMP are integrated into the SAM prompt encoder. The final loss, StaRFM = SAM + 𝜆1 𝐼3𝐷(𝜃) + 𝜆2CMP3𝐷, combines the original SAM loss with FIP and CMP to produce robust and aligned predictions. and Reyes (2019) [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Comparison of StaRFM versus baselines. Top left shows CLIP’s overconfident misclassification on ImageNet-R. Top right shows StaRFM’s calibrated predictions. Bottom left shows SAM-FT’s false positives (red) in BraTS. Bottom right shows StaRFM’s corrected segmentation (green) with uncertainty quantification. 5.4. Ablation study 5.4.1. Hyperparameter sensitivity in vision-language To check the tradeoff between FIP and … view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

24 extracted references · 13 canonical work pages

  1. [8]

    Assessing reliability and challenges of uncertainty estimations for medical image segmentation, in: Medical Image Computing and Computer Assisted Intervention–MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13–17, 2019, Proceedings, Part II 22, Springer. pp. 48–56. Kamnitsas, K., Baumgartner, C., Ledig, C., Newcombe, V., Simpson, J., ...

    work page 2019

  2. [9]

    Unsupervised domain adaptation in brain lesion segmentation with adversarial networks, in: Information Processing in Medical Imaging: 25thInternationalConference,IPMI2017,Boone,NC,USA,June25-30, 2017, Proceedings 25, Springer. pp. 597–609. Karimi, D., Salcudean, S.E.,

    work page 2017

  3. [13]

    IEEE transactions on medical imaging 34, 1993–2024

    The multimodal brain tumor image segmentation benchmark (brats). IEEE transactions on medical imaging 34, 1993–2024. Mirza, B., Syed, T.Q., Khan, B., Malik, Y.,

    work page 1993

  4. [17]

    arXiv preprint arXiv:1701.06548

    Regularizingneuralnetworksbypenalizingconfidentoutputdistributions. arXiv preprint arXiv:1701.06548 . Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.,

    arXiv

  5. [19]

    arXiv preprint arXiv:1212.0402

    Ucf101: A dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv:1212.0402 . Sudre, C.H., Li, W., Vercauteren, T., Ourselin, S., Jorge Cardoso, M.,

    arXiv

  6. [20]

    Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations, in: Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support: Third International Workshop, DLMIA 2017, and 7th International Workshop, ML-CDS 2017, Held in Conjunction with MICCAI 2017, Québec City, QC, Canada, September 14,...

    work page 2017

  7. [22]

    arXiv preprint arXiv:2006.10726

    Tent: Fully test-time adaptation by entropy minimization. arXiv preprint arXiv:2006.10726 . Wang, H., Ge, S., Lipton, Z., Xing, E.P.,

    arXiv 2006

  8. [23]

    arXiv preprint arXiv:2410.02681

    Understanding and mitigating miscali- bration in prompt tuning for vision-language models. arXiv preprint arXiv:2410.02681 . Behraj and Tahir et. al:Preprint submitted to Elsevier Page 12 of 13 Robust and Calibrated Foundation Models Xiao,J.,Hays,J.,Ehinger,K.A.,Oliva,A.,Torralba,A.,2010. Sundatabase: Large-scalescenerecognitionfromabbeytozoo,in:2010IEEEc...

    arXiv

Show all 24 references
  1. [24]

    arXiv preprint arXiv:2303.00915

    Biomedclip: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs. arXiv preprint arXiv:2303.00915 . Zhou, K., Yang, J., Loy, C.C., Liu, Z.,

    arXiv

  2. [2004]

    Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories, in: 2004 conference on computer vision and pattern recognition workshop, IEEE. pp. 178–178. Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Laroche...

    2004

  3. [2008]

    Automated flower classification over a large number of classes, in: 2008 Sixth Indian conference on computer vision, graphics & image processing, IEEE. pp. 722–729. Oquab, M., Darcet, T., Moutakanni, T., Vo, H., Szafraniec, M., Khalidov, V., Fernandez, P., Haziza, D., Massa, F...

    2008

  4. [2009]

    Imagenet: A large-scale hierarchical image database, in: 2009 IEEE conference on computer vision and pattern recognition, Ieee. pp. 248–255. Fang, Y., Lin, W., Chen, Z., Tsai, C.M., Lin, C.W.,

    2009

  5. [2012]

    Cats and dogs, in: 2012 IEEE conference on computer vision and pattern recognition, IEEE. pp. 3498–3505. Peng, X., Bai, Q., Xia, X., Huang, Z., Saenko, K., Wang, B.,

    2012

  6. [2013]

    arXiv preprint arXiv:1306.5151

    Fine- grained visual classification of aircraft. arXiv preprint arXiv:1306.5151 . Menze, B.H., Jakab, A., Bauer, S., Kalpathy-Cramer, J., Farahani, K., Kirby, J., Burren, Y., Porz, N., Slotboom, J., Wiest, R., et al.,

    arXiv

  7. [2014]

    Food-101–mining discriminative components with random forests, in: Computer vision– ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceedings, part VI 13, Springer. pp. 446–461. Chen, H., Xu, Y., Xu, Y., Zhang, Y., Cui, L.,

    2014

  8. [2015]

    U-net: Convolutional networks for biomedical image segmentation, in: Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18, Springer. pp. 234–241. Shimodaira, H.,

    2015

  9. [2016]

    1099–1104

    Sparkr: Scaling r programs with spark, in: Proceedings of the 2016 International Conference on Management of Data, pp. 1099–1104. Wang,C.,2023. Calibrationindeeplearning:Asurveyofthestate-of-the-art. arXiv preprint arXiv:2308.01222 . Wang, D., Shelhamer, E., Liu, S., Olshausen...

    2016 arXiv

  10. [2017]

    On calibration of modern neural networks, in: International conference on machine learning, PMLR. pp. 1321–1330. Helber,P.,Bischke,B.,Dengel,A.,Borth,D.,2019. Eurosat:Anoveldataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected...

    2019

  11. [2019]

    arXiv preprint arXiv:1903.09734

    Regularized learning for domain adaptation under label shifts. arXiv preprint arXiv:1903.09734 . Bossard, L., Guillaumin, M., Van Gool, L.,

    1903 arXiv

  12. [2020]

    arXiv preprint arXiv:2007.01434

    In search of lost domain generalization. arXiv preprint arXiv:2007.01434 . Guo, C., Pleiss, G., Sun, Y., Weinberger, K.Q.,

    2007 arXiv

  13. [2021]

    arXivpreprint arXiv:2102.07623

    Fedbn: Federated learningonnon-iidfeaturesvialocalbatchnormalization. arXivpreprint arXiv:2102.07623 . Liew,S.L.,Lo,B.P.,Donnelly,M.R.,Zavaliangos-Petropulu,A.,Jeong,J.N., Barisano, G., Hutton, A., Simon, J.P., Juliano, J.M., Suri, A., et al.,

    arXiv

  14. [2023]

    arXiv preprint arXiv:2304.07193

    Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193 . Ovadia, Y., Fertig, E., Ren, J., Nado, Z., Sculley, D., Nowozin, S., Dillon, J., Lakshminarayanan, B., Snoek, J.,

    arXiv

  15. [2024]

    Test-time medical image segmentation using clip-guided sam adaptation, in: 2024 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), IEEE. pp. 1866–1873. Cimpoi, M., Maji, S., Kokkinos, I., Mohamed, S., Vedaldi, A.,

    2024

  16. [2025]

    arXiv preprint arXiv:2501.17595

    Technical report on label-informed logit redistribution for better domain generalization in low-shot classification with foundation models. arXiv preprint arXiv:2501.17595 . Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg,...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.