REVIEW 17 cited by
Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shift
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Covariate shift has been shown to sharply degrade both predictive accuracy and the calibration of uncertainty estimates for deep learning models. This is worrying, because covariate shift is prevalent in a wide range of real world deployment settings. However, in this paper, we note that frequently there exists the potential to access small unlabeled batches of the shifted data just before prediction time. This interesting observation enables a simple but surprisingly effective method which we call prediction-time batch normalization, which significantly improves model accuracy and calibration under covariate shift. Using this one line code change, we achieve state-of-the-art on recent covariate shift benchmarks and an mCE of 60.28\% on the challenging ImageNet-C dataset; to our knowledge, this is the best result for any model that does not incorporate additional data augmentation or modification of the training pipeline. We show that prediction-time batch normalization provides complementary benefits to existing state-of-the-art approaches for improving robustness (e.g. deep ensembles) and combining the two further improves performance. Our findings are supported by detailed measurements of the effect of this strategy on model behavior across rigorous ablations on various dataset modalities. However, the method has mixed results when used alongside pre-training, and does not seem to perform as well under more natural types of dataset shift, and is therefore worthy of additional study. We include links to the data in our figures to improve reproducibility, including a Python notebooks that can be run to easily modify our analysis at https://colab.research.google.com/drive/11N0wDZnMQQuLrRwRoumDCrhSaIhkqjof.
Forward citations
Cited by 17 Pith papers
-
SAFE-Merge: Data-Free Continual Model Merging with General Knowledge Preservation
SAFE-Merge masks risk-prone parameter updates and recovers lost task information with a constrained low-rank correction, achieving the best H-score in data-free continual model merging benchmarks.
-
Test-Time Curriculum for Open-Set AIGC Detection
A curriculum-based test-time adaptation method, using balanced confident pseudo-labels and multi-scale refinement, improves AIGC detector accuracy on unseen generators by 11 to 29 points over its starting detector.
-
CRISP: Constrained Refinement via Iterative Squeezing Process for Robust Medical Image Segmentation under Domain Shift
CRISP improves source-only medical image segmentation under domain shift by iteratively refining high-precision and high-recall masks derived from perturbation-based rank stability.
-
Adapt in the Wild: Test-Time Entropy Minimization with Sharpness and Feature Regularization
Test-time adaptation is stabilized by filtering unreliable samples, seeking flat entropy minima, and applying redundancy and inequity regularizers to pseudo-labeled class centroids.
-
TAPS : Frustratingly Simple Test Time Active Learning for VLMs
TAPS queries uncertain test images in a single-sample stream, stores them in a class-balanced buffer, and updates CLIP prompts to achieve small improvements over test-time tuning baselines.
-
CTA: Cross-Task Alignment for Better Test Time Training
CTA aligns a SimCLR-trained encoder to a frozen supervised encoder, then adapts only the self-supervised encoder at test time, improving corrupted-image classification over prior test-time training methods.
-
Align the GAP: Prior-based Unified Multi-Task Remote Physiological Measurement Framework For Domain Generalization and Personalization
A prior-based unified framework, GAP, jointly handles multi-source domain generalization and per-user test-time adaptation for multi-task rPPG, beating prior domain generalization and test-time adaptation methods on s...
-
Exploring Audio Cues for Enhanced Test-Time Video Model Adaptation
Using audio-assisted pseudo-labels generated by a pretrained audio model and an LLM improves test-time adaptation of video classifiers on corrupted videos.
-
SRPL-SFDA: SAM-Guided Reliable Pseudo-Labels for Source-Free Domain Adaptation in Medical Image Segmentation
SRPL-SFDA refines source-model pseudo-labels with SAM and consistency-based reliability selection, reporting near-supervised adaptation on two MRI segmentation benchmarks.
-
Frequency Composition for Compressed and Domain-Adaptive Neural Networks
Training quantized models on low-frequency images plus frequency-aware batch normalization at test time improves both compression and domain-shift robustness.
-
SMART-PC: Skeletal Model Adaptation for Robust Test-Time Training in Point Clouds
Skeleton-based pretraining plus BatchNorm-only test-time adaptation gives fast, accurate 3D point cloud classification under corruption on ModelNet40-C and ScanObjectNN-C, but not uniformly across all tested benchmarks.
-
Test-Time Adaptation with Binary Feedback
BiTTA guides test-time model adaptation with a few binary correct/incorrect feedback labels, combining feedback-guided updates on uncertain samples with agreement-based self-adaptation on confident ones.
-
Single Image Test-Time Adaptation via Multi-View Co-Training
MuVi achieves near-supervised breast MRI tumor segmentation by adapting a pretrained 3D model on a single test image via multi-view co-training with entropy-thresholded pseudolabels.
-
Maximizing Confidence Alone Improves Reasoning
Reinforcement learning with a reward equal to the negative entropy of the model's own token distribution improves benchmark accuracy without any labeled answers.
-
Test-Time Learning for Large Language Models
TLM adapts LLMs at test time by minimizing input perplexity on unlabeled data, using high-perplexity sample selection and LoRA updates.
-
Domain Adaptation via Feature Refinement
DAFR2 combines target-data batch normalization adaptation, feature distillation, and hypothesis transfer to make models robust to image corruption without target labels.
-
Domain Borders Are There to Be Crossed With Federated Few-Shot Adaptation
FedAcross+ couples prototype-based few-shot adaptation with stream sampling on federated clients, but the experiments validate only the static configuration carried over from the authors' prior FedAcross work.
Discussion (0). Sign in to comment.