{"id":"513ef1d5-cb32-4ab2-9cb0-6ee1bff00bcd","arxiv_id":"2505.12912","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"UnInfo adapts CLIP at test time by updating a low-rank adapter on the image encoder with a combination of confidence maximization, uniformity loss and EMA distillation, improving zero-shot accuracy under image corruptions.","lead":"CLIP models lose accuracy when test images are corrupted by blur, noise or weather, because their image embeddings become less informative. The new UnInfo method adapts the image encoder during testing by balancing prediction confidence and embedding uniformity, improving accuracy on ImageNet-C and ImageNet-C-bar.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"UnInfo's information-theoretic foundation is invalid: Eq. 4 treats H(Z|X)=0 for a deterministic encoder, which is false under stochastic corruption; L_unif is not shown to measure I(X;Z).","rationale":"The reader's weakest assumption identified the same load-bearing gap: the connection between the data processing inequality and the uniformity loss is not derived. I agree, and the flaw is more specific than a missing derivation. Equation (4) explicitly assumes H(Z|X)=0 because the encoder is deterministic, but conditional entropy is taken with respect to the clean image X, not the corrupted input X'. Since the corruption process is stochastic for noise-type degradations, Z is random even when X is fixed, so H(Z|X)>0. The claimed upper bound on H(Z) therefore does not follow. Separately, uniformity of the embedding marginal is not equivalent to preservation of task-relevant information; a deterministic mapping can spread embeddings uniformly while discarding class information. The paper's own Table I shows the uniformity loss rises under corruption, but this correlation does not establish causation or identify the measure with information content. The empirical results are still credible and the method may be effective; the issue is that the proposed explanation is unsupported. This strengthens the reader's conditional verdict rather than changing it: the paper should either correct the information-theoretic argument or explicitly reframe the uniformity objective as a purely empirical regularizer. A secondary empirical concern, that hyperparameters were selected on a few ImageNet-C corruption types without specifying which, is real but less load-bearing because the reported sensitivity analysis in Figure 2 shows modest variation across the chosen ranges.","tokens_in":19729,"tokens_out":8931,"duration_ms":101120,"concrete_test":"On a held-out subset of ImageNet-C (e.g., five corruption types not used for hyperparameter selection), estimate I(X;Z) with a nonparametric estimator such as the KSG estimator on 10k paired clean-corrupted image embeddings, measured before and after running UnInfo, and compare the change in I(X;Z) with the change in L_unif. If L_unif decreases while I(X;Z) does not increase, the uniformity loss is not acting as an information-retention measure and the central mechanism is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central mechanism—that minimizing L_unif retains input information—rests on the Markov-chain argument in Section III. The argument claims H(Z|X)=0 because the CLIP image encoder is deterministic, and therefore that corruption lowers the upper bound on H(Z). This step is wrong: in the chain X -> X' -> Z, X' is a stochastic corruption, so Z = f_img(X') is not determined by X. Conditioning on X leaves randomness from X', so H(Z|X) = H(f_img(X')|X) > 0, and this term can grow with noise entropy. The data processing inequality only gives H(Z) <= I(X;X') + H(Z|X), not a smaller H(Z). Moreover, the uniformity loss in Eq. 3 is a pairwise-distance statistic on the marginal distribution of embeddings, not an estimator of mutual information between X and Z. A deterministic encoder can produce near-uniform embeddings from class-uninformative or purely noisy inputs, so lower L_unif does not by itself imply that classification-relevant information is retained. The objective in Eq. 15 may still work empirically as a regularizer that prevents entropy-minimization collapse, but the paper's claim that UnInfo improves accuracy 'by retaining information in terms of uniformity' is not established by the presented theory.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies test-time adaptation (TTA) of CLIP for zero-shot classification under image corruption, which the authors call sensor degradation. Through a preliminary experiment on ImageNet-C, they observe that corrupted images increase both prediction entropy and a uniformity loss over image embeddings, and that simple prompt ensembling does not recover accuracy. They propose UnInfo, which updates the image encoder with LoRA adapters using an objective that combines entropy minimization, a uniformity loss, and knowledge distillation from an EMA teacher. The entropy and uniformity terms are balanced by an adaptive weight w = exp(I(z;ŷ) - I_0), where I(z;ŷ) is estimated from the current batch. Experiments on ImageNet-C and ImageNet-C-bar report that UnInfo outperforms existing CLIP TTA methods (TPT, TDA, ZERO, MTA) and few-shot adaptation baselines, with ablations showing each component contributes and a throughput analysis showing a computational advantage over augmentation-based baselines.","tokens_in":19986,"tokens_out":5139,"duration_ms":54170,"significance":"If the results hold, the paper offers a potentially useful TTA recipe for CLIP under image corruption, with a clear empirical comparison, ablations, sensitivity analysis, efficiency measurements, and released code. The claimed mechanism, however, is that corruptions reduce the information retained in image embeddings and that minimizing a uniformity loss recovers this information. This theoretical framing is not currently supported by the derivation in Section III, and the main experimental comparison is weakened by hyperparameter selection on a subset of the test distribution. The empirical method may still be valuable as a regularized entropy-minimization approach, but the paper's central explanatory claim needs substantial revision.","major_comments":[{"comment":"The derivation of the information-theoretic motivation is invalid. In the Markov chain X -> X' -> Z, X' is a stochastic corruption, so the image encoder, while deterministic given X', does not make Z deterministic given X. Consequently H(Z|X) = H(f_img(X')|X) is not zero and is not in general a constant; it can grow with the entropy of the corruption process. The data processing inequality I(X;Z) <= I(X;X') therefore does not imply that H(Z) becomes smaller under corruption. Moreover, the uniformity loss in Eq. (3) is a pairwise RBF statistic of the marginal distribution of embeddings, not an estimator of H(Z) or I(X;Z). A deterministic encoder can map noisy inputs to near-uniform embeddings that carry little class-relevant information, so minimizing this loss does not by itself establish that classification-relevant information is retained. The central claim that UnInfo improves accuracy 'by retaining information in terms of uniformity' is not supported by the presented theory. Please either provide a valid derivation or reframe the uniformity loss as an empirical regularizer that prevents entropy-minimization collapse.","section":"Section III, Eq. (4)"},{"comment":"The hyperparameters λ=1 and I_0=3 are selected using 'a few corruption types in ImageNet-C', which is part of the test distribution. In the TTA protocol, the test distribution is unknown and labels are unavailable; selecting hyperparameters on a subset of the evaluation set and then reporting the average over all corruptions inflates the reported gains and breaks the evaluation protocol. This issue is load-bearing because the information-balancing mechanism depends on the threshold I_0, and the claim that the balancing adaptively detects the current regime is weakened if I_0 is tuned on the test distribution. Please specify exactly which corruption types were used for selection, report results excluding those types, or choose hyperparameters using a disjoint validation set that is not part of the test benchmark.","section":"Section V-B"},{"comment":"The claim that existing TTA methods 'fail to adapt to distribution shifts caused by sensor degradation' is too strong. Table III shows that TPT improves mean accuracy from 23.09 to 25.07 and TDA from 23.09 to 25.54, with both methods improving on most individual corruption types. The results show limited or inconsistent gains, not a complete failure. The text should be revised to reflect this more accurately, since the overstatement affects the paper's motivation and framing.","section":"Abstract and Table III"}],"minor_comments":[{"comment":"The notation in Eq. (10) is misleading: the first equality writes I(z;ŷ) = H(z) - H(z|ŷ), but the computed quantities are H(ŷ) - H(ŷ|z) using the discrete entropy of the predicted-label distribution. Since mutual information is symmetric this is numerically correct, but the notation should be fixed to avoid confusion.","section":"Section IV-B, Eq. (10)"},{"comment":"The LoRA rank and alpha are said to be set to default values, but the actual values are not reported. Please specify these hyperparameters for reproducibility.","section":"Section V-B"},{"comment":"The discussion emphasizes UnInfo's throughput advantage, but Table VII shows that UnInfo also has the second-highest GPU memory usage (11736 MiB). This trade-off should be acknowledged explicitly.","section":"Section V-D4, Table VII"},{"comment":"The preliminary experiment in Section III uses a ViT-B/16 CLIP trained on LAION, while the main experiments use a ViT-B/16 CLIP trained on DataComp-1B. The choice of different checkpoints should be clarified earlier, since the preliminary observations motivate the method but are not directly obtained with the model used in the main evaluation.","section":"Section III and Section V-A"},{"comment":"The qualitative claim that uniformity is improved after TTA is based on a 2D spherical PCA visualization. Reporting the numerical uniformity loss before and after adaptation would strengthen this claim.","section":"Section V-D5, Figure 4"},{"comment":"The sensitivity of the EMA momentum parameter m is not reported, even though the ablation shows that the EMA teacher plays a large role in preventing collapse. A sensitivity analysis for m would be useful.","section":"Section V-B"}],"recommendation":"major_revision","confidential_remarks":"The paper has a solid empirical core, but the theoretical justification in Section III is formally incorrect and the hyperparameter selection on a subset of ImageNet-C undermines the fairness of the main comparison. Both issues are fixable: the theory can be removed or reframed as a heuristic regularizer, and the hyperparameters can be selected on a disjoint validation set or justified with a separate sensitivity study. I recommend major revision rather than rejection because the method itself may still be valuable and the experimental infrastructure is comprehensive."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Kazuki et al. have a real empirical result: CLIP zero-shot is badly degraded by ImageNet-C, and their LoRA-based TTA (UnInfo) gets consistent gains over no adaptation and over the prompt/embedding baselines (23.09 -> 27.10 mean on ImageNet-C; 32.12 -> 36.13 on C-bar). The experiments are honest and reasonably thorough, with ablations, sensitivity, throughput, and code. Credit where due: the paper identifies sensor degradation as a distinct failure mode for CLIP TTA, and shows that updating the image encoder (rather than prompts or logits) plus a uniformity regularizer and EMA distillation is a workable recipe.\n\nThe soft spots are real. Section III's information-theoretic story is wrong. The Markov chain is X -> X' -> Z; X' is stochastic given X, so H(Z|X) is not zero just because the encoder is deterministic. Eq. (4) does not imply corruption lowers H(Z), and the uniformity loss (Eq. 3) is not an estimator of I(X;Z). So the claim that UnInfo works 'by retaining information in terms of uniformity' is not established. I'd advise the authors to present the uniformity loss as a heuristic regularizer, or actually measure information retention.\n\nAlso, the hyperparameters (lambda, I0, LR) were tuned on a subset of ImageNet-C—the same test distribution—so the reported numbers are optimistic; that needs disclosure or a validation split. And the text overstates: TPT and TDA do improve over no-adapt (25.07 and 25.54 vs 23.09), so 'fail' is too strong. The ablation shows the EMA teacher alone gives 25.54; adding uniformity and balancing adds about 1.5 points. The title's 'Uniformity First' is partially supported but the teacher is doing heavy lifting.\n\nNone of this is fatal. The empirical finding and the method are useful, and the flaws are fixable in revision. I'd send it to review, with a request to fix the theory, disclose the tuning, and soften the claims. For a TTA researcher it's worth reading; for a practitioner it's a usable baseline. I'd cite it as an empirical result.","headline":"Useful empirical result with a broken theoretical motivation: the uniformity-based TTA works modestly, but the Section III information story is wrong and hyperparameters leak the test distribution.","tokens_in":20551,"tokens_out":3190,"would_cite":true,"duration_ms":31646,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CLIP's image embeddings lose uniformity — a marker of lost information — when images are corrupted by blur, noise, or weather, and the paper shows test-time adaptation must update the image encoder itself, not just prompts or embeddings.","keywords":["test-time adaptation","vision-language models","CLIP","image corruption","sensor degradation","uniformity","low-rank adaptation","zero-shot classification"],"falsifier":"Compute a ground-truth information measure for the same corruptions — for instance, the mutual information between the image embeddings and the true class labels, or the accuracy of a fixed linear probe trained on clean ImageNet features — and track it alongside the uniformity loss and UnInfo's accuracy across all 15 ImageNet-C corruptions at several severities; the central claim predicts that improvements in uniformity coincide with recoveries of label-relevant information, so a corruption where uniformity rises without the probe recovering, or where accuracy improves without uniformity improving, would falsify the mechanism.","tokens_in":19500,"feed_emoji":"🌫️","tokens_out":16900,"duration_ms":147973,"temperature":0.7,"pith_summary":"The paper sets out to show that CLIP's zero-shot classification fails on sensor-corrupted images (blur, noise, weather, compression artifacts) for a reason earlier test-time adaptation methods do not address: the image encoder's embeddings lose uniformity, which the paper reads as a loss of retained input information, whereas under ordinary domain shifts the embeddings stay informative and only the modality gap between image and text matters. It matters because deployed systems such as autonomous driving and surveillance face unexpected sensor degradation, and existing test-time fixes that tune prompts or embeddings leave the encoder fixed and so cannot restore information that was never encoded. The paper's method, UnInfo, updates the image encoder with low-rank adapters under an information-balanced loss that restores uniformity first and sharpens prediction confidence afterward, with an EMA teacher for stability. On ImageNet-C and the deliberately dissimilar ImageNet-C-bar, UnInfo achieves the best mean accuracy among the compared methods (27.10% and 36.13%), with the largest gains on exactly the corruption types where uniformity loss is highest. If the claim holds, the practical lesson is that test-time adaptation to corrupted imagery should act on the encoder itself and treat information retention, not prompt alignment, as the primary target.","feed_headline":"Updating the encoder lifts CLIP 4 points on corrupted images","feed_subtitle":"A low-rank adapter on the image encoder recovers what prompts cannot under blur, noise, and digital corruption.","key_machinery":"The load-bearing object is the uniformity loss $L_{\\mathrm{unif}} = \\log \\frac{1}{B^2} \\sum_{i,j} \\exp(-\\|z_i - z_j\\|_2^2)$, which measures how evenly the batch's image embeddings spread over the unit hypersphere; a lower value is read as more retained input information, an interpretation the paper supports with the data-processing-inequality bound $H(Z) \\le H(X) - H(X \\mid X')$. Around this it builds the information-balanced objective $\\min_{\\phi_{\\mathrm{img}}} w L_{\\mathrm{ent}} + \\lambda w^{-1} L_{\\mathrm{unif}} + L_{\\mathrm{pl}}$, in which the weight $w = \\exp(I(z;\\hat{y}) - I_0)$ uses the mutual information between embeddings and predictions to decide whether to push uniformity first (severe corruption) or confidence first (mild corruption). The image encoder is updated only through LoRA adapters on the attention layers, and an EMA teacher supplies both the distillation target $L_{\\mathrm{pl}}$ and the weights used at inference, preventing the collapse to a single-class solution that pure entropy minimization produces.","core_discovery":"On the paper's own terms, the discovery is that sensor degradation damages CLIP differently from ordinary domain shifts: it does not much widen the modality gap between image and text embeddings, but it does corrupt the image embeddings in terms of uniformity — the embeddings clump instead of spreading over the unit hypersphere, and the uniformity loss rises from 0.513 on clean ImageNet to 0.855 under Gaussian noise while prediction entropy climbs. The authors read this through the data processing inequality: for the corruption chain $X \\to X' \\to Z$, the mutual information bound implies $H(Z) \\le H(X) - H(X \\mid X')$, so corruption shrinks the upper bound on the entropy of the embedding, meaning less input information is retained. Because previously proposed TTA methods for CLIP leave the image encoder fixed and only modify prompts, text embeddings, or predictions, they cannot recover information that was never encoded; even appending corruption names to the prompt fails because CLIP cannot reliably recognize corruption types. UnInfo instead updates the image encoder with LoRA, minimizing an information-balanced objective $w L_{\\mathrm{ent}} + \\lambda w^{-1} L_{\\mathrm{unif}} + L_{\\mathrm{pl}}$ whose weight $w = \\exp(I(z;\\hat{y}) - I_0)$ is set by the mutual information between embeddings and predictions, so that uniformity is restored first under severe corruption and confidence is sharpened afterward. The same objective, stabilized by an EMA teacher used both for distillation and for inference, attains the highest mean accuracy among all compared methods on ImageNet-C (27.10%) and on ImageNet-C-bar (36.13%).","pith_inferences":["A testable extension the paper leaves implicit: the mutual-information weight $w = \\exp(I(z;\\hat{y}) - I_0)$ could double as an unsupervised domain-change detector — a sudden drop in $I(z;\\hat{y})$ would signal the onset of a new corruption type and could trigger re-weighting in the continual or fully online setting the authors name as future work.","Because the paper links uniformity to information through an inequality rather than an equality, a calibration study that measures the mutual information between adapted embeddings and clean class labels as the uniformity loss changes would turn the proxy into a quantitative law that later methods could optimize directly.","If uniformity is the real bottleneck, the same loss-balancing recipe should transfer to other encoders and to mixed or multi-severity corruptions, where prompt-based methods have even less leverage and the predictability of the method ranking would give a sharp test of the theory.","The results on ImageNet-C-bar, which was designed to be dissimilar from ImageNet-C, suggest the mechanism is not overfitting to a corruption family; pushing the same procedure to adversarial or other out-of-distribution inputs that also collapse uniformity would show whether the improvement is specific to sensor degradation."],"forward_implications":["Prompt-side fixes are almost ineffective under sensor degradation: ensembling 80 prompts gains about one percentage point, and adding corruption names to the prompt can lower accuracy; only updating the image encoder recovers the lost information.","The effective order of operations depends on corruption severity: uniformity must be restored before entropy is minimized for severe corruption such as Gaussian noise, while entropy can be minimized first for mild corruption such as brightness, and the mutual-information weight $w = \\exp(I(z;\\hat{y}) - I_0)$ supplies this ordering without knowing the corruption type.","Entropy minimization alone is destructive on corrupted inputs — the ablation collapses to 0.10% accuracy, effectively predicting a single class — so practical test-time adaptation under corruption needs a stabilizing mechanism such as the EMA-teacher distillation.","Because the adapted knowledge accumulates in the LoRA parameters, adaptation can be stopped once the distribution is stable and the LoRA merged into the base model, making inference as cheap as no adaptation at all.","The accuracy gains concentrate on the corruption types with the highest uniformity loss (blur and noise), consistent with the claim that the mechanism is information retention rather than generic feature tuning."],"supporting_citations":[{"why":"The CLIP vision-language model itself: the zero-shot classification setup in Equation (1) and the phenomenon of degraded accuracy under corruption are defined against it.","marker":"[1]"},{"why":"Test-time prompt tuning — the representative prompt-side baseline the paper shows fails under sensor degradation and that UnInfo must beat.","marker":"[15]"},{"why":"Supplies the entropy-minimization confidence objective on which the uniformity-aware confidence maximization is built.","marker":"[22]"},{"why":"Supplies ImageNet-C, the 15-type corruption benchmark that defines the sensor-degradation setting.","marker":"[37]"},{"why":"Supplies ImageNet-C-bar, the deliberately dissimilar ten-corruption benchmark used to test whether adaptation generalizes beyond ImageNet-C.","marker":"[44]"},{"why":"Supplies the uniformity loss and the reading of uniform hypersphere embeddings as information-preserving, the paper's central diagnostic.","marker":"[45]"},{"why":"LoRA — the low-rank adapter mechanism used to update the image encoder while avoiding catastrophic forgetting.","marker":"[46]"},{"why":"The contrastive (InfoNCE) objective underlying CLIP training and the reference for the uniformity measure.","marker":"[53]"},{"why":"Provides the specific LoRA implementation on CLIP attention layers that UnInfo adopts.","marker":"[60]"},{"why":"Supplies the EMA teacher and the distillation loss that stabilize the encoder update and the inference weights.","marker":"[61]"}],"fun_headline_variants":["CLIP's uniformity loss fixed via encoder LoRA for sensors","Test-time adapt by updating image encoder, not prompts","UnInfo: EMA teacher and info-balanced loss for CLIP TTA","Image embeddings clump under noise: UnInfo restores uniformity","Drop prompt tuning: adapt CLIP's image encoder instead"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entirety of the method rests on treating the uniformity loss — how evenly the adapted image embeddings spread over the unit sphere — as a faithful measure of how much classification-relevant information the encoder has retained; if that proxy separates from the true information content of the embeddings, the information-balancing argument loses its grounding even though the accuracy gains might persist.","fun_headline_variants_meta":{"raw":{"variants":["CLIP's uniformity loss fixed via encoder LoRA for sensors","Test-time adapt by updating image encoder, not prompts","UnInfo: EMA teacher and info-balanced loss for CLIP TTA","Image embeddings clump under noise: UnInfo restores uniformity","Drop prompt tuning: adapt CLIP's image encoder instead"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000466,"raw_usage":{"total_tokens":2441,"prompt_tokens":1178,"completion_tokens":1263,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":794,"completion_tokens_details":{"reasoning_tokens":1178}},"tokens_in":794,"tokens_out":1263,"duration_ms":9714,"temperature":1.0,"reasoning_tokens":1178,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:24:08.251254+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute a ground-truth information measure for the same corruptions — for instance, the mutual information between the image embeddings and the true class labels, or the accuracy of a fixed linear probe trained on clean ImageNet features — and track it alongside the uniformity loss and UnInfo's accuracy across all 15 ImageNet-C corruptions at several severities; the central claim predicts that improvements in uniformity coincide with recoveries of label-relevant information, so a corruption where uniformity rises without the probe recovering, or where accuracy improves without uniformity improving, would falsify the mechanism.","supporting_citations":[{"cited_title":"Test-time prompt tuning for zero-shot generalization in vision-language models,","cited_arxiv_id":null,"evidence_quote":"Test-time prompt tuning — the representative prompt-side baseline the paper shows fails under sensor degradation and that UnInfo must beat."},{"cited_title":"Tent: Fully Test-Time Adaptation by Entropy Minimization,","cited_arxiv_id":null,"evidence_quote":"Supplies the entropy-minimization confidence objective on which the uniformity-aware confidence maximization is built."},{"cited_title":"Benchmarking Neural Network Ro- bustness to Common Corruptions and Perturbations,","cited_arxiv_id":null,"evidence_quote":"Supplies ImageNet-C, the 15-type corruption benchmark that defines the sensor-degradation setting."},{"cited_title":"On interaction between augmenta- tions and corruptions in natural corruption robustness,","cited_arxiv_id":null,"evidence_quote":"Supplies ImageNet-C-bar, the deliberately dissimilar ten-corruption benchmark used to test whether adaptation generalizes beyond ImageNet-C."},{"cited_title":"Low-rank few-shot adaptation of vision- language models,","cited_arxiv_id":null,"evidence_quote":"Provides the specific LoRA implementation on CLIP attention layers that UnInfo adopts."},{"cited_title":"Continual test-time domain adaptation,","cited_arxiv_id":null,"evidence_quote":"Supplies the EMA teacher and the distillation loss that stabilize the encoder update and the inference weights."}],"review_version":1}