{"id":"d46fd37c-adb5-40f9-bb98-ef3f9e37ea47","arxiv_id":"2606.06943","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"SS-TPT uses stability and suitability scores on augmented views to guide test-time prompt tuning and weighted prediction for improved adversarial robustness in VLMs with better robustness-throughput balance.","lead":"SS-TPT is a test-time method that scores augmented views of an image using stability (prediction consistency under weak changes) and suitability (feature density) to guide prompt tuning and final prediction in vision-language models like CLIP. A smart generalist might read it to see one practical approach for making multimodal AI more resistant to adversarial attacks while keeping inference fast.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Stability measured only under weak augmentations may not identify views trustworthy under adversarial perturbations.","rationale":"The reader's weakest assumption matches the load-bearing point exactly. Because the full text was initially unavailable the verdict stayed UNVERDICTED; the concrete test above would either confirm or refute the assumption with a single, falsifiable measurement and would move the verdict to CONDITIONAL if the correlation is low.","tokens_in":1709,"tokens_out":289,"duration_ms":11784,"concrete_test":"Compute Pearson correlation between per-view SS scores and per-view top-1 accuracy on adversarial test examples (using the same attack strength as the main experiments); if correlation < 0.4 on two datasets, re-run the main robustness-throughput curves with random view selection instead of SS weighting to isolate the contribution.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim requires that the two SS scores reliably select and weight views for both the consistency loss during prompt tuning and the final weighted prediction. Stability is defined via invariance to weak augmentations (not adversarial ones), while suitability uses feature-space density; nothing in the abstract or described method shows these quantities correlate with per-view accuracy on adversarial examples or that they avoid new failure modes such as suppressing correct but low-density views. If this correlation is weak, the robustness-throughput gains cannot be attributed to the SS guidance rather than other implementation choices.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper proposes SS-TPT, a test-time prompt tuning approach for improving adversarial robustness of vision-language models such as CLIP. It computes two scores per augmented view—stability (prediction invariance under weak augmentations) and suitability (feature-space density)—and uses them to drive an SS-guided consistency loss during prompt adaptation as well as an SS-weighted ensemble prediction at inference. The central claim is that this guidance yields superior robustness-throughput trade-offs compared with prior state-of-the-art test-time adaptation methods across multiple datasets and different numbers of views.","tokens_in":1790,"tokens_out":467,"duration_ms":17750,"significance":"If the empirical claims hold after addressing the correlation concern below, the work would provide a practical engineering advance for deploying robust VLMs under realistic compute budgets. The dual use of the same scores for both adaptation and inference is a coherent design choice, and the public code release is a clear strength for reproducibility.","major_comments":[{"comment":"§3 (method): Stability is defined exclusively via invariance to weak augmentations, yet the central claim requires that this quantity reliably identifies views that remain accurate under adversarial perturbations. No correlation analysis, per-view accuracy plots under attack, or ablation isolating the contribution of the SS scores versus other implementation choices is presented; without such evidence the robustness gains cannot be attributed to the proposed guidance rather than confounding factors.","section":"§3"},{"comment":"§4 (experiments): The manuscript asserts outperformance “across diverse datasets and varying numbers of views,” but the provided text supplies no attack strengths, baseline implementations, statistical tests, or ablations that remove the SS weighting. These details are load-bearing for the robustness-throughput claim and must be supplied with concrete numbers and controls.","section":"§4"}],"minor_comments":[{"comment":"Notation for the two SS scores and the precise form of the consistency loss should be introduced with an equation rather than prose only.","section":"§3"},{"comment":"Figure captions should explicitly state the attack norm and strength used for the reported robustness numbers.","section":null}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive feedback. The comments correctly identify areas where additional empirical evidence would strengthen the attribution of gains to the SS guidance mechanism. We will revise the manuscript to address both points with new analyses and details.","responses":[{"response":"We agree that a direct link between weak-augmentation stability and adversarial-view accuracy must be demonstrated rather than assumed. In the revised manuscript we will add (i) scatter plots and Pearson/Spearman correlations between per-view stability scores and adversarial accuracy, (ii) per-view accuracy plots under PGD attack, and (iii) an ablation that replaces the SS-guided loss and weighting with uniform averaging while keeping all other implementation choices fixed. These additions will allow readers to quantify how much of the reported robustness improvement is attributable to the SS scores.","revision_made":"yes","referee_comment":"[§3] §3 (method): Stability is defined exclusively via invariance to weak augmentations, yet the central claim requires that this quantity reliably identifies views that remain accurate under adversarial perturbations. No correlation analysis, per-view accuracy plots under attack, or ablation isolating the contribution of the SS scores versus other implementation choices is presented; without such evidence the robustness gains cannot be attributed to the proposed guidance rather than confounding factors."},{"response":"We acknowledge that the experimental section must be expanded for full reproducibility and attribution. The revision will include: explicit attack parameters (PGD-10, ε=4/255, step size 1/255), code-level descriptions of all baseline re-implementations, standard deviations over three random seeds together with paired t-test p-values, and a dedicated ablation table that removes the SS weighting (both in the consistency loss and in the ensemble) while reporting accuracy and throughput (images/sec) for 1-, 4-, 8-, and 16-view settings on ImageNet and CIFAR-10. These concrete numbers and controls will directly support the robustness-throughput claims.","revision_made":"yes","referee_comment":"[§4] §4 (experiments): The manuscript asserts outperformance “across diverse datasets and varying numbers of views,” but the provided text supplies no attack strengths, baseline implementations, statistical tests, or ablations that remove the SS weighting. These details are load-bearing for the robustness-throughput claim and must be supplied with concrete numbers and controls."}],"tokens_in":1352,"tokens_out":506,"duration_ms":14642,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main point is that this paper gives a concrete way to cut the cost of multi-view test-time defenses for models like CLIP by scoring each augmented view on two axes and then using those scores both to shape the prompt update and to weight the final output.\n\nWhat is new is the specific combination: stability (prediction invariance under weak augmentations) plus suitability (feature-space density) applied at both adaptation and inference stages. Earlier test-time robustness work tends to treat views more uniformly, so the guided consistency loss and weighted prediction are a distinct implementation choice.\n\nThe paper does a service by naming the robustness-throughput tradeoff as the central deployment problem and by releasing code. That makes the engineering contribution easier to inspect.\n\nThe soft spot is the one flagged in the stress-test note. Stability is defined only against weak augmentations, not against the adversarial perturbations the method is meant to resist. The abstract offers no evidence that these scores actually track per-view accuracy under attack or that they avoid discarding correct but low-density views. Without ablations that isolate the scoring mechanism or tests across stronger attacks, it is hard to know whether the reported gains come from the SS guidance or from other tuning details. The low soundness score in the report is fair given how little experimental detail is visible.\n\nThis is for readers already working on test-time adaptation or efficient adversarial defenses for vision-language models. Someone in that niche could extract a usable baseline or idea even if they later modify the scoring.\n\nIt is worth sending to peer review. The problem is real and the method is specified enough to be checked.","headline":"SS-TPT pairs weak-aug stability with feature density to guide view selection in test-time VLM prompt tuning, but the link to actual adversarial trustworthiness remains unshown.","tokens_in":2279,"tokens_out":402,"would_cite":false,"duration_ms":15655,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"SS-TPT scores augmented views for stability and suitability to guide prompt tuning and weighted prediction, raising adversarial robustness in vision-language models while cutting the cost of multi-view processing.","keywords":["test-time adaptation","adversarial robustness","prompt tuning","vision-language models","augmented views","stability score"],"falsifier":"A test set or attack where views that receive high stability and suitability scores produce lower accuracy than either using every view equally or selecting views at random.","tokens_in":2580,"feed_emoji":"🛡️","tokens_out":613,"duration_ms":18277,"temperature":0.7,"pith_summary":"The paper introduces SS-TPT to make vision-language models like CLIP more resistant to adversarial attacks during test time. It assigns each augmented view two scores: stability, which checks if predictions stay the same under small changes, and suitability, which checks how densely the view sits among other views in feature space. These scores then steer a consistency loss during prompt adaptation and determine how much each view contributes to the final prediction. The method aims to keep the benefits of many views without paying the full speed penalty of prior approaches. A reader would care because it targets the practical barrier that has kept robust test-time defenses from seeing wide use.","feed_headline":"SS scores lift VLM robustness without extra slowdown","feed_subtitle":"By weighting only stable and suitable augmented views, the method improves both attack resistance and inference speed.","key_machinery":"Stability and suitability (SS) scores on augmented views, which control an SS-guided consistency loss for prompt tuning and an SS-weighted prediction at inference time.","core_discovery":"SS-TPT evaluates the quality of each augmented view via two complementary scores: stability, measuring prediction invariance to weak augmentations, and suitability, measuring feature-space density among views. These stability and suitability scores guide both adaptation and inference through an SS-guided consistency loss and an SS-weighted prediction, amplifying trustworthy views while suppressing corrupted ones.","pith_inferences":["If the SS scores prove reliable, the same weighting idea could be added to other test-time adaptation techniques beyond prompt tuning.","The method might transfer to image-only models that lack the language component.","Fewer required views would directly lower memory and energy costs at deployment.","The scores could be checked against new attack families to see whether they remain predictive."],"forward_implications":["SS-TPT significantly outperforms prior state-of-the-art methods on robustness.","It achieves superior robustness-throughput trade-offs across diverse datasets.","Performance holds when the number of augmented views is varied.","The approach shows practicality and generality without model-specific tuning."],"fun_headline_variants":["SS scores guide VLM adaptation and inference","Stability and suitability evaluate augmented views","SS-TPT tunes prompts using SS scores in VLMs","Stability measures invariance in VLM predictions","Suitability assesses density among VLM views"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The two SS scores reliably mark which augmented views are trustworthy enough to steer prompt adaptation and final predictions without opening new failure modes.","fun_headline_variants_meta":{"raw":{"variants":["SS scores guide VLM adaptation and inference","Stability and suitability evaluate augmented views","SS-TPT tunes prompts using SS scores in VLMs","Stability measures invariance in VLM predictions","Suitability assesses density among VLM views"]},"model":"grok-4.3","cost_usd":0.00578,"raw_usage":{"total_tokens":2731,"prompt_tokens":623,"num_sources_used":0,"completion_tokens":65,"cost_in_usd_ticks":57799500,"prompt_tokens_details":{"text_tokens":623,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2043,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":623,"tokens_out":65,"duration_ms":12260,"temperature":1.0,"reasoning_tokens":2043,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-27T22:23:47.581214+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"A test set or attack where views that receive high stability and suitability scores produce lower accuracy than either using every view equally or selecting views at random.","supporting_citations":[],"review_version":1}