{"id":"5a628727-60c0-44c5-9eaf-5399a6dff5d9","arxiv_id":"2507.18675","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":1,"one_line_summary":"CLIP misclassifies human actions under masking, and the proposed class-specific noise augmentation is claimed to help but lacks quantitative validation.","lead":"A study tests how well CLIP recognizes human actions when parts of the video are hidden, then proposes adding class-specific noise to make it more reliable. The improvement is shown only in a few hand-picked examples, with no quantitative results.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The proposed class-specific noise cannot be applied to unlabeled test frames: if N_c is chosen by ground-truth label, the claimed gains are label leakage; if chosen by prediction, the method is circular.","rationale":"The reader's verdict of REJECT is correct: the central claim is unsupported. The reader identified the absence of a training loop, hyperparameters, and quantitative comparisons. My stress-test sharpens the objection: even if those details were supplied, the method as written has a more fundamental flaw. The noise is class-specific, so at inference the model must know the class to choose N_c; the paper never specifies this. Using the ground-truth label leaks the answer; using the predicted class makes the augmentation circular. Either way, the reported improvements cannot be trusted. Additionally, the triplet loss's positive term cancels N_c, meaning the optimization objective does not do what the authors claim. These are structural issues, not mere missing details. Thus the paper cannot currently support its headline claim, and the verdict should remain REJECT. I partially agree with the reader because they correctly rejected the paper, but I believe the more decisive objection is the undefined/leaky inference procedure rather than only the lack of quantitative results.","tokens_in":14407,"tokens_out":4223,"duration_ms":43891,"concrete_test":"Inspect the released code (https://github.com/s4nyam/HAR-CLIP) and trace how N_c is selected in the CLIP+CSN inference path, checking whether the ground-truth class label is used to index D_c. Independently, run a controlled experiment on UCF-101 frames: compute baseline CLIP top-1 predictions, then for each test frame retrieve N_c using (a) the ground-truth label and (b) the model's own baseline top-1 prediction. Apply the noise to the CLIP image embedding, recompute top-1 accuracy, and compare. If only (a) improves, the reported gain is label leakage; if neither (b) improves, the method fails to generalize to truly unlabeled frames.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that class-specific noise (CSN) improves CLIP's accuracy and confidence is not supported because the inference protocol is undefined. In Section 4, N_c is learned per class and stored in dictionary D_c, but a test frame's class is unknown. The paper never states which N_c is added at test time. If the ground-truth label is used to retrieve N_c (as Figure 6's 'GT Action Image' panel suggests), then the improvement is label leakage: the model is told the answer before it predicts. If instead N_c is chosen based on the model's own top-1 prediction, then the augmentation is circular: the noise simply reinforces the model's prior guess, so the reported top-3 histograms reflect the dictionary entry rather than a genuine gain. A second, independent flaw: the triplet loss in Section 4 is degenerate. Both anchor and positive share the same N_c, so d(f(X_c)+N_c, f(X_positive)+N_c) = d(f(X_c), f(X_positive)); the positive distance is independent of N_c. Only the negative term depends on N_c, so the objective does not enforce intra-class compactness and may simply push N_c to an arbitrary large vector. Consequently, the paper's asserted improvement is not merely unquantified; as described, the method cannot be evaluated meaningfully at all.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper evaluates the CLIP vision-language model on the UCF-101 human action recognition dataset under three masking strategies (random percentage/shape-based black masking, feature-specific masking via SAM, and isolation masking that retains only class-specific regions), and proposes a class-specific noise (CSN) augmentation learned via a triplet loss to improve classification accuracy, confidence, and label dispersion. The results are presented primarily as frequency histogram charts and qualitative observations, with full prediction tables for Task 1 in the appendix. The paper concludes by discussing challenges and future directions for applying CLIP to healthcare-related action recognition.","tokens_in":14633,"tokens_out":2603,"duration_ms":26649,"significance":"If the proposed class-specific noise augmentation were rigorously validated, it could offer a simple plug-in mechanism to improve zero-shot CLIP robustness under occlusion and domain shift, which would be relevant to healthcare monitoring applications. The paper also contributes a useful descriptive evaluation of CLIP on UCF-101 under various masking perturbations, and it makes its code and full per-class prediction tables publicly available. However, the central claim of improved accuracy, confidence, and reduced bias is supported only by a few qualitative histograms, with no quantitative metrics, no comparison baselines, no error bars, and no clearly defined inference protocol. The method as described in Section 4 is also internally degenerate, as the triplet loss does not actually depend on the noise for the anchor-positive pair. As a result, the paper does not, in its current form, provide a reliable basis for its headline conclusions.","major_comments":[{"comment":"The inference protocol for the proposed class-specific noise is undefined. The noise N_c is learned per class and stored in dictionary D_c, but at test time the action class is unknown. The paper never states how N_c is selected for a query frame. If the ground-truth label is used to retrieve N_c (as the 'GT Action Image' panel in Figure 6 suggests), the improvement is label leakage; if the model's own top-1 prediction is used to select N_c, the augmentation is circular and merely reinforces the prior guess. This makes the claimed gains in accuracy and confidence in Section 5.4 impossible to interpret or reproduce.","section":"§4 and §5.4, Class-Specific Noise"},{"comment":"The triplet loss is degenerate as written. Both the anchor and the positive sample are augmented with the same noise N_c, so d(f(X_c)+N_c, f(X_positive)+N_c) = d(f(X_c), f(X_positive)), which is independent of N_c. Only the negative distance depends on N_c, so minimizing the loss can arbitrarily increase the norm of N_c or push it toward the negative class without enforcing intra-class compactness. The paper's claim that this loss 'ensures that the noise N_c is optimized to highlight class-specific features' is therefore not supported by the stated objective.","section":"§4, Triplet Loss Equation"},{"comment":"The central claim that class-specific noise improves accuracy, confidence, and reduces bias is supported only by qualitative top-3 histogram examples. No accuracy numbers, no baseline CLIP comparison, no error bars, and no statistical analysis are provided. The paper itself states in §5.4 that 'more experiments can be carried using tasks 1 to 4 in order to verify methods for task 5,' which explicitly acknowledges that the method has not been properly evaluated. Without quantitative results, the abstract's assertion of 'improved classification accuracy and model confidence' is unsubstantiated.","section":"§5.4 and Figure 6"},{"comment":"The feature-specific and isolation masking experiments are conducted on a single action class (Cricket Shot), and the paper notes that 'none of the segmented images were labelled as the ground truth.' This single-class, all-failure result is presented without any quantitative summary and without extending to other classes, so it cannot support the paper's broader generalization claims about CLIP's behavior under masking.","section":"§5.3 (Tasks 3 and 4)"}],"minor_comments":[{"comment":"There are several incomplete or informal sentences, e.g., 'In s study by [7], evaluated the effectiveness of CLIP' and '[2]' in the references has no venue. The prose would benefit from a careful editing pass.","section":"§2"},{"comment":"The abstract and Task 2 description mention 'shape-based' black masking, but the implementation describes only random percentage-based pixel masking. The paper should clarify whether shape-based masking was actually performed and, if so, how the shapes were generated.","section":"§5.2"},{"comment":"The caption states 'with this particular experiment we kept only four labels in the ground truth labels for CLIP to take as query.' This is a substantial restriction on the label space that is not mentioned in the main text, and it makes the Task 2 histogram results non-comparable to standard 101-way classification. This should be stated clearly in the main text.","section":"Table 4 (Appendix)"},{"comment":"The tables in the appendix are useful, but the paper does not provide a summary statistic such as top-1 accuracy over the 101 classes. Adding mean accuracy and confidence would make these tables more interpretable.","section":"§5.1 and Tables 1-3"},{"comment":"The figure is difficult to read because the panels are small and the histograms are not clearly labeled. A higher-resolution figure with explicit axis labels and a clearer description of the three prediction conditions would improve reproducibility.","section":"Figure 6"},{"comment":"The notation for the augmented features is inconsistent: F_c, F_aug, and F_fine-tuned are used for essentially the same operation f(X) + N_c. Please use a single consistent symbol and define all variables explicitly.","section":"§4, Notation"}],"recommendation":"reject","confidential_remarks":"The paper presents an interesting idea but the experimental validation falls far short of what is needed for a serious journal. In particular, the proposed method has no defined test-time inference rule, the triplet loss is degenerate as written, and the central claim rests entirely on a few qualitative histograms. The authors themselves acknowledge that more experiments are needed. This is not a matter of polishing; the core method as described cannot be meaningfully evaluated. I would not encourage resubmission unless the method is substantially redesigned and properly validated with quantitative results on multiple classes and with a clear inference protocol."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read this as exploratory evaluation, not a validated contribution. The descriptive part—per-class prediction histograms in the appendix and Task 2 masking examples—gives a concrete picture of how CLIP predictions shift under random black masking. That is genuinely useful for people studying CLIP robustness, and the code is released. But the abstract-level claim that class-specific noise improves accuracy and confidence is unsupported.\n\nThe problems are load-bearing. The only evidence for the improvement is Figure 6, which is qualitative: no accuracy numbers, no baseline, no error bars, only a few classes. The authors themselves concede in Section 5.4 that the method is only 'proposed and explored' and that more experiments are needed to verify it. Honest limitation, but it means the paper does not deliver what the abstract promises.\n\nThe stress-test concerns land. The inference protocol is undefined: for a test frame whose class is unknown, the paper never says which N_c to add. Using ground truth would be label leakage; using the model's prediction would be circular. Second, the triplet loss as written is degenerate: the same N_c is added to anchor and positive, so d(f(X_c)+N_c, f(X_pos)+N_c) = d(f(X_c), f(X_pos)) regardless of N_c. Only the negative term is affected, so the objective cannot enforce intra-class compactness and N_c can simply grow large to push negatives away. As described, the method cannot be evaluated meaningfully.\n\nThe masking observations could seed a future study, but the paper reads like an internship report with an overreaching abstract. The class-specific noise method needs a proper training loop, a defined inference procedure, and quantitative comparison on more than one class per task before it deserves peer review. I would not cite it in current form, and I would not send it to referees as is.","headline":"Exploratory CLIP masking study with an unvalidated and internally flawed class-specific noise method; the descriptive histogram analysis has some value, but the central claim does not hold.","tokens_in":701,"tokens_out":1941,"would_cite":false,"duration_ms":43865,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper evaluates CLIP's zero-shot action recognition under masking and claims that a per-class learned noise vector added to its frozen features reduces label dispersion, raising accuracy and confidence.","keywords":["CLIP","human action recognition","UCF-101","zero-shot classification","label dispersion","class-specific noise","triplet loss","masking strategies"],"falsifier":"Learn $N_c$ on a designated subset of UCF-101 frames per class using the described triplet loss, then evaluate on a disjoint held-out subset of the same classes, with and without 10/30/50 percent black masking, and compare top-1 accuracy and average confidence against plain CLIP. If the noise-augmented version does not beat plain CLIP on held-out frames, the claimed mechanism fails.","tokens_in":14231,"feed_emoji":"🎬","tokens_out":5406,"duration_ms":48496,"temperature":0.7,"pith_summary":"This paper evaluates the CLIP vision-language model on the UCF-101 human action recognition dataset under three kinds of perturbation: random black masking at 10, 30, and 50 percent, feature-specific masking that removes background elements, and isolation masking that keeps only class-defining regions. The reported histograms show that CLIP frequently misclassifies actions and spreads its predictions across many labels, a behaviour the paper calls label dispersion. To counter this, the authors propose learning a class-specific noise vector for each action class, optimized with a triplet loss and added to CLIP's feature representation, so that the model attends to class-defining features. They claim this augmentation improves classification accuracy and model confidence while reducing bias. The stated motivation is healthcare applications such as fall detection, patient monitoring, and surgical skill assessment, where action recognition must generalize across domains.","feed_headline":"Adding per-class noise helps CLIP keep action labels straight","feed_subtitle":"On UCF-101 frames with blacked-out cues, CLIP scatters predictions; a learned noise vector is claimed to fix it.","key_machinery":"The load-bearing mechanism is an additive class-specific noise vector $N_c$, one per action class, stored in a dictionary $D_c$ and added to CLIP's extracted feature representation: $F_c = f(X_c) + N_c$. The noise is learned with a triplet loss that pulls augmented anchor and positive frame features from the same class together while pushing apart features from a different class, with a margin $\\alpha$. In the masking experiments, the Segment Anything Model (SAM) supplies the masks used for feature-specific and isolation masking. The noise is doing the work: it is the only proposed remedy for the label dispersion the masking tasks expose.","core_discovery":"On its own terms, the paper's central discovery is that CLIP's zero-shot action recognition on UCF-101 is unstable under masking: random black masking at 10–50 percent scatters predictions across unrelated classes, feature-specific masking of backgrounds such as grass or pitch leads to complete misclassification of Cricket Shot, and isolation masking does not recover the ground truth. The paper then proposes class-specific noise $N_c$, learned per class through a triplet loss and added to the frozen visual features as $F_c = f(X_c) + N_c$, and asserts in Section 5.4 that this augmentation helps the model prevent label dispersion and produce high-confidence predictions. The claim is that adding this per-class learned perturbation to CLIP's features reinforces class-defining cues and improves both label accuracy and confidence, on ordinary frames and on SAM-segmented frames alike.","pith_inferences":["The paper's own evidence for the noise augmentation is a single illustrative figure with no training details or quantitative comparison; the claim should be read as proposed and illustrated rather than established.","Because the noise $N_c$ is an additive vector in feature space, it is equivalent to a per-class prototype shift; that invites a direct comparison against prototype-based few-shot action recognition methods.","The paper's 'domain-independent generalization' is not tested, since all experiments use UCF-101; an obvious test is to learn noise dictionaries on one dataset and evaluate on HMDB51 or Kinetics-400.","A concrete robustness test would apply the same learned noise dictionary under unseen mask patterns or object occlusions to see whether the improvement persists beyond the perturbations studied."],"forward_implications":["On UCF-101, plain CLIP's predictions scatter across many labels and misclassify actions when key cues are blacked out or backgrounds are removed; the paper treats this as label dispersion.","If class-specific noise works as claimed, CLIP can be strengthened for action recognition without retraining the vision encoder, since the noise is added to frozen features.","The proposed pipeline is aimed at healthcare scenarios such as fall detection, patient monitoring, and surgical skill assessment, where partial occlusion and domain shift are common."],"supporting_citations":[{"why":"Supplies the CLIP model whose zero-shot features and classification behavior are evaluated under masking.","marker":"[20]"},{"why":"Supplies the Segment Anything Model used to create feature-specific and isolation masks.","marker":"[17]"},{"why":"Prior work on class-specific noise injection that the paper adapts to feature-space augmentation.","marker":"[10]"},{"why":"Source of the label-dispersion concept that the paper aims to mitigate.","marker":"[4]"},{"why":"Documents vision-language models' fragility on fine-grained visual patterns, motivating the robustness study.","marker":"[21]"}],"fun_headline_variants":["CLIP's masking chaos fixed by per-class noise","Learned per-class noise steadies CLIP's action labels","CLIP mislabels masked actions; noise vector rescues","Class-specific noise keeps CLIP confident on UCF-101","For CLIP, a noise boost beats blacked-out cues"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a noise vector $N_c$, learned from a small set of frames of class $c$ with a triplet loss, transfers to other frames of the same class and can be added to CLIP's frozen features to improve zero-shot predictions on those other frames.","fun_headline_variants_meta":{"raw":{"variants":["CLIP's masking chaos fixed by per-class noise","Learned per-class noise steadies CLIP's action labels","CLIP mislabels masked actions; noise vector rescues","Class-specific noise keeps CLIP confident on UCF-101","For CLIP, a noise boost beats blacked-out cues"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000429,"raw_usage":{"total_tokens":2195,"prompt_tokens":952,"completion_tokens":1243,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":568,"completion_tokens_details":{"reasoning_tokens":1160}},"tokens_in":568,"tokens_out":1243,"duration_ms":9620,"temperature":1.0,"reasoning_tokens":1160,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:11:51.635927+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Learn $N_c$ on a designated subset of UCF-101 frames per class using the described triplet loss, then evaluate on a disjoint held-out subset of the same classes, with and without 10/30/50 percent black masking, and compare top-1 accuracy and average confidence against plain CLIP. If the noise-augmented version does not beat plain CLIP on held-out frames, the claimed mechanism fails.","supporting_citations":[{"cited_title":"Learning transferable visual models from nat- ural language supervision","cited_arxiv_id":null,"evidence_quote":"Supplies the CLIP model whose zero-shot features and classification behavior are evaluated under masking."},{"cited_title":"Class-Specific Noise Injection for Improved Road Segmentation","cited_arxiv_id":null,"evidence_quote":"Prior work on class-specific noise injection that the paper adapts to feature-space augmentation."},{"cited_title":"When deep learners change their mind: Learning dynamics for active learning","cited_arxiv_id":null,"evidence_quote":"Source of the label-dispersion concept that the paper aims to mitigate."}],"review_version":1}