{"id":"16651638-96f6-44d4-9593-1297241595ae","arxiv_id":"2505.13123","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"PI-VAD distills pose, depth, panoptic, flow, and language cues into an RGB-only weakly supervised anomaly detector, reporting SOTA AUC/AP on UCF-Crime, XD-Violence, and MSAD.","lead":"This paper trains a video anomaly detector on five extra cues (body pose, depth, object masks, motion, and text) but only needs ordinary video at test time. It reports the best published scores on three surveillance benchmarks, UCF-Crime, XD-Violence, and MSAD.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The SOTA gain is attributed to five modality targets, but no control shows those targets' specific content matters; random or shuffled targets may perform equally.","rationale":"The reader's weakest assumption concerns information loss in the PMG reconstruction, evidenced by the 0.25% gap between PI-VAD and true-modality CMI. My concern is upstream and complementary: even a perfect PMG would not validate the multi-modal claim unless the target contents themselves are shown to matter. This is not a logical contradiction or an accusation; the paper includes solid ablations showing PMG, alignment, and distillation are all necessary, and the code is released. But the absence of a random-target/shuffle control leaves open a plausible alternative explanation for the +3.61 AUC jump from the baseline to CMI with true modality features (Table 3): the extra transformer capacity and distillation pressure could deliver most of the gain regardless of what the five auxiliary inputs encode. Since the paper explicitly frames its contribution as augmenting RGB with five modalities, this control is needed before the 'poly-modal' interpretation can be accepted. The reader's CONDITIONAL verdict already asks for more validation, and my concern specifies one concrete experiment that would settle it, so no verdict change is needed.","tokens_in":12988,"tokens_out":7518,"duration_ms":83094,"concrete_test":"Run the full PI-VAD pipeline on UCF-Crime with identical hyperparameters but replace the five ground-truth embedding sets e_j in Eq. (1) and in the CMI input with (a) random Gaussian tensors of the same shapes, and (b) shuffled modality assignments (e.g., depth targets used as pose, pose targets as depth). If either control reaches roughly 90.3 AUC, the modality-specific content is not causally load-bearing and the SOTA claim should be reframed as an auxiliary-training effect; if both drop toward the 86-87 baseline, the modality semantics are confirmed as the source of the gain.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.2-4.3 and Table 3 support the claim that PMG+CMI raise UCF-Crime AUC from 86.97 (UR-DMU) to 90.33, with true modality features giving 90.58. However, the load-bearing assumption is not merely that Eq. (1) reconstructs the modality embeddings without too much loss; it is that the semantics of those five targets are causally responsible for the gain. The pose, depth, panoptic, and flow embeddings are all produced by frozen networks operating on the same RGB snippets, and the text embedding is derived from label prompts rather than an independent sensor. Thus the 'modalities' are auxiliary regression targets computed from the same input stream, not independent measurements. Table 6 shows each single modality improves AUC by only 0.68-0.95 points over baseline (87.65-87.92 vs 86.97), and the differences between modalities are small (0.27 AUC range). No ablation replaces these targets with uninformative controls of the same shape, so the large CMI gain (90.58) could come from the CMI transformer plus distillation loss alone, with the modality labels contributing little specific information. If that were true, the paper's central interpretation 'RGB features are insufficient, modalities add salient cues' would be unsupported, even though the numerical SOTA might still hold.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper introduces PI-VAD, a weakly-supervised video anomaly detection framework that augments RGB snippet features with five pseudo-modalities (pose, depth, panoptic masks, optical flow, and text) which are generated from the RGB features themselves by a Pseudo Modality Generation (PMG) module and integrated by a Cross Modal Induction (CMI) module through contrastive alignment and distillation from a frozen UR-DMU teacher. The modality backbones are used only during training; at inference, only the student RGB encoder and the lightweight PI modules are needed. Experiments on UCF-Crime, XD-Violence, and MSAD report high AUC/AP values, and ablations show contributions from the alignment and distillation losses and from individual modalities. The paper claims state-of-the-art accuracy while maintaining RGB-only inference and reports a favorable computational profile.","tokens_in":13231,"tokens_out":6403,"duration_ms":66733,"significance":"The central idea is attractive and practically relevant: frozen external modality embeddings can serve as auxiliary training targets that transfer multi-modal knowledge into a model that is RGB-only at inference. The paper is commendable for releasing code, for ablating the two proposed modules (Table 3), and for providing a per-modality analysis (Table 6 and Figures 4-6). The efficiency result (30 FPS, 19.88 GFLOPs versus 2,561.40 GFLOPs for the five modality backbones) is a concrete practical strength. However, the state-of-the-art claim is weakened by a higher-scoring multi-modal method in the paper's own comparison table, and the causal role of the specific modality targets is not isolated by a control experiment. If these two issues are resolved, the work would constitute a solid contribution to WSVAD.","major_comments":[{"comment":"The paper claims in §5.1 that 'Compared with current multi-modal SoTA approaches, π-VAD demonstrates superior capabilities in both UCF-Crime and XD-Violence datasets,' and the abstract claims unqualified 'state-of-the-art accuracy' on three datasets. However, Table 1 lists PEMIL (I3D+Text) under 'SoTA with multi-modality at inference' with an XD-Violence AP of 88.21, which is 2.84 points higher than the reported π-VAD AP of 85.37. This directly contradicts the stated claim. The authors should either benchmark against PEMIL under the same protocol, explicitly explain why PEMIL is excluded from the comparison, or restrict the claim to 'state-of-the-art among methods with RGB-only inference.' As written, the headline contribution is not supported by the reported numbers.","section":"§5.1, Table 1"},{"comment":"The load-bearing interpretation of the paper is that the five external modalities contribute salient anomaly-relevant cues beyond RGB. Yet no ablation replaces the modality targets with non-informative controls of the same shape, such as random targets, shuffled targets, or targets from a fixed untrained encoder. Table 3 shows that the full CMI pipeline (row 4) reaches 90.58 AUC versus 86.97 for the UR-DMU baseline, but Table 6 shows each individual modality improves AUC by only 0.68-0.95 points, and the differences between modalities are small. Without a control, the large CMI gain could be due to the extra transformer blocks, the contrastive alignment, and the distillation loss rather than to the semantic content of the pose/depth/panoptic/flow/text targets. Adding such a control is necessary to support the central claim that RGB features are insufficient and that the five modalities add salient cues.","section":"§4.2-4.3, Table 3"}],"minor_comments":[{"comment":"The text states that PMG has 'one encoder and five parallel decoders' and then says 'The six decoders operate in a mutually exclusive manner'; this count is inconsistent and should be corrected.","section":"§4.2"},{"comment":"The InfoNCE denominator sums over k≠i, which omits the positive pair, and the temperature τ appears only in the numerator and not in the denominator. This is inconsistent with the standard InfoNCE formulation described in the text and should be fixed.","section":"Eq. (2)"},{"comment":"The hyperparameters λ1, λ2, and τ are introduced but their values are not reported in the main text; Section A is referenced but not included in the manuscript text provided. The authors should ensure all training details, including these values, are fully specified in the appendix or supplement.","section":"§4.4, Eq. (6)"},{"comment":"The text-only configuration in Table 6 yields AUC_A of 69.45, which is below the RGB baseline's 70.81. This contradicts the unqualified statement in §6.1 that 'each modality is able to enhance the RGB features' and the conclusion's claim that all five modalities are useful; the statement should be qualified to mention the AUC_A regression for text.","section":"Table 6, §6.1"},{"comment":"No error bars or multiple-seed statistics are reported. Given that some comparisons in Table 3 differ by only 0.25 AUC points, reporting mean±std over at least three runs would strengthen the conclusions.","section":"Tables 1-3, 6"}],"recommendation":"major_revision","confidential_remarks":"The PEMIL comparison in Table 1 is the most serious issue because it directly affects the headline SOTA claim. If the authors cannot reconcile the comparison, they should restrict the claim to RGB-only-inference methods. The missing control for target semantics is experimentally fixable and should be added before acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nRead the PI-VAD paper. The short version: this is a real, practical contribution—extending pi-ViT's pose induction idea to five modalities for weakly-supervised VAD, with RGB-only inference and a clean, reproducible training setup. They release code. The ablations are honestly laid out: Table 3 shows that both alignment and distillation matter, Table 6 shows each individual modality nudges the baseline up, and Table 5 makes the inference-efficiency claim concrete. The reported numbers on UCF-Crime, XD-Violence, and MSAD are a solid step above prior work.\n\nThe biggest issue is not the reconstruction loss, which the oracle comparison in Table 3 (90.58 vs 90.33) suggests is nearly lossless. It is that all five modality targets are computed from the same RGB snippets by frozen networks, so they are not independent measurements; and, more importantly, there is no control with random or shuffled target embeddings. The single-modality gains are small and within a narrow band (0.68–0.95 AUC over baseline), so the large jump to 90.33 could plausibly come from the CMI transformer plus the distillation loss acting as regularizers, with the specific semantic content of the targets playing a minor role. That would not invalidate the numerical result, but it would undercut the paper's central narrative—that pose/depth/panoptic/flow/text specific cues are what add value. A reviewer should ask for a control where the PMG targets are replaced by random vectors of the same shape, or by shuffled embeddings from other videos. That's a cheap experiment and it directly tests the paper's interpretation.\n\nOther soft spots are real but minor: no error bars or variance estimates (given the 0.25% gap to the oracle, we need to know if that gap is meaningful), some metrics are defined loosely (AUC A vs AP A), and training details are in the appendix. None of these are deal-breakers.\n\nWho is this for? Anyone working on weakly-supervised VAD or on distilling auxiliary cues into RGB-only models. It deserves serious peer review; it would likely be accepted after adding the random-target control and variance estimates.\n\nRecommendation: engage with it, but push for the control.","headline":"Solid multi-modal WSVAD paper with real but possibly over-interpreted gains; the missing random-target control is the key thing a referee should demand.","tokens_in":13831,"tokens_out":2600,"would_cite":true,"duration_ms":26300,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"PI-VAD shows that a video anomaly detector can be trained on five auxiliary modalities and then run on RGB alone, reaching state-of-the-art scores on UCF-Crime, XD-Violence, and MSAD.","keywords":["weakly-supervised video anomaly detection","multi-modal representation learning","pseudo-modality generation","cross-modal induction","RGB-only inference","video anomaly detection benchmarks","teacher-student distillation"],"falsifier":"Take the trained PI-VAD and, at inference, replace the generated pseudo-modalities with random noise of the same shape; if UCF-Crime AUC stays close to 90.33, the gains come from the alignment and distillation machinery rather than from modality content. A second test is to train on UCF-Crime and evaluate on MSAD without retuning; if accuracy falls to the level of the RGB-only teacher on MSAD, the pseudo-modalities are not transferring across domains.","tokens_in":12809,"feed_emoji":"🎥","tokens_out":8493,"duration_ms":84808,"temperature":0.7,"pith_summary":"This paper argues that weakly-supervised video anomaly detection is limited by relying on RGB appearance alone, because subtle anomalies such as shoplifting can look similar to normal actions. To fix this, PI-VAD trains a video transformer with five extra modalities—pose, depth, panoptic masks, optical flow, and text—and at the same time learns to synthesize those modalities from RGB features. The extra modality backbones are used only during training; at inference the model needs only RGB and still reports top accuracy on UCF-Crime, XD-Violence, and MSAD. If correct, this means multi-modal training can enrich an RGB detector without the cost of running several heavy networks per video.","feed_headline":"Five training modalities, one RGB stream catches video anomalies","feed_subtitle":"PI-VAD tops UCF-Crime, XD-Violence, and MSAD without running pose, depth, mask, flow, or text backbones at test time.","key_machinery":"The load-bearing mechanism is the Poly-modal Inductor (PI), a training-only plug-in placed at early and late transformer blocks of a teacher-student architecture. It has two parts: PMG, a shared encoder with five decoders that predicts each modality's embedding from the student's RGB features under an MSE reconstruction loss, and CMI, which applies a snippet-level bi-directional InfoNCE contrastive loss between each pseudo-modality and RGB, fuses the aligned embeddings with transformer blocks, and distills the fused output toward the frozen teacher using an MSE distillation loss. PI is what lets the heavy modality encoders be removed at inference while their signal remains present in the RGB student.","core_discovery":"The central claim is that the latent embeddings of five auxiliary modalities can be approximated from RGB features well enough to improve weakly-supervised anomaly detection. A Pseudo-modality Generation (PMG) module maps the student's intermediate RGB representation into five pseudo-embeddings by reconstructing the outputs of pretrained pose, depth, panoptic, flow, and text models. A Cross Modal Induction (CMI) module aligns each pseudo-modality to its RGB snippet with a contrastive loss, lets transformer blocks select the relevant cues, and distills the result toward a frozen teacher's anomaly-aware features. With this training-only module inserted at early and late stages of the student, the method reaches UCF-Crime AUC 90.33, XD-Violence AP 85.37, and MSAD AUC 88.68 while processing about 30 frames per second at inference. The paper's ablation also shows that using the true modality features instead of generated pseudo-modalities raises UCF-Crime AUC by only about 0.25 points, indicating the reconstruction is nearly lossless.","pith_inferences":["Inference: if the 0.25 AUC reconstruction gap is stable across datasets, the same pseudo-modality trick could transfer to other label-sparse video tasks such as temporal action localization, where multi-modal ground truth is even scarcer.","Inference: the class-wise results suggest a budget-driven choice a practitioner could test: two or three modalities such as depth, text, and pose may capture most of the gain for human-centered anomaly categories.","Inference: the method's ceiling is set by the frozen per-modality backbones; swapping those backbones for stronger or surveillance-tuned versions is a direct test of how much headroom remains.","Inference: because text and panoptic masks dominate scene-based anomalies while pose and depth dominate human-based ones, a modular deployment could select modalities according to the anomaly types a given camera site prioritizes."],"forward_implications":["The method needs only video-level labels, so the same pseudo-modality recipe applies to other weakly supervised video tasks without frame-level annotation.","At test time the model runs at about 30 frames per second, making it plausible for live surveillance, whereas running all five modality backbones would be far more expensive.","Because different anomalies activate different modalities, the framework can also serve as an explanation tool: a high pose-modality activation flags human-centered anomalies, while text and panoptic activations flag scene-level events.","The paper's sequential-modality ablation improves accuracy at every step, suggesting the poly-modal contributions are complementary rather than redundant.","Reconstruction costs only about 0.25 AUC compared with using true modality features, so the efficiency gain comes at a very small measured accuracy cost on UCF-Crime."],"supporting_citations":[{"why":"Supplies the panoptic-mask ground-truth embeddings used to train the pseudo-modality generator.","marker":"[12]"},{"why":"Defines the snippet-level bi-directional InfoNCE loss that aligns pseudo-modalities with RGB.","marker":"[22]"},{"why":"Supplies the text and language ground-truth embeddings for the text modality.","marker":"[25]"},{"why":"Provides the UCF-Crime benchmark and its weak video-level labels.","marker":"[27]"},{"why":"Supplies the optical-flow ground-truth embeddings for the motion pseudo-modality.","marker":"[28]"},{"why":"Supplies the pose ground-truth embeddings for the pose pseudo-modality.","marker":"[31]"},{"why":"Provides the XD-Violence benchmark, including audio as an extra training modality.","marker":"[32]"},{"why":"Supplies the depth ground-truth embeddings for the depth pseudo-modality.","marker":"[34]"},{"why":"Serves as the frozen teacher and supplies the MIL anomaly-detection training objective.","marker":"[41]"},{"why":"Provides the MSAD benchmark used to test the method's generalization.","marker":"[42]"}],"fun_headline_variants":["Train five, test one: PI-VAD anomaly detection","Five modality teachers, zero inference overhead","Pseudo-modalities make RGB features anomaly-aware","Multi-modal training, unimodal testing for VAD","Distill five senses into a single RGB stream"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a small network can reproduce each auxiliary modality's learned representation from RGB features closely enough to keep the anomaly signal; if the reproduced versions lose too much, the reported gains shrink toward the RGB-only baseline.","fun_headline_variants_meta":{"raw":{"variants":["Train five, test one: PI-VAD anomaly detection","Five modality teachers, zero inference overhead","Pseudo-modalities make RGB features anomaly-aware","Multi-modal training, unimodal testing for VAD","Distill five senses into a single RGB stream"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000442,"raw_usage":{"total_tokens":2281,"prompt_tokens":1032,"completion_tokens":1249,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":648,"completion_tokens_details":{"reasoning_tokens":1177}},"tokens_in":648,"tokens_out":1249,"duration_ms":12177,"temperature":1.0,"reasoning_tokens":1177,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:19:07.593632+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the trained PI-VAD and, at inference, replace the generated pseudo-modalities with random noise of the same shape; if UCF-Crime AUC stays close to 90.33, the gains come from the alignment and distillation machinery rather than from modality content. A second test is to train on UCF-Crime and evaluate on MSAD without retuning; if accuracy falls to the level of the RGB-only teacher on MSAD, the pseudo-modalities are not transferring across domains.","supporting_citations":[{"cited_title":"Yolov7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors","cited_arxiv_id":null,"evidence_quote":"Supplies the pose ground-truth embeddings for the pose pseudo-modality."},{"cited_title":"Fine-tuned clip models are efficient video learners","cited_arxiv_id":null,"evidence_quote":"Supplies the text and language ground-truth embeddings for the text modality."},{"cited_title":"Real-world anomaly detection in surveillance videos","cited_arxiv_id":null,"evidence_quote":"Provides the UCF-Crime benchmark and its weak video-level labels."},{"cited_title":"Not only look, but also listen: Learning multimodal violence detection under weak supervision","cited_arxiv_id":null,"evidence_quote":"Provides the XD-Violence benchmark, including audio as an extra training modality."},{"cited_title":"Advancing video anomaly detection: A concise re- view and a new dataset","cited_arxiv_id":null,"evidence_quote":"Provides the MSAD benchmark used to test the method's generalization."}],"review_version":1}