{"id":"d95e0162-3283-43bc-8484-5f1630663db8","arxiv_id":"2411.19731","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":9,"one_line_summary":"A YOLO plus VGG19-GRU system detects fight, gunshot, and fire anomalies on proprietary videos, with no public benchmark or code release.","lead":"This doctoral thesis builds a surveillance anomaly detector that pairs YOLO object detection with a VGG19-GRU temporal classifier to spot fights, gunshots, and fires. It reports strong accuracy on the author's own videos, but the thesis itself concedes the system does not run in true real time.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Quantitative support for the 'up to 90%' claim is undermined by the use of the validation split as the test bed and by precision/recall values that cannot be reproduced from the published confusion matrices.","rationale":"The reader's verdict is CONDITIONAL and focuses on the lack of public benchmarks and real-surveillance representativeness. I partially agree: the proprietary-data issue is real. However, the more elementary problem is that the reported numbers are not trustworthy even on the proprietary data. The validation set is used to select the architecture and hyperparameters, so reported performance is optimistically biased; in addition, several precision values conflict with the confusion matrices. Because the central claim is specifically 'reaching up to 90% precision and recall in certain cases', the absence of a valid test-set evaluation leaves that claim unverifiable. This does not mean the engineering work is worthless; it means the paper should not be accepted as evidence for the claimed accuracy until a held-out evaluation is supplied. I recommend UNVERDICTED rather than REJECT because the underlying models may perform well once properly evaluated.","tokens_in":55320,"tokens_out":10742,"duration_ms":89856,"concrete_test":"Hold out a random 20% of the proprietary videos as an untouched test set before any further training or hyperparameter selection; retrain the final binary and multi-class models on the remaining 80% using the already-chosen hyperparameters, and report precision, recall, and F1 on this held-out split. If the held-out F1 for the Fight, Gunshot, and Fire classes falls more than a few points below the reported 90%, or if the reported metrics cannot be recomputed from the published confusion matrices, the central claim is not supported.","verdict_should_be":"UNVERDICTED","load_bearing_attack":"The central claim is numerical: the system reaches up to 90% precision and recall, as in Table 4.43 (Fire/Normal, sequence detection: precision 93.24%, recall 90.03%). For this claim to hold, the evaluation must be valid. Two problems make it invalid as reported. First, all quantitative results are computed on the 'Validation' split of Tables 3.1 and 3.2, and Section 4.6.1 uses exactly that split to choose the sequence length, the number of GRU neurons, the dropout rate, the classifier depth, and the VGG retraining layer. No separate test set is used for any reported number; the only held-out data are ten videos, described qualitatively in Section 4.6.2. The conclusion later calls this split a 'test set', but because it was used for model selection, it functions as a validation set. Second, the confusion matrices do not reproduce the reported metrics: in Table 4.34 (Gunshot, sequence detection), precision is reported as 35.1%, while the matrix, with rows summing to 100%, implies a precision of 84.8% for the Gunshot row; in Table 4.43, the Fire row implies 83.67% precision, not 93.24%. This discrepancy is not a rounding effect. Together these facts mean the 'up to 90%' headline is not established, even for the author's proprietary videos.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript, a 2023 doctoral thesis posted to arXiv, presents a video anomaly-detection system that couples YOLO object detection (spatial analysis) with a VGG19-GRU temporal classifier ('CGRU'), usable in serial or parallel modes, extended with hand-crafted correction rules (flame implies fire; firearm overlapping a person implies gunshot) and with explainability outputs (saliency and activation maps adapted to video, a contour method, and a 'badBox' evaluation metric). Two proprietary datasets were assembled, one of images (firearms, flames, humans) and one of videos (fights, gunshots, fires versus normal). On its validation videos the system is reported to reach up to 90% precision and recall (e.g., Table 4.43, fire/normal sequence detection), and the Conclusion summarizes the system as providing 'very good results, reaching up to 90% precision and recall in certain cases.' The title claims real-time detection in video streams.","tokens_in":55678,"tokens_out":22638,"duration_ms":163990,"significance":"If the reported results were robust, this would be a useful engineering contribution: an interpretable surveillance-assistance pipeline with two deployment modes, documented per-decision latencies, and a notably candid limitations section (Conclusion, Section 10, admits the absence of surveillance-camera footage and the non-real-time operation). The experimental breadth is a genuine strength: C3D, ConvLSTM, and CNN+RNN variants are compared across several backbones (Tables 4.27-4.31), model selection is documented in detail, the extension of saliency and activation maps to time-distributed video models is concretely described, and the badBox metric is an original evaluation element. However, the central claims are not established at the level asserted: all reported numbers come from the same split used for model selection, several reported metrics cannot be reproduced from the published confusion matrices, the real-time claim contradicts the author's own conclusion, and there is no evaluation on public benchmarks or any statistical analysis.","major_comments":[{"comment":"All reported quantitative results are computed on the split called 'Validation' in Tables 3.1 and 3.2, and the same split is used in Ch. 4, Section 6.1 to select the image size (Table 4.1), the sequence length (Table 4.2), the VGG19 retraining layer (Tables 4.28-4.29), the recurrent cell (Table 4.31), the classifier depth (Table 4.32), and the GRU size and dropout (Figures 4.6-4.9). No separate test split is defined anywhere in the dataset description, and the only data that are truly held out are the ten unedited videos discussed qualitatively in Section 4.6.2. Conclusion, Section 10 confirms that 'performance metrics were thus based solely on videos from our test set,' which is the same validation split. Because this split was used for hyperparameter selection, the reported precision and recall figures (including the 'up to 90%' headline) are selection-fitted estimates on the validation distribution, not estimates of generalization to unseen data; the paper must either evaluate on a genuinely held-out test set or scale its claims down accordingly.","section":"Ch. 3, Section 4.3, Tables 3.1-3.2; Ch. 4, Section 6; Conclusion, Section 10"},{"comment":"The reported metrics cannot be reconciled with the published confusion matrices. The matrices are row-normalized (rows sum to 100%), so the diagonal entry of a class row is that class's recall. In Table 4.33 (fight, sequence detection) the reported recall is 60.3% while the matrix diagonal is 78.3%, and the reported tuple (precision 93.6%, recall 60.3%, accuracy 63.1%) is impossible with that matrix under any class prior: matching the reported accuracy forces a normal:fight prior of about 5:1, which yields a precision near 27%, not 93.6%. Table 4.35 (fire, sequence detection) reports recall 96.2% against a diagonal of 60.5%; Table 4.43 (fire/normal) reports recall 90.03% (sequence) and 87.87% (video) against diagonals of 83.67% and 93.33%; and Table 4.40 (normal/abnormal, sequence) reports precision 94.82% when the accuracy-consistent prior gives about 55%. In Table 4.34 the reported recall (84.8%) agrees with the matrix, but the reported precision (35.1%) requires an undocumented evaluation-set prior of roughly 20 normal sequences per gunshot sequence. These are not rounding effects, and since these tables are the only quantitative evidence for the central claim, the metrics, the normalization convention, and the class priors must be restated and verified.","section":"Tables 4.33, 4.35, 4.40, 4.43 (and 4.34)"},{"comment":"The title's claim of real-time detection is contradicted by the manuscript itself. Ch. 1, Section 5 sets the goal as near real-time, Ch. 4, Section 5 states that the two parallel models are 'not yet capable of processing videos in real time,' and Conclusion, Section 9 states that 'the anomaly is detected a posteriori. It therefore does not operate in real-time.' The timing tables (4.25-4.26) show per-decision times of 0.5-1.1 s in parallel mode and 1.0-1.5 s in serial mode, with total processing times of the same order as (or longer than) the video duration, and Table 4.44 shows the same pattern on the ten held-out videos. The title and abstract should be changed to a defensible statement such as 'near-real-time anomaly detection,' with the measured latency reported explicitly, or the authors must demonstrate processing at a rate faster than the input frame rate.","section":"Title; Ch. 1, Section 5; Ch. 4, Section 5; Conclusion, Section 9"},{"comment":"Tables 4.23 and 4.24 are verbatim duplicates, including the erroneous 'RCNN + YOLO' caption, so the 'reducing false positives' experiment is reported twice as two different tables. Moreover, Table 4.6 reports exactly identical metrics for CGRU alone and for CGRU fed with YOLO bounding-box overlays on the gunshot/normal task (accuracy 91.89%, precision 35.14%, recall 84.86%, F1 49.70%), and Table 4.20 repeats those identical values, with the text conceding that 'displaying the bounding boxes around objects has no impact on the performance of our model.' Since the combination of spatial and temporal analysis is one of the two stated contributions, the serial-mode fusion is shown to be ineffective for the gunshot class, and the gains in parallel mode come from the hand-crafted correction rules rather than from learned fusion; the contribution claim should be re-scoped to the configurations that actually change the results, and the duplicate tables should be removed.","section":"Tables 4.6, 4.20, 4.23-4.24"},{"comment":"The evaluation is a single-run, single-domain assessment: there are no error bars, no repeated training runs, no significance tests, no public benchmark results (UCF Crime, ShanghaiTech, Hockey Fight, and AIRTLab are surveyed in Chapter 1 but never used), and the datasets are proprietary, so the results cannot be checked by other researchers. With only 64 gunshot and 61 fire videos in the validation split (Table 3.1), the 'up to 90%' claim is not robustly supported. In addition, the parallel-mode improvement rests on deterministic rules (flame present implies fire; firearm with IoU greater than 0 with a person implies gunshot; Ch. 3, Sections 5 and 7) whose thresholds (confidence 55%, IoU > 0) are asserted without sensitivity analysis and which the author concedes (Conclusion, Section 10) misclassify legitimate situations such as armed personnel in airports. An ablation and threshold study is needed, and the stated scope should reflect this limitation.","section":"Ch. 3, Sections 5 and 7; Ch. 4, Sections 6-7; Conclusion, Section 10"}],"minor_comments":[{"comment":"Typos and terminology: 'Convolutional Neuronal Network,' 'Recurrent Neronal Network,' 'You Only Looks Once,' 'YOLO+CRU' (Table 4.6), 'RCNN + YOLO' (Tables 4.23-4.24), 'A V AD' (Ch. 1, Section 3), and 'Segment Anything Mode' (Conclusion, Section 11) should be corrected.","section":"Throughout"},{"comment":"Cross-references are broken ('Section ??' in Ch. 4, Section 2; 'see Section 6' in Ch. 3, Section 4; 'page 23' in Ch. 3, Section 4.3), and figure numbering is duplicated (Figure 2.1 appears in both Chapter 2 and Chapter 3; Figure 4.2 appears twice), which will confuse readers.","section":"Ch. 2, Ch. 3, Ch. 4"},{"comment":"The 'Elements' column is never defined (number of videos versus number of sequences), and the confusion matrices do not state whether percentages are normalized by row or by column; both must be specified for the metrics to be verifiable, and the number of decimal places should be made consistent.","section":"Tables 4.33-4.43"},{"comment":"The dataset table lists Fight, Gunshot, and Fire but omits the number of Normal videos, although Normal is one of the two classes in most of the reported binary experiments; the Normal counts should be added.","section":"Table 3.1"},{"comment":"The 'X' entries for Optical Flow and Mask Optical Flow are unexplained; the caption should note that the model could not be trained with these preprocessing options.","section":"Table 4.3"},{"comment":"The claimed novelty ('novel approach combining temporal and spatial analysis'; an 'original method' for explainability) should be positioned against the already-surveyed combined approaches (Doshi and Yilmaz 2020; Mustafa 2022) and against the author's own prior publications (Poirier et al. 2022, 2023a, 2023b), so that the incremental contribution is clear.","section":"Introduction; Ch. 2, Section 4"},{"comment":"The summary sentence 'reaching up to 90% precision and recall in certain cases' should name the specific configuration (Table 4.43, fire/normal, sequence detection) and should carry the caveat that the numbers come from the validation split; in its current form it overstates what the data show.","section":"Conclusion, Section 9"},{"comment":"The Conclusion chapter numbers its sections 9, 10, and 11 while the table of contents lists only an unnumbered 'Conclusion'; the headings should be renumbered consistently.","section":"Conclusion"}],"recommendation":"major_revision","confidential_remarks":"This is the text of a 2023 PhD thesis, not a journal article; the state-of-the-art chapters are tutorial-level, and the three core contributions already appear in the author's cited publications (EGC 2022; AICCSA 2023; arXiv 2023). If the journal considers theses, the manuscript would need substantial restructuring and the re-evaluation described in the major comments. The object-detection review relies heavily on blog posts and informal sources; a journal version should lean on peer-reviewed references. The proprietary datasets and the absence of released code make independent verification impossible at present."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a real engineering effort with a clear head on its shoulders, but the headline numbers are not established. The validation split does double duty as the test set, and the reported precision/recall figures don't match the confusion matrices printed in the same chapter. Those two facts sink the 'up to 90%' claim, and the title's 'Real-Time' is disowned in the conclusion (near-real-time, detection a posteriori).\n\nWhat's genuinely good: the thesis is upfront about its constraints. It builds two proprietary datasets, tests a wide set of architectures (C3D, ConvLSTM, CNN+RNN with VGG19/ResNet/Inception/Xception), compares serial vs parallel fusion with YOLO, and adds two original evaluation/visualization tools: badBox metric and a contour visualization method for activation maps. The failure analysis is thoughtful (e.g., why inter-image difference hurts on smartphone footage, why gunshot detection doesn't benefit from YOLO bounding boxes unless you add person-gun IoU). The author also plainly states the system was never tested on actual surveillance cameras and that rules like 'flame implies fire' may not transfer to contexts like airports. That honesty is real.\n\nThe soft spots are in the evaluation, not the effort. All quantitative results come from the 'Validation' split described in Tables 3.1-3.2, and Section 4.6.1 uses exactly that split to pick sequence length, GRU size, dropout, classifier depth, and VGG retraining layer. So the numbers are fitted, not tested. The conclusion calls it a 'test set' — that's a category error. Worse, the confusion matrices don't reproduce the stated metrics. In Table 4.34 (Gunshot, sequence detection) the Gunshot row sums to 84.8% but precision is reported as 35.1%; in Table 4.43 (Fire/Normal) the Fire row implies 83.67% precision, not 93.24%. Rounding doesn't explain an ordering inversion. I checked the stress-test note against the thesis text; it holds up.\n\nThere are smaller issues: duplicated tables (4.23/4.24), no error bars anywhere, and the CGRU vs CGRU+YOLO identical result on gunshot is suspicious but the text explains it as the model ignoring bounding boxes — plausible but still odd.\n\nWho is this for? Someone working on applied video anomaly detection in constrained industrial settings might find the engineering comparisons useful. An evaluator would not use the reported numbers as evidence. As a PhD thesis it's a solid piece of work that needed a more careful evaluation chapter; as an arXiv paper it's overclaiming the title.\n\nRecommendation: send it to review. There's enough substance and the flaws are fixable with a proper held-out test set, honest metric correction, and releasing code/data (or at least a public benchmark). But the current version needs major revision before any numbers can be trusted.","headline":"A thorough engineering thesis with honest limitations, but the headline accuracy numbers don't survive contact with its own confusion matrices.","tokens_in":56172,"tokens_out":2387,"would_cite":false,"duration_ms":21280,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This thesis claims that fusing YOLO object detection with a VGG19-GRU temporal network can spot fights, gunshots, and fires in video streams with up to 90 percent precision and recall, while also providing visual explanations for each…","keywords":["anomaly detection","video streams","real-time","YOLO","VGG19","gated recurrent unit","object detection","explainability"],"falsifier":"Take the trained system as described and run it on a held-out set of genuine surveillance-camera recordings containing fights, gunshots, and fires that were not part of the author's datasets; if precision and recall fall far below the reported 90 percent, the claim of real-world anomaly detection is refuted. A simpler check is to remove the hand-written correction rules and observe whether performance on fire and gunshot classes collapses, which would show the temporal model alone does not carry the result.","tokens_in":55112,"feed_emoji":"🚨","tokens_out":6749,"duration_ms":51649,"temperature":0.7,"pith_summary":"This thesis aims to show that dangerous events in video streams—fights, gunshots, and fires—can be detected reliably by combining two complementary analyses: spatial object detection and temporal sequence analysis. The author proposes an architecture that pairs YOLO, which locates people, firearms, and flames in each frame, with a convolutional recurrent network (VGG19 followed by a GRU) that reads 20-frame sequences to decide whether an anomaly is happening. The system is trained and evaluated on two proprietary datasets built from smartphone and web videos, and its reported accuracy reaches up to 90 percent precision and recall in certain classes and configurations. The motivation is practical: automated assistance for surveillance operators who currently monitor many cameras at once, where speed and interpretability matter.","feed_headline":"Two-model AI system flags fights, gunshots, and fires at up to 90%","feed_subtitle":"Combining object detection with temporal analysis gives surveillance operators a fast, explainable alert within about a second.","key_machinery":"The load-bearing mechanism is a Convolutional Gated Recurrent Unit (CGRU): VGG19, applied through a time-distributed wrapper to each frame of a 20-image sequence, extracts spatial features, and a GRU with a multi-layer perceptron classifier reads the resulting feature sequence to label it as normal or one of the anomaly types. This temporal core is combined, in parallel mode, with YOLOv7 object detection and hand-written correction rules—a detected flame overrides the predicted class to fire, and a firearm whose bounding box has nonzero intersection over union with a person overrides to gunshot. The correction rules are the explicit coupling that lets object presence influence the temporal decision.","core_discovery":"The central claim is that combining YOLOv7 object detection with a temporal model built from VGG19 and a gated recurrent unit (GRU) produces a video anomaly detection system that can identify fights, gunshots, and fires with high accuracy on its own curated videos, in both whole-video and per-sequence evaluations. The author argues that the fusion of spatial information (the presence and position of key objects) with temporal dynamics (how those objects move) is what makes the detection work, and that a binary normal/abnormal model generalizes better than a multi-class model when the goal is simply to raise an alert. The system also provides per-frame saliency and activation maps, adapted to the time-distributed convolution, so an operator can see which regions drove the decision.","pith_inferences":["If the correction rules were replaced with a learned fusion module, the system might adapt more gracefully to contexts where a weapon is present but not dangerous, such as airport security—an example the author cites as a known failure mode.","The reported per-sequence accuracy suggests the 20-frame window and dynamic-step generator may be a limiting factor for true streaming detection; testing shorter or overlapping windows on the same data would isolate whether window length or model capacity is the bottleneck.","Because the datasets are proprietary and collected from online videos, the results are not directly comparable to public benchmarks; a natural next step would be to retrain the same architecture on a public anomaly dataset and compare per-class numbers.","The explainability module, which highlights low-activation gun regions, could be turned into a weakly supervised localization signal to eventually output bounding boxes around the anomaly, which the author lists as a missing feature."],"forward_implications":["A binary normal/abnormal model outperforms the multi-class model, so for pure alerting the recommended configuration is binary classification combined with YOLO.","Serial mode, where YOLO preprocesses frames before the CGRU, is more reliable, while parallel mode is faster; neither processes video at full frame rate, with average detection times below about 1.5 seconds.","Per-sequence evaluation, which corresponds to continuous-stream monitoring, gives lower accuracy than whole-video evaluation, so the system is better at classifying completed clips than at flagging events mid-stream.","Specialized single-anomaly models beat the multi-class model for a given anomaly type, while the multi-class model remains viable when the precise type must be identified."],"supporting_citations":[{"why":"Supplies the original YOLO object detector that grounds the spatial analysis component.","marker":"Redmon, Divvala, et al. (2015)"},{"why":"The specific YOLOv7 model selected for the final system's spatial analysis and pose estimation.","marker":"C.-Y. Wang, Bochkovskiy, and H.-Y. M. Liao (2022)"},{"why":"VGG19 is the convolutional feature extractor inside the temporal CGRU model.","marker":"Simonyan and Zisserman (2014)"},{"why":"Reports comparative evidence that convolutional GRU is more effective and less costly than convolutional LSTM, guiding the temporal architecture choice.","marker":"Ravi and Karray (2021)"},{"why":"The widely used public video-anomaly benchmark that the author examined and found insufficient, motivating creation of the proprietary video dataset.","marker":"Sultani, C. Chen, and Shah (2018)"},{"why":"Prior work combining YOLO with temporal features, which the thesis extends to a GRU-based architecture.","marker":"Doshi and Yilmaz (2020)"},{"why":"General survey defining anomaly detection and its categories, which frames the problem statement.","marker":"Chandola, Banerjee, and Kumar (2009)"}],"fun_headline_variants":["YOLO plus temporal AI flags fights, gunshots, and fires","Real-time video anomaly detection with explainable fusion","Spatial-temporal model spots dangers in live streams","Two-model AI system detects anomalies in under a second","VGG19-GRU fusion raises alerts for violence and fires"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire evaluation rests on the assumption that the author's two proprietary datasets—images and videos collected from smartphones and the web—are representative enough of real surveillance conditions for fights, gunshots, and fires that a model trained on them will behave the same way on genuine surveillance footage.","fun_headline_variants_meta":{"raw":{"variants":["YOLO plus temporal AI flags fights, gunshots, and fires","Real-time video anomaly detection with explainable fusion","Spatial-temporal model spots dangers in live streams","Two-model AI system detects anomalies in under a second","VGG19-GRU fusion raises alerts for violence and fires"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000191,"raw_usage":{"total_tokens":1360,"prompt_tokens":979,"completion_tokens":381,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":595,"completion_tokens_details":{"reasoning_tokens":300}},"tokens_in":595,"tokens_out":381,"duration_ms":3623,"temperature":1.0,"reasoning_tokens":300,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T05:52:49.506689+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the trained system as described and run it on a held-out set of genuine surveillance-camera recordings containing fights, gunshots, and fires that were not part of the author's datasets; if precision and recall fall far below the reported 90 percent, the claim of real-world anomaly detection is refuted. A simpler check is to remove the hand-written correction rules and observe whether performance on fire and gunshot classes collapses, which would show the temporal model alone does not carry the result.","supporting_citations":[],"review_version":1}