Counteracting temporal attacks in Video Copy Detection

Katarzyna Fojcik; Piotr Syga

REVIEW 4 major objections 5 minor 18 references

Counteracting temporal attacks in Video Copy Detection

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Selecting frames halfway between scene changes preserves video copy detection accuracy while foiling temporal attacks that break the previous best method.

desk verdict The targeted-blackout robustness finding is real and valuable, but the in-sample tuning and uncontrolled baseline make the headline µAP comparability claim untrustworthy. read the letter →

arxiv 2501.11171 v1 pith:ADPRLEBX submitted 2025-01-19 cs.CV cs.AIcs.IRcs.LGcs.MM

Katarzyna Fojcik , Piotr Syga This is my paper · ORCID

classification cs.CVcs.AIcs.IRcs.LGcs.MM

keywords videocopydetectionframeselectioninterframedifferenceHanningwindowtemporalattackmicro-averageprecisionDVSC2023

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the standard one-frame-per-second sampling used in video copy detection is a weak point: it breaks under targeted temporal attacks such as blacking out the middle frame of each second, and it wastes computation on redundant frames. The authors propose selecting frames from the middles of segments between scene boundaries, found via local maxima of the interframe-difference curve smoothed with a Hanning window. On the DVSC2023 benchmark, this selection keeps micro-average precision within half a percentage point of the Dual-level detection winner (0.9300 vs 0.9343) while cutting representation size by 56% and more than halving inference time. Crucially, it stays robust to targeted frame blackouts (µAP 0.8835 vs 0.3705) and to speed changes where the baseline degrades by up to 7%. If correct, the method offers a cheap, attack-resistant frame-sampling rule for real-world copy-detection systems.

What carries the argument

The load-bearing object is the interframe-difference curve: for each consecutive frame pair, the sum of absolute pixel-wise differences divided by pixel count. This curve is smoothed with a Hanning window of size 30 (or 50) to suppress noise, and its local maxima mark scene-change moments. The winning selection strategy, 'middle between local maxima,' chooses the frame lying halfway between two successive maxima, thereby avoiding the exact boundary frames that an adversary can corrupt. The Hanning window size trades off compactness against information retention: size 30 preserves accuracy, size 50 halves the descriptor size, and size 100 drops µAP by about 10 points.

What would settle it

Run the exact same frame-selection rules on a held-out split or a different video corpus that was not used to choose the window size, and compare µAP under the same targeted blackout and speed-modification attacks; if the robustness gap over Dual-level shrinks below the reported roughly 50-point advantage under targeted blackouts, the central claim of attack resistance would be falsified.

Watch

Extended reading notes

Core claim

The central discovery is that the temporal vulnerability of the Dual-level detector comes from deterministic frame sampling, not from the matching stage: when the middle frame of every second is blacked out, a fixed one-frame-per-second sampler consistently picks corrupted frames and its µAP collapses by over 60%, while sampling from the middle of each smoothed interframe-difference segment leaves accuracy nearly intact. The paper also documents that the VED component of Dual-level detection misclassifies exact copies as non-copies in 100 out of 100 tested queries. Combined, these findings support a frame-selection rule based on local maxima of interframe differences with Hanning smoothing, selecting the frame between consecutive maxima; this rule matches the baseline's accuracy on ordinary data, withstands random and targeted blackouts, and is invariant to video speed modification.

Load-bearing premise

The reported numbers come from the same 1,681-query validation subset used to pick the Hanning window size and the middle-frame rule, so the configuration is tuned to that data and no held-out set confirms that the accuracy and robustness numbers generalize.

Editorial extensions

If this is right

If the middle-frame selection rule is adopted, video copy detection systems can process more than twice as many videos per second and store 56% less descriptor data without sacrificing published accuracy.
The method's invariance to speed modification (µAP constant across 0.5x, 1.2x, and 1.5x speed) means playback-rate changes no longer require re-encoding or re-sampling strategies.
Targeted frame blackouts—an imperceptible attack that breaks the Dual-level baseline—lose most of their effectiveness; the proposed method loses only about 5% µAP under the same attack.
The reduced frame count (40 to 150 times fewer frames than raw video, versus 24 times for 1 FPS) directly lowers storage and matching costs for large video databases.
The documented failure of VED on exact copies suggests that copy-detection pipelines should not treat unedited queries as non-copies without explicit verification.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The interframe-difference sampling rule has only one tunable parameter (the Hanning window size) and could plausibly transfer to other descriptor-based retrieval tasks beyond copy detection, such as near-duplicate image or audio retrieval.
Because the method selects frames by content change rather than by time, it may also be more robust to frame dropping and insertion attacks beyond the blackout and speed modifications tested; this is a testable extension the paper does not run.
The 56% size reduction suggests a scaling law: using the middle-between-maxima rule with larger windows could push memory savings further at a predictable µAP cost, letting systems choose an operating point on an explicit accuracy-efficiency frontier.
The paper's finding that deterministic sampling is the attack surface implies that future copy-detection methods should report robustness against targeted temporal perturbations, not only average accuracy.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

The thing worth remembering about this paper: the targeted blackout result is probably real. A frame selection based on the midpoint between scene-change peaks survives dropping the middle frame of every second (µAP 0.88 vs 0.37 for the dual-level baseline), and that is a large enough margin to be credible even if the comparison is sloppy. The paper also documents a genuinely awkward failure of the META-winning VED component on exact copies, which nobody else seems to have pointed out.

The new content is the temporal-attack analysis, not the frame selection itself; the frame selection is lifted from the authors' own [6]. That is fine as an extension, but don't credit the method as novel.

The soft spots are exactly where the stress-test note lands, plus a couple more. The parameter search in Table 1 (two strategies x three window sizes, six configurations) is done on the same 1681-query subset that produces the headline µAP in Tables 2-4. The differences at stake are tiny (0.9261 to 0.9300), so selection noise can easily explain the reported edge. No held-out test, no confidence intervals on the no-attack and targeted-attack numbers. Second, the Dual-level numbers are taken from the original paper [16] rather than a re-run on the same code and subset; the phrase 'we conduct our experiments using the same data' does not guarantee the same SSCD version, decoder, matching code, or even the same implementation. Third, the speed-modification attack is implemented by changing the fps metadata without altering the frame sequence (the text says 'it keeps all the frames'), so the proposed method's 'invariance' is trivial—it samples the same frames regardless. The dual-level baseline's 7% drop is an artifact of its fixed 1fps sampling, not evidence of the attack's strength. Fourth, the abstract overreaches: the 56% size reduction and 'more than 2x' speedup are only true for the window-50 variant, while the robustness tables lead with window-30; the 1.4-5.8x efficiency claim is not consistently defined.

The paper is a reasonable workshop-level empirical study with one striking finding, but as written the central 'comparable µAP' claim is not established. I'd send it to peer review rather than desk reject—the flaws are concrete and fixable, and the robustness observation deserves a proper test. But I'd expect a major revision with a held-out split, a real baseline re-run, and a corrected speed-attack evaluation.

Read it if you work on VCD or adversarial video perturbations; otherwise skim the tables.

Referee Report

4 major / 5 minor

Summary. The paper analyzes the Dual-level detection method for video copy detection on the DVSC2023 dataset, identifies a failure of its VED component on exact copies, and proposes a frame selection strategy based on local maxima (or midpoints between local maxima) of the Hanning-smoothed interframe difference curve. It reports that the 'Local-max-mid-window30' configuration achieves µAP 0.9300 versus the Dual-level baseline's 0.9343 while reducing descriptor storage by about 56% (with window size 50) and improving inference throughput, and that it is substantially more robust to targeted frame blackouts (µAP 0.8835 vs. 0.3705) and speed modifications than the baseline.

Significance. The paper addresses a practical problem and proposes a simple, computationally cheap frame-selection heuristic. Its experiments cover three temporal attack types and compare several window sizes, and the large targeted-blackout margin suggests the robustness phenomenon is real. If the evaluation were controlled and validated on held-out data, the contribution would be useful for resource-constrained video copy detection. Currently, however, the central quantitative claims rest on in-sample parameter selection and an uncontrolled baseline comparison.

major comments (4)

[§6.1, Table 1] The Hanning window size and the frame-selection strategy are chosen by inspecting µAP on the same 1681-query subset that is then used for all headline results in Tables 2–4. This makes the reported µAP comparability and robustness figures in-sample estimates; selection noise among the six configurations in Table 1 could account for the small gap to Dual-level (0.9300 vs. 0.9343). Please provide a held-out test set, cross-validation, or confidence intervals to support the claim of comparable performance.
[§6.2, Table 2] The Dual-level baseline numbers appear to be taken from reference [16] rather than measured with the same pipeline. Differences in SSCD version, frame decoding, matching code, and the exact query subset could change both µAP and throughput. A controlled re-implementation of the baseline on the same machine and the same 1681 queries is needed to support the efficiency and comparability claims.
[§5.2, Table 4] The speed-modification attack as described (changing only the container fps while keeping all frames) does not alter the decoded frame content or order, so the interframe difference curve and the selected frames are unchanged by construction. The observed exact invariance (0.9300 in all columns) is therefore an artifact of the attack implementation rather than evidence of robustness. Please implement speed modification that actually duplicates or skips frames (e.g., with setpts or by re-encoding at a different frame rate) and re-evaluate.
[§4] The VED failure experiment is reported only as "100 queries ... none recognized"; no details are given on how the exact-copy queries were selected, how VED was applied, or what score threshold was used. Since this motivates the paper, please provide the full protocol.

minor comments (5)

[Conclusion, §5.2] There are typos such as "Meta AI Challange" and "ffmeg"; please proofread.
[§3.1] The relation between the "Validation Split" (8,295 queries) and the "1681-query subset" is not defined precisely; please specify how the subset was derived and whether it is the same subset as used in [16].
[Abstract, §6.2] The phrase "1.4 to 5.8 times more efficient" is used in the abstract and conclusion, but Table 2 reports inference speed relative to Dual-level, not to the standard 1 FPS baseline; the comparison base should be stated consistently.
[Tables 2–4] No confidence intervals or significance tests are reported; given the small µAP differences between the proposed method and the baseline, stating variance across queries or across random attack runs would be informative.
[General] The paper does not state whether code or reproducible scripts will be released; an availability statement would strengthen reproducibility.

Circularity Check

1 steps flagged · score 6.0 of 10

Headline µAP is the argmax over six configurations on the same 1681-query subset used for final comparisons; no held-out split, so the 'comparable µAP' claim is in-sample by construction.

fitted input called prediction [Section 6.1 (Table 1) and Section 6.2 (Table 2)]
"The winning team extracted a smaller validation set from the original training dataset for their experiments, reducing its size by a factor of four (1681 queries). We conduct our experiments using the same data. ... One can note that the highest efficacy is obtained by the second selection strategy with Hanning window of size 30 ..."

The configuration Local-max-mid-window30 is selected because it achieves the highest µAP on the 1681-query subset (Table 1), and that same subset and configuration are then used to report the headline µAP of 0.9300 in Table 2 and the robustness numbers in Tables 3–4. The reported 'comparable µAP' is therefore the very objective optimized during parameter selection, not an out-of-sample prediction. No held-out split or confidence interval is provided, so the central accuracy claim reduces to the selection criterion by construction.

full rationale

The paper's central methodological content—frame selection by local maxima of interframe differences and the middle-frame variant—is a heuristic adopted from the authors' prior work [6] and extended to temporal attacks. That part is not circular: it is not derived from the target result, and the current paper re-evaluates the method empirically. The circularity is confined to the evaluation protocol. Section 6.1 uses the same 1681-query validation subset both to choose the Hanning window size and frame-selection strategy and to report all downstream results. Since the chosen configuration is the argmax over six configurations on exactly this data, the headline µAP and robustness margins are in-sample estimates of the selection objective, not independent tests. I do not count the self-citation [6] as load-bearing because it is used as background motivation and the paper's own experiments provide the evidence, however in-sample. The efficiency and representation-size comparisons are less affected by this selection issue, but they are measured on the same pipeline. Overall, one central claim—comparable µAP with improved robustness—is partially circular in its current evidence base, so the score is 6.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claim rests on the choice of frame selection parameters (window size, strategy, local-maxima definition), which are tuned on the same evaluation subset. The method also assumes the interframe-difference proxy is valid and that the baseline comparison is faithful; these are domain assumptions rather than independently verified facts.

free parameters (3)

Hanning window size = 30 (selected from 30, 50, 100)
Chosen based on µAP on the validation subset (Table 1); affects the number of selected frames and robustness to attacks.
Frame selection strategy = middle between local maxima
Chosen over the local-maxima strategy due to higher robustness to targeted attacks (Table 1); a design choice tuned on the validation subset.
Local maxima detection parameters = not specified
The definition of a local maximum (e.g., minimum prominence, neighborhood size) is not given, yet it determines which frames are selected.

assumptions (3)

domain assumption Interframe difference (sum of absolute pixel differences) is a reliable proxy for scene changes and for selecting representative frames.
Assumed throughout Section 5.1, inherited from the authors' prior work [6]; no independent justification is given in this paper.
domain assumption The 1681-query subset extracted by the winning team is representative of the full DVSC2023 distribution and suitable for parameter selection and final evaluation.
Section 6 states experiments use the same subset, but no evidence for representativeness or generalizability is provided.
domain assumption The Dual-level method [16] was correctly reimplemented or its reported numbers are directly comparable to the proposed method's results.
The paper compares against Dual-level numbers in Tables 2-4 but does not describe its implementation or whether it was rerun under identical conditions on the same hardware and data.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Counteracting temporal attacks in Video Copy Detection." pith.science (2026). https://pith.science/paper/ADPRLEBX

@misc{pith2026250111171,
  author       = {Pith},
  title        = {Pith review of: Counteracting temporal attacks in Video Copy Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ADPRLEBX}},
  note         = {Machine review of arXiv:2501.11171}
}

abstract

Video Copy Detection (VCD) plays a crucial role in copyright protection and content verification by identifying duplicates and near-duplicates in large-scale video databases. The META AI Challenge on video copy detection provided a benchmark for evaluating state-of-the-art methods, with the Dual-level detection approach emerging as a winning solution. This method integrates Video Editing Detection and Frame Scene Detection to handle adversarial transformations and large datasets efficiently. However, our analysis reveals significant limitations in the VED component, particularly in its ability to handle exact copies. Moreover, Dual-level detection shows vulnerability to temporal attacks. To address it, we propose an improved frame selection strategy based on local maxima of interframe differences, which enhances robustness against adversarial temporal modifications while significantly reducing computational overhead. Our method achieves an increase of 1.4 to 5.8 times in efficiency over the standard 1 FPS approach. Compared to Dual-level detection method, our approach maintains comparable micro-average precision ($\mu$AP) while also demonstrating improved robustness against temporal attacks. Given 56\% reduced representation size and the inference time of more than 2 times faster, our approach is more suitable to real-world resource restriction.

Figures

Figures reproduced from arXiv: 2501.11171 by the authors.

**Figure 2.** Interframe differences curve before and after smoothing with Hanning [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

**Figure 3.** Interframe differences curve before and after smoothing with Hanning [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: Interframe differences curve before and after smoothing with Hanning [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

**Figure 5.** Figure 5: Selected frames from the first 10 seconds of a sample video obtained using [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

18 extracted references · 5 canonical work pages

[6]

Pattern Recognition158, 111016 (2025)

Fojcik, K., Syga, P., Klonowski, M.: Extremely compact video representation for efficient near-duplicates detection. Pattern Recognition158, 111016 (2025)

2025
[16]

A Dual-level Detection Method for Video Copy Detection

Wang, T., Ma, F., Liu, Z., Rao, F.: A dual-level detection method for video copy detection. arXiv preprint arXiv:2305.12361 (2023)

work page Pith review arXiv 2023
[1]

In: Proceedings of the IEEE/CVF international confer- ence on computer vision

Arnab, A., Dehghani, M., Heigold, G., Sun, C., Lučić, M., Schmid, C.: Vivit: A video vision transformer. In: Proceedings of the IEEE/CVF international confer- ence on computer vision. pp. 6836–6846 (2021)

2021
[2]

In: Proceedings of the IEEE/CVF International Conference on Computer Vision

Black, A., Jenni, S., Bui, T., Tanjim, M.M., Petrangeli, S., Sinha, R., Swami- nathan, V., Collomosse, J.: Vader: Video alignment differencing and retrieval. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 22357–22367 (2023) 14 K. Fojcik, P. Syga

work page 2023
[3]

Pattern Recognition130, 108807 (2022)

Chiang, T.H., Tseng, Y.C., Tseng, Y.C.: A multi-embedding neural model for in- cident video retrieval. Pattern Recognition130, 108807 (2022)

2022
[4]

In: 2023 IEEE International Conference on Image Processing (ICIP)

Deng, R., Wu, Q., Li, Y.: 3d-csl: self-supervised 3d context similarity learning for near-duplicate video retrieval. In: 2023 IEEE International Conference on Image Processing (ICIP). pp. 2880–2884. IEEE (2023)

2023
[5]

In: ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Deng, R., Wu, Q., Li, Y., Fu, H.: Differentiable resolution compression and align- ment for efficient video classification and retrieval. In: ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). pp. 3200–3204. IEEE (2024)

2024
[7]

He,K.,Zhang,X.,Ren,S.,Sun,J.:Deepresiduallearningforimagerecognition.In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770–778 (2016)

2016

Show all 18 references

[8]

Kim, J., Woo, S., Nang, J.: Relational self-supervised distillation with compact descriptors for image copy detection (2024),https://arxiv.org/abs/2405.17928

2024 arXiv
[9]

In: Proceedings of the IEEE/CVF international conference on computer vision

Kordopatis-Zilos, G., Papadopoulos, S., Patras, I., Kompatsiaris, I.: Visil: Fine-grained spatio-temporal video similarity learning. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 6351–6360 (2019)

2019
[10]

In: Proceedings of the IEEE international conference on computer vision workshops

Kordopatis-Zilos, G., Papadopoulos, S., Patras, I., Kompatsiaris, Y.: Near- duplicate video retrieval with deep metric learning. In: Proceedings of the IEEE international conference on computer vision workshops. pp. 347–356 (2017)

2017
[11]

Multimedia Tools and Applications79(7-8), 4749–4761 (2020)

Li, J., Zhang, H., Wan, W., Sun, J.: Two-class 3d-cnn classifiers combination for video copy detection. Multimedia Tools and Applications79(7-8), 4749–4761 (2020)

2020
[12]

In:ProceedingsoftheAAAIConferenceonArtificialIntelligence.vol.38,pp.4126– 4135 (2024)

Ma, Z., Dong, J., Ji, S., Liu, Z., Zhang, X., Wang, Z., He, S., Qian, F., Zhang, X., Yang, L.: Let all be whitened: Multi-teacher distillation for efficient visual retrieval. In:ProceedingsoftheAAAIConferenceonArtificialIntelligence.vol.38,pp.4126– 4135 (2024)

2024
[13]

Computer Vision and Image Understanding243, 103997 (2024)

Pizzi, E., Kordopatis-Zilos, G., Patel, H., Postelnicu, G., Ravindra, S.N., Gupta, A., Papadopoulos, S., Tolias, G., Douze, M.: The 2023 video similarity dataset and challenge. Computer Vision and Image Understanding243, 103997 (2024)

2024
[14]

Pizzi, E., Roy, S.D., Ravindra, S.N., Goyal, P., Douze, M.: A self-supervised de- scriptor for image copy detection. Proc. CVPR (2022)

2022
[15]

In: Bengio, Y., LeCun, Y

Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: Bengio, Y., LeCun, Y. (eds.) 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings (2015)

2015
[17]

In: Proceedings of the 15th ACM international conference on Multimedia

Wu, X., Hauptmann, A.G., Ngo, C.W.: Practical elimination of near-duplicates from web video search. In: Proceedings of the 15th ACM international conference on Multimedia. pp. 218–227 (2007)

2007
[18]

Journal of Information Security and Applications 85, 103863 (2024)

Zhong, J.L., Gan, Y.F., Yang, J.X.: Efficient detection of intra/inter-frame video copy-move forgery: A hierarchical coarse-to-fine method. Journal of Information Security and Applications 85, 103863 (2024). https://doi.org/https://doi. org/10.1016/j.jisa.2024.103863, https://...

2024

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [6]

Pattern Recognition158, 111016 (2025)

Fojcik, K., Syga, P., Klonowski, M.: Extremely compact video representation for efficient near-duplicates detection. Pattern Recognition158, 111016 (2025)

2025

[2] [16]

A Dual-level Detection Method for Video Copy Detection

Wang, T., Ma, F., Liu, Z., Rao, F.: A dual-level detection method for video copy detection. arXiv preprint arXiv:2305.12361 (2023)

work page Pith review arXiv 2023

[3] [1]

In: Proceedings of the IEEE/CVF international confer- ence on computer vision

Arnab, A., Dehghani, M., Heigold, G., Sun, C., Lučić, M., Schmid, C.: Vivit: A video vision transformer. In: Proceedings of the IEEE/CVF international confer- ence on computer vision. pp. 6836–6846 (2021)

2021

[4] [2]

In: Proceedings of the IEEE/CVF International Conference on Computer Vision

Black, A., Jenni, S., Bui, T., Tanjim, M.M., Petrangeli, S., Sinha, R., Swami- nathan, V., Collomosse, J.: Vader: Video alignment differencing and retrieval. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 22357–22367 (2023) 14 K. Fojcik, P. Syga

work page 2023

[5] [3]

Pattern Recognition130, 108807 (2022)

Chiang, T.H., Tseng, Y.C., Tseng, Y.C.: A multi-embedding neural model for in- cident video retrieval. Pattern Recognition130, 108807 (2022)

2022

[6] [4]

In: 2023 IEEE International Conference on Image Processing (ICIP)

Deng, R., Wu, Q., Li, Y.: 3d-csl: self-supervised 3d context similarity learning for near-duplicate video retrieval. In: 2023 IEEE International Conference on Image Processing (ICIP). pp. 2880–2884. IEEE (2023)

2023

[7] [5]

In: ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Deng, R., Wu, Q., Li, Y., Fu, H.: Differentiable resolution compression and align- ment for efficient video classification and retrieval. In: ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). pp. 3200–3204. IEEE (2024)

2024

[8] [7]

He,K.,Zhang,X.,Ren,S.,Sun,J.:Deepresiduallearningforimagerecognition.In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770–778 (2016)

2016