Pith. sign in

REVIEW 4 major objections 5 minor 2 cited by

MAVOS-DD: Multilingual Audio-Video Open-Set Deepfake Detection Benchmark

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper introduces MAVOS-DD, a 250-hour, eight-language deepfake benchmark whose open-set splits show that state-of-the-art audio-video detectors lose substantial accuracy on unseen generators and languages.

desk verdict Solid dataset, but the open-set splits may leak clip-level overlap; the generalization numbers need a disjointness check before they can be trusted. read the letter →

arxiv 2505.11109 v1 pith:OGJMVTWR submitted 2025-05-16 cs.CV cs.AIcs.LGcs.MM

Florinel-Alin Croitoru , Vlad Hondru , Marius Popescu , Radu Tudor Ionescu , Fahad Shahbaz Khan , Mubarak Shah This is my paper · ORCID
classification cs.CVcs.AIcs.LGcs.MM
keywords deepfakedetectionopen-setbenchmarkmultimodalaudio-videomultilingualdatasetgeneralizationtalking-headgenerationfaceswappingvideoforensics
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MAVOS-DD is a new benchmark built to test whether audio-video deepfake detectors can recognize fakes produced by generators and in languages they never saw during training. It contains more than 60,000 videos and 250 hours of content across eight languages, with fakes made by seven different generation models spanning talking-head synthesis, portrait animation, and face swapping. The paper's central finding is a generality gap: fine-tuned detectors score high in-domain, but lose a meaningful slice of accuracy on open-set splits, most sharply when generators are held out. The reason a reader should care is that real-world deepfake encounters are open-set by definition, so benchmarks that only test matched generators overstate a detector's practical value.

What carries the argument

The machine that carries the argument is the benchmark itself, MAVOS-DD, together with its four-way test split. Training and validation cover six languages and four generators; the in-domain test samples the same distribution, while open-set model adds three held-out generators, open-set language adds two held-out languages, and open-set full combines both. This split is what turns 'does the detector work?' into 'does the detector generalize?' by holding the training recipe fixed and varying only what is unseen at test time. The seven generation methods span talking-head synthesis, portrait animation, and face swapping, giving the open-set splits a realistic mix of forgery types.

What would settle it

Rebuild the open-set language test so the same talking-head generator checkpoint is used for all languages and face-swap source identities are excluded from the training real pool; if fine-tuned detector accuracy no longer drops, the claimed open-set failure is an artifact of per-language generator variants and identity overlap rather than genuine generalization.

Watch

Extended reading notes

Core claim

The paper's central claim is that MAVOS-DD is the first large-scale open-set benchmark for multilingual audio-video deepfake detection, and that state-of-the-art detectors cannot hold their performance under its open-set conditions. The evidence is a measured degradation: the strongest fine-tuned detector reaches 86.93% accuracy on the in-domain test set but 75.34% when fakes from unseen generators are added and 77.68% on the combined open-set full set; the other detectors show similar or larger relative declines. The paper argues this gap demonstrates that current models learn generator- and language-specific shortcuts rather than a robust notion of authenticity.

Load-bearing premise

The result depends on the real YouTube videos being genuinely authentic and the generated fakes, made with specific models and source identities, being representative enough that the drop on held-out languages and generators is due to generalization rather than to dataset artifacts.

Editorial extensions

If this is right

  • Fine-tuned detectors that exceed 90% accuracy on in-domain data fall to 75 to 79% accuracy on open-set model and full splits, so accuracy reported on matched generator and language benchmarks overstates deployment readiness.
  • The largest drops occur when unseen generators are introduced, indicating detectors latch onto generator-specific cues instead of a general forgery signal.
  • Unimodal video analysis lags behind audio-visual fusion on this benchmark, supporting the design of detectors that combine both modalities.
  • Pre-trained, off-the-shelf detectors perform near chance on MAVOS-DD, meaning the generated fakes are realistic enough to challenge existing public models.
  • MAVOS-DD provides a reusable stress test for future detectors and is designed to be extended with new generators as they are released.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not isolate whether the open-set language drop comes from language itself or from per-language generator checkpoints; a single generator checkpoint across all languages would separate the two.
  • Because face-swap source identities are drawn partly from the same real-video pool used in training, part of in-domain accuracy may reflect identity familiarity; a disjoint-identity split would test this.
  • The benchmark's open-set protocol could be extended to timestamp-level localization, which would reveal whether detectors rely on genuine artifacts rather than clip-level biases.
  • The near-chance performance of pre-trained detectors suggests that many existing public checkpoints are tuned to generator-specific traces; retraining with method augmentation or feature-level noise may close some of the gap.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces MAVOS-DD, a large-scale multilingual audio-video deepfake detection benchmark comprising 60,364 real and synthetic videos in eight languages, generated with seven modern deepfake methods (talking-head, portrait animation, and face swapping). The authors define training, validation, and test splits to create four evaluation scenarios: in-domain, open-set model, open-set language, and open-set full. They evaluate three recent detectors (AVFF, MRDF, TALL) in both pre-trained and fine-tuned form, reporting accuracy, AUC, and mAP. The central empirical claim is that state-of-the-art detectors degrade substantially when tested on unseen generators or languages, with the best fine-tuned detector (AVFF) dropping from 86.93% in-domain accuracy to 75.34% on the open-set model split and 77.68% on the open-set full split. The data and code are publicly released.

Significance. If the reported results are valid, MAVOS-DD is a valuable community resource: it is the largest multilingual audio-video deepfake benchmark with explicit open-set train/test splits, and it provides a concrete stress test that separates in-domain proficiency from cross-generator and cross-language generalization. The benchmark design, with seven generators spanning three generative tasks and a relatively balanced language distribution, addresses a real gap in existing datasets. The paper also ships a reproducible evaluation protocol (public data, code, and fine-tuning hyperparameters) and applies McNemar's test for the fine-tuning comparison, which strengthens the empirical section. The central qualitative finding—that detectors fine-tuned in-domain lose performance in open-set settings—is supported by the reported numbers, provided the splits are cleanly disjoint at the content level, which is not yet explicitly established.

major comments (4)
  1. [Section 3, 'Deepfake videos' and Table 2] The open-set generalization claims require that the train/validation/test splits are disjoint at the level of the underlying real video clips and their derived fake videos. The paper does not state this disjointness. Sections 3 states that talking-head generation uses audio that 'originates from the real video set', LivePortrait uses 'source videos are represented by the real videos collected from YouTube', and face swaps use 'target videos selected from the collected set of real YouTube videos', with source identities including 'identities found in our real videos'. Because Table 2 defines splits only by language and generator membership, a real clip (or a fake video derived from it) could appear in both the training set and the open-set test sets. If a detector sees the original real clip or its near-duplicate fake during fine-tuning, the reported drops, especially in the open-set model and open-set full scenarios, could partly reflect content leakage rather than generalization failure. The authors must either confirm that all splits are disjoint at the source/target video level, or revise the benchmark and the headline claims accordingly.
  2. [Section 3, 'Deepfake videos'] The per-language fine-tuning of EchoMimic changes the identity of the generator across languages. The text says that EchoMimic is 'individually fine-tune[d] on additional languages, such as Romanian and Arabic' before generation, with 1,000 real videos per language and 10 epochs. If EchoMimic is also fine-tuned on German and Hindi (the held-out languages) before generating their fakes, then the open-set language test does not present a single EchoMimic generator unseen at training time; it presents language-specific variants of EchoMimic. This confounds the 'open-set language' condition with a generator-variant condition. The authors should specify exactly which EchoMimic variant was used for each language and, if held-out languages used a differently fine-tuned model, report whether results change when excluding EchoMimic from the open-set language analysis.
  3. [Abstract and Section 3] The paper claims that '60% of data being generated', but Table 2 gives 35,169 fake videos out of 60,364 total, which is 58.3%. The 60% figure appears to be a rounding that could mislead readers; the exact fraction should be reported in the abstract and Section 3, or the dataset construction should be adjusted so the stated number matches.
  4. [Section 3, 'Real videos'] The benchmark's authenticity assumption for real videos is reasonable but should be stated more carefully. The real videos are scraped from YouTube and filtered by TalkNet and a face detector; there is no guarantee that some 'real' clips are not themselves deepfakes or contain manipulation. Given that the central claim is about generalization, the authors should acknowledge this as a residual risk and provide at least a basic manual or automated sanity check (e.g., a small annotation study) to estimate the contamination rate.
minor comments (5)
  1. [Section 2, paragraph after Table 1] There is a typo in 'MA VOD-DD' in the sentence 'To this end, MA VOD-DD contains samples ...' — should be 'MAVOS-DD'.
  2. [Section 4, 'Baselines and hyperparameters'] The phrase 'three state-of-the-art deepfake detectors' is written as 'thee state-of-the-art'; please fix the typo.
  3. [Figure 1] The figure would benefit from explicitly indicating which real videos are used as sources/targets for fake generation, to make the generation pipeline transparent and to support the split-disjointness claim.
  4. [Section 5, 'Broader Impact and Limitations'] The broader impact section focuses on hardware limitations and demographic bias, but it would be helpful to also mention the potential for benchmark-specific overfitting: detectors fine-tuned on MAVOS-DD may not generalize to other data, which is a general limitation of any benchmark.
  5. [Ethical Statement] The ethical statement mentions a license and user opt-out, but does not address consent for using individuals' faces in the dataset; consider discussing how identity rights are handled beyond the right-to-be-forgotten request.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the benchmark is constructed independently of the evaluated detectors, and the reported performance drops are measured outcomes rather than derived quantities.

full rationale

The paper's central claims are empirical: it constructs a multilingual open-set deepfake detection benchmark and measures how three external detectors perform on it. There is no derivation chain in which an input is defined in terms of an output or a fitted parameter is renamed as a prediction. The detectors (AVFF, MRDF, TALL) are prior published methods, used as baselines; their fine-tuning uses the training split, and the reported in-domain versus open-set accuracy values in Table 3 are experimental measurements, not consequences of the benchmark's definitions. The splits are defined by language and generative-model membership (Table 2), and the open-set test sets are constructed by adding unseen languages and unseen generators. No load-bearing step reduces to a self-citation: the authors' self-citations are literature surveys and are not used to justify the benchmark's validity or the measured results. A separate concern, raised in the accompanying skeptic note, is that the fake videos are generated from real YouTube clips that may overlap across splits at the level of source or target content, which would threaten the openness of the evaluation. However, that is a dataset-construction validity issue, not circular reasoning: even if such leakage existed, it would make the reported drops less clean without making the conclusion equivalent to its inputs by construction. Therefore, the appropriate circularity score is 0.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central claim rests on dataset construction choices rather than a mathematical derivation. The free parameters are hand-set generation and filtering settings that shape the fake video distribution. The axioms are domain assumptions about authenticity, representativeness, and the absence of shortcut cues. No invented entities are introduced.

free parameters (4)
  • EchoMimic per-language fine-tuning epochs = 10
    Hand-set; makes the EchoMimic model language-specific, a potential confound in open-set language evaluation.
  • FLUX denoising steps = 50
    Hand-set for portrait generation; affects realism and diversity of source identities.
  • FLUX guidance scale = 3.5
    Hand-set CFG scale for portraits; affects generation diversity.
  • Face detection frame step = 15
    Hand-set sampling interval for filtering real videos without faces; affects the real video pool.
assumptions (3)
  • domain assumption The seven generation models are representative of current high-quality deepfake tools.
    The difficulty of the benchmark rests on this; the authors select 'based on the quality of the generated content' without quantitative criteria.
  • domain assumption Real YouTube videos are authentic and unmanipulated.
    Labels for the real class depend on this; no verification beyond face and speaker detection.
  • domain assumption Open-set splits expose generalization rather than dataset-specific artifacts.
    Assumes no shortcuts from resolution, compression, audio codec, or language-dependent features; the paper does not analyze these.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MAVOS-DD: Multilingual Audio-Video Open-Set Deepfake Detection Benchmark." pith.science (2026). https://pith.science/paper/OGJMVTWR

@misc{pith2026250511109,
  author       = {Pith},
  title        = {Pith review of: MAVOS-DD: Multilingual Audio-Video Open-Set Deepfake Detection Benchmark},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OGJMVTWR}},
  note         = {Machine review of arXiv:2505.11109}
}
read the original abstract

We present the first large-scale open-set benchmark for multilingual audio-video deepfake detection. Our dataset comprises over 250 hours of real and fake videos across eight languages, with 60% of data being generated. For each language, the fake videos are generated with seven distinct deepfake generation models, selected based on the quality of the generated content. We organize the training, validation and test splits such that only a subset of the chosen generative models and languages are available during training, thus creating several challenging open-set evaluation setups. We perform experiments with various pre-trained and fine-tuned deepfake detectors proposed in recent literature. Our results show that state-of-the-art detectors are not currently able to maintain their performance levels when tested in our open-set scenarios. We publicly release our data and code at: https://huggingface.co/datasets/unibuc-cs/MAVOS-DD.

Figures

Figures reproduced from arXiv: 2505.11109 by the authors.

Figure 1
Figure 1. In MAVOS-DD, the training set and in-domain test set contain real and fake videos sampled from the same distribution, comprising six languages and four generative models. The open-set model test set extends the in-domain test set with fake samples generated by unseen models (Sonic, HifiFace, Roop). The open-set language test set extends the in-domain test set with samples in unseen languages (German and Hindi). The … view at source ↗
Figure 2
Figure 2. Distribution of videos per language and per generative method. MAVOS-DD comprises [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 1
Figure 1. The first scenario, referred to as in-domain evaluation, uses a test set comprising the same 4 [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figures from the paper (3 more)
Figure 3
Figure 3. Figure 3: Fake video frames generated by each of the seven deepfake methods. Best viewed in color. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png]
Figure 4
Figure 4. Figure 4: Confusion matrices obtained by AVFF, MRDF and TALL after fine-tuning them on [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: A real video and its corresponding fake sample generated using LivePortrait. The MRDF [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Foundation Models are Implicit Deepfake Detectors

    cs.CV 2026-08 conditional novelty 6.0 of 10

    Fake images and videos systematically produce lower-magnitude features in pretrained models, so a simple norm-based score detects deepfakes without training a classifier.

  2. Detecting AI-Generated Video: A Vision-Language Dual-View Survey

    cs.CV 2026-07 conditional novelty 6.0 of 10

    AIGC-V detection should be treated as factual fidelity verification and organized by a four-layer vision-language dual-view taxonomy spanning cues, motion, cross-modal consistency, and world-level reasoning.

Reference graph

Works this paper leans on

80 extracted references · 70 canonical work pages · cited by 2 Pith papers

  1. [1]

    Detecting deep-fake videos from phoneme-viseme mismatches

    Shruti Agarwal, Hany Farid, Ohad Fried, and Maneesh Agrawala. Detecting deep-fake videos from phoneme-viseme mismatches. InProceedings of CVPR, pages 660–661, 2020

    work page 2020

  2. [2]

    Combin- ing automatic speaker verification and prosody analysis for synthetic speech detection

    Luigi Attorresi, Davide Salvi, Clara Borrelli, Paolo Bestagini, and Stefano Tubaro. Combin- ing automatic speaker verification and prosody analysis for synthetic speech detection. In Proceedings of ICPR, pages 247–263, 2023

    work page 2023

  3. [3]

    Exposing the deception: Uncovering more forgery clues for deepfake detection

    Zhongjie Ba, Qingyu Liu, Zhenguang Liu, Shuang Wu, Feng Lin, Li Lu, and Kui Ren. Exposing the deception: Uncovering more forgery clues for deepfake detection. InProceedings of AAAI, pages 719–728, 2024

    work page 2024

  4. [4]

    DeepSpeak Dataset v1.0.arXiv preprint arXiv:2408.05366, 2024

    Sarah Barrington, Matyas Bohacek, and Hany Farid. DeepSpeak Dataset v1.0.arXiv preprint arXiv:2408.05366, 2024

    arXiv 2024

  5. [5]

    Who Are You (I Really Wanna Know)? Detecting Audio DeepFakes Through V ocal Tract Reconstruction

    Logan Blue, Kevin Warren, Hadi Abdullah, Cassidy Gibson, Luis Vargas, Jessica O’Dell, Kevin Butler, and Patrick Traynor. Who Are You (I Really Wanna Know)? Detecting Audio DeepFakes Through V ocal Tract Reconstruction. InProceedings of USENIX, pages 2691–2708, 2022

    work page 2022

  6. [6]

    Video face manipulation detection through ensemble of CNNs

    Nicolo Bonettini, Edoardo Daniele Cannas, Sara Mandelli, Luca Bondi, Paolo Bestagini, and Stefano Tubaro. Video face manipulation detection through ensemble of CNNs. InProceedings of ICPR, pages 5012–5019, 2021

    work page 2021

  7. [7]

    Deepfake-Eval-2024: A Multi-Modal In-the-Wild Benchmark of Deepfakes Circulated in 2024.arXiv preprint arXiv:2503.02857, 2025

    Nuria Alina Chandra, Ryan Murtfeldt, Lin Qiu, Arnab Karmakar, Hannah Lee, Emmanuel Tanumihardja, Kevin Farhat, Ben Caffee, Sejin Paik, Changyeon Lee, Jongwook Choi, Aerin Kim, and Oren Etzioni. Deepfake-Eval-2024: A Multi-Modal In-the-Wild Benchmark of Deepfakes Circulated in 2024.arXiv preprint arXiv:2503.02857, 2025

    arXiv 2024

  8. [8]

    SimSwap: An Efficient Framework For High Fidelity Face Swapping

    Renwang Chen, Xuanhong Chen, Bingbing Ni, and Yanhao Ge. SimSwap: An Efficient Framework For High Fidelity Face Swapping. InProceedings of ACMMM, pages 2003–2011, 2020

    work page 2003

Show all 80 references
  1. [9]

    EchoMimic: Life- like Audio-Driven Portrait Animations through Editable Landmark Conditions

    Zhiyuan Chen, Jiajiong Cao, Zhiquan Chen, Yuming Li, and Chenguang Ma. EchoMimic: Life- like Audio-Driven Portrait Animations through Editable Landmark Conditions. InProceedings of AAAI, pages 2403–2410, 2024

    2024

  2. [10]

    DiffusionFace: Towards a Comprehensive Dataset for Diffusion-Based Face Forgery Analysis

    Zhongxi Chen, Ke Sun, Ziyin Zhou, Xianming Lin, Xiaoshuai Sun, Liujuan Cao, and Rongrong Ji. DiffusionFace: Towards a Comprehensive Dataset for Diffusion-Based Face Forgery Analysis. arXiv preprint arXiv:2403.18471, 2024

    2024 arXiv

  3. [11]

    Exploit- ing Style Latent Flows for Generalizing Deepfake Video Detection

    Jongwook Choi, Taehoon Kim, Yonghyun Jeong, Seungryul Baek, and Jongwon Choi. Exploit- ing Style Latent Flows for Generalizing Deepfake Video Detection. InProceedings of CVPR, pages 1133–1143, 2024

    2024

  4. [12]

    Deepfake Detection by Exploiting Surface Anomalies: The Surfake Approach

    Andrea Ciamarra, Roberto Caldelli, Federico Becattini, Lorenzo Seidenari, and Alberto Del Bimbo. Deepfake Detection by Exploiting Surface Anomalies: The Surfake Approach. In Proceedings of WACV, pages 1024–1033, 2024

    2024

  5. [13]

    Com- bining EfficientNet and Vision Transformers for Video Deepfake Detection

    Davide Alessandro Coccomini, Nicola Messina, Claudio Gennaro, and Fabrizio Falchi. Com- bining EfficientNet and Vision Transformers for Video Deepfake Detection. InProceedings of ICIAP, pages 219–229, 2022

    2022

  6. [14]

    Audio-visual person-of-interest deepfake detection

    Davide Cozzolino, Alessandro Pianese, Matthias Nießner, and Luisa Verdoliva. Audio-visual person-of-interest deepfake detection. InProceedings of CVPR, pages 943–952, 2023

    2023

  7. [15]

    Diffusion Models in Vision: A Survey.IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(9):10850–10869, 2023

    Florinel-Alin Croitoru, Vlad Hondru, Radu Tudor Ionescu, and Mubarak Shah. Diffusion Models in Vision: A Survey.IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(9):10850–10869, 2023. 10

    2023

  8. [16]

    Deepfake Media Generation and Detection in the Generative AI Era: A Survey and Outlook

    Florinel-Alin Croitoru, Andrei-Iulian Hiji, Vlad Hondru, Nicolae Catalin Ristea, Paul Irofti, Marius Popescu, Cristian Rusu, Radu Tudor Ionescu, Fahad Shahbaz Khan, and Mubarak Shah. Deepfake Media Generation and Detection in the Generative AI Era: A Survey and Outlook. arXiv ...

    2024 arXiv

  9. [17]

    On the detection of digital face manipulation

    Hao Dang, Feng Liu, Joel Stehouwer, Xiaoming Liu, and Anil K Jain. On the detection of digital face manipulation. InProceedings of CVPR, pages 5781–5790, 2020

    2020

  10. [18]

    The DeepFake Detection Challenge (DFDC) Dataset.arXiv preprint arXiv:2006.07397, 2020

    Brian Dolhansky, Joanna Bitton, Ben Pflaum, Jikuo Lu, Russ Howes, Menglin Wang, and Cristian Canton Ferrer. The DeepFake Detection Challenge (DFDC) Dataset.arXiv preprint arXiv:2006.07397, 2020

    2006 arXiv

  11. [19]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, et al. An image is worth 16x16 words: Transformers for image recognition at scale. InProceedings of ICLR, 2021

    2021

  12. [20]

    Subband fusion of complex spectrogram for fake speech detection.Speech Communication, 155:102988, 2023

    Cunhang Fan, Jun Xue, Shunbo Dong, Mingming Ding, Jiangyan Yi, Jinpeng Li, and Zhao Lv. Subband fusion of complex spectrogram for fake speech detection.Speech Communication, 155:102988, 2023

    2023

  13. [21]

    Deepfake Detection Model Combining Texture Differences and Frequency Domain Information.ACM Transactions on Privacy and Security, 28(2):21, 2025

    Shuaijv Fang, Zhiyong Zhang, and Bin Song. Deepfake Detection Model Combining Texture Differences and Frequency Domain Information.ACM Transactions on Privacy and Security, 28(2):21, 2025

    2025

  14. [22]

    Generalized Spoofing Detection Inspired from Audio Generation Artifacts

    Yang Gao, Tyler Vuong, Mahsa Elyasi, Gaurav Bharaj, and Rita Singh. Generalized Spoofing Detection Inspired from Audio Generation Artifacts. InProceedings of INTERSPEECH, pages 4184–4188, 2021

    2021

  15. [23]

    Gerstner and Hany Farid

    Candice R. Gerstner and Hany Farid. Detecting real-time deep-fake videos using active illumination. InProceedings of CVPR, pages 53–60, 2022

    2022

  16. [24]

    Delving into sequential patches for deepfake detection

    Jiazhi Guan, Hang Zhou, Zhibin Hong, Errui Ding, Jingdong Wang, Chengbin Quan, and Youjian Zhao. Delving into sequential patches for deepfake detection. InProceedings of NeurIPS, pages 4517–4530, 2022

    2022

  17. [25]

    LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Control

    Jianzhu Guo, Dingyun Zhang, Xiaoqiang Liu, Zhizhou Zhong, Yuan Zhang, Pengfei Wan, and Di Zhang. LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Control. arXiv preprint arxiv:2407.03168, 2024

    2024 arXiv

  18. [26]

    Leveraging real talking faces via self-supervision for robust forgery detection

    Alexandros Haliassos, Rodrigo Mira, Stavros Petridis, and Maja Pantic. Leveraging real talking faces via self-supervision for robust forgery detection. InProceedings of CVPR, pages 14930–14942, 2022

    2022

  19. [27]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InProceedings of CVPR, pages 770–778, 2016

    2016

  20. [28]

    ForgeryNet: A Versatile Benchmark for Comprehensive Forgery Analysis

    Yinan He, Bei Gan, Siyu Chen, Yichun Zhou, Guojun Yin, Luchuan Song, Lu Sheng, Jing Shao, and Ziwei Liu. ForgeryNet: A Versatile Benchmark for Comprehensive Forgery Analysis. In Proceedings of CVPR, pages 4360–4369, 2021

    2021

  21. [29]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Proceedings of NeurIPS, volume 33, pages 6840–6851, 2020

    2020

  22. [30]

    HeadNeRF: A Real-time NeRF-based Parametric Head Model

    Yang Hong, Bo Peng, Haiyao Xiao, Ligang Liu, and Juyong Zhang. HeadNeRF: A Real-time NeRF-based Parametric Head Model. InProceedings of CVPR, pages 20374–20384, 2022

    2022

  23. [31]

    A VFakeNet: A unified end-to-end Dense Swin Transformer deep learning model for audio-visual deepfakes detection.Applied Soft Computing, page 110124, 2023

    Hafsa Ilyas, Ali Javed, and Khalid Mahmood Malik. A VFakeNet: A unified end-to-end Dense Swin Transformer deep learning model for audio-visual deepfakes detection.Applied Soft Computing, page 110124, 2023

    2023

  24. [32]

    Sonic: Shifting focus to global audio perception in portrait animation

    Xiaozhong Ji, Xiaobin Hu, Zhihong Xu, Junwei Zhu, Chuming Lin, Qingdong He, Jiangning Zhang, Donghao Luo, Yi Chen, Qin Lin, Qinglin Lu, and Chengjie Wang. Sonic: Shifting focus to global audio perception in portrait animation. InProceedings of CVPR, 2025. 11

    2025

  25. [33]

    DeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detection

    Liming Jiang, Ren Li, Wayne Wu, Chen Qian, and Chen Change Loy. DeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detection. InProceedings of CVPR, pages 2886–2895, 2020

    2020

  26. [34]

    Ultralytics YOLO, January 2023

    Glenn Jocher, Jing Qiu, and Ayush Chaurasia. Ultralytics YOLO, January 2023. URLhttps: //github.com/ultralytics/ultralytics

    2023

  27. [35]

    Exemplar Fine-Tuning for 3D Human Model Fitting Towards In-the-Wild 3D Human Pose Estimation

    Hanbyul Joo, Natalia Neverova, and Andrea Vedaldi. Exemplar Fine-Tuning for 3D Human Model Fitting Towards In-the-Wild 3D Human Pose Estimation. InProceedings of IC3DV, pages 42–52, 2021

    2021

  28. [36]

    A style-based generator architecture for generative adversarial networks.IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(12): 4217–4228, 2021

    Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks.IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(12): 4217–4228, 2021

    2021

  29. [37]

    Hasam Khalid, Shahroz Tariq, Minha Kim, and Simon S. Woo. FakeA VCeleb: A novel audio-video multimodal deepfake dataset. InProceedings of NeurIPS, 2021

    2021

  30. [38]

    Robust multimedia spam filtering based on visual, textual, and audio deep features and random forest.Multimedia Tools and Applications, 82(26):40819– 40837, 2023

    Marouane Kihal and Lamia Hamza. Robust multimedia spam filtering based on visual, textual, and audio deep features and random forest.Multimedia Tools and Applications, 82(26):40819– 40837, 2023

    2023

  31. [39]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. InProceedings of ICLR, 2015

    2015

  32. [40]

    LBPNet: Exploiting texture descriptor for deepfake detection.F orensic Science International: Digital Investigation, 42–43:301452, 2022

    Staffy Kingra, Naveen Aggarwal, and Nirmal Kaur. LBPNet: Exploiting texture descriptor for deepfake detection.F orensic Science International: Digital Investigation, 42–43:301452, 2022

    2022

  33. [41]

    DeepFakes: a New Threat to Face Recognition? Assessment and Detection.arXiv preprint arXiv:1812.08685, 2018

    Pavel Korshunov and Sébastien Marcel. DeepFakes: a New Threat to Face Recognition? Assessment and Detection.arXiv preprint arXiv:1812.08685, 2018

    2018 arXiv

  34. [42]

    Faster than lies: Real-time deepfake detection using binary neural networks

    Romeo Lanzino, Federico Fontana, Anxhelo Diko, Marco Raoul Marini, and Luigi Cinque. Faster than lies: Real-time deepfake detection using binary neural networks. InProceedings of CVPR, pages 3771–3780, 2024

    2024

  35. [43]

    Maskgan: Towards diverse and interactive facial image manipulation

    Cheng-Han Lee, Ziwei Liu, Lingyun Wu, and Ping Luo. Maskgan: Towards diverse and interactive facial image manipulation. InProceedings of CVPR, pages 5548–5557, 2020

    2020

  36. [44]

    Advancing High Fidelity Identity Swapping for Forgery Detection

    Lingzhi Li, Jianmin Bao, Hao Yang, Dong Chen, and Fang Wen. Advancing High Fidelity Identity Swapping for Forgery Detection. InProceedings of CVPR, pages 5073–5082, 2020

    2020

  37. [45]

    Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensics

    Yuezun Li, Pu Sun, Honggang Qi, and Siwei Lyu. Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensics. InProceedings of CVPR, pages 3204–3213, 2020

    2020

  38. [46]

    ASVspoof 2021: Towards Spoofed and Deepfake Speech Detection in the Wild.IEEE/ACM Transactions on Audio, Speech, and Language Processing, 31:2507–2522, 2023

    Xuechen Liu, Xin Wang, Md Sahidullah, Jose Patino, Héctor Delgado, Tomi Kinnunen, Mas- similiano Todisco, Junichi Yamagishi, Nicholas Evans, Andreas Nautsch, and Kong Aik Lee. ASVspoof 2021: Towards Spoofed and Deepfake Speech Detection in the Wild.IEEE/ACM Transactions on Aud...

    2021

  39. [47]

    Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin Transformer: Hierarchical Vision Transformer using Shifted Windows. InProceedings of ICCV, pages 9992–10002, 2021

    2021

  40. [48]

    The Vicomtech Partial Deepfake Detection and Location System for the 2023 ADD Challenge

    Juan Manuel Martín-Doñas and Aitor Álvarez. The Vicomtech Partial Deepfake Detection and Location System for the 2023 ADD Challenge. InProceedings of IJCAI, pages 37–42, 2023

    2023

  41. [49]

    GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models

    Alexander Quinn Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob Mcgrew, Ilya Sutskever, and Mark Chen. GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models. InProceedings of ICML, pages 16784–16804, 2022

    2022

  42. [50]

    Frade: Forgery-aware audio-distilled multimodal learning for deepfake detection

    Fan Nie, Jiangqun Ni, Jian Zhang, Bin Zhang, and Weizhe Zhang. Frade: Forgery-aware audio-distilled multimodal learning for deepfake detection. InProceedings of ACMMM, page 6297–6306, 2024. 12

    2024

  43. [51]

    FSGAN: Subject Agnostic Face Swapping and Reenactment

    Yuval Nirkin, Yosi Keller, and Tal Hassner. FSGAN: Subject Agnostic Face Swapping and Reenactment. InProceedings of ICCV, pages 7184–7193, 2019

    2019

  44. [52]

    A VFF: Audio-Visual Feature Fusion for Video Deepfake Detection

    Trevine Oorloff, Surya Koppisetti, Nicolò Bonettini, Divyaraj Solanki, Ben Colman, Yaser Yacoob, Ali Shahriyari, and Gaurav Bharaj. A VFF: Audio-Visual Feature Fusion for Video Deepfake Detection. InProceedings of CVPR, pages 27102–27112, 2024

    2024

  45. [53]

    Hierarchical Text-Conditional Image Generation with CLIP Latents.arXiv preprint arXiv:2204.06125, 2022

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical Text-Conditional Image Generation with CLIP Latents.arXiv preprint arXiv:2204.06125, 2022

    2022 arXiv

  46. [54]

    Multimodaltrace: Deepfake Detection Using Audiovisual Representation Learning

    Muhammad Anas Raza and Khalid Mahmood Malik. Multimodaltrace: Deepfake Detection Using Audiovisual Representation Learning. InProceedings of CVPR, pages 993–1000, 2023

    2023

  47. [55]

    High- Resolution Image Synthesis with Latent Diffusion Models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- Resolution Image Synthesis with Latent Diffusion Models. InProceedings of CVPR, pages 10684–10695, 2022

    2022

  48. [56]

    FaceForensics++: Learning to Detect Manipulated Facial Images

    Andreas Rossler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, and Matthias Nießner. FaceForensics++: Learning to Detect Manipulated Facial Images. InProceedings of ICCV, pages 1–11, 2019

    2019

  49. [57]

    Photorealistic text-to-image diffusion models with deep language understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, Jonathan Ho, David Fleet, and Mohammad Norouzi. Photorealistic text-to-image diffusion models with deep language unde...

    2022

  50. [58]

    A Robust Approach to Multimodal Deepfake Detection.Journal of Imaging, 9 (6), 2023

    Davide Salvi, Honggu Liu, Sara Mandelli, Paolo Bestagini, Wenbo Zhou, Weiming Zhang, and Stefano Tubaro. A Robust Approach to Multimodal Deepfake Detection.Journal of Imaging, 9 (6), 2023

    2023

  51. [59]

    Generative modeling by estimating gradients of the data distribution

    Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. InProceedings of NeurIPS, pages 11918–11930, 2019

    2019

  52. [60]

    Investigation of Sub-Band Discriminative Information Between Spoofed and Genuine Speech

    Kaavya Sriskandaraja, Vidhyasaharan Sethu, Phu Ngoc Le, and Eliathamby Ambikairajah. Investigation of Sub-Band Discriminative Information Between Spoofed and Genuine Speech. InProceedings of INTERSPEECH, pages 1710–1714, 2016

    2016

  53. [61]

    Is Someone Speaking? Exploring Long-term Temporal Features for Audio-visual Active Speaker Detection

    Ruijie Tao, Zexu Pan, Rohan Kumar Das, Xinyuan Qian, Mike Zheng Shou, and Haizhou Li. Is Someone Speaking? Exploring Long-term Temporal Features for Audio-visual Active Speaker Detection. InProceedings of ACMMM, pages 3927–3935, 2021

    2021

  54. [62]

    EMO: Emote Portrait Alive Generating Expressive Portrait Videos with Audio2Video Diffusion Model Under Weak Conditions

    Linrui Tian, Qi Wang, Bang Zhang, and Liefeng Bo. EMO: Emote Portrait Alive Generating Expressive Portrait Videos with Audio2Video Diffusion Model Under Weak Conditions. In Proceedings of ECCV, pages 244–260, 2024

    2024

  55. [63]

    Detection of Cross-Dataset Fake Audio Based on Prosodic and Pronunciation Features

    Chenglong Wang, Jiangyan Yi, Jianhua Tao, Chu Yuan Zhang, Shuai Zhang, and Xun Chen. Detection of Cross-Dataset Fake Audio Based on Prosodic and Pronunciation Features. In Proceedings of INTERSPEECH, pages 3844–3848, 2023

    2023

  56. [64]

    EAT-Face: Emotion-Controllable Audio-Driven Talking Face Generation via Diffusion Model

    Haodi Wang, Xiaojun Jia, and Xiaochun Cao. EAT-Face: Emotion-Controllable Audio-Driven Talking Face Generation via Diffusion Model. InProceedings of FG, pages 1–10, 2024

    2024

  57. [65]

    Noise Based Deepfake Detection via Multi-Head Relative- Interaction

    Tianyi Wang and Kam Pui Chow. Noise Based Deepfake Detection via Multi-Head Relative- Interaction. InProceedings of AAAI, pages 14548–14556, 2023

    2023

  58. [66]

    ASVspoof 2019: A large-scale public database of synthesized, converted and replayed speech.Computer Speech & Language, 64:101114, 2020

    Xin Wang, Junichi Yamagishi, Massimiliano Todisco, Hector Delgado, Andreas Nautsch, Nicholas Evans, Md Sahidullah, Ville Vestman, Tomi Kinnunen, Kong Aik Lee, Lauri Juvela, Paavo Alku, Yu-Huai Peng, Hsin-Te Hwang, Yu Tsao, Hsin-Min Wang, Sebastien Le Maguer, Markus Becker, Fer...

    2019

  59. [67]

    Towards Real-World Blind Face Restoration with Generative Facial Prior

    Xintao Wang, Yu Li, Honglun Zhang, and Ying Shan. Towards Real-World Blind Face Restoration with Generative Facial Prior. InProceedings of CVPR, pages 9164–9174, 2021

    2021

  60. [68]

    HifiFace: 3D Shape and Semantic Prior Guided High Fidelity Face Swapping

    Yuhan Wang, Xu Chen, Junwei Zhu, Wenqing Chu, Ying Tai, Chengjie Wang, Jilin Li, Yongjian Wu, Feiyue Huang, and Rongrong Ji. HifiFace: 3D Shape and Semantic Prior Guided High Fidelity Face Swapping. InProceedings of IJCAI, pages 1136–1142, 2021

    2021

  61. [69]

    Hallo: Hierarchical audio-driven visual synthesis for portrait image animation.arXiv preprint arXiv:2406.08801, 2024

    Mingwang Xu, Hui Li, Qingkun Su, Hanlin Shang, Liwei Zhang, Ce Liu, Jingdong Wang, Yao Yao, and Siyu Zhu. Hallo: Hierarchical audio-driven visual synthesis for portrait image animation.arXiv preprint arXiv:2406.08801, 2024

    2024 arXiv

  62. [70]

    Vasa-1: Lifelike audio-driven talking faces generated in real time

    Sicheng Xu, Guojun Chen, Yu-Xiao Guo, Jiaolong Yang, Chong Li, Zhenyu Zang, Yizhong Zhang, Xin Tong, and Baining Guo. Vasa-1: Lifelike audio-driven talking faces generated in real time. InProceedings of NeurIPS, pages 660–684, 2024

    2024

  63. [71]

    TALL: Thumbnail Layout for Deepfake Video Detection

    Yuting Xu, Jian Liang, Gengyun Jia, Ziming Yang, Yanhao Zhang, and Ran He. TALL: Thumbnail Layout for Deepfake Video Detection. InProceedings of ICCV, pages 22601–22611, 2023

    2023

  64. [72]

    Audio Deepfake Detection Based on a Combination of F0 Information and Real Plus Imaginary Spectrogram Features

    Jun Xue, Cunhang Fan, Zhao Lv, Jianhua Tao, Jiangyan Yi, Chengshi Zheng, Zhengqi Wen, Minmin Yuan, and Shegang Shao. Audio Deepfake Detection Based on a Combination of F0 Information and Real Plus Imaginary Spectrogram Features. InProceedings of DDAM, pages 19–26, 2022

    2022

  65. [73]

    Significance of Subband Features for Synthetic Speech Detection.IEEE Transactions on Information F orensics and Security, 15:2160–2170, 2020

    Jichen Yang, Rohan Kumar Das, and Haizhou Li. Significance of Subband Features for Synthetic Speech Detection.IEEE Transactions on Information F orensics and Security, 15:2160–2170, 2020

    2020

  66. [74]

    Yibo Zhang, Weiguo Lin, and Junfeng Xu. Joint Audio-Visual Attention with Contrastive Learning for More General Deepfake Detection.ACM Transactions on Multimedia Computing, Communications and Applications, 20(5):137, 2024

    2024

  67. [75]

    MEMO: Memory-Guided Diffusion for Expressive Talking Video Generation.arXiv preprint arXiv:2412.04448, 2024

    Longtao Zheng, Yifan Zhang, Hanzhong Guo, Jiachun Pan, Zhenxiong Tan, Jiahao Lu, Chuanxin Tang, Bo An, and Shuicheng Yan. MEMO: Memory-Guided Diffusion for Expressive Talking Video Generation.arXiv preprint arXiv:2412.04448, 2024

    2024 arXiv

  68. [76]

    Exploring temporal coherence for more general video face forgery detection

    Yinglin Zheng, Jianmin Bao, Dong Chen, Ming Zeng, and Fang Wen. Exploring temporal coherence for more general video face forgery detection. InProceedings of ICCV, pages 15024–15034, 2021

    2021

  69. [77]

    Bühler, Xu Chen, Michael J

    Yufeng Zheng, Victoria Fernández Abrevaya, Marcel C. Bühler, Xu Chen, Michael J. Black, and Otmar Hilliges. IMavatar: Implicit Morphable Head Avatars from Videos. InProceedings of CVPR, pages 13545–13555, 2022

    2022

  70. [78]

    Joint Audio-Visual Deepfake Detection

    Yipin Zhou and Ser-Nam Lim. Joint Audio-Visual Deepfake Detection. InProceedings of ICCV, pages 14800–14809, 2021

    2021

  71. [79]

    WildDeepfake: A Challenging Real-World Dataset for Deepfake Detection

    Bojia Zi, Minghao Chang, Jingjing Chen, Xingjun Ma, and Yu-Gang Jiang. WildDeepfake: A Challenging Real-World Dataset for Deepfake Detection. InProceedings of ACMMM, pages 2382–2390, 2020

    2020

  72. [80]

    Cross- modality and within-modality regularization for audio-visual deepfake detection

    Heqing Zou, Meng Shen, Yuchen Hu, Chen Chen, Eng Siong Chng, and Deepu Rajan. Cross- modality and within-modality regularization for audio-visual deepfake detection. InProceedings of ICASSP, pages 4900–4904, 2024. A Ethical Statement We share MA VOS-DD under the International ...

    2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.