REVIEW 4 major objections 6 minor 19 references
Hindi audio-video-Deepfake (HAV-DF): A Hindi language-based Audio-video Deepfake Dataset
T0 review · 4 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read HAV-DF, a new Hindi audio-video deepfake dataset, yields lower detection accuracy across seven classifiers than the English-language FF-DF and DFDC benchmarks.
desk verdict A promising Hindi deepfake dataset paper, but the headline language-difficulty claim is under-supported and the data isn't public yet. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The carrying mechanism is the three-stage generation pipeline: face swap using FSGAN, FaceSwap, and DeepFaceLab; lip-sync using ReTalking to align mouth movements with the audio track; and voice cloning using the RVC model to produce manipulated Hindi speech. The dataset is organized into four sets: real audio with real video ($R_a+R_v$), fake audio with real video ($F_a+R_v$, 78 videos), real audio with fake video ($R_a+F_v$, 130 videos), and fake audio with fake video ($F_a+F_v$, 100 videos), which together allow both unimodal and multimodal deepfake evaluation.
What would settle it
Run the same seven detectors on a matched English-language deepfake corpus generated with the same pipeline (FSGAN, ReTalking, RVC) and matched in resolution, duration, frame rate, video count, and training/test split. If the English corpus yields accuracy comparable to HAV-DF, the central claim that Hindi content drives the difficulty is falsified.
Extended reading notes
Core claim
On its own terms, the paper's claim is that HAV-DF is a new benchmark resource that is harder for current detectors than existing English-centric datasets. On detection methods such as HeadPose, Two-stream, Mesonet, Xception-c23, Multi-task, FWA, and Xception-c40, HAV-DF attains the lowest accuracy among the compared audio-video datasets, with its best result at 64.7% using Xception-c40 versus 99.7% for FF-DF using Xception-c23. The authors attribute this gap to the Hindi-language focus and to the combination of face-swapped, lip-synced, and voice-cloned manipulations, and they position the dataset as the first Hindi-language audio-video deepfake corpus, filling a gap left by video-only and audio-only datasets.
Load-bearing premise
The claim that Hindi makes deepfakes harder to detect rests on the assumption that the lower accuracies on HAV-DF are caused by the language and the diversity of manipulations, not by differences in generation tools, video count, resolution, compression, or evaluation protocol.
Editorial extensions
If this is right
- Existing detectors evaluated on HAV-DF score below their performance on FF-DF and DFDC, indicating that the benchmark is a harder test set for current English-oriented methods.
- HAV-DF supplies a resource for training and evaluating models that must detect deepfakes in a low-resource language with audio and video manipulated simultaneously.
- Because the fake videos include both celebrity and non-celebrity Hindi speakers across news, technology, and podcast content, models that work on HAV-DF can be tested for generalization across varied Hindi speech and appearance.
- The three manipulation combinations ($F_a+R_v$, $R_a+F_v$, $F_a+F_v$) allow a detector to be assessed separately on audio-only, video-only, and joint audio-video forgery.
- The reported high SSIM scores indicate that most generated videos are visually close to their real counterparts, so the lower detection accuracy is not simply an artifact of obvious visual flaws.
Reading between the lines
- A direct test of the language hypothesis would generate an English-language set with the same FSGAN, ReTalking, and RVC pipeline, matched in resolution, duration, frame rate, and video count; if accuracies match HAV-DF, the gap comes from the generation pipeline rather than from Hindi itself.
- If Hindi-specific phonetics and gestures drive the difficulty, detectors fine-tuned on HAV-DF should transfer more strongly to other Indic languages than to typologically distant languages, a claim the paper does not test.
- The dataset's three manipulation splits support an ablation the paper leaves implicit: measuring how much of the accuracy drop comes from audio cloning, video swapping, and their combination.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces HAV-DF, a Hindi-language audio-video deepfake dataset containing real and manipulated videos generated by face-swapping, lip-sync, and voice cloning. It reports benchmark accuracies of several deepfake detectors on HAV-DF and compares them with published accuracies on English-centric datasets, concluding that HAV-DF is harder to detect and that this is possibly due to Hindi-language content. The paper also claims HAV-DF is the first audio-video deepfake dataset of its kind and includes qualitative samples, SSIM analysis, and a discussion of limitations.
Significance. If the dataset is publicly released and the comparison claims are properly controlled, HAV-DF would fill a genuine gap: no audio-video deepfake dataset exists specifically for Hindi, a major low-resource language with large at-risk populations. The paper's strengths include a reasonable generation pipeline, explicit selection criteria, and an attempt to benchmark multiple detectors. However, the quantitative claim that Hindi language causes lower detection accuracy is not established by the current experiments because the cross-dataset comparison lacks a matched English baseline and does not control for generation pipeline, dataset size, or evaluation protocol. The dataset counts are also internally inconsistent, and the data are not publicly available, limiting reproducibility.
major comments (4)
- [§3.1, §4.1.2, §5.2, Table 1, §6] The paper reports inconsistent dataset sizes: §3.1 says 508 videos (200 real + 308 fake); §4.1.2 lists 308 fake (78 Fa+Rv, 130 Ra+Fv, 100 Fa+Fv); §5.2 says 408 training / 100 test (total 508); but §6 and the Limitations say 500 videos, Table 1 reports 100/400, and §3.4.4 says '350 videos' were generated. These inconsistencies make it impossible to understand the actual dataset composition and must be corrected before the dataset can be used as a benchmark.
- [§5.2.2, Fig. 13, Tables 3–4] The central claim that HAV-DF is harder to detect than FF-DF and DFDC is based on an uncontrolled cross-dataset comparison. The paper does not state whether detectors were trained from scratch on HAV-DF, fine-tuned, or applied as pre-trained models, nor does it account for the order-of-magnitude differences in training set size (HAV-DF: 408 training videos; FF++: 1000 real/4000 fake; DFDC: 23,654 real/104,500 fake), different generation pipelines, resolutions, compression levels, and face-positioning constraints (admitted in §6.1). No error bars or statistical tests are given. The phrase 'possibly due to its focus on Hindi language content' is therefore an interpretation, not a tested hypothesis. A matched English baseline using the same generation tools and the same evaluation protocol is required to support the language-specific difficulty claim.
- [Abstract and §2.3] The abstract states that HAV-DF is 'unique of its kind as all of the previous datasets contain either deepfake videos or synthesized audio,' but the paper itself reviews FakeAVCeleb (§2.3), an audio-video multimodal deepfake dataset, and DFDC, which contains cloned audio and deepfake videos. The novelty claim should be narrowed to 'first Hindi-language audio-video deepfake dataset' or qualified accordingly.
- [Data Availability Statement] The dataset and code are only 'available from the corresponding author on reasonable request.' For a dataset paper, this severely limits reproducibility and community adoption, and it makes the reported benchmark numbers impossible to verify independently. A public release (e.g., a permanent repository with a DOI) or at least an anonymous access mechanism during review is needed for the contribution to be usable as a benchmark.
minor comments (6)
- [Throughout] The manuscript contains several typographical issues: 'accuracy's' should be 'accuracies' (abstract, §5.2.2), 'multimodel' should be 'multimodal' (e.g., §6), and 'annd'/'hannd' should be 'and'/'hand' (§3.4.3, §4.1.2).
- [§3.4.4] The statement 'Generating high-quality deepfakes for 350 videos' conflicts with the 308 fake video count reported in §4.1.2; this appears to be a typo and should be corrected.
- [§5.2.1 and Table 3] The dataset abbreviation 'DFD' used in Table 3 is inconsistent with 'DFDC' used in Figure 12 and Table 4; please unify the terminology.
- [Figures 12 and 13] The figures would benefit from error bars or confidence intervals, as the reported differences of a few percentage points are not shown to be statistically significant.
- [§5.2.2] The paper refers to 'FF-DF' in §5.2.2 but 'FF++' in earlier sections; please use a consistent abbreviation throughout.
- [§1.3 and §5] Research objective 2 in §1.3 mentions comparing performance on audio-video, audio-only, and video-only, but no audio-only evaluation is reported; either add such experiments or revise the objectives to match the actual scope.
Circularity Check
No significant circularity: the dataset is an empirical artifact and the detector benchmarks come from external methods; the language-difficulty claim is confounded but not circular.
full rationale
The paper's central contribution is a new Hindi audio-video deepfake dataset, constructed via a described pipeline (FSGAN/FaceSwap/DeepFaceLab for face swap, ReTalking for lip sync, RVC for voice cloning). The benchmark claim is that existing detectors (HeadPose, Two-stream, Mesonet, Xception-c23/c40, Multi-task, FWA) achieve lower accuracy on HAV-DF than on FF-DF and DFDC. These accuracies are empirical outputs of externally defined detection methods applied to the dataset; they are not derived from the dataset definition or from any fitted parameter renamed as a prediction. No equation in the paper defines a target quantity in terms of itself, no fitted input is later called a prediction, and no load-bearing premise is justified solely by the authors' own prior work. The statement that the low accuracy is 'possibly due to its focus on Hindi language content' is an interpretation, not a tested hypothesis, and the cross-dataset comparison in Section 5.2.2 is confounded by differences in generation pipeline, dataset size, resolution, and evaluation protocol. The paper also acknowledges limitations such as small size and the frontal-view face-positioning constraint. Those are correctness and generalizability concerns, not circular reasoning. Under the given criteria, an uncontrolled comparison is a missing-baseline problem rather than circularity, and no specific reduction from output to input can be quoted. Therefore the appropriate circularity score is 0.
Assumptions & free parameters
assumptions (3)
- domain assumption Real videos sourced from YouTube can be used to create and share manipulated deepfakes of identifiable individuals without explicit consent or ethics review.
- domain assumption The generated videos are genuine deepfakes of sufficient quality, with quality established by visual inspection and SSIM scores rather than by human perceptual studies or independent verification.
- ad hoc to paper Accuracy differences across datasets are causally attributable to Hindi language and cultural content rather than to generation pipeline, resolution, compression, or dataset size.
Cite this review
Pith. "Pith review of Hindi audio-video-Deepfake (HAV-DF): A Hindi language-based Audio-video Deepfake Dataset." pith.science (2026). https://pith.science/paper/BDQ3L2Y4
@misc{pith2026241115457,
author = {Pith},
title = {Pith review of: Hindi audio-video-Deepfake (HAV-DF): A Hindi language-based Audio-video Deepfake Dataset},
year = {2026},
howpublished = {\url{https://pith.science/paper/BDQ3L2Y4}},
note = {Machine review of arXiv:2411.15457}
}
read the original abstract
Deepfakes offer great potential for innovation and creativity, but they also pose significant risks to privacy, trust, and security. With a vast Hindi-speaking population, India is particularly vulnerable to deepfake-driven misinformation campaigns. Fake videos or speeches in Hindi can have an enormous impact on rural and semi-urban communities, where digital literacy tends to be lower and people are more inclined to trust video content. The development of effective frameworks and detection tools to combat deepfake misuse requires high-quality, diverse, and extensive datasets. The existing popular datasets like FF-DF (FaceForensics++), and DFDC (DeepFake Detection Challenge) are based on English language.. Hence, this paper aims to create a first novel Hindi deep fake dataset, named ``Hindi audio-video-Deepfake'' (HAV-DF). The dataset has been generated using the faceswap, lipsyn and voice cloning methods. This multi-step process allows us to create a rich, varied dataset that captures the nuances of Hindi speech and facial expressions, providing a robust foundation for training and evaluating deepfake detection models in a Hindi language context. It is unique of its kind as all of the previous datasets contain either deepfake videos or synthesized audio. This type of deepfake dataset can be used for training a detector for both deepfake video and audio datasets. Notably, the newly introduced HAV-DF dataset demonstrates lower detection accuracy's across existing detection methods like Headpose, Xception-c40, etc. Compared to other well-known datasets FF-DF, and DFDC. This trend suggests that the HAV-DF dataset presents deeper challenges to detect, possibly due to its focus on Hindi language content and diverse manipulation techniques. The HAV-DF dataset fills the gap in Hindi-specific deepfake datasets, aiding multilingual deepfake detection development.
Figures
Figures from the paper (10 more)
Reference graph
Works this paper leans on
-
[9]
arXiv preprint arXiv:1812.08685
Deepfakes: a new threat to face recognition? assessment and detection. arXiv preprint arXiv:1812.08685 . Page 21 of 22 HAV-DF: Deepfake Kwon, P., You, J., Nam, G., Park, S., Chae, G.,
-
[10]
arXiv preprint arXiv:1811.00656
Exposing deepfake videos by detecting face warping artif acts. arXiv preprint arXiv:1811.00656 . Li,Y.,Yang,X.,Sun,P.,Qi,H.,Lyu,S.,2020. Celeb-df: Alarge-scalechallengingdatasetfordeepfakeforensics,in: ProceedingsoftheIEEE/CVF conference on computer vision and pattern recognition, pp. 3207–3216. Liu, K., Perov, I., Gao, D., Chervoniy, N., Zhou, W., Zhang, W.,
arXiv 2020
-
[11]
Information Fusion 103, 102103
Generation and detection of manipulated multimodal audiovisual content: Advances, trends and open challenges. Information Fusion 103, 102103. Masood,M.,Nawaz,M.,Malik,K.M.,Javed,A.,Irtaza,A.,Malik,H.,2023. Deepfakesgenerationanddetection: State-of-the-art,openchallenges, countermeasures, and way forward. Applied intelligence 53, 3974–4026. Melnik, A., Mia...
work page 2023
-
[13]
Multi-task learning for detecting and segmenting manipulated facial images and videos, in: 2019 IEEE 10th International Conference on Biometrics Theory, Applications and Systems (BTAS), IEEE. pp. 1–8. Nirkin, Y., Keller, Y., Hassner, T.,
work page 2019
-
[14]
arXiv preprint arXiv:2005.05535
Deepfacelab: Integrated, flexible and extensible face-swapping framework. arXiv preprint arXiv:2005.05535 . Rabhi,M.,Bakiras,S.,DiPietro,R.,2024. Audio-deepfakedetection: Adversarialattacksandcountermeasures. ExpertSystemswithApplications 250, 123941. Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., Nießner, M.,
arXiv 2005
-
[15]
Multilingual speech synthesis for voice cloning, in: 2021 IEEE International Conference on Big Data and Smart Computing (BigComp), IEEE. pp. 313–316. Shelar,J.,Ghatole,D.,Pachpande,M.,Bhandari,D.,Shinde,S.,2022. Deepfakesforvideoconferencingusinggeneraladversarialnetworks(gans) and multilingual voice cloning, in: Computational Intelligence in Data Mining:...
work page 2021
-
[16]
Natural tts synthesis by conditioning wavenet on mel spectrogram predictions, in: 2018 IEEE international conference on acoustics, speech and signal processing (ICASSP), IEEE. pp. 4779–4783. Tolosana,R.,Vera-Rodriguez,R.,Fierrez,J.,Morales,A.,Ortega-Garcia,J.,2020. Deepfakesandbeyond: Asurveyoffacemanipulationandfake detection. Information Fusion 64, 131–...
work page 2018
-
[18]
Deepfake detection on publicly available datasets using modified alexnet, in: 2020 IEEE Symposium Series on Computational Intelligence (SSCI), IEEE. pp. 1866–1871. Yang, X., Li, Y., Lyu, S.,
work page 2020
Show all 19 references
-
[19]
Exposing deep fakes using inconsistent head poses, in: ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE. pp. 8261–8265. Zhou,P.,Han,X.,Morariu,V.I.,Davis,L.S.,2017.Two-streamneuralnetworksfortamperedfacedetection,in: 2017...
2019
-
[31]
Deeperforensics-1.0: Alarge-scaledatasetforreal-worldfaceforgerydetection
Jiang,L.,Li,R.,Wu,W.,Qian,C.,Loy,C.C.,2020. Deeperforensics-1.0: Alarge-scaledatasetforreal-worldfaceforgerydetection. arXivpreprint arXiv:2001.03024 . Kambali, S.P., Ansari, M.A., Srivastav, P.U., Aryan, M.D., Nanda, R.,
2020 arXiv
-
[147]
Xception: Deeplearningwithdepthwiseseparableconvolutions,in: ProceedingsoftheIEEEconferenceoncomputervisionand pattern recognition, pp
Chollet,F.,2017. Xception: Deeplearningwithdepthwiseseparableconvolutions,in: ProceedingsoftheIEEEconferenceoncomputervisionand pattern recognition, pp. 1251–1258. Dang, H., Liu, F., Stehouwer, J., Liu, X., Jain, A.K.,
2017
-
[294]
Fakeavceleb: Anovelaudio-videomultimodaldeepfakedataset
Khalid,H.,Tariq,S.,Kim,M.,Woo,S.S.,2021. Fakeavceleb: Anovelaudio-videomultimodaldeepfakedataset. arXivpreprintarXiv:2108.05080 . Korshunov, P., Marcel, S.,
2021 arXiv
-
[2017]
arXiv preprint arXiv:1703.10135
Tacotron: Towards end-to-end speech synthesis. arXiv preprint arXiv:1703.10135 . Wu, Z., Yamagishi, J., Kinnunen, T., Hanilçi, C., Sahidullah, M., Sizov, A., Evans, N., Todisco, M., Delgado, H.,
-
[2018]
Mesonet: a compact facial video forgery detection network, in: 2018 IEEE international workshop on information forensics and security (WIFS), IEEE. pp. 1–7. Arik,S.,Chen,J.,Peng,K.,Ping,W.,Zhou,Y.,2018. Neu-ralvoicecloningwithafewsamples.inadvancesinneuralinformationprocessing...
2018
-
[2019]
Felouat, H., Nguyen, H.H., Le, T.N., Yamagishi, J., Echizen, I.,
Deepfakes detection dataset by google & jigsaw, in: arXiv preprint arXiv:1901.08971. Felouat, H., Nguyen, H.H., Le, T.N., Yamagishi, J., Echizen, I.,
1901 arXiv
-
[2020]
arXiv preprint arXiv:2006.07397
The deepfake detection challenge (dfdc) dataset. arXiv preprint arXiv:2006.07397 . Dufour, N., Gully, A., Karlsson, P., Vorbyov, A.V., Leung, T., Childs, J., Bregler, C.,
2006 arXiv
-
[2021]
ACM computing surveys (CSUR) 54, 1–41
The creation and detection of deepfakes: A survey. ACM computing surveys (CSUR) 54, 1–41. Mubarak,R.,Alsboui,T.,Alshaikh,O.,Inuwa-Dute,I.,Khan,S.,Parkinson,S.,2023. Asurveyonthedetectionandimpactsofdeepfakesinvisual, audio, and textual formats. IEEE Access . Munir,S.,Sajjad,W....
2023
-
[2022]
Videoretalking: Audio-based lip synchronization for talking head video editing in the wild, in: SIGGRAPH Asia 2022 Conference Papers, pp. 1–9. Cheng, X.,
2022
-
[2024]
arXiv preprint arXiv:2405.08838
Polyglotfake: A novel multilingual and multimodal deepfake dataset. arXiv preprint arXiv:2405.08838 . Jia, Y., Zhang, Y., Weiss, R., Wang, Q., Shen, J., Ren, F., Nguyen, P., Pang, R., Lopez Moreno, I., Wu, Y., et al.,
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.