Motion-based AI video detectors exploit motion biases in evaluation datasets and drop to near-random performance on rebalanced data, while frequency-based detectors remain robust.
As Good As A Coin Toss: Human detection of AI-generated images, videos, audio, and audiovisual stimuli
2 Pith papers cite this work. Polarity classification is still indexing.
2
Pith papers citing it
abstract
One of the current principal defenses against weaponized synthetic media continues to be the ability of the targeted individual to visually or auditorily recognize AI-generated content when they encounter it. However, as the realism of synthetic media continues to rapidly improve, it is vital to have an accurate understanding of just how susceptible people currently are to potentially being misled by convincing but false AI generated content. We conducted a perceptual study with 1276 participants to assess how capable people were at distinguishing between authentic and synthetic images, audio, video, and audiovisual media. We find that on average, people struggled to distinguish between synthetic and authentic media, with the mean detection performance close to a chance level performance of 50%. We also find that accuracy rates worsen when the stimuli contain any degree of synthetic content, features foreign languages, and the media type is a single modality. People are also less accurate at identifying synthetic images when they feature human faces, and when audiovisual stimuli have heterogeneous authenticity. Finally, we find that higher degrees of prior knowledgeability about synthetic media does not significantly impact detection accuracy rates, but age does, with older individuals performing worse than their younger counterparts. Collectively, these results highlight that it is no longer feasible to rely on the perceptual capabilities of people to protect themselves against the growing threat of weaponized synthetic media, and that the need for alternative countermeasures is more critical than ever before.
verdicts
UNVERDICTED 2representative citing papers
The authors provide the first systematic benchmark of traditional ML, DNN, Transformer, state-space, and multimodal models for machine-generated music detection, augmented with XAI analysis, and report ResNet18 as the strongest performer on in-domain and out-of-domain tests.
citing papers explorer
-
Dataset Biases and Shortcut Learning in Motion-Based AI-Generated Video Detection
Motion-based AI video detectors exploit motion biases in evaluation datasets and drop to near-random performance on rebalanced data, while frequency-based detectors remain robust.
-
Explainable Detection of Machine Generated Music and Early Systematic Evaluation
The authors provide the first systematic benchmark of traditional ML, DNN, Transformer, state-space, and multimodal models for machine-generated music detection, augmented with XAI analysis, and report ResNet18 as the strongest performer on in-domain and out-of-domain tests.