Can We Defend Against AI-Generated Video Attacks on Real-World Crisis Events? A Systematic Evaluation of Detectors, Generators and Social Dissemination

Key points

RA-Bench contains 17,886 videos, made up of 1,830 real videos and 16,056 generated videos, and the real videos cover 10 social risk categories.
On public data, the AUC of 7 conventional detectors ranges from 67.6 percent to 98.6 percent. Their averages on RA-Bench across generators range only from 43.9 percent to 57.3 percent.
Humans identified 68.6 percent of open-source generated videos as synthetic, but only 52.9 percent of closed-generator videos. There were also 633 generated videos that all five people judged to be real.
When compression, downsampling, lower frame rate, and a news badge are all applied, the mean detection rate of a fine-tuned multimodal detector drops from 46.0 percent to 1.4 percent.
The experiments are limited to 9 image-based video generators and silent video detection. Real misinformation can also involve voice synthesis, editing, and repeated processing across multiple platforms.

External research summaries. These are not HDATF publications or measured product results.