Real or Deepfake Face Detection in Images and Video Data  using YOLO11 Algorithm

Wawan Kurniawan; Aliyah Kurniasih; Muhamad Abdul Ghani

doi:10.59934/jaiea.v4i2.939

Authors

Wawan Kurniawan Universitas Bina Sarana Informatika
Aliyah Kurniasih Universitas Ary Ginanjar
Muhamad Abdul Ghani Universitas Bina Sarana Informatika

DOI:

https://doi.org/10.59934/jaiea.v4i2.939

Keywords:

Deepfake, Object Detection, YOLO11

Abstract

This study focuses on the detection of real and deepfake faces in images and video data using the YOLO11 algorithm. Deepfakes, generated using advanced deep learning techniques, have the potential for misuse, including spreading false information and identity theft. The research employs public datasets, namely EC2-DeepFake and Deepfake Dataset, to train a YOLO11-based model. The performance of the model is evaluated using metrics such as mAP50 and mAP50-95. Results indicate moderate accuracy in distinguishing real from fake faces, with notable challenges in handling diverse data. The findings emphasize the need for improved training techniques and larger datasets to enhance detection performance. This work contributes to developing tools for mitigating the risks posed by deepfake technology.

Downloads

Download data is not yet available.

References

C. T. Noerman and A. L. Ibrahim, “Kriminalisasi Deepfake Di Indonesia Sebagai Bentuk Pelindungan Negara,” J. Usm Law Rev., vol. 7, no. 2, pp. 603–621, 2024.

S. A. U. Sijabat and D. Lukitasari, “Konten Gambar dan Video Pornografi Deepfake Sebagai Suatu Bentuk Tindak Pidana Pencemaran Nama Baik,” J. Huk. Pidana dan Penanggulangan Kejahatan, vol. 13, no. 2, pp. 179–194, 2024.

J. Mu, M. Adrezo, and A. N. Haikal, “Identifikasi Wajah Asli dan Buatan Deepfake Menggunakan Metode Convolutional Neural Network,” Teknika, vol. 13, no. 1, pp. 45–50, 2024.

L. Hae-rin, “More Koreans fall victim to deepfake sex crimes,” The Korea Times. [Online]. Available: https://www.koreatimes.co.kr/www/nation/2025/01/113_381251.html. [Accessed: 02-Jan-2025].

“Polisi Ungkap Sisi Kelam AI, untuk Penipuan hingga Pelecehan Seksual,” CNN Indonesia. [Online]. Available: https://www.cnnindonesia.com/teknologi/20241129161114-185-1172121/polisi-ungkap-sisi-kelam-ai-untuk-penipuan-hingga-pelecehan-seksual. [Accessed: 02-Jan-2025].

“Penipu di Lampung Pakai Deepfake AI Prabowo Raup Rp30 Juta,” CNN Indonesia. [Online]. Available: https://www.cnnindonesia.com/nasional/20250123165315-12-1190773/penipu-di-lampung-pakai-deepfake-ai-prabowo-raup-rp30-juta. [Accessed: 26-Jan-2025].

G. N. Rizkatama, A. Nugroho, and A. F. Suni, “Sistem Cerdas Penghitung Jumlah Mobil untuk Mengetahui Ketersediaan Lahan Parkir berbasis Python dan YOLO v4,” Edu Komputika J., vol. 8, no. 2, pp. 91–99, 2021.

Darmatasia, A. R. Ramli, A. Salsabila, and F. Adiba, “Analisis Performa Convolutional Neural Network dengan Hyperparameter Tuning dalam Mendeteksi Gambar Deepfake,” J. Insypro, vol. 9, no. November, pp. 1–8, 2024.

N. Jegham, C. Y. Koh, M. Abdelatti, and A. Hendawi, “Evaluating the Evolution of YOLO (You Only Look Once) Models: A Comprehensive Benchmark Study of YOLO11 and Its Predecessors,” arXiv, no. October, pp. 1–20, 2024.

R. Khanam and M. Hussain, “YOLOv11: An Overview of the Key Architectural Enhancements,” arXiv, no. October, pp. 1–9, 2024.

J. Huang, K. Wang, Y. Hou, and J. Wang, “LW-YOLO11: A Lightweight Arbitrary-Oriented Ship Detection Method Based on Improved YOLO11,” Sensors, vol. 25, no. 1, 2025.

EC2, “EC2-DeepFake Dataset,” Roboflow. [Online]. Available: https://universe.roboflow.com/ec2/ec2-deepfake-oxj9m. [Accessed: 16-Dec-2024].

N. Workspace, “Deepfake Dataset,” Roboflow. [Online]. Available: https://universe.roboflow.com/new-workspace-ugbw4/deepfake-gglmq. [Accessed: 16-Dec-2024].

M. Bakhara Alief Rachman, A. Kurniasih, A. Sundawijaya, and A. Nuraminah, “Penerapan Blok Se-Net Pada Deep Learning Inceptionv3 Untuk Meningkatkan Deteksi Penyakit Mpox Pada Manusia,” J. Teknol. Inf. dan Ilmu Komput., vol. 10, no. 7, pp. 1447–1452, 2023.

A. Kurniasih and L. P. Manik, “On the Role of Text Preprocessing in BERT Embedding-based DNNs for Classifying Informal Texts,” Int. J. Adv. Comput. Sci. Appl., vol. 13, no. 6, pp. 927–934, 2022.

M. A. R. Alif, “YOLOv11 for Vehicle Detection: Advancements, Performance, and Applications in Intelligent Transportation Systems,” arXiv, no. October, pp. 1–16, 2024.

Saurabhbagchi, “Deepfake image detection,” Kaggle Dataset, 2025. [Online]. Available: https://www.kaggle.com/datasets/saurabhbagchi/deepfake-image-detection. [Accessed: 29-Dec-2024].

LipSynthesis, “Deepfake Example. Original/Deepfake Elon Musk.,” LipSynthesis. [Online]. Available: https://www.youtube.com/watch?v=WFc6t-c892A. [Accessed: 29-Dec-2024].