Gamer Behavior Modeling Using Computer Vision and Artificial Intelligence

Faizal Anugrah Pratama; Diky Mulyadi

doi:10.59934/jaiea.v5i3.2416

Authors

Faizal Anugrah Pratama Universitas Majalengka
Diky Mulyadi Universitas Majalengka

DOI:

https://doi.org/10.59934/jaiea.v5i3.2416

Keywords:

Artificial intelligence, Computer Vision, Deep Learning, Eye-Tracking, Player Behavior Modeling

Abstract

This systematic literature review examined game-player behavior modeling that integrates computer vision and artificial intelligence. A total of 50 scientific articles from the Scopus database, published between 2017 and 2025, were analyzed using the PRISMA 2020 protocol to address five research questions. The results showed that eye-tracking metrics reliably indicated cognitive load and player attention in real time. Deep learning models could predict individual-specific behavioral styles in strategic games, and data-driven analysis effectively mapped players' spatial movement dynamics in large-scale game environments. Furthermore, physiological metrics in Virtual Reality proved effective as inputs to a dynamic difficulty adjustment system, and data-driven gaze-direction modeling significantly enhanced the realism of player interactions with virtual agents. These findings confirmed that the synergy between computer vision and artificial intelligence is a crucial foundation for creating adaptive and immersive gaming experiences.

Downloads

Download data is not yet available.

References

Sekhavat Y.A., Roohi S., Mohammadi H.S., Yannakakis G.N. (2022). Play with One's Feelings: A Study on Emotion Awareness for Player Experience. IEEE Transactions on Games, 14(1). DOI: 10.1109/TG.2020.3003324

Dobrovsky A., Borghoff U.M., Hofmann M. (2017). Applying and augmenting deep reinforcement learning in serious games through interaction. Periodica polytechnica Electrical engineering and computer science, 61(2). DOI: 10.3311/PPee.10313

Gaurav D., Kaushik Y., Supraja S., Yadav M., Gupta M.P., Chaturvedi M. (2022). Empirical Study of Adaptive Serious Games in Enhancing Learning Outcome. International Journal of Serious Games, 9(2). DOI: 10.17083/ijsg.v9i2.486

Khan A., Feng J., Liu S., Asghar M.Z. (2019). Optimal Skipping Rates: Training Agents with Fine-Grained Control Using Deep Reinforcement Learning. Journal of Robotics, 2019. DOI: 10.1155/2019/2970408

Melhart D., Yannakakis G.N., Liapis A. (2020). I Feel I Feel You: A Theory of Mind Experiment in Games. KI - Kunstliche Intelligenz, 34(1). DOI: 10.1007/s13218-020-00641-2

Liu X., Zhang H., Zhang Y., Shao L. (2020). Optimal Network Defense Strategy Selection Method Based on Evolutionary Network Game. Security and Communication Networks, 2020. DOI: 10.1155/2020/5381495

Jung S., Kim H., Park H., Choi A. (2025). Integrated AI System for Real-Time Sports Broadcasting: Player Behavior, Game Event Recognition, and Generative AI Commentary in Basketball Games. Applied Sciences (Switzerland), 15(3). DOI: 10.3390/app15031543

Zhang L. (2022). Behaviour Detection and Recognition of College Basketball Players Based on Multimodal Sequence Matching and Deep Neural Networks. Computational Intelligence and Neuroscience, 2022. DOI: 10.1155/2022/7599685

Pfau J., Liapis A., Yannakakis G.N., Malaka R. (2023). Dungeons & Replicants II: Automated Game Balancing Across Multiple Difficulty Dimensions via Deep Player Behavior Modeling. IEEE Transactions on Games, 15(2). DOI: 10.1109/TG.2022.3167728

Dias S.B., Jelinek H.F., Hadjileontiadis L.J. (2023). Multisensed Emotions as Adaptation Controllers in Human-to-Serious NeuroGames Communication. IEEE Communications Magazine, 61(10). DOI: 10.1109/MCOM.001.2200828

Chesani F., Galassi A., Lippi M., Mello P. (2018). Can deep networks learn to play by the rules? A case study on nine Men's Morris. IEEE Transactions on Games, 10(4). DOI: 10.1109/TG.2018.2804039

Pinto J.P., Pimenta A., Novais P. (2021). Deep learning and multivariate time series for cheat detection in video games. Machine Learning, 110(11-12). DOI: 10.1007/s10994-021-06055-x

Santos C.P., Hutchinson K., Khan V.-J., Markopoulos P. (2019). Profiling personality traits with games. ACM Transactions on Interactive Intelligent Systems, 9(2-3). DOI: 10.1145/3230738

Zeng Y., Xu K., Qin L., Yin Q. (2020). A Semi-Markov Decision Model with Inverse Reinforcement Learning for Recognizing the Destination of a Maneuvering Agent in Real Time Strategy Games. IEEE Access, 8. DOI: 10.1109/ACCESS.2020.2967642

Li S., Li S., Cao H., Meng K., Ding M. (2020). Study on the Strategy of Playing Doudizhu Game Based on Multirole Modeling. Complexity, 2020. DOI: 10.1155/2020/1764594

Mustač K., Bačić K., Skorin-Kapov L., Sužnjević M. (2022). Predicting Player Churn of a Free-to-Play Mobile Video Game Using Supervised Machine Learning. Applied Sciences (Switzerland), 12(6). DOI: 10.3390/app12062795

Duinkharjav B., Chakravarthula P., Brown R., Patney A., Sun Q. (2022). Image features influence reaction time: A Learned Probabilistic Perceptual Model for Saccade Latency. ACM Transactions on Graphics, 41(4). DOI: 10.1145/3528223.3530055

Acquah E.O., Katz H.T. (2020). Digital game-based L2 learning outcomes for primary through high-school students: A systematic literature review. Computers and Education, 143. DOI: 10.1016/j.compedu.2019.103667

Jallov D., Risi S., Togelius J. (2017). EvoCommander: A novel game based on evolving and switching between artificial brains. IEEE Transactions on Computational Intelligence and AI in Games, 9(2). DOI: 10.1109/TCIAIG.2016.2535416

Barros P., Sciutti A. (2022). All by Myself: Learning individualized competitive behavior with a contrastive reinforcement learning optimization. Neural Networks, 150. DOI: 10.1016/j.neunet.2022.03.013

Phutela N., Chowdary A.N., Anchlia S., Jaisinghani D., Gabrani G. (2022). Unlock Me: A Real-World Driven Smartphone Game to Stimulate COVID-19 Awareness. International Journal of Human Computer Studies, 164. DOI: 10.1016/j.ijhcs.2022.102818

Posada Trobo I., García Díaz V., Pascual Espada J., González Crespo R., Moreno-Ger P. (2019). Rapid modeling of human-defined AI behavior patterns in games. Journal of Ambient Intelligence and Humanized Computing, 10(7). DOI: 10.1007/s12652-018-0969-y

Rădulescu R., Verstraeten T., Zhang Y., Mannion P., Roijers D.M., Nowé A. (2022). Opponent learning awareness and modelling in multi-objective normal form games. Neural Computing and Applications, 34(3). DOI: 10.1007/s00521-021-06184-3

Kian T.W., Sunar M.S., Su G.E. (2022). The Analysis of Intrinsic Game Elements for Undergraduates Gamified Platform Based on Learner Type. IEEE Access, 10. DOI: 10.1109/ACCESS.2022.3218625

Cavalcanti J., Valls V., Contero M., Fonseca D. (2021). Gamification and hazard communication in virtual reality: A qualitative study. Sensors, 21(14). DOI: 10.3390/s21144663

Donekal Chandrashekar N., Lee A., Azab M., Gracanin D. (2024). Understanding User Behavior for Enhancing Cybersecurity Training with Immersive Gamified Platforms. Information (Switzerland), 15(12). DOI: 10.3390/info15120814

Bennett M.S., Hedley L., Love J., Houpt J.W., Brown S.D., Eidels A. (2025). Human Performance in Competitive and Collaborative Human–Machine Teams. Topics in Cognitive Science, 17(2). DOI: 10.1111/tops.12683

Han Y.-J., Moon J., Woo J. (2024). Prediction of Churning Game Users Based on Social Activity and Churn Graph Neural Networks. IEEE Access, 12. DOI: 10.1109/ACCESS.2024.3429559

Mitre-Ortiz A., Muñoz-Arteaga J., Cardona-Reyes H. (2023). Developing a model to evaluate and improve user experience with hand motions in virtual reality environments. Universal Access in the Information Society, 22(3). DOI: 10.1007/s10209-022-00882-y

Gu H., Wang H. (2020). A Distributed Caching Scheme Using Non-Cooperative Game for Mobile Edge Networks. IEEE Access, 8. DOI: 10.1109/ACCESS.2020.3009683

Agarwal S., Wallner G., Beck F. (2020). Bombalytics: Visualization of Competition and Collaboration Strategies of Players in a Bomb Laying Game. Computer Graphics Forum, 39(3). DOI: 10.1111/cgf.13965

Ghawi R., Müller S., Pfeffer J. (2021). Improving team performance prediction in MMOGs with temporal communication networks. Social Network Analysis and Mining, 11(1). DOI: 10.1007/s13278-021-00775-7

Hurtado L.A., Mocanu E., Nguyen P.H., Gibescu M., Kamphuis R.I.G. (2018). Enabling Cooperative Behavior for Building Demand Response Based on Extended Joint Action Learning. IEEE Transactions on Industrial Informatics, 14(1). DOI: 10.1109/TII.2017.2753408

Huang J., Zhang H., Wang J. (2017). Markov Evolutionary Games for Network Defense Strategy Selection. IEEE Access, 5. DOI: 10.1109/ACCESS.2017.2753278

Lima J.F., Acosta-Urigüen M.I., Orellana M. (2024). Machine learning and knowledge engineering for cognitive memory assessment of age groups by anomalies in a serious game. Intelligent Systems with Applications, 21. DOI: 10.1016/j.iswa.2023.200301

Yaldo L., Shamir L. (2017). Computational estimation of football player wages. International Journal of Computer Science in Sport, 16(1). DOI: 10.1515/ijcss-2017-0002

Chen J., Zhang D., Qu Z., Wang C. (2020). Artificial empathy: A new perspective for analyzing and designing multi-agent systems. IEEE Access, 8. DOI: 10.1109/ACCESS.2020.3029502

Kirchner-Krath J., Altmeyer M., Schürmann L., Kordyaka B., Morschheuser B., Klock A.C.T., Nacke L., Hamari J., von Korflesch H.F.O. (2024). Uncovering the theoretical basis of user types: An empirical analysis and critical discussion of user typologies in research on tailored gameful design. International Journal of Human Computer Studies, 190. DOI: 10.1016/j.ijhcs.2024.103314

Goodson S., Turner K.J., Pearson S.L., Carter P. (2021). Violent Video Games and the P300: No Evidence to Support the Neural Desensitization Hypothesis. Cyberpsychology, Behavior, and Social Networking, 24(1). DOI: 10.1089/cyber.2020.0029

Parras J., Zazo S. (2019). Repeated game analysis of a CSMA/CA network under a backoff attack. Sensors (Switzerland), 19(24). DOI: 10.3390/s19245393

Khan B.U.I., Anwar F., Olanrewaju R.F., Kiah M.L.B.M., Mir R.N. (2021). Game Theory Analysis and Modeling of Sophisticated Multi-Collusion Attack in MANETs. IEEE Access, 9. DOI: 10.1109/ACCESS.2021.3073343

Pirker J., Rattinger A., Drachen A., Sifa R. (2018). Analyzing player networks in Destiny. Entertainment Computing, 25. DOI: 10.1016/j.entcom.2017.12.001

Buckley D., Chen K., Knowles J. (2017). Rapid Skill Capture in a First-Person Shooter. IEEE Transactions on Computational Intelligence and AI in Games, 9(1). DOI: 10.1109/TCIAIG.2015.2494849

Rupp F., Eberhardinger M., Eckert K. (2024). Simulation-Driven Balancing of Competitive Game Levels With Reinforcement Learning. IEEE Transactions on Games, 16(4). DOI: 10.1109/TG.2024.3399536

Saaty M., Haqq D., Beyki M., Hassan T., Mccrickard D.S. (2022). Pokémon GO with Social Distancing: Social Media Analysis of Players' Experiences with Location-based Games. Proceedings of the ACM on Human-Computer Interaction, 6. DOI: 10.1145/3549512

Peters L., Rubies-Royo V., Tomlin C.J., Ferranti L., Alonso-Mora J., Stachniss C., Fridovich-Keil D. (2023). Online and offline learning of player objectives from partial observations in dynamic games. International Journal of Robotics Research, 42(10). DOI: 10.1177/02783649231182453

Wang X., Luo C., Ding S., Wang J. (2018). Imitating Contributed Players Promotes Cooperation in the Prisoner's Dilemma Game. IEEE Access, 6. DOI: 10.1109/ACCESS.2018.2870340

Soler-Domínguez J.L., Contero M., Alcañiz M. (2019). Workflow and tools to track and visualize behavioural data from a Virtual Reality environment using a lightweight GIS. SoftwareX, 10. DOI: 10.1016/j.softx.2019.100269

Ferguson B.L., Brown P.N., Marden J.R. (2022). The Effectiveness of Subsidies and Tolls in Congestion Games. IEEE Transactions on Automatic Control, 67(6). DOI: 10.1109/TAC.2021.3088412

Reyssier S., Hallifax S., Serna A., Marty J.-C., Simonian S., Lavoue E. (2022). The Impact of Game Elements on Learner Motivation: Influence of Initial Motivation and Player Profile. IEEE Transactions on Learning Technologies, 15(1). DOI: 10.1109/TLT.2022.3153239

Alsaggaf W., Tsaramirsis G., Al-Malki N., Khan F.Q., Almasry M., Serafi M.A., Almarzuqi A. (2020). Association of game events with facial animations of computer-controlled virtual characters based on probabilistic human reaction modeling. Applied Sciences (Switzerland), 10(16). DOI: 10.3390/app10165636

Poivet R., De Lagarde A., Pelachaud C., Auvray M. (2024). Evaluation of Virtual Agents’ Hostility in Video Games. IEEE Transactions on Affective Computing. DOI: 10.1109/TAFFC.2024.3390400

Alimin dan S. Riyadi (2022). Sistem Pengembangan Deteksi Kanker Prostat Berbasis Image Processing dengan Metode Convolutional Neural Network. Explore IT: Jurnal Keilmuan dan Aplikasi Teknik Informatika, 14(2), pp. 52-63. DOI: 10.35891/explorit.v14i2.3535

Haddaway N.R., Page M.J., Pritchard C.C., McGuinness L.A. (2022). PRISMA2020: An R package and Shiny app for producing PRISMA 2020-compliant flow diagrams, with interactivity for optimised digital transparency and Open Synthesis. Campbell Systematic Reviews, 18. DOI: 10.1002/cl2.1230

Port E., Jacobs C., Kider J.T. (2025). Understanding Player Dynamics in Battle Royale Environments: A Data-Driven Analysis Using the Caldera Dataset. Proceedings MIG 2025 - 18th ACM Conference on Motion, Interaction and Games. DOI: 10.1145/3769047.3769054

Carlini E., Lulli A. (2019). Analysis of Movement Features in Multiplayer Online Battle Arenas. Journal of Grid Computing, 17. DOI: 10.1007/s10723-018-9470-2

Simmons T., Dinakaran S., Jain N., Ghanote V., Wilds G., Cheng M., Chukoskie L. (2025). Gaze Behavior Reflects Expertise in Goal-Directed Multiple Object Tracking. Eye Tracking Research and Applications Symposium (ETRA). DOI: 10.1145/3715669.3723115