Understanding Emergent Non-Verbal Communication in the Delta Force Competitive Video Game through Multimodal AI Analysis

Presenter: Josef Spjut

Non-verbal communication plays a critical role in multiplayer games, especially in environments where verbal communication is limited or constrained by game mechanisms. Players often rely on gestures, movement patterns, item interactions, and UI signals to communicate intent, negotiate cooperation willingness, and avoid conflict. In our work, we use gameplay video as the input, and analyze it to extract non-verbal communication. We propose a multi-modal pipeline for detecting and interpreting non-verbal communication in gameplay videos. Our research combines pose estimation, visual-textual extraction, and audio analysis to capture diverse behavioral signals, which are then integrated using a large language model to infer player intent. Using gameplay data collected from publicly available online videos, we conduct preliminary analyses suggesting that a wide range of meaningful non-verbal interactions can be systematically extracted and interpreted. Through this study, we aim to develop an empirical understanding of emergent non-verbal communication in modern multiplayer games and a practical framework for analyzing such behaviors using AI.