Field Notes (14): Presence Detection in VR with Machine Learning
Measuring the Virtual
The concept of presence plays a critical role in shaping user experience and immersion. Presence, often defined as the sensation of "being there," serves as a foundational element in determining the effectiveness of a VR environment. Whether in entertainment, education, or training, the extent to which users feel immersed can differentiate a compelling experience from one that feels fragmented. Traditionally, presence has been measured through self-reported questionnaires, which introduce bias and fail to capture real-time shifts in user engagement. Addressing this gap, Saha et al. (2024) propose a machine learning-based framework for detecting and classifying presence in VR environments using psychophysiological signals.
This Field Note examines how machine learning, integrated with real-time data from physiological sensors, is revolutionizing the optimization of VR experiences. By analyzing signals such as electroencephalography (EEG) and electrodermal activity (EDA), Saha and colleagues offer an objective metric for presence, with the potential to dynamically adjust VR scenarios in response to user engagement.
The conventional reliance on post-experience questionnaires for measuring presence in VR is inherently limited. As Saha et al. (2024) observe, these methods are subject to recall bias and the inability to capture real-time fluctuations in user experience. Participants' memories of their VR interactions may degrade by the time they complete the survey, obscuring the nuances of their presence shifts during the experience.
In response, the authors developed a machine-learning model capable of detecting presence using EEG and EDA data. EEG captures brain activity, while EDA measures skin conductance linked to emotional and autonomic nervous system responses. These signals provide direct, real-time insights into a user's mental and emotional states, enabling continuous assessment of presence throughout the VR interaction. Multiple classifiers, including Support Vector Machines (SVM), k-Nearest Neighbors (KNN), and a Multi-Layer Perceptron (MLP), were tested to classify presence into high, medium, and low levels. The MLP model achieved the highest accuracy, with a macro average of 93%.
Factors Influencing Presence in VR
The experiment manipulated several factors to induce varying levels of presence:
Graphics Fidelity: The quality of visuals has a significant effect on presence. High-resolution, detailed graphics enhance immersion, allowing users to feel integrated into the virtual space. Conversely, lower graphics quality disrupts this sense of presence by breaking the illusion of reality.
Latency: Latency refers to the delay between user actions and the system's response. High latency disrupts immersion by causing a disjunction between user movements and what is displayed in the virtual environment. The study introduced varying latency levels, consistently finding that high-latency conditions led to lower presence scores.
Audio Cues: Sound is essential for creating an immersive experience. Spatial audio, which mimics the direction and distance of sounds in the physical world, heightened users' presence. The absence of sound or mismatched audio effects diminished the immersive experience.
Embodiment and Haptic Feedback: The sense of embodiment—where users feel that their virtual body is their own—was significantly enhanced when participants controlled an avatar and received haptic feedback. The study supports prior research that demonstrates the stronger the connection between user and avatar, the greater the sense of presence.
Role of Machine Learning in Real-Time Presence Detection
Machine learning proved essential in classifying different levels of presence. By training models on real-time EEG and EDA data, the researchers predicted, with high accuracy, the level of presence a participant experienced at any given moment. Feature extraction from the psychophysiological data was crucial to this success.
EEG Signals: Specific EEG signals, particularly from the frontal and parietal lobes, correlated with high presence levels. Increased beta wave activity in the frontal lobe signaled greater engagement and immersion, while theta and alpha waves provided insights into cognitive load and attention, which are directly tied to presence.
EDA Signals: Electrodermal activity, a measure of emotional arousal, offered critical information on participants' reactions to the VR environments. Higher levels of skin conductance corresponded to more immersive experiences. The machine learning model effectively distinguished between high and medium presence levels using EDA data.
The integration of EEG and EDA signals into machine learning algorithms enabled real-time presence classification, with the MLP model delivering the most accurate predictions across all presence levels.
Applications and Implications for VR Design
The ability to measure presence in real-time has far-reaching implications for VR design:
Gaming: Maximizing presence is crucial to player engagement in gaming. Real-time presence detection could allow games to dynamically adjust in response to players' states, modifying difficulty, pacing, or narrative elements to maintain immersion.
Education and Training: VR is increasingly used in educational settings and simulations. High levels of presence correlate with better learning outcomes. Real-time presence detection can help adjust scenarios to keep users engaged, improving knowledge retention and skill acquisition.
Therapeutic Environments: VR has proven effective in treating conditions such as PTSD and anxiety. High presence is necessary for therapeutic success. Presence detection in real-time could enhance therapy by providing clinicians with data on patient engagement and emotional state, allowing for more personalized interventions.
Challenges and Future Directions
Despite the promising results, challenges remain. The reliance on EEG and EDA equipment may limit the scalability of this technology for consumer applications. However, advancements in wearable technology could make these sensors more accessible in the future. Additionally, incorporating other physiological signals, such as heart rate variability and eye-tracking, may improve the robustness of presence detection models. Expanding the dataset to include more diverse VR scenarios and user populations will further enhance the generalizability of the findings.
Concluding Thoughts
The machine learning-based approach to presence detection represents a significant advancement in optimizing VR experiences. By utilizing real-time psychophysiological data, Saha et al. (2024) have introduced a more objective and dynamic method for measuring presence, offering new possibilities for creating immersive and responsive VR environments.
References
Saha, S., Dobbins, C., Gupta, A., & Dey, A. (2024). Machine learning-based classification of presence utilizing psychophysiological signals in immersive virtual environments. Scientific Reports, 14(21667). https://doi.org/10.1038/s41598-024-72376-1