In recent years, the advancement of virtual reality (VR) and mixed reality (MR) technologies has opened up new avenues for immersive digital experiences across various sectors such as gaming, education, and remote work. These innovative technologies have transformed the way users interact with virtual environments and avatars, providing experiences that were previously unimaginable. However, with these advancements come new privacy risks that are often overlooked.
One such privacy risk involves the exploitation of gaze-controlled typing systems by potential attackers. As users interact with virtual avatars, their gaze movements can be analyzed to infer the keystrokes they type, posing a significant threat to user privacy and security. Researchers have developed a novel exploit known as GAZEploit to address this emerging vulnerability. GAZEploit can remotely reconstruct keystrokes typed by users based on their gaze movements, extracted from avatar images.
The GAZEploit attack targets individuals, particularly those who engage in activities that involve controlling avatars through gaze movements in VR and MR settings. By capturing and analyzing gaze information, the attack can decipher the text being typed by users without their knowledge, potentially compromising sensitive information such as login credentials.
The attack begins by extracting biometric features such as eye aspect ratio (EAR) and gaze direction from the user’s avatar in a virtual environment. These features are crucial in distinguishing typing sessions from other activities and mapping gaze movements to individual keystrokes on a virtual keyboard. Machine learning techniques, specifically Recurrent Neural Networks (RNNs), are utilized to analyze the sequential nature of gaze movements and identify patterns associated with typing behavior.
GAZEploit further identifies individual keystrokes by analyzing gaze stability during typing sessions. By focusing on fixations and eye movements between keystrokes, the attack can accurately predict the keys being targeted by users. Additionally, an adaptive mapping system is employed to map gaze data to specific keys on a virtual keyboard accurately, even in dynamic virtual environments.
Overall, the GAZEploit attack showcases the growing sophistication of privacy risks in VR and MR environments. By leveraging gaze-controlled typing systems and machine learning algorithms, attackers can reconstruct typed text with high precision, highlighting the need for enhanced security measures in virtual spaces. As these technologies continue to evolve, addressing vulnerabilities like GAZEploit will be crucial to protecting user privacy and security in immersive digital environments.