Immersive virtual reality (VR) simulation is a multi-sensory experience where participants don headsets and are immersed inside a realistic, virtual clinical environment. Computer vision based hand tracking is the future of immersive VR simulation and is practical to use with the latest commercially available VR headsets. With this new technology, participants interact within the simulated clinical environment with their hands instead of controllers, thus allowing for a more natural and immersive experience. Hand tracking technology is critical to the realism and fidelity of immersive VR healthcare simulations, as it allows participants to use their hands as they would in a real-life clinical setting. Exciting developments have led to a democratization of immersive VR technology with intuitive VR simulation authoring tools that empower simulation educators, technologists, and operations specialists to use their existing expertise to independently design and develop VR simulation scenarios. This allows for the development of meaningful simulations that meet the unique objectives and outcomes of the specific target learners. The integration of hand tracking technology with these simulation authoring tools unlocks new mechanics for those designing immersive VR simulations for healthcare education. This includes the ability to assess fine psychomotor skills and nonverbal communication.

This presentation provides an overview of computer vision based hand tracking technology in immersive VR simulation. Attendees will learn about the benefits of hand tracking, the limitations, and the future applications of this exciting new technology in healthcare simulation.

Learning Objectives:  

• Discuss the use of hand tracking technology in immersive VR simulation
• Analyze the benefits and limitations of using hand tracking technology in immersive VR simulation
• Describe the future applications of hand tracking technology in immersive VR simulation