Effective clinical handovers are vital for seamless patient care and strong teamwork in healthcare.
To build essential skills, competence and confidence, Nursing students benefit greatly from realistic, interactive practice. The AI-Enhanced Virtual Reality Simulation System in Nursing practice, co-created by the innovative team led by Dr. Maggie Chan*, Dr. Benney Wong, Mr. Abraham Wan at the School of Nursing, and Dr. Carson Hung and Mr. Ziv Tai at TALIC, were designed to prepare nursing students for professional nursing communications through cutting edge simulation technology.
Advancing Skills Through Interactive Practice
Nursing students thrive when given opportunities for hands-on, contextual learning that mirrors real clinical environments. In the simulated program, nursing instructors were created as humanoid avatars, i.e. NPCs, who interact with the nursing students in the virtual clinical environment. AI technology is integrated into the nursing instructor avatars and virtual environment to provide realistic responses and feedback much like in real life clinical handover scenarios. These virtual simulations enable nursing students to safely cultivate robust communication skills in clinical handover scenarios such as using structured frameworks including ISBAR (Identification, Situation, Background, Assessment, Recommendation), in order that these skills can be applied to their real-life counterparts.
Co-Creating AI-Enhanced VR Simulations
Names (left to right):
- Mr. Wai Hin Wan, Assistant Lecturer, School of Nursing, Li Ka Shing Faculty of Medicine, HKU
- Mr. Ziv Tai, TALIC, HKU
- Dr. Carson Ka Shun Hung, Lecturer / E-learning Technologist, TALIC, HKU
- Dr. Jannie Roed, Director, TALIC, HKU
- Dr. Maggie Mee Kie Chan, Senior Lecturer, School of Nursing, PolyU
- Dr. Benney Yiu Cheong Wong, Lecturer, School of Nursing, Li Ka Shing Faculty of Medicine, HKU
- Mr. Pak Hin Lai, Student, School of Nursing, Li Ka Shing Faculty of Medicine, HKU
- Mr. Kin Fung Chan, Student, School of Nursing, Li Ka Shing Faculty of Medicine, HKU
The project’s strength lies in its dual focus: cutting-edge technology and a thoughtful co-creation process. Strong collaboration between faculty from the School of Nursing, TALIC and artificial intelligence tools is what made the VR project a success. Realistic clinical scenarios aligned with professional standards designed by nursing educators were transformed with TALICs facilitation into an effective VR training tool, with strong feedback from students who give valuable insights on how best the tool can support their training in clinical handover.
How It Works: Immersive Training with Real-Time Feedback
A huge dimension of the responsive and realistic nature of the VR tool is built up by AI technology. By integrating AI capabilities such as ChatGPT and Azure Speech Studio, the simulation enables fluid, unscripted conversations and real-time adaptation to individual needs.
Precisely how this AI technology is integrated comes down to specific input data and scenarios provided by nursing professionals and TALIC’s facilitation of AI technological use. The simulation immerses students in authentic clinical settings, such as a medical ward or Accident and Emergency Department (AED), tailored to their year level—Year 3 undergraduates handle simpler cases in 7 minutes, while Year 5 students tackle complex ones in 10-12 minutes. Participants are required to take care of a patient, including taking vital signs and providing nursing care according to the patient’s condition, and then deliver an ISBAR-structured clinical handover to an AI nurse afterwards. Powered by GPT-4o, the AI analyzes communication patterns, tone, and completeness in real-time, offering prompts for missing details and instant feedback. Post-session, students get an AI-generated report with scores, areas for improvement, and gamified elements like task-based points.
What Do Students Say? Feedback and Confidence-Boosting
The simulation was rolled out to over 430 nursing students across HKU’s Bachelor and Master programmes, and feedback from this run has shown clear benefits to student’s clinical handover training. Pre- and post-tests show enhanced self-efficacy, while rubrics for ISBAR and qualitative responses reveal improved communication skills. Students report feeling more prepared for practicums. Qualitative feedback from nursing students also show enjoyment and positive feedback: “It was a surprisingly fun and innovative way to perform clinical simulations”
A Blueprint for Future Professional Education
This AI-enhanced VR simulation demonstrates how collaborative, technology-integrated approaches can transform professional training at HKU. Looking ahead, exciting developments such as AI-driven character animation, dynamic facial expressions, and more natural voice synthesis will create an even richer and more immersive learning environment.
*Dr. Maggie Chan is the initiator of the project and is formerly affiliated with HKU.