Immersive Holographic Simulator for ACL Reconstruction Training with an LLM-based Voice Assistant

Andrea Gómez Díaz, Oscar Araque & Sergio Muñoz López (2026). Immersive Holographic Simulator for ACL Reconstruction Training with an LLM-based Voice Assistant. In XLIV Congreso Anual de la Sociedad Española de la Ingeniería Biomédica 2026.

Abstract:
Anterior cruciate ligament (ACL) reconstruction is one of the most frequent procedures in orthopaedic knee surgery, but its training is limited by the scarce availability of cadaver laboratories and supervised operating room hours. Existing extended reality simulators partly address this gap, yet head-mounted display platforms confine each user to an individual environment and current artificial intelligence modules act as procedural proctors rather than clinical knowledge assistants. This work presents a prototype surgical simulator that integrates two complementary subsystems: an immersive three-dimensional simulation of the four main ACL reconstruction phases, rendered on a large-format holographic display that supports multi-user visualisation; and a voice-based intelligent assistant that combines a large language model with a Retrieval-Augmented Generation pipeline grounded on a curated corpus of ACL clinical literature. The subsystems run independently and communicate through UDP. A proof-of-concept demonstration confirms the technical feasibility of this integration, with the trainee navigating the simulation by voice and consulting the assistant without leaving the immersive scene.