Design and Development of an Agentic RAG System for Generating Responsive Holographic Avatars in Virtual Museums

Fran Cantos. (2026). Design and Development of an Agentic RAG System for Generating Responsive Holographic Avatars in Virtual Museums. Final Career Project (TFM). Universidad Politécnica de Madrid, ETSI Telecomunicación.

Abstract:
Cultural heritage sites are increasingly adopting immersive technologies and artificial in- telligence to attract new audiences and support more active forms of cultural engagement. This thesis presents the design and development of an agentic RAG system for generating responsive holographic avatars in virtual museums. Building upon a previously developed holographic virtual museum that included several exhibits and a trivia game but no spoken interaction, this work extends its functionality by introducing a holographic avatar pow- ered by an agentic system that acts as the cognitive core, capable of maintaining casual conversation, explaining the artworks found in each exhibit, answering visitor questions, and generating personalized trivia questions informed by both the prior conversation and cultural information. The system combines LLMs with retrieval-augmented generation and prompt-engineering strategies to produce a spoken audioguide, together with dynamic difficulty-control methods based on Bloom’s Taxonomy and semantic similarity to generate question sets organized into easy, medium, and hard levels. The generated audioguide and trivia questions are evaluated using a multi-judge LLM-based pipeline, and the complete museum experience is assessed via a within-subjects post-experience questionnaire completed by 41 participants and analyzed using PLS-SEM. Results indicate that audioguides generated through retrieval-augmented generation are judged relevant to visitors’ questions, and that role-based prompting, combined with a hybrid difficulty-control method, produces a more consistent perceived ordering of difficulty across levels than the alternatives tested. The user study provides preliminary self-reported evidence that participants perceived the experience as personalized and preferred it over a non-personalized alternative, that ease of use positively affects gaming engagement, and, notably, that user satisfaction positively affects intentions to return. These findings should be interpreted as exploratory, given the absence of objective learning measures and a control condition.