An immersive VR training application for the Meta Quest 3 in which pharmaceutical field reps communicate with AI-driven virtual doctors — across realistic channels, with automated three-dimensional scoring and individual feedback.
Client
Internationales Pharmaunternehmen
Industry
Healthcare
Solution
eLearningConversational AI
Technology
Virtual RealitySpatial / XRAI / ML
Services
Creative Strategy & StorytellingUX DesignSoftware Development3D Art ProductionAI Architecture & Engineering
Communication training you don't read — you live it.
The challenge.
How can a pharmaceutical field sales force systematically and scalably train its conversational skills — without risking real doctor conversations? Role-plays are staff-intensive and hard to scale, classic e-learnings convey knowledge but not communicative action, and feedback stays subjective and hard to measure. What was needed was a training format that makes communicative skill tangible, is available anytime, and makes individual progress visible.
Our solution.
A complete VR training application for the Meta Quest 3. Field reps communicate with AI-driven virtual doctors in an immersive environment — across realistic communication channels, with automated scoring and individualized feedback. The app combines three core technologies into one seamless experience: virtual reality for maximum presence, generative AI for natural, context-aware reactions, and speech-to-text as the primary control of the entire application.
Virtual doctors with real character.
Personality-based training partners.
The training partners are not generic characters. Each virtual doctor has a defined personality profile that drives all of their communication behavior — reactions, preferred communication styles, and openness to specific content and channels. The system learns from the prior interaction history and responds context-aware to the user's training history.
A complete communication mix.
Email — AI-assisted.
Users dictate bullet points or the desired content — the AI turns them into a complete email draft. Via voice command it can be revised and optimized before being submitted for analysis. The virtual doctor's replies appear directly in the communication history.
Service drop — leave-behind.
A simulation of a document-based doctor interaction. From pre-configured clinical studies and documents — or self-created materials — users assemble an individual service package, complemented by an AI-generated cover letter and an optional business card. The AI assessment factors in the full description of each document.
1:1 meeting — in real time.
The most immersive format: a fully immersive virtual one-on-one conversation with the AI doctor. Users speak completely naturally — no preset answer options, no script. The virtual doctor reacts in real time based on its personality profile and the conversation so far. This trains exactly what matters in the real conversation.
Measurably better: three-dimensional AI scoring.
Communication strength
How well does the written or spoken communication fit the doctor's personality profile? Tonality, word choice, and further communicative elements are assessed.
Content impact
Does the communicated content match what the doctor needs to hear based on their profile and prior history? The AI evaluates against the complete interaction history.
Channel effectiveness
Was the chosen communication channel the right one? Assessed depending on content type, personality type, and prior communication.
Voice-first & mixed reality.
The entire app experience is voice-first. The primary input method is voice control — writing emails, creating cover letters, and describing documents all work via speech, accompanied by visual spectrogram feedback. It is complemented by controller-free gesture control via hand tracking. By default the app runs in mixed-reality mode: real environment and virtual UI elements at once — only the 1:1 meeting is fully immersive.
Enterprise-ready & privacy-compliant.
The app runs as a standalone application on the Meta Quest 3 — without a PC connection. Login is handled via enterprise single sign-on through the integrated VR web browser. All training data stays stored locally on the device; there is no central backend for user data. An online connection is required solely for the AI cloud services and authentication.
