Interactive Virtual ChatBots
The Institute for Digital Intermedia Arts (IDIA Lab) at Ball State University has developed a demonstration of how artificial intelligence can drive real-time gestural and vocal responses in 3D virtual characters within the Unity game engine. This project showcases a new generation of AI-enabled non-player characters (NPCs) capable of engaging users through natural, conversational interaction. Each virtual character is built from customizable persona profiles, allowing creators to define not only appearance but also personality, knowledge domain, tone, and behavior. Users can communicate with these NPCs through either voice or text, and the system leverages AI to interpret the sentiment, intent, and context of the conversation. The characters then respond with synchronized facial animation, gestures, and expressive vocal delivery, producing an interaction that feels dynamic and emotionally aware.
To achieve this, the IDIA Lab integrates ChatGPT and complementary AI technologies directly into Unity, creating a seamless pipeline between language processing, animation control, and real-time rendering. This architecture allows the virtual characters to adapt to an expansive range of scenarios and roles—from historically informed personae to specialized technical experts capable of guiding users through complex content. By combining large language models with responsive 3D animation systems, the Lab’s work demonstrates how AI can transform NPC interactions into rich, immersive exchanges that go far beyond scripted dialogue. This approach enables new forms of experiential learning, and simulation where every conversation can be unique, contextual, and interactive.