Inspiration & Concept
“Harry Potter” Film Series
In the Harry Potter movies, the magical portraits are incredibly fascinating. The characters within the portraits come to life and can engage in conversations. This greatly inspired me, and I came up with the idea of having players interact with various magical portraits in the game. Additionally, by incorporating AI into the game’s NPCs, this game allows AI to take on the roles of these portraits.
Portraits Fantasy
An game Proposal focus on AI-based NPCs, enabling players to have free-form voice conversations with magical portraits in an old mansion, unraveling mysteries through interactive dialogue.
Expected Gameplay
Exploration
In the game, players have the freedom to explore the old house, uncovering the background story through items like letters and photos. Guided by a narrator, players gain insights into the game's narrative and discover crucial clues. Subsequently, after obtaining these clues, further exploration is required to find their practical applications.
Puzzle Solveing
The game proposed in this concept aims to help a young girl uncover the secrets of her grandfather's old house and learn about her grandfather's past. Therefore, players need to engage in conversations with various magical portraits, search for clues, find key memory fragments, and ultimately piece together all the fragments into complete memories.
A crucial aspect of the puzzle-solving involves dialogues and even confrontations with the portraits, where the player's emotions can influence the portraits' responses.
Core Functions
Speech-to-Text
I tested the speech input functionality in Unity using the provided SDK feature by OpenAI. This feature, part of the speech dialogue, serves as the initial step for input. It exhibits high accuracy and supports multiple languages. This confirms the feasibility of integrating speech input functionality into the game.
AI-Based Conversation
Install the plugin provided by Sarge and import the AI conversation demo. After inputed my OpenAI API key, I can start the conversation within Unity. In the example in the images below, I had OpenAI play the role of a stern castle guard, where the player had to speak the password to pass through. I had control over its conversation style, response length, and conversation boundaries. During testing, I tried to deceive AI, but it consistently responded flexibly to reject the me. This is the first step to the AI-based NPCs.
Practical AI-NPC Solution
After conducting feasibility tests and research, I have confirmed the feasibility of integrating AI into NPCs. However, to truly enable natural voice conversations with NPCs in the game, it is necessary to combine the three core technologies. This involves writing code to pass content generated by different programs to the next program, completing the entire process of Speech-to-AI Speech, and optimizing the user experience.
*This image is generated using AI
Old House & Magical Portraits
In the game, a young girl takes on the role of the main character. She deeply loves her grandfather, but he’s quite eccentric. Fueled by curiosity, she decides to explore her grandfather’s old house after his passing, hoping to uncover the mysteries of her grandfather’s past.
Natural Language Dialogue
The core of this proposal lies in AI NPCs. Magic portraits in the game are powered by AI, allowing players to use natural language input. The portraits respond accordingly based on predefined character settings. If, for instance, a player speaks a specific content like a password, the portrait provides key information. Engaging in dialogue with these portraits enables players.
I tested the core functions based on the SDK created by a developer, Sarge who generously sharing it along with tutorial videos on YouTube.
Text-to-Speech
The functionality of text-to-speech is already widely used in the market with applications like Google Text-to-Speech, Amazon Polly, and Microsoft Azure, among others. I have also found relevant plugins in the Unity Store, all of which offer editable real-time speech synthesis capabilities. With this, the feasibility of output has been validated.
Future Plan
Further Application
During the research process, I discovered that there are already well-established solutions for AI-based NPCs, and they have been applied in numerous game projects. Moving forward, I plan to conduct further research in this field to explore a more feasible AI-based NPC production pipeline. The ultimate goal is to create a practical demo with significant value.
