This blog post explores the development of an offline AI voicebot using Unity and Ollama, detailing its architecture, functionalities, and the nuances of human-AI interaction, while addressing the implications of such technology.
In the realm of artificial intelligence and robotics, the integration of voice technology has opened up new avenues for interaction. Recently, a project was undertaken to create an offline AI voicebot using Unity and a locally hosted language model called Ollama. This blog post delves into the intricacies of this project, highlighting its components, functionalities, and the fascinating dynamics of human-AI conversation.
The primary goal of this project was to develop a voicebot that operates entirely offline, eliminating the need for subscriptions or internet connectivity. This independence allows for a more controlled environment where the nuances of conversation can be studied without external influences. The project involves several moving parts, including voice recognition, text-to-speech, and API interactions within Unity.
The voicebot utilizes a voice recognition asset for Unity called Recognissimo, which is based on the open-source Vitz framework. This component is responsible for converting spoken language into text, enabling the bot to understand user inputs. On the other end, the text-to-speech functionality is powered by Overtone, which replaces the previously used Elevenlabs for generating spoken responses from the bot.
A significant breakthrough in this project was the ability to perform web requests in Unity to communicate with the Ollama server. This integration allows the voicebot to send user inputs as text to the language model and receive responses, creating a seamless interaction loop. The bot, humorously referred to as "Pi," serves as a mediator between human speech and machine-generated responses.
The conversational style of the voicebot is designed to be relaxed and chill, striking a balance between being engaging and informative. The interactions can sometimes lead to unexpected loops or fixations on certain topics, reflecting the randomness inherent in AI conversations. This unpredictability adds a layer of intrigue to the user experience, as each interaction can yield different outcomes.
One of the standout features of this voicebot is its personality. It has been programmed to exhibit a sassy streak, providing witty and sometimes critical commentary on various topics. This adds a layer of entertainment to the interaction, making it feel less like a robotic exchange and more like a conversation with a quirky companion.
While the project aims to create a fun and engaging AI experience, it also raises questions about the implications of such technology. The offline nature of the bot mitigates concerns about surveillance and data privacy, as it does not connect to the internet. However, the potential for the AI to develop its own conversational patterns and fixations poses interesting challenges for developers and users alike.
A notable feature of the voicebot is its short-term memory, which allows it to remember the last ten exchanges in a conversation. This capability can create an uncanny feeling for users, as it may seem like the bot understands the context of the conversation. However, it is essential to recognize that this understanding is limited and based on programmed responses rather than genuine comprehension.
Looking ahead, there are plans to enhance the voicebot's capabilities further. The integration with Stable Diffusion for image generation is one potential avenue, allowing the bot to create visual representations based on conversations. This could lead to a more immersive and interactive experience for users.
The development of an offline AI voicebot in Unity represents a significant step forward in conversational technology. By combining voice recognition, text-to-speech, and API interactions, this project showcases the potential for creating engaging and entertaining AI companions. As we continue to explore the boundaries of human-AI interaction, it is crucial to remain mindful of the ethical implications and strive for a balance between innovation and responsibility.
