iTranslated by AI
Building an AITuber with ChatGPT API, VOICEVOX, and LangChain
Hello! In this article, I will show you how to create your own AITuber using OpenAI's ChatGPT API, VOICEVOX, and LangChain. By reading this, you will learn how to leverage AI technology to create a virtual character that outputs real-time conversations in voice.
Tools and Libraries Required
First, you will need the following tools and libraries:
- OpenAI ChatGPT API
- VOICEVOX
- Python virtual environment (venv)
- LangChain (optional)
Setting up a Virtual Environment
Use a Python virtual environment to install the necessary libraries. This allows you to manage project dependencies independently without interfering with other projects or the entire system.
python -m venv voicevox_env
voicevox_env\Scripts\activate
Setting up the ChatGPT API
Get your OpenAI API key and save it in a .env file. This allows you to manage your API key securely.
Installing Packages
pip install openai requests
Setting up VOICEVOX
We will use VOICEVOX to convert text to speech. Furthermore, we will implement a function to play the generated audio files.
Download and install VOICEVOX from the link below.
Installing Packages
pip install pyaudio SpeechRecognition
Setting up LangChain
LangChain is a tool for saving the conversation history of the ChatGPT API. We will use LangChain's PostgresChatMessageHistory to store the conversation history in a PostgreSQL database.
This is not strictly necessary, but it allows you to retain the memory of the ChatGPT API.
Installing PostgreSQL
Download PostgreSQL from the site below and install the file for your operating system.
You can refer to the following site for installation instructions.
During the installation, a prompt will appear asking you to set a password for the postgres user. This password will be needed later when accessing the database, so please be sure not to forget it.
You will also be asked for settings such as the port number (default is 5432) and locale. If the default settings are fine for you, you can proceed with them.
Once the installation is complete, a PostgreSQL management tool called pgAdmin will also be installed. Launch pgAdmin, expand "Servers" in the tree view on the left, and connect to the PostgreSQL server.
Within pgAdmin, right-click on the PostgreSQL server and select "Create" > "Database...". Enter a name for the new database and click OK to create it.
Installing Packages
pip install langchain[llms] psycopg psycopg-binary
Implementation of the Code
Converting Voice to Text
def voice_to_text():
r = sr.Recognizer()
with sr.Microphone(sample_rate=16000) as source:
print("Please say something")
audio = r.listen(source)
audio_data = BytesIO(audio.get_wav_data())
audio_data.name = 'from_mic.wav'
transcript = openai.Audio.transcribe('whisper-1', audio_data)
return transcript['text']
Getting Responses Using the ChatGPT API
def chat(messages: list) -> str:
result = openai.ChatCompletion.create(
model='gpt-3.5-turbo',
messages=messages
)
response_text = result['choices'][0]['message']['content']
return response_text
Converting Text to Voice Using VOICEVOX
def post_audio_query(text: str) -> dict:
params = {'text': text, 'speaker': 1}
res = requests.post('http://localhost:50021/audio_query', params=params)
return res.json()
def post_synthesis(audio_query_response: dict) -> bytes:
params = {'speaker': 1}
headers = {'content-type': 'application/json'}
audio_query_response_json = json.dumps(audio_query_response)
res = requests.post(
'http://localhost:50021/synthesis',
data=audio_query_response_json,
params=params,
headers=headers
)
return res.content
def text_to_voice(text: str):
audio_query_response = post_audio_query(text)
audio_data = post_synthesis(audio_query_response)
play_wavfile(audio_data)
Playing Audio Files
def play_wavfile(wav_file: bytes):
wr: wave.Wave_read = wave.open(io.BytesIO(wav_file))
p = pyaudio.PyAudio()
stream = p.open(
format=p.get_format_from_width(wr.getsampwidth()),
channels=wr.getnchannels(),
rate=wr.getframerate(),
output=True
)
chunk = 1024
data = wr.readframes(chunk)
while data:
stream.write(data)
data = wr.readframes(chunk)
sleep(0.5)
stream.close()
p.terminate()
Main Function
def main():
EXIT_PHRASE = 'exit'
messages = [
{'role': 'system', 'content': 'You are a helpful assistant.'},
{'role': 'user', 'content': f'Please return only {EXIT_PHRASE} if you are asked to end the conversation, such as "finish" or "stop".'}
]
exit_flag = False
while not exit_flag:
text = voice_to_text()
messages.append({'role': 'user', 'content': text})
response = chat(messages)
if response == EXIT_PHRASE:
exit_flag = True
response = 'See you later!'
messages.append({'role': 'assistant', 'content': response})
print(f'User : {text}')
print(f'ChatGPT: {response}')
audio_data = post_synthesis(post_audio_query(response))
play_wavfile(audio_data)
if __name__ == '__main__':
main()
(LangChain) Saving Chat Message History Using Postgres
Use the PostgresChatMessageHistory class to connect to the Postgres database to add and retrieve messages.
Specify the connection information for the Postgres database in connection_string.
In this code, I am using the username postgres, password mypassword, host localhost, and database name chat_history, but you will need to change these to match your actual environment.
class ChatHistory:
def __init__(self):
self.history = PostgresChatMessageHistory(
connection_string = DB_CONNECTION_STRING,
session_id="foo",
)
def add_user_message(self, message):
self.history.add_user_message(message)
def add_ai_message(self, message):
self.history.add_ai_message(message)
def get_messages(self):
return self.history.messages
Implementation and Troubleshooting
Various issues and errors may occur while implementing the actual code.
Getting an error when installing psycopg2 instead of psycopg.
ModuleNotFoundError: No module named 'psycopg'
Conclusion
I introduced how to create your own AITuber using the ChatGPT API, VOICEVOX, and LangChain. You can resolve most of your questions by asking ChatGPT if you get stuck. Please use this article as a reference and try creating your own AITuber!
References
Discussion