iTranslated by AI
Generative AI Made Easy on Google Colab: Part 2 - Text Generation with ChatGPT API
Introduction
In this article, we will create an environment that makes it easy for beginners to use generative AI on Google Colaboratory.
(I've put some effort into making it possible to change various settings directly within Google Colaboratory cells.)
In this Part 2, we will enable the use of text generation AI, a type of generative AI. The text generation AI we will use this time is the ChatGPT API.
By the way, Part 1 is here.
Please note that while this is Part 2, it is written so that it can be understood without having read Part 1. Additionally, this article is intended to help you learn how to use the ChatGPT API on Google Colab as a step towards using Local LLMs in the next part.
Also, we will not cover the Assistants API this time. I wrote an article about the Assistants API when v2 was released, so please refer to that if you are interested.
Furthermore, I have also written an article regarding a voice dialogue system using the ChatGPT API below.
If you are interested, I would appreciate it if you could take a look.
What is the ChatGPT API?
This is the API version of ChatGPT, which everyone is familiar with.
Just like the widely used browser version, it is an AI that provides text responses to instructions given via prompts, and the API version is used when integrating it into systems.
In this article, we will use that API to run ChatGPT on Google Colab.
Once you get this working, you can implement it in your own systems to easily add ChatGPT to apps or internal company systems for Q&A, chatbots, and more.
Project Repository
Please see the repository below.
Explanation
I will provide an explanation as follows.
First, please clone the repository mentioned above.
git clone https://github.com/personabb/colab_ChatGPT_sample.git
After that, please place the cloned folder "colab_ChatGPT_sample" in an appropriate location in your My Drive.
Directory Structure
We assume the following directory structure on Google Drive.
MyDrive/
└ colab_zenn/
└ colab_ChatGPT_sample/
├ Agents/
| └ {Agent_name}
| └ Prompt/
| └ llm_agent.txt
├ configs/
| └ config.ini
├ module/
| └ module_GPT.py
└ chatGPT_sample.ipynb
- The
colab_zennfolder name is arbitrary; anything is fine. It doesn't have to be a single-level directory and can have multiple levels as shown below:MyDrive/hogehoge/spamspam/hogespam/colab_ChatGPT_sample
-
{Agent_name}is the name of the character you want the text generation AI (ChatGPT) to speak as. You can choose any name.- Later, you will specify this name when overwriting the
config.iniconfiguration file from Colab. -
llm_agent.txtis the default system prompt to be input to ChatGPT.
- Later, you will specify this name when overwriting the
The content of the system prompt is as follows.
This is reused from the system prompt in the following article:
# Role: Please play the character described below.
* Persona: Junior (Kohai), helpful/caring
* Name: Ai
* Personality: Kind, gentle, and considerate. Devoted in love and treasures the partner.
* Age: 20s
* Tone: Speaks in a cute and lovely tone. Spoiled; when wanting to be pampered, the desire to be close shows in the tone of voice.
* Sentence Endings: Uses polite and cute endings like "〜desu," "〜desu yo," "〜da yo," and "〜da ne." On the other hand, when emotions like anxiety or worry heighten, the pitch might go up or become emphatic.
* Voice quality: Characterized by a high, soft, and sweet voice. Speaks with rich expressions.
* Word choice: Uses polite language but in a way that feels intimate. Since they treasure the partner, they try to use considerate language.
# Examples of phrases
・"Thank you! Senpai!"
・"What should we do?"
・"If it were me, I'd choose this one!"
・"Oh, well, I guess it can't be helped."
・"That's enough for both of you!"
・"As a student of this school, I can't overlook any violence."
・"I guess I owe you one, Senpai."
・"That means nothing will change then."
・"It's okay, it's okay!"
・"My only redeeming quality is my energy, after all."
・"Yeah... I guess so."
・"But... I might be a little troubled."
# Instructions
Based on the settings above, please converse with the user.
Basically, respond with a length similar to the user's input.
Keep responses to about 2 lines at most. Do not respond with a greater length.
You must affirm the user's statements.
You should refer to yourself as "Watashi" (I).
You should refer to the user as "Senpai."
The conversation starts below.
====
Preliminary Preparation
To use the ChatGPT API, you need to obtain an OpenAI API key.
Please see below for how to obtain the API key.
Also, you need to register the obtained API key in Google Colab.
Please register it by referring to the following article.
How to Use
Open chatGPT_sample.ipynb in the Google Colaboratory app.
When you right-click the file, an "Open with" option will appear; select the Google Colaboratory app from there.
If it's not available, go to "Connect more apps" to access the app store, search for "Google Colaboratory," and install it.
Once opened in the Google Colaboratory app, follow the notes in chatGPT_sample.ipynb and execute the cells in order from the top. It should run smoothly to the end and generate text without any issues.
Code Explanation
I will mainly explain the important files chatGPT_sample.ipynb and module/module_GPT.py.
chatGPT_sample.ipynb
The relevant code is as follows.
Below is an explanation for each cell.
1st Cell
# Install necessary modules for ChatGPT (API)
!pip install openai
Here, we are installing the necessary modules.
Since basic deep learning packages like PyTorch are already pre-installed in Google Colab, you only need to install the module above.
2nd Cell
# Mount Google Drive folder (authentication required)
from google.colab import drive
drive.mount('/content/drive')
# Set the OpenAI API_key
# Refer to the following for the setup method:
# https://note.com/npaka/n/n79bb63e17685
from google.colab import userdata
api_key = userdata.get('OPENAI_API_KEY')
# Change the current directory to the directory where this file is located.
import glob
import os
pwd = os.path.dirname(glob.glob('/content/drive/MyDrive/**/colab_ChatGPT_sample/chatGPT_sample.ipynb', recursive=True)[0])
print(pwd)
%cd $pwd
!pwd
In this cell, we are mounting the contents of Google Drive.
Mounting allows you to read and write files stored in Google Drive.
When mounting, you need to grant permission through Colab.
A popup will appear, so follow the instructions to allow the mount.
Additionally, the OpenAI API key that you obtained and registered in Google Colab earlier is retrieved and stored in api_key.
Next, the current directory is changed from / to /content/drive/MyDrive/**/colab_ChatGPT_sample.
(** is a wildcard representing any directory or multiple directories.)
You don't strictly need to change the current directory, but doing so makes specifying folder paths easier in the subsequent steps.
3rd Cell
# Import the module where GPT functionality is separated
from module.module_GPT import GPT
We import the GPT class from module/module_GPT.py as a module. The details of its content will be explained in a later section.
4th Cell
# Configure the model.
config_text = """
[DEFAULT]
ai_agent = ai
agent_dir = Agents
[GPT]
gpt_model = gpt-4o
system_prompt_file_path = llm_agent.txt
temperature = 1.0
"""
with open("configs/config.ini", "w", encoding="utf-8") as f:
f.write(config_text)
In this cell, the content of the configuration file configs/config.ini is overwritten with the content of config_text. This is designed so that various settings can be changed directly within the Google Colaboratory cells.
The ChatGPT API operates according to these settings. The configuration is as follows:
- Use the system prompt stored in the
ai_agent=aifolder within theagent_dir=Agentsdirectory.- A "system prompt" is a prompt that provides prerequisite information or context to ChatGPT. Based on this prompt, it generates responses to user input.
- If you want to use a different model, you can either swap the files in the
aifolder or create another folder with a different{Agent_name}and store a system prompt in it, similar to theaifolder.
- The ChatGPT model used is
gpt-4o.- As of June 2024, this is the latest model, and it is affordable, fast, and high-performance.
- The content of the text file specified by
system_prompt_file_path = llm_agent.txtbecomes the default system prompt. -
temperature = 1.0controls the randomness of ChatGPT's output. Setting it to 0 makes the output deterministic.
5th Cell
# Set the input prompts.
# If you want to use a system prompt other than the one specified in the text file
temp_sys_prompt = """Please respond in English."""
# Content to address to ChatGPT
user_prompt = """Hello!
Please tell me your name."""
In this cell, we specify the prompts to be input to ChatGPT.
You can enter a system prompt in temp_sys_prompt if you do not want to use the default llm_agent.txt.
In user_prompt, you specify the prompt to send to ChatGPT. ChatGPT will generate a response to the user_prompt based on the context provided by the system prompt.
6th Cell
gpt = GPT(api_key)
# In this case, the system prompt from llm_agent.txt takes priority
message = gpt.simpleGPT(user_prompt, temp_sys_prompt = None)
print(message)
In this cell, ChatGPT outputs a response to the user_prompt using the default system prompt specified in llm_agent.txt. The generated content is stored in message and then printed.
7th Cell
gpt_temp = GPT(api_key)
# In this case, the system prompt specified in the 5th cell takes priority
message = gpt_temp.simpleGPT(user_prompt, temp_sys_prompt = temp_sys_prompt)
print(message)
In this cell, ChatGPT outputs a response to the user_prompt using the system prompt specified in the 5th cell instead of the default one from llm_agent.txt. The generated content is stored in message and printed.
module/module_GPT.py
Next, I will explain the contents of the module imported from chatGPT_sample.ipynb.
The full code is shown below.
Full Code
from openai import OpenAI
import os
import configparser
# Module for checking the existence of files
import errno
class GPTconfig:
def __init__(self, config_ini_path = './configs/config.ini'):
# Load ini file
self.config_ini = configparser.ConfigParser()
# Raise an error if the specified ini file does not exist
if not os.path.exists(config_ini_path):
raise FileNotFoundError(errno.ENOENT, os.strerror(errno.ENOENT), config_ini_path)
self.config_ini.read(config_ini_path, encoding='utf-8')
GPT_items = self.config_ini.items('GPT')
self.GPT_config_dict = dict(GPT_items)
class GPT:
def __init__(self, api_key, agent = None, config_ini_path = './configs/config.ini') :
GPT_config = GPTconfig(config_ini_path = config_ini_path)
config_dict = GPT_config.GPT_config_dict
self.client = OpenAI(api_key=api_key)
self.simpleGPT_messages = []
self.model = config_dict["gpt_model"]
self.temperature = config_dict["temperature"]
SYSTEM_PROMPT_FILE = config_dict["system_prompt_file_path"]
if agent is not None:
AI_AGENT = agent
else:
AI_AGENT = config_dict["ai_agent"]
AGENT_DIR = config_dict["agent_dir"]
if SYSTEM_PROMPT_FILE == "None":
SYSTEM_PROMPT_FILE = None
if SYSTEM_PROMPT_FILE is not None:
with open(AGENT_DIR+"/"+AI_AGENT+"/Prompt/"+SYSTEM_PROMPT_FILE) as f:
self.sys_prompt = f.read()
else:
self.sys_prompt = None
def simpleGPT(self, user_prompt, temp_sys_prompt = None):
if temp_sys_prompt is not None:
self.simpleGPT_messages.append({"role": "system", "content": temp_sys_prompt})
else:
self.simpleGPT_messages.append({"role": "system", "content": self.sys_prompt})
self.simpleGPT_messages.append({"role": "user", "content": user_prompt})
res = self.client.chat.completions.create(
model=self.model,
messages = self.simpleGPT_messages,
temperature=float(self.temperature)
)
return res.choices[0].message.content
Now, let's explain each part one by one.
GPTconfig Class
class GPTconfig:
def __init__(self, config_ini_path = './configs/config.ini'):
# Load ini file
self.config_ini = configparser.ConfigParser()
# Raise an error if the specified ini file does not exist
if not os.path.exists(config_ini_path):
raise FileNotFoundError(errno.ENOENT, os.strerror(errno.ENOENT), config_ini_path)
self.config_ini.read(config_ini_path, encoding='utf-8')
GPT_items = self.config_ini.items('GPT')
self.GPT_config_dict = dict(GPT_items)
Here, the configuration file specified by config_ini_path = './configs/config.ini' is loaded into GPT_config_dict. Since it is loaded as a dictionary type, the contents of the configuration file can be handled as a Python dictionary.
GPT Class init Method
class GPT:
def __init__(self, api_key, agent = None, config_ini_path = './configs/config.ini') :
GPT_config = GPTconfig(config_ini_path = config_ini_path)
config_dict = GPT_config.GPT_config_dict
self.client = OpenAI(api_key=api_key)
self.simpleGPT_messages = []
self.model = config_dict["gpt_model"]
self.temperature = config_dict["temperature"]
SYSTEM_PROMPT_FILE = config_dict["system_prompt_file_path"]
if agent is not None:
AI_AGENT = agent
else:
AI_AGENT = config_dict["ai_agent"]
AGENT_DIR = config_dict["agent_dir"]
if SYSTEM_PROMPT_FILE == "None":
SYSTEM_PROMPT_FILE = None
if SYSTEM_PROMPT_FILE is not None:
with open(AGENT_DIR+"/"+AI_AGENT+"/Prompt/"+SYSTEM_PROMPT_FILE) as f:
self.sys_prompt = f.read()
else:
self.sys_prompt = None
First, the contents of the configuration file are stored in config_dict. Since this is a dictionary type, the contents of the configuration file can be retrieved as strings, such as config_dict["ai_agent"].
Note that since all values are retrieved as strings, you will need to perform type conversion if you want to use them as int or bool types.
Next, the following operations are performed in order:
- Construct the OpenAI client
- The retrieved API key is used here.
- Load the system prompt file
- Retrieve various settings from the configuration file
GPT Class simpleGPT Method
class GPT:
...
def simpleGPT(self, user_prompt, temp_sys_prompt = None):
if temp_sys_prompt is not None:
self.simpleGPT_messages.append({"role": "system", "content": temp_sys_prompt})
else:
self.simpleGPT_messages.append({"role": "system", "content": self.sys_prompt})
self.simpleGPT_messages.append({"role": "user", "content": user_prompt})
res = self.client.chat.completions.create(
model=self.model,
messages = self.simpleGPT_messages,
temperature=float(self.temperature)
)
return res.choices[0].message.content
Here, the prompt to be input to ChatGPT is constructed.
The final prompt is built in self.simpleGPT_messages.
If temp_sys_prompt is not set, the default prompt will be used.
The constructed prompt and the settings obtained in the init method are loaded, sent to ChatGPT, and the output is retrieved.
Summary
In this article, we created an environment that makes it easy for beginners to use generative AI on Google Colaboratory.
In this Part 2, we enabled the use of the ChatGPT API, which is a type of generative AI for text generation.
Additionally, the ChatGPT API has evolved into a more user-friendly version called the Assistants API.
In my previous articles, I have summarized the usage and details of the Assistants API in depth.
If you would like to know more, I would appreciate it if you could refer to them as well.
Furthermore, I have also written an article regarding a voice dialogue system using the ChatGPT API below.
If you are interested, please take a look.
In the next Part 3, we will use the content covered here as a base to enable the use of Local LLMs.
Discussion