GPTCacheを試してみる

!pip install -q gptcache

import os
from google.colab import userdata

os.environ["OPENAI_API_KEY"] = userdata.get('OPENAI_API_KEY')

import time
import openai


def response_text(openai_resp):
    return openai_resp['choices'][0]['message']['content']


question = 'what‘s github?'

# OpenAI API original usage
start_time = time.time()
response = openai.ChatCompletion.create(
  model='gpt-3.5-turbo',
  messages=[
    {
        'role': 'user',
        'content': question
    }
  ],
)
print(f'Question: {question}')
print("Time consuming: {:.2f}s".format(time.time() - start_time))
print(f'Answer: {response_text(response)}\n')

Question: what‘s github?
Time consuming: 3.11s
Answer: GitHub is a web-based platform that allows developers to collaborate on and share code. It provides a version control system called Git, which allows multiple people to work on a project simultaneously and track changes made to the code. GitHub also offers features like issue tracking, project management, and code review, making it a popular platform for software development. It is widely used in the open-source community and by many companies for collaborative software development.

import time


def response_text(openai_resp):
    return openai_resp['choices'][0]['message']['content']

print("Cache loading.....")

# To use GPTCache, that's all you need
# -------------------------------------------------
from gptcache import cache
from gptcache.adapter import openai

cache.init()
cache.set_openai_key()
# -------------------------------------------------

question = "what's github"
for _ in range(2):
    start_time = time.time()
    response = openai.ChatCompletion.create(
      model='gpt-3.5-turbo',
      messages=[
        {
            'role': 'user',
            'content': question
        }
      ],
    )
    print(f'Question: {question}')
    print("Time consuming: {:.2f}s".format(time.time() - start_time))
    print(f'Answer: {response_text(response)}\n')

Cache loading.....
start to install package: redis
successfully installed package: redis
start to install package: redis-om
successfully installed package: redis-om
Question: what's github
Time consuming: 2.72s
Answer: GitHub is a web-based platform for version control and collaboration that allows developers to work together on projects, record changes made to code, and share their work with others. It uses the Git version control system, which allows multiple developers to make changes to a project simultaneously, while keeping track of those changes and allowing for easy collaboration and merging of code. GitHub provides a graphical interface and various features for managing repositories, tracking issues, and facilitating project collaboration. It is widely used in the software development community for managing and hosting source code.

start to install package: tiktoken
successfully installed package: tiktoken
Question: what's github
Time consuming: 7.84s
Answer: GitHub is a web-based platform for version control and collaboration that allows developers to work together on projects, record changes made to code, and share their work with others. It uses the Git version control system, which allows multiple developers to make changes to a project simultaneously, while keeping track of those changes and allowing for easy collaboration and merging of code. GitHub provides a graphical interface and various features for managing repositories, tracking issues, and facilitating project collaboration. It is widely used in the software development community for managing and hosting source code.

question = "what's Google Colaboratory"

Cache loading.....
Question: what's Google Colaboratory
Time consuming: 2.45s
Answer: Google Colaboratory, also known as Colab, is a free cloud-based Jupyter notebook environment offered by Google. It allows users to write and execute Python code in a web browser without the need for any setup or installation. Colab provides access to powerful computing resources, including GPUs and TPUs, and allows collaborative editing, sharing, and commenting on notebooks. Users can import data, install packages, and run code cells to perform data analysis, deep learning, machine learning, and other data-related tasks conveniently.

Question: what's Google Colaboratory
Time consuming: 0.00s
Answer: Google Colaboratory, also known as Colab, is a free cloud-based Jupyter notebook environment offered by Google. It allows users to write and execute Python code in a web browser without the need for any setup or installation. Colab provides access to powerful computing resources, including GPUs and TPUs, and allows collaborative editing, sharing, and commenting on notebooks. Users can import data, install packages, and run code cells to perform data analysis, deep learning, machine learning, and other data-related tasks conveniently.

from gptcache import cache
from gptcache.adapter import openai

cache.init()
cache.set_openai_key()

import time


def response_text(openai_resp):
    return openai_resp['choices'][0]['message']['content']

from gptcache import cache
from gptcache.adapter import openai
from gptcache.embedding import Onnx
from gptcache.manager import CacheBase, VectorBase, get_data_manager
from gptcache.similarity_evaluation.distance import SearchDistanceEvaluation

print("Cache loading.....")

onnx = Onnx()
data_manager = get_data_manager(CacheBase("sqlite"), VectorBase("faiss", dimension=onnx.dimension))
cache.init(
    embedding_func=onnx.to_embeddings,
    data_manager=data_manager,
    similarity_evaluation=SearchDistanceEvaluation(),
    )
cache.set_openai_key()

questions = [
    "what's github",
    "can you explain what GitHub is",
    "can you tell me more about GitHub",
    "what is the purpose of GitHub"
]

for question in questions:
    start_time = time.time()
    response = openai.ChatCompletion.create(
        model='gpt-3.5-turbo',
        messages=[
            {
                'role': 'user',
                'content': question
            }
        ],
    )
    print(f'Question: {question}')
    print("Time consuming: {:.2f}s".format(time.time() - start_time))
    print(f'Answer: {response_text(response)}\n')

Cache loading.....
start to install package: transformers
successfully installed package: transformers
start to install package: onnxruntime==1.14.1
successfully installed package: onnxruntime==1.14.1
Downloading (…)okenizer_config.json: 100%
465/465 [00:00<00:00, 13.9kB/s]
Downloading (…)lve/main/config.json: 100%
827/827 [00:00<00:00, 26.8kB/s]
Downloading spiece.model: 100%
760k/760k [00:00<00:00, 5.22MB/s]
Downloading (…)/main/tokenizer.json: 100%
1.31M/1.31M [00:00<00:00, 9.53MB/s]
Downloading (…)cial_tokens_map.json: 100%
245/245 [00:00<00:00, 10.9kB/s]
Downloading model.onnx: 100%
46.9M/46.9M [00:00<00:00, 54.6MB/s]
start to install package: faiss-cpu
successfully installed package: faiss-cpu
Question: what's github
Time consuming: 4.07s
Answer: GitHub is a development platform that allows developers to store and manage their code repositories in a centralized and distributed version control system called Git. It provides a web-based interface for collaborating with other developers, managing projects, tracking issues, and organizing code contributions. GitHub also allows users to showcase their work by creating and hosting websites directly from their repositories. It is widely used in the software development industry for open-source projects, as well as for private code repositories and team collaborations.

Question: can you explain what GitHub is
Time consuming: 1.33s
Answer: GitHub is a development platform that allows developers to store and manage their code repositories in a centralized and distributed version control system called Git. It provides a web-based interface for collaborating with other developers, managing projects, tracking issues, and organizing code contributions. GitHub also allows users to showcase their work by creating and hosting websites directly from their repositories. It is widely used in the software development industry for open-source projects, as well as for private code repositories and team collaborations.

Question: can you tell me more about GitHub
Time consuming: 1.07s
Answer: GitHub is a development platform that allows developers to store and manage their code repositories in a centralized and distributed version control system called Git. It provides a web-based interface for collaborating with other developers, managing projects, tracking issues, and organizing code contributions. GitHub also allows users to showcase their work by creating and hosting websites directly from their repositories. It is widely used in the software development industry for open-source projects, as well as for private code repositories and team collaborations.

Question: what is the purpose of GitHub
Time consuming: 0.80s
Answer: GitHub is a development platform that allows developers to store and manage their code repositories in a centralized and distributed version control system called Git. It provides a web-based interface for collaborating with other developers, managing projects, tracking issues, and organizing code contributions. GitHub also allows users to showcase their work by creating and hosting websites directly from their repositories. It is widely used in the software development industry for open-source projects, as well as for private code repositories and team collaborations.

from gptcache.core import cache, Config
from gptcache.manager import get_data_manager, CacheBase, VectorBase
from gptcache.similarity_evaluation.distance import SearchDistanceEvaluation
from gptcache.embedding import OpenAI

openai_embedding = OpenAI()

data_manager = get_data_manager(CacheBase("sqlite"), VectorBase("faiss", dimension=openai_embedding.dimension))
cache.init(embedding_func=openai_embedding.to_embeddings,
           data_manager=data_manager,
           similarity_evaluation=SearchDistanceEvaluation(),
           )
cache.set_openai_key()

from gptcache.manager import get_data_manager, CacheBase, VectorBase
from gptcache.embedding import OpenAI

openai_embedding = OpenAI()

data_manager = get_data_manager(CacheBase("mysql"), VectorBase("milvus", dimension=openai_embedding.dimension), max_size=100, eviction='LRU') 

!pip install -q gptcache redis redis-om openai faiss-cpu tiktoken
!apt update && apt install sqlite3

import os
from google.colab import userdata

os.environ["OPENAI_API_KEY"] = userdata.get('OPENAI_API_KEY')

import time
from gptcache import cache
from gptcache.adapter import openai
from gptcache.manager import CacheBase, VectorBase, get_data_manager
from gptcache.similarity_evaluation.distance import SearchDistanceEvaluation
from gptcache.embedding import OpenAI


def response_text(openai_resp):
    return openai_resp['choices'][0]['message']['content']


print("Cache loading.....")

openai_embedding = OpenAI()
data_manager = get_data_manager(CacheBase("sqlite"), VectorBase("chromadb", dimension=openai_embedding.dimension))
cache.init(
    embedding_func=openai_embedding.to_embeddings,
    data_manager=data_manager,
    similarity_evaluation=SearchDistanceEvaluation(),
    )
cache.set_openai_key()

questions = [
    "what's github",
    "can you explain what GitHub is",
    "can you tell me more about GitHub",
    "what is the purpose of GitHub"
]

for question in questions:
    start_time = time.time()
    response = openai.ChatCompletion.create(
        model='gpt-3.5-turbo',
        messages=[
            {
                'role': 'user',
                'content': question
            }
        ],
    )
    print(f'Question: {question}')
    print("Time consuming: {:.2f}s".format(time.time() - start_time))
    print(f'Answer: {response_text(response)}\n')

Question: what's github
Time consuming: 2.62s
Answer: GitHub is a web-based platform used for version control and collaboration in software development projects. It provides features like code repository hosting, issue tracking, pull requests, and code review. GitHub allows developers to work together on projects, share and contribute to open-source projects, and keep track of changes made to the codebase. It is widely used in the software development community and has a vast collection of public repositories covering various programming languages and frameworks.

Question: can you explain what GitHub is
Time consuming: 1.44s
Answer: GitHub is a web-based platform used for version control and collaboration in software development projects. It provides features like code repository hosting, issue tracking, pull requests, and code review. GitHub allows developers to work together on projects, share and contribute to open-source projects, and keep track of changes made to the codebase. It is widely used in the software development community and has a vast collection of public repositories covering various programming languages and frameworks.

Question: can you tell me more about GitHub
Time consuming: 0.15s
Answer: GitHub is a web-based platform used for version control and collaboration in software development projects. It provides features like code repository hosting, issue tracking, pull requests, and code review. GitHub allows developers to work together on projects, share and contribute to open-source projects, and keep track of changes made to the codebase. It is widely used in the software development community and has a vast collection of public repositories covering various programming languages and frameworks.

Question: what is the purpose of GitHub
Time consuming: 0.14s
Answer: GitHub is a web-based platform used for version control and collaboration in software development projects. It provides features like code repository hosting, issue tracking, pull requests, and code review. GitHub allows developers to work together on projects, share and contribute to open-source projects, and keep track of changes made to the codebase. It is widely used in the software development community and has a vast collection of public repositories covering various programming languages and frameworks.

Question: what's github
Time consuming: 0.14s
Answer: GitHub is a web-based platform used for version control and collaboration in software development projects. It provides features like code repository hosting, issue tracking, pull requests, and code review. GitHub allows developers to work together on projects, share and contribute to open-source projects, and keep track of changes made to the codebase. It is widely used in the software development community and has a vast collection of public repositories covering various programming languages and frameworks.

Question: can you explain what GitHub is
Time consuming: 0.13s
Answer: GitHub is a web-based platform used for version control and collaboration in software development projects. It provides features like code repository hosting, issue tracking, pull requests, and code review. GitHub allows developers to work together on projects, share and contribute to open-source projects, and keep track of changes made to the codebase. It is widely used in the software development community and has a vast collection of public repositories covering various programming languages and frameworks.

Question: can you tell me more about GitHub
Time consuming: 0.14s
Answer: GitHub is a web-based platform used for version control and collaboration in software development projects. It provides features like code repository hosting, issue tracking, pull requests, and code review. GitHub allows developers to work together on projects, share and contribute to open-source projects, and keep track of changes made to the codebase. It is widely used in the software development community and has a vast collection of public repositories covering various programming languages and frameworks.

Question: what is the purpose of GitHub
Time consuming: 0.14s
Answer: GitHub is a web-based platform used for version control and collaboration in software development projects. It provides features like code repository hosting, issue tracking, pull requests, and code review. GitHub allows developers to work together on projects, share and contribute to open-source projects, and keep track of changes made to the codebase. It is widely used in the software development community and has a vast collection of public repositories covering various programming languages and frameworks.

$ sqlite3 sqlite.db 
SQLite version 3.37.2 2022-01-06 13:25:41
Enter ".help" for usage hints.
sqlite> .tables
gptcache_answer        gptcache_question_dep  gptcache_session     
gptcache_question      gptcache_report   

sqlite> .schema gptcache_question
CREATE TABLE gptcache_question (
        id INTEGER NOT NULL, 
        question VARCHAR(3000) NOT NULL, 
        create_on DATETIME, 
        last_access DATETIME, 
        embedding_data BLOB, 
        deleted INTEGER, 
        PRIMARY KEY (id)
);
sqlite> .schema gptcache_answer 
CREATE TABLE gptcache_answer (
        id INTEGER NOT NULL, 
        question_id INTEGER NOT NULL, 
        answer VARCHAR(3000) NOT NULL, 
        answer_type INTEGER NOT NULL, 
        PRIMARY KEY (id)
);
sqlite> .schema gptcache_question_dep 
CREATE TABLE gptcache_question_dep (
        id INTEGER NOT NULL, 
        question_id INTEGER NOT NULL, 
        dep_name VARCHAR(1000) NOT NULL, 
        dep_data VARCHAR(3000) NOT NULL, 
        dep_type INTEGER NOT NULL, 
        PRIMARY KEY (id)
);
sqlite> .schema gptcache_session
CREATE TABLE gptcache_session (
        id INTEGER NOT NULL, 
        question_id INTEGER NOT NULL, 
        session_id VARCHAR(1000) NOT NULL, 
        session_question VARCHAR(3000) NOT NULL, 
        PRIMARY KEY (id)
);
sqlite> .schema gptcache_report
CREATE TABLE gptcache_report (
        id INTEGER NOT NULL, 
        user_question VARCHAR(3000) NOT NULL, 
        cache_question_id INTEGER NOT NULL, 
        cache_question VARCHAR(3000) NOT NULL, 
        cache_answer VARCHAR(3000) NOT NULL, 
        similarity FLOAT NOT NULL, 
        cache_delta_time FLOAT NOT NULL, 
        cache_time DATETIME, 
        extra VARCHAR(3000), 
        PRIMARY KEY (id)
);

import chromadb

chroma_client = chromadb.Client()
print(chroma_client.list_collections())

[Collection(name=gptcache)]

collection = chroma_client.get_collection("gptcache")
print(collection.get())

{'ids': [], 'embeddings': None, 'metadatas': [], 'documents': []}

questions = [
    "what's github",
    "can you explain what GitHub is",
    "can you tell me more about GitHub",
    "what is the purpose of GitHub"
]

question = questions[0]

start_time = time.time()
response = openai.ChatCompletion.create(
    model='gpt-3.5-turbo',
    messages=[
        {
            'role': 'user',
            'content': question
        }
    ],
)
print(f'Question: {question}')
print("Time consuming: {:.2f}s".format(time.time() - start_time))
print(f'Answer: {response_text(response)}\n')

Question: what's github
Time consuming: 2.84s
Answer: GitHub is a web-based platform used for version control and collaboration. It allows software developers to track changes to their code, manage and share their code repositories, and collaborate on projects with other developers. GitHub provides features such as code hosting, pull requests, issue tracking, and project management tools. It is a widely used platform for open-source projects and a popular choice among developers for sharing and contributing to code repositories.

sqlite> .mode line
sqlite> select * from gptcache_question;
            id = 1
      question = what's github
     create_on = 2023-11-05 11:57:46.577177
   last_access = 2023-11-05 11:57:46.577183
embedding_data = 8kF|v<Ms7<C;&<:41,%=Cͼ[fna@=='w;}݌<s%g?ؼu<w:jZPC(y:h<i˼;WPCoe;5V<A<1<ӕGnI<_źPX+z<<])ݺ݇S<Il< <) ݌<};NvǼСeD<u9fq<;0p<<&Q<@<%ݮÔ%<6k[<ל'/5;&<̼
     ӼU<
        Tf<<#h<|<$5;A<<;[<aM;#;:>P<1b<jZ<Tf.M;A(
-(C=>мx<Tf.1='<"<
       deleted = 0

sqlite> select * from gptcache_answer;
         id = 1
question_id = 1
     answer = GitHub is a web-based platform used for version control and collaboration. It allows software developers to track changes to their code, manage and share their code repositories, and collaborate on projects with other developers. GitHub provides features such as code hosting, pull requests, issue tracking, and project management tools. It is a widely used platform for open-source projects and a popular choice among developers for sharing and contributing to code repositories.
answer_type = 0

sqlite> select * from gptcache_question_dep;;
sqlite> select * from gptcache_session;
sqlite> select * from gptcache_report;

from pprint import pprint

collection = chroma_client.get_collection("gptcache")
pprint(collection.get(include=["embeddings", "metadatas", "documents"]))

{'documents': [None],
 'embeddings': [[-0.012110523879528046,
                 0.008817311376333237,
                 -0.011196923442184925,
(snip)
                 -0.006296055857092142,
                 -0.004369704052805901,
                 -0.005417866166681051]],
 'ids': ['1'],
 'metadatas': [None]}