GPUStackでAPIを叩いて出力する方法

GPUStackにはAPIがあるようで、それを叩けたらプログラム内で便利だと思ったので、試してみました。

前提条件

• 自分のPCにGPUStackをインストールしていて、IPアドレスで接続確認できること。
• GPUStack内でAPIキーを発行していること。
• 対象のモデルをGPUStack内にSaveしていること。

コード詳細

下記コードを、test.pyなどの名前で適当に保存して実行します。

import requests
import json

# ユーザーにプロンプトをTerminal上で入力させる
user_input = input("プロンプトを入力してください: ")

url = "http://192.XXX.XX.XX/v1-openai/chat/completions" #自身のPCのIPアドレスを入力する

headers = {
    "accept": "application/json",
    "Authorization": "Basic XXXXXXXXXXXXXXXXXX", #APIKeyを入力する
    "Content-Type": "application/json"
}

payload = {
    "model": "llama3.2",　#対象のモデルを入力する
    "messages": [
        {"role": "user", "content": user_input}
    ]
}

response = requests.post(url, headers=headers, data=json.dumps(payload))

print("Status code:", response.status_code)
print("Response body:", response.text)

このプログラムでは、プロンプトを入力するメッセージが出るので、そこに適当な内容を入力します。

プロンプトを入力してください: Please explain the theory of Darwinism comparing other evolutuion theory.

そうすると、下記のように出力が出ます。

出力

Status code: 200
Response body: {"id":"chatcmpl-FZwktT6dM7xcGWlRLjYP3YskLSSoxZ2Y","created":1749279870,"model":"llama3.2","object":"chat.completion","choices":[{"index":0,"finish_reason":"stop","message":{"content":"Charles Darwin's theory of evolution by natural selection is one of the most well-known and widely accepted scientific theories of its time. In this explanation, I'll compare Darwin's theory to other major evolution theories.\n\nDarwin's Theory of Evolution by Natural Selection\n\nIn 1859, Darwin proposed that all living organisms share a common ancestor and that species evolve through the process of variation, mutation, genetic drift, and gene flow, ultimately leading to the diversity of life on Earth. He argued that all species, including humans, are descendants of a single species of protozoan.\n\nKey components of Darwin's theory:\n\n1. Variation: All individuals within a population exhibit genetic variation, which can result from mutation, genetic drift, or gene flow.\n2. Heritability: The characteristics of an individual are determined by its genetic makeup, which is inherited from its parents.\n3. Adaptation: The variation in individuals is favored and retained in the population if it provides a selective advantage, leading to the survival and reproduction of the most fit individuals.\n4. Speciation: The process of evolution resulting in the creation of new species is thought to occur over time, leading to the diversity of life on Earth.\n\nComparing Darwin's theory to other evolution theories:\n\n1. Theistic Evolution Theory (Creationism):\n\t* Theistic evolution theory proposes that God created the universe and all living organisms, and that humans are uniquely created by God.\n\t* Key features of theistic evolution theory include a literal interpretation of the Bible and the idea that God intervened in the evolutionary process.\n\t* While theistic evolution theory acknowledges the complexity and diversity of life, it often rejects the idea of gradual evolution through natural selection.\n2. Intelligent Design Theory:\n\t* Intelligent design theory proposes that certain features of living organisms are too complex to have evolved through natural means.\n\t* Key features of intelligent design theory include the complexity of the human genome and the presence of "irreducible complexity," which suggests that certain biological processes are incompatible with evolutionary theory.\n\t* Intelligent design theory rejects the idea of natural selection and instead proposes that a designer intervened in the evolutionary process.\n3. Evolution by Genetic Drift:\n\t* Evolution by genetic drift proposes that small changes in the gene pool of a population can result in the evolution of new traits over time.\n\t* Key features of evolutionary by genetic drift include the random nature of genetic changes and the potential for rapid evolution.\n\t* This theory is often seen as a precursor to the modern evolutionary synthesis, which combines elements of Darwin's theory with the idea of genetic drift.\n4. Neutral Evolutionary Theory:\n\t* Neutral evolution theory proposes that all evolutionary changes occur due to random genetic mutations and drift, with no selective advantage.\n\t* Key features of neutral evolutionary theory include the idea that the process of evolution is driven by the same forces that drive genetic drift, rather than natural selection.\n\t* Neutral evolution theory is often seen as a more accurate representation of the process of evolution, as it acknowledges the role of randomness and genetic drift in shaping the evolution of species.\n\nKey differences between Darwin's theory and other evolution theories:\n\n1. Mechanism of evolution: Darwin's theory proposes that evolution is driven by natural selection, while intelligent design theory proposes that a designer intervened in the evolutionary process.\n2. Mechanism of natural selection: Darwin's theory describes natural selection as a process of favoring the survival and reproduction of the most fit individuals, while neutral evolution theory proposes that natural selection is a random process.\n3. Mechanism of genetic drift: Darwin's theory acknowledges the role of genetic drift in shaping the evolution of species, while intelligent design theory rejects the idea of genetic drift as a viable mechanism of evolution.\n\nConclusion\n\nCharles Darwin's theory of evolution by natural selection remains one of the most widely accepted scientific theories of its time. While other evolution theories have been proposed over the years, Darwin's theory has stood the test of time as the most comprehensive and widely accepted explanation of the evolution of life on Earth. However, the scientific community remains open to new evidence and alternative theories, and ongoing research continues to refine and expand our understanding of evolution.","role":"assistant"},"logprobs":null}],"usage":{"completion_tokens":859,"prompt_tokens":25,"total_tokens":884,"time_to_first_token_ms":729.202,"time_per_output_token_ms":23.01157275902212,"tokens_per_second":43.4563952004511}}

まとめ

GPUStack内のAPIを叩けると、別のプログラム内で様々なモデルを呼び出せるようになります。

GPUStackはローカルLLMを動作させる基盤として使えるので、様々なローカルLLMをパッとインストールして使えるとワクワクしてます。これを使って色々試していきます！