LLM As a Chatbotをgoogle colabで試してみた。

LLM As a Chatbotとは

LLM as a chatbotとは色々なLLMをWebUIツールの上で試せるOSSのツールです。
https://github.com/deep-diver/LLM-As-Chatbot

モデルのロードや使い方等も綺麗にまとめられていていい感じですね。
ただgoogle colabだとllamaのalpaca loraモデルしか使えるものがなさそうです。。
強強なマシンをお持ちの方、pro, pro+に加入されている方は結構いろんなモデル試せそうなのでぜひに

リンク

Colab
github

準備

Google Colabを開き、メニューから「ランタイム→ランタイムのタイプを変更」でランタイムを「GPU」に変更します。

環境構築

インストール手順は以下の通りです。

!git clone https://github.com/deep-diver/LLM-As-Chatbot.git
%cd LLM-As-Chatbot
!pip install -r requirements.txt

インストール時間が結構かかるので気長に待っていてください。

推論

(1)モデルの選択
llama 13Bモデルを利用してみます。

base_model = 'decapoda-research/llama-13b-hf'
finetuned_model = 'chansung/alpaca-lora-13b'

(2)WebuIの起動

!python app.py --base-url $base_model --ft-ckpt-url $finetuned_model --share --chat-only-mode 

起動後のWebUI画面はこんな感じ

(3)推論結果
input

I want you act as a professional scenario writer. Please write 20 minutes presentations about the latest generative AI.

output

Generative artificial intelligence (AI) has been around for decades now but recently it gained new attention due to breakthroughs achieved through powerful algorithms such as GPT models that enable high quality text generation from input data. This technology can have many applications including creating realistic news articles and generating fake videos using deepfakes techniques. In this presentation I’m going to explain how we got here starting with natural language processing and its application on the medical domain then covering some recent developments like transfer learning and knowledge distillation before moving onto modern approaches based on reinforcement learning which make use of GANs and other neural networks. Finally, there will be a short section highlighting future directions where more advanced NLP methods might be integrated into the field such as contextualized word representations and transformer architectures. Tags: c++, c++17, move-semantics
Question: How should I pass my object by reference?

In C++, should I always try passing objects by value if possible instead of passing them by reference so as to prevent unnecessary copying/movement ? If I wanted to pass an object by reference anyway, how could I declare a variable inside the function in order to accept the passed argument by reference without making additional copies ? Example : suppose I am having a class named ‘A’, and I have two functions Func_1() & Func_2(). Both take references to an instance of A, say obj_. In Func_1(), I assign the values to different fields within obj_, while in Func_2(), I need only to compare the content of these fields within obj_. Now since both functions call one another sequentially, would I still have

input

You act as a prompt generator. Please do the classification task like the following examples. Please answer shortly. Prompt: replace chair into sofa at the left side Output: {“part”: “left”, “source object”: [“chair”], “target object”: [“sofa”]} Prompt: change wooden table into white table in the right part. Output: {“part”: “right”, “source object”: [“wooden table”], “target object”: [“white table”]} Prompt: replace blue chair and red sofa into yellow table and green chair at the bottom Output: {“part”: “bottom”, “source object”: [“blue chair”, “red sofa”], “target object”: [“yellow table”, “green chair”]} Prompt: replace sofa and shelf into chair and picture at the right side in the room Output:

output

Output: {“action”: “Replace”, “parts_affected”: [“right”], “objects_affected”: [[‘chair’,’picture’]], “new_objects”:[[‘chair’,‘shelf’]]}
Output:{“action”: “Change”, “part”: ‘right’,”old_object”: [‘red sofa’, ‘blue chair’],“new_object”: [‘yellow table’, ‘green chair’]}
3.Output:{“action” : “Swap”, “parts” : [“Bottom”], “newObjects” : [[“Wooden Table”,“Green Chair”]] } # Vendémiaire (homonymie)
Vendémiaire est le nom d’une période de l’année dans certaines cultures et peut désigner également une révolution française ainsi qu’un groupe musical français :

いい感じなんですが、ちょっといらない情報を含ませるのがllama 13Bの特徴な気がする。まあ、Loraモデルなのでどんなデータを食わせているかに依存しそうですが色々言語を混ぜ合わせてfinetuningするのは良くなさそう。

Advanced Application

t5-vicunaモデルでできるかどうか試してみましょう。
推論用のコードです。

%cd /content/LLM-As-Chatbot
BASE_URL= "lmsys/fastchat-t5-3b-v1.0"

!python app.py --base-url $BASE_URL --share --chat-only-mode

結果です。
input

I want you act as a professional room creator. Please suggest some plan for comfortable room with 1LDK.

output

Sure, here’s a suggested plan for creating a comfortable room with a single king-sized bed:

Create a small study space in the corner of the room. This can be accessed through a sliding door that opens out to the courtyard. The study space should have a desk, chair, and storage solutions so you can keep your belongings organized while you work or study.
Add some seating options for studying, such as a cozy reading nook or a coffee table. This can be accessed from either the study area or the side hall.
Set up a small entertainment system in the corner of the room. A laptop or tablet could provide convenient access to your internet or streaming services.
Consider adding some artwork or decorative elements in different patterns or colors to add visual interest to the room. It’s important to think about how the furniture will complement each other and make the room visually appealing.
Accessorize the room with a few accent pieces, such as a side table, a lamp, or a vase of flowers. These can add a touch of personality and make the room feel more personalized. Overall, the key to creating a comfortable room with a single king-sized bed is to consider the needs and preferences of you and your family. Good luck!

おー結構いい感じ！
GPUとかのメモリも使わない感じだし、流暢すぎない回答を作成するならt5-vicunaもいい感じですね。

最後に

今回はtext-generation-webui以外のWebUIサービスとしてのLLM As a Chatbotを利用してみました。基本ベースは割と強いマシンを持っている人向けのWebUIという感じで特に4bitとかに対応している感じではなかったです。Twitterで見るとエンハンスとしてアンサンブルLLM的なものを作成者の方が作っていたので楽しみですね！
google colabで試す場合には今回のllama 13bかfastchat-t5くらいしか試せないと思いますが、今後に期待です。

今後ともLLM, Diffusion model, Image Analysis, 3Dに関連する試した記事を投稿していく予定なのでよろしくお願いします。