Stable Vicuna13Bをgoogle colab freeで動かしてみた

Stable Vicunaとは

Vicuna 13Bと言うLLMモデル（versionはv0）をさらに細かく指示で微調整し、RLHFで学習させたバージョンで、細かく命令調整したものです。Vicuna13B自身はLLaMa(MetaAI)の13BモデルをShareGPTから得られたユーザー同士の会話を利用してfinetuningされたモデルになっています。
Stable Vicunaについては以下のStability AIの記事が詳しいです。

前提

Stable Vicuna 13Bモデルを実際に試そうとすると26GBの重みを読み込んで、GPU runtimeに載せないといけないために30GBメモリ程度のCPUと30GBメモリ程度のGPU(もしくは複数台)必要になります。今回はGoogle colabのfree枠で利用できるようにするために4bit化したものを利用して動かしてみたいと思います。

リンク等

今回利用したcolabは以下です。サクッと試してみたい方はいますぐにGo!
Colab
Github
Medium

環境のセットアップ

まずgoogle colabを開いて「ランタイム」から「ランタイムのタイプを変更」をクリックして、

上記の写真のようにGPUのsettingsをします。
次に、環境構築からしていきます。今回使用するのは以下の2つ
https://github.com/oobabooga/text-generation-webui
https://github.com/oobabooga/GPTQ-for-LLaMa

text-generation-webuiはWebUIでchat, APIなど様々な形式でLLMのモデルを試せるすごいOSSです。
GPTQ-for-LLaMaは4bit化されたLLMモデルを推論させる際に利用します。このレポジトリを利用すると色々なLLMモデルを4bit化して低メモリで動かせる重みが手に入るのですごくいいです。

それではInstallationです。

!git clone https://github.com/oobabooga/text-generation-webui
%cd text-generation-webui
!pip install -r requirements.txt
!git clone https://github.com/oobabooga/GPTQ-for-LLaMa.git -b cuda
%cd GPTQ-for-LLaMa
!python setup_cuda.py install
!cp -r /content/text-generation-webui/GPTQ-for-LLaMa/* /content/text-generation-webui/

setting upには5~10分ほどかかるのでゆっくり待ってください。

推論

モデルのダウンロード

まずhuggingfaceからモデルをダウンロードしてきましょう。ダウンロードする際には、text-generation-webuiのdownload-model.pyを利用します。

# download model
%cd /content/text-generation-webui
!python download-model.py TheBloke/stable-vicuna-13B-GPTQ

今回利用させていただいたhuggingfaceのモデルは以下の通りです。このTheBlokeさんのレポジトリは他にも色々上がってて参考になります。

https://huggingface.co/TheBloke/stable-vicuna-13B-GPTQ

チャット形式での推論実行

次にchat形式で推論を実行します。

%cd /content/text-generation-webui
!python3 server.py --model TheBloke_stable-vicuna-13B-GPTQ --wbits 4 --group 128 --chat --share --auto-devices  --model_type=llama

このコマンドを実行後、https://***********.gradio.live/　のようなリンクがcolab上のconsoleに表示されるのでそのリンクを踏むとWebUIに飛べる。
推論実行後のWebUIは以下の通り。

推論結果

GPT3.5-turboと比較してみましょう。

• room creator
  input

You are a professional room planner. Please tell me about room creation tips in detail

GPT3.5-turbo

  Certainly! As a professional room planner, I can provide you with some   useful tips to create a well-designed and functional room. Here are some tips:

Determine the Purpose of the Room: The first step in creating a room is to determine its purpose. Will it be a living room, bedroom, home office, or entertainment space? This will help you decide on the layout, furniture, and decor of the room.

Measure the Space: Before you start designing the room, measure the space carefully to ensure that you know the dimensions of the room. This will help you determine the appropriate furniture and layout for the space.

Choose a Focal Point: Every room should have a focal point that draws the eye and anchors the space. This could be a fireplace, a piece of art, or a large window with a view. Once you have chosen your focal point, arrange your furniture around it to create a cohesive and inviting space.

Consider the Traffic Flow: When designing a room, it's important to consider the traffic flow. You don't want furniture to block doorways or walkways, as this can create a cramped and uncomfortable space. Make sure there is enough space to move around and that the flow is logical and intuitive.

Choose the Right Furniture: The furniture you choose should be both functional and stylish. Consider the size of the room and the layout when selecting furniture. Choose pieces that are proportional to the space and that fit with the overall style and aesthetic of the room.

Add Lighting: Lighting is an important element of room design. Make sure to include a variety of lighting options, including overhead lighting, table lamps, and floor lamps. This will allow you to create the right ambiance and mood in the room.

Use Color and Texture: Color and texture can add depth and interest to a room. Choose a color scheme that complements the furniture and decor, and use a variety of textures to add visual interest.

Add Accessories: Accessories can help tie a room together and add personality and style. Choose accessories that fit with the overall aesthetic of the room, such as artwork, throw pillows, and rugs.

By following these tips, you can create a well-designed and functional room that is both comfortable and stylish.

StableVicuna

  Here are some general tips for creating rooms:

1. Start by measuring the space you have available, including any doors or windows that may affect your layout options.
2. Determine what function each room will serve (e.g., bedroom, living room, kitchen). This can help guide decisions on furniture placement and design elements.
3. Consider the size of the room when choosing furniture. Make sure to leave enough walking space around beds, sofas, etc.
4. Use rugs or area rugs to define different areas within a larger room. For example, a rug under a dining table creates a separate seating area from the rest of the floor.
5. Choose lighting fixtures based on both style and functionality. Include task lights over work surfaces like desks or countertops if needed.
6. Don't be afraid to mix and match styles – combining modern pieces with traditional

• Task Planner
  input

You act as a professional prompt generator.  Please generate response like the following. 
Examples:
Input: {"object detection": {"chair": 0.997, "couch": 0.965, "table": [0.563, 0.674]}, "image caption": "kitchen room with european style"} Output: In this picture, there are 4 objects. 1 chair with score 99.7%, 1 couch with score 96.5% and 2 tables with score 56.3% and 67.4%. I performed object detection and image caption. and it looks like kitchen room with european style. 

Practice:
Input: {"object detection": {"cat": 0.97, "dog": 0.83, "bird": [0.763, 0.68, 0.986], "giraffe": [0.95, 0.37]}, "image caption": "amazon animals in the zoo"} Output:

GPT3.5-turbo

I ran object detection on this image and identified several animals. There is 1 cat with a confidence score of 97%, 1 dog with a confidence score of 83%, 3 birds with confidence scores of 76.3%, 68%, and 98.6%, and 2 giraffes with confidence scores of 95% and 37%. Based on my analysis, this appears to be a picture of animals at an Amazon zoo.

Stable Vicuna

There are 4 objects detected in this photo. The cat has a high accuracy rate of 97%, while the dog is less accurate at 83%. Two birds were identified with scores of 76.3%, 68%, and 98.6%, respectively. Finally, a giraffe was also spotted with a score of 95%. Overall, this appears to be a scene featuring various Amazonian animals in a zoo setting.

• Summarizer/Extractor
  input

Context: 
Certainly! As a professional room planner, I can provide you with some   useful tips to create a well-designed and functional room. Here are some tips:

Determine the Purpose of the Room: The first step in creating a room is to determine its purpose. Will it be a living room, bedroom, home office, or entertainment space? This will help you decide on the layout, furniture, and decor of the room.

Measure the Space: Before you start designing the room, measure the space carefully to ensure that you know the dimensions of the room. This will help you determine the appropriate furniture and layout for the space.

Choose a Focal Point: Every room should have a focal point that draws the eye and anchors the space. This could be a fireplace, a piece of art, or a large window with a view. Once you have chosen your focal point, arrange your furniture around it to create a cohesive and inviting space.

Consider the Traffic Flow: When designing a room, it's important to consider the traffic flow. You don't want furniture to block doorways or walkways, as this can create a cramped and uncomfortable space. Make sure there is enough space to move around and that the flow is logical and intuitive.

Choose the Right Furniture: The furniture you choose should be both functional and stylish. Consider the size of the room and the layout when selecting furniture. Choose pieces that are proportional to the space and that fit with the overall style and aesthetic of the room.

Add Lighting: Lighting is an important element of room design. Make sure to include a variety of lighting options, including overhead lighting, table lamps, and floor lamps. This will allow you to create the right ambiance and mood in the room.

Use Color and Texture: Color and texture can add depth and interest to a room. Choose a color scheme that complements the furniture and decor, and use a variety of textures to add visual interest.

Add Accessories: Accessories can help tie a room together and add personality and style. Choose accessories that fit with the overall aesthetic of the room, such as artwork, throw pillows, and rugs.

By following these tips, you can create a well-designed and functional room that is both comfortable and stylish.  

Please extract keywords from the above context.

GPT3.5-turbo

Keywords:

- Room planning
- Purpose of the room
- Measuring the space
- Focal point
- Traffic flow
- Furniture selection
- Lighting
- Color and texture
- Accessories

Stable Vicuna

- Tips for creating a well-designed and functional room
- Determine the purpose of the room before starting
- Measure the space accurately
- Choose a focal point for the room
- Consider the traffic flow when arranging furniture
- Select the right furniture proportionally sized for the room
- Add lighting to set the desired atmosphere
- Utilize color and texture to enhance the look of the room
- Decorate with accessories to complete the theme

• 日本語への対応
  input

サッカーを上手になる方法を教えて

GPT3.5-turbo

サッカーを上手になるためには、以下のような方法があります。

1.基本技術の練習：サッカーには基本的な技術があります。例えば、ボールコントロール、パス、シュートなどです。これらの基本技術を継続的に練習することで、自分のレベルを上げることができます。

2.フィジカルトレーニング：サッカーは体力が重要な要素です。走力やスタミナ、筋力を高めるトレーニングを行い、試合中に疲れにくくなるようにしましょう。

3.戦術の理解：サッカーには様々な戦術があります。自分のポジションに応じた役割や戦術を理解することで、チーム全体の動きを良くすることができます。

4.試合の観察：プロの試合や自分たちの試合を観察し、選手たちがどのような動きをしているか、何がうまくいっているかを学びましょう。

5.メンタルトレーニング：サッカーはメンタル面も重要です。自分の自信を高め、集中力を維持するために、メンタルトレーニングを行うことが大切です。

以上のような方法を継続的に取り入れることで、サッカーのスキルを向上させることができます。また、常に練習や試合での反省を行い、改善していくことも重要です。

Stable Vicuna

サッカーを上手になるためには、基本的な技術や戦術を身に付けることが重要です。まずは足球ボールを正しく操作するためのトレーニングが必要です。次に、速度や反発力を高めるためのエキササイズやゲームパターンを実践することが大切です。また、チームワークやセンスを向上させるために、相手との信頼関係を建設することも重要です。さらに、自分自身の状態や健康管理も素直に取り入れることが求められます

まとめ

Stable Vicuna 13BをGoogle colab freeで動かしてみました。GPU8GB, CPU4GB程度あれば動いてしまうので本当にすごい世界です。。GPT3.5-turboと推論結果を比較してみましたが、少し精度は劣るみたいですね。ただ他のLLMモデルと比較して日本語への対応度合いが良いのはさすがと言うべきでしょうか。

今後とも簡単AIツールを利用していく情報発信をしていくつもりなのでよろしくお願いします。