OpenAssistant Llama 13Bをgoogle colabで試してみた。



OpenAssistantの目指しているVisionは誰でも容易に拡張することが可能かつPersonalにカスタマイズされたAI Assistantを構築したいというもののようです。

  1. 高品質な指示-達成サンプルを収集するためのクラウドソーシングプロセスを設計する。
  2. 収集されたデータをランキング化して報酬モデルをトレーニングする。
  3. トップ貢献者にSwagを贈呈し、コミュニティをモチベーションづける。

今回はGoogle Colabのfreeで動かせそうなLlama製の13Bモデルの4bitversionをtext-generation-webuiで利用してみたいと思います。




Google Colabを開き、メニューから「ランタイム→ランタイムのタイプを変更」でランタイムを「GPU」に変更します。




!git clone https://github.com/oobabooga/text-generation-webui
%cd text-generation-webui
!pip install -r requirements.txt
!git clone https://github.com/oobabooga/GPTQ-for-LLaMa.git -b cuda
%cd GPTQ-for-LLaMa
!python setup_cuda.py install
!cp -r /content/text-generation-webui/GPTQ-for-LLaMa/* /content/text-generation-webui/


(1) モデルのダウンロード

%cd /content/text-generation-webui
!python download-model.py 4bit/oasst-llama13b-4bit-128g

(2) WebUIの起動

%cd /content/text-generation-webui
!python3 server.py --model 4bit_oasst-llama13b-4bit-128g --wbits 4 --group 128 --chat --share --auto-devices  --model_type=llama

起動するとコンソール上にRunning on public URL: https://*****.gradio.live と出るのでそこをクリックするとText generation WebUIのページに遷移します。


(1)1LDK room planner

I want you act as a professional room creator. Please suggest some plan for comfortable room with 1LDK.

OpenAssistant LLama 13B

Sure! Here are some suggestions for creating a comfortable living space in a 1-Bedroom, 1-Living Room, 1-Dining Area (1LDK) apartment layout:

For the bedroom:

1. Place the bed against one wall, leaving enough walking space around it on all sides.
2. Install curtains or blinds that can be closed at night to block out light and create a more restful environment.
3. Consider adding decorative elements such as lamps, plants, or artwork to make the space feel cozier.
For the living area:

4. Position furniture so there's plenty of room to move around and socialize comfortably.
Include seating options like a couch, armchair, and/or stools to accommodate different types of activities (e.g., watching TV, reading, having conversations).
5. Add decor items such as potted plants, throw pillows, and other accessories to enhance the overall look and feel of the space.
For the dining area:

6. Choose chairs that are comfortable and supportive for long meals and gatherings.
7. Select tableware and serving pieces that fit your style and functional needs (e.g., plates, bowls, glasses, utensils).
Consider incorporating design elements such as table runners, centerpieces, and lighting to create a welcoming atmosphere.


Certainly, I'd be happy to help! Here are some suggestions for a comfortable 1LDK room:

1. First, think about the layout. For a 1LDK room, you'll want to maximize the use of the space. Consider an open-plan living room and kitchen, with a separate bedroom and bathroom.
2. Choose comfortable furniture that is appropriately sized for the space. A small sofa or sectional, a coffee table, and a TV stand can work well in the living room, while a queen or full-sized bed, a dresser, and a nightstand can fit in the bedroom. You may also want to consider storage solutions, such as a bookshelf or storage ottoman.
3. Use lighting to create a cozy atmosphere. Consider a mix of overhead lighting, table lamps, and floor lamps to provide a variety of lighting options for different moods and activities.
4. Choose a color scheme that is calming and cohesive. Soft, neutral colors like beige, gray, and white can create a serene environment, while pops of color in pillows, rugs, or artwork can add interest and personality.
5. Don't forget about the bathroom! Choose soft, fluffy towels, a shower curtain or screen, and a bath mat to make the space feel luxurious and comfortable.
I hope these suggestions are helpful in creating a comfortable 1LDK room.


(2) Instruct
(2-1) Few shot learningのケース

Please do the classification task like the following examples.

Question: Can you tell me about the detail of this image? Answer: object detection+semantic segmentation+image caption

Question: Please tell me about the number of objects in this image. Answer: object detection

Question: can you extract the transcript from this audio file? Answer: speech to text + translation

Question: Please translate this sentence into English. Answer: translation

Question: Can you tell me about this video in detail? Answer:


Yes, here is some information on that image:

The image contains three objects: a car, a person and a building. The car is red and has four doors. The person is wearing a blue shirt and khaki pants. The building is white with two windows and a door. It also appears to be located near a tree and a street sign.


(2-2) zeroshot learningのケース

Please create task plan with AI model according to the following question. Question: Can you describe this image? Answer:


Sure! Here’s an example task plan for describing the image using an AI model:

Use object detection to identify all the objects present in the image (car, person, building).
For each detected object, use semantic segmentation to classify it as either background or foreground. This will help us determine which parts of the image are important for description.
Based on the importance of different regions, generate a hierarchical representation of the scene using graph neural networks. This will allow us to organize the information in a structured way.
Generate a descriptive caption by combining the region-specific features learned during step 3 with pretrained language models. We can fine-tune these models to improve their performance based on human feedback.



Please extract the keyword from the above sentence


Extracting keywords from the above sentence could look something like this:

Object Detection: car, person, building

Semantic Segmentation: car - foreground, person - foreground, building - foreground

Graph Neural Networks: hierarchy, structure, organization

Pretrained Language Models: captions, descriptions



今回はOpenAssistantのデータセットをLLama13Bでfinetuningされたモデルをtext-generation-webuiで利用してみました。SuggestionとかDraftを書くとかに関してはchatGPTと同等かそれ以上の性能があると思われます。ただQuestion AnsweringやText classificationといったタスクに関してはfinetuningされていないようでうまくできないようでした。
またLLMに関して色々なモデルを試してきたのでそのモデルの実行google colabと感想まとめみたいなのも近々更新しますね。

(注)30Bの4bitもありましたがgoogle colab freeでは動かなそうでした。おうちに24GBくらいのGPUがある方は以下をお試しください。

今後ともLLM, Diffusion model, Image Analysis, 3Dに関連する試した記事を投稿していく予定なのでよろしくお願いします。