Zenn
Open4

Llama3のFine-tuning

#
LLM
Koichiro MoriKoichiro Mori

Unsloth

https://unsloth.ai/

  • We support 16bit LoRA or 4bit QLoRA. Both 2x faster.
  • AutoModelForCausalLM を FastLanguageModel に置き換えるだけ
  • あとはtrlのSFTTrainerがそのまま使える
from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "unsloth/llama-3-8b-bnb-4bit",
    max_seq_length = max_seq_length,
    dtype = dtype,
    load_in_4bit = load_in_4bit,
)

https://note.com/npaka/n/na3f5abf30629