iTranslated by AI

The content below is an AI-generated translation. This is an experimental feature, and may contain errors. View original article
🪸

Fine-tuning Local LLM on Strix Halo

に公開
llm
Fine-tuning
生成 AI
ローカル LLM
evox2
tech

Introduction

Donato Capitella, who has been developing useful tools for Strix Halo one after another, has released a new video. This time, he has made public a tool that enables full fine-tuning on Strix Halo, and it looked interesting, so I decided to run it on the EVO-X2.

So, in this article, I will summarize the steps from setup, training, to inference, following the README.md as a reference, as a memo for myself.

https://www.youtube.com/watch?v=nxugSRDg_jg

Environment

  • EVO-X2 (Ryzen AI MAX+395, 128GB)
🐔@EVO-X2:~$ amd-smi
+------------------------------------------------------------------------------+
| AMD-SMI 26.0.2+39589fda      amdgpu version: 6.12.12  ROCm version: 7.0.2    |
| Platform: Linux Baremetal                                                    |
|-------------------------------------+----------------------------------------|
| BDF                        GPU-Name | Mem-Uti   Temp   UEC       Power-Usage |
| GPU  HIP-ID  OAM-ID  Partition-Mode | GFX-Uti    Fan               Mem-Usage |
|=====================================+========================================|
| 0000:c6:00.0    AMD Radeon Graphics | N/A        N/A   0             N/A/0 W |
|   0       0     N/A             N/A | N/A        N/A              433/512 MB |
+-------------------------------------+----------------------------------------+
+------------------------------------------------------------------------------+
| Processes:                                                                   |
|  GPU        PID  Process Name          GTT_MEM  VRAM_MEM  MEM_USAGE     CU % |
|==============================================================================|
|  No running processes found                                                  |
+------------------------------------------------------------------------------+

Setup

$ git clone https://github.com/kyuz0/amd-strix-halo-llm-finetuning.git
Cloning into 'amd-strix-halo-llm-finetuning'...
remote: Enumerating objects: 13, done.
remote: Counting objects: 100% (13/13), done.
remote: Compressing objects: 100% (13/13), done.
remote: Total 13 (delta 0), reused 13 (delta 0), pack-reused 0 (from 0)
Receiving objects: 100% (13/13), 18.01 KiB | 3.60 MiB/s, done.
$ toolbox create strix-halo-llm-finetuning \
  --image docker.io/kyuz0/amd-strix-halo-llm-finetuning:latest \
  -- --device /dev/dri --device /dev/kfd \
  --group-add video --group-add render --security-opt seccomp=unconfined
Image required to create toolbox container.
Download docker.io/kyuz0/amd-strix-halo-llm-finetuning:latest (500MB)? [y/N]: y
Created container: strix-halo-llm-finetuning
Enter with: toolbox enter strix-halo-llm-finetuning
$ toolbox enter strix-halo-llm-finetuning # Enter the toolbox

███████╗████████╗██████╗ ██╗██╗  ██╗      ██╗  ██╗ █████╗ ██╗      ██████╗ 
██╔════╝╚══██╔══╝██╔══██╗██║╚██╗██╔╝      ██║  ██║██╔══██╗██║     ██╔═══██╗
███████╗   ██║   ██████╔╝██║ ╚███╔╝       ███████║███████║██║     ██║   ██║
╚════██║   ██║   ██╔══██╗██║ ██╔██╗       ██╔══██║██╔══██║██║     ██║   ██║
███████║   ██║   ██║  ██║██║██╔╝ ██╗      ██║  ██║██║  ██║███████╗╚██████╔╝
╚══════╝   ╚═╝   ╚═╝  ╚═╝╚═╝╚═╝  ╚═╝      ╚═╝  ╚═╝╚═╝  ╚═╝╚══════╝ ╚═════╝ 

                        L L M   F I N E - T U N I N G                        


AMD STRIX HALO — LLM Finetuning (gfx1151, ROCm via TheRock)
ROCm nightly: 7.10.0a20251015

Machine: GMKtec NucBox_EVO-X2
GPU    : AMD RYZEN AI MAX+ 395 w/ Radeon 8060S

Repo   : https://github.com/kyuz0/amd-strix-halo-llm-finetuning
Image  : docker.io/kyuz0/amd-strix-halo-llm-finetuning:latest

Quickstart:
  - 1. Copy notebooks to home directory → mkdir -p ~/finetuning-workspace; cp -r /opt/workspace/* ~/finetuning-workspace/
  - 2. Start Jupyter Lab → jupyter lab --notebook-dir ~/finetuning-workspace/

SSH tip: ssh -L 8888:localhost:8888 user@host

Launch

$ mkdir -p ~/finetuning-workspace/
$ cp -r /opt/workspace ~/finetuning-workspace/
$ jupyter lab --notebook-dir ~/finetuning-workspace/
[I 2025-10-20 23:45:36.172 ServerApp] jupyter_lsp | extension was successfully linked.
[I 2025-10-20 23:45:36.174 ServerApp] jupyter_server_terminals | extension was successfully linked.
[I 2025-10-20 23:45:36.177 ServerApp] jupyterlab | extension was successfully linked.
[I 2025-10-20 23:45:36.179 ServerApp] Writing Jupyter server cookie secret to /home/gosrum/.local/share/jupyter/runtime/jupyter_cookie_secret
[I 2025-10-20 23:45:36.307 ServerApp] notebook_shim | extension was successfully linked.
[I 2025-10-20 23:45:36.318 ServerApp] notebook_shim | extension was successfully loaded.
[I 2025-10-20 23:45:36.319 ServerApp] jupyter_lsp | extension was successfully loaded.
[I 2025-10-20 23:45:36.320 ServerApp] jupyter_server_terminals | extension was successfully loaded.
[I 2025-10-20 23:45:36.321 LabApp] JupyterLab extension loaded from /usr/lib/python3.13/site-packages/jupyterlab
[I 2025-10-20 23:45:36.321 LabApp] JupyterLab application directory is /usr/share/jupyter/lab
[I 2025-10-20 23:45:36.322 LabApp] Extension Manager is 'pypi'.
[I 2025-10-20 23:45:36.331 ServerApp] jupyterlab | extension was successfully loaded.
[I 2025-10-20 23:45:36.331 ServerApp] Serving notebooks from local directory: /home/gosrum/finetuning-workspace
[I 2025-10-20 23:45:36.331 ServerApp] Jupyter Server 2.15.0 is running at:
[I 2025-10-20 23:45:36.331 ServerApp] http://localhost:8888/lab?token=3dfd4fb4cb01518e046a74bf1f7c7da62b1edcb533f7ceb7
[I 2025-10-20 23:45:36.331 ServerApp]     http://127.0.0.1:8888/lab?token=3dfd4fb4cb01518e046a74bf1f7c7da62b1edcb533f7ceb7
[I 2025-10-20 23:45:36.331 ServerApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 2025-10-20 23:45:36.337 ServerApp] 
    
    To access the server, open this file in a browser:
        file:///home/gosrum/.local/share/jupyter/runtime/jpserver-58308-open.html
    Or copy and paste one of these URLs:
        http://localhost:8888/lab?token=3dfd4fb4cb01518e046a74bf1f7c7da62b1edcb533f7ceb7
        http://127.0.0.1:8888/lab?token=3dfd4fb4cb01518e046a74bf1f7c7da62b1edcb533f7ceb7
gio: file:///home/gosrum/.local/share/jupyter/runtime/jpserver-58308-open.html: Failed to find default application for content type 'text/plain'
[I 2025-10-20 23:45:36.439 ServerApp] Skipped non-installed server(s): bash-language-server, dockerfile-language-server-nodejs, javascript-typescript-langserver, jedi-language-server, julia-language-server, pyright, python-language-server, python-lsp-server, r-languageserver, sql-language-server, texlab, typescript-language-server, unified-language-server, vscode-css-languageserver-bin, vscode-html-languageserver-bin, vscode-json-languageserver-bin, yaml-language-server

Once you reach this point, you can launch Jupyter Notebook by accessing http://localhost:8888/lab?token=***** in your web browser.

Let's Try Full Fine-tuning Right Away

Open workspace/gemma-finetuning.ipynb.

If you run the cells in order from the top without thinking, you'll encounter an error. To prevent this, click Switch kernel in the top right and switch to Python (venv). This should resolve the issue.


Continuing to execute the cells in order, an error occurs at the following cell:

model = AutoModelForCausalLM.from_pretrained(
    MODEL, dtype="auto", device_map="auto", attn_implementation="eager",
)
tokenizer = AutoTokenizer.from_pretrained(MODEL)
torch_dtype = model.dtype
print(f"Weights footprint: {model.get_memory_footprint()/1e9:.2f} GB")

Checking the error message, it states that to access the Gemma model, you must agree to Google's usage license. Since I had already agreed, I added the Hugging Face access token (hf_your_token) as shown below, which resolved the issue.

After the model download is complete, executing the cell below it starts the training.

For reference, here is the amdgpu_top status during training.

Full fine-tuning of Gemma-3 1B-IT completed in about 3 minutes.
Peak memory was approximately 19GB.

Inference

At the very bottom of the notebook, there is a cell for performing inference with the obtained model.

Other Notes

  • As mentioned in the video, it seems that training with the unsloth framework is currently not possible.

  • This article only demonstrated the full fine-tuning method, but it also supports creating LoRA and QLoRA.

    • As shown below, it seems possible to create LoRA for gpt-oss-20B.

Performance on Strix Halo

Below is a rough guide for the VRAM and time required for post-training on Strix Halo.
With 128GB, full fine-tuning appears to be limited to 12B.

Model Full FT LoRA 8-bit + LoRA QLoRA
Gemma-3 1B-IT 19 GB / 2m52s 15 GB / 2m 13 GB / 8m 13 GB / 9m
Gemma-3 4B-IT 46 GB / 9m 30 GB / 5m 21 GB / 41m 13 GB / 9m
Gemma-3 12B-IT 115 GB / 25m 67 GB / 13m 43 GB / 2h38m 26 GB / 23m
Gemma-3 27B-IT OOM OOM 32 GB unstable 19 GB runs
GPT-OSS-20B (MXFP4) - 32-38 GB / ~1h - -
  • Source

https://github.com/kyuz0/amd-strix-halo-llm-finetuning

https://zenn.dev/robustonian/scraps/b503f3f69646a1

Summary

In this article, I summarized how to perform full fine-tuning of a local LLM on Strix Halo, based on the amd-strix-halo-llm-finetuning repository prepared by Donato Capitella and his video, and confirmed that it works on the EVO-X2.

Honestly, this is almost my first experience with post-training. I would like to deepen my understanding of the mathematical meaning of post-training while referring to this introductory video and others. I also want to try changing the training data or creating a fine-tuned model with a 12B model.

Discussion