🎨

改めてAutomatic1111

2023/01/14に公開

Stable Diffusion

tech

2022/01/14
GitHubからBANされる前後でわちゃわちゃしたようなので、改めて環境構築手順をメモ

環境

Windows 10 Pro 22H2
なんらかのPython3.10環境を準備済み
- 今回はminiconda
gitコマンドインストール済み
作業はVSCode上のPowershellで行う

まずは入れる

まずはCloneしてくる。以前の環境と区別するためにディレクトリ名は変更する。

(base) > git clone https://gitgud.io/AUTOMATIC1111/stable-diffusion-webui.git stable-diffusion-webui-gitgud

何も考えずに立ち上げてみる。まぁ怒られるよね。というか、modelがなかっただけか。

(base) > cd stable-diffusion-webui-gitgud
(base) > ./webui-user.bat

Creating venv in directory venv using python "C:\Users\xxx\miniconda3\python.exe"
venv "X:\Repositories\stable-diffusion-webui-gitgud\venv\Scripts\Python.exe"
Python 3.10.8 | packaged by conda-forge | (main, Nov 24 2022, 14:07:00) [MSC v.1916 64 bit (AMD64)]
Commit hash: cca6e5212ddd77be7efdce2c27e475d3af436d85
Installing torch and torchvision
Installing gfpgan
Installing clip
Installing open_clip
Cloning Taming Transformers into repositories\taming-transformers...
Cloning K-diffusion into repositories\k-diffusion...
Cloning CodeFormer into repositories\CodeFormer...
Cloning BLIP into repositories\BLIP...
Installing requirements for CodeFormer
Installing requirements for Web UI
Launching Web UI with arguments:
No module 'xformers'. Proceeding without it.
No checkpoints found. When searching for checkpoints, looked at:
 - file X:\Repositories\stable-diffusion-webui-gitgud\model.ckpt
 - directory X:\Repositories\stable-diffusion-webui-gitgud\models\Stable-diffusion
Can't run without a checkpoint. Find and place a .ckpt file into any of those locations. The program will exit.
Press any key to continue . . .

ここで、venvができているので、入っておく。

(base) > .\venv\Scripts\Activate.ps1
(venv) >

適当なmodelを用意して再度起動。

(venv) > cp sd-v1-5-pruned-emaonly.ckpt .\model\Stable-diffusion
(venv) > .\webui-user.bat

venv "X:\Repositories\stable-diffusion-webui-gitgud\venv\Scripts\Python.exe"
Python 3.10.8 | packaged by conda-forge | (main, Nov 24 2022, 14:07:00) [MSC v.1916 64 bit (AMD64)]
Commit hash: cca6e5212ddd77be7efdce2c27e475d3af436d85
Installing requirements for Web UI
Launching Web UI with arguments:
No module 'xformers'. Proceeding without it.
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
Loading weights [81761151] from X:\Repositories\stable-diffusion-webui-gitgud\models\Stable-diffusion\sd-v1-5-pruned-emaonly.ckpt
Applying cross attention optimization (Doggettx).
Textual inversion embeddings loaded(0):
Model loaded.
Running on local URL:  http://127.0.0.1:7860

一応動きました。

dreamboothを入れる

webui上から、sd_dreambooth_extentionを入れる。案の定エラー。

#######################################################################################################
Initializing Dreambooth
If submitting an issue on github, please provide the below text for debugging purposes:

Python revision: 3.10.8 | packaged by conda-forge | (main, Nov 24 2022, 14:07:00) [MSC v.1916 64 bit (AMD64)]
Dreambooth revision: 17c3864803ebb50615205271de687be96cfc96e8
SD-WebUI revision: cca6e5212ddd77be7efdce2c27e475d3af436d85

[!] bitsandbytes NOT installed.
[!] diffusers NOT installed.
[!] transformers version 4.19.2 installed.
[ ] xformers version N/A installed.
[+] torch version 1.12.1+cu113 installed.
[+] torchvision version 0.13.1+cu113 installed.

#######################################################################################################

Closing server running on port: 7860
Restarting UI...
Exception importing SD-WebUI module.
Error loading script: api.py
Traceback (most recent call last):
  File "X:\Repositories\stable-diffusion-webui-gitgud\modules\scripts.py", line 205, in load_scripts
    module = script_loading.load_module(scriptfile.path)
    exec(compiled, module.__dict__)
  File "X:\Repositories\stable-diffusion-webui-gitgud\extensions\sd_dreambooth_extension\scripts\api.py", line 21, in <module>
    from extensions.sd_dreambooth_extension.dreambooth.db_config import from_file, DreamboothConfig
  File "X:\Repositories\stable-diffusion-webui-gitgud\extensions\sd_dreambooth_extension\dreambooth\db_config.py", line 6, in <module>
    from extensions.sd_dreambooth_extension.dreambooth.db_concept import Concept
  File "X:\Repositories\stable-diffusion-webui-gitgud\extensions\sd_dreambooth_extension\dreambooth\db_concept.py", line 5, in <module>
    from extensions.sd_dreambooth_extension.dreambooth.utils import get_images
  File "X:\Repositories\stable-diffusion-webui-gitgud\extensions\sd_dreambooth_extension\dreambooth\utils.py", line 18, in <module>
    import tensorflow
ModuleNotFoundError: No module named 'tensorflow'

Error loading script: dreambooth.py
Traceback (most recent call last):
  File "X:\Repositories\stable-diffusion-webui-gitgud\modules\scripts.py", line 205, in load_scripts
    module = script_loading.load_module(scriptfile.path)
  File "X:\Repositories\stable-diffusion-webui-gitgud\modules\script_loading.py", line 13, in load_module
    exec(compiled, module.__dict__)
  File "X:\Repositories\stable-diffusion-webui-gitgud\extensions\sd_dreambooth_extension\scripts\dreambooth.py", line 11, in <module>
    from diffusers.utils import logging as dl
ModuleNotFoundError: No module named 'diffusers'

Error loading script: main.py
Traceback (most recent call last):
  File "X:\Repositories\stable-diffusion-webui-gitgud\modules\scripts.py", line 205, in load_scripts
    module = script_loading.load_module(scriptfile.path)
  File "X:\Repositories\stable-diffusion-webui-gitgud\modules\script_loading.py", line 13, in load_module
    exec(compiled, module.__dict__)
  File "X:\Repositories\stable-diffusion-webui-gitgud\extensions\sd_dreambooth_extension\scripts\main.py", line 6, in <module>
    from extensions.sd_dreambooth_extension.dreambooth.db_config import save_config, from_file
  File "X:\Repositories\stable-diffusion-webui-gitgud\extensions\sd_dreambooth_extension\dreambooth\db_config.py", line 6, in <module>
    from extensions.sd_dreambooth_extension.dreambooth.db_concept import Concept
  File "X:\Repositories\stable-diffusion-webui-gitgud\extensions\sd_dreambooth_extension\dreambooth\db_concept.py", line 5, in <module>
    from extensions.sd_dreambooth_extension.dreambooth.utils import get_images
  File "X:\Repositories\stable-diffusion-webui-gitgud\extensions\sd_dreambooth_extension\dreambooth\utils.py", line 18, in <module>
    import tensorflow
ModuleNotFoundError: No module named 'tensorflow'

Running on local URL:  http://127.0.0.1:7860

webui-user-dreambooth.batで起動し直す。

webui-user-dreambooth.bat

@echo off

set PYTHON=
set GIT=
set VENV_DIR=
set COMMANDLINE_ARGS=
:: Use the below argument if getting OOM extracting checkpoints
:: set COMMANDLINE_ARGS=--ckptfix
set "REQS_FILE=.\extensions\sd_dreambooth_extension\requirements.txt"
:: Uncomment below to skip trying to install automatically on launch.
:: set "DREAMBOOTH_SKIP_INSTALL=True"
:: Use this to launch with accelerate (Run 'accelerate config' first, launch once without to install dependencies)
:: set ACCELERATE="True"

call webui.bat

(venv) > .\extensions\sd_dreambooth_extension\webui-user-dreambooth.bat

venv "X:\Repositories\stable-diffusion-webui-gitgud\venv\Scripts\Python.exe"
Python 3.10.8 | packaged by conda-forge | (main, Nov 24 2022, 14:07:00) [MSC v.1916 64 bit (AMD64)]
Commit hash: cca6e5212ddd77be7efdce2c27e475d3af436d85
Installing xformers
Installing requirements for Web UI
Traceback (most recent call last):
  File "X:\Repositories\stable-diffusion-webui-gitgud\launch.py", line 306, in <module>
    prepare_environment()
  File "X:\Repositories\stable-diffusion-webui-gitgud\launch.py", line 258, in prepare_environment
    run_pip(f"install -r {requirements_file}", "requirements for Web UI")
  File "X:\Repositories\stable-diffusion-webui-gitgud\launch.py", line 93, in run_pip
    return run(f'"{python}" -m pip {args} --prefer-binary{index_url_line}', desc=f"Installing {desc}", errdesc=f"Couldn't install {desc}")
  File "X:\Repositories\stable-diffusion-webui-gitgud\launch.py", line 64, in run
    raise RuntimeError(message)
RuntimeError: Couldn't install requirements for Web UI.
Command: "X:\Repositories\stable-diffusion-webui-gitgud\venv\Scripts\python.exe" -m pip install -r .\extensions\sd_dreambooth_extension\requirements.txt --prefer-binary
Error code: 1

...
中略
...

Installing collected packages: libclang, flatbuffers, bitsandbytes, zipp, wrapt, threadpoolctl, termcolor, tensorflow-io-gcs-filesystem, tensorflow-estimator, requests, protobuf, opt-einsum, opencv-python-headless, keras, joblib, h5py, google-pasta, gast, astunparse, starlette, scikit-learn, importlib-metadata, discord-webhook, accelerate, transformers, qudida, modelcards, fastapi, diffusers, clean-fid, tensorboard, albumentations, tensorflow-intel, tensorflow

stderr: ERROR: Could not install packages due to an OSError: [Errno 2] No such file or directory: 'X:\\Repositories\\stable-diffusion-webui-gitgud\\venv\\Lib\\site-packages\\bitsandbytes\\__init__.py'

Press any key to continue . . .

またbitsandbytesか...昔もなんか色々やった気がする。

bitsandbytesを入れる

参考はこの辺。

oh...ってか、上のエラーと全く一緒だな。

(venv) > pip install bitsandbytes==0.35.0 

Collecting bitsandbytes==0.35.0
  Using cached bitsandbytes-0.35.0-py3-none-any.whl (62.5 MB)
Installing collected packages: bitsandbytes
ERROR: Could not install packages due to an OSError: [Errno 2] No such file or directory: 'X:\\Repositories\\stable-diffusion-webui-gitgud\\venv\\Lib\\site-packages\\bitsandbytes\\__init__.py'

調べたら、site-package以下にbitsandbytesという名前のファイルがあったせいで、インストールに失敗していた。なぜこうなったか不明。dreambooth_extentionのインストーラが悪さしたか？

取り急ぎ、当該ファイルを削除の上個別でbitsandbytesをインストール。普通に通った。

(venv) > del ./venv/Lib/site-packages/bitsandbytes
(venv) > pip install bitsandbytes==0.35.0

Collecting bitsandbytes==0.35.0
  Using cached bitsandbytes-0.35.0-py3-none-any.whl (62.5 MB)
Installing collected packages: bitsandbytes
Successfully installed bitsandbytes-0.35.0

再度webui-user-dreambooth.batで起動。通った。
bitsandbytesのDLL関連はsd_dreambooth_extentionが面倒見てくれてるはず。

(venv) > .\extentions\sd_dreambooth_extention\webui-user-dreambooth.bat

venv "X:\Repositories\stable-diffusion-webui-gitgud\venv\Scripts\Python.exe"
Python 3.10.8 | packaged by conda-forge | (main, Nov 24 2022, 14:07:00) [MSC v.1916 64 bit (AMD64)]
Commit hash: cca6e5212ddd77be7efdce2c27e475d3af436d85
Installing requirements for Web UI

#######################################################################################################
Initializing Dreambooth
If submitting an issue on github, please provide the below text for debugging purposes:

Python revision: 3.10.8 | packaged by conda-forge | (main, Nov 24 2022, 14:07:00) [MSC v.1916 64 bit (AMD64)]
Dreambooth revision: 17c3864803ebb50615205271de687be96cfc96e8
SD-WebUI revision: cca6e5212ddd77be7efdce2c27e475d3af436d85

[+] bitsandbytes version 0.35.0 installed.
[+] diffusers version 0.10.2 installed.
[+] transformers version 4.25.1 installed.
[+] xformers version 0.0.14.dev0 installed.
[+] torch version 1.12.1+cu113 installed.
[+] torchvision version 0.13.1+cu113 installed.

#######################################################################################################

Launching Web UI with arguments:
No module 'xformers'. Proceeding without it.
SD-Webui API layer loaded
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
Loading weights [81761151] from X:\Repositories\stable-diffusion-webui-gitgud\models\Stable-diffusion\sd-v1-5-pruned-emaonly.ckpt
Applying cross attention optimization (Doggettx).
Textual inversion embeddings loaded(0): 
Model loaded.
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.

仕上げ

torchのバージョンがcu113になっているが、自分の用途上cu116が欲しいので変更。

(venv) > pip install torch==1.12.1+cu116 torchvision==0.13.1+cu116 --extra-

index-url https://download.pytorch.org/whl/cu116

...
中略
...

Looking in indexes: https://pypi.org/simple, 
Installing collected packages: torch, torchvision
  Attempting uninstall: torch
    Found existing installation: torch 1.12.1+cu113
    Uninstalling torch-1.12.1+cu113:
      Successfully uninstalled torch-1.12.1+cu113
  Attempting uninstall: torchvision
    Uninstalling torchvision-0.13.1+cu113:
      Successfully uninstalled torchvision-0.13.1+cu113
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
xformers 0.0.14.dev0 requires pyre-extensions==0.0.23, which is not installed.
open-clip-torch 2.7.0 requires protobuf==3.20.0, but you have protobuf 3.19.6 which is incompatible.
Successfully installed torch-1.12.1+cu116 torchvision-0.13.1+cu116

cu116のインストールは成功したが、いくつか依存性問題があるようなので、解消したrequirementsを作っておく。

(venv) > cp .\extentions\sd_dreambooth_extention\requirements.txt .\requirements_mod.txt

requirements_mod.txt

前略
...

+ pyre-extensions
+ protobuf
+ open-clip-torch
+ onnx

(venv) > pip install -r ./requirements_mod.txt

...
中略
...

Successfully installed onnx-1.12.0 open-clip-torch-2.6.1 protobuf-3.19.6 mypy-extensions-0.4.3 pyre-extensions-0.0.23 typing-inspect-0.8.0

上記に合わせて起動スクリプトを書き換え。

(venv) > cp .\extentions\sd_dreambooth_extention\webui-user-dreambooth.bat .\webui-user-dreambooth-mod.bat

webui-user-dreambooth-mod.bat

@echo off

set PYTHON=
set GIT=
set VENV_DIR=
set "COMMANDLINE_ARGS=--xformers --deepdanbooru --theme dark --api --listen --port 7860"
:: Use the below argument if getting OOM extracting checkpoints
:: set COMMANDLINE_ARGS=--ckptfix
set "TORCH_COMMAND=pip install torch==1.12.1+cu116 torchvision==0.13.1+cu116 --extra-index-url https://download.pytorch.org/whl/cu116"
set "REQS_FILE=.\requirements_mod.txt"
:: Uncomment below to skip trying to install automatically on launch.
:: set "DREAMBOOTH_SKIP_INSTALL=True"
:: Use this to launch with accelerate (Run 'accelerate config' first, launch once without to install dependencies)
:: set ACCELERATE="True"

call webui.bat

(venv) > .\webui-user-dreambooth-mod.bat

venv "X:\Repositories\stable-diffusion-webui-gitgud\venv\Scripts\Python.exe"
Python 3.10.8 | packaged by conda-forge | (main, Nov 24 2022, 14:07:00) [MSC v.1916 64 bit (AMD64)]
Commit hash: cca6e5212ddd77be7efdce2c27e475d3af436d85
Installing requirements for Web UI

#######################################################################################################
Initializing Dreambooth
If submitting an issue on github, please provide the below text for debugging purposes:

Python revision: 3.10.8 | packaged by conda-forge | (main, Nov 24 2022, 14:07:00) [MSC v.1916 64 bit (AMD64)]
Dreambooth revision: 17c3864803ebb50615205271de687be96cfc96e8
SD-WebUI revision: cca6e5212ddd77be7efdce2c27e475d3af436d85

[+] bitsandbytes version 0.35.0 installed.
[+] diffusers version 0.10.2 installed.
[+] transformers version 4.25.1 installed.
[+] xformers version 0.0.14.dev0 installed.
[+] torch version 1.12.1+cu116 installed.
[+] torchvision version 0.13.1+cu116 installed.

#######################################################################################################

Launching Web UI with arguments: --xformers --deepdanbooru --theme dark --api --listen --port 7860
SD-Webui API layer loaded
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
Loading weights [81761151] from X:\Repositories\stable-diffusion-webui-gitgud\models\Stable-diffusion\sd-v1-5-pruned-emaonly.ckpt
Applying xformers cross attention optimization.
Textual inversion embeddings loaded(0): 
Model loaded.
Running on local URL:  http://0.0.0.0:7860

To create a public link, set `share=True` in `launch()`.

一旦終了！

環境

まずは入れる

dreamboothを入れる

bitsandbytesを入れる

仕上げ

Discussion