Zenn
📒

Proxmox VEのバージョンアップをしてみるメモ 7->8

2023/12/28に公開
memo
upgrade
Proxmox
tech

Proxmox VE のバージョンアップ

自宅で長めに運用しているProxmox VE がVersion7で最新版の8にしたくなったので更新するメモです。
他にもたくさんやっている人は居るし、公式ドキュメントも充実しているのでコケることは無いはず……

前提条件

現状の確認をします。

         .://:`              `://:.            
       `hMMMMMMd/          /dMMMMMMh`          ----------------
        `sMMMMMMMd:      :mMMMMMMMs`           OS: Proxmox VE 7.4-17 x86_64
`-/+oo+/:`.yMMMMMMMh-  -hMMMMMMMy.`:/+oo+/-`   Host: PC1PF57K ThinkCentre M75q-1
`:oooooooo/`-hMMMMMMMyyMMMMMMMh-`/oooooooo:`   Kernel: 5.15.131-2-pve
  `/oooooooo:`:mMMMMMMMMMMMMm:`:oooooooo/`     Uptime: 12 days, 18 hours, 42 mins
    ./ooooooo+- +NMMMMMMMMN+ -+ooooooo/.       Packages: 700 (dpkg)
      .+ooooooo+-`oNMMMMNo`-+ooooooo+.         Shell: bash 5.1.4
        -+ooooooo/.`sMMs`./ooooooo+-           Terminal: /dev/pts/0
          :oooooooo/`..`/oooooooo:             CPU: AMD Ryzen 5 PRO 3400GE w/ Radeon Vega Graphics (8) @ 3.300GHz
          :oooooooo/`..`/oooooooo:             GPU: AMD ATI 03:00.0 Picasso
        -+ooooooo/.`sMMs`./ooooooo+-           Memory: 1638MiB / 38020MiB
      .+ooooooo+-`oNMMMMNo`-+ooooooo+.
    ./ooooooo+- +NMMMMMMMMN+ -+ooooooo/.
  `/oooooooo:`:mMMMMMMMMMMMMm:`:oooooooo/`
`:oooooooo/`-hMMMMMMMyyMMMMMMMh-`/oooooooo:`
`-/+oo+/:`.yMMMMMMMh-  -hMMMMMMMy.`:/+oo+/-`
        `sMMMMMMMm:      :dMMMMMMMs`
       `hMMMMMMd/          /dMMMMMMh`
         `://:`              `://:`

Upgraded to the latest version of Proxmox VE 7.4 on all nodes.
Ensure your node(s) have correct package repository configuration (web UI, Node -> Repositories) if your pve-manager version isn't at least 7.4-15.
Hyper-converged Ceph: upgrade any Ceph Octopus or Ceph Pacific cluster to Ceph 17.2 Quincy before you start the Proxmox VE upgrade to 8.0.
Follow the guide Ceph Octopus to Pacific and Ceph Pacific to Quincy, respectively.
Co-installed Proxmox Backup Server: see the Proxmox Backup Server 2 to 3 upgrade how-to
Reliable access to the node. It's recommended to have access over a host independent channel like iKVM/IPMI or physical access.
If only SSH is available we recommend testing the upgrade on an identical, but non-production machine first.
A healthy cluster
Valid and tested backup of all VMs and CTs (in case something goes wrong)
At least 5 GB free disk space on the root mount point.
Check known upgrade issues

  1. 最新版にする
    # apt update && apt upgrade

  2. Ceph は未使用

  3. Proxmox Backup Server は入れていません

  4. SSH接続で作業しています

  5. 健全なクラスタ->○

  6. 壊れても良いVMしかありませんのでこのまま行きます

  7. 空きスペースの確認

        # df -h /
     Filesystem            Size  Used Avail Use% Mounted on
     /dev/mapper/pve-root   39G  7.5G   30G  21% /

  8. リンク先を確認しました
    特に問題はなさそう？

アップグレードテスト

GUIコンソールだと接続が途切れてしまうらしいのでSSHか直接接続のコンソール接続をするらしい。
今回はSSHでrootログインしてやってみます。
pve7to8というコマンドがあり問題点を診断してくれるらしいですね。早速試してみます。

# pve7to8 --full

pve7to8 --full
= CHECKING VERSION INFORMATION FOR PVE PACKAGES =

Checking for package updates..
PASS: all packages up-to-date

Checking proxmox-ve package version..
PASS: proxmox-ve package has version >= 7.4-1

Checking running kernel version..
PASS: running kernel '5.15.131-2-pve' is considered suitable for upgrade.

= CHECKING CLUSTER HEALTH/SETTINGS =

PASS: systemd unit 'pve-cluster.service' is in state 'active'
PASS: systemd unit 'corosync.service' is in state 'active'
PASS: Cluster Filesystem is quorate.

Analzying quorum settings and state..
INFO: configured votes - nodes: 2
INFO: configured votes - qdevice: 0
INFO: current expected votes: 2
INFO: current total votes: 2
WARN: cluster consists of less than three quorum-providing nodes!

Checking nodelist entries..
PASS: nodelist settings OK

Checking totem settings..
PASS: totem settings OK

INFO: run 'pvecm status' to get detailed cluster status..

= CHECKING HYPER-CONVERGED CEPH STATUS =

SKIP: no hyper-converged ceph setup detected!

= CHECKING CONFIGURED STORAGES =

SKIP: storage 'KIOXIA960GB' disabled.
PASS: storage 'TOSHIBA640GB' enabled and active.
PASS: storage 'local' enabled and active.
PASS: storage 'local-lvm' enabled and active.
PASS: storage 'synology-nfs-share' enabled and active.
INFO: Checking storage content type configuration..
PASS: no storage content problems found
PASS: no storage re-uses a directory for multiple content types.

= MISCELLANEOUS CHECKS =

INFO: Checking common daemon services..
PASS: systemd unit 'pveproxy.service' is in state 'active'
PASS: systemd unit 'pvedaemon.service' is in state 'active'
PASS: systemd unit 'pvescheduler.service' is in state 'active'
PASS: systemd unit 'pvestatd.service' is in state 'active'
INFO: Checking for supported & active NTP service..
PASS: Detected active time synchronisation unit 'chrony.service'
INFO: Checking for running guests..
PASS: no running guest detected.
INFO: Checking if the local node's hostname 'proxmoxve01' is resolvable..
INFO: Checking if resolved IP is configured on local node..
PASS: Resolved node IP '192.168.100.3' configured and active on single interface.
INFO: Check node certificate's RSA key size
PASS: Certificate 'pve-root-ca.pem' passed Debian Busters (and newer) security level for TLS connections (4096 >= 2048)
PASS: Certificate 'pve-ssl.pem' passed Debian Busters (and newer) security level for TLS connections (2048 >= 2048)
INFO: Checking backup retention settings..
PASS: no backup retention problems found.
INFO: checking CIFS credential location..
PASS: no CIFS credentials at outdated location found.
INFO: Checking permission system changes..
INFO: Checking custom role IDs for clashes with new 'PVE' namespace..
PASS: no custom roles defined, so no clash with 'PVE' role ID namespace enforced in Proxmox VE 8
INFO: Checking if LXCFS is running with FUSE3 library, if already upgraded..
SKIP: not yet upgraded, no need to check the FUSE library version LXCFS uses
INFO: Checking node and guest description/note length..
PASS: All node config descriptions fit in the new limit of 64 KiB
PASS: All guest config descriptions fit in the new limit of 8 KiB
INFO: Checking container configs for deprecated lxc.cgroup entries
PASS: No legacy 'lxc.cgroup' keys found.
INFO: Checking if the suite for the Debian security repository is correct..
PASS: found no suite mismatch
INFO: Checking for existence of NVIDIA vGPU Manager..
PASS: No NVIDIA vGPU Service found.
INFO: Checking bootloader configuration...
SKIP: not yet upgraded, no need to check the presence of systemd-boot
INFO: Check for dkms modules...
SKIP: could not get dkms status
SKIP: No containers on node detected.

= SUMMARY =

TOTAL:    38
PASSED:   31
SKIPPED:  6
WARNINGS: 1
FAILURES: 0

ATTENTION: Please check the output for detailed information!

警告が出ましたが、クラスタが3未満なのは問題ないので無視して次のステップに行きましょう。

アップグレード前の設定

アップグレード前にリポジトリを更新する必要があるようです。
一度現状確認してから置き換えます。

# apt update
Hit:1 http://security.debian.org bullseye-security InRelease
Hit:2 http://ftp.jp.debian.org/debian bullseye InRelease
Hit:3 http://ftp.jp.debian.org/debian bullseye-updates InRelease
Hit:4 http://download.proxmox.com/debian/pve bullseye InRelease
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
All packages are up to date.
# apt dist-upgrade
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Calculating upgrade... Done
The following packages were automatically installed and are no longer required:
  pve-kernel-5.15.108-1-pve pve-kernel-5.15.85-1-pve
Use 'apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
# pveversion
pve-manager/7.4-17/513c62be (running kernel: 5.15.131-2-pve)

ドキュメントによると、sed -i 's/bullseye/bookworm/g' /etc/apt/sources.listコマンドでリストの更新をするとあります。
/etc/apt/sources.list.d/以下に追加している場合こちらも更新が必要そうですね。

# grep -R bullseye /etc/apt/
grep: /etc/apt/trusted.gpg.d/debian-archive-bullseye-security-automatic.gpg: binary file matches
grep: /etc/apt/trusted.gpg.d/debian-archive-bullseye-automatic.gpg: binary file matches
grep: /etc/apt/trusted.gpg.d/debian-archive-bullseye-stable.gpg: binary file matches
/etc/apt/sources.list:deb http://ftp.jp.debian.org/debian bullseye main contrib
/etc/apt/sources.list:deb http://ftp.jp.debian.org/debian bullseye-updates main contrib
/etc/apt/sources.list:deb http://security.debian.org bullseye-security main contrib
/etc/apt/sources.list.d/pve-no-subscription.list:deb http://download.proxmox.com/debian/pve bullseye pve-no-subscription
/etc/apt/sources.list.d/pve-enterprise.list:#deb https://enterprise.proxmox.com/debian/pve bullseye pve-enterprise

# sed -i 's/bullseye/bookworm/g' /etc/apt/sources.list
# sed -i 's/bullseye/bookworm/g' /etc/apt/sources.list.d/pve-enterprise.list
# sed -i 's/bullseye/bookworm/g' /etc/apt/sources.list.d/pve-no-subscription.list

# grep -R bullseye /etc/apt/
grep: /etc/apt/trusted.gpg.d/debian-archive-bullseye-security-automatic.gpg: binary file matches
grep: /etc/apt/trusted.gpg.d/debian-archive-bullseye-automatic.gpg: binary file matches
grep: /etc/apt/trusted.gpg.d/debian-archive-bullseye-stable.gpg: binary file matches

一通り置換が終わったのでupdateできるか試していきます。
GPGキーがbullseyeを見ているのは問題ないのかしら、まああとで確認します。

アップデート

bookwormのリポジトリからアップデートできるか試します。

~# apt update
Get:1 http://security.debian.org bookworm-security InRelease [48.0 kB]
Get:2 http://security.debian.org bookworm-security/main amd64 Packages [131 kB]
Get:3 http://security.debian.org bookworm-security/main Translation-en [77.9 kB]
Get:4 http://security.debian.org bookworm-security/contrib amd64 Packages [644 B]
Get:5 http://security.debian.org bookworm-security/contrib Translation-en [372 B]
Get:6 http://ftp.jp.debian.org/debian bookworm InRelease [151 kB]
Get:7 http://ftp.jp.debian.org/debian bookworm-updates InRelease [52.1 kB]
Get:8 http://ftp.jp.debian.org/debian bookworm/main amd64 Packages [8,787 kB]
Get:9 http://download.proxmox.com/debian/pve bookworm InRelease [2,768 B]
Get:10 http://download.proxmox.com/debian/pve bookworm/pve-no-subscription amd64 Packages [225 kB]
Get:11 http://ftp.jp.debian.org/debian bookworm/main Translation-en [6,109 kB]
Get:12 http://ftp.jp.debian.org/debian bookworm/contrib amd64 Packages [54.1 kB]
Get:13 http://ftp.jp.debian.org/debian bookworm/contrib Translation-en [48.7 kB]
Get:14 http://ftp.jp.debian.org/debian bookworm-updates/main amd64 Packages [11.5 kB]
Get:15 http://ftp.jp.debian.org/debian bookworm-updates/main Translation-en [13.1 kB]
Fetched 15.7 MB in 3s (4,923 kB/s)
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
615 packages can be upgraded. Run 'apt list --upgradable' to see them.

できてそうですね！
アップグレードします。

アップグレード

おなじみのapt dist-upgradeでアップグレードします。
途中でadduserのドキュメントが表示されましたが、置き換え系のメモ。

Configuration file '/etc/issue'
 ==> Modified (by you or by a script) since installation.
 ==> Package distributor has shipped an updated version.
   What would you like to do about it ?  Your options are:
    Y or I  : install the package maintainer's version
    N or O  : keep your currently-installed version
      D     : show the differences between the versions
      Z     : start a shell to examine the situation
 The default action is to keep your current version.
*** issue (Y/I/N/O/D/Z) [default=N] ? N

公式ドキュメントにもあるようにデフォルトのNで

Configuration file '/etc/rsyslog.conf'
 ==> Modified (by you or by a script) since installation.
 ==> Package distributor has shipped an updated version.
   What would you like to do about it ?  Your options are:
    Y or I  : install the package maintainer's version
    N or O  : keep your currently-installed version
      D     : show the differences between the versions
      Z     : start a shell to examine the situation
 The default action is to keep your current version.
*** rsyslog.conf (Y/I/N/O/D/Z) [default=N] ? D -> Y

Rsyslogを入れていましたね、Diffで確認して自分で追加した部分のみ控えておいて置き換え後更新することにします。
-*.* @192.168.100.250:514 Syslog Serverへの転送設定のみ後で追記。

Configuration file '/etc/lvm/lvm.conf'
 ==> Modified (by you or by a script) since installation.
 ==> Package distributor has shipped an updated version.
   What would you like to do about it ?  Your options are:
    Y or I  : install the package maintainer's version
    N or O  : keep your currently-installed version
      D     : show the differences between the versions
      Z     : start a shell to examine the situation
 The default action is to keep your current version.
*** lvm.conf (Y/I/N/O/D/Z) [default=N] ? D -> Y

公式ドキュメントにあるとおり、LVM回りは自分では調整していなかったのでYで置き換えます。

Configuration file '/etc/apt/sources.list.d/pve-enterprise.list'
 ==> Modified (by you or by a script) since installation.
 ==> Package distributor has shipped an updated version.
   What would you like to do about it ?  Your options are:
    Y or I  : install the package maintainer's version
    N or O  : keep your currently-installed version
      D     : show the differences between the versions
      Z     : start a shell to examine the situation
 The default action is to keep your current version.
*** pve-enterprise.list (Y/I/N/O/D/Z) [default=N] ? D
--- /etc/apt/sources.list.d/pve-enterprise.list 2023-12-28 10:31:05.364437449 +0900
+++ /etc/apt/sources.list.d/pve-enterprise.list.dpkg-new        2023-11-23 19:20:00.000000000 +0900
@@ -1 +1 @@
-#deb https://enterprise.proxmox.com/debian/pve bookworm pve-enterprise
+deb https://enterprise.proxmox.com/debian/pve bookworm pve-enterprise

ソースリストの更新、どうせ無効にしますがYで更新。

再起動前にpve7to8を実行したところ次のような警告が出ました。

WARN: System booted in uefi mode but grub-efi-amd64 meta-package not installed, new grub versions will not be installed to /boot/efi! Install grub-efi-amd64.

grub-efi-amd64がインストールされていないようで、起動しなくなる可能性があるとBard君が言っていたのでインストールしておきます。

= SUMMARY =

TOTAL:    38
PASSED:   30
SKIPPED:  4
WARNINGS: 4
FAILURES: 0

# apt install grub-efi-amd64

# pve7to8 --full
= SUMMARY =

TOTAL:    38
PASSED:   31
SKIPPED:  4
WARNINGS: 3
FAILURES: 0

残りは再起動すれば良さそうな物だけだったので再起動します。

再起動とチェック

無事立ち上がってクラスタが復活したのをWEBコンソールから確認しました。
消してなかったVMが勝手に立ち上がるのまで問題なかったのでコンソールで見てみます。

# pveversion
pve-manager/8.1.3/b46aac3b42da5d15 (running kernel: 6.5.11-7-pve)
# neofetch
         .://:`              `://:.            
       `hMMMMMMd/          /dMMMMMMh`          ----------------
        `sMMMMMMMd:      :mMMMMMMMs`           OS: Proxmox VE 8.1.3 x86_64
`-/+oo+/:`.yMMMMMMMh-  -hMMMMMMMy.`:/+oo+/-`   Host: PC1PF57K ThinkCentre M75q-1
`:oooooooo/`-hMMMMMMMyyMMMMMMMh-`/oooooooo:`   Kernel: 6.5.11-7-pve
  `/oooooooo:`:mMMMMMMMMMMMMm:`:oooooooo/`     Uptime: 3 mins
    ./ooooooo+- +NMMMMMMMMN+ -+ooooooo/.       Packages: 846 (dpkg)
      .+ooooooo+-`oNMMMMNo`-+ooooooo+.         Shell: bash 5.2.15
        -+ooooooo/.`sMMs`./ooooooo+-           Terminal: /dev/pts/0
          :oooooooo/`..`/oooooooo:             CPU: AMD Ryzen 5 PRO 3400GE w/ Radeon Vega Graphics (8) @ 3.300GHz
          :oooooooo/`..`/oooooooo:             GPU: AMD ATI Radeon Vega Series / Radeon Vega Mobile Series
        -+ooooooo/.`sMMs`./ooooooo+-           Memory: 2317MiB / 38015MiB
      .+ooooooo+-`oNMMMMNo`-+ooooooo+.
    ./ooooooo+- +NMMMMMMMMN+ -+ooooooo/.
  `/oooooooo:`:mMMMMMMMMMMMMm:`:oooooooo/`
`:oooooooo/`-hMMMMMMMyyMMMMMMMh-`/oooooooo:`
`-/+oo+/:`.yMMMMMMMh-  -hMMMMMMMy.`:/+oo+/-`
        `sMMMMMMMm:      :dMMMMMMMs`
       `hMMMMMMd/          /dMMMMMMh`
         `://:`              `://:`

/etc/rsyslog.confでSyslogサーバへの送信設定を入れて終わりです。

オマケでクラスタを組んでいるもう一台も
こっちは最近入れたばかりなので実は最新版
異なるバージョンでもクラスタリングが問題なく組めていたのでそのテストにもなって良かったです。

         .://:`              `://:.            
       `hMMMMMMd/          /dMMMMMMh`          ----------------
        `sMMMMMMMd:      :mMMMMMMMs`           OS: Proxmox VE 8.1.3 x86_64
`-/+oo+/:`.yMMMMMMMh-  -hMMMMMMMy.`:/+oo+/-`   Host: MINI S
`:oooooooo/`-hMMMMMMMyyMMMMMMMh-`/oooooooo:`   Kernel: 6.5.11-7-pve
  `/oooooooo:`:mMMMMMMMMMMMMm:`:oooooooo/`     Uptime: 8 days, 18 hours, 3 mins
    ./ooooooo+- +NMMMMMMMMN+ -+ooooooo/.       Packages: 787 (dpkg)
      .+ooooooo+-`oNMMMMNo`-+ooooooo+.         Shell: bash 5.2.15
        -+ooooooo/.`sMMs`./ooooooo+-           Terminal: /dev/pts/2
          :oooooooo/`..`/oooooooo:             CPU: Intel N100 (4) @ 3.400GHz
          :oooooooo/`..`/oooooooo:             GPU: Intel Alder Lake-N [UHD Graphics]
        -+ooooooo/.`sMMs`./ooooooo+-           Memory: 4133MiB / 15762MiB
      .+ooooooo+-`oNMMMMNo`-+ooooooo+.
    ./ooooooo+- +NMMMMMMMMN+ -+ooooooo/.
  `/oooooooo:`:mMMMMMMMMMMMMm:`:oooooooo/`
`:oooooooo/`-hMMMMMMMyyMMMMMMMh-`/oooooooo:`
`-/+oo+/:`.yMMMMMMMh-  -hMMMMMMMy.`:/+oo+/-`
        `sMMMMMMMm:      :dMMMMMMMs`
       `hMMMMMMd/          /dMMMMMMh`
         `://:`              `://:`

オマケ

Non valid subscription の警告を消す方法。
次のワンライナーで警告表示をしなくする。
sed -Ezi.bak "s/(Ext.Msg.show\(\{\s+title: gettext\('No valid sub)/void\(\{ \/\/\1/g" /usr/share/javascript/proxmox-widget-toolkit/proxmoxlib.js && systemctl restart pveproxy.service

参考
proxmox wiki
Proxmox バージョンのアップグレード [Proxmox シリーズ #7]
Proxmox VE: 7系から8系へupgradeしてみたメモ

Discussion