🏋

ソフトウェアRAID

2023/09/27に公開

環境

Arm64のKVMで動かす Debian12の仮想環境でテストします。
mdadm の他に、パーティション作成で利用する parted もインストールします。

# cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 12 (bookworm)"
NAME="Debian GNU/Linux"
VERSION_ID="12"
VERSION="12 (bookworm)"
VERSION_CODENAME=bookworm
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

# cat /etc/debian_version
12.0

# apt-get install --no-install-recommends mdadm parted

利用するパーティションを切ります。
vdb1 と vdc1 で raid を組む想定です。

# lsblk
NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINTS
vda    254:0    0   16G  0 disk
|-vda1 254:1    0  487M  0 part /boot
|-vda2 254:2    0 15.1G  0 part /
`-vda3 254:3    0  465M  0 part [SWAP]
vdb    254:16   0    8G  0 disk
vdc    254:32   0    8G  0 disk

# parted --script /dev/vdb -- mklabel msdos mkpart primary 0% 100%  print
Model: Virtio Block Device (virtblk)
Disk /dev/vdb: 8590MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:

Number  Start   End     Size    Type     File system  Flags
 1      1049kB  8590MB  8589MB  primary               lba

# parted --script /dev/vdc -- mklabel msdos mkpart primary 0% 100%  print
Model: Virtio Block Device (virtblk)
Disk /dev/vdc: 8590MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:

Number  Start   End     Size    Type     File system  Flags
 1      1049kB  8590MB  8589MB  primary               lba

raid0

vdb1 と vdc1 で raid0 を組みます。

# mdadm --create /dev/md0 -l raid0 -n 2 /dev/vdb1 /dev/vdc1
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md0 started.

結果を確認します。

# cat /proc/mdstat
Personalities : [raid0]
md0 : active raid0 vdc1[1] vdb1[0]
      16764928 blocks super 1.2 512k chunks

unused devices: <none>

# mdadm --detail /dev/md0
/dev/md0:
           Version : 1.2
     Creation Time : Sun Sep 24 10:46:36 2023
        Raid Level : raid0
        Array Size : 16764928 (15.99 GiB 17.17 GB)
      Raid Devices : 2
     Total Devices : 2
       Persistence : Superblock is persistent

       Update Time : Sun Sep 24 10:46:36 2023
             State : clean
    Active Devices : 2
   Working Devices : 2
    Failed Devices : 0
     Spare Devices : 0

            Layout : -unknown-
        Chunk Size : 512K

Consistency Policy : none

              Name : debian12:0  (local to host debian12)
              UUID : 06c0ef59:9a700fc7:5803ba70:fabcd4e3
            Events : 0

    Number   Major   Minor   RaidDevice State
       0     254       17        0      active sync   /dev/vdb1
       1     254       33        1      active sync   /dev/vdc1

OS再起動した時にも自動で raid が構成されるように、情報を保存します。

# mdadm --detail --scan
ARRAY /dev/md0 metadata=1.2 name=debian12:0 UUID=06c0ef59:9a700fc7:5803ba70:fabcd4e3

# mdadm --detail --scan | tee /etc/mdadm.conf
ARRAY /dev/md0 metadata=1.2 name=debian12:0 UUID=06c0ef59:9a700fc7:5803ba70:fabcd4e3

通常のパーティションと同じように、ファイルシステムを作成し、マウントポイントにマウントして利用します。

# mkfs -t ext4 /dev/md0
mke2fs 1.47.0 (5-Feb-2023)
Creating filesystem with 4191232 4k blocks and 1048576 inodes
Filesystem UUID: ee5fe7c1-003b-4d16-b143-b95c04e91dcf
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
        4096000

Allocating group tables: done
Writing inode tables: done
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done

停止と再開

停止します。

# mdadm --stop /dev/md0
mdadm: stopped /dev/md0

# cat /proc/mdstat
Personalities : [raid1] [raid0]
unused devices: <none>

再開します。

# mdadm --assemble /dev/md0 /dev/vdb1 /dev/vdc1
mdadm: /dev/md0 has been started with 2 drives.

# cat /proc/mdstat
Personalities : [raid1] [raid0]
md0 : active raid0 vdb1[0] vdc1[1]
      16764928 blocks super 1.2 512k chunks

unused devices: <none>

# mdadm --detail /dev/md0
/dev/md0:
           Version : 1.2
     Creation Time : Tue Sep 26 07:42:37 2023
        Raid Level : raid0
        Array Size : 16764928 (15.99 GiB 17.17 GB)
      Raid Devices : 2
     Total Devices : 2
       Persistence : Superblock is persistent

       Update Time : Tue Sep 26 07:42:37 2023
             State : clean
    Active Devices : 2
   Working Devices : 2
    Failed Devices : 0
     Spare Devices : 0

            Layout : -unknown-
        Chunk Size : 512K

Consistency Policy : none

              Name : debian12:0  (local to host debian12)
              UUID : be6f1244:d1e04ccf:44b1e2b4:c7635f88
            Events : 0

    Number   Major   Minor   RaidDevice State
       0     254       17        0      active sync   /dev/vdb1
       1     254       33        1      active sync   /dev/vdc1

利用終了

停止した後、スーパーブロックをクリアします。

# mdadm --stop /dev/md0
mdadm: stopped /dev/md0

# mdadm --zero-superblock /dev/vdb1 /dev/vdc1

# rm /etc/mdadm.conf

raid1

vdb1 と vdc1 で raid1 を組みます。

# mdadm --create /dev/md0 -l raid1 -n 2 /dev/vdb1 /dev/vdc1
mdadm: Note: this array has metadata at the start and
    may not be suitable as a boot device.  If you plan to
    store '/boot' on this device please ensure that
    your boot-loader understands md/v1.x metadata, or use
    --metadata=0.90
Continue creating array? y
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md0 started.

結果を確認します。ミラーリングの同期が走ります。

# cat /proc/mdstat
Personalities : [raid0] [raid1]
md0 : active raid1 vdc1[1] vdb1[0]
      8382464 blocks super 1.2 [2/2] [UU]
      [===>.................]  resync = 19.0% (1596032/8382464) finish=1.1min speed=99752K/sec

unused devices: <none>

# mdadm --detail /dev/md0
/dev/md0:
           Version : 1.2
     Creation Time : Sun Sep 24 10:51:09 2023
        Raid Level : raid1
        Array Size : 8382464 (7.99 GiB 8.58 GB)
     Used Dev Size : 8382464 (7.99 GiB 8.58 GB)
      Raid Devices : 2
     Total Devices : 2
       Persistence : Superblock is persistent

       Update Time : Sun Sep 24 10:51:38 2023
             State : clean, resyncing
    Active Devices : 2
   Working Devices : 2
    Failed Devices : 0
     Spare Devices : 0

Consistency Policy : resync

     Resync Status : 37% complete

              Name : debian12:0  (local to host debian12)
              UUID : e6205a36:af6159e5:cfa82350:77ff5545
            Events : 7

    Number   Major   Minor   RaidDevice State
       0     254       17        0      active sync   /dev/vdb1
       1     254       33        1      active sync   /dev/vdc1

同期が終了した状態

# cat /proc/mdstat
Personalities : [raid0] [raid1]
md0 : active raid1 vdc1[1] vdb1[0]
      8382464 blocks super 1.2 [2/2] [UU]

unused devices: <none>

# mdadm --detail /dev/md0
/dev/md0:
           Version : 1.2
     Creation Time : Sun Sep 24 10:51:09 2023
        Raid Level : raid1
        Array Size : 8382464 (7.99 GiB 8.58 GB)
     Used Dev Size : 8382464 (7.99 GiB 8.58 GB)
      Raid Devices : 2
     Total Devices : 2
       Persistence : Superblock is persistent

       Update Time : Sun Sep 24 10:52:36 2023
             State : clean
    Active Devices : 2
   Working Devices : 2
    Failed Devices : 0
     Spare Devices : 0

Consistency Policy : resync

              Name : debian12:0  (local to host debian12)
              UUID : e6205a36:af6159e5:cfa82350:77ff5545
            Events : 18

    Number   Major   Minor   RaidDevice State
       0     254       17        0      active sync   /dev/vdb1
       1     254       33        1      active sync   /dev/vdc1

OS再起動した時にも自動で raid が構成されるように、情報を保存します。

# mdadm --detail --scan
ARRAY /dev/md0 metadata=1.2 name=debian12:0 UUID=e6205a36:af6159e5:cfa82350:77ff5545

# mdadm --detail --scan | tee /etc/mdadm.conf
ARRAY /dev/md0 metadata=1.2 name=debian12:0 UUID=e6205a36:af6159e5:cfa82350:77ff5545

ディスク交換の練習

交換するディスクに faulty のマークをします。

# mdadm --fail /dev/md0 /dev/vdb1
mdadm: set /dev/vdb1 faulty in /dev/md0

# cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 vdc1[2] vdb1[0](F)
      8382464 blocks super 1.2 [2/1] [_U]

unused devices: <none>

# mdadm --detail /dev/md0
/dev/md0:
           Version : 1.2
     Creation Time : Sun Sep 24 11:00:01 2023
        Raid Level : raid1
        Array Size : 8382464 (7.99 GiB 8.58 GB)
     Used Dev Size : 8382464 (7.99 GiB 8.58 GB)
      Raid Devices : 2
     Total Devices : 2
       Persistence : Superblock is persistent

       Update Time : Tue Sep 26 06:48:34 2023
             State : clean, degraded
    Active Devices : 1
   Working Devices : 1
    Failed Devices : 1
     Spare Devices : 0

Consistency Policy : resync

              Name : debian12:0  (local to host debian12)
              UUID : bd41c118:e82d16bd:c3c01df0:38124705
            Events : 48

    Number   Major   Minor   RaidDevice State
       -       0        0        0      removed
       2     254       33        1      active sync   /dev/vdc1

       0     254       17        -      faulty   /dev/vdb1

raid からディスクを除去します。

# mdadm --remove /dev/md0 /dev/vdb1
mdadm: hot removed /dev/vdb1 from /dev/md0

# cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 vdc1[2]
      8382464 blocks super 1.2 [2/1] [_U]

unused devices: <none>

# mdadm --detail /dev/md0
/dev/md0:
           Version : 1.2
     Creation Time : Sun Sep 24 11:00:01 2023
        Raid Level : raid1
        Array Size : 8382464 (7.99 GiB 8.58 GB)
     Used Dev Size : 8382464 (7.99 GiB 8.58 GB)
      Raid Devices : 2
     Total Devices : 1
       Persistence : Superblock is persistent

       Update Time : Tue Sep 26 06:50:03 2023
             State : clean, degraded
    Active Devices : 1
   Working Devices : 1
    Failed Devices : 0
     Spare Devices : 0

Consistency Policy : resync

              Name : debian12:0  (local to host debian12)
              UUID : bd41c118:e82d16bd:c3c01df0:38124705
            Events : 49

    Number   Major   Minor   RaidDevice State
       -       0        0        0      removed
       2     254       33        1      active sync   /dev/vdc1

物理的に古いディスクを取り外し、新しいディスクを付けた後(仮想環境でテストしているの、論理的に、ですが)
最初と同様にパーティションを切ります。

# lsblk
NAME    MAJ:MIN RM  SIZE RO TYPE  MOUNTPOINTS
vda     254:0    0   16G  0 disk
|-vda1  254:1    0  487M  0 part  /boot
|-vda2  254:2    0 15.1G  0 part  /
`-vda3  254:3    0  465M  0 part  [SWAP]
vdb     254:16   0    8G  0 disk
vdc     254:32   0    8G  0 disk
`-vdc1  254:33   0    8G  0 part
  `-md0   9:0    0    8G  0 raid1

# parted --script /dev/vdb -- mklabel msdos mkpart primary 0% 100%  print
Model: Virtio Block Device (virtblk)
Disk /dev/vdb: 8590MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:

Number  Start   End     Size    Type     File system  Flags
 1      1049kB  8590MB  8589MB  primary               lba

準備したパーティションを raid に追加します。

# mdadm --add /dev/md0 /dev/vdb1
mdadm: added /dev/vdb1

同期が走ります。

# cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 vdb1[3] vdc1[2]
      8382464 blocks super 1.2 [2/1] [_U]
      [=>...................]  recovery =  6.1% (516608/8382464) finish=0.7min speed=172202K/sec

unused devices: <none>

# mdadm --detail /dev/md0
/dev/md0:
           Version : 1.2
     Creation Time : Sun Sep 24 11:00:01 2023
        Raid Level : raid1
        Array Size : 8382464 (7.99 GiB 8.58 GB)
     Used Dev Size : 8382464 (7.99 GiB 8.58 GB)
      Raid Devices : 2
     Total Devices : 2
       Persistence : Superblock is persistent

       Update Time : Tue Sep 26 06:58:23 2023
             State : clean, degraded, recovering
    Active Devices : 1
   Working Devices : 2
    Failed Devices : 0
     Spare Devices : 1

Consistency Policy : resync

    Rebuild Status : 9% complete

              Name : debian12:0  (local to host debian12)
              UUID : bd41c118:e82d16bd:c3c01df0:38124705
            Events : 53

    Number   Major   Minor   RaidDevice State
       3     254       17        0      spare rebuilding   /dev/vdb1
       2     254       33        1      active sync   /dev/vdc1

同期が終了した状態

# cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 vdb1[3] vdc1[2]
      8382464 blocks super 1.2 [2/2] [UU]

unused devices: <none>

# mdadm --detail /dev/md0
/dev/md0:
           Version : 1.2
     Creation Time : Sun Sep 24 11:00:01 2023
        Raid Level : raid1
        Array Size : 8382464 (7.99 GiB 8.58 GB)
     Used Dev Size : 8382464 (7.99 GiB 8.58 GB)
      Raid Devices : 2
     Total Devices : 2
       Persistence : Superblock is persistent

       Update Time : Tue Sep 26 07:00:06 2023
             State : clean
    Active Devices : 2
   Working Devices : 2
    Failed Devices : 0
     Spare Devices : 0

Consistency Policy : resync

              Name : debian12:0  (local to host debian12)
              UUID : bd41c118:e82d16bd:c3c01df0:38124705
            Events : 69

    Number   Major   Minor   RaidDevice State
       3     254       17        0      active sync   /dev/vdb1
       2     254       33        1      active sync   /dev/vdc1

停止と再開

停止します。

# mdadm --stop /dev/md0
mdadm: stopped /dev/md0

# cat /proc/mdstat
Personalities : [raid1]
unused devices: <none>

再開します。

# mdadm --assemble /dev/md0 /dev/vdb1 /dev/vdc1
mdadm: /dev/md0 has been started with 2 drives.

# cat /proc/mdstat
Personalities : [raid1]
md0 : active (auto-read-only) raid1 vdb1[3] vdc1[2]
      8382464 blocks super 1.2 [2/2] [UU]

unused devices: <none>

# mdadm --detail /dev/md0
/dev/md0:
           Version : 1.2
     Creation Time : Sun Sep 24 11:00:01 2023
        Raid Level : raid1
        Array Size : 8382464 (7.99 GiB 8.58 GB)
     Used Dev Size : 8382464 (7.99 GiB 8.58 GB)
      Raid Devices : 2
     Total Devices : 2
       Persistence : Superblock is persistent

       Update Time : Tue Sep 26 07:06:11 2023
             State : clean
    Active Devices : 2
   Working Devices : 2
    Failed Devices : 0
     Spare Devices : 0

Consistency Policy : resync

              Name : debian12:0  (local to host debian12)
              UUID : bd41c118:e82d16bd:c3c01df0:38124705
            Events : 69

    Number   Major   Minor   RaidDevice State
       3     254       17        0      active sync   /dev/vdb1
       2     254       33        1      active sync   /dev/vdc1

利用終了

停止した後、スーパーブロックをクリアします。

# mdadm --stop /dev/md0
mdadm: stopped /dev/md0

# mdadm --zero-superblock /dev/vdb1 /dev/vdc1

# rm /etc/mdadm.conf

raid5

vdb1、vdc1、vdd1、vde1 で raid5 を組む準備をします。

# lsblk
NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINTS
vda    254:0    0   16G  0 disk
|-vda1 254:1    0  487M  0 part /boot
|-vda2 254:2    0 15.1G  0 part /
`-vda3 254:3    0  465M  0 part [SWAP]
vdb    254:16   0    8G  0 disk
`-vdb1 254:17   0    8G  0 part
vdc    254:32   0    8G  0 disk
`-vdc1 254:33   0    8G  0 part
vdd    254:48   0    8G  0 disk
vde    254:64   0    8G  0 disk

# parted --script /dev/vdd -- mklabel msdos mkpart primary 0% 100%  print
Model: Virtio Block Device (virtblk)
Disk /dev/vdd: 8590MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:

Number  Start   End     Size    Type     File system  Flags
 1      1049kB  8590MB  8589MB  primary               lba


# parted --script /dev/vde -- mklabel msdos mkpart primary 0% 100%  prin
Model: Virtio Block Device (virtblk)
Disk /dev/vde: 8590MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:

Number  Start   End     Size    Type     File system  Flags
 1      1049kB  8590MB  8589MB  primary               lba

raid5 を組みます。vde1 はスペアディスクとします。

# mdadm --create /dev/md0 -l raid5 -n 3 /dev/vdb1 /dev/vdc1 /dev/vdd1 --spare-devices=1 /dev/vde1
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md0 started.

結果を確認します。ミラーリングの同期が走ります。

# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active raid5 vdd1[4] vde1[3](S) vdc1[1] vdb1[0]
      16764928 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/2] [UU_]
      [>....................]  recovery =  4.7% (395772/8382464) finish=2.3min speed=56538K/sec

# mdadm --detail /dev/md0
/dev/md0:
           Version : 1.2
     Creation Time : Wed Sep 27 05:19:06 2023
        Raid Level : raid5
        Array Size : 16764928 (15.99 GiB 17.17 GB)
     Used Dev Size : 8382464 (7.99 GiB 8.58 GB)
      Raid Devices : 3
     Total Devices : 4
       Persistence : Superblock is persistent

       Update Time : Wed Sep 27 05:19:19 2023
             State : clean, degraded, recovering
    Active Devices : 2
   Working Devices : 4
    Failed Devices : 0
     Spare Devices : 2

            Layout : left-symmetric
        Chunk Size : 512K

Consistency Policy : resync

    Rebuild Status : 12% complete

              Name : debian12:0  (local to host debian12)
              UUID : 5fff5a4d:579c17ce:497a40ce:27185fb1
            Events : 3

    Number   Major   Minor   RaidDevice State
       0     254       17        0      active sync   /dev/vdb1
       1     254       33        1      active sync   /dev/vdc1
       4     254       49        2      spare rebuilding   /dev/vdd1

       3     254       65        -      spare   /dev/vde1

同期が終了した状態

# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active raid5 vdd1[4] vde1[3](S) vdc1[1] vdb1[0]
      16764928 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU]

unused devices: <none>

# mdadm --detail /dev/md0
/dev/md0:
           Version : 1.2
     Creation Time : Wed Sep 27 05:19:06 2023
        Raid Level : raid5
        Array Size : 16764928 (15.99 GiB 17.17 GB)
     Used Dev Size : 8382464 (7.99 GiB 8.58 GB)
      Raid Devices : 3
     Total Devices : 4
       Persistence : Superblock is persistent

       Update Time : Wed Sep 27 05:22:18 2023
             State : clean
    Active Devices : 3
   Working Devices : 4
    Failed Devices : 0
     Spare Devices : 1

            Layout : left-symmetric
        Chunk Size : 512K

Consistency Policy : resync

              Name : debian12:0  (local to host debian12)
              UUID : 5fff5a4d:579c17ce:497a40ce:27185fb1
            Events : 19

    Number   Major   Minor   RaidDevice State
       0     254       17        0      active sync   /dev/vdb1
       1     254       33        1      active sync   /dev/vdc1
       4     254       49        2      active sync   /dev/vdd1

       3     254       65        -      spare   /dev/vde1

OS再起動した時にも自動で raid が構成されるように、情報を保存します。

# mdadm --detail --scan
ARRAY /dev/md0 metadata=1.2 name=debian12:0 UUID=5fff5a4d:579c17ce:497a40ce:27185fb1

# mdadm --detail --scan | tee /etc/mdadm.conf
ARRAY /dev/md0 metadata=1.2 name=debian12:0 UUID=5fff5a4d:579c17ce:497a40ce:27185fb1

ディスク交換の練習

交換するディスクに faulty のマークをします。

# mdadm --fail /dev/md0 /dev/vdb1
mdadm: set /dev/vdb1 faulty in /dev/md0

同期が走ります。

# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active raid5 vdd1[4] vde1[3] vdc1[1] vdb1[0](F)
      16764928 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/2] [_UU]
      [=>...................]  recovery =  8.6% (727944/8382464) finish=2.8min speed=45496K/sec

unused devices: <none>

# mdadm --detail /dev/md0
/dev/md0:
           Version : 1.2
     Creation Time : Wed Sep 27 05:19:06 2023
        Raid Level : raid5
        Array Size : 16764928 (15.99 GiB 17.17 GB)
     Used Dev Size : 8382464 (7.99 GiB 8.58 GB)
      Raid Devices : 3
     Total Devices : 4
       Persistence : Superblock is persistent

       Update Time : Wed Sep 27 05:23:55 2023
             State : clean, degraded, recovering
    Active Devices : 2
   Working Devices : 3
    Failed Devices : 1
     Spare Devices : 1

            Layout : left-symmetric
        Chunk Size : 512K

Consistency Policy : resync

    Rebuild Status : 12% complete

              Name : debian12:0  (local to host debian12)
              UUID : 5fff5a4d:579c17ce:497a40ce:27185fb1
            Events : 23

    Number   Major   Minor   RaidDevice State
       3     254       65        0      spare rebuilding   /dev/vde1
       1     254       33        1      active sync   /dev/vdc1
       4     254       49        2      active sync   /dev/vdd1

       0     254       17        -      faulty   /dev/vdb1

同期が終了した状態

# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active raid5 vdd1[4] vde1[3] vdc1[1] vdb1[0](F)
      16764928 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU]

unused devices: <none>

# mdadm --detail /dev/md0
/dev/md0:
           Version : 1.2
     Creation Time : Wed Sep 27 05:19:06 2023
        Raid Level : raid5
        Array Size : 16764928 (15.99 GiB 17.17 GB)
     Used Dev Size : 8382464 (7.99 GiB 8.58 GB)
      Raid Devices : 3
     Total Devices : 4
       Persistence : Superblock is persistent

       Update Time : Wed Sep 27 05:26:45 2023
             State : clean
    Active Devices : 3
   Working Devices : 3
    Failed Devices : 1
     Spare Devices : 0

            Layout : left-symmetric
        Chunk Size : 512K

Consistency Policy : resync

              Name : debian12:0  (local to host debian12)
              UUID : 5fff5a4d:579c17ce:497a40ce:27185fb1
            Events : 38

    Number   Major   Minor   RaidDevice State
       3     254       65        0      active sync   /dev/vde1
       1     254       33        1      active sync   /dev/vdc1
       4     254       49        2      active sync   /dev/vdd1

       0     254       17        -      faulty   /dev/vdb1

raid からディスクを除去します。

# mdadm --remove /dev/md0 /dev/vdb1
mdadm: hot removed /dev/vdb1 from /dev/md0

# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active raid5 vdd1[4] vde1[3] vdc1[1]
      16764928 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU]

unused devices: <none>

# mdadm --detail /dev/md0
/dev/md0:
           Version : 1.2
     Creation Time : Wed Sep 27 05:19:06 2023
        Raid Level : raid5
        Array Size : 16764928 (15.99 GiB 17.17 GB)
     Used Dev Size : 8382464 (7.99 GiB 8.58 GB)
      Raid Devices : 3
     Total Devices : 3
       Persistence : Superblock is persistent

       Update Time : Wed Sep 27 05:28:10 2023
             State : clean
    Active Devices : 3
   Working Devices : 3
    Failed Devices : 0
     Spare Devices : 0

            Layout : left-symmetric
        Chunk Size : 512K

Consistency Policy : resync

              Name : debian12:0  (local to host debian12)
              UUID : 5fff5a4d:579c17ce:497a40ce:27185fb1
            Events : 39

    Number   Major   Minor   RaidDevice State
       3     254       65        0      active sync   /dev/vde1
       1     254       33        1      active sync   /dev/vdc1
       4     254       49        2      active sync   /dev/vdd1

ホットスペアを追加します。

# mdadm --add /dev/md0 /dev/vdb1
mdadm: added /dev/vdb1

# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active raid5 vdb1[5](S) vdd1[4] vde1[3] vdc1[1]
      16764928 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU]

unused devices: <none>

# mdadm --detail /dev/md0
/dev/md0:
           Version : 1.2
     Creation Time : Wed Sep 27 05:19:06 2023
        Raid Level : raid5
        Array Size : 16764928 (15.99 GiB 17.17 GB)
     Used Dev Size : 8382464 (7.99 GiB 8.58 GB)
      Raid Devices : 3
     Total Devices : 4
       Persistence : Superblock is persistent

       Update Time : Wed Sep 27 05:31:58 2023
             State : clean
    Active Devices : 3
   Working Devices : 4
    Failed Devices : 0
     Spare Devices : 1

            Layout : left-symmetric
        Chunk Size : 512K

Consistency Policy : resync

              Name : debian12:0  (local to host debian12)
              UUID : 5fff5a4d:579c17ce:497a40ce:27185fb1
            Events : 40

    Number   Major   Minor   RaidDevice State
       3     254       65        0      active sync   /dev/vde1
       1     254       33        1      active sync   /dev/vdc1
       4     254       49        2      active sync   /dev/vdd1

       5     254       17        -      spare   /dev/vdb1

停止と再開

停止します。

# mdadm --stop /dev/md0
mdadm: stopped /dev/md0

# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
unused devices: <none>

再開します。

# mdadm --assemble /dev/md0 /dev/vdb1 /dev/vdc1 /dev/vdd1 /dev/vde1
mdadm: /dev/md0 has been started with 3 drives and 1 spare.

# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active (auto-read-only) raid5 vde1[3] vdb1[5](S) vdd1[4] vdc1[1]
      16764928 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU]

unused devices: <none>

# mdadm --detail /dev/md0
/dev/md0:
           Version : 1.2
     Creation Time : Wed Sep 27 05:19:06 2023
        Raid Level : raid5
        Array Size : 16764928 (15.99 GiB 17.17 GB)
     Used Dev Size : 8382464 (7.99 GiB 8.58 GB)
      Raid Devices : 3
     Total Devices : 4
       Persistence : Superblock is persistent

       Update Time : Wed Sep 27 05:31:58 2023
             State : clean
    Active Devices : 3
   Working Devices : 4
    Failed Devices : 0
     Spare Devices : 1

            Layout : left-symmetric
        Chunk Size : 512K

Consistency Policy : resync

              Name : debian12:0  (local to host debian12)
              UUID : 5fff5a4d:579c17ce:497a40ce:27185fb1
            Events : 40

    Number   Major   Minor   RaidDevice State
       3     254       65        0      active sync   /dev/vde1
       1     254       33        1      active sync   /dev/vdc1
       4     254       49        2      active sync   /dev/vdd1

       5     254       17        -      spare   /dev/vdb1

利用終了

停止した後、スーパーブロックをクリアします。

# mdadm --stop /dev/md0
mdadm: stopped /dev/md0

# mdadm --zero-superblock /dev/vdb1 /dev/vdc1 /dev/vdd1 /dev/vde1

# rm /etc/mdadm.conf
GitHubで編集を提案

Discussion