Zenn
Open8

おうちK8sクラスタでmisskeyを建てる

Kubernetes
Misskey
海都海都

まずPersistentVolume用にiSCSIストレージを用意しなければならないので、どこのご家庭にもあるSynology NASにiSCSIの設定をする。
つってもやることはSAN ManagerからLUNを生やすだけである。

iSCSI建立の図

LUN番号が後々必要になってくるが、管理画面ではわからん。今回は2つめの建立だったのだが、どうやら1始まりのようなので、2が正解だった。

許可はちゃんと設定すべき。後でやっておく。
あとCHAP認証もかけたいが、Proxmoxの管理画面では設定できなくて、/etc/iscsi/iscsid.confを直いじりしなきゃならんらしいのでまだやってない。

https://www.youtube.com/watch?v=oSjiX05IL3k

海都海都

とりあえずRedisから。

長いので折りたたみ
apiVersion: v1
kind: Service
metadata:
  name: redis
  namespace: misskey
spec:
  selector:
    app: redis
  ports:
    - name: http
      port: 6379
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: misskey-redis-pv
  namespace: misskey
spec:
  capacity:
    storage: 1Gi
  accessModes:
    - ReadWriteOnce
  storageClassName: misskey-redis
  iscsi:
    targetPortal: <NAS IP>:3260
    iqn: <Synology iSCSI Target IQN>
    lun: 2
    readOnly: false
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: misskey-redis-pvc
  namespace: misskey
spec:
  resources:
    requests:
      storage: 1Gi
  accessModes:
    - ReadWriteOnce
  storageClassName: misskey-redis
---
apiVersion: v1
kind: Pod
metadata:
  name: redis
  namespace: misskey
  labels:
    name: redis
spec:
  containers:
    - name: redis
      image: redis:7
      resources:
        limits:
          memory: "128Mi"
          cpu: "250m"
      ports:
        - containerPort: 6379
      volumeMounts:
        - mountPath: /data
          name: redis-volume
  volumes:
    - name: redis-volume
      persistentVolumeClaim:
        claimName: misskey-redis-pvc
  restartPolicy: Always

参考サイトはNFSを使っているので、そこだけiSCSIに書き換えた。
残りのコードは参考サイトと大体同じ。

個々のプロパティの意味を雰囲気レベルでしか掴んでないので適当に。
lunにLUN番号書かないといけないけど、何かNAS上で確認する方法はないのだろうか。

海都海都
$ kubectl describe pods -A

Events:
  Type     Reason                  Age                  From                     Message
  ----     ------                  ----                 ----                     -------
  Warning  FailedScheduling        9m50s                default-scheduler        0/4 nodes are available: pod has unbound immediate PersistentVolumeClaims. preemption: 0/4 nodes are available: 4 No preemption victims found for incoming pod..
  Normal   Scheduled               9m49s                default-scheduler        Successfully assigned misskey/postgres to k8sworker02
  Warning  FailedAttachVolume      9m49s                attachdetach-controller  Multi-Attach error for volume "misskey-postgres-pv" Volume is already used by pod(s) redis
  Warning  FailedMount             5m32s                kubelet                  Unable to attach or mount volumes: unmounted volumes=[postgres-volume], unattached volumes=[kube-api-access-lhqqw postgres-volume]: timed out waiting for the condition
  Warning  FailedMount             61s (x3 over 7m46s)  kubelet                  Unable to attach or mount volumes: unmounted volumes=[postgres-volume], unattached volumes=[postgres-volume kube-api-access-lhqqw]: timed out waiting for the condition
  Normal   SuccessfulAttachVolume  20s                  attachdetach-controller  AttachVolume.Attach succeeded for volume "misskey-postgres-pv"
  Normal   Pulled                  2s (x3 over 19s)     kubelet                  Container image "postgres:15" already present on machine
  Normal   Created                 2s (x3 over 19s)     kubelet                  Created container postgres
  Normal   Started                 2s (x3 over 18s)     kubelet                  Started container postgres
  Warning  BackOff                 1s (x3 over 16s)     kubelet                  Back-off restarting failed container postgres in pod postgres_misskey(ab905306-1dce-45fa-8540-4bc5ee480ba3)

Multi-Attach error for volume "misskey-postgres-pv" Volume is already used by pod(s) redis

なんやこれ

海都海都
$ kubectl logs --namespace misskey postgres
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

initdb: error: directory "/var/lib/postgresql/data" exists but is not empty
initdb: detail: It contains a lost+found directory, perhaps due to it being a mount point.
initdb: hint: Using a mount point directly as the data directory is not recommended.
Create a subdirectory under the mount point.

なんかpostgresのデータディレクトリが空じゃないみたい
PGDATA環境変数でサブディレクトリに設定して解決

海都海都

postgres。

長いので折りたたみ
---
apiVersion: v1
kind: Secret
metadata:
  name: postgres-secret
  namespace: misskey
data:
  POSTGRES_PASSWORD: <base64 password>
type: Opaque
stringData:
  POSTGRES_USER: misskey
  POSTGRES_DB: misskey
  PGDATA: /var/lib/postgresql/data/misskey
  POSTGRES_INITDB_ARGS: --encoding=UTF-8 --locale=C
---
apiVersion: v1
kind: Service
metadata:
  name: postgres
  namespace: misskey
spec:
  selector:
    app: postgres
  ports:
    - name: http
      port: 5432
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: misskey-postgres-pv
  namespace: misskey
spec:
  capacity:
    storage: 4Gi
  accessModes:
    - ReadWriteOnce
  storageClassName: misskey-postgres
  iscsi:
    targetPortal: <NAS IP>:3260
    iqn: <Synology iSCSI Target IQN>
    lun: 3
    readOnly: false
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: misskey-postgres-pvc
  namespace: misskey
spec:
  resources:
    requests:
      storage: 4Gi
  accessModes:
    - ReadWriteOnce
  storageClassName: misskey-postgres
---
apiVersion: v1
kind: Pod
metadata:
  name: postgres
  namespace: misskey
  labels:
    name: postgres
spec:
  containers:
    - name: postgres
      image: postgres:15
      resources:
        limits:
          memory: "800Mi"
          cpu: "1024m"
      ports:
        - containerPort: 5432
      volumeMounts:
        - mountPath: /var/lib/postgresql/data
          name: postgres-volume
      envFrom:
        - secretRef:
            name: postgres-secret
  volumes:
    - name: postgres-volume
      persistentVolumeClaim:
        claimName: misskey-postgres-pvc