Open2

Kubernetes

kyamikyami

ある日コントロールプレーンでapt updateするとubuntuのKubernetesパッケージで502 Bad Gatewayとなったので解決の備忘

kyami@k8s-master:~$ sudo apt update -y
[sudo] password for kyami:
~略~
Reading state information... Done
38 packages can be upgraded. Run 'apt list --upgradable' to see them.
W: Failed to fetch https://apt.kubernetes.io/dists/kubernetes-ubuntu/InRelease  502  Bad Gateway [IP: 172.217.25.174 443]
W: Some index files failed to download. They have been ignored, or old ones used instead.

こちらでレガシーパッケージの扱いについて記載がありました。
どうやらapt.kubernetes.ioは2023 年 9 月 13 日以降は 非推奨となり、凍結されているそうです。
https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/change-package-repository/#before-you-begin
なので、新しいパッケージを入れます。
https://kubernetes.io/blog/2023/08/15/pkgs-k8s-io-introduction/#how-to-migrate-deb

kyami@k8s-master:~$ echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.28/deb/ /" | sudo tee /etc/apt/sources.list.d/kubernetes.list
deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.28/deb/ /
kyami@k8s-master:~$ curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.28/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
File '/etc/apt/keyrings/kubernetes-apt-keyring.gpg' exists. Overwrite? (y/N) y
kyami@k8s-master:~$ sudo apt-get update

上記対応でもまだ同じ502エラーが出る場合は、レガシーパッケージを参照する設定が残っている可能性があるので確認し、必要であれば削除します。

kyami@k8s-master:~$ grep -r "apt.kubernetes.io" /etc/apt/sources.list /etc/apt/sources.list.d/
/etc/apt/sources.list.d/archive_uri-https_apt_kubernetes_io_-jammy.list:deb https://apt.kubernetes.io/ kubernetes-ubuntu main
/etc/apt/sources.list.d/archive_uri-https_apt_kubernetes_io_-jammy.list:# deb-src https://apt.kubernetes.io/ kubernetes-ubuntu main
kyami@k8s-master:~$ sudo rm /etc/apt/sources.list.d/archive_uri-https_apt_kubernetes_io_-jammy.list
kyami@k8s-master:~$ sudo apt-get clean
kyami@k8s-master:~$ sudo rm -rf /var/lib/apt/lists/*
kyami@k8s-master:~$ sudo apt-get update
Get:1 https://download.docker.com/linux/ubuntu jammy InRelease [48.8 kB]
Get:2 https://download.docker.com/linux/ubuntu jammy/stable amd64 Packages [42.5 kB]
Get:3 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.28/deb  InRelease [1192 B]
Get:4 http://archive.ubuntu.com/ubuntu jammy InRelease [270 kB]
Get:5 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB]
Get:6 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.28/deb  Packages [21.3 kB]
Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [2006 kB]
Get:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB]
Get:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB]
Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 Packages [1395 kB]
Get:11 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [317 kB]
Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2711 kB]
Get:13 http://archive.ubuntu.com/ubuntu jammy/main Translation-en [510 kB]
Get:14 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [472 kB]
Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [958 kB]
Get:16 http://archive.ubuntu.com/ubuntu jammy/restricted amd64 Packages [129 kB]
Get:17 http://archive.ubuntu.com/ubuntu jammy/restricted Translation-en [18.6 kB]
Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 Packages [14.1 MB]
Get:19 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [204 kB]
Get:20 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.6 kB]
Get:21 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [8260 B]
Get:22 http://archive.ubuntu.com/ubuntu jammy/universe Translation-en [5652 kB]
Get:23 http://archive.ubuntu.com/ubuntu jammy/multiverse amd64 Packages [217 kB]
Get:24 http://archive.ubuntu.com/ubuntu jammy/multiverse Translation-en [112 kB]
Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2252 kB]
Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [380 kB]
Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2838 kB]
Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [494 kB]
Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1181 kB]
Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [288 kB]
Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [44.5 kB]
Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [11.5 kB]
Get:33 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.7 kB]
Get:34 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB]
Get:35 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.9 kB]
Get:36 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB]
Fetched 37.2 MB in 8s (4893 kB/s)
Reading package lists... Done
kyami@k8s-master:~$
kyamikyami

Building Minio on Local Kubernetres

構築手順

kyami@k8s-master:~/minio$ sudo helm repo add minio-operator https://operator.min.io
"minio-operator" has been added to your repositories

PVとPVCの作成

apiVersion: v1
kind: PersistentVolume
metadata:
  name: minio-pv0
spec:
  capacity:
    storage: 10Gi
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: data0-myminio-pool-0-0
  local:
    path: /mnt/data/minio/data0
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - k8s-worker01
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: data0-myminio-pool-0-0
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi
  storageClassName: data0-myminio-pool-0-0

上記PVとPVCを以下の4つ分作成する

  • data0-myminio-pool-0-0
  • data1-myminio-pool-0-0
  • data2-myminio-pool-0-0
  • data3-myminio-pool-0-0

NodePort用の定義ファイルの用意
https://github.com/minio/operator/blob/v7.0.1/examples/kustomization/tenant-nodeport/nodeport-service.yaml

---
apiVersion: v1
kind: Service
metadata:
  name: minio-nodeport
  labels:
    v1.min.io/tenant: myminio
spec:
  ports:
    - name: https-minio
      port: 443
      targetPort: 9000
    - name: http-minio
      port: 80
      targetPort: 9000
  selector:
    v1.min.io/tenant: myminio
  type: NodePort
---
apiVersion: v1
kind: Service
metadata:
  name: console-nodeport
  labels:
    v1.min.io/console: myminio-console
spec:
  ports:
    - name: http-console
      port: 9090
      targetPort: 9090
    - name: https-console
      port: 9443
      targetPort: 9443
  selector:
    v1.min.io/tenant: myminio
  type: NodePort

参考文献

https://min.io/docs/minio/kubernetes/upstream/operations/install-deploy-manage/deploy-operator-helm.html
https://min.io/docs/minio/kubernetes/upstream/reference/tenant-chart-values.html