🕸️
helm 学習のために WSL2 に minikube で k8s 環境を作る
目的
helm とか helmfile 勉強したいモチベーションはあるがそもそも k8s 環境が無いので作る
(ただし EKS とか GKE 使うほどガチで k8s 環境組みたいわけではないのでローカルでなんとかしたい
先に記しておくと、途中枝葉の部分で詰まったのを含めて所要時間は2時間半程度でした。
きちんと環境整備してある人なら1時間くらいで終わるのではと思います。
やること
- kubectl インストール
- minikube でローカルに環境構築
- helm, helmfile diff, helmfile インストール
kubectl インストール
大したことないので割愛。
install コマンド実行結果
> sudo apt-get update && sudo apt-get install -y apt-transport-https gnupg2
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee -a /etc/apt/sources.list.d/kubernetes.list
sudo apt-get update
sudo apt-get install -y kubectl
Get:1 file:/var/cuda-repo-wsl-ubuntu-11-7-local InRelease [1575 B]
Get:1 file:/var/cuda-repo-wsl-ubuntu-11-7-local InRelease [1575 B]
Get:2 https://dl.yarnpkg.com/debian stable InRelease [17.1 kB]
Get:3 https://packages.microsoft.com/ubuntu/20.04/prod focal InRelease [3632 B]
Get:4 https://download.docker.com/linux/ubuntu focal InRelease [57.7 kB]
Err:2 https://dl.yarnpkg.com/debian stable InRelease
The following signatures were invalid: EXPKEYSIG 23E7166788B63E1E Yarn Packaging <yarn@dan.cx>
Get:5 https://dl.google.com/linux/chrome/deb stable InRelease [1825 B]
Err:5 https://dl.google.com/linux/chrome/deb stable InRelease
The following signatures couldn't be verified because the public key is not available: NO_PUBKEY E88979FB9B30ACF2
Get:6 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB]
Hit:7 http://ppa.launchpad.net/git-core/ppa/ubuntu focal InRelease
Get:8 http://packages.cloud.google.com/apt cloud-sdk InRelease [6361 B]
Hit:9 http://archive.ubuntu.com/ubuntu focal InRelease
Err:8 http://packages.cloud.google.com/apt cloud-sdk InRelease
The following signatures couldn't be verified because the public key is not available: NO_PUBKEY B53DC80D13EDEF05
Get:10 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB]
Hit:11 http://ppa.launchpad.net/longsleep/golang-backports/ubuntu focal InRelease
Ign:12 https://apt.fury.io/ascii-image-converter InRelease
Ign:13 https://apt.fury.io/ascii-image-converter Release
Get:14 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [2773 kB]
Ign:15 https://apt.fury.io/ascii-image-converter Packages
Ign:16 https://apt.fury.io/ascii-image-converter Translation-en
Hit:17 http://archive.ubuntu.com/ubuntu focal-backports InRelease
Get:15 https://apt.fury.io/ascii-image-converter Packages
Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3151 kB]
Ign:16 https://apt.fury.io/ascii-image-converter Translation-en
Ign:16 https://apt.fury.io/ascii-image-converter Translation-en
Get:19 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [421 kB]
Ign:16 https://apt.fury.io/ascii-image-converter Translation-en
Get:20 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [2636 kB]
Get:21 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [368 kB]
Ign:16 https://apt.fury.io/ascii-image-converter Translation-en
Get:22 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [948 kB]
Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [503 kB]
Ign:16 https://apt.fury.io/ascii-image-converter Translation-en
Get:24 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [2753 kB]
Ign:16 https://apt.fury.io/ascii-image-converter Translation-en
Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [384 kB]
Get:26 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1174 kB]
Get:27 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [199 kB]
Get:28 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [281 kB]
Fetched 15.9 MB in 5s (3479 kB/s)
Reading package lists... Done
W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: https://dl.yarnpkg.com/debian stable InRelease: The following signatures were invalid: EXPKEYSIG 23E7166788B63E1E Yarn Packaging <yarn@dan.cx>
W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: https://dl.google.com/linux/chrome/deb stable InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY E88979FB9B30ACF2
W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: http://packages.cloud.google.com/apt cloud-sdk InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY B53DC80D13EDEF05
W: Failed to fetch https://dl.google.com/linux/chrome/deb/dists/stable/InRelease The following signatures couldn't be verified because the public key is not available: NO_PUBKEY E88979FB9B30ACF2
W: Failed to fetch http://packages.cloud.google.com/apt/dists/cloud-sdk/InRelease The following signatures couldn't be verified because the public key is not available: NO_PUBKEY B53DC80D13EDEF05
W: Failed to fetch https://dl.yarnpkg.com/debian/dists/stable/InRelease The following signatures were invalid: EXPKEYSIG 23E7166788B63E1E Yarn Packaging <yarn@dan.cx>
W: Some index files failed to download. They have been ignored, or old ones used instead.
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
golang-1.19-go golang-1.19-src libxmlb1
Use 'sudo apt autoremove' to remove them.
The following NEW packages will be installed:
gnupg2
The following packages will be upgraded:
apt-transport-https
1 upgraded, 1 newly installed, 0 to remove and 296 not upgraded.
Need to get 7020 B of archives.
After this operation, 51.2 kB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 apt-transport-https all 2.0.10 [1704 B]
Get:2 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 gnupg2 all 2.2.19-3ubuntu2.2 [5316 B]
Fetched 7020 B in 1s (13.0 kB/s)
(Reading database ... 197257 files and directories currently installed.)
Preparing to unpack .../apt-transport-https_2.0.10_all.deb ...
Unpacking apt-transport-https (2.0.10) over (2.0.9) ...
Selecting previously unselected package gnupg2.
Preparing to unpack .../gnupg2_2.2.19-3ubuntu2.2_all.deb ...
Unpacking gnupg2 (2.2.19-3ubuntu2.2) ...
Setting up gnupg2 (2.2.19-3ubuntu2.2) ...
Setting up apt-transport-https (2.0.10) ...
Processing triggers for man-db (2.9.1-1) ...
OK
deb https://apt.kubernetes.io/ kubernetes-xenial main
Get:1 file:/var/cuda-repo-wsl-ubuntu-11-7-local InRelease [1575 B]
Get:1 file:/var/cuda-repo-wsl-ubuntu-11-7-local InRelease [1575 B]
Hit:2 https://download.docker.com/linux/ubuntu focal InRelease
Get:3 https://dl.yarnpkg.com/debian stable InRelease [17.1 kB]
Hit:4 https://packages.microsoft.com/ubuntu/20.04/prod focal InRelease
Get:5 https://dl.google.com/linux/chrome/deb stable InRelease [1825 B]
Err:3 https://dl.yarnpkg.com/debian stable InRelease
The following signatures were invalid: EXPKEYSIG 23E7166788B63E1E Yarn Packaging <yarn@dan.cx>
Ign:6 https://packages.cloud.google.com/apt kubernetes-xenial InRelease
Err:5 https://dl.google.com/linux/chrome/deb stable InRelease
The following signatures couldn't be verified because the public key is not available: NO_PUBKEY E88979FB9B30ACF2
Hit:7 http://security.ubuntu.com/ubuntu focal-security InRelease
Err:8 https://packages.cloud.google.com/apt kubernetes-xenial Release
404 Not Found [IP: 142.250.199.110 443]
Get:9 http://packages.cloud.google.com/apt cloud-sdk InRelease [6361 B]
Err:9 http://packages.cloud.google.com/apt cloud-sdk InRelease
The following signatures couldn't be verified because the public key is not available: NO_PUBKEY B53DC80D13EDEF05
Hit:10 http://archive.ubuntu.com/ubuntu focal InRelease
Hit:11 http://archive.ubuntu.com/ubuntu focal-updates InRelease
Ign:12 https://apt.fury.io/ascii-image-converter InRelease
Hit:13 http://archive.ubuntu.com/ubuntu focal-backports InRelease
Ign:14 https://apt.fury.io/ascii-image-converter Release
Ign:15 https://apt.fury.io/ascii-image-converter Packages
Hit:16 http://ppa.launchpad.net/git-core/ppa/ubuntu focal InRelease
Ign:17 https://apt.fury.io/ascii-image-converter Translation-en
Get:15 https://apt.fury.io/ascii-image-converter Packages
Hit:18 http://ppa.launchpad.net/longsleep/golang-backports/ubuntu focal InRelease
Ign:17 https://apt.fury.io/ascii-image-converter Translation-en
Ign:17 https://apt.fury.io/ascii-image-converter Translation-en
Ign:17 https://apt.fury.io/ascii-image-converter Translation-en
Ign:17 https://apt.fury.io/ascii-image-converter Translation-en
Ign:17 https://apt.fury.io/ascii-image-converter Translation-en
Ign:17 https://apt.fury.io/ascii-image-converter Translation-en
Reading package lists... Done
W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: https://dl.yarnpkg.com/debian stable InRelease: The following signatures were invalid: EXPKEYSIG 23E7166788B63E1E Yarn Packaging <yarn@dan.cx>
W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: https://dl.google.com/linux/chrome/deb stable InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY E88979FB9B30ACF2
E: The repository 'https://apt.kubernetes.io kubernetes-xenial Release' does not have a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: http://packages.cloud.google.com/apt cloud-sdk InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY B53DC80D13EDEF05
Reading package lists... Done
Building dependency tree
Reading state information... Done
kubectl is already the newest version (1.24.4-00).
The following packages were automatically installed and are no longer required:
golang-1.19-go golang-1.19-src libxmlb1
Use 'sudo apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 296 not upgraded.
minikube インストールと k8s 環境構築
インストール
公式 の手順に従うのみです
下記は Linux, x86-64, Satble, Debian package を選択した場合の実行例です。
> curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube_latest_amd64.deb
sudo dpkg -i minikube_latest_amd64.deb
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 28.9M 100 28.9M 0 0 7723k 0 0:00:03 0:00:03 --:--:-- 7721k
[sudo] password for minatosingull:
Selecting previously unselected package minikube.
(Reading database ... 197256 files and directories currently installed.)
Preparing to unpack minikube_latest_amd64.deb ...
Unpacking minikube (1.32.0-0) ...
Setting up minikube (1.32.0-0) ...
クラスター作成
> minikube start
😄 minikube v1.32.0 on Ubuntu 20.04 (amd64)
👎 Unable to pick a default driver. Here is what was considered, in preference order:
▪ docker: Not healthy: "docker version --format {{.Server.Os}}-{{.Server.Version}}:{{.Server.Platform.Name}}" exit status 1: Cannot connect to the Docker daemon at tcp://localhost:2375. Is the docker daemon running?
▪ docker: Suggestion: Start the Docker service <https://minikube.sigs.k8s.io/docs/drivers/docker/>
💡 Alternatively you could install one of these drivers:
▪ kvm2: Not installed: exec: "virsh": executable file not found in $PATH
▪ qemu2: Not installed: stat /usr/share/OVMF/OVMF_CODE.fd: no such file or directory
▪ podman: Not installed: exec: "podman": executable file not found in $PATH
▪ virtualbox: Not installed: unable to find VBoxManage in $PATH
❌ Exiting due to DRV_DOCKER_NOT_RUNNING: Found docker, but the docker service isn't running. Try restarting the docker service.
失敗した...
docker version --format {{.Server.Os}}-{{.Server.Version}}:{{.Server.Platform.Name}}" exit status 1: Cannot connect to the Docker daemon at tcp://localhost:2375.
同コマンドを実行してみる
> docker version --format {{.Server.Os}}-{{.Server.Version}}:{{.Server.Platform.Name}}
linux-20.10.24:Docker Desktop
実行できた。なぜ。
どうも sudo あり docker と sudo 無し docker で挙動が違うらしい。
※すごい昔に alias docker="sudo docker" を設定していたらしい
> docker ps
Cannot connect to the Docker daemon at tcp://localhost:2375. Is the docker daemon running?
~ .............................................................................................................................................................................................................................................. 00:49:25
> sudo docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
8d93f5139046 mysql:8.0 "docker-entrypoint.s…" 8 months ago Up 4 minutes 33060/tcp, 0.0.0.0:3308->3306/tcp xxxxx_test
"docker Cannot connect to the Docker daemon at tcp://localhost:2375."
で ggl と以下の記事を発見。
WSL2でDOCKER_HOSTが指定されているとDockerが動かない
実際、元々 WSL で docker を使っていて後から WSL に上げたという経緯がある。
.zshrc を見てみると記事中にあるとおり DOCKER_HOST 指定があったので外して再度 docker コマンドの挙動を確認
> docker version
Client: Docker Engine - Community
Version: 23.0.1
API version: 1.42
Go version: go1.19.5
Git commit: a5ee5b1
Built: Thu Feb 9 19:46:56 2023
OS/Arch: linux/amd64
Context: default
Server: Docker Engine - Community
Engine:
Version: 23.0.1
API version: 1.42 (minimum version 1.12)
Go version: go1.19.5
Git commit: bc3805a
Built: Thu Feb 9 19:46:56 2023
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.6.18
GitCommit: 2456e983eb9e37e47538f59ea18f2043c9a73640
runc:
Version: 1.1.4
GitCommit: v1.1.4-0-g5fd4c4d
docker-init:
Version: 0.19.0
GitCommit: de40ad0
動いたので再度 minikube の起動を試みる
> minikube start
😄 minikube v1.32.0 on Ubuntu 20.04 (amd64)
✨ Automatically selected the docker driver. Other choices: none, ssh
📌 Using Docker driver with root privileges
👍 Starting control plane node minikube in cluster minikube
🚜 Pulling base image ...
💾 Downloading Kubernetes v1.28.3 preload ...
> preloaded-images-k8s-v18-v1...: 403.35 MiB / 403.35 MiB 100.00% 5.48 Mi
> gcr.io/k8s-minikube/kicbase...: 453.90 MiB / 453.90 MiB 100.00% 5.75 Mi
🔥 Creating docker container (CPUs=2, Memory=2200MB) ...
🐳 Preparing Kubernetes v1.28.3 on Docker 24.0.7 ...
▪ Generating certificates and keys ...
▪ Booting up control plane ...
▪ Configuring RBAC rules ...
🔗 Configuring bridge CNI (Container Networking Interface) ...
▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
🔎 Verifying Kubernetes components...
🌟 Enabled addons: storage-provisioner, default-storageclass
❗ /usr/bin/kubectl is version 1.24.4, which may have incompatibilities with Kubernetes 1.28.3.
▪ Want kubectl v1.28.3? Try 'minikube kubectl -- get pods -A'
🏄 Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default
(kubectl のバージョンの warning 出てるけどまぁいったん無視しましょう。
default の namespace 指定でつながるように設定までしてくれたらしいので早速繋いでみましょう。
> minikube kubectl -- get pods -A
> kubectl.sha256: 64 B / 64 B [-------------------------] 100.00% ? p/s 0s
> kubectl: 47.56 MiB / 47.56 MiB [-------------] 100.00% 6.76 MiB p/s 7.2s
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system coredns-5dd5756b68-zml9z 1/1 Running 0 4m49s
kube-system etcd-minikube 1/1 Running 0 5m
kube-system kube-apiserver-minikube 1/1 Running 0 5m
kube-system kube-controller-manager-minikube 1/1 Running 0 5m
kube-system kube-proxy-gtzjp 1/1 Running 0 4m50s
kube-system kube-scheduler-minikube 1/1 Running 0 5m4s
kube-system storage-provisioner 1/1 Running 0 4m58s
Done 🎉
適当に公式のサンプルをデプロイしてみます。
> kubectl apply -f hello-world.yml
deployment.apps/nginx-deployment created
> kc get pod
NAME READY STATUS RESTARTS AGE
nginx-deployment-86dcfdf4c6-bv65m 1/1 Running 0 65s
nginx-deployment-86dcfdf4c6-vnkqk 1/1 Running 0 65s
nginx-deployment-86dcfdf4c6-vvq4d 1/1 Running 0 65s
無事デプロイできたことが確認できました 🎊
helm, helmfile diff, helmfile をインストール
helm インストール(スクリプト), helm チャートリポジトリ初期化
> curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3
~/workdir/minikube-deploy ...................................................................................... 01:44:17
> chmod 700 get_helm.sh
~/workdir/minikube-deploy ...................................................................................... 01:44:23
> ./get_helm.sh
Downloading https://get.helm.sh/helm-v3.14.2-linux-amd64.tar.gz
Verifying checksum... Done.
Preparing to install helm into /usr/local/bin
[sudo] password for minatosingull:
helm installed into /usr/local/bin/helm
~/workdir/minikube-deploy ................................................................... 8s minikube kube 01:44:34
> helm version
version.BuildInfo{Version:"v3.14.2", GitCommit:"c309b6f0ff63856811846ce18f3bdc93d2b4d54b", GitTreeState:"clean", GoVersion:"go1.21.7"}
~/workdir/minikube-deploy ....................................................................... minikube kube 01:44:40
> helm repo add stable https://charts.helm.sh/stable
"stable" has been added to your repositories
以下は こちら を参考に実施
helmfile diff
install log
> helm plugin install https://github.com/databus23/helm-diff
Downloading https://github.com/databus23/helm-diff/releases/latest/download/helm-diff-linux-amd64.tgz
Preparing to install into /home/minatosingull/.local/share/helm/plugins/helm-diff
helm-diff installed into /home/minatosingull/.local/share/helm/plugins/helm-diff/helm-diff
The Helm Diff Plugin
* Shows a diff explaining what a helm upgrade would change:
This fetches the currently deployed version of a release
and compares it to a local chart plus values. This can be
used visualize what changes a helm upgrade will perform.
* Shows a diff explaining what had changed between two revisions:
This fetches previously deployed versions of a release
and compares them. This can be used visualize what changes
were made during revision change.
* Shows a diff explaining what a helm rollback would change:
This fetches the currently deployed version of a release
and compares it to the previously deployed version of the release, that you
want to rollback. This can be used visualize what changes a
helm rollback will perform.
Usage:
diff [flags]
diff [command]
Available Commands:
completion Generate the autocompletion script for the specified shell
release Shows diff between release's manifests
revision Shows diff between revision's manifests
rollback Show a diff explaining what a helm rollback could perform
upgrade Show a diff explaining what a helm upgrade would change.
version Show version of the helm diff plugin
Flags:
--allow-unreleased enables diffing of releases that are not yet deployed via Helm
-a, --api-versions stringArray Kubernetes api versions used for Capabilities.APIVersions
--color color output. You can control the value for this flag via HELM_DIFF_COLOR=[true|false]. If both --no-color and --color are unspecified, coloring enabled only when the stdout is a term and TERM is not "dumb"
-C, --context int output NUM lines of context around changes (default -1)
--detailed-exitcode return a non-zero exit code when there are changes
--devel use development versions, too. Equivalent to version '>0.0.0-0'. If --version is set, this is ignored.
--disable-openapi-validation disables rendered templates validation against the Kubernetes OpenAPI Schema
--disable-validation disables rendered templates validation against the Kubernetes cluster you are currently pointing to. This is the same validation performed on an install
--dry-run string[="client"] --dry-run, --dry-run=client, or --dry-run=true disables cluster access and show diff as if it was install. Implies --install, --reset-values, and --disable-validation. --dry-run=server enables the cluster access with helm-get and the lookup template function.
--enable-dns enable DNS lookups when rendering templates
-D, --find-renames float32 Enable rename detection if set to any value greater than 0. If specified, the value denotes the maximum fraction of changed content as lines added + removed compared to total lines in a diff for considering it a rename. Only objects of the same Kind are attempted to be matched
-h, --help help for diff
--include-tests enable the diffing of the helm test hooks
--insecure-skip-tls-verify skip tls certificate checks for the chart download
--install enables diffing of releases that are not yet deployed via Helm (equivalent to --allow-unreleased, added to match "helm upgrade --install" command
--kube-version string Kubernetes version used for Capabilities.KubeVersion
--kubeconfig string This flag is ignored, to allow passing of this top level flag to helm
--no-color remove colors from the output. If both --no-color and --color are unspecified, coloring enabled only when the stdout is a term and TERM is not "dumb"
--no-hooks disable diffing of hooks
--normalize-manifests normalize manifests before running diff to exclude style differences from the output
--output string Possible values: diff, simple, template, dyff. When set to "template", use the env var HELM_DIFF_TPL to specify the template. (default "diff")
--post-renderer string the path to an executable to be used for post rendering. If it exists in $PATH, the binary will be used, otherwise it will try to look for the executable at the given path
--post-renderer-args stringArray an argument to the post-renderer (can specify multiple)
--repo string specify the chart repository url to locate the requested chart
--reset-values reset the values to the ones built into the chart and merge in any new values
--reuse-values reuse the last release's values and merge in any new values. If '--reset-values' is specified, this is ignored
--set stringArray set values on the command line (can specify multiple or separate values with commas: key1=val1,key2=val2)
--set-file stringArray set values from respective files specified via the command line (can specify multiple or separate values with commas: key1=path1,key2=path2)
--set-json stringArray set JSON values on the command line (can specify multiple or separate values with commas: key1=jsonval1,key2=jsonval2)
--set-string stringArray set STRING values on the command line (can specify multiple or separate values with commas: key1=val1,key2=val2)
--show-secrets do not redact secret values in the output
--strip-trailing-cr strip trailing carriage return on input
--suppress stringArray allows suppression of the kinds listed in the diff output (can specify multiple, like '--suppress Deployment --suppress Service')
--suppress-output-line-regex stringArray a regex to suppress diff output lines that match
-q, --suppress-secrets suppress secrets in the output
--three-way-merge use three-way-merge to compute patch and generate diff output
-f, --values valueFiles specify values in a YAML file (can specify multiple) (default [])
--version string specify the exact chart version to use. If this is not specified, the latest version is used
Additional help topics:
diff
Use "diff [command] --help" for more information about a command.
Installed plugin: diff
helmfile
install log
> wget https://github.com/roboll/helmfile/releases/download/v0.144.0/helmfile_linux_amd64
sudo mv helmfile_linux_amd64 /usr/local/bin/helmfile
chmod +x /usr/local/bin/helmfile
--2024-03-09 02:10:39-- https://github.com/roboll/helmfile/releases/download/v0.144.0/helmfile_linux_amd64
Resolving github.com (github.com)... 20.27.177.113
Connecting to github.com (github.com)|20.27.177.113|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/74499101/51afb07f-4998-43c2-9f51-b5375ff292b2?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAVCODYLSA53PQK4ZA%2F20240308%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240308T171040Z&X-Amz-Expires=300&X-Amz-Signature=394c2d563b628799f7dc9423d5eb54711072a7db201a01fd2c494116918956e5&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=74499101&response-content-disposition=attachment%3B%20filename%3Dhelmfile_linux_amd64&response-content-type=application%2Foctet-stream [following]
--2024-03-09 02:10:39-- https://objects.githubusercontent.com/github-production-release-asset-2e65be/74499101/51afb07f-4998-43c2-9f51-b5375ff292b2?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAVCODYLSA53PQK4ZA%2F20240308%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240308T171040Z&X-Amz-Expires=300&X-Amz-Signature=394c2d563b628799f7dc9423d5eb54711072a7db201a01fd2c494116918956e5&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=74499101&response-content-disposition=attachment%3B%20filename%3Dhelmfile_linux_amd64&response-content-type=application%2Foctet-stream
Resolving objects.githubusercontent.com (objects.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ...
Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.108.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 46608813 (44M) [application/octet-stream]
Saving to: ‘helmfile_linux_amd64’
helmfile_linux_amd64 100%[====================================================>] 44.45M 11.0MB/s in 4.0s
2024-03-09 02:10:44 (11.0 MB/s) - ‘helmfile_linux_amd64’ saved [46608813/46608813]
公式の Getting Started を試しに動かしてみる (prometheus デプロイ)
repositories:
- name: prometheus-community
url: https://prometheus-community.github.io/helm-charts
releases:
- name: prom-norbac-ubuntu
namespace: prometheus
chart: prometheus-community/prometheus
set:
- name: rbac.create
value: false
helmfile appply
> helmfile apply
Adding repo prometheus-community https://prometheus-community.github.io/helm-charts
"prometheus-community" has been added to your repositories
Comparing release=prom-norbac-ubuntu, chart=prometheus-community/prometheus
********************
Release was not present in Helm. Diff will show entire contents as new.
********************
prometheus, prom-norbac-ubuntu-alertmanager, ConfigMap (v1) has been added:
-
+ # Source: prometheus/charts/alertmanager/templates/configmap.yaml
+ apiVersion: v1
+ kind: ConfigMap
+ metadata:
+ name: prom-norbac-ubuntu-alertmanager
+ labels:
+ helm.sh/chart: alertmanager-1.9.0
+ app.kubernetes.io/name: alertmanager
+ app.kubernetes.io/instance: prom-norbac-ubuntu
+ app.kubernetes.io/version: "v0.27.0"
+ app.kubernetes.io/managed-by: Helm
+ namespace: prometheus
+ data:
+ alertmanager.yml: |
+ global: {}
+ receivers:
+ - name: default-receiver
+ route:
+ group_interval: 5m
+ group_wait: 10s
+ receiver: default-receiver
+ repeat_interval: 3h
+ templates:
+ - /etc/alertmanager/*.tmpl
prometheus, prom-norbac-ubuntu-alertmanager, Service (v1) has been added:
... ~~ 色々差分が表示される.あまりにも長いので省略 ~~
Upgrading release=prom-norbac-ubuntu, chart=prometheus-community/prometheus
Release "prom-norbac-ubuntu" does not exist. Installing it now.
NAME: prom-norbac-ubuntu
LAST DEPLOYED: Sat Mar 9 02:16:01 2024
NAMESPACE: prometheus
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
The Prometheus server can be accessed via port 80 on the following DNS name from within your cluster:
prom-norbac-ubuntu-prometheus-server.prometheus.svc.cluster.local
Get the Prometheus server URL by running these commands in the same shell:
export POD_NAME=$(kubectl get pods --namespace prometheus -l "app.kubernetes.io/name=prometheus,app.kubernetes.io/instance=prom-norbac-ubuntu" -o jsonpath="{.items[0].metadata.name}")
kubectl --namespace prometheus port-forward $POD_NAME 9090
The Prometheus alertmanager can be accessed via port 9093 on the following DNS name from within your cluster:
prom-norbac-ubuntu-alertmanager.prometheus.svc.cluster.local
Get the Alertmanager URL by running these commands in the same shell:
export POD_NAME=$(kubectl get pods --namespace prometheus -l "app.kubernetes.io/name=alertmanager,app.kubernetes.io/instance=prom-norbac-ubuntu" -o jsonpath="{.items[0].metadata.name}")
kubectl --namespace prometheus port-forward $POD_NAME 9093
#################################################################################
###### WARNING: Pod Security Policy has been disabled by default since #####
###### it deprecated after k8s 1.25+. use #####
###### (index .Values "prometheus-node-exporter" "rbac" #####
###### . "pspEnabled") with (index .Values #####
###### "prometheus-node-exporter" "rbac" "pspAnnotations") #####
###### in case you still need it. #####
#################################################################################
The Prometheus PushGateway can be accessed via port 9091 on the following DNS name from within your cluster:
prom-norbac-ubuntu-prometheus-pushgateway.prometheus.svc.cluster.local
Get the PushGateway URL by running these commands in the same shell:
export POD_NAME=$(kubectl get pods --namespace prometheus -l "app=prometheus-pushgateway,component=pushgateway" -o jsonpath="{.items[0].metadata.name}")
kubectl --namespace prometheus port-forward $POD_NAME 9091
For more information on running Prometheus, visit:
https://prometheus.io/
Listing releases matching ^prom-norbac-ubuntu$
prom-norbac-ubuntu prometheus 1 2024-03-09 02:16:01.343767992 +0900 JST deployed prometheus-25.16.0 v2.50.1
UPDATED RELEASES:
NAME CHART VERSION
prom-norbac-ubuntu prometheus-community/prometheus 25.16.0
※namespace=prometheus で apply されるので注意
> kc get pod -n prometheus
NAME READY STATUS RESTARTS AGE
prom-norbac-ubuntu-alertmanager-0 1/1 Running 0 3m36s
prom-norbac-ubuntu-kube-state-metrics-68668c5849-xqwk4 1/1 Running 0 3m36s
prom-norbac-ubuntu-prometheus-node-exporter-m72d6 1/1 Running 0 3m36s
prom-norbac-ubuntu-prometheus-pushgateway-664559586-45q4s 1/1 Running 0 3m36s
prom-norbac-ubuntu-prometheus-server-6f6dfdc6cb-rl4l4 2/2 Running 0 3m36s
後は、 helmfile diff
デプロイしたものを削除して、 diff を見てみる
helmfile diff
> helmfile destroy
Adding repo prometheus-community https://prometheus-community.github.io/helm-charts
"prometheus-community" has been added to your repositories
Listing releases matching ^prom-norbac-ubuntu$
prom-norbac-ubuntu prometheus 1 2024-03-09 02:16:01.343767992 +0900 JST deployed prometheus-25.16.0 v2.50.1
Deleting prom-norbac-ubuntu
release "prom-norbac-ubuntu" uninstalled
DELETED RELEASES:
NAME
prom-norbac-ubuntu
> helmfile diff
Adding repo prometheus-community https://prometheus-community.github.io/helm-charts
"prometheus-community" has been added to your repositories
Comparing release=prom-norbac-ubuntu, chart=prometheus-community/prometheus
********************
Release was not present in Helm. Diff will show entire contents as new.
********************
prometheus, prom-norbac-ubuntu-alertmanager, ConfigMap (v1) has been added:
-
+ # Source: prometheus/charts/alertmanager/templates/configmap.yaml
+ apiVersion: v1
+ kind: ConfigMap
+ metadata:
+ name: prom-norbac-ubuntu-alertmanager
+ labels:
+ helm.sh/chart: alertmanager-1.9.0
+ app.kubernetes.io/name: alertmanager
+ app.kubernetes.io/instance: prom-norbac-ubuntu
+ app.kubernetes.io/version: "v0.27.0"
+ app.kubernetes.io/managed-by: Helm
+ namespace: prometheus
+ data:
+ alertmanager.yml: |
+ global: {}
+ receivers:
+ - name: default-receiver
+ route:
+ group_interval: 5m
+ group_wait: 10s
+ receiver: default-receiver
... ~~ 先ほどのインストール時と同様の差分 ~~
無事動作確認できました 💪
あとがき
実際には、クラウドの k8s クラスターにアクセスしたり image pull の設定とかあるかと思いますがひとまず local で学習用環境を作るのであれば割と手軽にできました。こんな短時間で手軽に準備できるとは... k8s,helm 関係者様様ですね。
また、後からつまらない「おま環」問題で詰まってしまうことになるので、なるべくイレギュラーな環境設定はしないようにした方が良いですね。
今回で言うと「毎回 sudo docker するの面倒だから alias 貼っとくか...」ってしてたのが仇になった点です。(まぁ、恐らく当時の自分としては効率化を図った上での対処なのだとは思いますが...)
Discussion