✨
Vagrant＋VirtualboxのK8sクラスタにMetrics Server導入
自分のラップトップ上で構築したK8sクラスタにMetrics Serverを導入してみる。
kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml
Output:
serviceaccount/metrics-server created
clusterrole.rbac.authorization.k8s.io/system:aggregated-metrics-reader created
clusterrole.rbac.authorization.k8s.io/system:metrics-server created
rolebinding.rbac.authorization.k8s.io/metrics-server-auth-reader created
clusterrolebinding.rbac.authorization.k8s.io/metrics-server:system:auth-delegator created
clusterrolebinding.rbac.authorization.k8s.io/system:metrics-server created
service/metrics-server created
deployment.apps/metrics-server created
apiservice.apiregistration.k8s.io/v1beta1.metrics.k8s.io created
さっそくうまくいかない
kubectl get deployment metrics-server -n kube-system
NAME READY UP-TO-DATE AVAILABLE AGE
metrics-server 0/1 1 0 101s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 92s default-scheduler Successfully assigned kube-system/metrics-server-6f7dd4c4c4-wxsxq to node01
Normal Pulling 92s kubelet Pulling image "registry.k8s.io/metrics-server/metrics-server:v0.7.2"
Normal Pulled 87s kubelet Successfully pulled image "registry.k8s.io/metrics-server/metrics-server:v0.7.2" in 5.504s (5.504s including waiting). Image size: 19494617 bytes.
Normal Created 87s kubelet Created container: metrics-server
Normal Started 87s kubelet Started container metrics-server
Warning Unhealthy 8s (x7 over 64s) kubelet Readiness probe failed: HTTP probe failed with statuscode: 500
Metrics Serverのログを確認
kubectl logs -n kube-system deployment/metrics-server
ログ
E0209 13:17:28.886388 1 scraper.go:149] "Failed to scrape node" err="Get \"https://192.168.56.21:10250/metrics/resource\": tls: failed to verify certificate: x509: cannot validate certificate for 192.168.56.21 because it doesn't contain any IP SANs" node="node01"
E0209 13:17:28.898963 1 scraper.go:149] "Failed to scrape node" err="Get \"https://192.168.56.22:10250/metrics/resource\": tls: failed to verify certificate: x509: cannot validate certificate for 192.168.56.22 because it doesn't contain any IP SANs" node="node02"
E0209 13:17:28.900577 1 scraper.go:149] "Failed to scrape node" err="Get \"https://192.168.56.11:10250/metrics/resource\": tls: failed to verify certificate: x509: cannot validate certificate for 192.168.56.11 because it doesn't contain any IP SANs" node="controlplane"
I0209 13:17:31.391720 1 server.go:191] "Failed probe" probe="metric-storage-ready" err="no metrics to serve"
I0209 13:17:41.390757 1 server.go:191] "Failed probe" probe="metric-storage-ready" err="no metrics to serve"
E0209 13:17:43.887482 1 scraper.go:149] "Failed to scrape node" err="Get \"https://192.168.56.22:10250/metrics/resource\": tls: failed to verify certificate: x509: cannot validate certificate for 192.168.56.22 because it doesn't contain any IP SANs" node="node02"
E0209 13:17:43.891846 1 scraper.go:149] "Failed to scrape node" err="Get \"https://192.168.56.11:10250/metrics/resource\": tls: failed to verify certificate: x509: cannot validate certificate for 192.168.56.11 because it doesn't contain any IP SANs" node="controlplane"
E0209 13:17:43.893459 1 scraper.go:149] "Failed to scrape node" err="Get \"https://192.168.56.21:10250/metrics/resource\": tls: failed to verify certificate: x509: cannot validate certificate for 192.168.56.21 because it doesn't contain any IP SANs" node="node01"
I0209 13:17:51.392105 1 server.go:191] "Failed probe" probe="metric-storage-ready" err="no metrics to serve"
E0209 13:17:58.880873 1 scraper.go:149] "Failed to scrape node" err="Get \"https://192.168.56.21:10250/metrics/resource\": tls: failed to verify certificate: x509: cannot validate certificate for 192.168.56.21 because it doesn't contain any IP SANs" node="node01"
E0209 13:17:58.888557 1 scraper.go:149] "Failed to scrape node" err="Get \"https://192.168.56.22:10250/metrics/resource\": tls: failed to verify certificate: x509: cannot validate certificate for 192.168.56.22 because it doesn't contain any IP SANs" node="node02"
E0209 13:17:58.893290 1 scraper.go:149] "Failed to scrape node" err="Get \"https://192.168.56.11:10250/metrics/resource\": tls: failed to verify certificate: x509: cannot validate certificate for 192.168.56.11 because it doesn't contain any IP SANs" node="controlplane"
I0209 13:24:47.958552 1 server.go:191] "Failed probe" probe="metric-storage-ready" err="no metrics to serve"
TLS verificationが邪魔をしているっぽい。以下のコマンドで治す。
kubectl patch deployment metrics-server -n kube-system --type='json' -p='[
{"op": "add", "path": "/spec/template/spec/containers/0/args/-", "value": "--kubelet-insecure-tls"},
{"op": "add", "path": "/spec/template/spec/containers/0/args/-", "value": "--kubelet-preferred-address-types=InternalIP"}
]'
調べてみたらここでも同じことが書かれてあった。
deploymentを再度スタートする。
kubectl rollout restart deployment metrics-server -n kube-system
これで走るようになった。
kubectl get deployment metrics-server -n kube-system
NAME READY UP-TO-DATE AVAILABLE AGE
metrics-server 1/1 1 1 24m
そして以下のコマンドも走るようになった。
kubectl top nodes
NAME CPU(cores) CPU(%) MEMORY(bytes) MEMORY(%)
controlplane 189m 4% 773Mi 13%
node01 34m 1% 320Mi 17%
node02 34m 1% 402Mi 21%
kubectl top pods
NAME CPU(cores) MEMORY(bytes)
nginx 0m 2Mi
web-deploy-66fbbff65d-cxcjc 0m 3Mi
web-deploy-66fbbff65d-f6bgd 0m 8Mi
web-deploy-66fbbff65d-ghklh 0m 8Mi
Discussion