kube-system の中身ってこうなってるんだなって話。
よくわからないんだけど ConfigMap が取れてないことがわかったので後半に追加。
$ kubectl get all -n kube-system
NAME READY STATUS RESTARTS AGE
pod/azure-ip-masq-agent-68j88 1/1 Running 0 7d2h
pod/azure-ip-masq-agent-dwjtg 1/1 Running 0 3m43s
pod/azure-ip-masq-agent-t4ddc 1/1 Running 0 7d2h
# daemonSet なので 3 つある
pod/coredns-69c47794-6xnlq 1/1 Running 0 12h
pod/coredns-69c47794-cgn9k 1/1 Running 0 7d2h
# coredns-autoscaler によって設定されているので現状では 2 pods
pod/coredns-autoscaler-5f85dc856b-vbhbb 1/1 Running 0 11d
# coredns の replicas を扱ってる
pod/csi-azuredisk-node-74qr5 3/3 Running 0 7d2h
pod/csi-azuredisk-node-f96pj 3/3 Running 0 3m43s
pod/csi-azuredisk-node-lvc76 3/3 Running 0 7d2h
# Azure Disk を PV として扱うため、たぶん
pod/csi-azurefile-node-2sxj9 3/3 Running 0 3m43s
pod/csi-azurefile-node-gpwv5 3/3 Running 0 10d
pod/csi-azurefile-node-nml4l 3/3 Running 0 10d
# 同じく Azure Files を PV として扱うため
pod/konnectivity-agent-586fcb7848-bzz4s 1/1 Running 0 11d
pod/konnectivity-agent-586fcb7848-d9sss 1/1 Running 0 11d
# Kuberenetes の Konnectivity という機能を使って AKS Control-plane <-> Nodes の間で VPN を張っているらしい。cf.) https://jpaztech.github.io/blog/containers/aks-control-plane-node-communication/
pod/kube-proxy-jct8w 1/1 Running 0 5m25s
pod/kube-proxy-pjztn 1/1 Running 0 5m13s
pod/kube-proxy-wf7qw 1/1 Running 0 3m43s
# 内部的な proxy だった気がする
pod/metrics-server-774f99dbf4-6v9b8 1/1 Running 1 11d
# たぶん Azure Monitor とつながってる
reverse proxy とか load balancer の機能を持っている Service たち。
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kube-dns ClusterIP 10.0.0.10 <none> 53/UDP,53/TCP 11d
# 内部的な DNS サーバ
service/metrics-server ClusterIP 10.0.131.223 <none> 443/TCP 11d
# たぶんここにメトリックを送ると Azure Monitor の metric として出てくるんだと思う
1 node に 1 つおかれる DaemonSet たち。
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
daemonset.apps/azure-ip-masq-agent 3 3 3 3 3 <none> 11d
daemonset.apps/csi-azuredisk-node 3 3 3 3 3 <none> 11d
daemonset.apps/csi-azuredisk-node-win 0 0 0 0 0 <none> 11d
# たぶん Azure Disk を PV として使うためのもの
daemonset.apps/csi-azurefile-node 3 3 3 3 3 <none> 11d
daemonset.apps/csi-azurefile-node-win 0 0 0 0 0 <none> 11d
# 同じく Azure Files のやつだと思う
daemonset.apps/kube-proxy 3 3 3 3 3 <none> 11d
# 内部的な proxy をやってる
pod の上位概念らしい Deployment。
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/coredns 2/2 2 2 11d
# 内部 DNS サーバ
deployment.apps/coredns-autoscaler 1/1 1 1 11d
# coredns pod の scale を扱ってる。node の数とか、合計 core 数を監視してる
deployment.apps/konnectivity-agent 2/2 2 2 11d
deployment.apps/metrics-server 1/1 1 1 11d
# たぶん Azure Monitor とつながってる
pod の上位概念の一つの ReplicaSet。直接デプロイしたわけじゃないから触らないほうがいいはず、たぶん
NAME DESIRED CURRENT READY AGE
replicaset.apps/coredns-69c47794 2 2 2 7d2h
# coredns のそれ
replicaset.apps/coredns-autoscaler-5f85dc856b 1 1 1 11d
# coredns の autoscaler、replicaSet である意味はあんまりわかってない
replicaset.apps/konnectivity-agent-586fcb7848 2 2 2 11d
replicaset.apps/metrics-server-774f99dbf4 1 1 1 11d
# これもなんで replicaSet なのかわかってない
追加で ConfigMap も。
$ kubectl get configmaps -n kube-system
NAME DATA AGE
azure-ip-masq-agent-config 1 11d
coredns 1 11d
coredns-autoscaler 1 11d
coredns-custom 0 11d
extension-apiserver-authentication 6 11d
kube-root-ca.crt 1 11d
overlay-upgrade-data 4 11d
例えば coredns だとこんな感じ。
ゾーン情報が書いてあある感じっぽい。
AKS private cluter を使ってるので一部置き換えてある。
data: が 1 つしかない。
$ kubectl get configmaps coredns -n kube-system -o yaml
apiVersion: v1
data:
Corefile: |
private-cl-aks-xxxxxx-xxxxxxxx.xxxxxxxx-xxxx-xxxx-xxxx-52cd5b9677e5.privatelink.southeastasia.azmk8s.io {
hosts privatelink.hosts private-cl-aks-xxxxxx-xxxxxxxx.xxxxxxxx-xxxx-xxxx-xxxx-52cd5b9677e5.privatelink.southeastasia.azmk8s.io {
172.16.10.4 private-cl-aks-xxxxxx-xxxxxxxx.xxxxxxxx-xxxx-xxxx-xxxx-52cd5b9677e5.privatelink.southeastasia.azmk8s.io
fallthrough
}
}
.:53 {
errors
ready
health
kubernetes cluster.local in-addr.arpa ip6.arpa {
pods insecure
fallthrough in-addr.arpa ip6.arpa
}
prometheus :9153
forward . /etc/resolv.conf
cache 30
loop
reload
loadbalance
import custom/*.override
}
import custom/*.server
kind: ConfigMap
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"v1","data":{"Corefile":"private-cl-aks-xxxxxx-xxxxxxxx.xxxxxxxx-xxxx-xxxx-xxxx-52cd5b9677e5.privatelink.southeastasia.azmk8s.io {\n hosts privatelink.hosts private-cl-aks-xxxxxx-xxxxxxxx.xxxxxxxx-xxxx-xxxx-xxxx-52cd5b9677e5.privatelink.southeastasia.azmk8s.io {\n 172.16.10.4 private-cl-aks-xxxxxx-xxxxxxxx.xxxxxxxx-xxxx-xxxx-xxxx-52cd5b9677e5.privatelink.southeastasia.azmk8s.io\n fallthrough\n }\n}\n.:53 {\n errors\n ready\n health\n kubernetes cluster.local in-addr.arpa ip6.arpa {\n pods insecure\n fallthrough in-addr.arpa ip6.arpa\n }\n prometheus :9153\n forward . /etc/resolv.conf\n cache 30\n loop\n reload\n loadbalance\n import custom/*.override\n}\nimport custom/*.server\n"},"kind":"ConfigMap","metadata":{"annotations":{},"labels":{"addonmanager.kubernetes.io/mode":"Reconcile","k8s-app":"kube-dns","kubernetes.io/cluster-service":"true"},"name":"coredns","namespace":"kube-system"}}
creationTimestamp: "2022-04-17T06:14:44Z"
labels:
addonmanager.kubernetes.io/mode: Reconcile
k8s-app: kube-dns
kubernetes.io/cluster-service: "true"
name: coredns
namespace: kube-system
resourceVersion: "2073"
uid: xxxxxxxx-xxxx-xxxx-xxxx-ab14331457c9
overlay-upgrade-data だと data: が 4 つある。
全体は短いけど。
$ kubectl get configmaps overlay-upgrade-data -n kube-system -o yaml
apiVersion: v1
data:
ccpEtag: xxxxxxxx-xxxx-xxxx-xxxx-ff6bee335c57
isSLA: "false"
lastUpgradeTime: "2022-04-29T02:50:05.004653003Z"
overlaymgrversionGoal: 'Version: v20220417.220420.2 - Branch: refs/heads/official/v20220417
- git SHA: 47716d522 - Build date / time: 2022-04-20T12:20:39Z'
kind: ConfigMap
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"v1","data":{"ccpEtag":"xxxxxxxx-xxxx-xxxx-xxxx-ff6bee335c57","isSLA":"false","lastUpgradeTime":"2022-04-29T02:50:05.004653003Z","overlaymgrversionGoal":"Version: v20220417.220420.2 - Branch: refs/heads/official/v20220417 - git SHA: 47716d522 - Build date / time: 2022-04-20T12:20:39Z"},"kind":"ConfigMap","metadata":{"annotations":{},"labels":{"addonmanager.kubernetes.io/mode":"Reconcile","k8s-app":"overlay-upgrade"},"name":"overlay-upgrade-data","namespace":"kube-system"}}
creationTimestamp: "2022-04-17T06:14:47Z"
labels:
addonmanager.kubernetes.io/mode: Reconcile
k8s-app: overlay-upgrade
name: overlay-upgrade-data
namespace: kube-system
resourceVersion: "3385444"
uid: xxxxxxxx-xxxx-xxxx-xxxx-d50bad8a67c2
Discussion