Closed35
Linkerd を試す with AWS Load Balancer Controller
junya0530以下のツールがインストール済であること
- CLI
- linkerd CLI
- smallstep/step
- Controller
- AWS Load Balancer Controller
- ExternalDNS
junya0530事前チェック
$ linkerd version
Client version: edge-25.7.5
Server version: unavailable
$ linkerd check --pre
kubernetes-api
--------------
√ can initialize the client
√ can query the Kubernetes API
kubernetes-version
------------------
√ is running the minimum Kubernetes API version
pre-kubernetes-setup
--------------------
√ control plane namespace does not already exist
√ can create non-namespaced resources
√ can create ServiceAccounts
√ can create Services
√ can create Deployments
√ can create CronJobs
√ can create ConfigMaps
√ can create Secrets
√ can read Secrets
√ can read extension-apiserver-authentication configmap
√ no clock skew detected
linkerd-version
---------------
√ can determine the latest version
√ cli is up-to-date
Status check results are √
mTLSに必要な証明書作成
$ step certificate create root.linkerd.cluster.local ca.crt ca.key \
--profile root-ca --no-password --insecure
Your certificate has been saved in ca.crt.
Your private key has been saved in ca.key.
$ ll
total 16
-rw-------@ 1 taniaijunya staff 599B 7 25 21:54 ca.crt
-rw-------@ 1 taniaijunya staff 227B 7 25 21:54 ca.key
$ step certificate create identity.linkerd.cluster.local issuer.crt issuer.key \
--profile intermediate-ca --not-after 8760h --no-password --insecure \
--ca ca.crt --ca-key ca.key
Your certificate has been saved in issuer.crt.
Your private key has been saved in issuer.key.
$ ll
total 32
-rw-------@ 1 taniaijunya staff 599B 7 25 21:54 ca.crt
-rw-------@ 1 taniaijunya staff 227B 7 25 21:54 ca.key
-rw-------@ 1 taniaijunya staff 652B 7 25 21:55 issuer.crt
-rw-------@ 1 taniaijunya staff 227B 7 25 21:55 issuer.key
Linkerdのインストール
LinkerdはGateway APIのxRoute（HTTPRrouteなど）を利用するので、一緒にGateway APIのCRDもインストールする。
$ helm install linkerd-crds linkerd-edge/linkerd-crds \
-n linkerd --create-namespace
NAME: linkerd-crds
LAST DEPLOYED: Mon Jul 28 10:37:14 2025
NAMESPACE: linkerd
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
The linkerd-crds chart was successfully installed 🎉
To complete the linkerd core installation, please now proceed to install the
linkerd-control-plane chart in the linkerd namespace.
Looking for more? Visit https://linkerd.io/2/getting-started/
$ kubectl api-resources | grep linkerd
serviceprofiles sp linkerd.io/v1alpha2 true ServiceProfile
authorizationpolicies authzpolicy policy.linkerd.io/v1alpha1 true AuthorizationPolicy
egressnetworks policy.linkerd.io/v1alpha1 true EgressNetwork
httplocalratelimitpolicies policy.linkerd.io/v1alpha1 true HTTPLocalRateLimitPolicy
httproutes policy.linkerd.io/v1beta3 true HTTPRoute
meshtlsauthentications meshtlsauthn policy.linkerd.io/v1alpha1 true MeshTLSAuthentication
networkauthentications netauthn,networkauthn policy.linkerd.io/v1alpha1 true NetworkAuthentication
serverauthorizations saz,serverauthz,srvauthz policy.linkerd.io/v1beta1 true ServerAuthorization
servers srv policy.linkerd.io/v1beta3 true Server
externalworkloads workload.linkerd.io/v1beta1 true ExternalWorkload
Linkerdのインストール
$ cd step-crt
$ ll ~/step-crt
total 32
-rw-------@ 1 taniaijunya staff 599B 7 25 21:54 ca.crt
-rw-------@ 1 taniaijunya staff 227B 7 25 21:54 ca.key
-rw-------@ 1 taniaijunya staff 652B 7 25 21:55 issuer.crt
-rw-------@ 1 taniaijunya staff 227B 7 25 21:55 issuer.key
$ helm install linkerd-control-plane \
-n linkerd \
--set-file identityTrustAnchorsPEM=ca.crt \
--set-file identity.issuer.tls.crtPEM=issuer.crt \
--set-file identity.issuer.tls.keyPEM=issuer.key \
linkerd-edge/linkerd-control-plane
NAME: linkerd-control-plane
LAST DEPLOYED: Mon Jul 28 10:39:32 2025
NAMESPACE: linkerd
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
The Linkerd control plane was successfully installed 🎉
To help you manage your Linkerd service mesh you can install the Linkerd CLI by running:
curl -sL https://run.linkerd.io/install | sh
Alternatively, you can download the CLI directly via the Linkerd releases page:
https://github.com/linkerd/linkerd2/releases/
To make sure everything works as expected, run the following:
linkerd check
The viz extension can be installed by running:
helm install linkerd-viz linkerd/linkerd-viz
Looking for more? Visit https://linkerd.io/2/getting-started/
viz拡張機能もインストールする。
linkerd viz install | kubectl apply -f -
namespace/linkerd-viz created
clusterrole.rbac.authorization.k8s.io/linkerd-linkerd-viz-metrics-api created
clusterrolebinding.rbac.authorization.k8s.io/linkerd-linkerd-viz-metrics-api created
serviceaccount/metrics-api created
clusterrole.rbac.authorization.k8s.io/linkerd-linkerd-viz-prometheus created
clusterrolebinding.rbac.authorization.k8s.io/linkerd-linkerd-viz-prometheus created
serviceaccount/prometheus created
clusterrole.rbac.authorization.k8s.io/linkerd-linkerd-viz-tap created
clusterrole.rbac.authorization.k8s.io/linkerd-linkerd-viz-tap-admin created
clusterrolebinding.rbac.authorization.k8s.io/linkerd-linkerd-viz-tap created
clusterrolebinding.rbac.authorization.k8s.io/linkerd-linkerd-viz-tap-auth-delegator created
serviceaccount/tap created
rolebinding.rbac.authorization.k8s.io/linkerd-linkerd-viz-tap-auth-reader created
secret/tap-k8s-tls created
apiservice.apiregistration.k8s.io/v1alpha1.tap.linkerd.io created
role.rbac.authorization.k8s.io/web created
rolebinding.rbac.authorization.k8s.io/web created
clusterrole.rbac.authorization.k8s.io/linkerd-linkerd-viz-web-check created
clusterrolebinding.rbac.authorization.k8s.io/linkerd-linkerd-viz-web-check created
clusterrolebinding.rbac.authorization.k8s.io/linkerd-linkerd-viz-web-admin created
clusterrole.rbac.authorization.k8s.io/linkerd-linkerd-viz-web-api created
clusterrolebinding.rbac.authorization.k8s.io/linkerd-linkerd-viz-web-api created
serviceaccount/web created
service/metrics-api created
deployment.apps/metrics-api created
server.policy.linkerd.io/metrics-api created
authorizationpolicy.policy.linkerd.io/metrics-api created
meshtlsauthentication.policy.linkerd.io/metrics-api-web created
networkauthentication.policy.linkerd.io/kubelet created
configmap/prometheus-config created
service/prometheus created
deployment.apps/prometheus created
server.policy.linkerd.io/prometheus-admin created
authorizationpolicy.policy.linkerd.io/prometheus-admin created
service/tap created
deployment.apps/tap created
server.policy.linkerd.io/tap-api created
authorizationpolicy.policy.linkerd.io/tap created
clusterrole.rbac.authorization.k8s.io/linkerd-tap-injector created
clusterrolebinding.rbac.authorization.k8s.io/linkerd-tap-injector created
serviceaccount/tap-injector created
secret/tap-injector-k8s-tls created
mutatingwebhookconfiguration.admissionregistration.k8s.io/linkerd-tap-injector-webhook-config created
service/tap-injector created
deployment.apps/tap-injector created
server.policy.linkerd.io/tap-injector-webhook created
authorizationpolicy.policy.linkerd.io/tap-injector created
networkauthentication.policy.linkerd.io/kube-api-server created
service/web created
deployment.apps/web created
serviceprofile.linkerd.io/metrics-api.linkerd-viz.svc.cluster.local created
serviceprofile.linkerd.io/prometheus.linkerd-viz.svc.cluster.local created
ここまでで事前の準備は完了。
次から機能を試していく。
junya0530Fault Injection
以下の公式ドキュメントに沿って作業する。
junya0530まずはサンプルアプリをインストールする。
$ kubectl -n booksapp apply -f https://run.linkerd.io/booksapp.yml
service/webapp created
serviceaccount/webapp created
deployment.apps/webapp created
service/authors created
serviceaccount/authors created
deployment.apps/authors created
service/books created
serviceaccount/books created
deployment.apps/books created
serviceaccount/traffic created
deployment.apps/traffic created
デフォルトでauthorsは50%のリクエストを拒否するようになっているので、設定用の環境変数を削除する。
$ kubectl -n booksapp patch deploy authors \
--type='json' \
-p='[{"op":"remove", "path":"/spec/template/spec/containers/0/env/2"}]'
次にIngressを作成する。
ingressClassNameはAWS Load Balancer Controllerのものを指定する。
$ kubectl apply -f - <<EOF
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: webapp
namespace: booksapp
annotations:
external-dns.alpha.kubernetes.io/hostname: "test.jnytnai.click"
alb.ingress.kubernetes.io/scheme: "internet-facing"
alb.ingress.kubernetes.io/target-type: "ip"
alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}]'
spec:
ingressClassName: alb
rules:
- host: "test.jnytnai.click"
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: webapp
port:
number: 7000
EOF
ingress.networking.k8s.io/webapp created
$ kubectl -n booksapp get ing
NAME CLASS HOSTS ADDRESS PORTS AGE
webapp alb test.jnytnai.click k8s-booksapp-webapp-64f1066be2-1846711455.ap-northeast-1.elb.amazonaws.com 80 7s
次にアプリケーションにLinkerdをProxyとして、SidecarをInjectする。
$ kubectl get -n booksapp deploy -o yaml | linkerd inject - | kubectl apply -f -
deployment "authors" injected
deployment "books" injected
deployment "traffic" injected
deployment "webapp" injected
deployment.apps/authors configured
deployment.apps/books configured
deployment.apps/traffic configured
deployment.apps/webapp configured
Injectすると、以下のようにMeshが構成されていることが確認できる。
$ linkerd viz -n booksapp stat deploy
NAME MESHED SUCCESS RPS LATENCY_P50 LATENCY_P95 LATENCY_P99 TCP_CONN
authors 1/1 100.00% 8.1rps 2ms 26ms 42ms 8
books 1/1 100.00% 9.7rps 4ms 74ms 95ms 9
traffic 1/1 100.00% 0.3rps 1ms 3ms 3ms 1
webapp 3/3 100.00% 9.6rps 15ms 81ms 96ms 10
Fault Injectionのためにエラーを発生させるPodをデプロイする。
$ cat << EOT | k apply -f -
apiVersion: v1
kind: ConfigMap
metadata:
name: error-injector
namespace: booksapp
data:
nginx.conf: |-
events {}
http {
server {
listen 8080;
location / {
return 500;
}
}
}
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: error-injector
namespace: booksapp
labels:
app: error-injector
spec:
selector:
matchLabels:
app: error-injector
replicas: 1
template:
metadata:
labels:
app: error-injector
spec:
containers:
- name: nginx
image: nginx:alpine
volumeMounts:
- name: nginx-config
mountPath: /etc/nginx/nginx.conf
subPath: nginx.conf
volumes:
- name: nginx-config
configMap:
name: error-injector
---
apiVersion: v1
kind: Service
metadata:
name: error-injector
namespace: booksapp
spec:
ports:
- name: service
port: 8080
selector:
app: error-injector
EOF
configmap/error-injector created
deployment.apps/error-injector created
service/error-injector created
これにもSidecarをInjectしてMeshに参加させる。
$ kubectl get -n booksapp deploy error-injector -o yaml | linkerd inject - | kubectl apply -f -
deployment "error-injector" injected
deployment.apps/error-injector configured
最後の準備として、HTTPRouteをデプロイして、10%の割合で上記Podに作成させるようにしている。
$ kubectl apply -f - <<EOF
apiVersion: policy.linkerd.io/v1beta2
kind: HTTPRoute
metadata:
name: error-split
namespace: booksapp
spec:
parentRefs:
- name: books
kind: Service
group: core
port: 7002
rules:
- backendRefs:
- name: books
port: 7002
weight: 90
- name: error-injector
port: 8080
weight: 10
EOF
httproute.policy.linkerd.io/error-split created
以下のように連続してリスエストを送ると、一定の割合でステータスコード500が返ってきているのが確認できる。
$ for i in `seq 20`; do curl --url "http://test.jnytnai.click" -o /dev/null -w '%{http_code}\n' -s; done
500
200
200
200
200
200
200
200
500
200
200
200
200
200
200
200
200
200
200
200
~ $
Dynamic Request Routing
以下の公式ドキュメントに沿って作業する。
これを利用することでプログレッシブデリバリ時に動的なルーティングが可能となる。
junya0530まずはサンプルアプリをインストールする。
このアプリはFrontendと２つのBackendで構成されており、Backendに/envでリスエストを送ると、それぞれ「A backend」と「B backend」を返すようにする。
$ helm upgrade --install --wait frontend \
--namespace test \
--set replicaCount=1 \
--set backend=http://backend-podinfo:9898/env \
podinfo/podinfo
Release "frontend" does not exist. Installing it now.
NAME: frontend
LAST DEPLOYED: Mon Jul 28 16:05:53 2025
NAMESPACE: test
STATUS: deployed
REVISION: 1
NOTES:
1. Get the application URL by running these commands:
echo "Visit http://127.0.0.1:8080 to use your application"
kubectl -n test port-forward deploy/frontend-podinfo 8080:9898
$ helm upgrade --install --wait backend-a \
--namespace test \
--set ui.message="A backend" \
podinfo/podinfo
Release "backend-a" does not exist. Installing it now.
NAME: backend-a
LAST DEPLOYED: Mon Jul 28 16:08:39 2025
NAMESPACE: test
STATUS: deployed
REVISION: 1
NOTES:
1. Get the application URL by running these commands:
echo "Visit http://127.0.0.1:8080 to use your application"
kubectl -n test port-forward deploy/backend-a-podinfo 8080:9898
$ helm upgrade --install --wait backend-b \
--namespace test \
--set ui.message="B backend" \
podinfo/podinfo
Release "backend-b" does not exist. Installing it now.
NAME: backend-b
LAST DEPLOYED: Mon Jul 28 16:08:52 2025
NAMESPACE: test
STATUS: deployed
REVISION: 1
NOTES:
1. Get the application URL by running these commands:
echo "Visit http://127.0.0.1:8080 to use your application"
kubectl -n test port-forward deploy/backend-b-podinfo 8080:9898
次にサンプルアプリにSidecarをInjectし、Meshを構成する。
$ kubectl get -n test deploy -o yaml | linkerd inject - | kubectl apply -f -
deployment "backend-a-podinfo" injected
deployment "backend-b-podinfo" injected
deployment "frontend-podinfo" injected
Warning: resource deployments/backend-a-podinfo is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
deployment.apps/backend-a-podinfo configured
Warning: resource deployments/backend-b-podinfo is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
deployment.apps/backend-b-podinfo configured
Warning: resource deployments/frontend-podinfo is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
deployment.apps/frontend-podinfo configured
$ linkerd viz -n test stat deploy
NAME MESHED SUCCESS RPS LATENCY_P50 LATENCY_P95 LATENCY_P99 TCP_CONN
backend-a-podinfo 1/1 100.00% 0.1rps 1ms 1ms 1ms 1
backend-b-podinfo 1/1 100.00% 0.0rps 1ms 1ms 1ms 1
frontend-podinfo 1/1 100.00% 0.2rps 1ms 1ms 1ms 2
最後にFrontendにアクセスするIngressを作成する。
cat << EOT | k apply -f -
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: front
namespace: test
annotations:
external-dns.alpha.kubernetes.io/hostname: "test.jnytnai.click"
alb.ingress.kubernetes.io/scheme: "internet-facing"
alb.ingress.kubernetes.io/target-type: "ip"
alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}]'
spec:
ingressClassName: alb
rules:
- host: "test.jnytnai.click"
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: frontend-podinfo
port:
number: 9898
EOT
ingress.networking.k8s.io/front create
HTTPRouteリソースを適用し、x-request-id: alternative ヘッダがあるの場合はB backendに流すようにルーティングする。
cat <<EOF | kubectl -n test apply -f -
apiVersion: policy.linkerd.io/v1beta2
kind: HTTPRoute
metadata:
name: backend-router
namespace: test
spec:
parentRefs:
- name: backend-a-podinfo
kind: Service
group: core
port: 9898
rules:
- matches:
- headers:
- name: "x-request-id"
value: "alternative"
backendRefs:
- name: "backend-b-podinfo"
port: 9898
- backendRefs:
- name: "backend-a-podinfo"
port: 9898
EOF
以下のように確認してみると、HTTPRouteで設定したようにルーティングすることを確認できる。
$ curl -sX POST test.jnytnai.click/echo | grep -o 'PODINFO_UI_MESSAGE=. backend'
PODINFO_UI_MESSAGE=A backend
$ curl -sX POST -H 'x-request-id: alternative' test.jnytnai.click/echo | grep -o 'PODINFO_UI_MESSAGE=. backend'
PODINFO_UI_MESSAGE=B backend
Circuit Breaking
以下の公式ドキュメントに沿って作業する。
junya0530まずは、サンプルアプリをインストールしつつ、SidecarをInjectする。
$ cat <<EOF | linkerd inject - | kubectl apply -f -
---
apiVersion: v1
kind: Namespace
metadata:
name: circuit-breaking-demo
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: good
namespace: circuit-breaking-demo
spec:
replicas: 1
selector:
matchLabels:
class: good
template:
metadata:
labels:
class: good
app: bb
spec:
containers:
- name: terminus
image: buoyantio/bb:v0.0.6
args:
- terminus
- "--h1-server-port=8080"
ports:
- containerPort: 8080
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: bad
namespace: circuit-breaking-demo
spec:
replicas: 1
selector:
matchLabels:
class: bad
template:
metadata:
labels:
class: bad
app: bb
spec:
containers:
- name: terminus
image: buoyantio/bb:v0.0.6
args:
- terminus
- "--h1-server-port=8080"
- "--percent-failure=100"
ports:
- containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
name: bb
namespace: circuit-breaking-demo
spec:
ports:
- name: http
port: 8080
targetPort: 8080
selector:
app: bb
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: slow-cooker
namespace: circuit-breaking-demo
spec:
replicas: 1
selector:
matchLabels:
app: slow-cooker
template:
metadata:
labels:
app: slow-cooker
spec:
containers:
- args:
- -c
- |
sleep 5 # wait for pods to start
/slow_cooker/slow_cooker --qps 10 http://bb:8080
command:
- /bin/sh
image: buoyantio/slow_cooker:1.3.0
name: slow-cooker
EOF <....
namespace "circuit-breaking-demo" annotated
deployment "good" injected
deployment "bad" injected
service "bb" skipped
deployment "slow-cooker" injected
namespace/circuit-breaking-demo created
deployment.apps/good created
deployment.apps/bad created
service/bb created
deployment.apps/slow-cooker created
早速Meshを確認すると、badの成功率が高い、かつRPSが一番高いので、slow-cookerからはほぼbadにアクセスされていることが確認できる。
$ linkerd viz -n circuit-breaking-demo stat deploy
NAME MESHED SUCCESS RPS LATENCY_P50 LATENCY_P95 LATENCY_P99 TCP_CONN
bad 1/1 4.33% 4.2rps 1ms 1ms 1ms 3
good 1/1 100.00% 2.3rps 1ms 1ms 1ms 3
slow-cooker 1/1 100.00% 0.2rps 1ms 2ms 2ms 1
さらにslow-cookerは成功率がやはり低い。
$ linkerd viz -n circuit-breaking-demo stat deploy/slow-cooker --to svc/bb
NAME MESHED SUCCESS RPS LATENCY_P50 LATENCY_P95 LATENCY_P99 TCP_CONN
slow-cooker 1/1 35.00% 10.0rps 1ms 1ms 2ms 2
サーキットブレイカーで通信を遮断したいので、アノテーションをサービスに設定する。
$ kubectl annotate -n circuit-breaking-demo svc/bb balancer.linkerd.io/failure-accrual=consecutive
service/bb annotated
ポリシーを確認すると、7回失敗すると通信を遮断するようになっていることがわかる。
$ linkerd diagnostics policy -n circuit-breaking-demo svc/bb 8080 -o json | jq '.protocol.Kind.Detect.http1.failure_accrual'
{
"Kind": {
"ConsecutiveFailures": {
"max_failures": 7,
"backoff": {
"min_backoff": {
"seconds": 1
},
"max_backoff": {
"seconds": 60
},
"jitter_ratio": 0.5
}
}
}
}
このポリシーは以下の通りの説明がある。
In this failure accrual policy, an endpoint is marked as failing after a configurable number of failures occur consecutively (i.e., without any successes). For example, if the maximum number of failures is 7, the endpoint is made unavailable once 7 failures occur in a row with no successes.
さらに追加のパラメータも設定でき、ポリシーの挙動を変更することも可能。
Set this annotation on a Service to enable meshed clients to use circuit breaking when sending traffic to that Service:
- balancer.linkerd.io/failure-accrual: Selects the failure accrual policy used when communicating with this Service. If this is not present, no failure accrual is performed. Currently, the only supported value for this annotation is "consecutive", to perform consecutive failures failure accrual.
When the failure accrual mode is "consecutive", the following annotations configure parameters for the consecutive-failures failure accrual policy:
- balancer.linkerd.io/failure-accrual-consecutive-max-failures: Sets the number of consecutive failures which must occur before an endpoint is made unavailable. Must be an integer. If this annotation is not present, the default value is 7.
- balancer.linkerd.io/failure-accrual-consecutive-min-penalty: Sets the minumum penalty duration for which an endpoint will be marked as unavailable after max-failures consecutive failures occur. After this period of time elapses, the endpoint will be probed. This duration must be non-zero, and may not be greater than the max-penalty duration. If this annotation is not present, the default value is one second (1s).
- balancer.linkerd.io/failure-accrual-consecutive-max-penalty: Sets the maximum penalty duration for which an endpoint will be marked as unavailable after max-failures consecutive failures occur. This is an upper bound on the duration between probe requests. This duration must be non-zero, and must be greater than the min-penalty duration. If this annotation is not present, the default value is one minute (1m).
- balancer.linkerd.io/failure-accrual-consecutive-jitter-ratio: Sets the jitter ratio used for probation backoffs. This is a floating-point number, and must be between 0.0 and 100.0. If this annotation is not present, the default value is 0.5.
junya0530goodのRPSは高くなっており、badは低くなっているので、slow-cookerからbadへのトラフィックがほぼ遮断されていることがわかる。
$ linkerd viz -n circuit-breaking-demo stat deploy
NAME MESHED SUCCESS RPS LATENCY_P50 LATENCY_P95 LATENCY_P99 TCP_CONN
bad 1/1 94.74% 0.3rps 1ms 1ms 1ms 3
good 1/1 100.00% 10.3rps 1ms 1ms 1ms 4
slow-cooker 1/1 100.00% 0.3rps 1ms 2ms 2ms 1
ほぼgoodに飛んでいるので、slow-cookerの成功率が高い
$ linkerd viz -n circuit-breaking-demo stat deploy/slow-cooker --to svc/bb
NAME MESHED SUCCESS RPS LATENCY_P50 LATENCY_P95 LATENCY_P99 TCP_CONN
slow-cooker 1/1 99.83% 10.0rps 1ms 2ms 3ms 4
Progressive Delivery /w Argo Rollouts
公式ドキュメントに極力沿うように作業を進めるが、yamlに修正が必要箇所が何個かあるので、以降の手順を正とする。
なお、Argo Rolloutsでは、Linkerd Pluginがないため、KubernetesビルトインのGateway API用Pluginを利用する。
junya0530まずはArgo Rolloutsをインストールする。
kubectl create namespace argo-rollouts && \
kubectl apply -n argo-rollouts -f https://github.com/argoproj/argo-rollouts/releases/latest/download/install.yaml
namespace/argo-rollouts created
customresourcedefinition.apiextensions.k8s.io/analysisruns.argoproj.io created
customresourcedefinition.apiextensions.k8s.io/analysistemplates.argoproj.io created
customresourcedefinition.apiextensions.k8s.io/clusteranalysistemplates.argoproj.io created
customresourcedefinition.apiextensions.k8s.io/experiments.argoproj.io created
customresourcedefinition.apiextensions.k8s.io/rollouts.argoproj.io created
serviceaccount/argo-rollouts created
clusterrole.rbac.authorization.k8s.io/argo-rollouts created
clusterrole.rbac.authorization.k8s.io/argo-rollouts-aggregate-to-admin created
clusterrole.rbac.authorization.k8s.io/argo-rollouts-aggregate-to-edit created
clusterrole.rbac.authorization.k8s.io/argo-rollouts-aggregate-to-view created
clusterrolebinding.rbac.authorization.k8s.io/argo-rollouts created
configmap/argo-rollouts-config created
secret/argo-rollouts-notification-secret created
service/argo-rollouts-metrics created
deployment.apps/argo-rollouts created
$ kubectl -n argo-rollouts get all
NAME READY STATUS RESTARTS AGE
pod/argo-rollouts-68bffbdf98-plmq5 1/1 Running 0 21s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/argo-rollouts-metrics ClusterIP 172.20.201.219 <none> 8090/TCP 21s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/argo-rollouts 1/1 1 1 21s
NAME DESIRED CURRENT READY AGE
replicaset.apps/argo-rollouts-68bffbdf98 1 1 1 21s
Argo Rollouts用でConfigmapとClusterrole、Clusterrolebindingsを作成する。
ConfigmapでKubernetesビルトインのGateway API用Pluginを定義する。
さらにArgo RolloutsはConfigmapを作成権限が必要となるため、それ用の権限も渡している。
これはRolloutsリソースとアプリがあるnamespaceに作成されるargo-gatewayapi-configmapの管理用権限みたい。
$ kubectl apply -f - <<EOF
apiVersion: v1
kind: ConfigMap
metadata:
name: argo-rollouts-config
namespace: argo-rollouts
data:
trafficRouterPlugins: |-
- name: "argoproj-labs/gatewayAPI"
location: "https://github.com/argoproj-labs/rollouts-plugin-trafficrouter-gatewayapi/releases/download/v0.6.0/gatewayapi-plugin-linux-amd64"
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: argo-controller-role
namespace: argo-rollouts
rules:
- apiGroups:
- gateway.networking.k8s.io
resources:
- httproutes
verbs:
- "*"
- apiGroups: [""]
resources: ["configmaps"]
verbs: ["create", "update", "patch", "get", "list", "watch", "delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: argo-controller
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: argo-controller-role
subjects:
- namespace: argo-rollouts
kind: ServiceAccount
name: argo-rollouts
EOF
configmap/argo-rollouts-config configured
clusterrole.rbac.authorization.k8s.io/argo-controller-role created
clusterrolebinding.rbac.authorization.k8s.io/argo-controller created
次にnamespaceを作成し、Gateway APIのAPIを利用し、HTTPRouteリソースを作成し、dynamic routingを実現させる。
同時にArgo Rolloutsリソースも作成し、10秒毎にPromoteさせるように設定する。
Argo Rolloutsリソースの.spec.template.metadata.annotationにLinkerdをSidecarとしてInjectさせるためのアノテーションを追加している。
$ kubectl create ns test
namespace/test created
$ kubectl apply -f - <<EOF
apiVersion: gateway.networking.k8s.io/v1beta1
kind: HTTPRoute
metadata:
name: argo-rollouts-http-route
namespace: test
spec:
parentRefs:
- name: podinfo
namespace: test
kind: Service
group: core
port: 9898
rules:
- backendRefs:
- name: podinfo-stable
namespace: test
port: 9898
- name: podinfo-canary
namespace: test
port: 9898
---
apiVersion: v1
kind: Service
metadata:
name: podinfo-canary
namespace: test
spec:
ports:
- port: 9898
targetPort: 9898
protocol: TCP
name: http
selector:
app: podinfo
---
apiVersion: v1
kind: Service
metadata:
name: podinfo-stable
namespace: test
spec:
ports:
- port: 9898
targetPort: 9898
protocol: TCP
name: http
selector:
app: podinfo
---
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
name: rollouts-demo
namespace: test
spec:
replicas: 2
strategy:
canary:
canaryService: podinfo-canary # our created canary service
stableService: podinfo-stable # our created stable service
trafficRouting:
plugins:
argoproj-labs/gatewayAPI:
httpRoute: argo-rollouts-http-route # our created httproute
namespace: test
steps:
- setWeight: 30
- pause: { duration: 10 }
- setWeight: 40
- pause: { duration: 10 }
- setWeight: 60
- pause: { duration: 10 }
- setWeight: 80
- pause: { duration: 10 }
revisionHistoryLimit: 2
selector:
matchLabels:
app: podinfo
template:
metadata:
labels:
app: podinfo
annotations:
linkerd.io/inject: enabled
spec:
containers:
- name: podinfod
image: quay.io/stefanprodan/podinfo:1.7.0
ports:
- containerPort: 9898
protocol: TCP
EOF
httproute.gateway.networking.k8s.io/argo-rollouts-http-route created
service/podinfo-canary created
service/podinfo-stable created
rollout.argoproj.io/rollouts-demo created
最後にRolloutsリソースで作成されたアプリへのServiceとIngressを作成する。
$ cat << EOT | kubectl apply -f -
∙ apiVersion: v1
kind: Service
metadata:
name: podinfo
namespace: test
labels:
app.kubernetes.io/name: loadtester
app.kubernetes.io/instance: flagger
spec:
type: ClusterIP
ports:
- port: 9898
selector:
app: podinfo
∙ EOT
service/podinfo created
$ cat << EOT | k apply -f -
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: pod-info
namespace: test
annotations:
external-dns.alpha.kubernetes.io/hostname: "test.jnytnai.click"
alb.ingress.kubernetes.io/scheme: "internet-facing"
alb.ingress.kubernetes.io/target-type: "ip"
alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}]'
spec:
ingressClassName: alb
rules:
- host: "test.jnytnai.click"
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: podinfo
port:
number: 9898
EOT
ingress/pod-info created
ここまでくると、argo-rollouts Podで以下のようなログが確認できる。
plugin.gatewayAPI: time="2025-07-30T08:00:15Z" level=info msg="[SetWeight] plugin \"argoproj-labs/gatewayAPI\" controls HTTPRoutes: [argo-rollouts-http-route]" plugin=trafficrouter からPluginが読み込まれ、HTTPRouteリソースが利用されていることがわかる。
$ kubectl logs -n argo-rollouts -l app.kubernetes.io/name=argo-rollouts --tail=100
:
time="2025-07-30T08:00:15Z" level=info msg="Started syncing rollout" generation=2 namespace=test resourceVersion=111485 rollout=rollouts-demo
time="2025-07-30T08:00:15Z" level=info msg="Found 1 TrafficRouting Reconcilers" namespace=test rollout=rollouts-demo
time="2025-07-30T08:00:15Z" level=info msg="Reconciling TrafficRouting with type 'GatewayAPI'" namespace=test rollout=rollouts-demo
2025-07-30T08:00:15.803Z [DEBUG] plugin.gatewayAPI: time="2025-07-30T08:00:15Z" level=info msg="[RemoveManagedRoutes] plugin \"argoproj-labs/gatewayAPI\" controls HTTPRoutes: [argo-rollouts-http-route]" plugin=trafficrouter
2025-07-30T08:00:15.828Z [DEBUG] plugin.gatewayAPI: time="2025-07-30T08:00:15Z" level=info msg="[SetWeight] plugin \"argoproj-labs/gatewayAPI\" controls HTTPRoutes: [argo-rollouts-http-route]" plugin=trafficrouter
2025-07-30T08:00:15.858Z [DEBUG] plugin.gatewayAPI: time="2025-07-30T08:00:15Z" level=info msg="[SetWeight] plugin \"argoproj-labs/gatewayAPI\" controls GRPCRoutes: []" plugin=trafficrouter
2025-07-30T08:00:15.858Z [DEBUG] plugin.gatewayAPI: time="2025-07-30T08:00:15Z" level=info msg="[SetWeight] plugin \"argoproj-labs/gatewayAPI\" controls TCPRoutes: []" plugin=trafficrouter
time="2025-07-30T08:00:15Z" level=info msg="Desired weight (stepIdx: 8) 0 verified" namespace=test rollout=rollouts-demo
time="2025-07-30T08:00:15Z" level=info msg="No StableRS exists to reconcile or matches newRS" namespace=test rollout=rollouts-demo
time="2025-07-30T08:00:15Z" level=info msg="No Steps remain in the canary steps" namespace=test rollout=rollouts-demo
time="2025-07-30T08:00:15Z" level=info msg="No status changes. Skipping patch" generation=2 namespace=test resourceVersion=111485 rollout=rollouts-demo
time="2025-07-30T08:00:15Z" level=info msg="Reconciliation completed" generation=2 namespace=test resourceVersion=111485 rollout=rollouts-demo time_ms=58.343944
もしGateway API Pluginが読み取れない旨のエラーが出てていたら、argo-rollouts namespaceのargo-rollouts Deploymentsを再起動する。
junya0530さらにkubectlのtree Pluginを使うと以下のように親子関係も確認可能。
$ kubectl tree rollouts/rollouts-demo -n test
NAMESPACE NAME READY REASON AGE
test Rollout/rollouts-demo - 2m18s
test └─ReplicaSet/rollouts-demo-77c4c4ccd9 - 2m18s
test ├─Pod/rollouts-demo-77c4c4ccd9-h4ph7 True 2m18s
test └─Pod/rollouts-demo-77c4c4ccd9-sd7cr True 2m18s
ここで、イメージをUpgradeしてみる。
$ kubectl argo rollouts -n test set image rollouts-demo \
podinfod=quay.io/stefanprodan/podinfo:1.7.1
rollout "rollouts-demo" image updated
すると、以下のように、Revisionが切り替わり、Imageの入れ替えができていることが確認できる。
$ kubectll argo rollouts -n test get rollout rollouts-demo --watch
:
NAME KIND STATUS AGE INFO
⟳ rollouts-demo Rollout ✔ Healthy 8m14s
├──# revision:2
│ └──⧉ rollouts-demo-9b46f8dfb ReplicaSet ✔ Healthy 3m27s stable
│ ├──□ rollouts-demo-9b46f8dfb-v2wcv Pod ✔ Running 3m27s ready:2/2
│ └──□ rollouts-demo-9b46f8dfb-7qzlc Pod ✔ Running 2m59s ready:2/2
└──# revision:1
└──⧉ rollouts-demo-5db59bd5ff ReplicaSet • ScaledDown 8m14s
Name: rollouts-demo
Namespace: test
Status: ✔ Healthy
Strategy: Canary
Step: 8/8
SetWeight: 100
ActualWeight: 100
Images: quay.io/stefanprodan/podinfo:1.7.1 (stable)
Replicas:
Desired: 2
Current: 2
Updated: 2
Ready: 2
Available: 2
同時に、Ingressにリクエストを送り続けていたら、Imageのタグがv1.7.0のcanaryとv1.7.1のstableへのリクエストが流れて、最後はstableのみになっていることも確認でき、カナリアリリースできていた。
$ while true
do
curl http://test.jnytnai.click
done
{
"hostname": "rollouts-demo-9b46f8dfb-v2wcv",
"version": "1.7.1",
"revision": "c9dc78f29c5087e7c181e58a56667a75072e6196",
"color": "blue",
"message": "greetings from podinfo v1.7.1",
"goos": "linux",
"goarch": "amd64",
"runtime": "go1.11.12",
"num_goroutine": "7",
"num_cpu": "2"
}{
"hostname": "rollouts-demo-5db59bd5ff-l5sfl",
"version": "1.7.0",
"revision": "4fc593f42c7cd2e7319c83f6bfd3743c05523883",
"color": "blue",
"message": "greetings from podinfo v1.7.0",
"goos": "linux",
"goarch": "amd64",
"runtime": "go1.11.2",
"num_goroutine": "6",
"num_cpu": "2"
}{
"hostname": "rollouts-demo-5db59bd5ff-8jvng",
"version": "1.7.0",
"revision": "4fc593f42c7cd2e7319c83f6bfd3743c05523883",
"color": "blue",
"message": "greetings from podinfo v1.7.0",
"goos": "linux",
"goarch": "amd64",
"runtime": "go1.11.2",
"num_goroutine": "6",
"num_cpu": "2"
}{
"hostname": "rollouts-demo-9b46f8dfb-v2wcv",
"version": "1.7.1",
"revision": "c9dc78f29c5087e7c181e58a56667a75072e6196",
"color": "blue",
"message": "greetings from podinfo v1.7.1",
"goos": "linux",
"goarch": "amd64",
"runtime": "go1.11.12",
"num_goroutine": "7",
"num_cpu": "2"
}
:
{
"hostname": "rollouts-demo-9b46f8dfb-v2wcv",
"version": "1.7.1",
"revision": "c9dc78f29c5087e7c181e58a56667a75072e6196",
"color": "blue",
"message": "greetings from podinfo v1.7.1",
"goos": "linux",
"goarch": "amd64",
"runtime": "go1.11.12",
"num_goroutine": "6",
"num_cpu": "2"
}{
"hostname": "rollouts-demo-9b46f8dfb-v2wcv",
"version": "1.7.1",
"revision": "c9dc78f29c5087e7c181e58a56667a75072e6196",
"color": "blue",
"message": "greetings from podinfo v1.7.1",
"goos": "linux",
"goarch": "amd64",
"runtime": "go1.11.12",
"num_goroutine": "6",
"num_cpu": "2"
}{
"hostname": "rollouts-demo-9b46f8dfb-7qzlc",
"version": "1.7.1",
"revision": "c9dc78f29c5087e7c181e58a56667a75072e6196",
"color": "blue",
"message": "greetings from podinfo v1.7.1",
"goos": "linux",
"goarch": "amd64",
"runtime": "go1.11.12",
"num_goroutine": "6",
"num_cpu": "2"
}
ここまでArgo Rolloutsのみでカナリアリリースしたが、RolloutsリソースをGitHubに置き、Kustomizeのkustomization.yamlで管理し、Argo CDのApplicationで監視することで、楽にリリースもできる。
この時、Argo CDのArgo Rollouts拡張機能をインストールすることでUIでもリリース状況を確認できる。
このスクラップは3ヶ月前にクローズされました