⛳
O-RAN SC RIC Platform 環境構築メモ
初めに
この記事は個人的なメモです.
間違い・抜け漏れが含まれている可能性があります.
積極的に読む意味はあまりないです.
もしO-RAN SC E-Releaseの環境構築をしたい方がいた場合には, 何らかの助けになるかもしれません.
書くこと
- O-RAN SC (E-Release)のDeployに関しての備忘録
- sample xApp deployでつまづいた点, ワークアラウンド
書かないこと
- O-RAN SCのアーキテクチャ概観・解説
- deploy後の動作確認等
今後やること
- deploy後の動作確認等
- sample xAppの実装・Deploy with golang
- O-RAN SCのアーキテクチャ理解・解説作成
参考書籍
以下Documentを中心として, 適宜O-RAN周辺ドキュメントを参照.
前提・環境
- AWS EC2 (Single Node)
version
# ubuntu 18.04 LTS
$ git log
commit 9f95be3aecb47b69cd918ae1a04cbbcc129a4e01 (HEAD -> master, origin/master, origin/HEAD)
Author: czichy <thoralf.czichy@nokia.com>
Date: Fri Dec 17 11:24:33 2021 +0200
Updating to latest ric-dep submodule change-id
Signed-off-by: czichy <thoralf.czichy@nokia.com>
Change-Id: I7cf6f8c6ae6c45d3335fc00ad6a12cc1cffe7182
$ docker version
Client:
Version: 20.10.7
API version: 1.41
Go version: go1.13.8
Git commit: 20.10.7-0ubuntu5~18.04.3
Built: Mon Nov 1 01:04:14 2021
OS/Arch: linux/amd64
Context: default
Experimental: true
Server:
Engine:
Version: 20.10.7
API version: 1.41 (minimum version 1.12)
Go version: go1.13.8
Git commit: 20.10.7-0ubuntu5~18.04.3
Built: Fri Oct 22 00:57:37 2021
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.5.5-0ubuntu3~18.04.1
GitCommit:
runc:
Version: 1.0.1-0ubuntu2~18.04.1
GitCommit:
docker-init:
Version: 0.19.0
GitCommit:
$ kubeadm version
kubeadm version: &version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.0", GitCommit:"9e991415386e4cf155a24b1da15becaa390438d8", GitTreeState:"clean", BuildDate:"2020-03-25T14:56:30Z", GoVersion:"go1.13.8", Compiler:"gc", Platform:"linux/amd64"}
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.0", GitCommit:"9e991415386e4cf155a24b1da15becaa390438d8", GitTreeState:"clean", BuildDate:"2020-03-25T14:58:59Z", GoVersion:"go1.13.8", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.0", GitCommit:"9e991415386e4cf155a24b1da15becaa390438d8", GitTreeState:"clean", BuildDate:"2020-03-25T14:50:46Z", GoVersion:"go1.13.8", Compiler:"gc", Platform:"linux/amd64"}
$ kubelet --version
Kubernetes v1.18.0
$ helm version
version.BuildInfo{Version:"v3.2.2", GitCommit:"a6ea66349ae3015618da4f547677a14b9ecc09b3", GitTreeState:"clean", GoVersion:"go1.13.12"}
root@ip-10-0-102-250:~/xapps/hw-python/init#
手順
Infra Install
$ sudo -i
$ swapoff -a
$ git clone http://gerrit.o-ran-sc.org/r/it/dep
$ cd dep
$ git submodule update --init --recursive --remote
$ cd tools/k8s/bin
$ vim ../etc/infra.rc
$ cat ../etc/infra.rc
# modify below for RIC infrastructure (docker-k8s-helm) component versions
# RIC tested
INFRA_DOCKER_VERSION=""
INFRA_HELM_VERSION="3.2.2"
INFRA_K8S_VERSION="1.18.0"
#INFRA_HELM_VERSION="2.17.0"
#INFRA_K8S_VERSION="1.16.0"
INFRA_CNI_VERSION=""
# older RIC tested
#INFRA_DOCKER_VERSION=""
#INFRA_HELM_VERSION="2.12.3"
#INFRA_K8S_VERSION="1.13.3"
#INFRA_CNI_VERSION="0.6.0"
# ONAP Frankfurt
#INFRA_DOCKER_VERSION="18.09.7"
#INFRA_K8S_VERSION="1.15.9"
#INFRA_CNI_VERSION="0.7.5"
#INFRA_HELM_VERSION="2.16.6"
$ ./gen-cloud-init.sh
上記手順まで実施した後, 生成されるk8s-1node-cloud-init-k_1_16-h_2_12-d_cur.shを実行すると, 当該スクリプト290行目の実行・HelmのInstallに失敗するため, その後のStepに進めない.
wget https://storage.googleapis.com/kubernetes-helm/helm-v${HELMVERSION}-linux-amd64.tar.gz
よって, 別途Helm v3をInstallする手順を挟む.
$ curl https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3 | bash
$ helm version
Client: &version.Version{SemVer:"v2.17.0", GitCommit:"a690bad98af45b015bd3da1a41f6218b1a451dbe", GitTreeState:"clean"}
Error: Get "http://localhost:8080/api/v1/namespaces/kube-system/pods?labelSelector=app%3Dhelm%2Cname%3Dtiller": dial tcp 127.0.0.1:8080: connect: connection refused
$ ./k8s-1node-cloud-init-k_1_18-h_3_2-d_cur.sh
数分後Rebootが走るので, 再ログイン後, podを確認.
$ sudo -i
$ kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system coredns-66bff467f8-6x4f2 1/1 Running 1 2m2s
kube-system coredns-66bff467f8-mhcn8 1/1 Running 1 2m2s
kube-system etcd-ip-10-0-102-250 1/1 Running 1 2m18s
kube-system kube-apiserver-ip-10-0-102-250 1/1 Running 1 2m18s
kube-system kube-controller-manager-ip-10-0-102-250 1/1 Running 1 2m18s
kube-system kube-flannel-ds-5sl27 1/1 Running 1 2m3s
kube-system kube-proxy-qm6tc 1/1 Running 2 2m3s
kube-system kube-scheduler-ip-10-0-102-250 1/1 Running 1 2m17s
8 Pods起動していることを確認.
各種k8s toolのinstall
飛ばしてもいいところ.
$ echo 'source <(kubectl completion bash)' >>~/.bashrc
$ (
set -x; cd "$(mktemp -d)" &&
OS="$(uname | tr '[:upper:]' '[:lower:]')" &&
ARCH="$(uname -m | sed -e 's/x86_64/amd64/' -e 's/\(arm\)\(64\)\?.*/\1\2/' -e 's/aarch64$/arm64/')" &&
KREW="krew-${OS}_${ARCH}" &&
curl -fsSLO "https://github.com/kubernetes-sigs/krew/releases/latest/download/${KREW}.tar.gz" &&
tar zxvf "${KREW}.tar.gz" &&
./"${KREW}" install krew
)
$ echo 'export PATH="${KREW_ROOT:-$HOME/.krew}/bin:$PATH"' >> ~/.bashrc
$ source ~/.bashrc
$ kubectl krew
$ kubectl krew install ns
RIC Recipe Deploy
$ cd dep/bin
# E-ReleaseをInstall
$ ./deploy-ric-platform -f ../RECIPE_EXAMPLE/PLATFORM/example_recipe_oran_e_release.yaml
$ kubectl ns ricplt
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
deployment-ricplt-a1mediator-b4576889d-slwf5 1/1 Running 0 3m7s
deployment-ricplt-alarmmanager-f59846448-js9w7 1/1 Running 0 2m8s
deployment-ricplt-appmgr-7cfbff4f7d-r8ks9 1/1 Running 0 4m4s
deployment-ricplt-e2mgr-7f57cb799b-mwdzr 1/1 Running 0 3m35s
deployment-ricplt-e2term-alpha-79784bd9fd-4frrt 1/1 Running 0 3m21s
deployment-ricplt-o1mediator-f7dd5fcc8-tdwnz 1/1 Running 0 2m23s
deployment-ricplt-rtmgr-7455599d58-fnhbv 1/1 Running 2 3m50s
deployment-ricplt-submgr-6cd6775cd6-t2rqv 1/1 Running 0 2m53s
deployment-ricplt-vespamgr-757b6cc5dc-kvj22 1/1 Running 0 2m37s
r4-infrastructure-kong-646b68bd88-7bqss 2/2 Running 1 4m34s
r4-infrastructure-prometheus-alertmanager-75dff54776-6jfwv 2/2 Running 0 4m34s
r4-infrastructure-prometheus-server-5fd7695-97bxr 1/1 Running 0 4m34s
statefulset-ricplt-dbaas-server-0 1/1 Running 0 4m19s
Trouble Shooting (1)
上記手順では, 実はricinfra namespaceの tiller Podの起動に失敗している.
Helm v3ではtiller podは不要なので放置してもいいかもしれないが, 気持ち悪いので修正する.
$ kubectl get pods -A
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system coredns-66bff467f8-6x4f2 1/1 Running 1 7m49s
kube-system coredns-66bff467f8-mhcn8 1/1 Running 1 7m49s
kube-system etcd-ip-10-0-102-250 1/1 Running 1 8m5s
kube-system kube-apiserver-ip-10-0-102-250 1/1 Running 1 8m5s
kube-system kube-controller-manager-ip-10-0-102-250 1/1 Running 1 8m5s
kube-system kube-flannel-ds-5sl27 1/1 Running 1 7m50s
kube-system kube-proxy-qm6tc 1/1 Running 2 7m50s
kube-system kube-scheduler-ip-10-0-102-250 1/1 Running 1 8m4s
ricinfra deployment-tiller-ricxapp-5cd69f688b-fzndf 0/1 Running 0 10s
ricinfra deployment-tiller-ricxapp-755fbd9fbc-rp2qb 0/1 ImagePullBackOff 0 4m38s
ricinfra tiller-secret-generator-mcvvm 0/1 Completed 0 4m38s
ricplt deployment-ricplt-a1mediator-687b54f649-zmknk 1/1 Running 0 3m48s
ricplt deployment-ricplt-alarmmanager-6dc48dbb97-txlkh 1/1 Running 0 3m14s
ricplt deployment-ricplt-appmgr-74d5f56b67-b67zn 1/1 Running 0 4m21s
ricplt deployment-ricplt-e2mgr-5d7fddc7b4-vxp8l 1/1 Running 3 4m4s
ricplt deployment-ricplt-e2term-alpha-7f64b9cc44-wkfj5 1/1 Running 0 3m56s
ricplt deployment-ricplt-o1mediator-5c76d6577-g66kv 1/1 Running 0 3m22s
ricplt deployment-ricplt-rtmgr-5fdbd9854f-t4q7t 1/1 Running 1 4m13s
ricplt deployment-ricplt-submgr-67d4f45c96-bc7qh 1/1 Running 0 3m39s
ricplt deployment-ricplt-vespamgr-745b7968cd-lwckf 1/1 Running 0 3m31s
ricplt r4-infrastructure-kong-7c974fb98b-srj7t 2/2 Running 2 4m38s
ricplt r4-infrastructure-prometheus-alertmanager-757c4d8d66-kcqq7 2/2 Running 0 4m38s
ricplt r4-infrastructure-prometheus-server-6b57df6c85-fhmcb 1/1 Running 0 4m38s
ricplt statefulset-ricplt-dbaas-server-0 1/1 Running 0 4m30s
起動失敗の理由について調べると(調べるまでもないが), manifest内で利用しているImage Repository (gcr.io/kubernetes-helm/tiller:v2.12.3)が既に失われている.
$ docker pull gcr.io/kubernetes-helm/tiller:v2.12.3
Error response from daemon: Head https://gcr.io/v2/kubernetes-helm/tiller/manifests/v2.12.3: unknown: Project 'project:kubernetes-helm' not found or deleted.
そのため, W/Aとして同Versionのtillerを配布しているDocker HubのRepo (fishead/gcr.io.kubernetes-helm.tiller:v2.12.3)を指定するように変更する.
$ kubectl edit deployments.apps -n ricinfra deployment-tiller-ricxapp
確認したら正常に起動していたのでOK!
$ kubectl get pod -n ricinfra
NAME READY STATUS RESTARTS AGE
deployment-tiller-ricxapp-5cd69f688b-fzndf 1/1 Running 0 78s
tiller-secret-generator-mcvvm 0/1 Completed 0 5m46s
Trouble Shooting (2)
xAppsのManagement CLI Toolである, dms_cliを用いてSample ChartをDeployする際に必要となるLocal repositoryも立てておく.
下記Manifestを作成し, apply.
ricplt-xapp-onboarder-http.yaml
---
apiVersion: apps/v1
kind: Deployment
metadata:
creationTimestamp: null
labels:
app: deployment-ricplt-xapp-onboarder-http
name: deployment-ricplt-xapp-onboarder-http
spec:
replicas: 1
selector:
matchLabels:
app: deployment-ricplt-xapp-onboarder-http
strategy: {}
template:
metadata:
creationTimestamp: null
labels:
app: deployment-ricplt-xapp-onboarder-http
spec:
containers:
- image: chartmuseum/chartmuseum
name: chartmuseum
securityContext:
runAsUser: 0
volumeMounts:
- name: charts
mountPath: /charts
env:
- name: DEBUG
value: "1"
- name: STORAGE
value: "local"
- name: STORAGE_LOCAL_ROOTDIR
value: "/charts"
volumes:
- name: charts
hostPath:
path: /root/charts
---
apiVersion: v1
kind: Service
metadata:
creationTimestamp: null
labels:
app: service-ricplt-xapp-onboarder-http
name: service-ricplt-xapp-onboarder-http
spec:
ports:
- name: default
port: 8080
protocol: TCP
targetPort: 8080
selector:
app: deployment-ricplt-xapp-onboarder-http
type: ClusterIP
$ kubectl apply -f ricplt-xapp-onboarder-http.yaml
$ kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
aux-entry ClusterIP 10.98.0.36 <none> 80/TCP,443/TCP 16m
r4-infrastructure-kong-proxy NodePort 10.111.0.86 <none> 32080:32080/TCP,32443:32443/TCP 16m
r4-infrastructure-prometheus-alertmanager ClusterIP 10.101.34.5 <none> 80/TCP 16m
r4-infrastructure-prometheus-server ClusterIP 10.104.40.46 <none> 80/TCP 16m
service-ricplt-a1mediator-http ClusterIP 10.99.227.215 <none> 10000/TCP 15m
service-ricplt-a1mediator-rmr ClusterIP 10.106.239.193 <none> 4561/TCP,4562/TCP 15m
service-ricplt-alarmmanager-http ClusterIP 10.99.111.90 <none> 8080/TCP 14m
service-ricplt-alarmmanager-rmr ClusterIP 10.106.206.229 <none> 4560/TCP,4561/TCP 14m
service-ricplt-appmgr-http ClusterIP 10.101.231.239 <none> 8080/TCP 16m
service-ricplt-appmgr-rmr ClusterIP 10.111.159.227 <none> 4561/TCP,4560/TCP 16m
service-ricplt-dbaas-tcp ClusterIP None <none> 6379/TCP 16m
service-ricplt-e2mgr-http ClusterIP 10.102.244.208 <none> 3800/TCP 15m
service-ricplt-e2mgr-rmr ClusterIP 10.105.153.16 <none> 4561/TCP,3801/TCP 15m
service-ricplt-e2term-prometheus-alpha ClusterIP 10.110.65.135 <none> 8088/TCP 15m
service-ricplt-e2term-rmr-alpha ClusterIP 10.108.254.46 <none> 4561/TCP,38000/TCP 15m
service-ricplt-e2term-sctp-alpha NodePort 10.110.165.173 <none> 36422:32222/SCTP 15m
service-ricplt-o1mediator-http ClusterIP 10.101.33.111 <none> 9001/TCP,8080/TCP,3000/TCP 14m
service-ricplt-o1mediator-tcp-netconf NodePort 10.100.91.23 <none> 830:30830/TCP 14m
service-ricplt-rtmgr-http ClusterIP 10.103.98.119 <none> 3800/TCP 16m
service-ricplt-rtmgr-rmr ClusterIP 10.99.218.193 <none> 4561/TCP,4560/TCP 16m
service-ricplt-submgr-http ClusterIP None <none> 3800/TCP 15m
service-ricplt-submgr-rmr ClusterIP None <none> 4560/TCP,4561/TCP 15m
service-ricplt-vespamgr-http ClusterIP 10.102.64.112 <none> 8080/TCP,9095/TCP 14m
service-ricplt-xapp-onboarder-http ClusterIP 10.104.49.4 <none> 8080/TCP 77s
Deploy Sample xApp
dms_cliのInstall
言われるとおり, そのまま実行. (Package managerはaptに変更)
# Git clone appmgr
$ git clone "https://gerrit.o-ran-sc.org/r/ric-plt/appmgr"
# Change dir to xapp_onboarder
$ cd appmgr/xapp_orchestrater/dev/xapp_onboarder
# If pip3 is not installed, install using the following command
$ apt install -y python3-pip
# In case dms_cli binary is already installed, it can be uninstalled using following command
$ pip3 uninstall xapp_onboarder
# Install xapp_onboarder using following command
$ pip3 install ./
Chart Repo. のClusterIP=10.107.141.229なので, dms_cli向けの環境変数を設定.
$ echo 'export CHART_REPO_URL=http://10.107.141.229:8080' >> ~/.bashrc
$ source ~/.bashrc
$ dms_cli health
True
hwxapp Deploy
1. Onboard
CHART_REPO_URLをservice-ricplt-xapp-onboarder-httpに切り替えて, dms_cliを使って再度onboard.
$ pwd
/root
$ mkdir xapps
$ cd ~/xapps
$ git clone "https://github.com/o-ran-sc/ric-app-hw.git"
$ cd ric-app-hw/init
$ dms_cli onboard config-file.json schema.json
{
"status": "Created"
}
$ dms_cli get_charts_list
{
...,
"hwxapp": [
{
"apiVersion": "v1",
"appVersion": "1.0",
"created": "2022-01-05T13:34:50.444607264Z",
"description": "Standard xApp Helm Chart",
"digest": "4066af47e8190743ae550463ba884826683b10f3b6521dac0def7bbafa37a5f3",
"name": "hwxapp",
"urls": [
"charts/hwxapp-1.0.0.tgz"
],
"version": "1.0.0"
}
],
...,
}
2. Deploy
r4-infrastructure-kong-proxy Serviceの, <NodePort>:32080 Port向けに下記のJSON RequestをPost．
Helm v3を使う場合は, 以下コマンドでデプロイ可能.
$ dms_cli install hwxapp 1.0.0 ricxapp
status: OK
成功！
$ dms_cli install hwxapp 1.0.0 ricxapp
status: OK
root@ip-10-0-102-250:~/xapps/ric-app-hw/init# kubectl get all -n ricxapp
NAME READY STATUS RESTARTS AGE
pod/ricxapp-hwxapp-595cfc4b6-hx2ls 1/1 Running 0 12s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/aux-entry ClusterIP 10.104.147.183 <none> 80/TCP,443/TCP 22h
service/service-ricxapp-hwxapp-rmr ClusterIP 10.101.205.57 <none> 4560/TCP,4561/TCP 12s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/ricxapp-hwxapp 1/1 1 1 12s
NAME DESIRED CURRENT READY AGE
replicaset.apps/ricxapp-hwxapp-595cfc4b6 1 1 1 12s
$ kubectl logs -n ricxapp ricxapp-hwxapp-595cfc4b6-hx2ls hwxapp
{"ts":1641389756764,"crit":"INFO","id":"hw_xapp_main","mdc":{},"msg":"Starting thread 140339566057280"}
{"ts":1641389756764,"crit":"INFO","id":"hw_xapp_main","mdc":{},"msg":"Xapp Name set to hwxapp from environment variable"}
{"ts":1641389756764,"crit":"INFO","id":"hw_xapp_main","mdc":{},"msg":"Xapp ID set to hwxapp from environment variable"}
{"ts":1641389756764,"crit":"INFO","id":"hw_xapp_main","mdc":{},"msg":"Ports set to 4560 from environment variable"}
1641389756 8/RMR [INFO] ric message routing library on SI95/g mv=3 flg=00 (f1d10cd 4.0.5 built: May 7 2020)
{"ts":1641389756765,"crit":"INFO","id":"hw_xapp_main","mdc":{},"msg":">>> waiting for RMR, file= xapp-utils/xapp_rmr.cc, line=59"}
{"ts":1641389757765,"crit":"INFO","id":"hw_xapp_main","mdc":{},"msg":"RMR Context is Ready, file= xapp-utils/xapp_rmr.cc, line=63"}
{"ts":1641389757765,"crit":"INFO","id":"hw_xapp_main","mdc":{},"msg":"Created Hello World Xapp Instance"}
1641389757 8/RMR [INFO] sends: ts=1641389757 src=service-ricxapp-hwxapp-rmr.ricxapp:4560 target=service-ricplt-a1mediator-rmr.ricplt:4562 open=0 succ=0 fail=0 (hard=0 soft=0)
1641389757 8/RMR [INFO] sends: ts=1641389757 src=service-ricxapp-hwxapp-rmr.ricxapp:4560 target=service-ricplt-e2term-rmr-alpha.ricplt:38000 open=0 succ=0 fail=0 (hard=0 soft=0)
{"ts":1641389758769,"crit":"INFO","id":"hw_xapp_main","mdc":{},"msg":"GNB List in R-NIB {\"gnb_list\":null} "}
{"ts":1641389758769,"crit":"INFO","id":"hw_xapp_main","mdc":{},"msg":"Preparing to send subscription in file= xapp.cc, line=127"}
{"ts":1641389758769,"crit":"INFO","id":"hw_xapp_main","mdc":{},"msg":"Subscriptions cannot be sent as GNBList in RNIB is NULL"}
{"ts":1641389758769,"crit":"INFO","id":"hw_xapp_main","mdc":{},"msg":"Sending request for policy id 2 "}
{"ts":1641389758769,"crit":"INFO","id":"hw_xapp_main","mdc":{},"msg":"Sending thread 140339566057280"}
{"ts":1641389758770,"crit":"INFO","id":"hw_xapp_main","mdc":{},"msg":"Message Sent: RMR State = RMR_OK"}
{"ts":1641389758770,"crit":"INFO","id":"hw_xapp_main","mdc":{},"msg":"Starting Listener Threads. Number of Workers = 1"}
{"ts":1641389758770,"crit":"INFO","id":"hw_xapp_main","mdc":{},"msg":"Receiver Thread file= xapp.cc, line=103"}
{"ts":1641389758770,"crit":"INFO","id":"hw_xapp_main","mdc":{},"msg":"Starting receiver thread 140338719090432"}
{"ts":1641389758770,"crit":"INFO","id":"hw_xapp_main","mdc":{},"msg":"Listening at Thread: 140338719090432"}
(動作確認方法がわからないけど)とりあえず動いていそうなのでOK.
ちゃんとした動作確認とか，アーキテクチャの理解とか，サンプルAppの実装・デプロイ等は，ここから始めていく.
ric-app-hw-go Deploy
1. Onboard
$ cd ~/xapps
$ git clone https://github.com/o-ran-sc/ric-app-hw-go.git -b e-release
$ cd ric-app-hw-go/config/
$ dms_cli onboard config-file.json schema.json
$ dms_cli install hw-go 1.0.0 ricxapp
付録
記事執筆段階での最終的な環境をメモ.
$ helm list
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
r4-a1mediator ricplt 1 2022-01-04 15:23:20.787884156 +0000 UTC deployed a1mediator-3.0.0 1.0
r4-alarmmanager ricplt 1 2022-01-04 15:23:54.574164915 +0000 UTC deployed alarmmanager-5.0.0 1.0
r4-appmgr ricplt 1 2022-01-04 15:22:47.111845471 +0000 UTC deployed appmgr-3.0.0 1.0
r4-dbaas ricplt 1 2022-01-04 15:22:38.679434629 +0000 UTC deployed dbaas-2.0.0 1.0
r4-e2mgr ricplt 1 2022-01-04 15:23:03.980225737 +0000 UTC deployed e2mgr-3.0.0 1.0
r4-e2term ricplt 1 2022-01-04 15:23:12.38211552 +0000 UTC deployed e2term-3.0.0 1.0
r4-infrastructure ricplt 1 2022-01-04 15:22:29.360168823 +0000 UTC deployed infrastructure-3.0.0 1.0
r4-o1mediator ricplt 1 2022-01-04 15:23:46.063492132 +0000 UTC deployed o1mediator-3.0.0 1.0
r4-rtmgr ricplt 1 2022-01-04 15:22:55.552944237 +0000 UTC deployed rtmgr-3.0.0 1.0
r4-submgr ricplt 1 2022-01-04 15:23:29.242091718 +0000 UTC deployed submgr-3.0.0 1.0
r4-vespamgr ricplt 1 2022-01-04 15:23:37.637942928 +0000 UTC deployed vespamgr-3.0.0 1.0
$ kubectl get all -A -o wide
NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
kube-system pod/coredns-66bff467f8-6x4f2 1/1 Running 2 22h 10.244.0.36 ip-10-0-102-250 <none> <none>
kube-system pod/coredns-66bff467f8-mhcn8 1/1 Running 2 22h 10.244.0.37 ip-10-0-102-250 <none> <none>
kube-system pod/etcd-ip-10-0-102-250 1/1 Running 2 22h 10.0.102.250 ip-10-0-102-250 <none> <none>
kube-system pod/kube-apiserver-ip-10-0-102-250 1/1 Running 2 22h 10.0.102.250 ip-10-0-102-250 <none> <none>
kube-system pod/kube-controller-manager-ip-10-0-102-250 1/1 Running 2 22h 10.0.102.250 ip-10-0-102-250 <none> <none>
kube-system pod/kube-flannel-ds-5sl27 1/1 Running 2 22h 10.0.102.250 ip-10-0-102-250 <none> <none>
kube-system pod/kube-proxy-qm6tc 1/1 Running 4 22h 10.0.102.250 ip-10-0-102-250 <none> <none>
kube-system pod/kube-scheduler-ip-10-0-102-250 1/1 Running 2 22h 10.0.102.250 ip-10-0-102-250 <none> <none>
ricinfra pod/deployment-tiller-ricxapp-5cd69f688b-fzndf 1/1 Running 1 22h 10.244.0.28 ip-10-0-102-250 <none> <none>
ricinfra pod/tiller-secret-generator-mcvvm 0/1 Completed 0 22h 10.244.0.8 ip-10-0-102-250 <none> <none>
ricplt pod/deployment-ricplt-a1mediator-687b54f649-zmknk 1/1 Running 2 22h 10.244.0.31 ip-10-0-102-250 <none> <none>
ricplt pod/deployment-ricplt-alarmmanager-6dc48dbb97-txlkh 1/1 Running 1 22h 10.244.0.35 ip-10-0-102-250 <none> <none>
ricplt pod/deployment-ricplt-appmgr-74d5f56b67-b67zn 1/1 Running 1 22h 10.244.0.25 ip-10-0-102-250 <none> <none>
ricplt pod/deployment-ricplt-e2mgr-5d7fddc7b4-vxp8l 1/1 Running 6 22h 10.244.0.30 ip-10-0-102-250 <none> <none>
ricplt pod/deployment-ricplt-e2term-alpha-7f64b9cc44-wkfj5 1/1 Running 1 22h 10.244.0.33 ip-10-0-102-250 <none> <none>
ricplt pod/deployment-ricplt-o1mediator-5c76d6577-g66kv 1/1 Running 1 22h 10.244.0.32 ip-10-0-102-250 <none> <none>
ricplt pod/deployment-ricplt-rtmgr-5fdbd9854f-t4q7t 1/1 Running 2 22h 10.244.0.27 ip-10-0-102-250 <none> <none>
ricplt pod/deployment-ricplt-submgr-67d4f45c96-bc7qh 1/1 Running 2 22h 10.244.0.38 ip-10-0-102-250 <none> <none>
ricplt pod/deployment-ricplt-vespamgr-745b7968cd-lwckf 1/1 Running 1 22h 10.244.0.24 ip-10-0-102-250 <none> <none>
ricplt pod/deployment-ricplt-xapp-onboarder-http-7dcdd9786b-4sxsq 1/1 Running 1 22h 10.244.0.29 ip-10-0-102-250 <none> <none>
ricplt pod/r4-infrastructure-kong-7c974fb98b-srj7t 2/2 Running 4 22h 10.244.0.34 ip-10-0-102-250 <none> <none>
ricplt pod/r4-infrastructure-prometheus-alertmanager-757c4d8d66-kcqq7 2/2 Running 2 22h 10.244.0.23 ip-10-0-102-250 <none> <none>
ricplt pod/r4-infrastructure-prometheus-server-6b57df6c85-fhmcb 1/1 Running 1 22h 10.244.0.39 ip-10-0-102-250 <none> <none>
ricplt pod/statefulset-ricplt-dbaas-server-0 1/1 Running 1 22h 10.244.0.26 ip-10-0-102-250 <none> <none>
ricxapp pod/ricxapp-hwxapp-595cfc4b6-hx2ls 1/1 Running 0 25m 10.244.0.52 ip-10-0-102-250 <none> <none>
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
default service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 22h <none>
kube-system service/kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP,9153/TCP 22h k8s-app=kube-dns
ricinfra service/service-tiller-ricxapp ClusterIP 10.111.75.117 <none> 44134/TCP 22h app=helm,name=tiller
ricplt service/aux-entry ClusterIP 10.103.63.147 <none> 80/TCP,443/TCP 22h <none>
ricplt service/r4-infrastructure-kong-proxy NodePort 10.108.190.92 <none> 32080:32080/TCP,32443:32443/TCP 22h app.kubernetes.io/component=app,app.kubernetes.io/instance=r4-infrastructure,app.kubernetes.io/name=kong
ricplt service/r4-infrastructure-prometheus-alertmanager ClusterIP 10.97.92.216 <none> 80/TCP 22h app=prometheus,component=alertmanager,release=r4-infrastructure
ricplt service/r4-infrastructure-prometheus-server ClusterIP 10.111.14.66 <none> 80/TCP 22h app=prometheus,component=server,release=r4-infrastructure
ricplt service/service-ricplt-a1mediator-http ClusterIP 10.103.59.50 <none> 10000/TCP 22h app=ricplt-a1mediator,release=r4-a1mediator
ricplt service/service-ricplt-a1mediator-rmr ClusterIP 10.97.211.45 <none> 4561/TCP,4562/TCP 22h app=ricplt-a1mediator,release=r4-a1mediator
ricplt service/service-ricplt-alarmmanager-http ClusterIP 10.106.187.163 <none> 8080/TCP 22h app=ricplt-alarmmanager,release=r4-alarmmanager
ricplt service/service-ricplt-alarmmanager-rmr ClusterIP 10.105.238.44 <none> 4560/TCP,4561/TCP 22h app=ricplt-alarmmanager,release=r4-alarmmanager
ricplt service/service-ricplt-appmgr-http ClusterIP 10.109.70.163 <none> 8080/TCP 22h app=ricplt-appmgr,release=r4-appmgr
ricplt service/service-ricplt-appmgr-rmr ClusterIP 10.107.101.193 <none> 4561/TCP,4560/TCP 22h app=ricplt-appmgr,release=r4-appmgr
ricplt service/service-ricplt-dbaas-tcp ClusterIP None <none> 6379/TCP 22h app=ricplt-dbaas,release=r4-dbaas
ricplt service/service-ricplt-e2mgr-http ClusterIP 10.105.216.61 <none> 3800/TCP 22h app=ricplt-e2mgr,release=r4-e2mgr
ricplt service/service-ricplt-e2mgr-rmr ClusterIP 10.101.45.75 <none> 4561/TCP,3801/TCP 22h app=ricplt-e2mgr,release=r4-e2mgr
ricplt service/service-ricplt-e2term-prometheus-alpha ClusterIP 10.99.130.142 <none> 8088/TCP 22h app=ricplt-e2term-alpha,release=r4-e2term
ricplt service/service-ricplt-e2term-rmr-alpha ClusterIP 10.99.31.100 <none> 4561/TCP,38000/TCP 22h app=ricplt-e2term-alpha,release=r4-e2term
ricplt service/service-ricplt-e2term-sctp-alpha NodePort 10.111.148.228 <none> 36422:32222/SCTP 22h app=ricplt-e2term-alpha,release=r4-e2term
ricplt service/service-ricplt-o1mediator-http ClusterIP 10.110.107.249 <none> 9001/TCP,8080/TCP,3000/TCP 22h app=ricplt-o1mediator,release=r4-o1mediator
ricplt service/service-ricplt-o1mediator-tcp-netconf NodePort 10.104.71.48 <none> 830:30830/TCP 22h app=ricplt-o1mediator,release=r4-o1mediator
ricplt service/service-ricplt-rtmgr-http ClusterIP 10.109.79.31 <none> 3800/TCP 22h app=ricplt-rtmgr,release=r4-rtmgr
ricplt service/service-ricplt-rtmgr-rmr ClusterIP 10.111.85.186 <none> 4561/TCP,4560/TCP 22h app=ricplt-rtmgr,release=r4-rtmgr
ricplt service/service-ricplt-submgr-http ClusterIP None <none> 3800/TCP 22h app=ricplt-submgr,release=r4-submgr
ricplt service/service-ricplt-submgr-rmr ClusterIP None <none> 4560/TCP,4561/TCP 22h app=ricplt-submgr,release=r4-submgr
ricplt service/service-ricplt-vespamgr-http ClusterIP 10.99.216.122 <none> 8080/TCP,9095/TCP 22h app=ricplt-vespamgr,release=r4-vespamgr
ricplt service/service-ricplt-xapp-onboarder-http ClusterIP 10.107.141.229 <none> 8080/TCP 22h app=deployment-ricplt-xapp-onboarder-http
ricxapp service/aux-entry ClusterIP 10.104.147.183 <none> 80/TCP,443/TCP 22h <none>
ricxapp service/service-ricxapp-hwxapp-rmr ClusterIP 10.101.205.57 <none> 4560/TCP,4561/TCP 25m app=ricxapp-hwxapp,release=hwxapp
NAMESPACE NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE CONTAINERS IMAGES SELECTOR
kube-system daemonset.apps/kube-flannel-ds 1 1 1 1 1 <none> 22h kube-flannel quay.io/coreos/flannel:v0.15.1 app=flannel
kube-system daemonset.apps/kube-proxy 1 1 1 1 1 kubernetes.io/os=linux 22h kube-proxy k8s.gcr.io/kube-proxy:v1.18.0 k8s-app=kube-proxy
NAMESPACE NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
kube-system deployment.apps/coredns 2/2 2 2 22h coredns k8s.gcr.io/coredns:1.6.7 k8s-app=kube-dns
ricinfra deployment.apps/deployment-tiller-ricxapp 1/1 1 1 22h tiller fishead/gcr.io.kubernetes-helm.tiller:v2.12.3 app=helm,name=tiller
ricplt deployment.apps/deployment-ricplt-a1mediator 1/1 1 1 22h container-ricplt-a1mediator nexus3.o-ran-sc.org:10002/o-ran-sc/ric-plt-a1:2.5.1 app=ricplt-a1mediator,release=r4-a1mediator
ricplt deployment.apps/deployment-ricplt-alarmmanager 1/1 1 1 22h container-ricplt-alarmmanager nexus3.o-ran-sc.org:10002/o-ran-sc/ric-plt-alarmmanager:0.5.11 app=ricplt-alarmmanager,release=r4-alarmmanager
ricplt deployment.apps/deployment-ricplt-appmgr 1/1 1 1 22h container-ricplt-appmgr nexus3.o-ran-sc.org:10002/o-ran-sc/ric-plt-appmgr:0.5.4 app=ricplt-appmgr,release=r4-appmgr
ricplt deployment.apps/deployment-ricplt-e2mgr 1/1 1 1 22h container-ricplt-e2mgr nexus3.o-ran-sc.org:10002/o-ran-sc/ric-plt-e2mgr:5.4.19 app=ricplt-e2mgr,release=r4-e2mgr
ricplt deployment.apps/deployment-ricplt-e2term-alpha 1/1 1 1 22h container-ricplt-e2term nexus3.o-ran-sc.org:10002/o-ran-sc/ric-plt-e2:5.5.0 app=ricplt-e2term-alpha,release=r4-e2term
ricplt deployment.apps/deployment-ricplt-o1mediator 1/1 1 1 22h container-ricplt-o1mediator nexus3.o-ran-sc.org:10002/o-ran-sc/ric-plt-o1:0.5.3 app=ricplt-o1mediator,release=r4-o1mediator
ricplt deployment.apps/deployment-ricplt-rtmgr 1/1 1 1 22h container-ricplt-rtmgr nexus3.o-ran-sc.org:10002/o-ran-sc/ric-plt-rtmgr:0.8.2 app=ricplt-rtmgr,release=r4-rtmgr
ricplt deployment.apps/deployment-ricplt-submgr 1/1 1 1 22h container-ricplt-submgr nexus3.o-ran-sc.org:10002/o-ran-sc/ric-plt-submgr:0.8.2 app=ricplt-submgr,release=r4-submgr
ricplt deployment.apps/deployment-ricplt-vespamgr 1/1 1 1 22h container-ricplt-vespamgr nexus3.o-ran-sc.org:10002/o-ran-sc/ric-plt-vespamgr:0.7.5 app=ricplt-vespamgr,release=r4-vespamgr
ricplt deployment.apps/deployment-ricplt-xapp-onboarder-http 1/1 1 1 22h chartmuseum chartmuseum/chartmuseum app=deployment-ricplt-xapp-onboarder-http
ricplt deployment.apps/r4-infrastructure-kong 1/1 1 1 22h ingress-controller,proxy kong/kubernetes-ingress-controller:0.7.0,kong:1.4 app.kubernetes.io/component=app,app.kubernetes.io/instance=r4-infrastructure,app.kubernetes.io/name=kong
ricplt deployment.apps/r4-infrastructure-prometheus-alertmanager 1/1 1 1 22h prometheus-alertmanager,prometheus-alertmanager-configmap-reload prom/alertmanager:v0.20.0,jimmidyson/configmap-reload:v0.3.0 app=prometheus,component=alertmanager,release=r4-infrastructure
ricplt deployment.apps/r4-infrastructure-prometheus-server 1/1 1 1 22h prometheus-server prom/prometheus:v2.18.1 app=prometheus,component=server,release=r4-infrastructure
ricxapp deployment.apps/ricxapp-hwxapp 1/1 1 1 25m hwxapp nexus3.o-ran-sc.org:10002/o-ran-sc/ric-app-hw:1.0.6 app=ricxapp-hwxapp,release=hwxapp
NAMESPACE NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR
kube-system replicaset.apps/coredns-66bff467f8 2 2 2 22h coredns k8s.gcr.io/coredns:1.6.7 k8s-app=kube-dns,pod-template-hash=66bff467f8
ricinfra replicaset.apps/deployment-tiller-ricxapp-5cd69f688b 1 1 1 22h tiller fishead/gcr.io.kubernetes-helm.tiller:v2.12.3 app=helm,name=tiller,pod-template-hash=5cd69f688b
ricinfra replicaset.apps/deployment-tiller-ricxapp-755fbd9fbc 0 0 0 22h tiller gcr.io/kubernetes-helm/tiller:v2.12.3 app=helm,name=tiller,pod-template-hash=755fbd9fbc
ricplt replicaset.apps/deployment-ricplt-a1mediator-687b54f649 1 1 1 22h container-ricplt-a1mediator nexus3.o-ran-sc.org:10002/o-ran-sc/ric-plt-a1:2.5.1 app=ricplt-a1mediator,pod-template-hash=687b54f649,release=r4-a1mediator
ricplt replicaset.apps/deployment-ricplt-alarmmanager-6dc48dbb97 1 1 1 22h container-ricplt-alarmmanager nexus3.o-ran-sc.org:10002/o-ran-sc/ric-plt-alarmmanager:0.5.11 app=ricplt-alarmmanager,pod-template-hash=6dc48dbb97,release=r4-alarmmanager
ricplt replicaset.apps/deployment-ricplt-appmgr-74d5f56b67 1 1 1 22h container-ricplt-appmgr nexus3.o-ran-sc.org:10002/o-ran-sc/ric-plt-appmgr:0.5.4 app=ricplt-appmgr,pod-template-hash=74d5f56b67,release=r4-appmgr
ricplt replicaset.apps/deployment-ricplt-e2mgr-5d7fddc7b4 1 1 1 22h container-ricplt-e2mgr nexus3.o-ran-sc.org:10002/o-ran-sc/ric-plt-e2mgr:5.4.19 app=ricplt-e2mgr,pod-template-hash=5d7fddc7b4,release=r4-e2mgr
ricplt replicaset.apps/deployment-ricplt-e2term-alpha-7f64b9cc44 1 1 1 22h container-ricplt-e2term nexus3.o-ran-sc.org:10002/o-ran-sc/ric-plt-e2:5.5.0 app=ricplt-e2term-alpha,pod-template-hash=7f64b9cc44,release=r4-e2term
ricplt replicaset.apps/deployment-ricplt-o1mediator-5c76d6577 1 1 1 22h container-ricplt-o1mediator nexus3.o-ran-sc.org:10002/o-ran-sc/ric-plt-o1:0.5.3 app=ricplt-o1mediator,pod-template-hash=5c76d6577,release=r4-o1mediator
ricplt replicaset.apps/deployment-ricplt-rtmgr-5fdbd9854f 1 1 1 22h container-ricplt-rtmgr nexus3.o-ran-sc.org:10002/o-ran-sc/ric-plt-rtmgr:0.8.2 app=ricplt-rtmgr,pod-template-hash=5fdbd9854f,release=r4-rtmgr
ricplt replicaset.apps/deployment-ricplt-submgr-67d4f45c96 1 1 1 22h container-ricplt-submgr nexus3.o-ran-sc.org:10002/o-ran-sc/ric-plt-submgr:0.8.2 app=ricplt-submgr,pod-template-hash=67d4f45c96,release=r4-submgr
ricplt replicaset.apps/deployment-ricplt-vespamgr-745b7968cd 1 1 1 22h container-ricplt-vespamgr nexus3.o-ran-sc.org:10002/o-ran-sc/ric-plt-vespamgr:0.7.5 app=ricplt-vespamgr,pod-template-hash=745b7968cd,release=r4-vespamgr
ricplt replicaset.apps/deployment-ricplt-xapp-onboarder-http-7dcdd9786b 1 1 1 22h chartmuseum chartmuseum/chartmuseum app=deployment-ricplt-xapp-onboarder-http,pod-template-hash=7dcdd9786b
ricplt replicaset.apps/r4-infrastructure-kong-7c974fb98b 1 1 1 22h ingress-controller,proxy kong/kubernetes-ingress-controller:0.7.0,kong:1.4 app.kubernetes.io/component=app,app.kubernetes.io/instance=r4-infrastructure,app.kubernetes.io/name=kong,pod-template-hash=7c974fb98b
ricplt replicaset.apps/r4-infrastructure-prometheus-alertmanager-757c4d8d66 1 1 1 22h prometheus-alertmanager,prometheus-alertmanager-configmap-reload prom/alertmanager:v0.20.0,jimmidyson/configmap-reload:v0.3.0 app=prometheus,component=alertmanager,pod-template-hash=757c4d8d66,release=r4-infrastructure
ricplt replicaset.apps/r4-infrastructure-prometheus-server-6b57df6c85 1 1 1 22h prometheus-server prom/prometheus:v2.18.1 app=prometheus,component=server,pod-template-hash=6b57df6c85,release=r4-infrastructure
ricxapp replicaset.apps/ricxapp-hwxapp-595cfc4b6 1 1 1 25m hwxapp nexus3.o-ran-sc.org:10002/o-ran-sc/ric-app-hw:1.0.6 app=ricxapp-hwxapp,pod-template-hash=595cfc4b6,release=hwxapp
NAMESPACE NAME READY AGE CONTAINERS IMAGES
ricplt statefulset.apps/statefulset-ricplt-dbaas-server 1/1 22h container-ricplt-dbaas-redis nexus3.o-ran-sc.org:10002/o-ran-sc/ric-plt-dbaas:0.5.5
NAMESPACE NAME COMPLETIONS DURATION AGE CONTAINERS IMAGES SELECTOR
ricinfra job.batch/tiller-secret-generator 1/1 14s 22h tiller-secret-generator nexus3.o-ran-sc.org:10002/o-ran-sc/it-dep-secret:0.0.2 controller-uid=c2876eea-a835-4173-a4ad-c66ea0616a82
Discussion
出来れば、Amazon EC2のインスタンス名についても言及して頂けるとサイジングの意味で助かります。