👻
【MS Learn】Azure Container Instances でサイドカー構成を使用してアプリのセキュリティを保護するをやってみた
タイトルが入力文字数限界です。ギリギリ。
今回は MS Learnの『Azure Container Instances でサイドカー構成を使用してアプリのセキュリティを保護する』をやってみた、です。
そのままコマンド実行していくはずなんですけど、うまく動かなかったりしたので、実際の挙動ログとして残しておきます。
コマンドは Azure Cloud Shell の Bash で実行します。
基本的にコマンドのところのみ書いていきますので、番号も本家に合わせています。
ユニット2: 仮想ネットワークに Azure Container Instances をデプロイする
環境を初期化する
- 次のコマンドを実行して、このユニットの変数を定義します。location は japaneast に変更。
rg=acilab
location=japaneast
aci_name=learnaci
aci_dns=${aci_name}${RANDOM}
vnet_name=acivnet
vnet_prefix=192.168.0.0/16
vm_subnet_name=vm
vm_subnet_prefix=192.168.1.0/24
aci_subnet_name=aci
aci_subnet_prefix=192.168.2.0/24
- テスト目的で使用するリソース グループと仮想マシンを作成します。
$ az group create -n $rg -l $location
{
"id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab",
"location": "japaneast",
"managedBy": null,
"name": "acilab",
"properties": {
"provisioningState": "Succeeded"
},
"tags": null,
"type": "Microsoft.Resources/resourceGroups"
}
$ az vm create -n test-vm -g $rg -l $location --image ubuntuLTS --generate-ssh-keys \
--public-ip-address test-vm-pip --vnet-name $vnet_name \
--vnet-address-prefix $vnet_prefix --subnet $vm_subnet_name --subnet-address-prefix $vm_subnet_prefix
{
"fqdns": "",
"id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Compute/virtualMachines/test-vm",
"location": "japaneast",
"macAddress": "00-22-48-E7-D4-18",
"powerState": "VM running",
"privateIpAddress": "192.168.1.4",
"publicIpAddress": "20.78.102.185",
"resourceGroup": "acilab",
"zones": ""
}
$ vm_pip=$(az network public-ip show -n test-vm-pip -g $rg --query ipAddress -o tsv) && echo $vm_pip
20.78.102.185
ssh 接続で確認
$ ssh $vm_pip
The authenticity of host '20.78.102.185 (20.78.102.185)' can't be established.
ED25519 key fingerprint is SHA256:OmpGWK7+3PtXN4u3iZFcXzfKtp8chN2QZoMKr1QXLPc.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '20.78.102.185' (ED25519) to the list of known hosts.
Welcome to Ubuntu 18.04.6 LTS (GNU/Linux 5.4.0-1109-azure x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
System information as of Thu Sep 28 05:47:20 UTC 2023
System load: 0.19 Processes: 109
Usage of /: 4.5% of 28.89GB Users logged in: 0
Memory usage: 5% IP address for eth0: 192.168.1.4
Swap usage: 0%
Expanded Security Maintenance for Infrastructure is not enabled.
0 updates can be applied immediately.
Enable ESM Infra to receive additional future security updates.
See https://ubuntu.com/esm or run: sudo pro status
The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.
To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.
*****@test-vm:~$ exit
logout
Connection to 20.78.102.185 closed.
- アプリケーションの接続先となるデータベースを作成します。
$ sql_server_name=sqlserver$RANDOM
sql_db_name=mydb
sql_username=azure
sql_password=Microsoft123!
$ az sql server create -n $sql_server_name -g $rg -l $location --admin-user $sql_username --admin-password $sql_password
{
"administratorLogin": "azure",
"administratorLoginPassword": null,
"administrators": null,
"externalGovernanceStatus": "Disabled",
"federatedClientId": null,
"fullyQualifiedDomainName": "sqlserver15595.database.windows.net",
"id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Sql/servers/sqlserver15595",
"identity": null,
"keyId": null,
"kind": "v12.0",
"location": "japaneast",
"minimalTlsVersion": "None",
"name": "sqlserver15595",
"primaryUserAssignedIdentityId": null,
"privateEndpointConnections": [],
"publicNetworkAccess": "Enabled",
"resourceGroup": "acilab",
"restrictOutboundNetworkAccess": "Disabled",
"state": "Ready",
"tags": null,
"type": "Microsoft.Sql/servers",
"version": "12.0",
"workspaceFeature": null
}
$ sql_server_fqdn=$(az sql server show -n $sql_server_name -g $rg -o tsv --query fullyQualifiedDomainName)
$ az sql db create -n $sql_db_name -s $sql_server_name -g $rg -e Basic -c 5 --no-wait
パスワードポリシーが厳しくなったのか、記号がないと怒られるので、後続で出てくるパスワードをセットします。
(PasswordNotComplex) Password validation failed. The password does not meet policy requirements because it is not complex enough.
Code: PasswordNotComplex
Message: Password validation failed. The password does not meet policy requirements because it is not complex enough.
仮想ネットワーク内に Azure Container Instance を作成する
- 環境変数に、Azure SQL Database の完全修飾ドメイン名と資格情報を指定して、Azure Container Instance がそれに接続できるようにします。
$ az network vnet subnet create -g $rg --vnet-name $vnet_name -n $aci_subnet_name --address-prefix $aci_subnet_prefix
{
"addressPrefix": "192.168.2.0/24",
"delegations": [],
"etag": "W/\"08b0d45e-fdd6-42fc-95b2-34ef48960c0d\"",
"id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Network/virtualNetworks/acivnet/subnets/aci",
"name": "aci",
"privateEndpointNetworkPolicies": "Disabled",
"privateLinkServiceNetworkPolicies": "Enabled",
"provisioningState": "Succeeded",
"resourceGroup": "acilab",
"type": "Microsoft.Network/virtualNetworks/subnets"
}
$ vnet_id=$(az network vnet show -n $vnet_name -g $rg --query id -o tsv)
$ aci_subnet_id=$(az network vnet subnet show -n $aci_subnet_name --vnet-name $vnet_name -g $rg --query id -o tsv)
$ $ az container create -n $aci_name -g $rg -e "SQL_SERVER_USERNAME=$sql_username" \
"SQL_SERVER_PASSWORD=$sql_password" \
"SQL_SERVER_FQDN=${sql_server_fqdn}" \
--image erjosito/sqlapi:1.0 \
--ip-address private --ports 8080 --vnet $vnet_id --subnet $aci_subnet_id
{
"confidentialComputeProperties": null,
"containers": [
{
"command": null,
"environmentVariables": [
{
"name": "SQL_SERVER_USERNAME",
"secureValue": null,
"value": "azure"
},
{
"name": "SQL_SERVER_PASSWORD",
"secureValue": null,
"value": "Microsoft123!"
},
{
"name": "SQL_SERVER_FQDN",
"secureValue": null,
"value": "sqlserver15595.database.windows.net"
}
],
"image": "erjosito/sqlapi:1.0",
"instanceView": {
"currentState": {
"detailStatus": "",
"exitCode": null,
"finishTime": null,
"startTime": "2023-09-28T05:54:03.672000+00:00",
"state": "Running"
},
"events": [
{
"count": 1,
"firstTimestamp": "2023-09-28T05:53:31+00:00",
"lastTimestamp": "2023-09-28T05:53:31+00:00",
"message": "pulling image \"erjosito/sqlapi@sha256:a1233cdfcd53fe513fe0ea2920e3fa9bb388b5722e28ee19c218361845206899\"",
"name": "Pulling",
"type": "Normal"
},
{
"count": 1,
"firstTimestamp": "2023-09-28T05:53:51+00:00",
"lastTimestamp": "2023-09-28T05:53:51+00:00",
"message": "Successfully pulled image \"erjosito/sqlapi@sha256:a1233cdfcd53fe513fe0ea2920e3fa9bb388b5722e28ee19c218361845206899\"",
"name": "Pulled",
"type": "Normal"
},
{
"count": 1,
"firstTimestamp": "2023-09-28T05:54:03+00:00",
"lastTimestamp": "2023-09-28T05:54:03+00:00",
"message": "Started container",
"name": "Started",
"type": "Normal"
}
],
"previousState": null,
"restartCount": 0
},
"livenessProbe": null,
"name": "learnaci",
"ports": [
{
"port": 8080,
"protocol": "TCP"
}
],
"readinessProbe": null,
"resources": {
"limits": null,
"requests": {
"cpu": 1.0,
"gpu": null,
"memoryInGb": 1.5
}
},
"securityContext": null,
"volumeMounts": null
}
],
"diagnostics": null,
"dnsConfig": null,
"encryptionProperties": null,
"extensions": null,
"id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.ContainerInstance/containerGroups/learnaci",
"identity": null,
"imageRegistryCredentials": null,
"initContainers": [],
"instanceView": {
"events": [],
"state": "Running"
},
"ipAddress": {
"autoGeneratedDomainNameLabelScope": null,
"dnsNameLabel": null,
"fqdn": null,
"ip": "192.168.2.4",
"ports": [
{
"port": 8080,
"protocol": "TCP"
}
],
"type": "Private"
},
"location": "japaneast",
"name": "learnaci",
"osType": "Linux",
"priority": null,
"provisioningState": "Succeeded",
"resourceGroup": "acilab",
"restartPolicy": "Always",
"sku": "Standard",
"subnetIds": [
{
"id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Network/virtualNetworks/acivnet/subnets/aci",
"name": null,
"resourceGroup": "acilab"
}
],
"tags": {},
"type": "Microsoft.ContainerInstance/containerGroups",
"volumes": null,
"zones": null
}
- 仮想マシンに接続できるかAPIを叩いて確認。
$ aci_ip=$(az container show -n $aci_name -g $rg --query 'ipAddress.ip' -o tsv) && echo $aci_ip
ssh -n -o BatchMode=yes -o StrictHostKeyChecking=no $vm_pip "curl -s http://$aci_ip:8080/api/healthcheck"
192.168.2.4
{
"health": "OK",
"version": "1.0"
}
- Azure SQL ファイアウォール ルールを更新。APIを叩いてアクセスできるか確認。
$ aci_pip=$(ssh -n -o BatchMode=yes -o StrictHostKeyChecking=no $vm_pip "curl -s http://$aci_ip:8080/api/ip" | jq -r .my_public_ip) && echo $aci_pip
az sql server firewall-rule create -g $rg -s $sql_server_name -n public_sqlapi_aci-source --start-ip-address $aci_pip --end-ip-address $aci_pip
ssh -n -o BatchMode=yes -o StrictHostKeyChecking=no $vm_pip "curl -s http://$aci_ip:8080/api/sqlversion"
ssh -n -o BatchMode=yes -o StrictHostKeyChecking=no $vm_pip "curl -s http://$aci_ip:8080/api/sqlsrcip"
20.44.190.204
{
"endIpAddress": "20.44.190.204",
"id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Sql/servers/sqlserver11156/firewallRules/public_sqlapi_aci-source",
"name": "public_sqlapi_aci-source",
"resourceGroup": "acilab",
"startIpAddress": "20.44.190.204",
"type": "Microsoft.Sql/servers/firewallRules"
}
{
"sql_output": "Microsoft SQL Azure (RTM) - 12.0.2000.8 \n\tSep 18 2023 12:22:37 \n\tCopyright (C) 2022 Microsoft Corporation\n"
}
{
"sql_output": "20.44.190.204"
}
- コンテナの削除
$ az container delete -n $aci_name -g $rg -y
{
"confidentialComputeProperties": null,
"containers": [
{
"command": null,
"environmentVariables": [
{
"name": "SQL_SERVER_USERNAME",
"secureValue": null,
"value": "azure"
},
{
"name": "SQL_SERVER_PASSWORD",
"secureValue": null,
"value": "Microsoft123!"
},
{
"name": "SQL_SERVER_FQDN",
"secureValue": null,
"value": "sqlserver11156.database.windows.net"
}
],
"image": "erjosito/sqlapi:1.0",
"instanceView": {
"currentState": {
"detailStatus": "",
"exitCode": null,
"finishTime": null,
"startTime": "2023-09-28T02:18:35.543000+00:00",
"state": "Running"
},
"events": [
{
"count": 1,
"firstTimestamp": "2023-09-28T02:18:01+00:00",
"lastTimestamp": "2023-09-28T02:18:01+00:00",
"message": "pulling image \"erjosito/sqlapi@sha256:a1233cdfcd53fe513fe0ea2920e3fa9bb388b5722e28ee19c218361845206899\"",
"name": "Pulling",
"type": "Normal"
},
{
"count": 1,
"firstTimestamp": "2023-09-28T02:18:24+00:00",
"lastTimestamp": "2023-09-28T02:18:24+00:00",
"message": "Successfully pulled image \"erjosito/sqlapi@sha256:a1233cdfcd53fe513fe0ea2920e3fa9bb388b5722e28ee19c218361845206899\"",
"name": "Pulled",
"type": "Normal"
},
{
"count": 1,
"firstTimestamp": "2023-09-28T02:18:35+00:00",
"lastTimestamp": "2023-09-28T02:18:35+00:00",
"message": "Started container",
"name": "Started",
"type": "Normal"
}
],
"previousState": null,
"restartCount": 0
},
"livenessProbe": null,
"name": "learnaci",
"ports": [
{
"port": 8080,
"protocol": "TCP"
}
],
"readinessProbe": null,
"resources": {
"limits": null,
"requests": {
"cpu": 1.0,
"gpu": null,
"memoryInGb": 1.5
}
},
"securityContext": null,
"volumeMounts": null
}
],
"diagnostics": null,
"dnsConfig": null,
"encryptionProperties": null,
"extensions": null,
"id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.ContainerInstance/containerGroups/learnaci",
"identity": null,
"imageRegistryCredentials": null,
"initContainers": [],
"instanceView": {
"events": [],
"state": "Running"
},
"ipAddress": {
"autoGeneratedDomainNameLabelScope": null,
"dnsNameLabel": null,
"fqdn": null,
"ip": "192.168.2.4",
"ports": [
{
"port": 8080,
"protocol": "TCP"
}
],
"type": "Private"
},
"location": "japaneast",
"name": "learnaci",
"osType": "Linux",
"priority": null,
"provisioningState": "Succeeded",
"resourceGroup": "acilab",
"restartPolicy": "Always",
"sku": "Standard",
"subnetIds": [
{
"id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Network/virtualNetworks/acivnet/subnets/aci",
"name": null,
"resourceGroup": "acilab"
}
],
"tags": {},
"type": "Microsoft.ContainerInstance/containerGroups",
"volumes": null,
"zones": null
}
ユニット3: Azure Container Instances YAML 定義について
既存のコンテナー グループから YAML コードを抽出する
- コンテナの再作成(さっき消さなかった場合)
$ az container create -n $aci_name -g $rg -e "SQL_SERVER_USERNAME=$sql_username" \
"SQL_SERVER_PASSWORD=$sql_password" \
"SQL_SERVER_FQDN=${sql_server_fqdn}" \
--image erjosito/sqlapi:1.0 \
--ip-address private --ports 8080 --vnet $vnet_id --subnet $aci_subnet_id
- YAML コードをファイルに保存
$ az container export -n $aci_name -g $rg -f /tmp/aci.yaml
$ more /tmp/aci.yaml
/tmp/aci.yaml
additional_properties: {}
apiVersion: '2023-05-01'
extended_location: null
location: japaneast
name: learnaci
properties:
containers:
- name: learnaci
properties:
environmentVariables:
- name: SQL_SERVER_USERNAME
value: azure
- name: SQL_SERVER_PASSWORD
value: Microsoft123!
- name: SQL_SERVER_FQDN
value: sqlserver15595.database.windows.net
image: erjosito/sqlapi:1.0
ports:
- port: 8080
protocol: TCP
resources:
requests:
cpu: 1.0
memoryInGB: 1.5
initContainers: []
ipAddress:
ip: 192.168.2.4
ports:
- port: 8080
protocol: TCP
type: Private
isCustomProvisioningTimeout: false
osType: Linux
provisioningTimeoutInSeconds: 1800
restartPolicy: Always
sku: Standard
subnetIds:
- id: /subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Network/virtualNetworks/acivnet/subnets
/aci
tags: {}
type: Microsoft.ContainerInstance/containerGroups
YAML ファイルを変更してデプロイする
- 自動的に生成されたファイルを変更する
$ sed -i 's/ value: Microsoft123!/ secureValue: Microsoft123!/g' /tmp/aci.yaml
- 再デプロイする
$ az container delete -n $aci_name -g $rg -y
$ az container create -g $rg --file /tmp/aci.yaml
けど、API versionが古くて？サポートしてないというエラーがでる。
(CustomProvisioningTimeoutInSecondsNotSuppported) Custom provisioning timeout is only supported for API version '2023-02-01-preview'.
Code: CustomProvisioningTimeoutInSecondsNotSuppported
Message: Custom provisioning timeout is only supported for API version '2023-02-01-preview'.
/tmp/aci.yaml の provisioningTimeoutInSeconds の行をコメントアウトする。
$ vi /tmp/aci.yaml
# provisioningTimeoutInSeconds: 1800
再デプロイ
$ az container create -g $rg --file /tmp/aci.yaml
- 新しい Azure Container Instance を別の YAML ファイル /tmp/aci2.yaml にエクスポート
$ az container export -n $aci_name -g $rg -f /tmp/aci2.yaml
diff を取るとパスワードが出力されないのがわかる
$ diff /tmp/aci.yaml /tmp/aci2.yaml
13,14d12
< - name: SQL_SERVER_PASSWORD
< secureValue: Microsoft123!
16a15
> - name: SQL_SERVER_PASSWORD
34c33
< # provisioningTimeoutInSeconds: 1800
---
> provisioningTimeoutInSeconds: 1800
- コンテナーを削除
az container delete -n $aci_name -g $rg -y
ユニット4: サイドカー コンテナーで Azure Container Instances をデプロイする
NGINX 構成を作成する
- 自己署名証明書を作成する
$ openssl req -new -newkey rsa:2048 -nodes -keyout ssl.key -out ssl.csr -subj "/C=US/ST=WA/L=Redmond/O=AppDev/OU=IT/CN=contoso.com"
Generating a RSA private key
...........+++++
...............................................................+++++
writing new private key to 'ssl.key'
-----
$ openssl x509 -req -days 365 -in ssl.csr -signkey ssl.key -out ssl.crt
Signature ok
subject=C = US, ST = WA, L = Redmond, O = AppDev, OU = IT, CN = contoso.com
Getting Private key
- nginx.conf を作成する
$ nginx_config_file=/tmp/nginx.conf
$ cat <<EOF > $nginx_config_file
user nginx;
worker_processes auto;
events {
worker_connections 1024;
}
pid /var/run/nginx.pid;
http {
server {
listen [::]:443 ssl;
listen 443 ssl;
server_name localhost;
ssl_protocols TLSv1.2;
ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:ECDHE-RSA-RC4-SHA:ECDHE-ECDSA-RC4-SHA:AES128:AES256:RC4-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!3DES:!MD5:!PSK;
ssl_prefer_server_ciphers on;
ssl_session_cache shared:SSL:10m; # a 1mb cache can hold about 4000 sessions, so we can hold 40000 sessions
ssl_session_timeout 24h;
keepalive_timeout 75; # up from 75 secs default
add_header Strict-Transport-Security 'max-age=31536000; includeSubDomains';
ssl_certificate /etc/nginx/ssl.crt;
ssl_certificate_key /etc/nginx/ssl.key;
location / {
proxy_pass http://127.0.0.1:8080 ;
proxy_set_header Connection "";
proxy_set_header Host \$host;
proxy_set_header X-Real-IP \$remote_addr;
proxy_set_header X-Forwarded-For \$remote_addr;
proxy_buffer_size 128k;
proxy_buffers 4 256k;
proxy_busy_buffers_size 256k;
}
}
}
EOF
- NGINX サイドカー コンテナーに渡す必要があるファイルを収集する
$ nginx_conf=$(cat $nginx_config_file | base64)
$ ssl_crt=$(cat ssl.crt | base64)
$ ssl_key=$(cat ssl.key | base64)
NGINX サイドカーによってコンテナー グループをデプロイする
1 & 2.
ここで、network profile を取得するという流れになっているんですが、コマンド実行しても空が返ってくるし、yamlにも出力されてないし、とだいぶハマったんですが、network profile は廃止されたようです。
代わりにサブネットを指定するようにします。
$aci_subnet_id は前に変数にセットしてますね。
$ aci_yaml_file=/tmp/aci_ssl.yaml
cat <<EOF > $aci_yaml_file
apiVersion: 2021-07-01
location: japaneast
name: $aci_name
properties:
containers:
- name: nginx
properties:
image: nginx
ports:
- port: 443
protocol: TCP
resources:
requests:
cpu: 1.0
memoryInGB: 1.5
volumeMounts:
- name: nginx-config
mountPath: /etc/nginx
- name: sqlapi
properties:
image: erjosito/sqlapi:1.0
environmentVariables:
- name: SQL_SERVER_USERNAME
value: $sql_username
- name: SQL_SERVER_PASSWORD
secureValue: $sql_password
- name: SQL_SERVER_FQDN
value: $sql_server_fqdn
ports:
- port: 8080
protocol: TCP
resources:
requests:
cpu: 1.0
memoryInGB: 1
volumes:
- secret:
ssl.crt: "$ssl_crt"
ssl.key: "$ssl_key"
nginx.conf: "$nginx_conf"
name: nginx-config
ipAddress:
ports:
- port: 443
protocol: TCP
type: Private
osType: Linux
subnetIds:
- id: $aci_subnet_id
name: aci
tags: null
type: Microsoft.ContainerInstance/containerGroups
EOF
- 作成したyamlからContainer Instance を作成します。
$ az container create -g $rg --file $aci_yaml_file
{
"extendedLocation": null,
"id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.ContainerInstance/containerGroups/learnaci",
"identity": null,
"kind": null,
"location": "japaneast",
"managedBy": null,
"name": "learnaci",
"plan": null,
"properties": {
"containers": [
{
"name": "nginx",
"properties": {
"environmentVariables": [],
"image": "nginx",
"instanceView": {
"currentState": {
"detailStatus": "",
"startTime": "2023-09-28T06:35:16.141Z",
"state": "Running"
},
"events": [
{
"count": 1,
"firstTimestamp": "2023-09-28T06:34:35Z",
"lastTimestamp": "2023-09-28T06:34:35Z",
"message": "pulling image \"nginx@sha256:b2888fc9cfe7cd9d6727aeb462d13c7c45dec413b66f2819a36c4a3cb9d4df75\"",
"name": "Pulling",
"type": "Normal"
},
{
"count": 1,
"firstTimestamp": "2023-09-28T06:34:57Z",
"lastTimestamp": "2023-09-28T06:34:57Z",
"message": "Successfully pulled image \"nginx@sha256:b2888fc9cfe7cd9d6727aeb462d13c7c45dec413b66f2819a36c4a3cb9d4df75\"",
"name": "Pulled",
"type": "Normal"
},
{
"count": 1,
"firstTimestamp": "2023-09-28T06:35:16Z",
"lastTimestamp": "2023-09-28T06:35:16Z",
"message": "Started container",
"name": "Started",
"type": "Normal"
}
],
"restartCount": 0
},
"ports": [
{
"port": 443,
"protocol": "TCP"
}
],
"resources": {
"requests": {
"cpu": 1.0,
"memoryInGB": 1.5
}
},
"volumeMounts": [
{
"mountPath": "/etc/nginx",
"name": "nginx-config"
}
]
}
},
{
"name": "sqlapi",
"properties": {
"environmentVariables": [
{
"name": "SQL_SERVER_USERNAME",
"value": "azure"
},
{
"name": "SQL_SERVER_FQDN",
"value": "sqlserver15595.database.windows.net"
},
{
"name": "SQL_SERVER_PASSWORD"
}
],
"image": "erjosito/sqlapi:1.0",
"instanceView": {
"currentState": {
"detailStatus": "",
"startTime": "2023-09-28T06:35:16.398Z",
"state": "Running"
},
"events": [
{
"count": 1,
"firstTimestamp": "2023-09-28T06:34:35Z",
"lastTimestamp": "2023-09-28T06:34:35Z",
"message": "pulling image \"erjosito/sqlapi@sha256:a1233cdfcd53fe513fe0ea2920e3fa9bb388b5722e28ee19c218361845206899\"",
"name": "Pulling",
"type": "Normal"
},
{
"count": 1,
"firstTimestamp": "2023-09-28T06:34:57Z",
"lastTimestamp": "2023-09-28T06:34:57Z",
"message": "Successfully pulled image \"erjosito/sqlapi@sha256:a1233cdfcd53fe513fe0ea2920e3fa9bb388b5722e28ee19c218361845206899\"",
"name": "Pulled",
"type": "Normal"
},
{
"count": 1,
"firstTimestamp": "2023-09-28T06:35:16Z",
"lastTimestamp": "2023-09-28T06:35:16Z",
"message": "Started container",
"name": "Started",
"type": "Normal"
}
],
"restartCount": 0
},
"ports": [
{
"port": 8080,
"protocol": "TCP"
}
],
"resources": {
"requests": {
"cpu": 1.0,
"memoryInGB": 1.0
}
}
}
}
],
"initContainers": [],
"instanceView": {
"events": [],
"state": "Running"
},
"ipAddress": {
"ip": "192.168.2.4",
"ports": [
{
"port": 443,
"protocol": "TCP"
},
{
"port": 8080,
"protocol": "TCP"
}
],
"type": "Private"
},
"isCustomProvisioningTimeout": false,
"osType": "Linux",
"provisioningState": "Succeeded",
"provisioningTimeoutInSeconds": 1800,
"sku": "Standard",
"subnetIds": [
{
"id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Network/virtualNetworks/acivnet/subnets/aci",
"name": "aci",
"resourceGroup": "acilab"
}
],
"volumes": [
{
"name": "nginx-config",
"secret": {}
}
]
},
"resourceGroup": "acilab",
"sku": null,
"tags": null,
"type": "Microsoft.ContainerInstance/containerGroups"
}
portalから Container Instance -> [設定：コンテナー] で確認すると、コンテナが2つになっていました。
- テスト仮想マシンから HTTPS 経由でアクセス
$ aci_ip=$(az container show -n $aci_name -g $rg --query 'ipAddress.ip' -o tsv) && echo $aci_ip
192.168.2.4
$ ssh -n -o BatchMode=yes -o StrictHostKeyChecking=no $vm_pip "curl -ks https://$aci_ip/api/healthcheck"
{
"health": "OK",
"version": "1.0"
}
ユニット5：Azure Container Instances からプライベート リンク対応サービスにアクセスする
プライベート エンドポイントの作成
- 仮想ネットワークに新しいサブネットを作成し、次にそのサブネットに Azure SQL プライベート エンドポイントを作成します。
$ sql_subnet_name=sql
$ sql_subnet_prefix=192.168.3.0/24
$ az network vnet subnet create --resource-group $rg --vnet-name $vnet_name \
--name $sql_subnet_name --address-prefix $sql_subnet_prefix \
--disable-private-endpoint-network-policies true
{
"addressPrefix": "192.168.3.0/24",
"delegations": [],
"etag": "W/\"574371d2-0023-4dec-b9e4-bce5bf86da4b\"",
"id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Network/virtualNetworks/acivnet/subnets/sql",
"name": "sql",
"privateEndpointNetworkPolicies": "Disabled",
"privateLinkServiceNetworkPolicies": "Enabled",
"provisioningState": "Succeeded",
"resourceGroup": "acilab",
"type": "Microsoft.Network/virtualNetworks/subnets"
}
$ sql_endpoint_name=sqlep
$ sql_server_id=$(az sql server show --name $sql_server_name --resource-group $rg --output tsv --query id)
$ az network private-endpoint create --name $sql_endpoint_name --resource-group $rg \
--vnet-name $vnet_name --subnet $sql_subnet_name \
--private-connection-resource-id $sql_server_id --group-id sqlServer \
--connection-name sqlConnection
{
"customDnsConfigs": [
{
"fqdn": "sqlserver15595.database.windows.net",
"ipAddresses": [
"192.168.3.4"
]
}
],
"customNetworkInterfaceName": "",
"etag": "W/\"ddf4dab7-3219-402b-a848-9e8c0ccb45d5\"",
"id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Network/privateEndpoints/sqlep",
"ipConfigurations": [],
"location": "japaneast",
"manualPrivateLinkServiceConnections": [],
"name": "sqlep",
"networkInterfaces": [
{
"id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Network/networkInterfaces/sqlep.nic.5c6311f4-c5fe-4533-b968-7107f16bff35",
"resourceGroup": "acilab"
}
],
"privateLinkServiceConnections": [
{
"etag": "W/\"ddf4dab7-3219-402b-a848-9e8c0ccb45d5\"",
"groupIds": [
"sqlServer"
],
"id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Network/privateEndpoints/sqlep/privateLinkServiceConnections/sqlConnection",
"name": "sqlConnection",
"privateLinkServiceConnectionState": {
"actionsRequired": "None",
"description": "Auto-approved",
"status": "Approved"
},
"privateLinkServiceId": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Sql/servers/sqlserver15595",
"provisioningState": "Succeeded",
"resourceGroup": "acilab",
"type": "Microsoft.Network/privateEndpoints/privateLinkServiceConnections"
}
],
"provisioningState": "Succeeded",
"resourceGroup": "acilab",
"subnet": {
"id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Network/virtualNetworks/acivnet/subnets/sql",
"resourceGroup": "acilab"
},
"type": "Microsoft.Network/privateEndpoints"
- プライベート エンドポイントは、Azure でネットワーク インターフェイス カード (NIC) として表されるため、プライベート エンドポイントに割り当てられた IP アドレスは、az network nic コマンドを使用して確認できます。
あ、そうなんですね。nic扱いなのか。言われてみればそうか。
$ sql_nic_id=$(az network private-endpoint show --name $sql_endpoint_name \
--resource-group $rg --query 'networkInterfaces[0].id' -o tsv)
$ sql_endpoint_ip=$(az network nic show --ids $sql_nic_id \
--query 'ipConfigurations[0].privateIpAddress' -o tsv) && echo $sql_endpoint_ip
$ ssh -n -o BatchMode=yes -o StrictHostKeyChecking=no $vm_pip "nslookup ${sql_server_name}.database.windows.net"
Server: 127.0.0.53
Address: 127.0.0.53#53
Non-authoritative answer:
sqlserver15595.database.windows.net canonical name = sqlserver15595.privatelink.database.windows.net.
sqlserver15595.privatelink.database.windows.net canonical name = dataslice6.japaneast.database.windows.net.
dataslice6.japaneast.database.windows.net canonical name = dataslice6japaneast.trafficmanager.net.
dataslice6japaneast.trafficmanager.net canonical name = cr10.japaneast1-a.control.database.windows.net.
Name: cr10.japaneast1-a.control.database.windows.net
Address: 13.78.104.32
DNS の解決
- プライベート リンクが構成された Azure SQL Database では中間ドメイン privatelink.database.windows.net が使用されるため、このドメインのプライベート ゾーンを作成し、先に作成した Azure SQL プライベート エンドポイントの IP アドレスの A レコードを追加します。
$ dns_zone_name=privatelink.database.windows.net
$ az network private-dns zone create --name $dns_zone_name --resource-group $rg
{
"etag": "6e4cc59c-0451-48f7-afc1-a3e6843178aa",
"id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Network/privateDnsZones/privatelink.database.windows.net",
"location": "global",
"maxNumberOfRecordSets": 25000,
"maxNumberOfVirtualNetworkLinks": 1000,
"maxNumberOfVirtualNetworkLinksWithRegistration": 100,
"name": "privatelink.database.windows.net",
"numberOfRecordSets": 1,
"numberOfVirtualNetworkLinks": 0,
"numberOfVirtualNetworkLinksWithRegistration": 0,
"provisioningState": "Succeeded",
"resourceGroup": "acilab",
"type": "Microsoft.Network/privateDnsZones"
}
$ az network private-dns link vnet create --resource-group $rg --zone-name $dns_zone_name \
--name myDnsLink --virtual-network $vnet_name --registration-enabled false
{
"etag": "\"1d04dd01-0000-0100-0000-651525390000\"",
"id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Network/privateDnsZones/privatelink.database.windows.net/virtualNetworkLinks/mydnslink",
"location": "global",
"name": "mydnslink",
"provisioningState": "Succeeded",
"registrationEnabled": false,
"resourceGroup": "acilab",
"type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks",
"virtualNetwork": {
"id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Network/virtualNetworks/acivnet",
"resourceGroup": "acilab"
},
"virtualNetworkLinkState": "Completed"
}
$ az network private-endpoint dns-zone-group create --endpoint-name $sql_endpoint_name \
--resource-group $rg --name zonegroup --zone-name zone1 --private-dns-zone $dns_zone_name
{
"etag": "W/\"06904254-a4df-42c1-a4bd-9cccb99837e2\"",
"id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Network/privateEndpoints/sqlep/privateDnsZoneGroups/zonegroup",
"name": "zonegroup",
"privateDnsZoneConfigs": [
{
"name": "zone1",
"privateDnsZoneId": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Network/privateDnsZones/privatelink.database.windows.net",
"recordSets": [
{
"fqdn": "sqlserver15595.privatelink.database.windows.net",
"ipAddresses": [
"192.168.3.4"
],
"provisioningState": "Succeeded",
"recordSetName": "sqlserver15595",
"recordType": "A",
"ttl": 10
}
]
}
],
"provisioningState": "Succeeded",
"resourceGroup": "acilab"
}
$ ssh -n -o BatchMode=yes -o StrictHostKeyChecking=no $vm_pip "nslookup ${sql_server_name}.database.windows.net"
Server: 127.0.0.53
Address: 127.0.0.53#53
Non-authoritative answer:
sqlserver15595.database.windows.net canonical name = sqlserver15595.privatelink.database.windows.net.
Name: sqlserver15595.privatelink.database.windows.net
Address: 192.168.3.4
- 各種接続確認
$ aci_ip=$(az container show --name $aci_name --resource-group $rg \
--query 'ipAddress.ip' --output tsv) && echo $aci_ip
192.168.2.4
$ ssh -n -o BatchMode=yes -o StrictHostKeyChecking=no $vm_pip "curl -ks https://$aci_ip/api/healthcheck"
{
"health": "OK",
"version": "1.0"
}
$ ssh -n -o BatchMode=yes -o StrictHostKeyChecking=no $vm_pip "curl -ks https://$aci_ip/api/dns?fqdn=${sql_server_name}.database.windows.net"
{
"fqdn": "sqlserver15595.database.windows.net",
"ip": "192.168.3.4"
}
$ ssh -n -o BatchMode=yes -o StrictHostKeyChecking=no $vm_pip "curl -ks https://$aci_ip/api/sqlversion"
{
"sql_output": "Microsoft SQL Azure (RTM) - 12.0.2000.8 \n\tJul 17 2023 18:40:52 \n\tCopyright (C) 2022 Microsoft Corporation\n"
}
$ ssh -n -o BatchMode=yes -o StrictHostKeyChecking=no $vm_pip "curl -ks https://$aci_ip/api/sqlsrcip"
{
"sql_output": "192.168.2.4"
}
ここまでのトポロジーはこれ。
- コンテナの削除
$ az container delete --name $aci_name --resource-group $rg --yes
ユニット6: 初期化コンテナーで Azure Container Instances をデプロイする
初期化スクリプトを作成する
- Azure サービス プリンシパルを作成。ここではContributerにしているけど、運用環境ではもうちょっと小さいロールにしてね、と言っている。
$ scope=$(az group show -n $rg --query id -o tsv)
$ new_sp=$(az ad sp create-for-rbac --scopes $scope --role Contributor --name acilab -o json)
WARNING: Creating 'Contributor' role assignment under scope '/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab'
WARNING: The output includes credentials that you must protect. Be sure that you do not include these credentials in your code or check the credentials into your source control. For more information, see https://aka.ms/azadsp-cli
$ sp_appid=$(echo $new_sp | jq -r '.appId') && echo $sp_appid
7dcd731d-b8ef-4bf9-8c83-e456d623024d
$ sp_tenant=$(echo $new_sp | jq -r '.tenant') && echo $sp_tenant
cb9bb346-c037-4fb2-a3ff-dd23544753e
$ sp_password=$(echo $new_sp | jq -r '.password')
- Azure プライベート DNS ゾーンを作成し、それを仮想ネットワークに関連付けます。
$ dns_zone_name=contoso.com
$ az network private-dns zone create -n $dns_zone_name -g $rg
{
"etag": "2200f2e9-7666-4075-b563-1eb68963d41e",
"id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Network/privateDnsZones/contoso.com",
"location": "global",
"maxNumberOfRecordSets": 25000,
"maxNumberOfVirtualNetworkLinks": 1000,
"maxNumberOfVirtualNetworkLinksWithRegistration": 100,
"name": "contoso.com",
"numberOfRecordSets": 1,
"numberOfVirtualNetworkLinks": 0,
"numberOfVirtualNetworkLinksWithRegistration": 0,
"provisioningState": "Succeeded",
"resourceGroup": "acilab",
"type": "Microsoft.Network/privateDnsZones"
}
$ az network private-dns link vnet create -g $rg -z $dns_zone_name -n contoso --virtual-network $vnet_name --registration-enabled false
{
"etag": "\"1e04740f-0000-0100-0000-65152af70000\"",
"id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Network/privateDnsZones/contoso.com/virtualNetworkLinks/contoso",
"location": "global",
"name": "contoso",
"provisioningState": "Succeeded",
"registrationEnabled": false,
"resourceGroup": "acilab",
"type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks",
"virtualNetwork": {
"id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Network/virtualNetworks/acivnet",
"resourceGroup": "acilab"
},
"virtualNetworkLinkState": "Completed"
}
- Azure ファイル共有を作成し、初期化スクリプトをアップロードします。
$ storage_account_name="acilab$RANDOM"
$ az storage account create -n $storage_account_name -g $rg --sku Standard_LRS --kind StorageV2
The public access to all blobs or containers in the storage account will be disallowed by default in the future, which means default value for --allow-blob-public-access is still null but will be equivalent to false.
{
"accessTier": "Hot",
"allowBlobPublicAccess": true,
"allowCrossTenantReplication": null,
"allowSharedKeyAccess": null,
"allowedCopyScope": null,
"azureFilesIdentityBasedAuthentication": null,
"blobRestoreStatus": null,
"creationTime": "2023-09-28T07:30:50.699353+00:00",
"customDomain": null,
"defaultToOAuthAuthentication": null,
"dnsEndpointType": null,
"enableHttpsTrafficOnly": true,
"enableNfsV3": null,
"encryption": {
"encryptionIdentity": null,
"keySource": "Microsoft.Storage",
"keyVaultProperties": null,
"requireInfrastructureEncryption": null,
"services": {
"blob": {
"enabled": true,
"keyType": "Account",
"lastEnabledTime": "2023-09-28T07:30:51.074438+00:00"
},
"file": {
"enabled": true,
"keyType": "Account",
"lastEnabledTime": "2023-09-28T07:30:51.074438+00:00"
},
"queue": null,
"table": null
}
},
"extendedLocation": null,
"failoverInProgress": null,
"geoReplicationStats": null,
"id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Storage/storageAccounts/acilab8342",
"identity": null,
"immutableStorageWithVersioning": null,
"isHnsEnabled": null,
"isLocalUserEnabled": null,
"isSftpEnabled": null,
"keyCreationTime": {
"key1": "2023-09-28T07:30:50.761858+00:00",
"key2": "2023-09-28T07:30:50.761858+00:00"
},
"keyPolicy": null,
"kind": "StorageV2",
"largeFileSharesState": null,
"lastGeoFailoverTime": null,
"location": "japaneast",
"minimumTlsVersion": "TLS1_0",
"name": "acilab8342",
"networkRuleSet": {
"bypass": "AzureServices",
"defaultAction": "Allow",
"ipRules": [],
"resourceAccessRules": null,
"virtualNetworkRules": []
},
"primaryEndpoints": {
"blob": "https://acilab8342.blob.core.windows.net/",
"dfs": "https://acilab8342.dfs.core.windows.net/",
"file": "https://acilab8342.file.core.windows.net/",
"internetEndpoints": null,
"microsoftEndpoints": null,
"queue": "https://acilab8342.queue.core.windows.net/",
"table": "https://acilab8342.table.core.windows.net/",
"web": "https://acilab8342.z11.web.core.windows.net/"
},
"primaryLocation": "japaneast",
"privateEndpointConnections": [],
"provisioningState": "Succeeded",
"publicNetworkAccess": null,
"resourceGroup": "acilab",
"routingPreference": null,
"sasPolicy": null,
"secondaryEndpoints": null,
"secondaryLocation": null,
"sku": {
"name": "Standard_LRS",
"tier": "Standard"
},
"statusOfPrimary": "available",
"statusOfSecondary": null,
"storageAccountSkuConversionStatus": null,
"tags": {},
"type": "Microsoft.Storage/storageAccounts"
}
$ storage_account_key=$(az storage account keys list --account-name $storage_account_name -g $rg --query '[0].value' -o tsv)
$ az storage share create --account-name $storage_account_name --account-key $storage_account_key --name initscript
{
"created": true
}
$ init_script_filename=init.sh
$ init_script_path=/tmp/
$ cat <<EOF > ${init_script_path}${init_script_filename}
> echo "Logging into Azure..."
az login --service-principal -u \$SP_APPID -p \$SP_PASSWORD --tenant \$SP_TENANT
echo "Finding out IP address..."
my_private_ip=\$(az container show -n \$ACI_NAME -g \$RG --query 'ipAddress.ip' -o tsv) && echo \$my_private_ip
echo "Creating DNS record..."
az network private-dns record-set a create -n \$HOSTNAME -z \$DNS_ZONE_NAME -g \$RG
az network private-dns record-set a add-record --record-set-name \$HOSTNAME -z \$DNS_ZONE_NAME -g \$RG -a \$my_private_ip
EOF
$ az storage file upload --account-name $storage_account_name --account-key $storage_account_key -s initscript --source ${init_script_path}${init_script_filename}
Finished[#############################################################] 100.0000%
{
"content_md5": "0xd90x630xa70x130x9e0xe50x4f0x300x3b0x850xa00x7f0xfb0x330x190x64",
"date": "2023-09-28T07:34:00+00:00",
"etag": "\"0x8DBBFF54897072B\"",
"file_last_write_time": "2023-09-28T07:34:00.8807211Z",
"last_modified": "2023-09-28T07:34:00+00:00",
"request_id": "022a576d-a01a-001a-44de-f1a8c0000000",
"request_server_encrypted": true,
"version": "2022-11-02"
}
init コンテナーによってコンテナー グループをデプロイする
- YAMLファイルを作成します。ここもユニット4同様、network profile を使った書き方なので、少し修正します。
azure-cli のイメージのレジストリが変わっているので、imageの参照先も変えています。
$ aci_yaml_file=/tmp/acilab.yaml
$ cat <<EOF > $aci_yaml_file
apiVersion: 2021-07-01
location: japaneast
name: $aci_name
properties:
initContainers:
- name: azcli
properties:
image: mcr.microsoft.com/azure-cli
command:
- "/bin/sh"
- "-c"
- "/mnt/init/$init_script_filename"
environmentVariables:
- name: RG
value: $rg
- name: SP_APPID
value: $sp_appid
- name: SP_PASSWORD
secureValue: $sp_password
- name: SP_TENANT
value: $sp_tenant
- name: DNS_ZONE_NAME
value: $dns_zone_name
- name: HOSTNAME
value: $aci_name
- name: ACI_NAME
value: $aci_name
volumeMounts:
- name: initscript
mountPath: /mnt/init/
containers:
- name: nginx
properties:
image: nginx
ports:
- port: 443
protocol: TCP
resources:
requests:
cpu: 1.0
memoryInGB: 1.5
volumeMounts:
- name: nginx-config
mountPath: /etc/nginx
- name: sqlapi
properties:
image: erjosito/sqlapi:1.0
environmentVariables:
- name: SQL_SERVER_FQDN
value: $sql_server_fqdn
- name: SQL_SERVER_USERNAME
value: $sql_username
- name: SQL_SERVER_PASSWORD
secureValue: $sql_password
ports:
- port: 8080
protocol: TCP
resources:
requests:
cpu: 1.0
memoryInGB: 1
volumeMounts:
volumes:
- secret:
ssl.crt: "$ssl_crt"
ssl.key: "$ssl_key"
nginx.conf: "$nginx_conf"
name: nginx-config
- name: initscript
azureFile:
readOnly: false
shareName: initscript
storageAccountName: $storage_account_name
storageAccountKey: $storage_account_key
ipAddress:
ports:
- port: 443
protocol: TCP
type: Private
osType: Linux
subnetIds:
- id: $aci_subnet_id
name: aci
tags: null
type: Microsoft.ContainerInstance/containerGroups
EOF
- Azure Container Instance を作成します。
$ az container create -g $rg --file $aci_yaml_file
{
"extendedLocation": null,
"id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.ContainerInstance/containerGroups/learnaci",
"identity": null,
"kind": null,
"location": "japaneast",
"managedBy": null,
"name": "learnaci",
"plan": null,
"properties": {
"containers": [
{
"name": "nginx",
"properties": {
"environmentVariables": [],
"image": "nginx",
"instanceView": {
"currentState": {
"detailStatus": "",
"startTime": "2023-09-28T07:47:13.046Z",
"state": "Running"
},
"events": [
{
"count": 1,
"firstTimestamp": "2023-09-28T07:45:48Z",
"lastTimestamp": "2023-09-28T07:45:48Z",
"message": "pulling image \"nginx@sha256:b2888fc9cfe7cd9d6727aeb462d13c7c45dec413b66f2819a36c4a3cb9d4df75\"",
"name": "Pulling",
"type": "Normal"
},
{
"count": 1,
"firstTimestamp": "2023-09-28T07:46:34Z",
"lastTimestamp": "2023-09-28T07:46:34Z",
"message": "Successfully pulled image \"nginx@sha256:b2888fc9cfe7cd9d6727aeb462d13c7c45dec413b66f2819a36c4a3cb9d4df75\"",
"name": "Pulled",
"type": "Normal"
},
{
"count": 1,
"firstTimestamp": "2023-09-28T07:47:13Z",
"lastTimestamp": "2023-09-28T07:47:13Z",
"message": "Started container",
"name": "Started",
"type": "Normal"
}
],
"restartCount": 0
},
"ports": [
{
"port": 443,
"protocol": "TCP"
}
],
"resources": {
"requests": {
"cpu": 1.0,
"memoryInGB": 1.5
}
},
"volumeMounts": [
{
"mountPath": "/etc/nginx",
"name": "nginx-config"
}
]
}
},
{
"name": "sqlapi",
"properties": {
"environmentVariables": [
{
"name": "SQL_SERVER_FQDN",
"value": "sqlserver15595.database.windows.net"
},
{
"name": "SQL_SERVER_USERNAME",
"value": "azure"
},
{
"name": "SQL_SERVER_PASSWORD"
}
],
"image": "erjosito/sqlapi:1.0",
"instanceView": {
"currentState": {
"detailStatus": "",
"startTime": "2023-09-28T07:47:13.253Z",
"state": "Running"
},
"events": [
{
"count": 1,
"firstTimestamp": "2023-09-28T07:45:48Z",
"lastTimestamp": "2023-09-28T07:45:48Z",
"message": "pulling image \"erjosito/sqlapi@sha256:a1233cdfcd53fe513fe0ea2920e3fa9bb388b5722e28ee19c218361845206899\"",
"name": "Pulling",
"type": "Normal"
},
{
"count": 1,
"firstTimestamp": "2023-09-28T07:46:34Z",
"lastTimestamp": "2023-09-28T07:46:34Z",
"message": "Successfully pulled image \"erjosito/sqlapi@sha256:a1233cdfcd53fe513fe0ea2920e3fa9bb388b5722e28ee19c218361845206899\"",
"name": "Pulled",
"type": "Normal"
},
{
"count": 1,
"firstTimestamp": "2023-09-28T07:47:13Z",
"lastTimestamp": "2023-09-28T07:47:13Z",
"message": "Started container",
"name": "Started",
"type": "Normal"
}
],
"restartCount": 0
},
"ports": [
{
"port": 8080,
"protocol": "TCP"
}
],
"resources": {
"requests": {
"cpu": 1.0,
"memoryInGB": 1.0
}
}
}
}
],
"initContainers": [
{
"name": "azcli",
"properties": {
"command": [
"/bin/sh",
"-c",
"/mnt/init/init.sh"
],
"environmentVariables": [
{
"name": "RG",
"value": "acilab"
},
{
"name": "SP_APPID",
"value": "7dcd731d-b8ef-4bf9-8c83-e456d623024d"
},
{
"name": "SP_TENANT",
"value": "cb9bb346-c037-4fb2-a3ff-dd23544753ea"
},
{
"name": "DNS_ZONE_NAME",
"value": "contoso.com"
},
{
"name": "HOSTNAME",
"value": "learnaci"
},
{
"name": "ACI_NAME",
"value": "learnaci"
},
{
"name": "SP_PASSWORD"
}
],
"image": "mcr.microsoft.com/azure-cli",
"instanceView": {
"currentState": {
"detailStatus": "Completed",
"exitCode": 0,
"finishTime": "2023-09-28T07:47:12.142Z",
"startTime": "2023-09-28T07:47:00.891Z",
"state": "Terminated"
},
"events": [],
"restartCount": 0
},
"volumeMounts": [
{
"mountPath": "/mnt/init/",
"name": "initscript"
}
]
}
}
],
"instanceView": {
"events": [
{
"count": 1,
"firstTimestamp": "2023-09-28T07:46:54.092Z",
"lastTimestamp": "2023-09-28T07:46:54.092Z",
"message": "Successfully mounted Azure File Volume.",
"name": "SuccessfulMountAzureFileVolume",
"type": "Normal"
}
],
"state": "Running"
},
"ipAddress": {
"ip": "192.168.2.4",
"ports": [
{
"port": 443,
"protocol": "TCP"
},
{
"port": 8080,
"protocol": "TCP"
}
],
"type": "Private"
},
"isCustomProvisioningTimeout": false,
"osType": "Linux",
"provisioningState": "Succeeded",
"provisioningTimeoutInSeconds": 1800,
"sku": "Standard",
"subnetIds": [
{
"id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Network/virtualNetworks/acivnet/subnets/aci",
"name": "aci",
"resourceGroup": "acilab"
}
],
"volumes": [
{
"name": "nginx-config",
"secret": {}
},
{
"azureFile": {
"readOnly": false,
"shareName": "initscript",
"storageAccountName": "acilab8342"
},
"name": "initscript"
}
]
},
"resourceGroup": "acilab",
"sku": null,
"tags": null,
"type": "Microsoft.ContainerInstance/containerGroups"
}
- SQL API エンドポイントを使用して、コンテナーに到達できるようになったことをテストします
$ aci_fqdn=${aci_name}.${dns_zone_name} && echo $aci_fqdn
learnaci.contoso.com
$ ssh -n -o BatchMode=yes -o StrictHostKeyChecking=no $vm_pip "nslookup $aci_fqdn"
Server: 127.0.0.53
Address: 127.0.0.53#53
Non-authoritative answer:
Name: learnaci.contoso.com
Address: 0.0.0.0
0.0.0.0 になってる？？
なんかIPの取得がうまくいかずに、DNSゾーンのAレコードに0.0.0.0で登録されたみたいです。
だいぶ疲れてきたのでチートします。
IPアドレスを $aci_ip の値にします。
もう一度リトライ
$ ssh -n -o BatchMode=yes -o StrictHostKeyChecking=no $vm_pip "nslookup $aci_fqdn"
Server: 127.0.0.53
Address: 127.0.0.53#53
Non-authoritative answer:
Name: learnaci.contoso.com
Address: 192.168.2.4
$ ssh -n -o BatchMode=yes -o StrictHostKeyChecking=no $vm_pip "curl -ks https://$aci_fqdn/api/healthcheck"
{
"health": "OK",
"version": "1.0"
}
$ ssh -n -o BatchMode=yes -o StrictHostKeyChecking=no $vm_pip "curl -ks https://$aci_fqdn/api/sqlversion"
{
"sql_output": "Microsoft SQL Azure (RTM) - 12.0.2000.8 \n\tJul 17 2023 18:40:52 \n\tCopyright (C) 2022 Microsoft Corporation\n"
}
$ ssh -n -o BatchMode=yes -o StrictHostKeyChecking=no $vm_pip "curl -ks https://$aci_fqdn/api/sqlsrcip"
{
"sql_output": "192.168.2.4"
}
疎通確認はできました。
が、このIPをちゃんと取得してくれないとinitの意味がないような？
もうちょっと理解できるようにならないと。。
- initコンテナのログ確認
Finding out IP addressの後が0.0.0.0になってるんだよなぁ。
$ az container logs -n $aci_name -g $rg --container-name azcli
Logging into Azure...
[
{
"cloudName": "AzureCloud",
"homeTenantId": "cb9bb346-c037-4fb2-a3ff-dd23544753ea",
"id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
"isDefault": true,
"managedByTenants": [],
"name": "検証用",
"state": "Enabled",
"tenantId": "cb9bb346-c037-4fb2-a3ff-dd23544753ea",
"user": {
"name": "7dcd731d-b8ef-4bf9-8c83-e456d623024d",
"type": "servicePrincipal"
}
}
]
Finding out IP address...
0.0.0.0
Creating DNS record...
{
"aRecords": [],
"etag": "637c952f-4b33-4353-880f-6cf264c5b3d9",
"fqdn": "learnaci.contoso.com.",
"id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Network/privateDnsZones/contoso.com/A/learnaci",
"isAutoRegistered": false,
"name": "learnaci",
"resourceGroup": "acilab",
"ttl": 3600,
"type": "Microsoft.Network/privateDnsZones/A"
}
{
"aRecords": [
{
"ipv4Address": "0.0.0.0"
}
],
"etag": "b76ea1f8-0d94-4308-97dc-1fb575b20652",
"fqdn": "learnaci.contoso.com.",
"id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Network/privateDnsZones/contoso.com/A/learnaci",
"isAutoRegistered": false,
"name": "learnaci",
"resourceGroup": "acilab",
"ttl": 3600,
"type": "Microsoft.Network/privateDnsZones/A"
}
- 後片付け。リソースグループごとリソースを削除します。
$ az group delete -n $rg -y --no-wait
結構簡単にできるかと思いきや、いろいろな罠にハマりました。
この記事も思ってたより長くなってしまったし。
MS Learnのメンテナンスって大変そうよね。。
あと、ネットワーク周り難しいーーーー
Discussion