Zenn
👻

【MS Learn】Azure Container Instances でサイドカー構成を使用してアプリのセキュリティを保護するをやってみた

2023/09/28に公開
Azure
sidecar
containerinstance
tech

タイトルが入力文字数限界です。ギリギリ。

今回は MS Learnの『Azure Container Instances でサイドカー構成を使用してアプリのセキュリティを保護する』をやってみた、です。
https://learn.microsoft.com/ja-jp/training/modules/secure-apps-azure-container-instances-sidecar/

そのままコマンド実行していくはずなんですけど、うまく動かなかったりしたので、実際の挙動ログとして残しておきます。

コマンドは Azure Cloud Shell の Bash で実行します。
基本的にコマンドのところのみ書いていきますので、番号も本家に合わせています。

ユニット2: 仮想ネットワークに Azure Container Instances をデプロイする

環境を初期化する

  1. 次のコマンドを実行して、このユニットの変数を定義します。location は japaneast に変更。
rg=acilab
location=japaneast
aci_name=learnaci
aci_dns=${aci_name}${RANDOM}
vnet_name=acivnet
vnet_prefix=192.168.0.0/16
vm_subnet_name=vm
vm_subnet_prefix=192.168.1.0/24
aci_subnet_name=aci
aci_subnet_prefix=192.168.2.0/24
  1. テスト目的で使用するリソース グループと仮想マシンを作成します。
$ az group create -n $rg -l $location
{
  "id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab",
  "location": "japaneast",
  "managedBy": null,
  "name": "acilab",
  "properties": {
    "provisioningState": "Succeeded"
  },
  "tags": null,
  "type": "Microsoft.Resources/resourceGroups"
}
$ az vm create -n test-vm -g $rg -l $location --image ubuntuLTS --generate-ssh-keys \
    --public-ip-address test-vm-pip --vnet-name $vnet_name \
    --vnet-address-prefix $vnet_prefix --subnet $vm_subnet_name --subnet-address-prefix $vm_subnet_prefix
{
  "fqdns": "",
  "id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Compute/virtualMachines/test-vm",
  "location": "japaneast",
  "macAddress": "00-22-48-E7-D4-18",
  "powerState": "VM running",
  "privateIpAddress": "192.168.1.4",
  "publicIpAddress": "20.78.102.185",
  "resourceGroup": "acilab",
  "zones": ""
}
$ vm_pip=$(az network public-ip show -n test-vm-pip -g $rg --query ipAddress -o tsv) && echo $vm_pip
20.78.102.185

ssh 接続で確認

$ ssh $vm_pip
The authenticity of host '20.78.102.185 (20.78.102.185)' can't be established.
ED25519 key fingerprint is SHA256:OmpGWK7+3PtXN4u3iZFcXzfKtp8chN2QZoMKr1QXLPc.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '20.78.102.185' (ED25519) to the list of known hosts.
Welcome to Ubuntu 18.04.6 LTS (GNU/Linux 5.4.0-1109-azure x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

  System information as of Thu Sep 28 05:47:20 UTC 2023

  System load:  0.19              Processes:           109
  Usage of /:   4.5% of 28.89GB   Users logged in:     0
  Memory usage: 5%                IP address for eth0: 192.168.1.4
  Swap usage:   0%

Expanded Security Maintenance for Infrastructure is not enabled.

0 updates can be applied immediately.

Enable ESM Infra to receive additional future security updates.
See https://ubuntu.com/esm or run: sudo pro status



The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.

To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.

*****@test-vm:~$ exit
logout
Connection to 20.78.102.185 closed.
  1. アプリケーションの接続先となるデータベースを作成します。
$ sql_server_name=sqlserver$RANDOM
sql_db_name=mydb
sql_username=azure
sql_password=Microsoft123!
$ az sql server create -n $sql_server_name -g $rg -l $location --admin-user $sql_username --admin-password $sql_password
{
  "administratorLogin": "azure",
  "administratorLoginPassword": null,
  "administrators": null,
  "externalGovernanceStatus": "Disabled",
  "federatedClientId": null,
  "fullyQualifiedDomainName": "sqlserver15595.database.windows.net",
  "id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Sql/servers/sqlserver15595",
  "identity": null,
  "keyId": null,
  "kind": "v12.0",
  "location": "japaneast",
  "minimalTlsVersion": "None",
  "name": "sqlserver15595",
  "primaryUserAssignedIdentityId": null,
  "privateEndpointConnections": [],
  "publicNetworkAccess": "Enabled",
  "resourceGroup": "acilab",
  "restrictOutboundNetworkAccess": "Disabled",
  "state": "Ready",
  "tags": null,
  "type": "Microsoft.Sql/servers",
  "version": "12.0",
  "workspaceFeature": null
}
$ sql_server_fqdn=$(az sql server show -n $sql_server_name -g $rg -o tsv --query fullyQualifiedDomainName)
$ az sql db create -n $sql_db_name -s $sql_server_name -g $rg -e Basic -c 5 --no-wait

パスワードポリシーが厳しくなったのか、記号がないと怒られるので、後続で出てくるパスワードをセットします。

(PasswordNotComplex) Password validation failed. The password does not meet policy requirements because it is not complex enough.
Code: PasswordNotComplex
Message: Password validation failed. The password does not meet policy requirements because it is not complex enough.

仮想ネットワーク内に Azure Container Instance を作成する

  1. 環境変数に、Azure SQL Database の完全修飾ドメイン名と資格情報を指定して、Azure Container Instance がそれに接続できるようにします。
$ az network vnet subnet create -g $rg --vnet-name $vnet_name -n $aci_subnet_name --address-prefix $aci_subnet_prefix
{
  "addressPrefix": "192.168.2.0/24",
  "delegations": [],
  "etag": "W/\"08b0d45e-fdd6-42fc-95b2-34ef48960c0d\"",
  "id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Network/virtualNetworks/acivnet/subnets/aci",
  "name": "aci",
  "privateEndpointNetworkPolicies": "Disabled",
  "privateLinkServiceNetworkPolicies": "Enabled",
  "provisioningState": "Succeeded",
  "resourceGroup": "acilab",
  "type": "Microsoft.Network/virtualNetworks/subnets"
}
$ vnet_id=$(az network vnet show -n $vnet_name -g $rg --query id -o tsv)
$ aci_subnet_id=$(az network vnet subnet show -n $aci_subnet_name --vnet-name $vnet_name -g $rg --query id -o tsv)
$ $ az container create -n $aci_name -g $rg -e "SQL_SERVER_USERNAME=$sql_username" \
  "SQL_SERVER_PASSWORD=$sql_password" \
  "SQL_SERVER_FQDN=${sql_server_fqdn}" \
  --image erjosito/sqlapi:1.0 \
  --ip-address private --ports 8080 --vnet $vnet_id --subnet $aci_subnet_id
{
  "confidentialComputeProperties": null,
  "containers": [
    {
      "command": null,
      "environmentVariables": [
        {
          "name": "SQL_SERVER_USERNAME",
          "secureValue": null,
          "value": "azure"
        },
        {
          "name": "SQL_SERVER_PASSWORD",
          "secureValue": null,
          "value": "Microsoft123!"
        },
        {
          "name": "SQL_SERVER_FQDN",
          "secureValue": null,
          "value": "sqlserver15595.database.windows.net"
        }
      ],
      "image": "erjosito/sqlapi:1.0",
      "instanceView": {
        "currentState": {
          "detailStatus": "",
          "exitCode": null,
          "finishTime": null,
          "startTime": "2023-09-28T05:54:03.672000+00:00",
          "state": "Running"
        },
        "events": [
          {
            "count": 1,
            "firstTimestamp": "2023-09-28T05:53:31+00:00",
            "lastTimestamp": "2023-09-28T05:53:31+00:00",
            "message": "pulling image \"erjosito/sqlapi@sha256:a1233cdfcd53fe513fe0ea2920e3fa9bb388b5722e28ee19c218361845206899\"",
            "name": "Pulling",
            "type": "Normal"
          },
          {
            "count": 1,
            "firstTimestamp": "2023-09-28T05:53:51+00:00",
            "lastTimestamp": "2023-09-28T05:53:51+00:00",
            "message": "Successfully pulled image \"erjosito/sqlapi@sha256:a1233cdfcd53fe513fe0ea2920e3fa9bb388b5722e28ee19c218361845206899\"",
            "name": "Pulled",
            "type": "Normal"
          },
          {
            "count": 1,
            "firstTimestamp": "2023-09-28T05:54:03+00:00",
            "lastTimestamp": "2023-09-28T05:54:03+00:00",
            "message": "Started container",
            "name": "Started",
            "type": "Normal"
          }
        ],
        "previousState": null,
        "restartCount": 0
      },
      "livenessProbe": null,
      "name": "learnaci",
      "ports": [
        {
          "port": 8080,
          "protocol": "TCP"
        }
      ],
      "readinessProbe": null,
      "resources": {
        "limits": null,
        "requests": {
          "cpu": 1.0,
          "gpu": null,
          "memoryInGb": 1.5
        }
      },
      "securityContext": null,
      "volumeMounts": null
    }
  ],
  "diagnostics": null,
  "dnsConfig": null,
  "encryptionProperties": null,
  "extensions": null,
  "id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.ContainerInstance/containerGroups/learnaci",
  "identity": null,
  "imageRegistryCredentials": null,
  "initContainers": [],
  "instanceView": {
    "events": [],
    "state": "Running"
  },
  "ipAddress": {
    "autoGeneratedDomainNameLabelScope": null,
    "dnsNameLabel": null,
    "fqdn": null,
    "ip": "192.168.2.4",
    "ports": [
      {
        "port": 8080,
        "protocol": "TCP"
      }
    ],
    "type": "Private"
  },
  "location": "japaneast",
  "name": "learnaci",
  "osType": "Linux",
  "priority": null,
  "provisioningState": "Succeeded",
  "resourceGroup": "acilab",
  "restartPolicy": "Always",
  "sku": "Standard",
  "subnetIds": [
    {
      "id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Network/virtualNetworks/acivnet/subnets/aci",
      "name": null,
      "resourceGroup": "acilab"
    }
  ],
  "tags": {},
  "type": "Microsoft.ContainerInstance/containerGroups",
  "volumes": null,
  "zones": null
}
  1. 仮想マシンに接続できるかAPIを叩いて確認。
$ aci_ip=$(az container show -n $aci_name -g $rg --query 'ipAddress.ip' -o tsv) && echo $aci_ip
ssh -n -o BatchMode=yes -o StrictHostKeyChecking=no $vm_pip "curl -s http://$aci_ip:8080/api/healthcheck"
192.168.2.4
{
  "health": "OK", 
  "version": "1.0"
}
  1. Azure SQL ファイアウォール ルールを更新。APIを叩いてアクセスできるか確認。
$ aci_pip=$(ssh -n -o BatchMode=yes -o StrictHostKeyChecking=no $vm_pip "curl -s http://$aci_ip:8080/api/ip" | jq -r .my_public_ip) && echo $aci_pip
az sql server firewall-rule create -g $rg -s $sql_server_name -n public_sqlapi_aci-source --start-ip-address $aci_pip --end-ip-address $aci_pip
ssh -n -o BatchMode=yes -o StrictHostKeyChecking=no $vm_pip "curl -s http://$aci_ip:8080/api/sqlversion"
ssh -n -o BatchMode=yes -o StrictHostKeyChecking=no $vm_pip "curl -s http://$aci_ip:8080/api/sqlsrcip"
20.44.190.204
{
  "endIpAddress": "20.44.190.204",
  "id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Sql/servers/sqlserver11156/firewallRules/public_sqlapi_aci-source",
  "name": "public_sqlapi_aci-source",
  "resourceGroup": "acilab",
  "startIpAddress": "20.44.190.204",
  "type": "Microsoft.Sql/servers/firewallRules"
}
{
  "sql_output": "Microsoft SQL Azure (RTM) - 12.0.2000.8 \n\tSep 18 2023 12:22:37 \n\tCopyright (C) 2022 Microsoft Corporation\n"
}
{
  "sql_output": "20.44.190.204"
}
  1. コンテナの削除
$ az container delete -n $aci_name -g $rg -y
{
  "confidentialComputeProperties": null,
  "containers": [
    {
      "command": null,
      "environmentVariables": [
        {
          "name": "SQL_SERVER_USERNAME",
          "secureValue": null,
          "value": "azure"
        },
        {
          "name": "SQL_SERVER_PASSWORD",
          "secureValue": null,
          "value": "Microsoft123!"
        },
        {
          "name": "SQL_SERVER_FQDN",
          "secureValue": null,
          "value": "sqlserver11156.database.windows.net"
        }
      ],
      "image": "erjosito/sqlapi:1.0",
      "instanceView": {
        "currentState": {
          "detailStatus": "",
          "exitCode": null,
          "finishTime": null,
          "startTime": "2023-09-28T02:18:35.543000+00:00",
          "state": "Running"
        },
        "events": [
          {
            "count": 1,
            "firstTimestamp": "2023-09-28T02:18:01+00:00",
            "lastTimestamp": "2023-09-28T02:18:01+00:00",
            "message": "pulling image \"erjosito/sqlapi@sha256:a1233cdfcd53fe513fe0ea2920e3fa9bb388b5722e28ee19c218361845206899\"",
            "name": "Pulling",
            "type": "Normal"
          },
          {
            "count": 1,
            "firstTimestamp": "2023-09-28T02:18:24+00:00",
            "lastTimestamp": "2023-09-28T02:18:24+00:00",
            "message": "Successfully pulled image \"erjosito/sqlapi@sha256:a1233cdfcd53fe513fe0ea2920e3fa9bb388b5722e28ee19c218361845206899\"",
            "name": "Pulled",
            "type": "Normal"
          },
          {
            "count": 1,
            "firstTimestamp": "2023-09-28T02:18:35+00:00",
            "lastTimestamp": "2023-09-28T02:18:35+00:00",
            "message": "Started container",
            "name": "Started",
            "type": "Normal"
          }
        ],
        "previousState": null,
        "restartCount": 0
      },
      "livenessProbe": null,
      "name": "learnaci",
      "ports": [
        {
          "port": 8080,
          "protocol": "TCP"
        }
      ],
      "readinessProbe": null,
      "resources": {
        "limits": null,
        "requests": {
          "cpu": 1.0,
          "gpu": null,
          "memoryInGb": 1.5
        }
      },
      "securityContext": null,
      "volumeMounts": null
    }
  ],
  "diagnostics": null,
  "dnsConfig": null,
  "encryptionProperties": null,
  "extensions": null,
  "id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.ContainerInstance/containerGroups/learnaci",
  "identity": null,
  "imageRegistryCredentials": null,
  "initContainers": [],
  "instanceView": {
    "events": [],
    "state": "Running"
  },
  "ipAddress": {
    "autoGeneratedDomainNameLabelScope": null,
    "dnsNameLabel": null,
    "fqdn": null,
    "ip": "192.168.2.4",
    "ports": [
      {
        "port": 8080,
        "protocol": "TCP"
      }
    ],
    "type": "Private"
  },
  "location": "japaneast",
  "name": "learnaci",
  "osType": "Linux",
  "priority": null,
  "provisioningState": "Succeeded",
  "resourceGroup": "acilab",
  "restartPolicy": "Always",
  "sku": "Standard",
  "subnetIds": [
    {
      "id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Network/virtualNetworks/acivnet/subnets/aci",
      "name": null,
      "resourceGroup": "acilab"
    }
  ],
  "tags": {},
  "type": "Microsoft.ContainerInstance/containerGroups",
  "volumes": null,
  "zones": null
}

ユニット3: Azure Container Instances YAML 定義について

既存のコンテナー グループから YAML コードを抽出する

  1. コンテナの再作成(さっき消さなかった場合)
$ az container create -n $aci_name -g $rg -e "SQL_SERVER_USERNAME=$sql_username" \
  "SQL_SERVER_PASSWORD=$sql_password" \
  "SQL_SERVER_FQDN=${sql_server_fqdn}" \
  --image erjosito/sqlapi:1.0 \
  --ip-address private --ports 8080 --vnet $vnet_id --subnet $aci_subnet_id
  1. YAML コードをファイルに保存
$ az container export -n $aci_name -g $rg -f /tmp/aci.yaml
$ more /tmp/aci.yaml
/tmp/aci.yaml
additional_properties: {}
apiVersion: '2023-05-01'
extended_location: null
location: japaneast
name: learnaci
properties:
  containers:
  - name: learnaci
    properties:
      environmentVariables:
      - name: SQL_SERVER_USERNAME
        value: azure
      - name: SQL_SERVER_PASSWORD
        value: Microsoft123!
      - name: SQL_SERVER_FQDN
        value: sqlserver15595.database.windows.net
      image: erjosito/sqlapi:1.0
      ports:
      - port: 8080
        protocol: TCP
      resources:
        requests:
          cpu: 1.0
          memoryInGB: 1.5
  initContainers: []
  ipAddress:
    ip: 192.168.2.4
    ports:
    - port: 8080
      protocol: TCP
    type: Private
  isCustomProvisioningTimeout: false
  osType: Linux
  provisioningTimeoutInSeconds: 1800
  restartPolicy: Always
  sku: Standard
  subnetIds:
  - id: /subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Network/virtualNetworks/acivnet/subnets
/aci
tags: {}
type: Microsoft.ContainerInstance/containerGroups

YAML ファイルを変更してデプロイする

  1. 自動的に生成されたファイルを変更する
$ sed -i 's/        value: Microsoft123!/        secureValue: Microsoft123!/g' /tmp/aci.yaml
  1. 再デプロイする
$ az container delete -n $aci_name -g $rg -y
$ az container create -g $rg --file /tmp/aci.yaml

けど、API versionが古くて？サポートしてないというエラーがでる。

(CustomProvisioningTimeoutInSecondsNotSuppported) Custom provisioning timeout is only supported for API version '2023-02-01-preview'.
Code: CustomProvisioningTimeoutInSecondsNotSuppported
Message: Custom provisioning timeout is only supported for API version '2023-02-01-preview'.

/tmp/aci.yaml の provisioningTimeoutInSeconds の行をコメントアウトする。

$ vi /tmp/aci.yaml
#  provisioningTimeoutInSeconds: 1800

再デプロイ

$ az container create -g $rg --file /tmp/aci.yaml
  1. 新しい Azure Container Instance を別の YAML ファイル /tmp/aci2.yaml にエクスポート
$ az container export -n $aci_name -g $rg -f /tmp/aci2.yaml

diff を取るとパスワードが出力されないのがわかる

$ diff /tmp/aci.yaml /tmp/aci2.yaml 
13,14d12
<       - name: SQL_SERVER_PASSWORD
<         secureValue: Microsoft123!
16a15
>       - name: SQL_SERVER_PASSWORD
34c33
< #  provisioningTimeoutInSeconds: 1800
---
>   provisioningTimeoutInSeconds: 1800
  1. コンテナーを削除
az container delete -n $aci_name -g $rg -y

ユニット4: サイドカー コンテナーで Azure Container Instances をデプロイする

NGINX 構成を作成する

  1. 自己署名証明書を作成する
$ openssl req -new -newkey rsa:2048 -nodes -keyout ssl.key -out ssl.csr -subj "/C=US/ST=WA/L=Redmond/O=AppDev/OU=IT/CN=contoso.com"
Generating a RSA private key
...........+++++
...............................................................+++++
writing new private key to 'ssl.key'
-----
$ openssl x509 -req -days 365 -in ssl.csr -signkey ssl.key -out ssl.crt
Signature ok
subject=C = US, ST = WA, L = Redmond, O = AppDev, OU = IT, CN = contoso.com
Getting Private key
  1. nginx.conf を作成する
$ nginx_config_file=/tmp/nginx.conf
$ cat <<EOF > $nginx_config_file
user nginx;
worker_processes auto;
events {
  worker_connections 1024;
}
pid        /var/run/nginx.pid;
http {
    server {
        listen [::]:443 ssl;
        listen 443 ssl;
        server_name localhost;
        ssl_protocols              TLSv1.2;
        ssl_ciphers                ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:ECDHE-RSA-RC4-SHA:ECDHE-ECDSA-RC4-SHA:AES128:AES256:RC4-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!3DES:!MD5:!PSK;
        ssl_prefer_server_ciphers  on;
        ssl_session_cache    shared:SSL:10m; # a 1mb cache can hold about 4000 sessions, so we can hold 40000 sessions
        ssl_session_timeout  24h;
        keepalive_timeout 75; # up from 75 secs default
        add_header Strict-Transport-Security 'max-age=31536000; includeSubDomains';
        ssl_certificate      /etc/nginx/ssl.crt;
        ssl_certificate_key  /etc/nginx/ssl.key;
        location / {
            proxy_pass http://127.0.0.1:8080 ;
            proxy_set_header Connection "";
            proxy_set_header Host \$host;
            proxy_set_header X-Real-IP \$remote_addr;
            proxy_set_header X-Forwarded-For \$remote_addr;
            proxy_buffer_size          128k;
            proxy_buffers              4 256k;
            proxy_busy_buffers_size    256k;
        }
    }
}
EOF
  1. NGINX サイドカー コンテナーに渡す必要があるファイルを収集する
$ nginx_conf=$(cat $nginx_config_file | base64)
$ ssl_crt=$(cat ssl.crt | base64)
$ ssl_key=$(cat ssl.key | base64)

NGINX サイドカーによってコンテナー グループをデプロイする

1 & 2.
ここで、network profile を取得するという流れになっているんですが、コマンド実行しても空が返ってくるし、yamlにも出力されてないし、とだいぶハマったんですが、network profile は廃止されたようです。
https://github.com/Azure/azure-cli/issues/20587


https://learn.microsoft.com/ja-jp/azure/container-instances/container-instances-vnet

代わりにサブネットを指定するようにします。
$aci_subnet_id は前に変数にセットしてますね。

$ aci_yaml_file=/tmp/aci_ssl.yaml
cat <<EOF > $aci_yaml_file
apiVersion: 2021-07-01
location: japaneast
name: $aci_name
properties:
  containers:
  - name: nginx
    properties:
      image: nginx
      ports:
      - port: 443
        protocol: TCP
      resources:
        requests:
          cpu: 1.0
          memoryInGB: 1.5
      volumeMounts:
      - name: nginx-config
        mountPath: /etc/nginx
  - name: sqlapi
    properties:
      image: erjosito/sqlapi:1.0
      environmentVariables:
      - name: SQL_SERVER_USERNAME
        value: $sql_username
      - name: SQL_SERVER_PASSWORD
        secureValue: $sql_password
      - name: SQL_SERVER_FQDN
        value: $sql_server_fqdn
      ports:
      - port: 8080
        protocol: TCP
      resources:
        requests:
          cpu: 1.0
          memoryInGB: 1
  volumes:
  - secret:
      ssl.crt: "$ssl_crt"
      ssl.key: "$ssl_key"
      nginx.conf: "$nginx_conf"
    name: nginx-config
  ipAddress:
    ports:
    - port: 443
      protocol: TCP
    type: Private
  osType: Linux
  subnetIds:
    - id: $aci_subnet_id
      name: aci
tags: null
type: Microsoft.ContainerInstance/containerGroups
EOF
  1. 作成したyamlからContainer Instance を作成します。
$ az container create -g $rg --file $aci_yaml_file
{
  "extendedLocation": null,
  "id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.ContainerInstance/containerGroups/learnaci",
  "identity": null,
  "kind": null,
  "location": "japaneast",
  "managedBy": null,
  "name": "learnaci",
  "plan": null,
  "properties": {
    "containers": [
      {
        "name": "nginx",
        "properties": {
          "environmentVariables": [],
          "image": "nginx",
          "instanceView": {
            "currentState": {
              "detailStatus": "",
              "startTime": "2023-09-28T06:35:16.141Z",
              "state": "Running"
            },
            "events": [
              {
                "count": 1,
                "firstTimestamp": "2023-09-28T06:34:35Z",
                "lastTimestamp": "2023-09-28T06:34:35Z",
                "message": "pulling image \"nginx@sha256:b2888fc9cfe7cd9d6727aeb462d13c7c45dec413b66f2819a36c4a3cb9d4df75\"",
                "name": "Pulling",
                "type": "Normal"
              },
              {
                "count": 1,
                "firstTimestamp": "2023-09-28T06:34:57Z",
                "lastTimestamp": "2023-09-28T06:34:57Z",
                "message": "Successfully pulled image \"nginx@sha256:b2888fc9cfe7cd9d6727aeb462d13c7c45dec413b66f2819a36c4a3cb9d4df75\"",
                "name": "Pulled",
                "type": "Normal"
              },
              {
                "count": 1,
                "firstTimestamp": "2023-09-28T06:35:16Z",
                "lastTimestamp": "2023-09-28T06:35:16Z",
                "message": "Started container",
                "name": "Started",
                "type": "Normal"
              }
            ],
            "restartCount": 0
          },
          "ports": [
            {
              "port": 443,
              "protocol": "TCP"
            }
          ],
          "resources": {
            "requests": {
              "cpu": 1.0,
              "memoryInGB": 1.5
            }
          },
          "volumeMounts": [
            {
              "mountPath": "/etc/nginx",
              "name": "nginx-config"
            }
          ]
        }
      },
      {
        "name": "sqlapi",
        "properties": {
          "environmentVariables": [
            {
              "name": "SQL_SERVER_USERNAME",
              "value": "azure"
            },
            {
              "name": "SQL_SERVER_FQDN",
              "value": "sqlserver15595.database.windows.net"
            },
            {
              "name": "SQL_SERVER_PASSWORD"
            }
          ],
          "image": "erjosito/sqlapi:1.0",
          "instanceView": {
            "currentState": {
              "detailStatus": "",
              "startTime": "2023-09-28T06:35:16.398Z",
              "state": "Running"
            },
            "events": [
              {
                "count": 1,
                "firstTimestamp": "2023-09-28T06:34:35Z",
                "lastTimestamp": "2023-09-28T06:34:35Z",
                "message": "pulling image \"erjosito/sqlapi@sha256:a1233cdfcd53fe513fe0ea2920e3fa9bb388b5722e28ee19c218361845206899\"",
                "name": "Pulling",
                "type": "Normal"
              },
              {
                "count": 1,
                "firstTimestamp": "2023-09-28T06:34:57Z",
                "lastTimestamp": "2023-09-28T06:34:57Z",
                "message": "Successfully pulled image \"erjosito/sqlapi@sha256:a1233cdfcd53fe513fe0ea2920e3fa9bb388b5722e28ee19c218361845206899\"",
                "name": "Pulled",
                "type": "Normal"
              },
              {
                "count": 1,
                "firstTimestamp": "2023-09-28T06:35:16Z",
                "lastTimestamp": "2023-09-28T06:35:16Z",
                "message": "Started container",
                "name": "Started",
                "type": "Normal"
              }
            ],
            "restartCount": 0
          },
          "ports": [
            {
              "port": 8080,
              "protocol": "TCP"
            }
          ],
          "resources": {
            "requests": {
              "cpu": 1.0,
              "memoryInGB": 1.0
            }
          }
        }
      }
    ],
    "initContainers": [],
    "instanceView": {
      "events": [],
      "state": "Running"
    },
    "ipAddress": {
      "ip": "192.168.2.4",
      "ports": [
        {
          "port": 443,
          "protocol": "TCP"
        },
        {
          "port": 8080,
          "protocol": "TCP"
        }
      ],
      "type": "Private"
    },
    "isCustomProvisioningTimeout": false,
    "osType": "Linux",
    "provisioningState": "Succeeded",
    "provisioningTimeoutInSeconds": 1800,
    "sku": "Standard",
    "subnetIds": [
      {
        "id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Network/virtualNetworks/acivnet/subnets/aci",
        "name": "aci",
        "resourceGroup": "acilab"
      }
    ],
    "volumes": [
      {
        "name": "nginx-config",
        "secret": {}
      }
    ]
  },
  "resourceGroup": "acilab",
  "sku": null,
  "tags": null,
  "type": "Microsoft.ContainerInstance/containerGroups"
}

portalから Container Instance -> [設定：コンテナー] で確認すると、コンテナが2つになっていました。

  1. テスト仮想マシンから HTTPS 経由でアクセス
$ aci_ip=$(az container show -n $aci_name -g $rg --query 'ipAddress.ip' -o tsv) && echo $aci_ip
192.168.2.4
$ ssh -n -o BatchMode=yes -o StrictHostKeyChecking=no $vm_pip "curl -ks https://$aci_ip/api/healthcheck"
{
  "health": "OK", 
  "version": "1.0"
}

ユニット5：Azure Container Instances からプライベート リンク対応サービスにアクセスする

プライベート エンドポイントの作成

  1. 仮想ネットワークに新しいサブネットを作成し、次にそのサブネットに Azure SQL プライベート エンドポイントを作成します。
$ sql_subnet_name=sql
$ sql_subnet_prefix=192.168.3.0/24
$ az network vnet subnet create --resource-group $rg --vnet-name $vnet_name \
    --name $sql_subnet_name --address-prefix $sql_subnet_prefix \
    --disable-private-endpoint-network-policies true
{
  "addressPrefix": "192.168.3.0/24",
  "delegations": [],
  "etag": "W/\"574371d2-0023-4dec-b9e4-bce5bf86da4b\"",
  "id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Network/virtualNetworks/acivnet/subnets/sql",
  "name": "sql",
  "privateEndpointNetworkPolicies": "Disabled",
  "privateLinkServiceNetworkPolicies": "Enabled",
  "provisioningState": "Succeeded",
  "resourceGroup": "acilab",
  "type": "Microsoft.Network/virtualNetworks/subnets"
}
$ sql_endpoint_name=sqlep
$ sql_server_id=$(az sql server show --name $sql_server_name --resource-group $rg --output tsv --query id)
$ az network private-endpoint create --name $sql_endpoint_name --resource-group $rg \
    --vnet-name $vnet_name --subnet $sql_subnet_name \
    --private-connection-resource-id $sql_server_id --group-id sqlServer \
    --connection-name sqlConnection
{
  "customDnsConfigs": [
    {
      "fqdn": "sqlserver15595.database.windows.net",
      "ipAddresses": [
        "192.168.3.4"
      ]
    }
  ],
  "customNetworkInterfaceName": "",
  "etag": "W/\"ddf4dab7-3219-402b-a848-9e8c0ccb45d5\"",
  "id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Network/privateEndpoints/sqlep",
  "ipConfigurations": [],
  "location": "japaneast",
  "manualPrivateLinkServiceConnections": [],
  "name": "sqlep",
  "networkInterfaces": [
    {
      "id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Network/networkInterfaces/sqlep.nic.5c6311f4-c5fe-4533-b968-7107f16bff35",
      "resourceGroup": "acilab"
    }
  ],
  "privateLinkServiceConnections": [
    {
      "etag": "W/\"ddf4dab7-3219-402b-a848-9e8c0ccb45d5\"",
      "groupIds": [
        "sqlServer"
      ],
      "id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Network/privateEndpoints/sqlep/privateLinkServiceConnections/sqlConnection",
      "name": "sqlConnection",
      "privateLinkServiceConnectionState": {
        "actionsRequired": "None",
        "description": "Auto-approved",
        "status": "Approved"
      },
      "privateLinkServiceId": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Sql/servers/sqlserver15595",
      "provisioningState": "Succeeded",
      "resourceGroup": "acilab",
      "type": "Microsoft.Network/privateEndpoints/privateLinkServiceConnections"
    }
  ],
  "provisioningState": "Succeeded",
  "resourceGroup": "acilab",
  "subnet": {
    "id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Network/virtualNetworks/acivnet/subnets/sql",
    "resourceGroup": "acilab"
  },
  "type": "Microsoft.Network/privateEndpoints"
  1. プライベート エンドポイントは、Azure でネットワーク インターフェイス カード (NIC) として表されるため、プライベート エンドポイントに割り当てられた IP アドレスは、az network nic コマンドを使用して確認できます。
    あ、そうなんですね。nic扱いなのか。言われてみればそうか。
$ sql_nic_id=$(az network private-endpoint show --name $sql_endpoint_name \
    --resource-group $rg --query 'networkInterfaces[0].id' -o tsv)
$ sql_endpoint_ip=$(az network nic show --ids $sql_nic_id \
    --query 'ipConfigurations[0].privateIpAddress' -o tsv) && echo $sql_endpoint_ip

$ ssh -n -o BatchMode=yes -o StrictHostKeyChecking=no $vm_pip "nslookup ${sql_server_name}.database.windows.net"
Server:         127.0.0.53
Address:        127.0.0.53#53

Non-authoritative answer:
sqlserver15595.database.windows.net     canonical name = sqlserver15595.privatelink.database.windows.net.
sqlserver15595.privatelink.database.windows.net canonical name = dataslice6.japaneast.database.windows.net.
dataslice6.japaneast.database.windows.net       canonical name = dataslice6japaneast.trafficmanager.net.
dataslice6japaneast.trafficmanager.net  canonical name = cr10.japaneast1-a.control.database.windows.net.
Name:   cr10.japaneast1-a.control.database.windows.net
Address: 13.78.104.32

DNS の解決

  1. プライベート リンクが構成された Azure SQL Database では中間ドメイン privatelink.database.windows.net が使用されるため、このドメインのプライベート ゾーンを作成し、先に作成した Azure SQL プライベート エンドポイントの IP アドレスの A レコードを追加します。
$ dns_zone_name=privatelink.database.windows.net
$ az network private-dns zone create --name $dns_zone_name --resource-group $rg 
{
  "etag": "6e4cc59c-0451-48f7-afc1-a3e6843178aa",
  "id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Network/privateDnsZones/privatelink.database.windows.net",
  "location": "global",
  "maxNumberOfRecordSets": 25000,
  "maxNumberOfVirtualNetworkLinks": 1000,
  "maxNumberOfVirtualNetworkLinksWithRegistration": 100,
  "name": "privatelink.database.windows.net",
  "numberOfRecordSets": 1,
  "numberOfVirtualNetworkLinks": 0,
  "numberOfVirtualNetworkLinksWithRegistration": 0,
  "provisioningState": "Succeeded",
  "resourceGroup": "acilab",
  "type": "Microsoft.Network/privateDnsZones"
}
$ az network private-dns link vnet create --resource-group $rg --zone-name $dns_zone_name \
    --name myDnsLink --virtual-network $vnet_name --registration-enabled false
{
  "etag": "\"1d04dd01-0000-0100-0000-651525390000\"",
  "id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Network/privateDnsZones/privatelink.database.windows.net/virtualNetworkLinks/mydnslink",
  "location": "global",
  "name": "mydnslink",
  "provisioningState": "Succeeded",
  "registrationEnabled": false,
  "resourceGroup": "acilab",
  "type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks",
  "virtualNetwork": {
    "id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Network/virtualNetworks/acivnet",
    "resourceGroup": "acilab"
  },
  "virtualNetworkLinkState": "Completed"
}
$ az network private-endpoint dns-zone-group create --endpoint-name $sql_endpoint_name \
    --resource-group $rg --name zonegroup --zone-name zone1 --private-dns-zone $dns_zone_name
{
  "etag": "W/\"06904254-a4df-42c1-a4bd-9cccb99837e2\"",
  "id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Network/privateEndpoints/sqlep/privateDnsZoneGroups/zonegroup",
  "name": "zonegroup",
  "privateDnsZoneConfigs": [
    {
      "name": "zone1",
      "privateDnsZoneId": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Network/privateDnsZones/privatelink.database.windows.net",
      "recordSets": [
        {
          "fqdn": "sqlserver15595.privatelink.database.windows.net",
          "ipAddresses": [
            "192.168.3.4"
          ],
          "provisioningState": "Succeeded",
          "recordSetName": "sqlserver15595",
          "recordType": "A",
          "ttl": 10
        }
      ]
    }
  ],
  "provisioningState": "Succeeded",
  "resourceGroup": "acilab"
}
$ ssh -n -o BatchMode=yes -o StrictHostKeyChecking=no $vm_pip "nslookup ${sql_server_name}.database.windows.net"
Server:         127.0.0.53
Address:        127.0.0.53#53

Non-authoritative answer:
sqlserver15595.database.windows.net     canonical name = sqlserver15595.privatelink.database.windows.net.
Name:   sqlserver15595.privatelink.database.windows.net
Address: 192.168.3.4
  1. 各種接続確認
$ aci_ip=$(az container show --name $aci_name --resource-group $rg \
    --query 'ipAddress.ip' --output tsv) && echo $aci_ip
192.168.2.4
$ ssh -n -o BatchMode=yes -o StrictHostKeyChecking=no $vm_pip "curl -ks https://$aci_ip/api/healthcheck"
{
  "health": "OK", 
  "version": "1.0"
}
$ ssh -n -o BatchMode=yes -o StrictHostKeyChecking=no $vm_pip "curl -ks https://$aci_ip/api/dns?fqdn=${sql_server_name}.database.windows.net"
{
  "fqdn": "sqlserver15595.database.windows.net", 
  "ip": "192.168.3.4"
}
$ ssh -n -o BatchMode=yes -o StrictHostKeyChecking=no $vm_pip "curl -ks https://$aci_ip/api/sqlversion"
{
  "sql_output": "Microsoft SQL Azure (RTM) - 12.0.2000.8 \n\tJul 17 2023 18:40:52 \n\tCopyright (C) 2022 Microsoft Corporation\n"
}
$ ssh -n -o BatchMode=yes -o StrictHostKeyChecking=no $vm_pip "curl -ks https://$aci_ip/api/sqlsrcip"
{
  "sql_output": "192.168.2.4"
}

ここまでのトポロジーはこれ。

  1. コンテナの削除
$ az container delete --name $aci_name --resource-group $rg --yes

ユニット6: 初期化コンテナーで Azure Container Instances をデプロイする

初期化スクリプトを作成する

  1. Azure サービス プリンシパルを作成。ここではContributerにしているけど、運用環境ではもうちょっと小さいロールにしてね、と言っている。
$ scope=$(az group show -n $rg --query id -o tsv)
$ new_sp=$(az ad sp create-for-rbac --scopes $scope --role Contributor --name acilab -o json)
WARNING: Creating 'Contributor' role assignment under scope '/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab'
WARNING: The output includes credentials that you must protect. Be sure that you do not include these credentials in your code or check the credentials into your source control. For more information, see https://aka.ms/azadsp-cli
$ sp_appid=$(echo $new_sp | jq -r '.appId') && echo $sp_appid
7dcd731d-b8ef-4bf9-8c83-e456d623024d
$ sp_tenant=$(echo $new_sp | jq -r '.tenant') && echo $sp_tenant
cb9bb346-c037-4fb2-a3ff-dd23544753e
$ sp_password=$(echo $new_sp | jq -r '.password')
  1. Azure プライベート DNS ゾーンを作成し、それを仮想ネットワークに関連付けます。
$ dns_zone_name=contoso.com
$ az network private-dns zone create -n $dns_zone_name -g $rg 
{
  "etag": "2200f2e9-7666-4075-b563-1eb68963d41e",
  "id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Network/privateDnsZones/contoso.com",
  "location": "global",
  "maxNumberOfRecordSets": 25000,
  "maxNumberOfVirtualNetworkLinks": 1000,
  "maxNumberOfVirtualNetworkLinksWithRegistration": 100,
  "name": "contoso.com",
  "numberOfRecordSets": 1,
  "numberOfVirtualNetworkLinks": 0,
  "numberOfVirtualNetworkLinksWithRegistration": 0,
  "provisioningState": "Succeeded",
  "resourceGroup": "acilab",
  "type": "Microsoft.Network/privateDnsZones"
}
$ az network private-dns link vnet create -g $rg -z $dns_zone_name -n contoso --virtual-network $vnet_name --registration-enabled false
{
  "etag": "\"1e04740f-0000-0100-0000-65152af70000\"",
  "id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Network/privateDnsZones/contoso.com/virtualNetworkLinks/contoso",
  "location": "global",
  "name": "contoso",
  "provisioningState": "Succeeded",
  "registrationEnabled": false,
  "resourceGroup": "acilab",
  "type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks",
  "virtualNetwork": {
    "id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Network/virtualNetworks/acivnet",
    "resourceGroup": "acilab"
  },
  "virtualNetworkLinkState": "Completed"
}
  1. Azure ファイル共有を作成し、初期化スクリプトをアップロードします。
$ storage_account_name="acilab$RANDOM"
$ az storage account create -n $storage_account_name -g $rg --sku Standard_LRS --kind StorageV2
The public access to all blobs or containers in the storage account will be disallowed by default in the future, which means default value for --allow-blob-public-access is still null but will be equivalent to false.
{
  "accessTier": "Hot",
  "allowBlobPublicAccess": true,
  "allowCrossTenantReplication": null,
  "allowSharedKeyAccess": null,
  "allowedCopyScope": null,
  "azureFilesIdentityBasedAuthentication": null,
  "blobRestoreStatus": null,
  "creationTime": "2023-09-28T07:30:50.699353+00:00",
  "customDomain": null,
  "defaultToOAuthAuthentication": null,
  "dnsEndpointType": null,
  "enableHttpsTrafficOnly": true,
  "enableNfsV3": null,
  "encryption": {
    "encryptionIdentity": null,
    "keySource": "Microsoft.Storage",
    "keyVaultProperties": null,
    "requireInfrastructureEncryption": null,
    "services": {
      "blob": {
        "enabled": true,
        "keyType": "Account",
        "lastEnabledTime": "2023-09-28T07:30:51.074438+00:00"
      },
      "file": {
        "enabled": true,
        "keyType": "Account",
        "lastEnabledTime": "2023-09-28T07:30:51.074438+00:00"
      },
      "queue": null,
      "table": null
    }
  },
  "extendedLocation": null,
  "failoverInProgress": null,
  "geoReplicationStats": null,
  "id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Storage/storageAccounts/acilab8342",
  "identity": null,
  "immutableStorageWithVersioning": null,
  "isHnsEnabled": null,
  "isLocalUserEnabled": null,
  "isSftpEnabled": null,
  "keyCreationTime": {
    "key1": "2023-09-28T07:30:50.761858+00:00",
    "key2": "2023-09-28T07:30:50.761858+00:00"
  },
  "keyPolicy": null,
  "kind": "StorageV2",
  "largeFileSharesState": null,
  "lastGeoFailoverTime": null,
  "location": "japaneast",
  "minimumTlsVersion": "TLS1_0",
  "name": "acilab8342",
  "networkRuleSet": {
    "bypass": "AzureServices",
    "defaultAction": "Allow",
    "ipRules": [],
    "resourceAccessRules": null,
    "virtualNetworkRules": []
  },
  "primaryEndpoints": {
    "blob": "https://acilab8342.blob.core.windows.net/",
    "dfs": "https://acilab8342.dfs.core.windows.net/",
    "file": "https://acilab8342.file.core.windows.net/",
    "internetEndpoints": null,
    "microsoftEndpoints": null,
    "queue": "https://acilab8342.queue.core.windows.net/",
    "table": "https://acilab8342.table.core.windows.net/",
    "web": "https://acilab8342.z11.web.core.windows.net/"
  },
  "primaryLocation": "japaneast",
  "privateEndpointConnections": [],
  "provisioningState": "Succeeded",
  "publicNetworkAccess": null,
  "resourceGroup": "acilab",
  "routingPreference": null,
  "sasPolicy": null,
  "secondaryEndpoints": null,
  "secondaryLocation": null,
  "sku": {
    "name": "Standard_LRS",
    "tier": "Standard"
  },
  "statusOfPrimary": "available",
  "statusOfSecondary": null,
  "storageAccountSkuConversionStatus": null,
  "tags": {},
  "type": "Microsoft.Storage/storageAccounts"
}
$ storage_account_key=$(az storage account keys list --account-name $storage_account_name -g $rg --query '[0].value' -o tsv)
$ az storage share create --account-name $storage_account_name --account-key $storage_account_key --name initscript
{
  "created": true
}
$ init_script_filename=init.sh
$ init_script_path=/tmp/
$ cat <<EOF > ${init_script_path}${init_script_filename}
> echo "Logging into Azure..."
az login --service-principal -u \$SP_APPID -p \$SP_PASSWORD --tenant \$SP_TENANT
echo "Finding out IP address..."
my_private_ip=\$(az container show -n \$ACI_NAME -g \$RG --query 'ipAddress.ip' -o tsv) && echo \$my_private_ip
echo "Creating DNS record..."
az network private-dns record-set a create -n \$HOSTNAME -z \$DNS_ZONE_NAME -g \$RG
az network private-dns record-set a add-record --record-set-name \$HOSTNAME -z \$DNS_ZONE_NAME -g \$RG -a \$my_private_ip
EOF
$ az storage file upload --account-name $storage_account_name --account-key $storage_account_key -s initscript --source ${init_script_path}${init_script_filename}
Finished[#############################################################]  100.0000%
{
  "content_md5": "0xd90x630xa70x130x9e0xe50x4f0x300x3b0x850xa00x7f0xfb0x330x190x64",
  "date": "2023-09-28T07:34:00+00:00",
  "etag": "\"0x8DBBFF54897072B\"",
  "file_last_write_time": "2023-09-28T07:34:00.8807211Z",
  "last_modified": "2023-09-28T07:34:00+00:00",
  "request_id": "022a576d-a01a-001a-44de-f1a8c0000000",
  "request_server_encrypted": true,
  "version": "2022-11-02"
}

init コンテナーによってコンテナー グループをデプロイする

  1. YAMLファイルを作成します。ここもユニット4同様、network profile を使った書き方なので、少し修正します。
    azure-cli のイメージのレジストリが変わっているので、imageの参照先も変えています。
$ aci_yaml_file=/tmp/acilab.yaml
$ cat <<EOF > $aci_yaml_file
apiVersion: 2021-07-01
location: japaneast
name: $aci_name
properties:
  initContainers:
  - name: azcli
    properties:
      image: mcr.microsoft.com/azure-cli
      command:
      - "/bin/sh"
      - "-c"
      - "/mnt/init/$init_script_filename"
      environmentVariables:
      - name: RG
        value: $rg
      - name: SP_APPID
        value: $sp_appid
      - name: SP_PASSWORD
        secureValue: $sp_password
      - name: SP_TENANT
        value: $sp_tenant
      - name: DNS_ZONE_NAME
        value: $dns_zone_name
      - name: HOSTNAME
        value: $aci_name
      - name: ACI_NAME
        value: $aci_name
      volumeMounts:
      - name: initscript
        mountPath: /mnt/init/
  containers:
  - name: nginx
    properties:
      image: nginx
      ports:
      - port: 443
        protocol: TCP
      resources:
        requests:
          cpu: 1.0
          memoryInGB: 1.5
      volumeMounts:
      - name: nginx-config
        mountPath: /etc/nginx
  - name: sqlapi
    properties:
      image: erjosito/sqlapi:1.0
      environmentVariables:
      - name: SQL_SERVER_FQDN
        value: $sql_server_fqdn
      - name: SQL_SERVER_USERNAME
        value: $sql_username
      - name: SQL_SERVER_PASSWORD
        secureValue: $sql_password
      ports:
      - port: 8080
        protocol: TCP
      resources:
        requests:
          cpu: 1.0
          memoryInGB: 1
      volumeMounts:
  volumes:
  - secret:
      ssl.crt: "$ssl_crt"
      ssl.key: "$ssl_key"
      nginx.conf: "$nginx_conf"
    name: nginx-config
  - name: initscript
    azureFile:
      readOnly: false
      shareName: initscript
      storageAccountName: $storage_account_name
      storageAccountKey: $storage_account_key
  ipAddress:
    ports:
    - port: 443
      protocol: TCP
    type: Private
  osType: Linux
  subnetIds:
    - id: $aci_subnet_id
      name: aci
tags: null
type: Microsoft.ContainerInstance/containerGroups
EOF
  1. Azure Container Instance を作成します。
$ az container create -g $rg --file $aci_yaml_file
{
  "extendedLocation": null,
  "id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.ContainerInstance/containerGroups/learnaci",
  "identity": null,
  "kind": null,
  "location": "japaneast",
  "managedBy": null,
  "name": "learnaci",
  "plan": null,
  "properties": {
    "containers": [
      {
        "name": "nginx",
        "properties": {
          "environmentVariables": [],
          "image": "nginx",
          "instanceView": {
            "currentState": {
              "detailStatus": "",
              "startTime": "2023-09-28T07:47:13.046Z",
              "state": "Running"
            },
            "events": [
              {
                "count": 1,
                "firstTimestamp": "2023-09-28T07:45:48Z",
                "lastTimestamp": "2023-09-28T07:45:48Z",
                "message": "pulling image \"nginx@sha256:b2888fc9cfe7cd9d6727aeb462d13c7c45dec413b66f2819a36c4a3cb9d4df75\"",
                "name": "Pulling",
                "type": "Normal"
              },
              {
                "count": 1,
                "firstTimestamp": "2023-09-28T07:46:34Z",
                "lastTimestamp": "2023-09-28T07:46:34Z",
                "message": "Successfully pulled image \"nginx@sha256:b2888fc9cfe7cd9d6727aeb462d13c7c45dec413b66f2819a36c4a3cb9d4df75\"",
                "name": "Pulled",
                "type": "Normal"
              },
              {
                "count": 1,
                "firstTimestamp": "2023-09-28T07:47:13Z",
                "lastTimestamp": "2023-09-28T07:47:13Z",
                "message": "Started container",
                "name": "Started",
                "type": "Normal"
              }
            ],
            "restartCount": 0
          },
          "ports": [
            {
              "port": 443,
              "protocol": "TCP"
            }
          ],
          "resources": {
            "requests": {
              "cpu": 1.0,
              "memoryInGB": 1.5
            }
          },
          "volumeMounts": [
            {
              "mountPath": "/etc/nginx",
              "name": "nginx-config"
            }
          ]
        }
      },
      {
        "name": "sqlapi",
        "properties": {
          "environmentVariables": [
            {
              "name": "SQL_SERVER_FQDN",
              "value": "sqlserver15595.database.windows.net"
            },
            {
              "name": "SQL_SERVER_USERNAME",
              "value": "azure"
            },
            {
              "name": "SQL_SERVER_PASSWORD"
            }
          ],
          "image": "erjosito/sqlapi:1.0",
          "instanceView": {
            "currentState": {
              "detailStatus": "",
              "startTime": "2023-09-28T07:47:13.253Z",
              "state": "Running"
            },
            "events": [
              {
                "count": 1,
                "firstTimestamp": "2023-09-28T07:45:48Z",
                "lastTimestamp": "2023-09-28T07:45:48Z",
                "message": "pulling image \"erjosito/sqlapi@sha256:a1233cdfcd53fe513fe0ea2920e3fa9bb388b5722e28ee19c218361845206899\"",
                "name": "Pulling",
                "type": "Normal"
              },
              {
                "count": 1,
                "firstTimestamp": "2023-09-28T07:46:34Z",
                "lastTimestamp": "2023-09-28T07:46:34Z",
                "message": "Successfully pulled image \"erjosito/sqlapi@sha256:a1233cdfcd53fe513fe0ea2920e3fa9bb388b5722e28ee19c218361845206899\"",
                "name": "Pulled",
                "type": "Normal"
              },
              {
                "count": 1,
                "firstTimestamp": "2023-09-28T07:47:13Z",
                "lastTimestamp": "2023-09-28T07:47:13Z",
                "message": "Started container",
                "name": "Started",
                "type": "Normal"
              }
            ],
            "restartCount": 0
          },
          "ports": [
            {
              "port": 8080,
              "protocol": "TCP"
            }
          ],
          "resources": {
            "requests": {
              "cpu": 1.0,
              "memoryInGB": 1.0
            }
          }
        }
      }
    ],
    "initContainers": [
      {
        "name": "azcli",
        "properties": {
          "command": [
            "/bin/sh",
            "-c",
            "/mnt/init/init.sh"
          ],
          "environmentVariables": [
            {
              "name": "RG",
              "value": "acilab"
            },
            {
              "name": "SP_APPID",
              "value": "7dcd731d-b8ef-4bf9-8c83-e456d623024d"
            },
            {
              "name": "SP_TENANT",
              "value": "cb9bb346-c037-4fb2-a3ff-dd23544753ea"
            },
            {
              "name": "DNS_ZONE_NAME",
              "value": "contoso.com"
            },
            {
              "name": "HOSTNAME",
              "value": "learnaci"
            },
            {
              "name": "ACI_NAME",
              "value": "learnaci"
            },
            {
              "name": "SP_PASSWORD"
            }
          ],
          "image": "mcr.microsoft.com/azure-cli",
          "instanceView": {
            "currentState": {
              "detailStatus": "Completed",
              "exitCode": 0,
              "finishTime": "2023-09-28T07:47:12.142Z",
              "startTime": "2023-09-28T07:47:00.891Z",
              "state": "Terminated"
            },
            "events": [],
            "restartCount": 0
          },
          "volumeMounts": [
            {
              "mountPath": "/mnt/init/",
              "name": "initscript"
            }
          ]
        }
      }
    ],
    "instanceView": {
      "events": [
        {
          "count": 1,
          "firstTimestamp": "2023-09-28T07:46:54.092Z",
          "lastTimestamp": "2023-09-28T07:46:54.092Z",
          "message": "Successfully mounted Azure File Volume.",
          "name": "SuccessfulMountAzureFileVolume",
          "type": "Normal"
        }
      ],
      "state": "Running"
    },
    "ipAddress": {
      "ip": "192.168.2.4",
      "ports": [
        {
          "port": 443,
          "protocol": "TCP"
        },
        {
          "port": 8080,
          "protocol": "TCP"
        }
      ],
      "type": "Private"
    },
    "isCustomProvisioningTimeout": false,
    "osType": "Linux",
    "provisioningState": "Succeeded",
    "provisioningTimeoutInSeconds": 1800,
    "sku": "Standard",
    "subnetIds": [
      {
        "id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Network/virtualNetworks/acivnet/subnets/aci",
        "name": "aci",
        "resourceGroup": "acilab"
      }
    ],
    "volumes": [
      {
        "name": "nginx-config",
        "secret": {}
      },
      {
        "azureFile": {
          "readOnly": false,
          "shareName": "initscript",
          "storageAccountName": "acilab8342"
        },
        "name": "initscript"
      }
    ]
  },
  "resourceGroup": "acilab",
  "sku": null,
  "tags": null,
  "type": "Microsoft.ContainerInstance/containerGroups"
}
  1. SQL API エンドポイントを使用して、コンテナーに到達できるようになったことをテストします
$ aci_fqdn=${aci_name}.${dns_zone_name} && echo $aci_fqdn
learnaci.contoso.com
$ ssh -n -o BatchMode=yes -o StrictHostKeyChecking=no $vm_pip "nslookup $aci_fqdn"
Server:         127.0.0.53
Address:        127.0.0.53#53

Non-authoritative answer:
Name:   learnaci.contoso.com
Address: 0.0.0.0

0.0.0.0 になってる？？
なんかIPの取得がうまくいかずに、DNSゾーンのAレコードに0.0.0.0で登録されたみたいです。

だいぶ疲れてきたのでチートします。
IPアドレスを $aci_ip の値にします。

もう一度リトライ

$ ssh -n -o BatchMode=yes -o StrictHostKeyChecking=no $vm_pip "nslookup $aci_fqdn"
Server:         127.0.0.53
Address:        127.0.0.53#53

Non-authoritative answer:
Name:   learnaci.contoso.com
Address: 192.168.2.4

$ ssh -n -o BatchMode=yes -o StrictHostKeyChecking=no $vm_pip "curl -ks https://$aci_fqdn/api/healthcheck"
{
  "health": "OK", 
  "version": "1.0"
}
$ ssh -n -o BatchMode=yes -o StrictHostKeyChecking=no $vm_pip "curl -ks https://$aci_fqdn/api/sqlversion"
{
  "sql_output": "Microsoft SQL Azure (RTM) - 12.0.2000.8 \n\tJul 17 2023 18:40:52 \n\tCopyright (C) 2022 Microsoft Corporation\n"
}
$ ssh -n -o BatchMode=yes -o StrictHostKeyChecking=no $vm_pip "curl -ks https://$aci_fqdn/api/sqlsrcip"
{
  "sql_output": "192.168.2.4"
}

疎通確認はできました。
が、このIPをちゃんと取得してくれないとinitの意味がないような？
もうちょっと理解できるようにならないと。。

  1. initコンテナのログ確認
    Finding out IP addressの後が0.0.0.0になってるんだよなぁ。
$ az container logs -n $aci_name -g $rg --container-name azcli
Logging into Azure...
[
  {
    "cloudName": "AzureCloud",
    "homeTenantId": "cb9bb346-c037-4fb2-a3ff-dd23544753ea",
    "id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
    "isDefault": true,
    "managedByTenants": [],
    "name": "検証用",
    "state": "Enabled",
    "tenantId": "cb9bb346-c037-4fb2-a3ff-dd23544753ea",
    "user": {
      "name": "7dcd731d-b8ef-4bf9-8c83-e456d623024d",
      "type": "servicePrincipal"
    }
  }
]
Finding out IP address...
0.0.0.0
Creating DNS record...
{
  "aRecords": [],
  "etag": "637c952f-4b33-4353-880f-6cf264c5b3d9",
  "fqdn": "learnaci.contoso.com.",
  "id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Network/privateDnsZones/contoso.com/A/learnaci",
  "isAutoRegistered": false,
  "name": "learnaci",
  "resourceGroup": "acilab",
  "ttl": 3600,
  "type": "Microsoft.Network/privateDnsZones/A"
}
{
  "aRecords": [
    {
      "ipv4Address": "0.0.0.0"
    }
  ],
  "etag": "b76ea1f8-0d94-4308-97dc-1fb575b20652",
  "fqdn": "learnaci.contoso.com.",
  "id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/acilab/providers/Microsoft.Network/privateDnsZones/contoso.com/A/learnaci",
  "isAutoRegistered": false,
  "name": "learnaci",
  "resourceGroup": "acilab",
  "ttl": 3600,
  "type": "Microsoft.Network/privateDnsZones/A"
}
  1. 後片付け。リソースグループごとリソースを削除します。
$ az group delete -n $rg -y --no-wait

結構簡単にできるかと思いきや、いろいろな罠にハマりました。
この記事も思ってたより長くなってしまったし。

MS Learnのメンテナンスって大変そうよね。。
あと、ネットワーク周り難しいーーーー

参考

Docker コンテナーで Azure CLI を実行する方法

Discussion