Zenn
💻

BastionのNSGの削除に失敗する

2022/08/29に公開
Azure
Terraform
tech

はじめに

TerraformでAzure Bastion用のネットワークセキュリティグループ（NSG）を作るのは問題ないのですが、destroyするとき、毎回エラーが出て、削除に失敗してました。Azure Portalからは普通に削除できるので、そこだけ毎回手で消してたのですが、調べたら、回避策があったので、それを残しておきます…と思ったのですが、色々と試した結果、現時点で最新のAzure Provider(3.20.0)を使用したら、何の問題もなく、消せました。。バグ解消された模様です。最新版では発生しないので、過去バージョンではこんなことあったな程度で残しておきたいと思います。

前提

以下のバージョンで確認してます。[1]
Terraform:1.2.8
Azure Provider(azurerm):3.17.0

対処前のコード

Bastion用のNSGとして以下のように定義してました。[2]
このコードで作成した環境をterraform destroyすると毎回NSGの削除に失敗します。[3]

main.tf
resource "azurerm_resource_group" "this" {
  name     = "dev-rg"
  location = var.resource_group_location
}

resource "azurerm_virtual_network" "this" {
  name                = "dev-vnet"
  location            = azurerm_resource_group.this.location
  resource_group_name = azurerm_resource_group.this.name
  address_space       = var.vnet_address
}

resource "azurerm_subnet" "bastion_subnet" {
  name                 = "AzureBastionSubnet"
  resource_group_name  = azurerm_resource_group.this.name
  virtual_network_name = azurerm_virtual_network.this.name
  address_prefixes     = var.vnet_bastion_subnet
}

resource "azurerm_network_security_group" "public" {
  name                = "dev-nsg-public"
  location            = azurerm_resource_group.this.location
  resource_group_name = azurerm_resource_group.this.name
}

resource "azurerm_network_security_rule" "allow_https_inbound_bastion" {
  name                        = "AllowHttpsInbound"
  priority                    = 100
  direction                   = "Inbound"
  access                      = "Allow"
  protocol                    = "Tcp"
  source_port_range           = "*"
  destination_port_range      = "443"
  source_address_prefix       = "Internet"
  destination_address_prefix  = "*"
  resource_group_name         = azurerm_resource_group.this.name
  network_security_group_name = azurerm_network_security_group.public.name
}

resource "azurerm_network_security_rule" "allow_communication_inbound_bastion" {
  name                        = "AllowBastionHostCommunicationInbound"
  priority                    = 110
  direction                   = "Inbound"
  access                      = "Allow"
  protocol                    = "*"
  source_port_range           = "*"
  destination_port_ranges     = ["8080", "5701"]
  source_address_prefix       = "VirtualNetwork"
  destination_address_prefix  = "VirtualNetwork"
  resource_group_name         = azurerm_resource_group.this.name
  network_security_group_name = azurerm_network_security_group.public.name
}

resource "azurerm_network_security_rule" "allow_gatewaymanager_inbound_bastion" {
  name                        = "AllowGatewayManagerInbound"
  priority                    = 120
  direction                   = "Inbound"
  access                      = "Allow"
  protocol                    = "Tcp"
  source_port_range           = "*"
  destination_port_range      = "443"
  source_address_prefix       = "GatewayManager"
  destination_address_prefix  = "*"
  resource_group_name         = azurerm_resource_group.this.name
  network_security_group_name = azurerm_network_security_group.public.name
}

resource "azurerm_network_security_rule" "allow_loadbalancer_inbound_bastion" {
  name                        = "AllowAzureLoadBalancerInbound"
  priority                    = 130
  direction                   = "Inbound"
  access                      = "Allow"
  protocol                    = "Tcp"
  source_port_range           = "*"
  destination_port_range      = "443"
  source_address_prefix       = "AzureLoadBalancer"
  destination_address_prefix  = "*"
  resource_group_name         = azurerm_resource_group.this.name
  network_security_group_name = azurerm_network_security_group.public.name
}

resource "azurerm_network_security_rule" "deny_all_inbound_bastion" {
  name                        = "DenyAllInbound"
  priority                    = 200
  direction                   = "Inbound"
  access                      = "Deny"
  protocol                    = "*"
  source_port_range           = "*"
  destination_port_range      = "*"
  source_address_prefix       = "*"
  destination_address_prefix  = "*"
  resource_group_name         = azurerm_resource_group.this.name
  network_security_group_name = azurerm_network_security_group.public.name
}


resource "azurerm_network_security_rule" "allow_sshrdp_outbound_bastion" {
  name                        = "AllowSshRdpOutbound"
  priority                    = 100
  direction                   = "Outbound"
  access                      = "Allow"
  protocol                    = "*"
  source_port_range           = "*"
  destination_port_ranges     = ["22", "3389"]
  source_address_prefix       = "*"
  destination_address_prefix  = "VirtualNetwork"
  resource_group_name         = azurerm_resource_group.this.name
  network_security_group_name = azurerm_network_security_group.public.name
}

resource "azurerm_network_security_rule" "allow_azurecloud_outbound_bastion" {
  name                        = "AllowAzureCloudOutbound"
  priority                    = 110
  direction                   = "Outbound"
  access                      = "Allow"
  protocol                    = "Tcp"
  source_port_range           = "*"
  destination_port_range      = "443"
  source_address_prefix       = "*"
  destination_address_prefix  = "AzureCloud"
  resource_group_name         = azurerm_resource_group.this.name
  network_security_group_name = azurerm_network_security_group.public.name
}

resource "azurerm_network_security_rule" "allow_communication_outbound_bastion" {
  name                        = "AllowBastionHostCommunicationOutbound"
  priority                    = 120
  direction                   = "Outbound"
  access                      = "Allow"
  protocol                    = "*"
  source_port_range           = "*"
  destination_port_ranges     = ["8080", "5701"]
  source_address_prefix       = "VirtualNetwork"
  destination_address_prefix  = "VirtualNetwork"
  resource_group_name         = azurerm_resource_group.this.name
  network_security_group_name = azurerm_network_security_group.public.name
}

resource "azurerm_network_security_rule" "allow_http_outbound_bastion" {
  name                        = "AllowGetSessionInformation"
  priority                    = 130
  direction                   = "Outbound"
  access                      = "Allow"
  protocol                    = "*"
  source_port_range           = "*"
  destination_port_range      = "80"
  source_address_prefix       = "*"
  destination_address_prefix  = "Internet"
  resource_group_name         = azurerm_resource_group.this.name
  network_security_group_name = azurerm_network_security_group.public.name
}

resource "azurerm_network_security_rule" "deny_all_outbound_bastion" {
  name                        = "DenyAllOutbound"
  priority                    = 200
  direction                   = "Outbound"
  access                      = "Deny"
  protocol                    = "*"
  source_port_range           = "*"
  destination_port_range      = "*"
  source_address_prefix       = "*"
  destination_address_prefix  = "*"
  resource_group_name         = azurerm_resource_group.this.name
  network_security_group_name = azurerm_network_security_group.public.name
}

resource "azurerm_subnet_network_security_group_association" "public" {
  subnet_id                 = azurerm_subnet.bastion_subnet.id
  network_security_group_id = azurerm_network_security_group.public.id
}
variables.tf
variable "resource_group_location" {
  default     = "japaneast"
  description = "Location of the resource group."
}

variable "vnet_bastion_subnet" {
  type        = list(string)
  default     = ["10.0.0.192/26"]
  description = "bastion subnet(require a minimum of /26)"
}

対処後のコード

以下のようにサブネットにNSGを適用するところで、セキュリティルールをdepends_onしてあげれば削除に失敗しなくなります。

main.tf
resource "azurerm_resource_group" "this" {
  name     = "dev-rg"
  location = var.resource_group_location
}

〜中略〜

resource "azurerm_subnet_network_security_group_association" "public" {
  subnet_id                 = azurerm_subnet.bastion_subnet.id
  network_security_group_id = azurerm_network_security_group.public.id
+  depends_on = [
+    azurerm_network_security_rule.allow_azurecloud_outbound_bastion,
+    azurerm_network_security_rule.allow_communication_inbound_bastion,
+    azurerm_network_security_rule.allow_communication_outbound_bastion,
+    azurerm_network_security_rule.allow_gatewaymanager_inbound_bastion,
+    azurerm_network_security_rule.allow_http_outbound_bastion,
+    azurerm_network_security_rule.allow_https_inbound_bastion,
+    azurerm_network_security_rule.allow_loadbalancer_inbound_bastion,
+    azurerm_network_security_rule.allow_sshrdp_outbound_bastion,
+    azurerm_network_security_rule.deny_all_inbound_bastion,
+    azurerm_network_security_rule.deny_all_outbound_bastion
  ]
}

おわりに

回避策見つけたーと思ってましたが、バグフィックスされてるというオチでした。でも調べる/試すというのが大事だなと改めて感じました。

参考にしたサイト

https://github.com/hashicorp/terraform-provider-azurerm/issues/5232

脚注

  1. 本記事を書くにあたって手元のバージョンが古かったのでTerraform自体とProviderのバージョンアップしたのですが、azurerm_network_security_ruleリソースの中のprotocolが大文字始まりになっていないと怒られました。。以前からも大文字推奨だった模様ですが、どこかで強制になってるみたいです。（tcpではなく、Tcp） ↩︎

  2. Bastion使うときにどんな通信を通さなければならないかはここに書いてあります。https://docs.microsoft.com/ja-jp/azure/bastion/bastion-nsg ↩︎

  3. こんな感じのエラーが出ます。Error: Deleting Security Rule: (Name "AllowAzureCloudOutbound" / Network Security Group Name "dev-nsg-public" / Resource Group "dev-rg"): network.SecurityRulesClient#Delete: Failure sending request: StatusCode=400 -- Original Error: Code="NetworkSecurityGroupNotCompliantForAzureBastionSubnet" Message="Network security group dev-nsg-public does not have necessary rules for Azure Bastion Subnet AzureBastionSubnet." Details=[] ↩︎

Discussion