🦔

TerraformでAWS Copilotを再現してみる

2021/12/03に公開

AWS

AWS Copilot

https://aws.github.io/copilot-cli/

AWS Coplotはアプリケーション環境を簡単に構築・反映・更新できるCLI。
これを使うことで、インフラ構成に時間をかけずに、アプリケーション開発そのものに注力しやすくなる。

Copilotにより事前に定義された構成のリソースが作成される。
これは、CloudFormationで管理されていて、ほぼほぼStackの更新で成り立っている。

作成されるリソースにはIAMを含め、必要なリソースが全て含まれている。
つまり、AWSにおけるインフラ構成を学ぶのにはちょうどよい。

そこで、TerraformでAWS Copilotを再現してみることで、
Terraform・AWS・Copilotをまとめて理解を深めてみたいと思う。

概要

再現すると言っても100％再現するのは少し面倒くさい部分があるので、
9割ぐらいの再現度を目指したいと思う。

まず、Copilotにより作成される、システム構成を確認する。

CopilotではApplication・Environment・Service・Job・Pipelineといった概念が登場する。
ApplicationはEnvironment・Service・Job・Pipelineを束ねる、一番大枠に位置する概念である。
Applicationの中には複数のEnvironmentがあり、Prod・Stg・Devのように環境を分けられるようになっている。
そして、各Environmentの中にServiceやJobを適宜配置していく。
Service・Jobには事前に定められた型が存在するので、配置する際にはその中から選択する形になる。
最後に、PipelineはデプロイするためのCI・CDパイプラインである。

図に示すと次のような感じになる。
（多少まちがっている部分もあるとは思うが、だいたいこんな感じなはず）

各EnvironmentごとにVPCを作成し、その中にECSのServiceやTaskを配置していく。
また、ECR RepositoryはApplication内で共有する形になる。これによりEnvironment(dev)で動作確認して、同じイメージをEnvironment(prod)に反映するといった事ができる。

図にはのっけていないが、Security GroupもEnvironment単位で共有する形になっていて、同じEnvironment間であれば通信可能な形となっている。ECS Service間の通信はService Discoveryを通してホスト名を解決できる。

で、これらをTerraformを使って再現してみる。

Application

まずは、Applicationのを作成する。
Application自体はParameter Storeに情報があるだけなので、指定の名前で保存できるようにする。

modules/app/main.tf

variable "app_name" {
  type = string
}

locals {
  tags = {
    application = var.app_name
  }
}

resource "aws_ssm_parameter" "this" {
  name = "/applications/${var.app_name}"
  type = "String"
  value = jsonencode({
    "name": var.app_name
  })
  tags = local.tags
}

また、各Serviceで使うECR RepositoryはApplication内で共有される。
そのため、Applicationと同じところで宣言できるよう、こちらもModuleを作っておく。

modules/app_svc/main.tf

variable "app_name" {
  type = string
}

variable "svc_name" {
  type = string
}

locals {
  tags = {
    application = var.app_name
    service = var.svc_name
  }
}

resource "aws_ecr_repository" "this" {
  name = "${var.app_name}/${var.svc_name}"
  image_tag_mutability = "MUTABLE"
  image_scanning_configuration {
    scan_on_push = true
  }
  tags = local.tags
}

これらのModuleを使ってApplicationを宣言すると次のような感じになる。
ここではApplication名を”tf_copilot”としている。

app/main.tf

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 3.0"
    }
  }
}

locals {
  app_name = "tf_copilot"
}

provider "aws" {
  region = "ap-northeast-1"
}

module "app" {
  source = "../modules/app"
  app_name = local.app_name
}

module "app_svc_rdws" {
  source = "../modules/app_svc"
  app_name = local.app_name
  svc_name = "rdws"
}

module "app_svc_lbws" {
  source = "../modules/app_svc"
  app_name = local.app_name
  svc_name = "lbws"
}

module "app_svc_bs" {
  source = "../modules/app_svc"
  app_name = local.app_name
  svc_name = "bs"
}

module "app_svc_ws" {
  source = "../modules/app_svc"
  app_name = local.app_name
  svc_name = "ws"
}

module "app_job_sj" {
  source = "../modules/app_job"
  app_name = local.app_name
  job_name = "sj"
}

これでterraform applyすると、各ServiceのECR Repositoryが作成されるので、事前にイメージを上げておくと良い。
イメージが無いと後の部分でterraform applyしたときに、そもそも反映できないので。

特に、App Runnerは反映に失敗した場合、30分ほど何も操作できない状態に陥るので注意したほうがよい。。。

Environment

次にEnvironmentを作成する。

Environmentでは、VPC・ECS Cluster・ALB・Private DNS Namespaceあたりを作成する必要がある。
VPCの中にはAZを2つ配置し、各AZ内にPublic Subnet・Private Subnetを1つずつ配置する。
また、ALBではHTTPSの通信を前提として、リクエストを受けられる状態にしておく。（実際のCopilotではHTTPSを強制されないが、ここでは簡略化してそうする）
Environment内で共有して使うSecurity Groupもここで作成する。

modules/env/main.tf

variable "app_name" {
  type = string
}

variable "env_name" {
  type = string
}

variable "certificate_arn" {
  type = string
}

variable "extra_certificate_arns" {
  type = list(string)
  default = []
}

variable "aliases" {
  type = list(object({
    zone_id = string
    name = string
  }))
  default = []
}

locals {
  vpc_name = "${var.app_name}-${var.env_name}"
  env_sg_name = "${var.app_name}-${var.env_name}-env"
  ecs_cluster_name = "${var.app_name}-${var.env_name}"
  lb_name = replace("${var.app_name}-${var.env_name}-public", "_", "-")
  lb_sg_name = "${var.app_name}-${var.env_name}-lb"
  discovery_dns_namespace = "${var.env_name}.${var.app_name}.local"
  extra_certificate_arns = toset(var.extra_certificate_arns)
  aliases = {for a in var.aliases : a.name => {
    zone_id = a.zone_id
    name = a.name
  }}
  tags = {
    application = var.app_name
    environment = var.env_name
  }
}

data "aws_region" "this" {}

##################################################
# Systems Manager Parameter Store
##################################################

resource "aws_ssm_parameter" "this" {
  name = "/applications/${var.app_name}/environments/${var.env_name}"
  type = "String"
  value = jsonencode({
    "name": var.env_name,
    "app_name": var.app_name,
  })
  tags = local.tags
}

##################################################
# VPC
##################################################

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "3.11.0"

  name = local.vpc_name
  cidr = "10.0.0.0/16"
  azs = ["${data.aws_region.this.name}a", "${data.aws_region.this.name}c"]
  public_subnets = ["10.0.0.0/24", "10.0.1.0/24"]
  private_subnets = ["10.0.2.0/24", "10.0.3.0/24"]
  public_subnet_tags = { public: "true" }
  private_subnet_tags = { public: "false" }
  enable_dns_hostnames = true
  enable_dns_support = true
  tags = local.tags
}

resource "aws_security_group" "lb" {
  name = local.lb_sg_name
  vpc_id = module.vpc.vpc_id
  ingress {
    description = "Allow HTTP"
    from_port = 80
    to_port = 80
    protocol = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
  ingress {
    description = "Allow HTTPS"
    from_port = 443
    to_port = 443
    protocol = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
  egress {
    description = ""
    from_port = 0
    to_port = 0
    protocol = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
  tags = merge({ Name: local.lb_sg_name }, local.tags)
}

resource "aws_security_group" "env" {
  name = local.env_sg_name
  vpc_id = module.vpc.vpc_id
  ingress {
    description = ""
    from_port = 0
    to_port = 0
    protocol = "-1"
    self = true
  }
  ingress {
    description = "Allow ALB to Environment"
    from_port = 0
    to_port = 0
    protocol = "-1"
    security_groups = [aws_security_group.lb.id]
  }
  egress {
    description = ""
    from_port = 0
    to_port = 0
    protocol = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
  tags = merge({ Name: local.env_sg_name }, local.tags)
}

##################################################
# ECS Cluster
##################################################

module "ecs" {
  source  = "terraform-aws-modules/ecs/aws"
  version = "3.4.1"

  name = local.ecs_cluster_name
  container_insights = true
  capacity_providers = ["FARGATE"]
  default_capacity_provider_strategy = [
    { capacity_provider = "FARGATE" }
  ]
  tags = local.tags
}

##################################################
# Public ALB
##################################################

resource "aws_lb" "this" {
  name = local.lb_name
  internal = false
  load_balancer_type = "application"
  security_groups = [aws_security_group.lb.id]
  subnets = module.vpc.public_subnets
  tags = local.tags
}

resource "aws_lb_listener" "http" {
  load_balancer_arn = aws_lb.this.arn
  port = "80"
  protocol = "HTTP"
  default_action {
    type = "redirect"
    redirect {
      port = "443"
      protocol = "HTTPS"
      status_code = "HTTP_301"
    }
  }
  tags = local.tags
}

resource "aws_lb_listener" "https" {
  load_balancer_arn = aws_lb.this.arn
  port = "443"
  protocol = "HTTPS"
  certificate_arn = var.certificate_arn
  default_action {
    type = "fixed-response"
    fixed_response {
      content_type = "text/plain"
      message_body = "503 Service Temporarily Unavailable"
      status_code = "503"
    }
  }
  tags = local.tags
}

resource "aws_alb_listener_certificate" "https" {
  for_each = local.extra_certificate_arns
  listener_arn = aws_lb.this.arn
  certificate_arn = each.key
}

resource "aws_route53_record" "this" {
  for_each = local.aliases
  zone_id = local.aliases[each.key].zone_id
  name = local.aliases[each.key].name
  type = "A"
  alias {
    name = aws_lb.this.dns_name
    zone_id = aws_lb.this.zone_id
    evaluate_target_health = true
  }
}

##################################################
# Service Discovery
##################################################

resource "aws_service_discovery_private_dns_namespace" "this" {
  name = local.discovery_dns_namespace
  vpc = module.vpc.vpc_id
}

このModuleを使ってEnvironmentを宣言すると次のような感じになる。
ここではEnvironment名を”dev”としている。

ただし、Hosted Zoneは事前に作成された状態であることとし、ACMに関してもModule外で作成することとする。

env_dev/main.tf

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 3.0"
    }
  }
}

locals {
  app_name = "tf_copilot"
  env_name = "dev"
}

provider "aws" {
  region = "ap-northeast-1"
}

data "aws_route53_zone" "aws" {
  name = "aws.okto.page"
}

module "acm" {
  source  = "terraform-aws-modules/acm/aws"
  version = "3.2.1"

  zone_id = data.aws_route53_zone.aws.id
  domain_name = "aws.okto.page"
  subject_alternative_names = [
    "lbws.aws.okto.page"
  ]
  wait_for_validation = true
}

module "env" {
  source = "../modules/env"
  app_name = local.app_name
  env_name = local.env_name
  certificate_arn = module.acm.acm_certificate_arn
  aliases = [
    {
      zone_id = data.aws_route53_zone.aws.zone_id
      name = "lbws.aws.okto.page"
    }
  ]
}

これでterraform applyするとEnvironmentが作成できる。
ApplicationとEnvironmentでそれぞれtfstateを別にしているので、先にEnvironment(dev)だけ構成を変えて、問題なければEnvironment(prod)の方も構成を変える、といった事がやりやすいかと思う。

Request-Driven Web Service

ここからは、各Service・Jobを作成していく。
まずは、Request-Driven Web Serviceから。

このServiceはApp Runnerを使っているので、基本的にはそれだけである。
また、Worker ServiceとのPub/Subができるよう、SNS Topicも必要に応じて作成する。

modules/svc_sns_topics/main.tf

variable "app_name" {
  type = string
}

variable "env_name" {
  type = string
}

variable "svc_name" {
  type = string
}

variable "topic_names" {
  type  = set(string)
}

locals {
  tags = {
    application = var.app_name
    environment = var.env_name
    service = var.svc_name
  }
}

data "aws_caller_identity" "this" {}

resource "aws_sns_topic" "this" {
  for_each = var.topic_names
  name = "${var.app_name}-${var.env_name}-${var.svc_name}-${each.key}"
  tags = local.tags
}

resource "aws_sns_topic_policy" "this" {
  for_each = var.topic_names
  arn = aws_sns_topic.this[each.key].arn
  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Principal = { AWS = "arn:aws:iam::${data.aws_caller_identity.this.account_id}:root" }
        Action = "sns:Subscribe"
        Resource = aws_sns_topic.this[each.key].arn
        Condition = {
          StringEquals = {
            "sns:Protocol": "sqs"
          }
        }
      }
    ]
  })
}

output "publish_to_sns_policy" {
  value = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Action = "sns:Publish"
        Resource = [for k, v in var.topic_names : aws_sns_topic.this[k].arn]
      }
    ]
  })
}

output "topic_arns_env_value" {
  value = jsonencode({for k, v in var.topic_names : k => aws_sns_topic.this[k].arn})
}

modules/env_svc_rdws/main.tf

variable "app_name" {
  type = string
}

variable "env_name" {
  type = string
}

variable "svc_name" {
  type = string
}

variable "port" {
  type = number
  default = 80
}

variable "cpu" {
  type = number
  default = 1024
}

variable "memory" {
  type = number
  default = 2048
}

variable "image_tag" {
  type = string
  default = "latest"
}

variable "topic_names" {
  type = set(string)
  default = []
}

locals {
  access_role_name = "${var.app_name}-${var.env_name}-${var.svc_name}-access_role"
  instance_role_name = "${var.app_name}-${var.env_name}-${var.svc_name}-instance_role"
  app_runner_service_name = "${var.app_name}-${var.env_name}-${var.svc_name}"
  ecr_image_name = "${var.app_name}/${var.svc_name}"
  tags = {
    application = var.app_name
    environment = var.env_name
    service = var.svc_name
  }
}

data "aws_caller_identity" "this" {}

data "aws_ecr_repository" "this" {
  name = local.ecr_image_name
}

data "aws_ecr_image" "this" {
  repository_name = data.aws_ecr_repository.this.name
  image_tag = var.image_tag
}

##################################################
# SNS Topics
##################################################

module "svc_sns_topics" {
  source = "../svc_sns_topics"
  app_name = var.app_name
  env_name = var.env_name
  svc_name = var.svc_name
  topic_names = var.topic_names
}

##################################################
# App Runner
##################################################

resource "aws_iam_role" "access_role" {
  name = local.access_role_name
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Principal = { Service = "build.apprunner.amazonaws.com" }
        Action = "sts:AssumeRole"
      }
    ]
  })
  managed_policy_arns = [
    "arn:aws:iam::aws:policy/service-role/AWSAppRunnerServicePolicyForECRAccess"
  ]
  tags = local.tags
}

resource "aws_iam_role" "instance_role" {
  name = local.instance_role_name
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Principal = { Service = "tasks.apprunner.amazonaws.com" }
        Action = "sts:AssumeRole"
      }
    ]
  })
  inline_policy {
    name = "deny_iam_except_tagged_roles"
    policy = jsonencode({
      Version = "2012-10-17"
      Statement = [
        {
          Effect = "Deny"
          Action = "iam:*"
          Resource = "*"
        },
        {
          Effect = "Allow"
          Action = "sts:AssumeRole"
          Resource = "arn:aws:iam::${data.aws_caller_identity.this.account_id}:role/*"
          Condition = {
            StringEquals: {
              "iam:ResourceTag/application": var.app_name
              "iam:ResourceTag/environment": var.env_name
            }
          }
        }
      ]
    })
  }
  dynamic "inline_policy" {
    for_each = length(var.topic_names) == 0 ? [] : [1]
    content {
      name = "publish_to_sns"
      policy = module.svc_sns_topics.publish_to_sns_policy
    }
  }
  tags = local.tags
}

resource "aws_apprunner_service" "this" {
  service_name = local.app_runner_service_name
  source_configuration {
    authentication_configuration {
      access_role_arn = aws_iam_role.access_role.arn
    }
    image_repository {
      image_configuration {
        port = tostring(var.port)
        runtime_environment_variables = {
          APPLICATION_NAME = var.app_name
          ENVIRONMENT_NAME = var.env_name
          SERVICE_NAME = var.svc_name
          SNS_TOPIC_ARNS = module.svc_sns_topics.topic_arns_env_value
        }
      }
      image_identifier = "${data.aws_ecr_repository.this.repository_url}:${data.aws_ecr_image.this.image_tag}"
      image_repository_type = "ECR"
    }
    auto_deployments_enabled = false
  }
  instance_configuration {
    cpu = var.cpu
    memory = var.memory
    instance_role_arn = aws_iam_role.instance_role.arn
  }
  tags = local.tags
}

このModuleを使ってServiceを宣言すると次のような感じになる。
ここではService名を”rdws”としている。

env_dev_svcs

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 3.0"
    }
  }
}

locals {
  app_name = "tf_copilot"
  env_name = "dev"
}

provider "aws" {
  region = "ap-northeast-1"
}

module "svc_rdws" {
  source = "../modules/env_svc_rdws"
  app_name = local.app_name
  env_name = local.env_name
  svc_name = "rdws"
}

これでterraform applyするとEnvironment内にServiceが作成できる。

Load Balanced Web Service

つぎは、Load Blanced Web Serviceを作成する。

このServiceはECS Serviceを使っている。
また、ALBを通じてインターネットからアクセス可能な形となる。

なので、ECS Task Definition・ECS Service・Listener Ruleあたりを作成する必要がある。

modules/env_svc_lbws/main.tf

variable "app_name" {
  type = string
}

variable "env_name" {
  type = string
}

variable "svc_name" {
  type = string
}

variable "port" {
  type = number
  default = 80
}

variable "cpu" {
  type = number
  default = 256
}

variable "memory" {
  type = number
  default = 512
}

variable "desired_count" {
  type = number
  default = 1
}

variable "log_retention" {
  type = number
  default = 30
}

variable "image_tag" {
  type = string
  default = "latest"
}

variable "alias_names" {
  type = set(string)
}

variable "health_check_path" {
  type = string
  default = "/"
}

variable "topic_names" {
  type = set(string)
  default = []
}

locals {
  execution_role_name = "${var.app_name}-${var.env_name}-${var.svc_name}-execution_role"
  task_role_name = "${var.app_name}-${var.env_name}-${var.svc_name}-task_role"
  task_def_log_group_name = "${var.app_name}-${var.env_name}-${var.svc_name}"
  task_def_name = "${var.app_name}-${var.env_name}-${var.svc_name}"
  task_def_container_name = var.svc_name
  ecr_image_name = "${var.app_name}/${var.svc_name}"
  ecs_cluster_name = "${var.app_name}-${var.env_name}"
  ecs_service_name = "${var.app_name}-${var.env_name}-${var.svc_name}"
  ecs_service_tg_name = replace("${var.app_name}-${var.env_name}-${var.svc_name}", "_", "-")
  env_sg_name = "${var.app_name}-${var.env_name}-env"
  lb_name = replace("${var.app_name}-${var.env_name}-public", "_", "-")
  discovery_dns_namespace = "${var.env_name}.${var.app_name}.local"
  tags = {
    application = var.app_name
    environment = var.env_name
    service = var.svc_name
  }
}

data "aws_caller_identity" "this" {}

data "aws_region" "this" {}

data "aws_ecr_repository" "this" {
  name = local.ecr_image_name
}

data "aws_ecr_image" "this" {
  repository_name = data.aws_ecr_repository.this.name
  image_tag = var.image_tag
}

data "aws_ecs_cluster" "this" {
  cluster_name = local.ecs_cluster_name
}

data "aws_vpc" "this" {
  tags = {
    application = var.app_name
    environment = var.env_name
  }
}

data "aws_security_group" "env" {
  vpc_id = data.aws_vpc.this.id
  tags = { Name = local.env_sg_name }
}

data "aws_subnet_ids" "public" {
  vpc_id = data.aws_vpc.this.id
  tags = { public = "true" }
}

data "aws_lb" "this" {
  name = local.lb_name
}

data "aws_lb_listener" "https" {
  load_balancer_arn = data.aws_lb.this.arn
  port = 443
}

##################################################
# SNS Topics
##################################################

module "svc_sns_topics" {
  source = "../svc_sns_topics"
  app_name = var.app_name
  env_name = var.env_name
  svc_name = var.svc_name
  topic_names = var.topic_names
}

##################################################
# ECS Task Definition
##################################################

resource "aws_iam_role" "execution_role" {
  name = local.execution_role_name
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Principal = { Service = "ecs-tasks.amazonaws.com" }
        Action = "sts:AssumeRole"
      }
    ]
  })
  managed_policy_arns = [
    "arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy"
  ]
  inline_policy {
    name = "secrets"
    policy = jsonencode({
      Version = "2012-10-17"
      Statement = [
        {
          Effect = "Allow"
          Action = "ssm:GetParameters"
          Resource = "arn:aws:ssm::${data.aws_region.this.name}:${data.aws_caller_identity.this.account_id}:parameter/*"
          Condition = {
            StringEquals: {
              "ssm:ResourceTag/application": var.app_name
              "ssm:ResourceTag/environment": var.env_name
            }
          }
        }
      ]
    })
  }
  tags = local.tags
}

resource "aws_iam_role" "task_role" {
  name = local.task_role_name
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Principal = { Service = "ecs-tasks.amazonaws.com" }
        Action = "sts:AssumeRole"
      }
    ]
  })
  inline_policy {
    name = "deny_iam_except_tagged_roles"
    policy = jsonencode({
      Version = "2012-10-17"
      Statement = [
        {
          Effect = "Deny"
          Action = "iam:*"
          Resource = "*"
        },
        {
          Effect = "Allow"
          Action = "sts:AssumeRole"
          Resource = "arn:aws:iam::${data.aws_caller_identity.this.account_id}:role/*"
          Condition = {
            StringEquals: {
              "iam:ResourceTag/application": var.app_name
              "iam:ResourceTag/environment": var.env_name
            }
          }
        }
      ]
    })
  }
  inline_policy {
    name = "execute_command"
    policy = jsonencode({
      Version = "2012-10-17"
      Statement = [
        {
          Effect = "Allow"
          Action = [
            "ssmmessages:CreateControlChannel",
            "ssmmessages:OpenControlChannel",
            "ssmmessages:CreateDataChannel",
            "ssmmessages:OpenDataChannel",
          ],
          Resource = "*"
        },
        {
          Effect = "Allow"
          Action = [
            "logs:CreateLogStream",
            "logs:DescribeLogGroups",
            "logs:DescribeLogStreams",
            "logs:PutLogEvents",
          ],
          Resource = "*"
        }
      ]
    })
  }
  dynamic "inline_policy" {
    for_each = length(var.topic_names) == 0 ? [] : [1]
    content {
      name = "publish_to_sns"
      policy = module.svc_sns_topics.publish_to_sns_policy
    }
  }
  tags = local.tags
}

resource "aws_cloudwatch_log_group" "this" {
  name = local.task_def_log_group_name
  retention_in_days = var.log_retention
  tags = local.tags
}

resource "aws_ecs_task_definition" "this" {
  family = local.task_def_name
  network_mode = "awsvpc"
  requires_compatibilities = ["FARGATE"]
  cpu = var.cpu
  memory = var.memory
  execution_role_arn = aws_iam_role.execution_role.arn
  task_role_arn = aws_iam_role.task_role.arn
  container_definitions = jsonencode([
    {
      name = local.task_def_container_name
      image = "${data.aws_ecr_repository.this.repository_url}:${data.aws_ecr_image.this.image_tag}"
      portMappings = [
        { containerPort: var.port }
      ]
      environment = [
        { name = "APPLICATION_NAME", value = var.app_name },
        { name = "ENVIRONMENT_NAME", value = var.env_name },
        { name = "SERVICE_NAME", value = var.svc_name },
        { name = "SNS_TOPIC_ARNS", value = module.svc_sns_topics.topic_arns_env_value },
        { name = "SERVICE_DISCOVERY_ENDPOINT", value = local.discovery_dns_namespace },
      ]
      logConfiguration = {
        logDriver = "awslogs"
        options = {
          awslogs-region: data.aws_region.this.name
          awslogs-group: aws_cloudwatch_log_group.this.name
          awslogs-stream-prefix: "service"
        }
      }
    }
  ])
  tags = local.tags
}

##################################################
# ECS Service
##################################################

resource "aws_lb_target_group" "this" {
  name = local.ecs_service_tg_name
  vpc_id = data.aws_vpc.this.id
  target_type = "ip"
  port = var.port
  protocol = "HTTP"
  deregistration_delay = 60
  health_check { path = var.health_check_path }
  tags = local.tags
}

resource "aws_ecs_service" "this" {
  name = local.ecs_service_name
  platform_version = "LATEST"
  cluster = data.aws_ecs_cluster.this.id
  task_definition = aws_ecs_task_definition.this.arn
  desired_count = var.desired_count
  deployment_minimum_healthy_percent = 100
  deployment_maximum_percent = 200
  propagate_tags = "SERVICE"
  enable_execute_command = true
  launch_type = "FARGATE"
  health_check_grace_period_seconds = 60
  deployment_circuit_breaker {
    enable = true
    rollback = true
  }
  network_configuration {
    assign_public_ip = true
    subnets = data.aws_subnet_ids.public.ids
    security_groups = [data.aws_security_group.env.id]
  }
  load_balancer {
    target_group_arn = aws_lb_target_group.this.arn
    container_name = local.task_def_container_name
    container_port = var.port
  }
  tags = local.tags
}

resource "aws_lb_listener_rule" "forward" {
  listener_arn = data.aws_lb_listener.https.arn
  priority = 50000
  action {
    type = "forward"
    target_group_arn = aws_lb_target_group.this.arn
  }
  condition {
    path_pattern { values = ["/*"] }
  }
  condition {
    host_header { values = var.alias_names }
  }
  tags = local.tags
}

このModuleを使ってServiceを宣言すると次のような感じになる。
ここではService名を”lbws”としている。

また、HTTPSでアクセスする際のドメイン名も指定している。
これに対応するHosted Zoneは事前に作成されていることとする。

env_dev_svcs/main.tf

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 3.0"
    }
  }
}

locals {
  app_name = "tf_copilot"
  env_name = "dev"
}

provider "aws" {
  region = "ap-northeast-1"
}

module "svc_rdws" {
  source = "../modules/env_svc_rdws"
  app_name = local.app_name
  env_name = local.env_name
  svc_name = "rdws"
}

module "svc_lbws" {
  source = "../modules/env_svc_lbws"
  app_name = local.app_name
  env_name = local.env_name
  svc_name = "lbws"
  topic_names = ["hello"]
  alias_names = ["lbws.aws.okto.page"]
}

これでterraform applyするとEnvironment内にServiceが作成できる。

Backend Service

つぎは、Backend Serviceを作成する。

このServiceはECS Serviceを使っている。
また、先ほどとは異なりインターネットからアクセス不可な形となる。
そのため、Service Discoveryを使い、VPC内で通信できるようにする。

なので、ECS Task Definition・ECS Service・Service Discovery Serviceあたりを作成する必要がある。

modules/env_svc_bs/main.tf

variable "app_name" {
  type = string
}

variable "env_name" {
  type = string
}

variable "svc_name" {
  type = string
}

variable "port" {
  type = number
  default = 80
}

variable "cpu" {
  type = number
  default = 256
}

variable "memory" {
  type = number
  default = 512
}

variable "desired_count" {
  type = number
  default = 1
}

variable "log_retention" {
  type = number
  default = 30
}

variable "image_tag" {
  type = string
  default = "latest"
}

variable "topic_names" {
  type = set(string)
  default = []
}

locals {
  execution_role_name = "${var.app_name}-${var.env_name}-${var.svc_name}-execution_role"
  task_role_name = "${var.app_name}-${var.env_name}-${var.svc_name}-task_role"
  task_def_log_group_name = "${var.app_name}-${var.env_name}-${var.svc_name}"
  task_def_name = "${var.app_name}-${var.env_name}-${var.svc_name}"
  task_def_container_name = var.svc_name
  ecr_image_name = "${var.app_name}/${var.svc_name}"
  ecs_cluster_name = "${var.app_name}-${var.env_name}"
  ecs_service_name = "${var.app_name}-${var.env_name}-${var.svc_name}"
  env_sg_name = "${var.app_name}-${var.env_name}-env"
  discovery_dns_namespace = "${var.env_name}.${var.app_name}.local"
  tags = {
    application = var.app_name
    environment = var.env_name
    service = var.svc_name
  }
}

data "aws_caller_identity" "this" {}

data "aws_region" "this" {}

data "aws_ecr_repository" "this" {
  name = local.ecr_image_name
}

data "aws_ecr_image" "this" {
  repository_name = data.aws_ecr_repository.this.name
  image_tag = var.image_tag
}

data "aws_ecs_cluster" "this" {
  cluster_name = local.ecs_cluster_name
}

data "aws_vpc" "this" {
  tags = {
    application = var.app_name
    environment = var.env_name
  }
}

data "aws_security_group" "env" {
  vpc_id = data.aws_vpc.this.id
  tags = { Name = local.env_sg_name }
}

data "aws_subnet_ids" "public" {
  vpc_id = data.aws_vpc.this.id
  tags = { public = "true" }
}

data "aws_service_discovery_dns_namespace" "this" {
  name = local.discovery_dns_namespace
  type = "DNS_PRIVATE"
}

##################################################
# SNS Topics
##################################################

module "svc_sns_topics" {
  source = "../svc_sns_topics"
  app_name = var.app_name
  env_name = var.env_name
  svc_name = var.svc_name
  topic_names = var.topic_names
}

##################################################
# ECS Task Definition
##################################################

resource "aws_iam_role" "execution_role" {
  name = local.execution_role_name
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Principal = { Service = "ecs-tasks.amazonaws.com" }
        Action = "sts:AssumeRole"
      }
    ]
  })
  managed_policy_arns = [
    "arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy"
  ]
  inline_policy {
    name = "secrets"
    policy = jsonencode({
      Version = "2012-10-17"
      Statement = [
        {
          Effect = "Allow"
          Action = "ssm:GetParameters"
          Resource = "arn:aws:ssm::${data.aws_region.this.name}:${data.aws_caller_identity.this.account_id}:parameter/*"
          Condition = {
            StringEquals: {
              "ssm:ResourceTag/application": var.app_name
              "ssm:ResourceTag/environment": var.env_name
            }
          }
        }
      ]
    })
  }
  tags = local.tags
}

resource "aws_iam_role" "task_role" {
  name = local.task_role_name
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Principal = { Service = "ecs-tasks.amazonaws.com" }
        Action = "sts:AssumeRole"
      }
    ]
  })
  inline_policy {
    name = "deny_iam_except_tagged_roles"
    policy = jsonencode({
      Version = "2012-10-17"
      Statement = [
        {
          Effect = "Deny"
          Action = "iam:*"
          Resource = "*"
        },
        {
          Effect = "Allow"
          Action = "sts:AssumeRole"
          Resource = "arn:aws:iam::${data.aws_caller_identity.this.account_id}:role/*"
          Condition = {
            StringEquals: {
              "iam:ResourceTag/application": var.app_name
              "iam:ResourceTag/environment": var.env_name
            }
          }
        }
      ]
    })
  }
  inline_policy {
    name = "execute_command"
    policy = jsonencode({
      Version = "2012-10-17"
      Statement = [
        {
          Effect = "Allow"
          Action = [
            "ssmmessages:CreateControlChannel",
            "ssmmessages:OpenControlChannel",
            "ssmmessages:CreateDataChannel",
            "ssmmessages:OpenDataChannel",
          ],
          Resource = "*"
        },
        {
          Effect = "Allow"
          Action = [
            "logs:CreateLogStream",
            "logs:DescribeLogGroups",
            "logs:DescribeLogStreams",
            "logs:PutLogEvents",
          ],
          Resource = "*"
        }
      ]
    })
  }
  dynamic "inline_policy" {
    for_each = length(var.topic_names) == 0 ? [] : [1]
    content {
      name = "publish_to_sns"
      policy = module.svc_sns_topics.publish_to_sns_policy
    }
  }
  tags = local.tags
}

resource "aws_cloudwatch_log_group" "this" {
  name = local.task_def_log_group_name
  retention_in_days = var.log_retention
  tags = local.tags
}

resource "aws_ecs_task_definition" "this" {
  family = local.task_def_name
  network_mode = "awsvpc"
  requires_compatibilities = ["FARGATE"]
  cpu = var.cpu
  memory = var.memory
  execution_role_arn = aws_iam_role.execution_role.arn
  task_role_arn = aws_iam_role.task_role.arn
  container_definitions = jsonencode([
    {
      name = local.task_def_container_name
      image = "${data.aws_ecr_repository.this.repository_url}:${data.aws_ecr_image.this.image_tag}"
      portMappings = [
        { containerPort: var.port }
      ]
      environment = [
        { name = "APPLICATION_NAME", value = var.app_name },
        { name = "ENVIRONMENT_NAME", value = var.env_name },
        { name = "SERVICE_NAME", value = var.svc_name },
        { name = "SNS_TOPIC_ARNS", value = module.svc_sns_topics.topic_arns_env_value },
        { name = "SERVICE_DISCOVERY_ENDPOINT", value = local.discovery_dns_namespace },
      ]
      logConfiguration = {
        logDriver = "awslogs"
        options = {
          awslogs-region: data.aws_region.this.name
          awslogs-group: aws_cloudwatch_log_group.this.name
          awslogs-stream-prefix: "service"
        }
      }
    }
  ])
  tags = local.tags
}

##################################################
# Service Discovery
##################################################

resource "aws_service_discovery_service" "this" {
  name = var.svc_name
  dns_config {
    namespace_id = data.aws_service_discovery_dns_namespace.this.id
    routing_policy = "MULTIVALUE"
    dns_records {
      ttl = 10
      type = "A"
    }
    dns_records {
      ttl = 10
      type = "SRV"
    }
  }
  health_check_custom_config {
    failure_threshold = 1
  }
  tags = local.tags
}

##################################################
# ECS Service
##################################################

resource "aws_ecs_service" "this" {
  name = local.ecs_service_name
  platform_version = "LATEST"
  cluster = data.aws_ecs_cluster.this.id
  task_definition = aws_ecs_task_definition.this.arn
  desired_count = var.desired_count
  deployment_minimum_healthy_percent = 100
  deployment_maximum_percent = 200
  propagate_tags = "SERVICE"
  enable_execute_command = true
  launch_type = "FARGATE"
  deployment_circuit_breaker {
    enable = true
    rollback = true
  }
  network_configuration {
    assign_public_ip = true
    subnets = data.aws_subnet_ids.public.ids
    security_groups = [data.aws_security_group.env.id]
  }
  service_registries {
    registry_arn = aws_service_discovery_service.this.arn
    port = var.port
  }
  tags = local.tags
}

このModuleを使ってServiceを宣言すると次のような感じになる。
ここではService名を”bs”としている。

env_dev_svcs/main.tf

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 3.0"
    }
  }
}

locals {
  app_name = "tf_copilot"
  env_name = "dev"
}

provider "aws" {
  region = "ap-northeast-1"
}

module "svc_rdws" {
  source = "../modules/env_svc_rdws"
  app_name = local.app_name
  env_name = local.env_name
  svc_name = "rdws"
}

module "svc_lbws" {
  source = "../modules/env_svc_lbws"
  app_name = local.app_name
  env_name = local.env_name
  svc_name = "lbws"
  topic_names = ["hello"]
  alias_names = ["lbws.aws.okto.page"]
}

module "svc_bs" {
  source = "../modules/env_svc_bs"
  app_name = local.app_name
  env_name = local.env_name
  svc_name = "bs"
}

これでterraform applyするとEnvironment内にServiceが作成できる。

Worker Service

つぎは、Worker Serviceを作成する。

このServiceはECS Serviceを使っている。
また、先ほどとは異なりPub/Sub形式でやり取りする形となる。
そのため、SNSからSQSへとPublishし、SQS QueueをSubscribeできるようにする。

なので、ECS Task Definition・ECS Service・SQS Queueあたりを作成する必要がある。

modules/env_svc_ws/main.tf

variable "app_name" {
  type = string
}

variable "env_name" {
  type = string
}

variable "svc_name" {
  type = string
}

variable "cpu" {
  type = number
  default = 256
}

variable "memory" {
  type = number
  default = 512
}

variable "desired_count" {
  type = number
  default = 1
}

variable "log_retention" {
  type = number
  default = 30
}

variable "image_tag" {
  type = string
  default = "latest"
}

variable "topics" {
  type = list(object({
    name = string
    svc_name = string
  }))
  default = []
}

locals {
  execution_role_name = "${var.app_name}-${var.env_name}-${var.svc_name}-execution_role"
  task_role_name = "${var.app_name}-${var.env_name}-${var.svc_name}-task_role"
  task_def_log_group_name = "${var.app_name}-${var.env_name}-${var.svc_name}"
  task_def_name = "${var.app_name}-${var.env_name}-${var.svc_name}"
  task_def_container_name = var.svc_name
  ecr_image_name = "${var.app_name}/${var.svc_name}"
  ecs_cluster_name = "${var.app_name}-${var.env_name}"
  ecs_service_name = "${var.app_name}-${var.env_name}-${var.svc_name}"
  env_sg_name = "${var.app_name}-${var.env_name}-env"
  events_sqs_queue_name = "${var.app_name}-${var.env_name}-${var.svc_name}-events"
  dead_letter_sqs_queue_name = "${var.app_name}-${var.env_name}-${var.svc_name}-dead_letter"
  topic_names = toset([for t in var.topics : "${var.app_name}-${var.env_name}-${t.svc_name}-${t.name}"])
  tags = {
    application = var.app_name
    environment = var.env_name
    service = var.svc_name
  }
}

data "aws_caller_identity" "this" {}

data "aws_region" "this" {}

data "aws_ecr_repository" "this" {
  name = local.ecr_image_name
}

data "aws_ecr_image" "this" {
  repository_name = data.aws_ecr_repository.this.name
  image_tag = var.image_tag
}

data "aws_ecs_cluster" "this" {
  cluster_name = local.ecs_cluster_name
}

data "aws_vpc" "this" {
  tags = {
    application = var.app_name
    environment = var.env_name
  }
}

data "aws_security_group" "env" {
  vpc_id = data.aws_vpc.this.id
  tags = { Name = local.env_sg_name }
}

data "aws_subnet_ids" "public" {
  vpc_id = data.aws_vpc.this.id
  tags = { public = "true" }
}

##################################################
# ECS Task Definition
##################################################

resource "aws_iam_role" "execution_role" {
  name = local.execution_role_name
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Principal = { Service = "ecs-tasks.amazonaws.com" }
        Action = "sts:AssumeRole"
      }
    ]
  })
  managed_policy_arns = [
    "arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy"
  ]
  inline_policy {
    name = "secrets"
    policy = jsonencode({
      Version = "2012-10-17"
      Statement = [
        {
          Effect = "Allow"
          Action = "ssm:GetParameters"
          Resource = format(
            "arn:aws:ssm::%s:%s:parameter/*",
            data.aws_region.this.name,
            data.aws_caller_identity.this.account_id
          )
          Condition = {
            StringEquals: {
              "ssm:ResourceTag/application": var.app_name
              "ssm:ResourceTag/environment": var.env_name
            }
          }
        }
      ]
    })
  }
  tags = local.tags
}

resource "aws_iam_role" "task_role" {
  name = local.task_role_name
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Principal = { Service = "ecs-tasks.amazonaws.com" }
        Action = "sts:AssumeRole"
      }
    ]
  })
  inline_policy {
    name = "deny_iam_except_tagged_roles"
    policy = jsonencode({
      Version = "2012-10-17"
      Statement = [
        {
          Effect = "Deny"
          Action = "iam:*"
          Resource = "*"
        },
        {
          Effect = "Allow"
          Action = "sts:AssumeRole"
          Resource = "arn:aws:iam::${data.aws_caller_identity.this.account_id}:role/*"
          Condition = {
            StringEquals: {
              "iam:ResourceTag/application": var.app_name
              "iam:ResourceTag/environment": var.env_name
            }
          }
        }
      ]
    })
  }
  inline_policy {
    name = "execute_command"
    policy = jsonencode({
      Version = "2012-10-17"
      Statement = [
        {
          Effect = "Allow"
          Action = [
            "ssmmessages:CreateControlChannel",
            "ssmmessages:OpenControlChannel",
            "ssmmessages:CreateDataChannel",
            "ssmmessages:OpenDataChannel",
          ],
          Resource = "*"
        },
        {
          Effect = "Allow"
          Action = [
            "logs:CreateLogStream",
            "logs:DescribeLogGroups",
            "logs:DescribeLogStreams",
            "logs:PutLogEvents",
          ],
          Resource = "*"
        }
      ]
    })
  }
  tags = local.tags
}

resource "aws_cloudwatch_log_group" "this" {
  name = local.task_def_log_group_name
  retention_in_days = var.log_retention
  tags = local.tags
}

resource "aws_ecs_task_definition" "this" {
  family = local.task_def_name
  network_mode = "awsvpc"
  requires_compatibilities = ["FARGATE"]
  cpu = var.cpu
  memory = var.memory
  execution_role_arn = aws_iam_role.execution_role.arn
  task_role_arn = aws_iam_role.task_role.arn
  container_definitions = jsonencode([
    {
      name = local.task_def_container_name
      image = "${data.aws_ecr_repository.this.repository_url}:${data.aws_ecr_image.this.image_tag}"
      environment = [
        { name = "APPLICATION_NAME", value = var.app_name },
        { name = "ENVIRONMENT_NAME", value = var.env_name },
        { name = "SERVICE_NAME", value = var.svc_name },
        { name = "QUEUE_URI", value = aws_sqs_queue.events.id },
      ]
      logConfiguration = {
        logDriver = "awslogs"
        options = {
          awslogs-region: data.aws_region.this.name
          awslogs-group: aws_cloudwatch_log_group.this.name
          awslogs-stream-prefix: "service"
        }
      }
    }
  ])
  tags = local.tags
}

##################################################
# ECS Service
##################################################

resource "aws_ecs_service" "this" {
  name = local.ecs_service_name
  platform_version = "LATEST"
  cluster = data.aws_ecs_cluster.this.id
  task_definition = aws_ecs_task_definition.this.arn
  desired_count = var.desired_count
  deployment_minimum_healthy_percent = 100
  deployment_maximum_percent = 200
  propagate_tags = "SERVICE"
  enable_execute_command = true
  launch_type = "FARGATE"
  deployment_circuit_breaker {
    enable = true
    rollback = true
  }
  network_configuration {
    assign_public_ip = true
    subnets = data.aws_subnet_ids.public.ids
    security_groups = [data.aws_security_group.env.id]
  }
  tags = local.tags
}

##################################################
# SQS Queue
##################################################

resource "aws_sqs_queue" "dead_letter" {
  name = local.dead_letter_sqs_queue_name
  message_retention_seconds = 1209600
}

resource "aws_sqs_queue_policy" "dead_letter" {
  queue_url = aws_sqs_queue.dead_letter.id
  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Principal = { AWS = aws_iam_role.task_role.arn }
        Action = [
          "sqs:ReceiveMessage",
          "sqs:DeleteMessage",
        ]
        Resource = aws_sqs_queue.dead_letter.arn
      }
    ]
  })
}

resource "aws_sqs_queue" "events" {
  name = local.events_sqs_queue_name
  redrive_policy = jsonencode({
    deadLetterTargetArn = aws_sqs_queue.dead_letter.arn
    maxReceiveCount = 4
  })
}

resource "aws_sqs_queue_policy" "events" {
  queue_url = aws_sqs_queue.events.id
  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Principal = { AWS = aws_iam_role.task_role.arn }
        Action = [
          "sqs:ReceiveMessage",
          "sqs:DeleteMessage",
        ]
        Resource = aws_sqs_queue.events.arn
      },
      {
        Effect = "Allow"
        Principal = { Service = "sns.amazonaws.com" }
        Action = "sqs:SendMessage"
        Resource = aws_sqs_queue.events.arn
        Condition = {
          ArnLike: {
            "aws:SourceArn": format(
              "arn:aws:sns:%s:%s:%s-%s-*",
              data.aws_region.this.name,
              data.aws_caller_identity.this.account_id,
              var.app_name,
              var.env_name
            )
          }
        }
      }
    ]
  })
}

resource "aws_sns_topic_subscription" "this" {
  for_each = local.topic_names
  topic_arn = format(
    "arn:aws:sns:%s:%s:%s",
    data.aws_region.this.name,
    data.aws_caller_identity.this.account_id,
    each.key
  )
  protocol = "sqs"
  endpoint = aws_sqs_queue.events.arn
}

このModuleを使ってServiceを宣言すると次のような感じになる。
ここではService名を”ws”としている。

env_dev_svcs/main.tf

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 3.0"
    }
  }
}

locals {
  app_name = "tf_copilot"
  env_name = "dev"
}

provider "aws" {
  region = "ap-northeast-1"
}

module "svc_rdws" {
  source = "../modules/env_svc_rdws"
  app_name = local.app_name
  env_name = local.env_name
  svc_name = "rdws"
}

module "svc_lbws" {
  source = "../modules/env_svc_lbws"
  app_name = local.app_name
  env_name = local.env_name
  svc_name = "lbws"
  topic_names = ["hello"]
  alias_names = ["lbws.aws.okto.page"]
}

module "svc_bs" {
  source = "../modules/env_svc_bs"
  app_name = local.app_name
  env_name = local.env_name
  svc_name = "bs"
}

module "svc_ws" {
  source = "../modules/env_svc_ws"
  app_name = local.app_name
  env_name = local.env_name
  svc_name = "ws"
  topics = [{ name = "hello", svc_name = "lbws" }]
  depends_on = [module.svc_lbws]
}

これでterraform applyするとEnvironment内にServiceが作成できる。

Scheduled Job

最後に、Scheduled Jobを作成する。

このJobはECS Serviceを使っている。
また、Serviceとは異なりECS Taskを使う形となる。
EventBridgeでスケジュール指定のEventを発生させ、ECS Taskを実行できるようにする。
（実際のCopilotでは間にStepFunctionsが入っているが、ここでは簡略化してそうする）

なので、ECS Task Definition・EventBridge Ruleあたりを作成する必要がある。

modules/job_sj/main.tf

variable "app_name" {
  type = string
}

variable "env_name" {
  type = string
}

variable "job_name" {
  type = string
}

variable "cpu" {
  type = number
  default = 256
}

variable "memory" {
  type = number
  default = 512
}

variable "desired_count" {
  type = number
  default = 1
}

variable "log_retention" {
  type = number
  default = 30
}

variable "image_tag" {
  type = string
  default = "latest"
}

variable "schedule_expression" {
  type = string
}

locals {
  execution_role_name = "${var.app_name}-${var.env_name}-${var.job_name}-execution_role"
  task_role_name = "${var.app_name}-${var.env_name}-${var.job_name}-task_role"
  task_def_log_group_name = "${var.app_name}-${var.env_name}-${var.job_name}"
  task_def_name = "${var.app_name}-${var.env_name}-${var.job_name}"
  task_def_container_name = var.job_name
  ecr_image_name = "${var.app_name}/${var.job_name}"
  ecs_cluster_name = "${var.app_name}-${var.env_name}"
  env_sg_name = "${var.app_name}-${var.env_name}-env"
  event_rule_name = "${var.app_name}-${var.env_name}-${var.job_name}"
  event_role_name = "${var.app_name}-${var.env_name}-${var.job_name}-event_role"
  tags = {
    application = var.app_name
    environment = var.env_name
    job = var.job_name
  }
}

data "aws_caller_identity" "this" {}

data "aws_region" "this" {}

data "aws_ecr_repository" "this" {
  name = local.ecr_image_name
}

data "aws_ecr_image" "this" {
  repository_name = data.aws_ecr_repository.this.name
  image_tag = var.image_tag
}

data "aws_ecs_cluster" "this" {
  cluster_name = local.ecs_cluster_name
}

data "aws_vpc" "this" {
  tags = {
    application = var.app_name
    environment = var.env_name
  }
}

data "aws_security_group" "env" {
  vpc_id = data.aws_vpc.this.id
  tags = { Name = local.env_sg_name }
}

data "aws_subnet_ids" "public" {
  vpc_id = data.aws_vpc.this.id
  tags = { public = "true" }
}

##################################################
# ECS Task Definition
##################################################

resource "aws_iam_role" "execution_role" {
  name = local.execution_role_name
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Principal = { Service = "ecs-tasks.amazonaws.com" }
        Action = "sts:AssumeRole"
      }
    ]
  })
  managed_policy_arns = [
    "arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy"
  ]
  inline_policy {
    name = "secrets"
    policy = jsonencode({
      Version = "2012-10-17"
      Statement = [
        {
          Effect = "Allow"
          Action = "ssm:GetParameters"
          Resource = format(
            "arn:aws:ssm::%s:%s:parameter/*",
            data.aws_region.this.name,
            data.aws_caller_identity.this.account_id
          )
          Condition = {
            StringEquals: {
              "ssm:ResourceTag/application": var.app_name
              "ssm:ResourceTag/environment": var.env_name
            }
          }
        }
      ]
    })
  }
  tags = local.tags
}

resource "aws_iam_role" "task_role" {
  name = local.task_role_name
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Principal = { Service = "ecs-tasks.amazonaws.com" }
        Action = "sts:AssumeRole"
      }
    ]
  })
  inline_policy {
    name = "deny_iam_except_tagged_roles"
    policy = jsonencode({
      Version = "2012-10-17"
      Statement = [
        {
          Effect = "Deny"
          Action = "iam:*"
          Resource = "*"
        },
        {
          Effect = "Allow"
          Action = "sts:AssumeRole"
          Resource = "arn:aws:iam::${data.aws_caller_identity.this.account_id}:role/*"
          Condition = {
            StringEquals: {
              "iam:ResourceTag/application": var.app_name
              "iam:ResourceTag/environment": var.env_name
            }
          }
        }
      ]
    })
  }
  inline_policy {
    name = "execute_command"
    policy = jsonencode({
      Version = "2012-10-17"
      Statement = [
        {
          Effect = "Allow"
          Action = [
            "ssmmessages:CreateControlChannel",
            "ssmmessages:OpenControlChannel",
            "ssmmessages:CreateDataChannel",
            "ssmmessages:OpenDataChannel",
          ],
          Resource = "*"
        },
        {
          Effect = "Allow"
          Action = [
            "logs:CreateLogStream",
            "logs:DescribeLogGroups",
            "logs:DescribeLogStreams",
            "logs:PutLogEvents",
          ],
          Resource = "*"
        }
      ]
    })
  }
  tags = local.tags
}

resource "aws_cloudwatch_log_group" "this" {
  name = local.task_def_log_group_name
  retention_in_days = var.log_retention
  tags = local.tags
}

resource "aws_ecs_task_definition" "this" {
  family = local.task_def_name
  network_mode = "awsvpc"
  requires_compatibilities = ["FARGATE"]
  cpu = var.cpu
  memory = var.memory
  execution_role_arn = aws_iam_role.execution_role.arn
  task_role_arn = aws_iam_role.task_role.arn
  container_definitions = jsonencode([
    {
      name = local.task_def_container_name
      image = "${data.aws_ecr_repository.this.repository_url}:${data.aws_ecr_image.this.image_tag}"
      environment = [
        { name = "APPLICATION_NAME", value = var.app_name },
        { name = "ENVIRONMENT_NAME", value = var.env_name },
        { name = "JOB_NAME", value = var.job_name },
      ]
      logConfiguration = {
        logDriver = "awslogs"
        options = {
          awslogs-region: data.aws_region.this.name
          awslogs-group: aws_cloudwatch_log_group.this.name
          awslogs-stream-prefix: "job"
        }
      }
    }
  ])
  tags = local.tags
}

##################################################
# EventBridge Rule
##################################################

resource "aws_iam_role" "ecs_scheduled_task" {
  name = local.event_role_name
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Principal = { Service = "events.amazonaws.com" }
        Action = "sts:AssumeRole"
      }
    ]
  })
  inline_policy {
    name = "pass_role"
    policy = jsonencode({
      Version = "2012-10-17"
      Statement = [
        {
          Effect = "Allow"
          Action = "iam:PassRole"
          Resource = [
            aws_iam_role.execution_role.arn,
            aws_iam_role.task_role.arn,
          ]
        }
      ]
    })
  }
  inline_policy {
    name = "run_task"
    policy = jsonencode({
      Version = "2012-10-17"
      Statement = [
        {
          Effect = "Allow"
          Action = "ecs:RunTask"
          Resource = aws_ecs_task_definition.this.arn
          Condition = {
            ArnEquals = {
              "ecs:cluster": data.aws_ecs_cluster.this.arn
            }
          }
        }
      ]
    })
  }
}

resource "aws_cloudwatch_event_rule" "ecs_scheduled_task" {
  name = local.event_rule_name
  schedule_expression = var.schedule_expression
  tags = local.tags
}

resource "aws_cloudwatch_event_target" "ecs_scheduled_task" {
  target_id = "ecs_scheduled_task"
  rule = aws_cloudwatch_event_rule.ecs_scheduled_task.name
  role_arn = aws_iam_role.ecs_scheduled_task.arn
  arn = data.aws_ecs_cluster.this.arn
  ecs_target {
    task_count = 1
    task_definition_arn = aws_ecs_task_definition.this.arn
    platform_version = "LATEST"
    launch_type = "FARGATE"
    propagate_tags = "TASK_DEFINITION"
    network_configuration {
      assign_public_ip = true
      subnets = data.aws_subnet_ids.public.ids
      security_groups = [data.aws_security_group.env.id]
    }
    tags = local.tags
  }
}

このModuleを使ってJobを宣言すると次のような感じになる。
ここではJop名を”sj”としている。

env_dev_svcs/main.tf

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 3.0"
    }
  }
}

locals {
  app_name = "tf_copilot"
  env_name = "dev"
}

provider "aws" {
  region = "ap-northeast-1"
}

module "svc_rdws" {
  source = "../modules/env_svc_rdws"
  app_name = local.app_name
  env_name = local.env_name
  svc_name = "rdws"
}

module "svc_lbws" {
  source = "../modules/env_svc_lbws"
  app_name = local.app_name
  env_name = local.env_name
  svc_name = "lbws"
  topic_names = ["hello"]
  alias_names = ["lbws.aws.okto.page"]
}

module "svc_bs" {
  source = "../modules/env_svc_bs"
  app_name = local.app_name
  env_name = local.env_name
  svc_name = "bs"
}

module "svc_ws" {
  source = "../modules/env_svc_ws"
  app_name = local.app_name
  env_name = local.env_name
  svc_name = "ws"
  topics = [{ name = "hello", svc_name = "lbws" }]
  depends_on = [module.svc_lbws]
}

module "job_sj" {
  source = "../modules/env_job_sj"
  app_name = local.app_name
  env_name = local.env_name
  job_name = "sj"
  schedule_expression = "rate(2 minutes)"
}

これでterraform applyするとEnvironment内にJobが作成できる。

完成

これで、9割ぐらいはAWS Copilotを再現できているんじゃないかと思う。
Pipelineに関してはデプロイ部分の話なので、今回は除外した。

全体のコードは↓に、いちおう置いてある。

まとめ

AWS Copilotによって作成される構成が、具体的にどうなっているのかより深く理解することができた。そして、AWSで構成を考える時に考慮すべきポイント的なのも、何となく間隔がつかめた感じがする。

ついでにTerraformも色々と使えたので良かった。

AWS Coplot自体はそれなりに制約を高めたものであり、必ずしも採用できるとは限らない。
だが、AWS Copilotで採用されているシステム構成は、誰でも真似ることができるので、それらを参考に今後のシステムを組んでいくと良さそうな感じがする。

AWS Copilot

概要

Application

Environment

Request-Driven Web Service

Load Balanced Web Service

Backend Service

Worker Service

Scheduled Job

完成

まとめ

Discussion