🦔
委任したAccountでAWS Configを観測 by Terraform
概要
またAWS Landing Zone環境構築の続き
今回はTerraformを用いてマルチアカウント環境でAWS Configを設定する
ポイントとしては以下
- Log Archive Accountに組織Aggregatorを委任
- 全アカウントのConfig計測データをLog Archive AccountのS3に集約
- 各アカウントのConfig設定を一つのterraformで実施
- 組織内全アカウントにConfig Ruleを適用
イメージとしては以下のような感じ
ディレクトリ構成
今回こんな感じで作成した
（やはりTerraform歴浅いのでまだまだ実力不足感を感じる
$ tree
.
├── backend.tf
├── main.tf
├── modules
│ └── config
│ ├── aggregator
│ │ ├── main.tf
│ │ ├── outputs.tf
│ │ └── variables.tf
│ ├── each-config
│ │ ├── main.tf
│ │ ├── outputs.tf
│ │ └── variables.tf
│ └── rules
│ ├── main.tf
│ ├── outputs.tf
│ └── variables.tf
├── provider.tf
└── versions.tf
5 directories, 13 files
やったこと
それぞれやったことのポイントをメモしておく
各アカウント情報の設定
各AWSアカウントにConfig設定するために
providerのassume_roleにrole_arnを指定する
main.tfでこのproviderのaliasを指定することで
対象のAWSアカウントに対してリソースを作成したりできる🤓
roleは前回のSSO記事で生成されたAdministratorAccessの付与されたrole使ってる
./provider.tf
provider "aws" {
region = "ap-northeast-1"
}
provider "aws" {
alias = "log-archive-production"
region = "ap-northeast-1"
assume_role {
role_arn = "arn:aws:iam::${aws_organizations_account.log_archive_production_account.id}:role/OrganizationAccountAccessRole"
}
}
provider "aws" {
alias = "ucwork-production-account"
region = "ap-northeast-1"
assume_role {
role_arn = "arn:aws:iam::${aws_organizations_account.ucwork_production_account.id}:role/OrganizationAccountAccessRole"
}
}
# ...アカウントの分だけproviderを用意する
各リソース作成指示
各種moduleやresourceの作成
Organizational Unit(OU), Account, SSOは
コメントに記載した別記事参照くだされ
./main.tf
# -----------------------------------------------------------
# Set up organizations
# -----------------------------------------------------------
// この辺参照ください
https://zenn.dev/ucwork/articles/abebfe4003a7ee
// OUとAccountを管理してる場所
# -----------------------------------------------------------
# Set up aggregator and each config in all account
# -----------------------------------------------------------
// terraformのresourceないみたいなのでawsコマンドで実行
// 登録済みの場合エラーになるのでon_failure = continueをセット
resource "null_resource" "config_delegated" {
provisioner "local-exec" {
command = "aws organizations register-delegated-administrator --account-id ${aws_organizations_account.log_archive_production_account.id} --service-principal config.amazonaws.com"
on_failure = continue
}
}
resource "null_resource" "config_multi_setup_delegated" {
provisioner "local-exec" {
command = "aws organizations register-delegated-administrator --account-id ${aws_organizations_account.log_archive_production_account.id} --service-principal config-multiaccountsetup.amazonaws.com"
on_failure = continue
}
depends_on = [ null_resource.config_delegated ]
}
module "config_aggregator" {
source = "./modules/config/aggregator"
aggregator_account_id = aws_organizations_account.log_archive_production_account.id
aggregator_s3_region = "ap-northeast-1"
providers = {
aws = aws.log-archive-production
}
}
// config_aggregatorで作成したS3使いたいので依存関係（depends_on）指定
module "config_management" {
source = "./modules/config/each-config"
bucket_arn = module.config_aggregator.config_s3_arn
bucket_id = module.config_aggregator.config_s3_id
depends_on = [
module.config_aggregator
]
}
// provider.tfで指定したaliasで各アカウントにresource作成
module "config_ucwork_production" {
source = "./modules/config/each-config"
bucket_arn = module.config_aggregator.config_s3_arn
bucket_id = module.config_aggregator.config_s3_id
providers = {
aws = aws.ucwork-production-account
}
depends_on = [
module.config_aggregator
]
}
# Config設定するアカウントの分だけ上記each-configのmoduleを追記する！
// moduleのfor_each使いたかったけど、providersのaliasにeach.value指定するとエラーで怒られたので愚直に全アカウント分書いた・・・
# -----------------------------------------------------------
# Set up organization config rules
# -----------------------------------------------------------
// organizationに紐づくアカウント全部にルール作成
// 各アカウントのconfig resourceないと怒られるのでdepends_onを指定
module "config_rules" {
source = "./modules/config/rules"
depends_on = [
module.config_management,
module.config_shared_services_production,
module.config_ucwork_production,
module.config_ucwork_sdlc
]
}
# -----------------------------------------------------------
# Set up Single Sign-On (SSO)
# -----------------------------------------------------------
// この辺見てね
https://zenn.dev/ucwork/articles/7f30488b6d2bbf
// SSOでマルチアカウントアクセスするための設定まわりが書いてある
Aggregatorアカウントの設定
./modules/config/aggregator/main.tf
# logging s3
resource "aws_s3_bucket" "config_bucket" {
bucket = "config-bucket-${var.aggregator_account_id}-${var.aggregator_s3_region}"
acl = "private"
server_side_encryption_configuration {
rule {
apply_server_side_encryption_by_default {
sse_algorithm = "AES256"
}
}
}
}
resource "aws_s3_bucket_policy" "config_logging_policy" {
bucket = aws_s3_bucket.config_bucket.id
policy = <<POLICY
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AWSConfigBucketPermissionsCheck",
"Effect": "Allow",
"Principal": {
"Service": [
"config.amazonaws.com"
]
},
"Action": "s3:GetBucketAcl",
"Resource": "${aws_s3_bucket.config_bucket.arn}"
},
{
"Sid": "AWSConfigBucketExistenceCheck",
"Effect": "Allow",
"Principal": {
"Service": [
"config.amazonaws.com"
]
},
"Action": "s3:ListBucket",
"Resource": "${aws_s3_bucket.config_bucket.arn}"
},
{
"Sid": "AWSConfigBucketDelivery",
"Effect": "Allow",
"Principal": {
"Service": [
"config.amazonaws.com"
]
},
"Action": "s3:PutObject",
"Resource": "${aws_s3_bucket.config_bucket.arn}/AWSLogs/*/Config/*",
"Condition": {
"StringEquals": {
"s3:x-amz-acl": "bucket-owner-full-control"
}
}
}
]
}
POLICY
}
# iam role
resource "aws_iam_role" "config_role" {
name = "config_aggregator_role"
assume_role_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"Service": "config.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
EOF
}
resource "aws_iam_policy" "config_org_policy" {
path = "/"
description = "IAM Policy for AWS Config"
name = "ConfigPolicy"
policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"config:GetOrganizationConfigRuleDetailedStatus",
"config:Put*",
"iam:GetPasswordPolicy",
"organizations:ListAccounts",
"organizations:DescribeOrganization",
"organizations:ListAWSServiceAccessForOrganization",
"organization:EnableAWSServiceAccess"
],
"Effect": "Allow",
"Resource": "*"
},
{
"Effect": "Allow",
"Action": ["s3:PutObject"],
"Resource": ["${aws_s3_bucket.config_bucket.arn}/AWSLogs/${var.aggregator_account_id}/*"],
"Condition": {
"StringLike": {
"s3:x-amz-acl": "bucket-owner-full-control"
}
}
},
{
"Effect": "Allow",
"Action": ["s3:GetBucketAcl"],
"Resource": "${aws_s3_bucket.config_bucket.arn}"
}
]
}
EOF
}
resource "aws_iam_role_policy_attachment" "config_org_policy_attach" {
role = aws_iam_role.config_role.name
policy_arn = aws_iam_policy.config_org_policy.arn
}
resource "aws_iam_role_policy_attachment" "config_policy_attach" {
role = aws_iam_role.config_role.name
policy_arn = "arn:aws:iam::aws:policy/service-role/AWSConfigRole"
}
resource "aws_iam_role_policy_attachment" "read_only_policy_attach" {
role = aws_iam_role.config_role.name
policy_arn = "arn:aws:iam::aws:policy/ReadOnlyAccess"
}
# -----------------------------------------------------------
# set up the aggregator account Config
# -----------------------------------------------------------
resource "aws_config_configuration_aggregator" "organization" {
name = "organization-aggregator"
organization_aggregation_source {
all_regions = true
role_arn = aws_iam_role.config_role.arn
}
}
resource "aws_config_configuration_recorder" "config_recorder" {
role_arn = aws_iam_role.config_role.arn
recording_group {
all_supported = true
include_global_resource_types = true
}
}
# Delivery channel resource and bucket location to specify configuration history location.
resource "aws_config_delivery_channel" "config_channel" {
s3_bucket_name = aws_s3_bucket.config_bucket.id
depends_on = [aws_config_configuration_recorder.config_recorder]
}
resource "aws_config_configuration_recorder_status" "config_recorder_status" {
name = aws_config_configuration_recorder.config_recorder.name
is_enabled = true
depends_on = [aws_config_delivery_channel.config_channel]
}
./modules/config/aggregator/variables.tf
variable "aggregator_account_id" {
type = string
default = ""
description = "config aggregator account id"
}
variable "aggregator_s3_region" {
type = string
default = ""
description = "config aggregator s3 region"
}
./modules/config/aggregator/outputs.tf
output "config_s3_arn" {
value = aws_s3_bucket.config_bucket.arn
description = "AWS Config Aggregator s3 bucket arn"
}
output "config_s3_id" {
value = aws_s3_bucket.config_bucket.id
description = "AWS Config Aggregator s3 bucket name(id)"
}
各アカウントのConfig設定
./modules/config/each-config/main.tf
resource "aws_iam_role" "config_role" {
name = "ConfigRecorderRole"
assume_role_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"Service": "config.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
EOF
}
resource "aws_iam_policy" "s3_config_org_policy" {
path = "/"
description = "S3ConfigOrganizationPolicy"
policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": ["s3:PutObject"],
"Resource": ["${var.bucket_arn}/AWSLogs/*/Config/*"],
"Condition": {
"StringLike": {
"s3:x-amz-acl": "bucket-owner-full-control"
}
}
},
{
"Effect": "Allow",
"Action": ["s3:GetBucketAcl"],
"Resource": "${var.bucket_arn}"
}
]
}
EOF
}
resource "aws_iam_role_policy_attachment" "config_s3_policy_attach" {
role = aws_iam_role.config_role.name
policy_arn = aws_iam_policy.s3_config_org_policy.arn
}
resource "aws_iam_role_policy_attachment" "read_only_attachment" {
role = aws_iam_role.config_role.name
policy_arn = "arn:aws:iam::aws:policy/ReadOnlyAccess"
}
resource "aws_iam_role_policy_attachment" "config_attachment" {
role = aws_iam_role.config_role.name
policy_arn = "arn:aws:iam::aws:policy/service-role/AWSConfigRole"
}
# -----------------------------------------------------------
# set up the Config
# -----------------------------------------------------------
resource "aws_config_configuration_recorder" "config_recorder" {
role_arn = aws_iam_role.config_role.arn
}
resource "aws_config_delivery_channel" "config_channel" {
s3_bucket_name = var.bucket_id
depends_on = [aws_config_configuration_recorder.config_recorder]
}
resource "aws_config_configuration_recorder_status" "config_recorder_status" {
name = aws_config_configuration_recorder.config_recorder.name
is_enabled = true
depends_on = [aws_config_delivery_channel.config_channel]
}
./modules/config/each-config/variables.tf
variable "bucket_arn" {
type = string
default = ""
description = "organization config s3 bucket arn"
}
variable "bucket_id" {
type = string
default = ""
description = "organization config s3 bucket name(id)"
}
config ruleの設定
./modules/config/rules/main.tf
# AWS Config Rule that manages IAM Password Policy
resource "aws_config_organization_managed_rule" "iam_policy_organization_config_rule" {
input_parameters = <<EOF
{
"RequireUppercaseCharacters": "true",
"RequireLowercaseCharacters": "true",
"RequireSymbols": "true",
"RequireNumbers": "true",
"MinimumPasswordLength": "9",
"PasswordReusePrevention": "5",
"MaxPasswordAge": "90"
}
EOF
name = "iam-password-policy"
rule_identifier = "IAM_PASSWORD_POLICY"
}
# AWS Config Rule that manages IAM Root Access Keys to see if they exist
resource "aws_config_organization_managed_rule" "iam_root_access_key_organization_config_rule" {
name = "iam-root-access-key-check"
rule_identifier = "IAM_ROOT_ACCESS_KEY_CHECK"
}
# AWS Config Rule that checks whether your AWS account is enabled to use multi-factor authentication (MFA)
# hardware device to sign in with root credentials.
resource "aws_config_organization_managed_rule" "root_hardware_mfa_organization_config_rule" {
name = "root-hardware-mfa"
rule_identifier = "ROOT_ACCOUNT_HARDWARE_MFA_ENABLED"
}
# AWS Config Rule that checks whether users of your AWS account require a multi-factor authentication (MFA)
# device to sign in with root credentials.
resource "aws_config_organization_managed_rule" "root_account_mfa_organization_config_rules" {
name = "root-account-mfa-enabled"
rule_identifier = "ROOT_ACCOUNT_MFA_ENABLED"
}
# AWS Config Rule that checks whether the required public access block settings are configured from account level.
# The rule is only NON_COMPLIANT when the fields set below do not match the corresponding fields in the configuration
# item.
resource "aws_config_organization_managed_rule" "s3_public_access_organization_config_rules" {
name = "s3-account-level-public-access-blocks"
rule_identifier = "S3_ACCOUNT_LEVEL_PUBLIC_ACCESS_BLOCKS"
}
# AWS Config Rule that checks whether logging is enabled for your S3 buckets.
resource "aws_config_organization_managed_rule" "s3_bucket_logging_organization_config_rules" {
name = "s3-bucket-logging-enabled"
rule_identifier = "S3_BUCKET_LOGGING_ENABLED"
}
# AWS Config Rule that checks whether logging is enabled for your S3 buckets.
resource "aws_config_organization_managed_rule" "s3_bucket_encryption_organization_config_rules" {
name = "s3-bucket-server-side-encryption-enabled"
rule_identifier = "S3_BUCKET_SERVER_SIDE_ENCRYPTION_ENABLED"
}
terraform適用
あとはいつもの通り以下でリソース作成
terraform init
terraform plan
terraform apply
動作確認
集約したアカウントにログイン
Configのアグリゲータにアクセスしたら
全てのアカウントの非準拠ルールとか出てる！きっとうまく行ってそう！
詰まったところ
別アカウントからのS3アクセスがうまくいかない
terraform applyするとconfig channel resourceあたりでこんなエラー出てだいぶ詰まった
nsufficient delivery policy to s3 bucket:<Bucket Name>, unable to write to bucket, provided s3 key prefix is 'null'.
ここの説明にある通り、集約するS3 bucketのpolicyとIAM RoleのPolicyをちゃんと合わせないとここから抜け出せない
moduleでfor_each, countうまくいかない
この方の記事にもあるとおり
カッコつけてmoduleで分離して色々variablesで渡してたら
こんなエラーメッセージが出て、なかなか抜け出せなかった・・・
The "for_each" value depends on resource attributes that cannot be determined
until apply, so Terraform cannot predict how many instances will be created.
To work around this, use the -target argument to first apply only the
resources that the for_each depends on.
ちょっと戦ったけど、module使わない方で逃げた🙈
まとめ
だいぶ時間かかった・・・
Configで遊び始めたらついに個人アカウントにも50円ほど課金が発生し始めた・・・
最近転職してお給金も減ったので
一旦このConfigのrecordを止めておこう。これで請求一旦止まるかな・・？
ここ変更してterraform applyすると記録止まりまっす
resource "aws_config_configuration_recorder_status" "config_recorder_status" {
name = aws_config_configuration_recorder.config_recorder.name
- is_enabled = true
+ is_enabled = false
depends_on = [aws_config_delivery_channel.config_channel]
}
Discussion