😸

【CloudFormation】プライベートサブネットにCloud9(Session Manager経由)をデプロイしてみる

2022/07/11に公開

AWS

1. はじめに

こんにちわ、Mitsuoです。
皆さん選挙には行かれましたか。
私は家族と行ったのですが、息子がおばあさま方に大人気でして、その息子の父親であることが嬉しかったです。
~~正直、中々帰れなくて辛かったです~~ 。

2. 今回のお題

さて、前回のお題と同様になりますが、「CloudFormationを用いてCloud9をデプロイ」してみました。
前回のトピックは、パブリック上にCloud9を配置するテンプレートのご紹介でしたが、今回はプライベートサブネットに配置するバージョンとなります。
接続方式はSystems Manager(Session Manager)です。
SSH用を希望される方がいない気はしていますが、プライベートサブネット上のCloud9はSSH接続が出来ない仕様のようです。
（どこかに書いていた気がするのですが、Cloud9にデフォルトでアタッチされるセキュリティグループのインバウンドは、Cloud9接続用のグローバルIPアドレスのCIDRレンジ、およびSSHポートを許可する設定になっています。
そのような仕様であるため、直接SSHアクセスする事が出来ず、Cloud9のIDE画面を表示出来ないと言うわけです。

公式ドキュメントにも記載がありました。

If your environment is accessing its EC2 instance directly though SSH, the instance can be launched into a public subnet only.

一方で、Session Managerの場合はどちらのサブネットでも利用する事が出来ます。

If you're accessing a no-ingress Amazon EC2 instance using Systems Manager, the instance can be launched into either a public or a private subnet.

Cloud9に限った話では無いですが、インターネット通信が必要な時は、Cloud9インスタンスにNAT Gatewayと接続出来る設定を行う必要があります。

If you're using a private subnet, allow the instance for the subnet to communicate with the internet by hosting a NAT gateway in a public subnet.

詳細は、公式ドキュメントを参照ください。

Amazon VPC requirements for AWS Cloud9

SSH host requirements

3. 作成したリソース

テンプレートで作成するリソースは以下の通りです。

VPC
Public Subnet
Private Subnet
InternetGateway
Elastic IP
NAT Gateway
NetworkACL
Route Table
Cloud9
IAMロール(Cloud9)
Security Group(Systems Manager、CodeCommmitおよびECRのVPCEndPoint用)
VPCEndpoint(Interface型：Systems Manager、CodeCommitおよびECR用)
VPCEndpoint(Gateway型：S3用)

4 テンプレート情報

作成したテンプレートになります。


AWSTemplateFormatVersion: 2010-09-09
Description: Create Cloud9 on Private subnet using SSM. Additionally this template includes related resources such as Network, route table and NACL.
# ------------------------------------------------------------#
#  Caution:
#  This template is for deploying resources in Tokyo region(ap-northeast-1).
#  If you deploy them in another region, make sure that you understand below content and make modification for parameters such as AvailabilityZone.
#  This server is for Amazon Linux 2. If you want to change other AMI or SSM Paths, you have to modify Cloud9 resources, which is ImageId of AWS::Cloud9::EnvironmentEC2.
#  For more information, see https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-cloud9-environmentec2.html#cfn-cloud9-environmentec2-imageid
#  The Cloud9 on private subnet has to disable AMTC because Cloud9-enabled AMTC cannot execute API command except public IP address to which Cloud9 is assigned, 
#  which is specification Cloud9 has. (As of July 4 2022)
# ------------------------------------------------------------#
#  Metadata:
#  This can privide details about the template.
#  For more information, see https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/metadata-section-structure.html.
#  As Metadata Keys, AWS::CloudFormation::Interface can Defines the grouping and ordering of input parameters
#  when they are displayed in the AWS CloudFormation console.
#  For more information, see https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-cloudformation-interface.html.
# ------------------------------------------------------------# 

Metadata: 
  "AWS::CloudFormation::Interface": 
    ParameterGroups:
      -  
        Parameters:
          - SystemName
          - EnvType 
          - Tagprefix
          - Owner
          - InstanceType
          - VPCCIDR
          - PublicSubnet1CIDR          
          - PrivateSubnet1CIDR
          - AvailabilityZone
          - RepositoryName
    ParameterLabels: 
      SystemName:
        default: "Type the Systemname"
      EnvType:
        default: "Select the environment in which you deploy resources"
      Tagprefix:
        default: "Type the Tagprefix which is logical unit name about this stack(e.g. Webserver,Monitoring,Logging)"
      Owner:
        default: "Type who owns these resources (e.g. project name or worker, whether this purpose is just verification or not)" 
      InstanceType:
        default: "Type InstanceType you want to create for Cloud9"     
      VPCCIDR: 
        default: "Type VPC's CIDR such as NN.NN.NN.NN/NN based on RFC 1918"
      PublicSubnet1CIDR:
        default: "Type your public subnet's CIDR in availability zone such as NN.NN.NN.NN/NN based on RFC 1918"
      PrivateSubnet1CIDR: 
        default: "Type your private subnet's CIDR in availability zone such as NN.NN.NN.NN/NN based on RFC 1918"
      AvailabilityZone:
        default: "Type your availability zone you want to use"
      RepositoryName:
        default: "Type Repository name in ECR"

# ------------------------------------------------------------#
# Parameters
# This Can enable templates to input custom values each time you create or update a stack.
# For more information, see https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/parameters-section-structure.html.
# ------------------------------------------------------------# 

Parameters:
  SystemName:
    Type: String
  EnvType:
    Type: String
    Default: "dev"
    AllowedValues:
      - dev
      - stg
      - prd 
  Tagprefix:
    Type: String      
  Owner:
    Type: String
  InstanceType:
    Type: String 
    Default: "t3.small"
    ConstraintDescription: "Type InstanceType you want to create for Cloud9"      
  VPCCIDR:
    Type: String
    Default: "10.10.0.0/16"
    ConstraintDescription: "Type VPC's CIDR such as NN.NN.NN.NN/NN based on RFC 1918"  
  PublicSubnet1CIDR:
    Type: String
    Default: "10.10.1.0/24"
    ConstraintDescription: "Type your public subnet's CIDR in availability zone such as NN.NN.NN.NN/NN based on RFC 1918"      
  PrivateSubnet1CIDR:
    Type: String
    Default: "10.10.100.0/24"
    ConstraintDescription: "Type your private subnet's CIDR in availability zone such as NN.NN.NN.NN/NN based on RFC 1918"       
  AvailabilityZone:
    Type: String
    Default: "ap-northeast-1a"
    AllowedValues:
      - ap-northeast-1a
      - ap-northeast-1c
      - ap-northeast-1d 
  RepositoryName:
    Type: String
    ConstraintDescription: "Type Repository name in ECR"   

# ------------------------------------------------------------#
# Resources
# This can declare the AWS resources that you want to include in the stack.
# For more information, see https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/resources-section-structure.html.
# ------------------------------------------------------------# 

Resources:
# ------------------------------------------------------------#
# Create VPC
# ------------------------------------------------------------#

  VPC:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: !Ref VPCCIDR
      EnableDnsHostnames: "true"
      EnableDnsSupport: "true"
      Tags:
        - Key: Name
          Value: !Sub ${SystemName}-${EnvType}-${Tagprefix}-Cloud9-VPC
        - Key: Owner
          Value: !Ref Owner

# ------------------------------------------------------------#
# Create public subnet
# ------------------------------------------------------------#

  PublicSubnet1:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref VPC
      CidrBlock: !Ref PublicSubnet1CIDR
      AvailabilityZone: !Ref AvailabilityZone
      Tags:
        - Key: Name
          Value: !Sub ${SystemName}-${EnvType}-${Tagprefix}-Cloud9-PublicSubnet
        - Key: Owner
          Value: !Ref Owner

# ------------------------------------------------------------#
# Create PrivateSubnet
# ------------------------------------------------------------#

  PrivateSubnet1:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref VPC
      CidrBlock: !Ref PrivateSubnet1CIDR
      AvailabilityZone: !Ref AvailabilityZone
      Tags:
        - Key: Name
          Value: !Sub ${SystemName}-${EnvType}-${Tagprefix}-Cloud9-PrivateSubnet
        - Key: Owner
          Value: !Ref Owner

# ------------------------------------------------------------#
# Create InternetGateway and associate it with VPC
# ------------------------------------------------------------#

  InternetGateway:
    Type: AWS::EC2::InternetGateway
    Properties:
      Tags:
        - Key: Name
          Value: !Sub ${SystemName}-${EnvType}-${Tagprefix}-Cloud9-Internet-Gateway
        - Key: Owner
          Value: !Ref Owner

  AttachGateway:
    Type: AWS::EC2::VPCGatewayAttachment
    Properties:
      VpcId: !Ref VPC
      InternetGatewayId: !Ref InternetGateway

# ------------------------------------------------------------#
# Create Nat Gateway
# ------------------------------------------------------------#

  NatGatewayEIP:
    Type: AWS::EC2::EIP
    Properties:
      Domain: vpc
      Tags:
        - Key: Name
          Value: !Sub ${SystemName}-${EnvType}-${Tagprefix}-Cloud9-EIP
        - Key: Owner
          Value: !Ref Owner

  NatGateway:
    Type: AWS::EC2::NatGateway
    Properties:
      AllocationId:
        Fn::GetAtt:
          - NatGatewayEIP
          - AllocationId
      SubnetId: !Ref PublicSubnet1
      ConnectivityType: public
      Tags:
        - Key: Name
          Value: !Sub ${SystemName}-${EnvType}-${Tagprefix}-Cloud9-NatGateway
        - Key: Owner
          Value: !Ref Owner

# ------------------------------------------------------------#
# Create NetworkACL and associate it with public subnet  
# ------------------------------------------------------------#
  NetworkACLForPublicSubnet1:
    Type: AWS::EC2::NetworkAcl
    Properties:
      VpcId: !Ref VPC
      Tags: 
        - Key: Name
          Value: !Sub ${SystemName}-${EnvType}-${Tagprefix}-Cloud9-PublicSubnet-Networkacl
        - Key: Owner
          Value: !Ref Owner          

  NetworkACLAssocForPublicSubnet1:
    Type: AWS::EC2::SubnetNetworkAclAssociation
    Properties: 
      NetworkAclId: !Ref NetworkACLForPublicSubnet1
      SubnetId: !Ref PublicSubnet1

# ------------------------------------------------------------#
# Add In/Outbound Rule to NetworkACL with public subnet  
# ------------------------------------------------------------#

  NaclInboundRuleForPublicSubnet1:
    Type: AWS::EC2::NetworkAclEntry
    Properties:
       NetworkAclId: !Ref NetworkACLForPublicSubnet1
       RuleNumber: 100
       Protocol: -1
       Egress: false
       RuleAction: allow
       CidrBlock: 0.0.0.0/0

  NaclOutboundRuleForPublicSubnet1:
    Type: AWS::EC2::NetworkAclEntry
    Properties:
       NetworkAclId: !Ref NetworkACLForPublicSubnet1
       RuleNumber: 100
       Protocol: -1
       Egress: true
       RuleAction: allow
       CidrBlock: 0.0.0.0/0

# ------------------------------------------------------------#
# Create NetworkACL and associate it with private subnet  
# ------------------------------------------------------------#
  NetworkACLForPrivateSubnet1:
    Type: AWS::EC2::NetworkAcl
    Properties:
      VpcId: !Ref VPC
      Tags: 
        - Key: Name
          Value: !Sub ${SystemName}-${EnvType}-${Tagprefix}-Cloud9-PrivateSubnet-Networkacl
        - Key: Owner
          Value: !Ref Owner          

  NetworkACLAssocForPrivateSubnet1:
    Type: AWS::EC2::SubnetNetworkAclAssociation
    Properties: 
      NetworkAclId: !Ref NetworkACLForPrivateSubnet1
      SubnetId: !Ref PrivateSubnet1

# ------------------------------------------------------------#
# Add In/Outbound Rule to NetworkACL with private subnet  
# ------------------------------------------------------------#

  NaclInboundRuleForPrivateSubnet1:
    Type: AWS::EC2::NetworkAclEntry
    Properties:
       NetworkAclId: !Ref NetworkACLForPrivateSubnet1
       RuleNumber: 100
       Protocol: -1
       Egress: false
       RuleAction: allow
       CidrBlock: 0.0.0.0/0

  NaclOutboundRuleForPrivateSubnet1:
    Type: AWS::EC2::NetworkAclEntry
    Properties:
       NetworkAclId: !Ref NetworkACLForPrivateSubnet1
       RuleNumber: 100
       Protocol: -1
       Egress: true
       RuleAction: allow
       CidrBlock: 0.0.0.0/0

# ------------------------------------------------------------#
# Create Route Table and associate with public subnet
# ------------------------------------------------------------#

  PublicRouteTable1:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref VPC
      Tags:
      - Key: Name
        Value: !Sub ${SystemName}-${EnvType}-${Tagprefix}-Cloud9-RouteTable-PublicSubnet
      - Key: Owner
        Value: !Ref Owner

  PublicSubnetRouteTableAssociation1:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref PublicSubnet1
      RouteTableId: !Ref PublicRouteTable1
      
# ------------------------------------------------------------#
# Add route in a route table with public subnet
# ------------------------------------------------------------#

  PublicRoute:
    Type: AWS::EC2::Route
    Properties:
       RouteTableId: !Ref PublicRouteTable1
       DestinationCidrBlock: 0.0.0.0/0
       GatewayId: !Ref InternetGateway

# ------------------------------------------------------------#
# Create Route Table and associate with Private Subnet
# ------------------------------------------------------------#

  PrivateRouteTable1:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref VPC
      Tags:
      - Key: Name
        Value: !Sub ${SystemName}-${EnvType}-${Tagprefix}-Cloud9-RouteTable-PrivateSubnet
      - Key: Owner
        Value: !Ref Owner

  PrivateSubnetRouteTableAssociation1:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref PrivateSubnet1
      RouteTableId: !Ref PrivateRouteTable1
    
# ------------------------------------------------------------#
# Add route in a route table with private subnet
# ------------------------------------------------------------#

  PrivateRoute:
    Type: AWS::EC2::Route
    Properties:
       RouteTableId: !Ref PrivateRouteTable1
       DestinationCidrBlock: 0.0.0.0/0
       NatGatewayId: !Ref NatGateway

# ------------------------------------------------------------#
# Create Cloud9
# ------------------------------------------------------------#

  Cloud9:
    Type: AWS::Cloud9::EnvironmentEC2
    Properties: 
      Description: "The Cloud9 on Private subnet connecting Via SSM. This server is for building Docker images so on."
      AutomaticStopTimeMinutes: 30
      ConnectionType: CONNECT_SSM
      ImageId: resolve:ssm:/aws/service/cloud9/amis/amazonlinux-2-x86_64
      InstanceType: !Ref InstanceType
      Name: !Sub ${SystemName}-${EnvType}-${Tagprefix}-Cloud9
      SubnetId: !Ref PrivateSubnet1
      Tags:
        - Key: Owner
          Value: !Ref Owner

# ------------------------------------------------------------#
# Create IAM resources for Cloud9
# ------------------------------------------------------------#
 
  IAMRoleforCloud9:
    Type: AWS::IAM::Role
    Properties:
      RoleName: !Sub ${SystemName}-${EnvType}-${Tagprefix}-Cloud9-IAM-Role1
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - ec2.amazonaws.com
                - cloud9.amazonaws.com
            Action:
              - sts:AssumeRole
      MaxSessionDuration: 3600
      Path: /
      ManagedPolicyArns: 
        - arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore      
      Tags:
        - Key: Name
          Value: !Sub ${SystemName}-${EnvType}-${Tagprefix}-Cloud9-IAM-Role1
        - Key: Owner
          Value: !Ref Owner

  InstanceProfileforCloud9:
    Type: 'AWS::IAM::InstanceProfile'
    Properties:
      InstanceProfileName: !Sub ${SystemName}-${EnvType}-${Tagprefix}-Cloud9-IAM-Role1
      Path: /
      Roles:
        - !Ref IAMRoleforCloud9

  IAMPolicyforCloud9:
    Type: 'AWS::IAM::ManagedPolicy'
    Properties:
      Description: Policy for Cloud9
      Path: /    
      ManagedPolicyName: !Sub ${SystemName}-${EnvType}-${Tagprefix}-Cloud9-IAM-Policy1
      PolicyDocument:
        Version: "2012-10-17"
        Statement:
          - Effect: Allow
            Action:
              - ecr:BatchCheckLayerAvailability
              - ecr:GetDownloadUrlForLayer
              - ecr:GetRepositoryPolicy
              - ecr:DescribeRepositories
              - ecr:ListImages
              - ecr:DescribeImages
              - ecr:BatchGetImage
              - ecr:GetLifecyclePolicy
              - ecr:GetLifecyclePolicyPreview
              - ecr:ListTagsForResource
              - ecr:DescribeImageScanFindings
              - ecr:InitiateLayerUpload
              - ecr:UploadLayerPart
              - ecr:CompleteLayerUpload
              - ecr:PutImage
            Resource: !Sub 'arn:aws:ecr:${AWS::Region}:${AWS::AccountId}:repository/${RepositoryName}'
          - Effect: Allow
            Action: ecr:GetAuthorizationToken
            Resource: '*'
          - Effect: Allow
            Action: s3:GetObject
            Resource: !Sub 'arn:aws:s3:::prod-${AWS::Region}-starport-layer-bucket/*'
          - Effect: Allow
            Action: codecommit:GitPull
            Resource: '*'
      Roles:
        - !Ref IAMRoleforCloud9

# ------------------------------------------------------------#
# Create Security Group for VPC EndPoint(Systems Manager)
# ------------------------------------------------------------#

  SecurityGroupEndpointsEc2messages:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupName: !Sub ${SystemName}-${EnvType}-${Tagprefix}-Cloud9-SG-ForEndpoint-Ec2messages
      GroupDescription: Security Group for SSM VPC Endpoint.
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 443
          ToPort: 443
          CidrIp: !Ref VPCCIDR
      VpcId: !Ref VPC
      Tags:
        - Key: Name
          Value: !Sub ${SystemName}-${EnvType}-${Tagprefix}-Cloud9-SG-ForEndpoint-Ec2messages
        - Key: Owner
          Value: !Ref Owner

  SecurityGroupEndpointsSsm:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupName: !Sub ${SystemName}-${EnvType}-${Tagprefix}-Cloud9-SG-ForEndpoint-Ssm
      GroupDescription: Security Group for SSM VPC Endpoint.
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 443
          ToPort: 443
          CidrIp: !Ref VPCCIDR
      VpcId: !Ref VPC
      Tags:
        - Key: Name
          Value: !Sub ${SystemName}-${EnvType}-${Tagprefix}-Cloud9-SG-ForEndpoint-Ssm
        - Key: Owner
          Value: !Ref Owner

  SecurityGroupEndpointsSsmmessages:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupName: !Sub ${SystemName}-${EnvType}-${Tagprefix}-Cloud9-SG-ForEndpoint-Ssmmessages
      GroupDescription: Security Group for SSM VPC Endpoint.
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 443
          ToPort: 443
          CidrIp: !Ref VPCCIDR
      VpcId: !Ref VPC
      Tags:
        - Key: Name
          Value: !Sub ${SystemName}-${EnvType}-${Tagprefix}-Cloud9-SG-ForEndpoint-Ssmmessages
        - Key: Owner
          Value: !Ref Owner

# ------------------------------------------------------------#
# Create Security Group for VPC End Point(CodeCommit)
# ------------------------------------------------------------#

  SecurityGroupEndpointsCodeCommit:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupName: !Sub ${SystemName}-${EnvType}-${Tagprefix}-Cloud9-SG-ForEndpoint-CodeCommit
      GroupDescription: Security Group for CodeCommit VPC Endpoint.
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 443
          ToPort: 443
          CidrIp: !Ref VPCCIDR
      VpcId: !Ref VPC
      Tags:
        - Key: Name
          Value: !Sub ${SystemName}-${EnvType}-${Tagprefix}-Cloud9-SG-ForEndpoint-CodeCommit
        - Key: Owner
          Value: !Ref Owner

  SecurityGroupEndpointsGitCodeCommit:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupName: !Sub ${SystemName}-${EnvType}-${Tagprefix}-Cloud9-SG-ForEndpoint-GitCodeCommit
      GroupDescription: Security Group for GitCodeCommit VPC Endpoint.
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 443
          ToPort: 443
          CidrIp: !Ref VPCCIDR
      VpcId: !Ref VPC
      Tags:
        - Key: Name
          Value: !Sub ${SystemName}-${EnvType}-${Tagprefix}-Cloud9-SG-ForEndpoint-GitCodeCommit
        - Key: Owner
          Value: !Ref Owner

# ------------------------------------------------------------#
# Create Security Group for VPC End Point(ECR)
# ------------------------------------------------------------#

  SecurityGroupEndpointsEcrdkr:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupName: !Sub ${SystemName}-${EnvType}-${Tagprefix}-Cloud9-SG-ForEndpoint-Ecrdkr
      GroupDescription: Security Group for ECR VPC Endpoint.
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 443
          ToPort: 443
          CidrIp: !Ref VPCCIDR
      VpcId: !Ref VPC
      Tags:
        - Key: Name
          Value: !Sub ${SystemName}-${EnvType}-${Tagprefix}-Cloud9-SG-ForEndpoint-Ecrdkr
        - Key: Owner
          Value: !Ref Owner

  SecurityGroupEndpointsEcrapi:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupName: !Sub ${SystemName}-${EnvType}-${Tagprefix}-Cloud9-SG-ForEndpoint-Ecrapi
      GroupDescription: Security Group for ECR VPC Endpoint.
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 443
          ToPort: 443
          CidrIp: !Ref VPCCIDR
      VpcId: !Ref VPC
      Tags:
        - Key: Name
          Value: !Sub ${SystemName}-${EnvType}-${Tagprefix}-Cloud9-SG-ForEndpoint-Ecrapi
        - Key: Owner
          Value: !Ref Owner

# ------------------------------------------------------------#
# Create Systems Manager End Point
# ------------------------------------------------------------#

  Ec2messagesEndpoint:
    Type: AWS::EC2::VPCEndpoint
    Properties:
      ServiceName: !Sub com.amazonaws.${AWS::Region}.ec2messages
      SubnetIds:
        - !Ref PrivateSubnet1
      VpcId: !Ref VPC
      VpcEndpointType: Interface
      SecurityGroupIds:
        - !Ref SecurityGroupEndpointsEc2messages
      PrivateDnsEnabled: true
    DependsOn: SecurityGroupEndpointsEc2messages

  SsmEndpoint:
    Type: AWS::EC2::VPCEndpoint
    Properties:
      ServiceName: !Sub com.amazonaws.${AWS::Region}.ssm
      SubnetIds:
        - !Ref PrivateSubnet1
      VpcId: !Ref VPC
      VpcEndpointType: Interface
      SecurityGroupIds:
        - !Ref SecurityGroupEndpointsSsm
      PrivateDnsEnabled: true
    DependsOn: SecurityGroupEndpointsSsm

  SsmmessagesEndpoint:
    Type: AWS::EC2::VPCEndpoint
    Properties:
      ServiceName: !Sub com.amazonaws.${AWS::Region}.ssmmessages
      SubnetIds:
        - !Ref PrivateSubnet1
      VpcId: !Ref VPC
      VpcEndpointType: Interface
      SecurityGroupIds:
        - !Ref SecurityGroupEndpointsSsmmessages
      PrivateDnsEnabled: true
    DependsOn: SecurityGroupEndpointsSsmmessages

# ------------------------------------------------------------#
# Create CodeCommit End Point
# ------------------------------------------------------------#

  CodeCommitEndpoint:
    Type: AWS::EC2::VPCEndpoint
    Properties:
      ServiceName: !Sub com.amazonaws.${AWS::Region}.codecommit
      SubnetIds:
        - !Ref PrivateSubnet1
      VpcId: !Ref VPC
      VpcEndpointType: Interface
      SecurityGroupIds:
        - !Ref SecurityGroupEndpointsCodeCommit
      PrivateDnsEnabled: true
    DependsOn: SecurityGroupEndpointsCodeCommit

  GitCodeCommitEndpoint:
    Type: AWS::EC2::VPCEndpoint
    Properties:
      ServiceName: !Sub com.amazonaws.${AWS::Region}.git-codecommit
      SubnetIds:
        - !Ref PrivateSubnet1
      VpcId: !Ref VPC
      VpcEndpointType: Interface
      SecurityGroupIds:
        - !Ref SecurityGroupEndpointsGitCodeCommit
      PrivateDnsEnabled: true
    DependsOn: SecurityGroupEndpointsGitCodeCommit

# ------------------------------------------------------------#
# Create ECR End Point
# Amazon ECS tasks hosted on Fargate using platform version 1.4.0 or later require both the com.amazonaws.region.ecr.dkr
# and com.amazonaws.region.ecr.api Amazon ECR VPC endpoints as well as the Amazon S3 gateway endpoint to take advantage of this feature.
# For more information, see https://docs.aws.amazon.com/AmazonECR/latest/userguide/vpc-endpoints.html
# ------------------------------------------------------------#

  EcrdkrEndpoint:
    Type: AWS::EC2::VPCEndpoint
    Properties:
      ServiceName: !Sub com.amazonaws.${AWS::Region}.ecr.dkr
      SubnetIds:
        - !Ref PrivateSubnet1
      VpcId: !Ref VPC
      VpcEndpointType: Interface
      SecurityGroupIds:
        - !Ref SecurityGroupEndpointsEcrdkr
      PrivateDnsEnabled: true
    DependsOn: SecurityGroupEndpointsEcrdkr

  EcrapiEndpoint:
    Type: AWS::EC2::VPCEndpoint
    Properties:
      ServiceName: !Sub com.amazonaws.${AWS::Region}.ecr.api
      SubnetIds:
        - !Ref PrivateSubnet1
      VpcId: !Ref VPC
      VpcEndpointType: Interface
      SecurityGroupIds:
        - !Ref SecurityGroupEndpointsEcrapi
      PrivateDnsEnabled: true
    DependsOn: SecurityGroupEndpointsEcrapi

# ------------------------------------------------------------#
# Create S3 End Point(Gateway)
# ------------------------------------------------------------#

  S3GatewayEndpoint:
    Type: 'AWS::EC2::VPCEndpoint'
    Properties:
      RouteTableIds:
        - !Ref PrivateRouteTable1
      ServiceName: !Sub com.amazonaws.${AWS::Region}.s3
      VpcId: !Ref VPC
      VpcEndpointType: Gateway

# ------------------------------------------------------------#
# Outputs
# This can output each value specified as output section.
# For more information, see https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/resources-section-structure.html.
# ------------------------------------------------------------# 

Outputs:
  VPC:
    Description: VPC
    Value: !Ref VPC

  PublicSubnet1:
    Description: PublicSubnet1
    Value: !Ref PublicSubnet1

  PrivateSubnet1:
    Description: PrivateSubnet1
    Value: !Ref PrivateSubnet1

  InternetGateway:
    Description: InternetGateway
    Value: !Ref InternetGateway
  
  NatGatewayEIP:
    Description: NatGatewayEIP
    Value: !Ref NatGatewayEIP  

  NatGateway:
    Description: NatGateway
    Value: !Ref NatGateway  

  NetworkACLForPublicSubnet1:
    Description: NetworkACLForPublicSubnet1
    Value: !Ref NetworkACLForPublicSubnet1  

  NetworkACLForPrivateSubnet1:
    Description: NetworkACLForPrivateSubnet1
    Value: !Ref NetworkACLForPrivateSubnet1  

  PublicRouteTable1:
    Description: PublicRouteTable1
    Value: !Ref PublicRouteTable1  

  PrivateRouteTable1:
    Description: PrivateRouteTable1
    Value: !Ref PrivateRouteTable1  

  Cloud9:
    Description: Cloud9
    Value: !Ref Cloud9  

  AZ1:
    Description: Availability Zone 1
    Value: !GetAtt
      - PrivateSubnet1
      - AvailabilityZone

# ------------------------------------------------------------#
# Output Security Groups
# ------------------------------------------------------------#

  SecurityGroupEndpointsEc2messages:
    Description: SecurityGroupEndpointsEc2messages
    Value: !Ref SecurityGroupEndpointsEc2messages

  SecurityGroupEndpointsSsm:
    Description: SecurityGroupEndpointsSsm
    Value: !Ref SecurityGroupEndpointsSsm

  SecurityGroupEndpointsSsmmessages:
    Description: SecurityGroupEndpointsSsmmessages
    Value: !Ref SecurityGroupEndpointsSsmmessages

  SecurityGroupEndpointsEcrdkr:
    Description: SecurityGroupEndpointsEcrdkr
    Value: !Ref SecurityGroupEndpointsEcrdkr

  SecurityGroupEndpointsEcrapi:
    Description: SecurityGroupEndpointsEcrapi
    Value: !Ref SecurityGroupEndpointsEcrapi

  SecurityGroupEndpointsCodeCommit:
    Description: SecurityGroupEndpointsCodeCommit
    Value: !Ref SecurityGroupEndpointsCodeCommit

  SecurityGroupEndpointsGitCodeCommit:
    Description: SecurityGroupEndpointsGitCodeCommit
    Value: !Ref SecurityGroupEndpointsGitCodeCommit

# ------------------------------------------------------------#
# Output VPC Endpoints using Interface Type
# ------------------------------------------------------------#

  Ec2messagesEndpoint:
    Description: Ec2messagesEndpoint
    Value: !Ref Ec2messagesEndpoint

  SsmEndpoint:
    Description: SsmEndpoint
    Value: !Ref SsmEndpoint

  SsmmessagesEndpoint:
    Description: SsmmessagesEndpoint
    Value: !Ref SsmmessagesEndpoint

  S3GatewayEndpoint:
    Description: S3GatewayEndpoint
    Value: !Ref S3GatewayEndpoint

  EcrdkrEndpoint:
    Description: EcrdkrEndpoint
    Value: !Ref EcrdkrEndpoint

  EcrapiEndpoint:
    Description: EcrapiEndpoint
    Value: !Ref EcrapiEndpoint

  CodeCommitEndpoint:
    Description: CodeCommitEndpoint
    Value: !Ref CodeCommitEndpoint

  GitCodeCommitEndpoint:
    Description: GitCodeCommitEndpoint
    Value: !Ref GitCodeCommitEndpoint

# ------------------------------------------------------------#
# Output VPC Endpoints using Gateway Type
# ------------------------------------------------------------#

  S3GatewayEndpoint:
    Description: S3GatewayEndpoint
    Value: !Ref S3GatewayEndpoint

# ------------------------------------------------------------#
# Output IAM resources
# ------------------------------------------------------------#

  IAMRoleforCloud9:
    Description: IAMRoleforCloud9
    Value: !Ref IAMRoleforCloud9    

  IAMPolicyforCloud9:
    Description: IAMPolicyforCloud9
    Value: !Ref IAMPolicyforCloud9

5. テンプレートの補足

テンプレートに関して補足します。
なお、以前のブログ【CloudFormation】パブリックサブネットにCloud9(SSH)をデプロイしてみるで補足した内容については、冗長になるので割愛します。

5.1 想定ユースケース

今回のテンプレートは、次の様なユースケースを想定して作成しました。

DockerFileをローカルからアップロードして、Build/Ship/Runを実行できる
したがって、NAT Gatewayを配置している
コンテナイメージは、ECRのプライベートリポジトリに格納する
セッションマネージャー経由でCloud9インスタンスにアクセスする
セッションマネージャーの諸設定(Preference)は行わない(ロギング設定等はしない)

5.2 セキュリティグループとIAMロール

Cloud9を生成する時、セキュリティグループとIAMロールが自動で生成、アタッチされます。CloudFormationのプロパティでも指定する事が出来ません。
セキュリティグループ名は、指定したCloud9のインスタンス名の末尾に-InstanceSecurityGroup-ランダム文字列を付与した形になります。
上記のような仕様のため、例えばSSM用のVPCエンドポイントにおけるセキュリティグループ設定で接続元をVPC CIDRではなく、セキュリティグループにすると、Cloud9のセキュリティグループ名と一致させることが出来ず、SSM経由のアクセスに失敗しテンプレートがロールバックしますので、ご注意ください。

別のブログにまとめようと思っていますが、Cloud9のインスタンス作成後かつデフォルトで生成されるセキュリティグループがアタッチされた後に、SSM用VPCエンドポイントのセキュリティグループの接続元に指定されたセキュリティグループをCloud9にアタッチすることでテンプレートを作成することが出来ます。

続いて、IAMロールですが、接続方式がSSMの場合、自動的にインスタンスプロファイルAWSCloud9SSMInstanceProfileが作成されCloud9にアタッチされます。
このインスタンスプロファイルによってCloud9にIAMロール(AWSCloud9SSMAccessRole)がアタッチされます。
このIAMロール(AWSCloud9SSMAccessRole)には許可ポリシー(AWSCloud9SSMInstanceProfile)が付与されています。

次の通り、AWSCloud9SSMInstanceProfileには、SSMに関するアクションの許可権限しかないです。

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ssmmessages:CreateControlChannel",
                "ssmmessages:CreateDataChannel",
                "ssmmessages:OpenControlChannel",
                "ssmmessages:OpenDataChannel",
                "ssm:UpdateInstanceInformation"
            ],
            "Resource": "*"
        }
    ]
}

したがって、CloudFormationで以下のIAMロールを作っています。
繰り返しになりますが、テンプレートのプロファイルではCloud9にIAMロールを直接付与することが出来ないので、作成後にIAMロールを付け替えるようにしてください。

  IAMRoleforCloud9:
    Type: AWS::IAM::Role
    Properties:
      RoleName: !Sub ${SystemName}-${EnvType}-${Tagprefix}-Cloud9-IAM-Role1
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - ec2.amazonaws.com
                - cloud9.amazonaws.com
            Action:
              - sts:AssumeRole
      MaxSessionDuration: 3600
      Path: /
      ManagedPolicyArns: 
        - arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore      
      Tags:
        - Key: Name
          Value: !Sub ${SystemName}-${EnvType}-${Tagprefix}-Cloud9-IAM-Role1
        - Key: Owner
          Value: !Ref Owner

  InstanceProfileforCloud9:
    Type: 'AWS::IAM::InstanceProfile'
    Properties:
      InstanceProfileName: !Sub ${SystemName}-${EnvType}-${Tagprefix}-Cloud9-IAM-Role1
      Path: /
      Roles:
        - !Ref IAMRoleforCloud9

  IAMPolicyforCloud9:
    Type: 'AWS::IAM::ManagedPolicy'
    Properties:
      Description: Policy for Cloud9
      Path: /    
      ManagedPolicyName: !Sub ${SystemName}-${EnvType}-${Tagprefix}-Cloud9-IAM-Policy1
      PolicyDocument:
        Version: "2012-10-17"
        Statement:
          - Effect: Allow
            Action:
              - ecr:BatchCheckLayerAvailability
              - ecr:GetDownloadUrlForLayer
              - ecr:GetRepositoryPolicy
              - ecr:DescribeRepositories
              - ecr:ListImages
              - ecr:DescribeImages
              - ecr:BatchGetImage
              - ecr:GetLifecyclePolicy
              - ecr:GetLifecyclePolicyPreview
              - ecr:ListTagsForResource
              - ecr:DescribeImageScanFindings
              - ecr:InitiateLayerUpload
              - ecr:UploadLayerPart
              - ecr:CompleteLayerUpload
              - ecr:PutImage
            Resource: !Sub 'arn:aws:ecr:${AWS::Region}:${AWS::AccountId}:repository/${RepositoryName}'
          - Effect: Allow
            Action: ecr:GetAuthorizationToken
            Resource: '*'
          - Effect: Allow
            Action: s3:GetObject
            Resource: !Sub 'arn:aws:s3:::prod-${AWS::Region}-starport-layer-bucket/*'
          - Effect: Allow
            Action: codecommit:GitPull
            Resource: '*'
      Roles:
        - !Ref IAMRoleforCloud9

必要に応じてポリシーを変更してください。

公式ドキュメント抜粋：

This policy grants administrative permissions that allow IAM users to read and write to repositories, but doesn't allow them to delete repositories or change the policy documents that are applied to them.

This policy includes the following permissions:

ecr – Allows principals to read and write to respositores, as well as read lifecycle policies. Principals aren't granted permission to delete repositories or change the lifecycle policies that are applied to them.

詳細は、公式ドキュメントを参照ください。

AWS managed policies for Amazon Elastic Container Registry

5.3 ECR・S3について

ECRへのアクセスはVPCエンドポイントを経由しますが。
主にイメージのPullおよびPushになると思います。

テンプレートにも記載がある通り、ECR用のエンドポイントを作成しています。

  EcrdkrEndpoint:
    Type: AWS::EC2::VPCEndpoint
    Properties:
      ServiceName: !Sub com.amazonaws.${AWS::Region}.ecr.dkr
      SubnetIds:
        - !Ref PrivateSubnet1
      VpcId: !Ref VPC
      VpcEndpointType: Interface
      SecurityGroupIds:
        - !Ref SecurityGroupEndpointsEcrdkr
      PrivateDnsEnabled: true
    DependsOn: SecurityGroupEndpointsEcrdkr

  EcrapiEndpoint:
    Type: AWS::EC2::VPCEndpoint
    Properties:
      ServiceName: !Sub com.amazonaws.${AWS::Region}.ecr.api
      SubnetIds:
        - !Ref PrivateSubnet1
      VpcId: !Ref VPC
      VpcEndpointType: Interface
      SecurityGroupIds:
        - !Ref SecurityGroupEndpointsEcrapi
      PrivateDnsEnabled: true
    DependsOn: SecurityGroupEndpointsEcrapi

ECR用のエンドポイントは2つあり、今回Cloud9用として両方作成しています。
これらのエンドポイントは、ECSのバージョンまたはデータノードがFargateかによって必要な数が異なるのですが、
Cloud9の場合でもECRにShipする場合もECR用のエンドポイントが必要になります。

Amazon ECS tasks hosted on Fargate using platform version 1.3.0 or earlier only require the com.amazonaws.region.ecr.dkr Amazon ECR VPC endpoint and the Amazon S3 gateway endpoint to take advantage of this feature.

Amazon ECS tasks hosted on Fargate using platform version 1.4.0 or later require both the com.amazonaws.region.ecr.dkr and com.amazonaws.region.ecr.api Amazon ECR VPC endpoints as well as the Amazon S3 gateway endpoint to take advantage of this feature.

ECRプライベートリポジトリのイメージをPullする時の注意点として、S3のエンドポイントも必要になります。
これは、ECRで保管するイメージの実態がS3になるからだと思いますが、次の様なARNをポリシーのResourceで指定する必要があります。

arn:aws:s3:::prod-region-starport-layer-bucket/*

したがって、S3エンドポイントのポリシーに次のような設定を行っています。

        Statement:
          - Effect: Allow
            Principal:
              AWS: !GetAtt IAMRoleforCloud9.Arn
            Action: s3:GetObject
            Resource: !Sub 'arn:aws:s3:::prod-${AWS::Region}-starport-layer-bucket/*'

詳細は公式ドキュメントを参照ください。

Amazon ECR interface VPC endpoints (AWS PrivateLink)

5.4 CodeCommitについて

CodeCommitのエンドポイントとGitアクションを行うために、codecommitおよびgit-codecommitのエンドポイントを作成する必要があります。

  CodeCommitEndpoint:
    Type: AWS::EC2::VPCEndpoint
    Properties:
      ServiceName: !Sub com.amazonaws.${AWS::Region}.codecommit
      SubnetIds:
        - !Ref PrivateSubnet1
      VpcId: !Ref VPC
      VpcEndpointType: Interface
      SecurityGroupIds:
        - !Ref SecurityGroupEndpointsCodeCommit
      PrivateDnsEnabled: true
    DependsOn: SecurityGroupEndpointsCodeCommit

  GitCodeCommitEndpoint:
    Type: AWS::EC2::VPCEndpoint
    Properties:
      ServiceName: !Sub com.amazonaws.${AWS::Region}.git-codecommit
      SubnetIds:
        - !Ref PrivateSubnet1
      VpcId: !Ref VPC
      VpcEndpointType: Interface
      SecurityGroupIds:
        - !Ref SecurityGroupEndpointsGitCodeCommit
      PrivateDnsEnabled: true
    DependsOn: SecurityGroupEndpointsGitCodeCommit

5.5 AWS managed temporary credentials(AMTC)について

Cloud9 IDEでAWS CLIを利用する際にAWS側で一時的なクレデンシャルが発行され、それをAWS managed temporary credentialsといいます。

結論から言うと、このクレデンシャルをプライベートサブネットで利用する事は(例外を除き)出来ません。

理由は、利用者を概ね同じ権限が付与されますが、接続元IPがCloud9のグローバルIPアドレスに制限されるためです。
プライベートサブネット上のIDEからのAPIコールが失敗します。

ですので、AWS managed temporary credentialsを無効化してからAWS CLIを実行する様にしてください。

クラスメソッド様の記事が非常に分かりやすかったのでこちらを参照いただくのが、良いかと思います。

Cloud9 IDE で AWS CLIを実行する際に注意したいネットワーク的制約

公式ドキュメントはこのリンクのActions supported by AWS managed temporary credentials
を参照ください。

6. まとめ

プライベートサブネット上でCloud9を利用する際に、DockerのBuild/Shipが実行出来てかつ他リソースと隔離されたテンプレートがあれば良いなと思い作成したリソースです。

この記事が誰かの役に立てれば嬉しいです。Mitsuoでした！