💭

LocalStack上にAWS SAMでAPI Gateway、Lambda、DynamoDBを構築してみる

2021/12/22に公開約26,100字

はじめに

LocalStack上にAWS SAMを使用して「LocalStack上にAPI Gateway、Lambda、DynamoDB環境を構築してみる」と同じ環境を構築する。
CurlでApiを叩き。LambdaでDynamoDBからデータを取得しレスポンスを返す処理を実装する。

動作確認環境

WSL2上のUbuntu 20.04でLocalStackを使ってみる」で構築した環境を使い作成する。

Guest OS (WSL2)

追加アイテムだけ記載する。

  • Ubuntu 20.04
    • SAM CLI: 1.36.0
    • aws-sam-cli-local: 1.1.0.1

準備

作業フォルダー作成

hoge@AA:~$ mkdir aws-sam
hoge@AA:~$ cd aws-sam

AWS SAM CLI for LocalStackインストール

samlocalコマンドと合わせてsamコマンド(AWS SAM)もインストールされる。[1]

hoge@AA:~/aws-sam$ pip3 install aws-sam-cli-local==1.1.0.1
hoge@AA:~/aws-sam$ samlocal --version
SAM CLI, version 1.36.0

docker-composeを準備する

docker-compose.ymlファイルを作成する。
必要なSERVICESを指定する。
serverlessが使用する、s3,iam,sts,cloudformationをSERVICESに追加している。[2]

serverless.ymlを編集する。

hoge@AA:~/aws-sam$ vi docker-compose.yml
docker-compose.yml 編集後の状態
version: "3.8"

services:
  localstack:
    container_name: "${LOCALSTACK_DOCKER_NAME-localstack_main}"
    image: localstack/localstack:0.13.1
    network_mode: bridge
    ports:
      - "4566:4566"
      - "4571:4571"
    environment:
      - SERVICES=apigateway,lambda,dynamodb,s3,iam,sts,cloudformation
      - DEBUG=${DEBUG- }
      - DATA_DIR=${DATA_DIR- }
      - LAMBDA_EXECUTOR=${LAMBDA_EXECUTOR- }
      - LOCALSTACK_API_KEY=${LOCALSTACK_API_KEY- }  # only required for Pro
      - HOST_TMP_FOLDER=${TMPDIR:-/tmp/}localstack
      - DOCKER_HOST=unix:///var/run/docker.sock
      - LAMBDA_EXECUTOR=docker-reuse
    volumes:
      - "${TMPDIR:-/tmp}/localstack:/tmp/localstack"
      - "/var/run/docker.sock:/var/run/docker.sock"

docker-compose.ymlの編集を保存して終了する。(escキーを押す。「:wq」を入力してエンターキーを押す。)

LocalStackを立ち上げる。[3]

hoge@AA:~/aws-sam$ docker-compose up -d

AWS SAMの設定

テンプレート作成

hoge@AA:~/aws-sam$ samlocal init
samlocal init選択とログ
Which template source would you like to use?
        1 - AWS Quick Start Templates
        2 - Custom Template Location
Choice: 1
What package type would you like to use?
        1 - Zip (artifact is a zip uploaded to S3)
        2 - Image (artifact is an image uploaded to an ECR image repository)
Package type: 1⏎

Which runtime would you like to use?
        1 - nodejs14.x
        2 - python3.9
        3 - ruby2.7
        4 - go1.x
        5 - java11
        6 - dotnetcore3.1
        7 - nodejs12.x
        8 - nodejs10.x
        9 - python3.8
        10 - python3.7
        11 - python3.6
        12 - python2.7
        13 - ruby2.5
        14 - java8.al2
        15 - java8
        16 - dotnetcore2.1
Runtime: 9⏎

Project name [sam-app]: python3_sam

Cloning from https://github.com/aws/aws-sam-cli-app-templates

AWS quick start application templates:
        1 - Hello World Example
        2 - EventBridge Hello World
        3 - EventBridge App from scratch (100+ Event Schemas)
        4 - Step Functions Sample App (Stock Trader)
        5 - Elastic File System Sample App
Template selection: 1⏎

    -----------------------
    Generating application:
    -----------------------
    Name: python3_sam
    Runtime: python3.8
    Architectures: x86_64
    Dependency Manager: pip
    Application Template: hello-world
    Output Directory: .

    Next application steps can be found in the README file at ./python3_sam/README.md


    Commands you can use next
    =========================
    [*] Create pipeline: cd python3_sam && sam pipeline init --bootstrap
    [*] Test Function in the Cloud: sam sync --stack-name {stack-name} --watch

テンプレートのフォルダー構造[4]

hoge@AA:~/aws-sam/python3_sam$ tree
.
├── README.md
├── __init__.py
├── events
│   └── event.json
├── hello_world
│   ├── __init__.py
│   ├── app.py
│   └── requirements.txt
├── template.yaml
└── tests
    ├── __init__.py
    ├── integration
    │   ├── __init__.py
    │   └── test_api_gateway.py
    ├── requirements.txt
    └── unit
        ├── __init__.py
        └── test_handler.py

必要最小限のファイルだけ書き換える。テストなどは変更しない。

template.yaml

hello_world/app.pyを編集する。

hoge@AA:~/aws-sam/python3_sam$ vi template.yaml
template.yaml
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
  python3_sam

  Sample SAM Template for python3_sam

# More info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rst
Globals:
  Function:
    Timeout: 3

Resources:
  UsersFunction:
    Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
    Properties:
      CodeUri: hello_world/
      Handler: app.getUsers
      Runtime: python3.8
      Architectures:
        - x86_64
      Events:
        Users:
          Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api
          Properties:
            Path: /users
            Method: get

  myDynamoDBTable: 
    Type: AWS::DynamoDB::Table
    Properties: 
      TableName: myUserTable
      AttributeDefinitions:
        - AttributeName: Name
          AttributeType: S
        - AttributeName: Index
          AttributeType: N
      KeySchema:
        - AttributeName: Name
          KeyType: HASH
        - AttributeName: Index
          KeyType: RANGE
      ProvisionedThroughput: 
        ReadCapacityUnits: 10
        WriteCapacityUnits: 5

Outputs:
  # ServerlessRestApi is an implicit API created out of Events key under Serverless::Function
  # Find out more about other implicit resources you can reference within SAM
  # https://github.com/awslabs/serverless-application-model/blob/master/docs/internals/generated_resources.rst#api
  UsersApi:
    Description: "API Gateway endpoint URL for Prod stage for Users function"
    Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/users/"
  UsersFunction:
    Description: "Users Lambda Function ARN"
    Value: !GetAtt UsersFunction.Arn
  UsersFunctionIamRole:
    Description: "Implicit IAM Role created for Users function"
    Value: !GetAtt UsersFunctionRole.Arn

template.yamlの編集を保存して終了する。(escキーを押す。「:wq」を入力してエンターキーを押す。)

hello_world/app.py

hello_world/app.pyを編集する。

hoge@AA:~/aws-sam/python3_sam$ vi hello_world/app.py
hello_world/app.py
import os
import boto3
from boto3.session import Session
from datetime import datetime

session = Session(
    aws_access_key_id='dummy',
    aws_secret_access_key='dummy',
    region_name='us-east-1'
)

if os.getenv('LOCALSTACK_HOSTNAME') is None:
    endpoint = 'http://localhost:4566'
else:
    endpoint=f"http://{os.environ['LOCALSTACK_HOSTNAME']}:4566"

dynamodb = session.resource(
    service_name='dynamodb', 
    endpoint_url=endpoint
)

def getUsers(event, context):
    table = dynamodb.Table('myUserTable')
    data = []
    response = table.scan()
    while True:
        data.extend(response["Items"])
        if "LastEvaluatedKey" not in response:
            break
        response = table.scan(ExclusiveStartKey=response["LastEvaluatedKey"])

    return {
        'statusCode': 200,
        'body':{ 
            'users':data
        }
    }

hello_world/app.pyの編集を保存して終了する。(escキーを押す。「:wq」を入力してエンターキーを押す。)

ビルド

プロジェクトをビルドする。

hoge@AA:~/aws-sam/python3_sam$ samlocal build
Building codeuri: /home/hoge/aws-sam/python3_sam/hello_world runtime: python3.8 metadata: {} architecture: x86_64 functions: ['UsersFunction']
Running PythonPipBuilder:ResolveDependencies
Running PythonPipBuilder:CopySource

Build Succeeded

Built Artifacts  : .aws-sam/build
Built Template   : .aws-sam/build/template.yaml

Commands you can use next
=========================
[*] Invoke Function: sam local invoke
[*] Test Function in the Cloud: sam sync --stack-name {stack-name} --watch
[*] Deploy: sam deploy --guided

動作確認

デプロイ

hoge@AA:~/aws-sam/python3_sam$ samlocal deploy --guided --region us-east-1 --stack-name my-stack
samlocal deploy選択とログ
Configuring SAM deploy
======================

        Looking for config file [samconfig.toml] :  Not found

        Setting default arguments for 'sam deploy'
        =========================================
        Stack Name [my-stack]:⏎
        AWS Region [us-east-1]:⏎
        #Shows you resources changes to be deployed and require a 'Y' to initiate deploy
        Confirm changes before deploy [y/N]: y⏎
        #SAM needs permission to be able to create roles to connect to the resources in your template
        Allow SAM CLI IAM role creation [Y/n]:⏎
        #Preserves the state of previously provisioned resources when an operation fails
        Disable rollback [y/N]:⏎
        UsersFunction may not have authorization defined, Is this okay? [y/N]: y⏎
        Save arguments to configuration file [Y/n]:⏎
        SAM configuration file [samconfig.toml]:⏎
        SAM configuration environment [default]:⏎

        Looking for resources needed for deployment:
        Creating the required resources...
        Successfully created!
         Managed S3 bucket: aws-sam-cli-managed-default-samclisourcebucket-f892687c
         A different default S3 bucket can be set in samconfig.toml

        Saved arguments to config file
        Running 'sam deploy' for future deployments will use the parameters saved above.
        The above parameters can be changed by modifying samconfig.toml
        Learn more about samconfig.toml syntax at
        https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-config.html

Uploading to my-stack/a9e5d830bc93ddfdde1a54688915befb  451291 / 451291  (100.00%)

        Deploying with following values
        ===============================
        Stack name                   : my-stack
        Region                       : us-east-1
        Confirm changeset            : True
        Disable rollback             : False
        Deployment s3 bucket         : aws-sam-cli-managed-default-samclisourcebucket-f892687c
        Capabilities                 : ["CAPABILITY_IAM"]
        Parameter overrides          : {}
        Signing Profiles             : {}

Initiating deployment
=====================
Uploading to my-stack/d8b82bb43720dc66b29a5d306ce757f6.template  1509 / 1509  (100.00%)

Waiting for changeset to be created..

CloudFormation stack changeset
-----------------------------------------------------------------------------------------------------------------------------
Operation                       LogicalResourceId               ResourceType                    Replacement
-----------------------------------------------------------------------------------------------------------------------------
+ Add                           myDynamoDBTable                 AWS::DynamoDB::Table            False
+ Add                           UsersFunction                   AWS::Lambda::Function           False
+ Add                           UsersFunctionRole               AWS::IAM::Role                  False
+ Add                           UsersFunctionUsersPermissionP   AWS::Lambda::Permission         False
                                rod
+ Add                           ServerlessRestApi               AWS::ApiGateway::RestApi        False
+ Add                           ServerlessRestApiDeployment8a   AWS::ApiGateway::Deployment     False
                                5f04c8fd
+ Add                           ServerlessRestApiProdStage      AWS::ApiGateway::Stage          False
-----------------------------------------------------------------------------------------------------------------------------

Changeset created successfully. arn:aws:cloudformation:us-east-1:000000000000:changeSet/samcli-deploy1640162327/2229ff87


Previewing CloudFormation changeset before deployment
======================================================
Deploy this changeset? [y/N]: y⏎

2020-10-09 13:02:09 - Waiting for stack create/update to complete

CloudFormation events from stack operations
-----------------------------------------------------------------------------------------------------------------------------
ResourceStatus                  ResourceType                    LogicalResourceId               ResourceStatusReason
-----------------------------------------------------------------------------------------------------------------------------
CREATE_COMPLETE                 AWS::CloudFormation::Stack      myDynamoDBTable                 -
CREATE_COMPLETE                 AWS::CloudFormation::Stack      ServerlessRestApi               -
CREATE_COMPLETE                 AWS::CloudFormation::Stack      UsersFunctionRole               -
CREATE_COMPLETE                 AWS::CloudFormation::Stack      ServerlessRestApiProdStage      -
CREATE_COMPLETE                 AWS::CloudFormation::Stack      ServerlessRestApiDeployment8a   -
                                                                5f04c8fd
CREATE_COMPLETE                 AWS::CloudFormation::Stack      UsersFunction                   -
CREATE_COMPLETE                 AWS::CloudFormation::Stack      my-stack                        -
CREATE_COMPLETE                 AWS::CloudFormation::Stack      UsersFunctionUsersPermissionP   -
                                                                rod
-----------------------------------------------------------------------------------------------------------------------------

CloudFormation outputs from deployed stack
-------------------------------------------------------------------------------------------------------------------------------
Outputs
-------------------------------------------------------------------------------------------------------------------------------
Key                 UsersApi
Description         API Gateway endpoint URL for Prod stage for Users function
Value               https://XXXXXXXX.execute-api.amazonaws.com:4566/Prod/users/

Key                 UsersFunction
Description         Users Lambda Function ARN
Value               arn:aws:lambda:us-east-1:000000000000:function:my-stack-UsersFunction-8ab12ade

Key                 UsersFunctionIamRole
Description         Implicit IAM Role created for Users function
Value               arn:aws:iam::000000000000:role/my-stack-UsersFunctionRole-7105ca15
-------------------------------------------------------------------------------------------------------------------------------

Successfully created/updated stack - my-stack in us-east-1

Rust API IDをメモしておく。UsersApiのValue「XXXXXXXX」の部分。

-------------------------------------------------------------------------------------------------------------------------------
Outputs
-------------------------------------------------------------------------------------------------------------------------------
Key                 UsersApi
Description         API Gateway endpoint URL for Prod stage for Users function
Value               https://XXXXXXXX.execute-api.amazonaws.com:4566/Prod/users/

StageNameの確認する。「Stage」と「Prod」が作成される。

awslocal apigateway get-stages --rest-api-id XXXXXXXX
{
    "item": [
        {
            "deploymentId": "YYYYYYYY",
            "stageName": "Stage",
            "description": "",
            "cacheClusterEnabled": false,
            "methodSettings": {},
            "variables": {}
        },
        {
            "deploymentId": "ZZZZZZZZ",
            "stageName": "Prod",
            "description": "",
            "cacheClusterEnabled": false,
            "methodSettings": {},
            "variables": {},
            "tags": {}
        }
    ]
}

アイテムの追加

DynamoDBアイテム追加する。

hoge@AA:~/aws-sam/python3_sam$ awslocal dynamodb put-item --table-name myUserTable \
--item '{"Name":{"S": "Ishida Mio"}, "Index":{"N": "1000"}, "Gender":{"S": "Female"}, "Tel":{"S": "0769625106"}, "Mail":{"S": "mio1094@ccfrevk.ll"}, "Birthday":{"S": "1982/02/26"}}' \
--profile=localstack
hoge@AA:~/aws-sam/python3_sam$ awslocal dynamodb put-item --table-name myUserTable \
--item '{"Name":{"S": "Shimura Kokoro"}, "Index":{"N": "1001"}, "Gender":{"S": "Female"}, "Tel":{"S": "0829490871"}, "Mail":{"S": "kokoro414@bevhlk.xju"}, "Birthday":{"S": "1990/02/17"}}' \
--profile=localstack
hoge@AA:~/aws-sam/python3_sam$ awslocal dynamodb put-item --table-name myUserTable \
--item '{"Name":{"S": "Ishimura Masaki"}, "Index":{"N": "1002"}, "Gender":{"S": "Male"}, "Tel":{"S": "0848254480"}, "Mail":{"S": "masaki_ishimura@kokn.qgj"}, "Birthday":{"S": "1992/06/20"}}' \
--profile=localstack
hoge@AA:~/aws-sam/python3_sam$ awslocal dynamodb put-item --table-name myUserTable \
--item '{"Name":{"S": "Shirai Tomoko"}, "Index":{"N": "1003"}, "Gender":{"S": "Female"}, "Tel":{"S": "0742216492"}, "Mail":{"S": "tomoko225@vlyqs.pxt"}, "Birthday":{"S": "1982/09/29"}}' \
--profile=localstack
hoge@AA:~/aws-sam/python3_sam$ awslocal dynamodb put-item --table-name myUserTable \
--item '{"Name":{"S": "Yasui Tomoyuki"}, "Index":{"N": "1004"}, "Gender":{"S": "Male"}, "Tel":{"S": "095953680"}, "Mail":{"S": "tomoyuki772@jheauwpl.lyo"}, "Birthday":{"S": "1996/04/23"}}' \
--profile=localstack

API呼び出し

APIを呼び出してレスポンスを確認する。
APIはデプロイのRust API ID「XXXXXXXX」部分をを置き換える。「Stage」or「Prod」に置き換えて呼び出してください。

hoge@AA:~/aws-sam/python3_sam$ curl -X GET -s http://localhost:4566/restapis/XXXXXXXX/Prod/_user_request_/users  | jq
{
  "users": [
    {
      "Index": 1000,
      "Tel": "0769625106",
      "Birthday": "1982/02/26",
      "Gender": "Female",
      "Mail": "mio1094@ccfrevk.ll",
      "Name": "Ishida Mio"
    },
    {
      "Index": 1002,
      "Tel": "0848254480",
      "Birthday": "1992/06/20",
      "Gender": "Male",
      "Mail": "masaki_ishimura@kokn.qgj",
      "Name": "Ishimura Masaki"
    },
    {
      "Index": 1001,
      "Tel": "0829490871",
      "Birthday": "1990/02/17",
      "Gender": "Female",
      "Mail": "kokoro414@bevhlk.xju",
      "Name": "Shimura Kokoro"
    },
〜略〜

削除

samlocal delete

作成時の問題か設定の問題か不明だが、「EnableTerminationProtection」Keyが無いというエラーになる。
調べてみたが設定方法がわからなかった。

hoge@AA:~/aws-sam/python3_sam$ samlocal delete --stack-name my-stack --region us-east-1
        Are you sure you want to delete the stack my-stack in the region us-east-1 ? [y/N]: y
Traceback (most recent call last):
〜中略〜
  File "/home/hoge/.local/lib/python3.8/site-packages/samcli/lib/delete/cfn_utils.py", line 33, in has_stack
    if stack["EnableTerminationProtection"]:
KeyError: 'EnableTerminationProtection'

Stack設定を確認すると確かに「EnableTerminationProtection」がセットされていない。

hoge@AA:~/aws-sam/python3_sam$ awslocal cloudformation describe-stacks --region us-east-1 --profile localstack --stack-name "my-stack" --output json
{
    "Stacks": [
        {
            "StackId": "arn:aws:cloudformation:us-east-1:000000000000:stack/my-stack/dab3349f",
            "StackName": "my-stack",
            "ChangeSetId": "arn:aws:cloudformation:us-east-1:000000000000:changeSet/samcli-deploy1640163633/69053da1",
            "Description": "Created by SAM CLI at 2020-10-09T13:02:09.145319 UTC",
            "Parameters": [],
            "CreationTime": "2020-10-09T13:02:09.483000+00:00",
            "StackStatus": "CREATE_COMPLETE",
            "StackStatusReason": "Deployment succeeded",
            "Capabilities": [],
            "Outputs": [
                {
                    "OutputKey": "UsersApi",
                    "OutputValue": "https://XXXXXXXX.execute-api.amazonaws.com:4566/Prod/users/",
                    "Description": "API Gateway endpoint URL for Prod stage for Users function",
                    "ExportName": ""
                },
                {
                    "OutputKey": "UsersFunction",
                    "OutputValue": "arn:aws:lambda:us-east-1:000000000000:function:my-stack-UsersFunction-44c51310",
                    "Description": "Users Lambda Function ARN",
                    "ExportName": ""
                },
                {
                    "OutputKey": "UsersFunctionIamRole",
                    "OutputValue": "arn:aws:iam::000000000000:role/my-stack-UsersFunctionRole-adf023c3",
                    "Description": "Implicit IAM Role created for Users function",
                    "ExportName": ""
                }
            ],
            "Tags": []
        }
    ]
}

awslocal cloudformation delete-stack

エラーも出ず削除されている?

hoge@AA:~/aws-sam/python3_sam$ awslocal cloudformation delete-stack --stack-name my-stack --region us-east-1

再デプロイするとStack作成時にエラーが出る。
何か削除手順が足りないようだ。

hoge@AA:~/aws-sam/python3_sam$ samlocal deploy --region us-east-1 --stack-name my-stack
〜略〜
CREATE_FAILED                   AWS::CloudFormation::Stack      my-stack                        -
samlocal deploy全ログ
File with same data already exists at my-stack/a9e5d830bc93ddfdde1a54688915befb, skipping upload

        Deploying with following values
        ===============================
        Stack name                   : my-stack
        Region                       : us-east-1
        Confirm changeset            : True
        Disable rollback             : True
        Deployment s3 bucket         : aws-sam-cli-managed-default-samclisourcebucket-6a41e267
        Capabilities                 : ["CAPABILITY_IAM"]
        Parameter overrides          : {}
        Signing Profiles             : {}

Initiating deployment
=====================
File with same data already exists at my-stack/db0adca694251624519f17da6c16f33b.template, skipping upload

Waiting for changeset to be created..

CloudFormation stack changeset
-----------------------------------------------------------------------------------------------------------------------------
Operation                       LogicalResourceId               ResourceType                    Replacement
-----------------------------------------------------------------------------------------------------------------------------
* Modify                        myDynamoDBTable                 AWS::DynamoDB::Table            False
* Modify                        UsersFunction                   AWS::Lambda::Function           False
* Modify                        UsersFunctionRole               AWS::IAM::Role                  False
* Modify                        UsersFunctionUsersPermissionP   AWS::Lambda::Permission         False
                                rod
* Modify                        ServerlessRestApi               AWS::ApiGateway::RestApi        False
* Modify                        ServerlessRestApiDeployment8a   AWS::ApiGateway::Deployment     False
                                5f04c8fd
* Modify                        ServerlessRestApiProdStage      AWS::ApiGateway::Stage          False
-----------------------------------------------------------------------------------------------------------------------------

Changeset created successfully. arn:aws:cloudformation:us-east-1:000000000000:changeSet/samcli-deploy1640164603/32d12aff


Previewing CloudFormation changeset before deployment
======================================================
Deploy this changeset? [y/N]: y

2020-10-10 19:32:46 - Waiting for stack create/update to complete

CloudFormation events from stack operations
-----------------------------------------------------------------------------------------------------------------------------
ResourceStatus                  ResourceType                    LogicalResourceId               ResourceStatusReason
-----------------------------------------------------------------------------------------------------------------------------
CREATE_COMPLETE                 AWS::CloudFormation::Stack      UsersFunctionRole               -
CREATE_COMPLETE                 AWS::CloudFormation::Stack      myDynamoDBTable                 -
CREATE_COMPLETE                 AWS::CloudFormation::Stack      ServerlessRestApi               -
UPDATE_COMPLETE                 AWS::CloudFormation::Stack      UsersFunctionUsersPermissionP   -
                                                                rod
CREATE_FAILED                   AWS::CloudFormation::Stack      my-stack                        -
-----------------------------------------------------------------------------------------------------------------------------

参考にしたサイト

脚注
  1. AWS SAMコマンドのみインストールしたい場合。zipインストールHomebrewインストールを参考にすると良い。 ↩︎

  2. どのサービスが足りなくてServerlessが動かないのかわからななくて。DEBUG=1へ変更して、docker-compose upでログを見ながら確認した。 ↩︎

  3. エラーが出る場合はdockerサービス立ち上がっているかを確認する。「sudo service docker start」 ↩︎

  4. treeコマンドは「sudo apt install tree」でインストールできる ↩︎

Discussion

ログインするとコメントできます