💭
LocalStack上にAWS SAMでAPI Gateway、Lambda、DynamoDBを構築してみる
はじめに
LocalStack上にAWS SAMを使用して「LocalStack上にAPI Gateway、Lambda、DynamoDB環境を構築してみる」と同じ環境を構築する。
CurlでApiを叩き。LambdaでDynamoDBからデータを取得しレスポンスを返す処理を実装する。
動作確認環境
「WSL2上のUbuntu 20.04でLocalStackを使ってみる」で構築した環境を使い作成する。
Guest OS (WSL2)
追加アイテムだけ記載する。
- Ubuntu 20.04
- SAM CLI: 1.36.0
- aws-sam-cli-local: 1.1.0.1
準備
作業フォルダー作成
hoge@AA:~$ mkdir aws-sam
hoge@AA:~$ cd aws-sam
AWS SAM CLI for LocalStackインストール
samlocalコマンドと合わせてsamコマンド（AWS SAM）もインストールされる。[1]
hoge@AA:~/aws-sam$ pip3 install aws-sam-cli-local==1.1.0.1
hoge@AA:~/aws-sam$ samlocal --version
SAM CLI, version 1.36.0
docker-composeを準備する
docker-compose.ymlファイルを作成する。
必要なSERVICESを指定する。
serverlessが使用する、s3,iam,sts,cloudformationをSERVICESに追加している。[2]
serverless.ymlを編集する。
hoge@AA:~/aws-sam$ vi docker-compose.yml
docker-compose.yml 編集後の状態
version: "3.8"
services:
localstack:
container_name: "${LOCALSTACK_DOCKER_NAME-localstack_main}"
image: localstack/localstack:0.13.1
network_mode: bridge
ports:
- "4566:4566"
- "4571:4571"
environment:
- SERVICES=apigateway,lambda,dynamodb,s3,iam,sts,cloudformation
- DEBUG=${DEBUG- }
- DATA_DIR=${DATA_DIR- }
- LAMBDA_EXECUTOR=${LAMBDA_EXECUTOR- }
- LOCALSTACK_API_KEY=${LOCALSTACK_API_KEY- } # only required for Pro
- HOST_TMP_FOLDER=${TMPDIR:-/tmp/}localstack
- DOCKER_HOST=unix:///var/run/docker.sock
- LAMBDA_EXECUTOR=docker-reuse
volumes:
- "${TMPDIR:-/tmp}/localstack:/tmp/localstack"
- "/var/run/docker.sock:/var/run/docker.sock"
docker-compose.ymlの編集を保存して終了する。（escキーを押す。「:wq」を入力してエンターキーを押す。）
LocalStackを立ち上げる。[3]
hoge@AA:~/aws-sam$ docker-compose up -d
AWS SAMの設定
テンプレート作成
hoge@AA:~/aws-sam$ samlocal init
samlocal init選択とログ
Which template source would you like to use?
1 - AWS Quick Start Templates
2 - Custom Template Location
Choice: 1
What package type would you like to use?
1 - Zip (artifact is a zip uploaded to S3)
2 - Image (artifact is an image uploaded to an ECR image repository)
Package type: 1⏎
Which runtime would you like to use?
1 - nodejs14.x
2 - python3.9
3 - ruby2.7
4 - go1.x
5 - java11
6 - dotnetcore3.1
7 - nodejs12.x
8 - nodejs10.x
9 - python3.8
10 - python3.7
11 - python3.6
12 - python2.7
13 - ruby2.5
14 - java8.al2
15 - java8
16 - dotnetcore2.1
Runtime: 9⏎
Project name [sam-app]: python3_sam
Cloning from https://github.com/aws/aws-sam-cli-app-templates
AWS quick start application templates:
1 - Hello World Example
2 - EventBridge Hello World
3 - EventBridge App from scratch (100+ Event Schemas)
4 - Step Functions Sample App (Stock Trader)
5 - Elastic File System Sample App
Template selection: 1⏎
-----------------------
Generating application:
-----------------------
Name: python3_sam
Runtime: python3.8
Architectures: x86_64
Dependency Manager: pip
Application Template: hello-world
Output Directory: .
Next application steps can be found in the README file at ./python3_sam/README.md
Commands you can use next
=========================
[*] Create pipeline: cd python3_sam && sam pipeline init --bootstrap
[*] Test Function in the Cloud: sam sync --stack-name {stack-name} --watch
テンプレートのフォルダー構造[4]
hoge@AA:~/aws-sam/python3_sam$ tree
.
├── README.md
├── __init__.py
├── events
│ └── event.json
├── hello_world
│ ├── __init__.py
│ ├── app.py
│ └── requirements.txt
├── template.yaml
└── tests
├── __init__.py
├── integration
│ ├── __init__.py
│ └── test_api_gateway.py
├── requirements.txt
└── unit
├── __init__.py
└── test_handler.py
必要最小限のファイルだけ書き換える。テストなどは変更しない。
template.yaml
hello_world/app.pyを編集する。
hoge@AA:~/aws-sam/python3_sam$ vi template.yaml
template.yaml
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
python3_sam
Sample SAM Template for python3_sam
# More info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rst
Globals:
Function:
Timeout: 3
Resources:
UsersFunction:
Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
Properties:
CodeUri: hello_world/
Handler: app.getUsers
Runtime: python3.8
Architectures:
- x86_64
Events:
Users:
Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api
Properties:
Path: /users
Method: get
myDynamoDBTable:
Type: AWS::DynamoDB::Table
Properties:
TableName: myUserTable
AttributeDefinitions:
- AttributeName: Name
AttributeType: S
- AttributeName: Index
AttributeType: N
KeySchema:
- AttributeName: Name
KeyType: HASH
- AttributeName: Index
KeyType: RANGE
ProvisionedThroughput:
ReadCapacityUnits: 10
WriteCapacityUnits: 5
Outputs:
# ServerlessRestApi is an implicit API created out of Events key under Serverless::Function
# Find out more about other implicit resources you can reference within SAM
# https://github.com/awslabs/serverless-application-model/blob/master/docs/internals/generated_resources.rst#api
UsersApi:
Description: "API Gateway endpoint URL for Prod stage for Users function"
Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/users/"
UsersFunction:
Description: "Users Lambda Function ARN"
Value: !GetAtt UsersFunction.Arn
UsersFunctionIamRole:
Description: "Implicit IAM Role created for Users function"
Value: !GetAtt UsersFunctionRole.Arn
template.yamlの編集を保存して終了する。（escキーを押す。「:wq」を入力してエンターキーを押す。）
hello_world/app.py
hello_world/app.pyを編集する。
hoge@AA:~/aws-sam/python3_sam$ vi hello_world/app.py
hello_world/app.py
import os
import boto3
from boto3.session import Session
from datetime import datetime
session = Session(
aws_access_key_id='dummy',
aws_secret_access_key='dummy',
region_name='us-east-1'
)
if os.getenv('LOCALSTACK_HOSTNAME') is None:
endpoint = 'http://localhost:4566'
else:
endpoint=f"http://{os.environ['LOCALSTACK_HOSTNAME']}:4566"
dynamodb = session.resource(
service_name='dynamodb',
endpoint_url=endpoint
)
def getUsers(event, context):
table = dynamodb.Table('myUserTable')
data = []
response = table.scan()
while True:
data.extend(response["Items"])
if "LastEvaluatedKey" not in response:
break
response = table.scan(ExclusiveStartKey=response["LastEvaluatedKey"])
return {
'statusCode': 200,
'body':{
'users':data
}
}
hello_world/app.pyの編集を保存して終了する。（escキーを押す。「:wq」を入力してエンターキーを押す。）
ビルド
プロジェクトをビルドする。
hoge@AA:~/aws-sam/python3_sam$ samlocal build
Building codeuri: /home/hoge/aws-sam/python3_sam/hello_world runtime: python3.8 metadata: {} architecture: x86_64 functions: ['UsersFunction']
Running PythonPipBuilder:ResolveDependencies
Running PythonPipBuilder:CopySource
Build Succeeded
Built Artifacts : .aws-sam/build
Built Template : .aws-sam/build/template.yaml
Commands you can use next
=========================
[*] Invoke Function: sam local invoke
[*] Test Function in the Cloud: sam sync --stack-name {stack-name} --watch
[*] Deploy: sam deploy --guided
動作確認
デプロイ
hoge@AA:~/aws-sam/python3_sam$ samlocal deploy --guided --region us-east-1 --stack-name my-stack
samlocal deploy選択とログ
Configuring SAM deploy
======================
Looking for config file [samconfig.toml] : Not found
Setting default arguments for 'sam deploy'
=========================================
Stack Name [my-stack]:⏎
AWS Region [us-east-1]:⏎
#Shows you resources changes to be deployed and require a 'Y' to initiate deploy
Confirm changes before deploy [y/N]: y⏎
#SAM needs permission to be able to create roles to connect to the resources in your template
Allow SAM CLI IAM role creation [Y/n]:⏎
#Preserves the state of previously provisioned resources when an operation fails
Disable rollback [y/N]:⏎
UsersFunction may not have authorization defined, Is this okay? [y/N]: y⏎
Save arguments to configuration file [Y/n]:⏎
SAM configuration file [samconfig.toml]:⏎
SAM configuration environment [default]:⏎
Looking for resources needed for deployment:
Creating the required resources...
Successfully created!
Managed S3 bucket: aws-sam-cli-managed-default-samclisourcebucket-f892687c
A different default S3 bucket can be set in samconfig.toml
Saved arguments to config file
Running 'sam deploy' for future deployments will use the parameters saved above.
The above parameters can be changed by modifying samconfig.toml
Learn more about samconfig.toml syntax at
https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-config.html
Uploading to my-stack/a9e5d830bc93ddfdde1a54688915befb 451291 / 451291 (100.00%)
Deploying with following values
===============================
Stack name : my-stack
Region : us-east-1
Confirm changeset : True
Disable rollback : False
Deployment s3 bucket : aws-sam-cli-managed-default-samclisourcebucket-f892687c
Capabilities : ["CAPABILITY_IAM"]
Parameter overrides : {}
Signing Profiles : {}
Initiating deployment
=====================
Uploading to my-stack/d8b82bb43720dc66b29a5d306ce757f6.template 1509 / 1509 (100.00%)
Waiting for changeset to be created..
CloudFormation stack changeset
-----------------------------------------------------------------------------------------------------------------------------
Operation LogicalResourceId ResourceType Replacement
-----------------------------------------------------------------------------------------------------------------------------
+ Add myDynamoDBTable AWS::DynamoDB::Table False
+ Add UsersFunction AWS::Lambda::Function False
+ Add UsersFunctionRole AWS::IAM::Role False
+ Add UsersFunctionUsersPermissionP AWS::Lambda::Permission False
rod
+ Add ServerlessRestApi AWS::ApiGateway::RestApi False
+ Add ServerlessRestApiDeployment8a AWS::ApiGateway::Deployment False
5f04c8fd
+ Add ServerlessRestApiProdStage AWS::ApiGateway::Stage False
-----------------------------------------------------------------------------------------------------------------------------
Changeset created successfully. arn:aws:cloudformation:us-east-1:000000000000:changeSet/samcli-deploy1640162327/2229ff87
Previewing CloudFormation changeset before deployment
======================================================
Deploy this changeset? [y/N]: y⏎
2020-10-09 13:02:09 - Waiting for stack create/update to complete
CloudFormation events from stack operations
-----------------------------------------------------------------------------------------------------------------------------
ResourceStatus ResourceType LogicalResourceId ResourceStatusReason
-----------------------------------------------------------------------------------------------------------------------------
CREATE_COMPLETE AWS::CloudFormation::Stack myDynamoDBTable -
CREATE_COMPLETE AWS::CloudFormation::Stack ServerlessRestApi -
CREATE_COMPLETE AWS::CloudFormation::Stack UsersFunctionRole -
CREATE_COMPLETE AWS::CloudFormation::Stack ServerlessRestApiProdStage -
CREATE_COMPLETE AWS::CloudFormation::Stack ServerlessRestApiDeployment8a -
5f04c8fd
CREATE_COMPLETE AWS::CloudFormation::Stack UsersFunction -
CREATE_COMPLETE AWS::CloudFormation::Stack my-stack -
CREATE_COMPLETE AWS::CloudFormation::Stack UsersFunctionUsersPermissionP -
rod
-----------------------------------------------------------------------------------------------------------------------------
CloudFormation outputs from deployed stack
-------------------------------------------------------------------------------------------------------------------------------
Outputs
-------------------------------------------------------------------------------------------------------------------------------
Key UsersApi
Description API Gateway endpoint URL for Prod stage for Users function
Value https://XXXXXXXX.execute-api.amazonaws.com:4566/Prod/users/
Key UsersFunction
Description Users Lambda Function ARN
Value arn:aws:lambda:us-east-1:000000000000:function:my-stack-UsersFunction-8ab12ade
Key UsersFunctionIamRole
Description Implicit IAM Role created for Users function
Value arn:aws:iam::000000000000:role/my-stack-UsersFunctionRole-7105ca15
-------------------------------------------------------------------------------------------------------------------------------
Successfully created/updated stack - my-stack in us-east-1
Rust API IDをメモしておく。UsersApiのValue「XXXXXXXX」の部分。
-------------------------------------------------------------------------------------------------------------------------------
Outputs
-------------------------------------------------------------------------------------------------------------------------------
Key UsersApi
Description API Gateway endpoint URL for Prod stage for Users function
Value https://XXXXXXXX.execute-api.amazonaws.com:4566/Prod/users/
StageNameの確認する。「Stage」と「Prod」が作成される。
awslocal apigateway get-stages --rest-api-id XXXXXXXX
{
"item": [
{
"deploymentId": "YYYYYYYY",
"stageName": "Stage",
"description": "",
"cacheClusterEnabled": false,
"methodSettings": {},
"variables": {}
},
{
"deploymentId": "ZZZZZZZZ",
"stageName": "Prod",
"description": "",
"cacheClusterEnabled": false,
"methodSettings": {},
"variables": {},
"tags": {}
}
]
}
アイテムの追加
DynamoDBアイテム追加する。
hoge@AA:~/aws-sam/python3_sam$ awslocal dynamodb put-item --table-name myUserTable \
--item '{"Name":{"S": "Ishida Mio"}, "Index":{"N": "1000"}, "Gender":{"S": "Female"}, "Tel":{"S": "0769625106"}, "Mail":{"S": "mio1094@ccfrevk.ll"}, "Birthday":{"S": "1982/02/26"}}' \
--profile=localstack
hoge@AA:~/aws-sam/python3_sam$ awslocal dynamodb put-item --table-name myUserTable \
--item '{"Name":{"S": "Shimura Kokoro"}, "Index":{"N": "1001"}, "Gender":{"S": "Female"}, "Tel":{"S": "0829490871"}, "Mail":{"S": "kokoro414@bevhlk.xju"}, "Birthday":{"S": "1990/02/17"}}' \
--profile=localstack
hoge@AA:~/aws-sam/python3_sam$ awslocal dynamodb put-item --table-name myUserTable \
--item '{"Name":{"S": "Ishimura Masaki"}, "Index":{"N": "1002"}, "Gender":{"S": "Male"}, "Tel":{"S": "0848254480"}, "Mail":{"S": "masaki_ishimura@kokn.qgj"}, "Birthday":{"S": "1992/06/20"}}' \
--profile=localstack
hoge@AA:~/aws-sam/python3_sam$ awslocal dynamodb put-item --table-name myUserTable \
--item '{"Name":{"S": "Shirai Tomoko"}, "Index":{"N": "1003"}, "Gender":{"S": "Female"}, "Tel":{"S": "0742216492"}, "Mail":{"S": "tomoko225@vlyqs.pxt"}, "Birthday":{"S": "1982/09/29"}}' \
--profile=localstack
hoge@AA:~/aws-sam/python3_sam$ awslocal dynamodb put-item --table-name myUserTable \
--item '{"Name":{"S": "Yasui Tomoyuki"}, "Index":{"N": "1004"}, "Gender":{"S": "Male"}, "Tel":{"S": "095953680"}, "Mail":{"S": "tomoyuki772@jheauwpl.lyo"}, "Birthday":{"S": "1996/04/23"}}' \
--profile=localstack
API呼び出し
APIを呼び出してレスポンスを確認する。
APIはデプロイのRust API ID「XXXXXXXX」部分をを置き換える。「Stage」or「Prod」に置き換えて呼び出してください。
hoge@AA:~/aws-sam/python3_sam$ curl -X GET -s http://localhost:4566/restapis/XXXXXXXX/Prod/_user_request_/users | jq
{
"users": [
{
"Index": 1000,
"Tel": "0769625106",
"Birthday": "1982/02/26",
"Gender": "Female",
"Mail": "mio1094@ccfrevk.ll",
"Name": "Ishida Mio"
},
{
"Index": 1002,
"Tel": "0848254480",
"Birthday": "1992/06/20",
"Gender": "Male",
"Mail": "masaki_ishimura@kokn.qgj",
"Name": "Ishimura Masaki"
},
{
"Index": 1001,
"Tel": "0829490871",
"Birthday": "1990/02/17",
"Gender": "Female",
"Mail": "kokoro414@bevhlk.xju",
"Name": "Shimura Kokoro"
},
〜略〜
削除
samlocal delete
作成時の問題か設定の問題か不明だが、「EnableTerminationProtection」Keyが無いというエラーになる。
調べてみたが設定方法がわからなかった。
hoge@AA:~/aws-sam/python3_sam$ samlocal delete --stack-name my-stack --region us-east-1
Are you sure you want to delete the stack my-stack in the region us-east-1 ? [y/N]: y
Traceback (most recent call last):
〜中略〜
File "/home/hoge/.local/lib/python3.8/site-packages/samcli/lib/delete/cfn_utils.py", line 33, in has_stack
if stack["EnableTerminationProtection"]:
KeyError: 'EnableTerminationProtection'
Stack設定を確認すると確かに「EnableTerminationProtection」がセットされていない。
hoge@AA:~/aws-sam/python3_sam$ awslocal cloudformation describe-stacks --region us-east-1 --profile localstack --stack-name "my-stack" --output json
{
"Stacks": [
{
"StackId": "arn:aws:cloudformation:us-east-1:000000000000:stack/my-stack/dab3349f",
"StackName": "my-stack",
"ChangeSetId": "arn:aws:cloudformation:us-east-1:000000000000:changeSet/samcli-deploy1640163633/69053da1",
"Description": "Created by SAM CLI at 2020-10-09T13:02:09.145319 UTC",
"Parameters": [],
"CreationTime": "2020-10-09T13:02:09.483000+00:00",
"StackStatus": "CREATE_COMPLETE",
"StackStatusReason": "Deployment succeeded",
"Capabilities": [],
"Outputs": [
{
"OutputKey": "UsersApi",
"OutputValue": "https://XXXXXXXX.execute-api.amazonaws.com:4566/Prod/users/",
"Description": "API Gateway endpoint URL for Prod stage for Users function",
"ExportName": ""
},
{
"OutputKey": "UsersFunction",
"OutputValue": "arn:aws:lambda:us-east-1:000000000000:function:my-stack-UsersFunction-44c51310",
"Description": "Users Lambda Function ARN",
"ExportName": ""
},
{
"OutputKey": "UsersFunctionIamRole",
"OutputValue": "arn:aws:iam::000000000000:role/my-stack-UsersFunctionRole-adf023c3",
"Description": "Implicit IAM Role created for Users function",
"ExportName": ""
}
],
"Tags": []
}
]
}
awslocal cloudformation delete-stack
エラーも出ず削除されている？
hoge@AA:~/aws-sam/python3_sam$ awslocal cloudformation delete-stack --stack-name my-stack --region us-east-1
再デプロイするとStack作成時にエラーが出る。
何か削除手順が足りないようだ。
hoge@AA:~/aws-sam/python3_sam$ samlocal deploy --region us-east-1 --stack-name my-stack
〜略〜
CREATE_FAILED AWS::CloudFormation::Stack my-stack -
samlocal deploy全ログ
File with same data already exists at my-stack/a9e5d830bc93ddfdde1a54688915befb, skipping upload
Deploying with following values
===============================
Stack name : my-stack
Region : us-east-1
Confirm changeset : True
Disable rollback : True
Deployment s3 bucket : aws-sam-cli-managed-default-samclisourcebucket-6a41e267
Capabilities : ["CAPABILITY_IAM"]
Parameter overrides : {}
Signing Profiles : {}
Initiating deployment
=====================
File with same data already exists at my-stack/db0adca694251624519f17da6c16f33b.template, skipping upload
Waiting for changeset to be created..
CloudFormation stack changeset
-----------------------------------------------------------------------------------------------------------------------------
Operation LogicalResourceId ResourceType Replacement
-----------------------------------------------------------------------------------------------------------------------------
* Modify myDynamoDBTable AWS::DynamoDB::Table False
* Modify UsersFunction AWS::Lambda::Function False
* Modify UsersFunctionRole AWS::IAM::Role False
* Modify UsersFunctionUsersPermissionP AWS::Lambda::Permission False
rod
* Modify ServerlessRestApi AWS::ApiGateway::RestApi False
* Modify ServerlessRestApiDeployment8a AWS::ApiGateway::Deployment False
5f04c8fd
* Modify ServerlessRestApiProdStage AWS::ApiGateway::Stage False
-----------------------------------------------------------------------------------------------------------------------------
Changeset created successfully. arn:aws:cloudformation:us-east-1:000000000000:changeSet/samcli-deploy1640164603/32d12aff
Previewing CloudFormation changeset before deployment
======================================================
Deploy this changeset? [y/N]: y
2020-10-10 19:32:46 - Waiting for stack create/update to complete
CloudFormation events from stack operations
-----------------------------------------------------------------------------------------------------------------------------
ResourceStatus ResourceType LogicalResourceId ResourceStatusReason
-----------------------------------------------------------------------------------------------------------------------------
CREATE_COMPLETE AWS::CloudFormation::Stack UsersFunctionRole -
CREATE_COMPLETE AWS::CloudFormation::Stack myDynamoDBTable -
CREATE_COMPLETE AWS::CloudFormation::Stack ServerlessRestApi -
UPDATE_COMPLETE AWS::CloudFormation::Stack UsersFunctionUsersPermissionP -
rod
CREATE_FAILED AWS::CloudFormation::Stack my-stack -
-----------------------------------------------------------------------------------------------------------------------------
参考にしたサイト
- AWS Serverless Application Model (AWS SAM) とは - AWS Serverless Application Model
- LocalStack AWS SAM | Docs
- LocalStackの提供するAWS SAMを使って、LocalStackにAmazon API Gateway＋AWS Lambdaをデプロイしてみる - CLOVER🍀
- AWS SAM + LocalStack によるサンプルアプリケーションの作成 - arailly books
- AWS SAM CLI と localstack を利用して Lambda をローカル実行してみよう
- AWS SAM + DockerでAWSサーバーレス環境をローカル構築する | 株式会社CyberOwl
- AWSCLIでCloudFormationのOutputsから値取得 - Qiita
Discussion