Zenn
Closed10

buildders.flash 「AWS Step Functions を使って時間ぴったりに処理を行うサーバーレススケジューラーを構築する」をやってみる

AWS
Step Functions
kenryokenryo

https://github.com/serverless-operations/serverless-dynamic-scheduler

アプリはCDKで管理されている。
npmをインストール（今回はv14.18.0を使用してみる）し、CDK（2.19.0）をインストールした。

satoukensuke@MacBook-Pro serverless-dynamic-scheduler-main % nvm use v14.18.0    
Now using node v14.18.0 (npm v6.14.15)
satoukensuke@MacBook-Pro serverless-dynamic-scheduler-main % npm install -g aws-cdk
/Users/satoukensuke/.nvm/versions/node/v14.18.0/bin/cdk -> /Users/satoukensuke/.nvm/versions/node/v14.18.0/lib/node_modules/aws-cdk/bin/cdk
+ aws-cdk@2.19.0
added 2 packages from 6 contributors in 2.019s
satoukensuke@MacBook-Pro serverless-dynamic-scheduler-main % cdk --version
2.19.0 (build e0d3e62)
kenryokenryo

東京リージョンにブートストラップする。

satoukensuke@MacBook-Pro .aws % aws sts get-caller-identity
{
    "UserId": "XXX",
    "Account": "XXX",
    "Arn": "arn:aws:iam::XXX:user/KensukeSatou"
}
satoukensuke@MacBook-Pro .aws % cdk bootstrap aws://XXX/ap-northeast-1
 ⏳  Bootstrapping environment aws://XXX/ap-northeast-1...
Trusted accounts for deployment: (none)
Trusted accounts for lookup: (none)
Using default execution policy of 'arn:aws:iam::aws:policy/AdministratorAccess'. Pass '--cloudformation-execution-policies' to customize.
CDKToolkit: creating CloudFormation changeset...


 ✅  Environment aws://XXX/ap-northeast-1 bootstrapped.
kenryokenryo

フォルダ直下でnpm installし、必要なモジュールをインストールしておく。

satoukensuke@MacBook-Pro serverless-dynamic-scheduler-main % npm install
npm WARN read-shrinkwrap This version of npm is compatible with lockfileVersion@1, but package-lock.json was generated for lockfileVersion@2. I'll try to do my best with it!
npm WARN step-functions-scheduler@0.1.0 No repository field.
npm WARN step-functions-scheduler@0.1.0 No license field.

added 891 packages from 592 contributors and audited 894 packages in 37.473s

38 packages are looking for funding
  run `npm fund` for details

found 11 critical severity vulnerabilities
  run `npm audit fix` to fix them, or `npm audit` for details
kenryokenryo

cdk deployでエラー。INVOKER_ALPHA_WEBHOOK_URLに値を設定する必要あり、READMEをよく読めという話だが、.envファイルを作成する。

satoukensuke@MacBook-Pro serverless-dynamic-scheduler-main % cdk deploy
Environment variable INVOKER_ALPHA_WEBHOOK_URL is not set.

Subprocess exited with error 1

なお、値となるWebhookURLの取得方法は以下を参考に取得。
Slackにもチャンネル作っておく必要あり。

https://zenn.dev/hotaka_noda/articles/4a6f0ccee73a18

kenryokenryo

.envに値を設定し、cdk deployを再実施、エラー。
docker起動してなくちゃいけないみたい。

satoukensuke@MacBook-Pro serverless-dynamic-scheduler-main % cdk deploy
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
(node:76138) UnhandledPromiseRejectionWarning: Error: docker exited with status 1
    at dockerExec (/Users/satoukensuke/handson/serverless-dynamic-scheduler-main/node_modules/aws-cdk-lib/core/lib/bundling.ts:311:11)
    at Function.fromBuild (/Users/satoukensuke/handson/serverless-dynamic-scheduler-main/node_modules/aws-cdk-lib/core/lib/bundling.ts:160:5)
    at new Bundling (/Users/satoukensuke/handson/serverless-dynamic-scheduler-main/node_modules/aws-cdk-lib/aws-lambda-nodejs/lib/bundling.ts:122:74)
    at Function.bundle (/Users/satoukensuke/handson/serverless-dynamic-scheduler-main/node_modules/aws-cdk-lib/aws-lambda-nodejs/lib/bundling.ts:59:17)
    at new NodejsFunction (/Users/satoukensuke/handson/serverless-dynamic-scheduler-main/node_modules/aws-cdk-lib/aws-lambda-nodejs/lib/function.ts:53:22)
    at new TriggerStandbyEventsStack (/Users/satoukensuke/handson/serverless-dynamic-scheduler-main/deploy/scheduler-events-stack.ts:41:36)
    at createApp (/Users/satoukensuke/handson/serverless-dynamic-scheduler-main/deploy/cdk-deploy.ts:48:3)
    at Object.<anonymous> (/Users/satoukensuke/handson/serverless-dynamic-scheduler-main/deploy/cdk-deploy.ts:79:1)
    at Module._compile (internal/modules/cjs/loader.js:1085:14)
    at Module.m._compile (/Users/satoukensuke/handson/serverless-dynamic-scheduler-main/node_modules/ts-node/src/index.ts:1056:23)
(Use `node --trace-warnings ...` to show where the warning was created)
(node:76138) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:76138) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Since this app includes more than a single stack, specify which stacks to use (wildcards are supported) or specify `--all`
Stacks: org-scheduler-dev-SchedulerDynamoStack · org-scheduler-dev-TriggerStandbyEventsStack
kenryokenryo

コマンド実行に成功。（cdk deploy --all ではなく、cdk deployでの結果です）

.
.
.
 => CACHED [7/9] RUN mkdir /tmp/yarn-cache &&     chmod -R 777 /tmp/yarn-cache &&     yarn config  0.0s
 => CACHED [8/9] RUN npm config --global set update-notifier false                                 0.0s
 => CACHED [9/9] RUN /sbin/useradd -u 1000 user && chmod 711 /                                     0.0s
 => exporting to image                                                                             0.0s
 => => exporting layers                                                                            0.0s
 => => writing image sha256:75408de6075fac460d05d5544240cfd09cf6f5c6d63179580d67455b731782b2       0.0s
 => => naming to docker.io/library/cdk-53fef9ced5a08335e81a426e8d6b14ac5a54fc9c4af4c97ca198768db2  0.0s

Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them
Bundling asset org-scheduler-dev-SchedulerAppStack/InvokerBeta/Code/Stage...
esbuild cannot run locally. Switching to Docker bundling.
WARNING: The requested image's platform (linux/arm64) does not match the detected host platform (linux/amd64) and no specific platform was requested

  asset-output/index.js      7.7kb
  asset-output/index.js.map  8.4kb

⚡ Done in 321ms

Since this app includes more than a single stack, specify which stacks to use (wildcards are supported) or specify `--all`
Stacks: org-scheduler-dev-SchedulerDynamoStack · org-scheduler-dev-SchedulerAppStack · org-scheduler-dev-TriggerStandbyEventsStack
kenryokenryo

cdk deploy --allを実行する必要があった。

✨  Deployment time: 171.83s

Stack ARN:
arn:aws:cloudformation:ap-northeast-1:XXX:stack/org-scheduler-dev-TriggerStandbyEventsStack/ab7d4f60-b5bf-11ec-ba05-06562795e5cd

✨  Total time: 199.71s


satoukensuke@MacBook-Pro serverless-dynamic-scheduler-main %

orgから始まるスタックが3つ作成された。

kenryokenryo

curlコマンド実行。

curl --location --request POST 'https://XXX.execute-api.ap-northeast-1.amazonaws.com/messages' --header 'Content-Type: application/json' --data-raw '{
  "publishTime": 1649430600,
  "channel": "invoker-alpha",
	"parameters": {
		"message": "Invocation test from invoker-alpha"
	}
}'

レコードが追加される。

DynamoDB Streamsに指定したLambdaが実行される。

StepFunctionsはステータスWaitStateになり、指定時間まで待機する。

時間になるとSlackにメッセージを投稿するLambdaが実行される。

kenryokenryo

ブログ記事にもあるように EventBridgeでの実行は数分遅れが発生する可能性があるし、Lambdaだと複数実行される場合がある。
StepFunctionsを使用することでこれらを解決している。

このスクラップは2022/04/09にクローズされました