GitHub Actionsでsam deployしたLambdaが動かない問題を解決する
- GitHub ActionsでSAMアプリケーションをデプロイする
- LambdaのランタイムはGo(provided.al2)
デプロイは正常に終了したが、Lambdaを実行したら Runtime.ExitError
INIT_START Runtime Version: provided:al2.v28 Runtime Version ARN: arn:aws:lambda:ap-northeast-1::runtime:8bdc1e5642bbcd08528c3948a723b8901ec98ab986e6bab636fae24882de1d6c
/var/task/bootstrap: /lib64/libc.so.6: version `GLIBC_2.32' not found (required by /var/task/bootstrap)
/var/task/bootstrap: /lib64/libc.so.6: version `GLIBC_2.34' not found (required by /var/task/bootstrap)
INIT_REPORT Init Duration: 8.38 ms Phase: init Status: error Error Type: Runtime.ExitError
/var/task/bootstrap: /lib64/libc.so.6: version `GLIBC_2.32' not found (required by /var/task/bootstrap)
/var/task/bootstrap: /lib64/libc.so.6: version `GLIBC_2.34' not found (required by /var/task/bootstrap)
INIT_REPORT Init Duration: 91.54 ms Phase: invoke Status: error Error Type: Runtime.ExitError
START RequestId: 14aca2ee-d2d5-4b42-b89e-11521998d45f Version: $LATEST
RequestId: 14aca2ee-d2d5-4b42-b89e-11521998d45f Error: Runtime exited with error: exit status 1
Runtime.ExitError
END RequestId: 14aca2ee-d2d5-4b42-b89e-11521998d45f
REPORT RequestId: 14aca2ee-d2d5-4b42-b89e-11521998d45f Duration: 112.10 ms Billed Duration: 113 ms Memory Size: 128 MB Max Memory Used: 3 MB
ローカル(WSL, ubuntu20.04)でビルド・デプロイする時は問題ないので、環境差分(というかActionsのRunnerの問題)だと推測
ビルドする時に --use-container
付ければ良いのでは?と思って付けてみる
→ ビルドが失敗
aws_lambda_builders.exceptions.WorkflowFailedError: GoModulesBuilder:Resolver - Path resolution for runtime: provided of binary: go was not successful
GitHub Actionのログ(debug含んで長いので折り畳み)
sam build --use-container
shell: /usr/bin/bash -e {0}
env:
AWS_DEFAULT_REGION: ***
AWS_REGION: ***
AWS_ACCESS_KEY_ID: ***
AWS_SECRET_ACCESS_KEY: ***
AWS_SESSION_TOKEN: ***
SAM CLI now collects telemetry to better understand customer needs.
You can OPT OUT and disable telemetry collection by setting the
environment variable SAM_CLI_TELEMETRY=0 in your shell.
Thanks for your help!
Learn More: https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-telemetry.html
2023-12-21 17:10:41,363 | Using config file: samconfig.toml, config environment: default
2023-12-21 17:10:41,363 | Expand command line arguments to:
2023-12-21 17:10:41,363 | --template_file=/home/runner/work/***Api/***Api/template.yaml --use_container --mount_with=READ --build_dir=.aws-sam/build --cache_dir=.aws-sam/cache
2023-12-21 17:10:41,399 | 'build' command is called
2023-12-21 17:10:41,399 | Starting Build inside a container
2023-12-21 17:10:41,409 | Collected default values for parameters: {}
2023-12-21 17:10:41,434 | There is no customer defined id or cdk path defined for resource RestApi, so we will use the resource logical id as the resource id
2023-12-21 17:10:41,434 | There is no customer defined id or cdk path defined for resource Function, so we will use the resource logical id as the resource id
2023-12-21 17:10:41,434 | There is no customer defined id or cdk path defined for resource FunctionLogGroup, so we will use the resource logical id as the resource id
2023-12-21 17:10:41,482 | Unable to resolve property Value: {'Fn::GetAtt': ['Function', 'Arn']}. Leaving as is.
2023-12-21 17:10:41,483 | 0 stacks found in the template
2023-12-21 17:10:41,483 | Collected default values for parameters: {}
2023-12-21 17:10:41,503 | There is no customer defined id or cdk path defined for resource RestApi, so we will use the resource logical id as the resource id
2023-12-21 17:10:41,503 | There is no customer defined id or cdk path defined for resource Function, so we will use the resource logical id as the resource id
2023-12-21 17:10:41,503 | There is no customer defined id or cdk path defined for resource FunctionLogGroup, so we will use the resource logical id as the resource id
2023-12-21 17:10:41,549 | Unable to resolve property Value: {'Fn::GetAtt': ['Function', 'Arn']}. Leaving as is.
2023-12-21 17:10:41,549 | 3 resources found in the stack
2023-12-21 17:10:41,549 | Found Serverless function with name='Function' and CodeUri='cmd/lambda/'
2023-12-21 17:10:41,549 | --base-dir is not presented, adjusting uri cmd/lambda/ relative to /home/runner/work/***Api/***Api/template.yaml
2023-12-21 17:10:41,560 | 3 resources found in the stack
2023-12-21 17:10:41,560 | Found Serverless function with name='Function' and CodeUri='cmd/lambda/'
2023-12-21 17:10:41,561 | Instantiating build definitions
2023-12-21 17:10:41,561 | No previous build graph found, generating new one
2023-12-21 17:10:41,561 | Unique function build definition found, adding as new (Function Build Definition: BuildDefinition(provided.al2, /home/runner/work/***Api/***Api/cmd/lambda, Zip, , 12e086fe-aff4-4b7d-8336-09422fb0a8ba, {'BuildMethod': 'go1.x'}, {}, x86_64, []), Function: Function(function_id='Function', name='Function', functionname='Env-***-api', runtime='provided.al2', memory=128, timeout=30, handler='bootstrap', imageuri=None, packagetype='Zip', imageconfig=None, codeuri='/home/runner/work/***Api/***Api/cmd/lambda', environment={'Variables': {'***_TABLE_NAME': 'Env-***', 'COIN_API_URL': 'CoinApiUrl'}}, rolearn=None, layers=[], events={'Post***s': {'Type': 'Api', 'Properties': {'Path': '/v1/***s', 'Method': 'post', 'RestApiId': 'RestApi'}}, 'Get***s': {'Type': 'Api', 'Properties': {'Path': '/v1/***s/{proxy+}', 'Method': 'get', 'RestApiId': 'RestApi'}}, 'Patch***': {'Type': 'Api', 'Properties': {'Path': '/v1/***s/{proxy+}', 'Method': 'patch', 'RestApiId': 'RestApi'}}, 'Delete***': {'Type': 'Api', 'Properties': {'Path': '/v1/***s/{proxy+}', 'Method': 'delete', 'RestApiId': 'RestApi'}}, 'Post***Tickets': {'Type': 'Api', 'Properties': {'Path': '/v1/***s/{proxy+}/tickets', 'Method': 'post', 'RestApiId': 'RestApi'}}, 'Post***Results': {'Type': 'Api', 'Properties': {'Path': '/v1/***s/{proxy+}/results', 'Method': 'post', 'RestApiId': 'RestApi'}}, 'Get***Odds': {'Type': 'Api', 'Properties': {'Path': '/v1/***s/{proxy+}/odds/{proxy+}', 'Method': 'get', 'RestApiId': 'RestApi'}}}, metadata={'BuildMethod': 'go1.x', 'SamResourceId': 'Function'}, inlinecode=None, codesign_config_arn=None, architectures=['x86_64'], function_url_config=None, function_build_info=<FunctionBuildInfo.BuildableZip: ('BuildableZip', 'Regular ZIP function which can be build with SAM CLI')>, stack_path='', runtime_management_config=None, logging_config=None))
2023-12-21 17:10:41,563 | Building codeuri: /home/runner/work/***Api/***Api/cmd/lambda runtime: provided.al2 metadata: {'BuildMethod': 'go1.x'} architecture: x86_64 functions: Function
2023-12-21 17:10:41,563 | Building to following folder /home/runner/work/***Api/***Api/.aws-sam/build/Function
2023-12-21 17:10:41,573 | Checking free port on 127.0.0.1:5723
Fetching public.ecr.aws/sam/build-provided.al2:latest-x86_64 Docker container image...................................................................................................................................................................................................................................................................................................................................
2023-12-21 17:11:17,889 | Mounting /home/runner/work/***Api/***Api/cmd/lambda as /tmp/samcli/source:ro,delegated, inside runtime container
Using the request object from command line argument
Loading workflow module 'aws_lambda_builders.workflows'
Registering workflow 'CustomMakeBuilder' with capability 'Capability(language='provided', dependency_manager=None, application_framework=None)'
Registering workflow 'DotnetCliPackageBuilder' with capability 'Capability(language='dotnet', dependency_manager='cli-package', application_framework=None)'
Registering workflow 'GoModulesBuilder' with capability 'Capability(language='go', dependency_manager='modules', application_framework=None)'
Registering workflow 'JavaGradleWorkflow' with capability 'Capability(language='java', dependency_manager='gradle', application_framework=None)'
Registering workflow 'JavaMavenWorkflow' with capability 'Capability(language='java', dependency_manager='maven', application_framework=None)'
Registering workflow 'NodejsNpmBuilder' with capability 'Capability(language='nodejs', dependency_manager='npm', application_framework=None)'
Registering workflow 'NodejsNpmEsbuildBuilder' with capability 'Capability(language='nodejs', dependency_manager='npm-esbuild', application_framework=None)'
Registering workflow 'PythonPipBuilder' with capability 'Capability(language='python', dependency_manager='pip', application_framework=None)'
Registering workflow 'RubyBundlerBuilder' with capability 'Capability(language='ruby', dependency_manager='bundler', application_framework=None)'
Registering workflow 'RustCargoLambdaBuilder' with capability 'Capability(language='rust', dependency_manager='cargo', application_framework=None)'
Found workflow 'GoModulesBuilder' to support capabilities 'Capability(language='go', dependency_manager='modules', application_framework=None)'
Workflow GoModulesBuilder does not support value "False" for building in source. Using default value "True".
Builder workflow failed
Traceback (most recent call last):
File "/usr/local/opt/sam-cli/lib64/python3.8/site-packages/aws_lambda_builders/workflow.py", line 77, in wrapper
binary_checker.resolver.exec_paths
File "/usr/local/opt/sam-cli/lib64/python3.8/site-packages/aws_lambda_builders/path_resolver.py", line 33, in exec_paths
return self._which()
File "/usr/local/opt/sam-cli/lib64/python3.8/site-packages/aws_lambda_builders/path_resolver.py", line 26, in _which
raise ValueError(
ValueError: Path resolution for runtime: provided of binary: go was not successful
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/opt/sam-cli/lib64/python3.8/site-packages/aws_lambda_builders/__main__.py", line 117, in main
builder.build(
File "/usr/local/opt/sam-cli/lib64/python3.8/site-packages/aws_lambda_builders/builder.py", line 170, in build
return workflow.run()
File "/usr/local/opt/sam-cli/lib64/python3.8/site-packages/aws_lambda_builders/workflow.py", line 82, in wrapper
raise WorkflowFailedError(workflow_name=self.NAME, action_name="Resolver", reason=str(ex))
aws_lambda_builders.exceptions.WorkflowFailedError: GoModulesBuilder:Resolver - Path resolution for runtime: provided of binary: go was not successful
2023-12-21 17:11:19,082 | Build inside container returned response {"jsonrpc": "2.0", "id": 1, "error": {"code": 400, "message": "GoModulesBuilder:Resolver - Path resolution for runtime: provided of binary: go was not successful"}}
2023-12-21 17:11:19,230 | Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics
Build Failed
2023-12-21 17:11:19,238 | Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics
2023-12-21 17:11:19,238 | Unable to find Click Context for getting session_id.
2023-12-21 17:11:19,238 | Sending Telemetry: {'metrics': [{'events': {'requestId': 'c751d61a-364f-4465-9dbf-6538dac3379e', 'installationId': '8601e8e3-ee62-46a6-9587-a89c584c916f', 'sessionId': '5dcf3c68-54bf-4208-b669-d154082f349a', 'executionEnvironment': 'GitHubAction', 'ci': True, 'pyversion': '3.10.12', 'samcliVersion': '1.105.0', 'commandName': 'sam build', 'metricSpecificAttributes': {'events': [{'event_name': 'SamConfigFileExtension', 'event_value': '.toml', 'thread_id': '5c156de52fe94def8d18e8def1ae7267', 'time_stamp': '2023-12-21 17:10:41.327', 'exception_name': None}, {'event_name': 'SamConfigFileExtension', 'event_value': '.toml', 'thread_id': 'b2f6ebcf5af8499d8db5cc851691e95b', 'time_stamp': '2023-12-21 17:10:41.363', 'exception_name': None}, {'event_name': 'BuildWorkflowUsed', 'event_value': 'go-modules', 'thread_id': '43d531dc96534b47884ef5e767cdda56', 'time_stamp': '2023-12-21 17:10:41.560', 'exception_name': None}, {'event_name': 'BuildFunctionRuntime', 'event_value': 'provided.al2', 'thread_id': 'da98c071c50d412685101efa035dde55', 'time_stamp': '2023-12-21 17:10:41.561', 'exception_name': None}, {'event_name': 'BuildWorkflowUsed', 'event_value': 'go-modules', 'thread_id': '7b30bfc15c844bb2a307f427d8491381', 'time_stamp': '2023-12-21 17:10:41.563', 'exception_name': None}]}}}]}
2023-12-21 17:11:19,239 | Sending Telemetry: {'metrics': [{'commandRun': {'requestId': 'b582a011-8e46-4dd9-b375-bbc8d45e9805', 'installationId': '8601e8e3-ee62-46a6-9587-a89c584c916f', 'sessionId': '5dcf3c68-54bf-4208-b669-d154082f349a', 'executionEnvironment': 'GitHubAction', 'ci': True, 'pyversion': '3.10.12', 'samcliVersion': '1.105.0', 'awsProfileProvided': False, 'debugFlagProvided': True, 'region': '', 'commandName': 'sam build', 'metricSpecificAttributes': {'projectType': 'CFN', 'gitOrigin': None, 'projectName': '27caac2342bb08ffd25c9e2e1983a86f99dbf85d806ec2dc63e02d8e560d9fa6', 'initialCommit': None}, 'duration': 37866, 'exitReason': 'BuildInsideContainerError', 'exitCode': 1}}]}
2023-12-21 17:11:19,785 | HTTPSConnectionPool(host='aws-serverless-tools-telemetry.us-west-2.amazonaws.com', port=443): Read timed out. (read timeout=0.1)
Error: GoModulesBuilder:Resolver - Path resolution for runtime: provided of binary: go was not successful
2023-12-21 17:11:19,788 | HTTPSConnectionPool(host='aws-serverless-tools-telemetry.us-west-2.amazonaws.com', port=443): Read timed out. (read timeout=0.1)
Error: Process completed with exit code 1.
--use-container
を付けた時、ランタイムが provided.al2
であれば sam/build-provided.al2
のイメージから起動するコンテナ内でビルドが実行されるが、このイメージにはgoのバイナリが含まれていないため、Goのビルドができない(!?!?!?)
このイメージにGo入れろやのPRもあったが蹴られている
特定の言語を考慮したものはカスタムランタイムのイメージに入れません、というスタンスは理解はできる
This doesn't look right to me. We don't install any other runtimes, so I'm not sure we should do this for Go.
I think there might be a confusion on building Go with the provided runtime, but I'll get back to you after chatting with the team.
さっきのissueで専用のオレオレビルドイメージ作るやで!って書いてあったので、真似してみた
sam build \
--use-container \
--build-image ghcr.io/miztch/sam-build-custom-image:v0.0.1-go1.21.5
ビルドしようとしたので前進はしたが、プロジェクトのディレクトリ構成の問題で蹴られる
(ただローカルでビルドすれば通るので……でわけわからなくなった)
Fetching ghcr.io/miztch/sam-build-custom-image:v0.0.1-go1.21.5 Docker container image.....................................................................................................................................................................................................................................................................................................................................................................................
2023-12-22 17:21:21,594 | Mounting /home/runner/work/***Api/***Api/cmd/lambda as /tmp/samcli/source:ro,delegated, inside runtime container
Using the request object from command line argument
Loading workflow module 'aws_lambda_builders.workflows'
Registering workflow 'CustomMakeBuilder' with capability 'Capability(language='provided', dependency_manager=None, application_framework=None)'
Registering workflow 'DotnetCliPackageBuilder' with capability 'Capability(language='dotnet', dependency_manager='cli-package', application_framework=None)'
Registering workflow 'GoModulesBuilder' with capability 'Capability(language='go', dependency_manager='modules', application_framework=None)'
Registering workflow 'JavaGradleWorkflow' with capability 'Capability(language='java', dependency_manager='gradle', application_framework=None)'
Registering workflow 'JavaMavenWorkflow' with capability 'Capability(language='java', dependency_manager='maven', application_framework=None)'
Registering workflow 'NodejsNpmBuilder' with capability 'Capability(language='nodejs', dependency_manager='npm', application_framework=None)'
Registering workflow 'NodejsNpmEsbuildBuilder' with capability 'Capability(language='nodejs', dependency_manager='npm-esbuild', application_framework=None)'
Registering workflow 'PythonPipBuilder' with capability 'Capability(language='python', dependency_manager='pip', application_framework=None)'
Registering workflow 'RubyBundlerBuilder' with capability 'Capability(language='ruby', dependency_manager='bundler', application_framework=None)'
Registering workflow 'RustCargoLambdaBuilder' with capability 'Capability(language='rust', dependency_manager='cargo', application_framework=None)'
Found workflow 'GoModulesBuilder' to support capabilities 'Capability(language='go', dependency_manager='modules', application_framework=None)'
Workflow GoModulesBuilder does not support value "False" for building in source. Using default value "True".
Running workflow 'GoModulesBuilder'
Running GoModulesBuilder:Build
go: go.mod file not found in current directory or any parent directory; see 'go help modules'
Go files not found. Attempting to build for Go files in a different directory
Go files not found at CodeUri /tmp/samcli/source . Descending into sub-directories to find the handler: /tmp/samcli/source/bootstrap
GoModulesBuilder:Build failed
Traceback (most recent call last):
File "/usr/local/opt/sam-cli/lib64/python3.8/site-packages/aws_lambda_builders/workflows/go_modules/actions.py", line 22, in execute
self.builder.build(self.source_dir, self.output_path)
File "/usr/local/opt/sam-cli/lib64/python3.8/site-packages/aws_lambda_builders/workflows/go_modules/builder.py", line 77, in build
raise BuilderError(message=p_err.decode("utf8").strip())
aws_lambda_builders.workflows.go_modules.builder.BuilderError: Builder Failed: go: go.mod file not found in current directory or any parent directory; see 'go help modules'
ここで当初のエラーに立ち返る
エラーメッセージから上記がヒット
go build するマシンの glibc のバージョンが go build した Go バイナリを動かすマシンの glibc のバージョンより新しいとこの事象が発生するようなので go build するマシンの glibc のバージョンを下げるか go build した Go バイナリを動かすマシンの glibc のバージョンを上げるかしたら事象は解消しそう。
glibcのバージョン調べるのめんどくないか?俺はLambda動かしたいだけだが
ひとまず調べた
GitHub Actions
- 2023/12現在選べるLinuxのRunnerは、
ubuntu-20.04
orubuntu-22.04
上記から引用
OS | バージョン | glibcのバージョン |
---|---|---|
Ubuntu | 20.04 | 2.31 |
Ubuntu | 22.04 | 2.35 |
Amazon Linux 2
リリースノートで確認できるアップデートを見るとglibcは2.26らしい
(Amazon Linux 2.0.20211103.0)
glibc-2.26-56.amzn2.aarch64
glibc-2.26-56.amzn2.x86_64
Amazon Linux 2023
glibcは2.34
GitHub Actionsのworkflowでは ubuntu-latest
を利用していたので、glibcのバージョンが新しすぎたということになる…
ubuntu-20.04
と provided.al2023
の組み合わせであればこれをクリアできそうなので、
-
sam build
をランナー上で実行 - ランナーのイメージは
ubuntu-20.04
- Lambdaのランタイムは
provided.al2023
でうまくいく気がする
ubuntu-20.04 と provided.al2023 の組み合わせ
でデプロイ成功し、Lambdaも動いた
ランタイムが provided.al2 でもなぜか動いている……?一旦解決したのでヨシ