Zenn
Closed11

AWS CloudTrail のログを Slack に通知したい

AWS
Slack
CloudTrail
wintwint

モチベ

AWS 公式の自動構築テンプレで CloudTrail の logs の metrics filter は作れるけど、ログの有無しか分からないのが足りない。 ChatBot で簡単に通知できるけど、ログごと送信したい。

参考

https://docs.aws.amazon.com/awscloudtrail/latest/userguide/use-cloudformation-template-to-create-cloudwatch-alarms.html

これは既存のアラートのIaC

実装

Metric
CloudTrailMetrics / AuthorizationFailureCount
Alarms
CloudTrailAuthorizationFailures

{ ($.errorCode = "*UnauthorizedOperation") || ($.errorCode = "AccessDenied*") }
wintwint

構想

CloudTrail → CloudWatch Logs → filter → AWS Lambda → Slack

以下 参考資料

https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/SubscriptionFilters.html

Logs … are Base64 encoded and compressed with the gzip format.

https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/SubscriptionFilters.html#LambdaFunctionExample

Subscription filters with AWS Lambda

https://qiita.com/miyaz/items/ed2e0f30d1cc7b96d7c2

https://masakimisawa.com/cwl-transfer-slack/

https://htnosm.hatenablog.com/entry/2016/05/05/090000

https://developer.dip-net.co.jp/entry/2020/12/02/CloudWatchLogsからLambda経由でログメッセージを通知する

https://dev.classmethod.jp/articles/notify-lambda-error-and-warning-log-to-slack/

https://serverlessrepo.aws.amazon.com/applications/arn:aws:serverlessrepo:us-east-1:289559741701:applications~cw-logs-to-slack

cw-logs-to-slack

wintwint

入力形式

再掲

https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/SubscriptionFilters.html

https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/SubscriptionFilters.html#LambdaFunctionExample

{ "awslogs": { "data": "BASE64ENCODED_GZIP_COMPRESSED_DATA" } }

{
  type: "object",
  properties: {
    awslogs: {
      type: "object",
      properties: {
        data: {
          type: "string",
          contentEncoding: "base64",
          contentMediaType: "application/gzip",
          description: "Base64 encoded GZIP compressed JSON",
        },
      },
      required: ["data"],
    },
  },
  required: ["awslogs"],
}

参考

https://www.serverless.com/framework/docs/providers/aws/events/apigateway/#request-schema-validators

Currently, API Gateway supports JSON Schema draft-04.

https://json-schema.org/understanding-json-schema/index.html

https://json-schema.org/understanding-json-schema/reference/non_json_data.html

New in draft 7

API GW には使えなさそう

https://developer.mozilla.org/ja/docs/Web/HTTP/Basics_of_HTTP/MIME_types/Common_types

https://nodejs.org/api/buffer.html

https://nodejs.org/api/zlib.html

wintwint

Slack

SDK

https://slack.dev/node-slack-sdk/

Node Slack SDK

https://slack.dev/node-slack-sdk/webhook

Node Slack SDK / Incoming Webhooks

https://www.npmjs.com/package/@slack/webhook

manifest

_metadata:
  major_version: 1
  minor_version: 1
display_information:
  name: AWS Log Alert
features:
  app_home:
    home_tab_enabled: false
    messages_tab_enabled: true
    messages_tab_read_only_enabled: true
  bot_user:
    display_name: AWS Log Alert
    always_online: false
oauth_config:
  scopes:
    bot:
      - incoming-webhook
settings:
  org_deploy_enabled: false
  socket_mode_enabled: false
  is_hosted: false
wintwint

JSON schema of the persed log

https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/SubscriptionFilters.html#LambdaFunctionExample

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "oneOf": [
    {
      "type": "object",
      "properties": {
        "owner": {
          "type": "string",
          "pattern": "^\\d{12}$"
        },
        "logGroup": {
          "type": "string",
          "pattern": "^[\\w/.#-]{1,512}$"
        },
        "logStream": {
          "type": "string",
          "pattern": "^[^:*]{1,512}$"
        },
        "subscriptionFilters": {
          "type": "array",
          "items": {
            "type": "string",
            "pattern": "^[^:*]{1,512}$"
          }
        },
        "messageType": {
          "const": "DATA_MESSAGE"
        },
        "logEvents": {
          "type": "array",
          "items": {
            "type": "object",
            "properties": {
              "id": {
                "type": "string"
              },
              "timestamp": {
                "type": "number",
                "minimum": 0
              },
              "message": {
                "type": "string",
                "description": "escaped JSON string"
              }
            }
          }
        }
      }
    },
    {
      "type": "object",
      "properties": {
        "messageType": {
          "const": "CONTROL_MESSAGE"
        }
      }
    }
  ]
}

ref

https://json-schema.org/understanding-json-schema/index.html

again

https://docs.aws.amazon.com/ja_jp/general/latest/gr/acct-identifiers.html

https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_CreateLogGroup.html

https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_CreateLogStream.html

https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_SubscriptionFilter.html#API_SubscriptionFilter_Contents

wintwint

Slack Block

{
	"blocks": [
		{
			"type": "header",
			"text": {
				"type": "plain_text",
				"text": ":rotating_light: Alert from CloudWatch Logs"
			}
		},
		{
			"type": "section",
			"fields": [
				{
					"type": "mrkdwn",
					"text": "*Log Group*"
				},
				{
					"type": "mrkdwn",
					"text": "*Log Stream*"
				},
				{
					"type": "plain_text",
					"text": "CloudTrail"
				},
				{
					"type": "plain_text",
					"text": "123456789012_CloudTrail_us-east-1"
				}
			]
		},
		{
			"type": "divider"
		},
		{
			"type": "section",
			"text": {
				"type": "mrkdwn",
				"text": "*Log Events*"
			}
		},
		{
			"type": "section",
			"text": {
				"type": "mrkdwn",
				"text": "```\nfoo\nbar```"
			}
		}
	]
}
wintwint

フィルター構文には2種類の括弧があるな

  • JSONマッチング: { SELECTOR NUMERIC_OPERATOR NUMBER }
    • JSONで構造化されてると便利
  • フィールドマッチング: [ f1 f2 f3 ]
    • "foo bar" とか [foo bar] とかをまとめてくれて便利

https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/FilterAndPatternSyntax.html#matching-terms-events

Using metric filters to extract values from space-delimited log events
To specify a metric filter pattern that parses space-delimited events, the metric filter pattern has to specify the fields with a name, separated by commas, with the entire pattern enclosed in square brackets. For example: [ip, user, username, timestamp, request, status_code, bytes].

このスクラップは2021/08/26にクローズされました