モチベ

AWS 公式の自動構築テンプレで CloudTrail の logs の metrics filter は作れるけど、ログの有無しか分からないのが足りない。 ChatBot で簡単に通知できるけど、ログごと送信したい。

参考

これは既存のアラートのIaC

実装

Metric
CloudTrailMetrics / AuthorizationFailureCount
Alarms
CloudTrailAuthorizationFailures

{ ($.errorCode = "*UnauthorizedOperation") || ($.errorCode = "AccessDenied*") }

wint 2021/06/07に更新

構想

CloudTrail → CloudWatch Logs → filter → AWS Lambda → Slack

以下参考資料

Logs … are Base64 encoded and compressed with the gzip format.

Subscription filters with AWS Lambda

cw-logs-to-slack

wint 2021/06/05

AWS Lambda を使うなら Serverless Framework を検討したい

ドンピシャのがあるじゃん

wint 2021/06/07に更新

入力形式

再掲

{ "awslogs": { "data": "BASE64ENCODED_GZIP_COMPRESSED_DATA" } }

{
  type: "object",
  properties: {
    awslogs: {
      type: "object",
      properties: {
        data: {
          type: "string",
          contentEncoding: "base64",
          contentMediaType: "application/gzip",
          description: "Base64 encoded GZIP compressed JSON",
        },
      },
      required: ["data"],
    },
  },
  required: ["awslogs"],
}

参考

Currently, API Gateway supports JSON Schema draft-04.

New in draft 7

API GW には使えなさそう

https://nodejs.org/api/buffer.html

https://nodejs.org/api/zlib.html

wint 2021/06/07

Serverless

wint 2021/06/07に更新

Slack

SDK

https://slack.dev/node-slack-sdk/

Node Slack SDK

Node Slack SDK / Incoming Webhooks

manifest

_metadata:
  major_version: 1
  minor_version: 1
display_information:
  name: AWS Log Alert
features:
  app_home:
    home_tab_enabled: false
    messages_tab_enabled: true
    messages_tab_read_only_enabled: true
  bot_user:
    display_name: AWS Log Alert
    always_online: false
oauth_config:
  scopes:
    bot:
      - incoming-webhook
settings:
  org_deploy_enabled: false
  socket_mode_enabled: false
  is_hosted: false

wint 2021/06/09に更新

JSON schema of the persed log

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "oneOf": [
    {
      "type": "object",
      "properties": {
        "owner": {
          "type": "string",
          "pattern": "^\\d{12}$"
        },
        "logGroup": {
          "type": "string",
          "pattern": "^[\\w/.#-]{1,512}$"
        },
        "logStream": {
          "type": "string",
          "pattern": "^[^:*]{1,512}$"
        },
        "subscriptionFilters": {
          "type": "array",
          "items": {
            "type": "string",
            "pattern": "^[^:*]{1,512}$"
          }
        },
        "messageType": {
          "const": "DATA_MESSAGE"
        },
        "logEvents": {
          "type": "array",
          "items": {
            "type": "object",
            "properties": {
              "id": {
                "type": "string"
              },
              "timestamp": {
                "type": "number",
                "minimum": 0
              },
              "message": {
                "type": "string",
                "description": "escaped JSON string"
              }
            }
          }
        }
      }
    },
    {
      "type": "object",
      "properties": {
        "messageType": {
          "const": "CONTROL_MESSAGE"
        }
      }
    }
  ]
}

ref

again

wint 2021/06/07

Slack Block

{
	"blocks": [
		{
			"type": "header",
			"text": {
				"type": "plain_text",
				"text": ":rotating_light: Alert from CloudWatch Logs"
			}
		},
		{
			"type": "section",
			"fields": [
				{
					"type": "mrkdwn",
					"text": "*Log Group*"
				},
				{
					"type": "mrkdwn",
					"text": "*Log Stream*"
				},
				{
					"type": "plain_text",
					"text": "CloudTrail"
				},
				{
					"type": "plain_text",
					"text": "123456789012_CloudTrail_us-east-1"
				}
			]
		},
		{
			"type": "divider"
		},
		{
			"type": "section",
			"text": {
				"type": "mrkdwn",
				"text": "*Log Events*"
			}
		},
		{
			"type": "section",
			"text": {
				"type": "mrkdwn",
				"text": "```\nfoo\nbar```"
			}
		}
	]
}

wint 2021/06/10に更新

フィルター構文には2種類の括弧があるな

JSONマッチング: { SELECTOR NUMERIC_OPERATOR NUMBER }
- JSONで構造化されてると便利
フィールドマッチング: [ f1 f2 f3 ]
- "foo bar" とか [foo bar] とかをまとめてくれて便利

Using metric filters to extract values from space-delimited log events
To specify a metric filter pattern that parses space-delimited events, the metric filter pattern has to specify the fields with a name, separated by commas, with the entire pattern enclosed in square brackets. For example: [ip, user, username, timestamp, request, status_code, bytes].

wint 2021/06/11に更新

known errors

"CloudTrail/DefaultLogGroup" logGroup for cloudwatchLog event is duplicated. This property can only be set once per CloudFormation stack.

code: https://github.com/serverless/serverless/blob/8d56d0e520db8068e89fa8d4d2bf4e64b0acd97b/lib/plugins/aws/package/compile/events/cloudWatchLog.js#L57-L65

モチベ

参考

実装

構想

以下 参考資料

入力形式

参考

Slack

SDK

manifest

JSON schema of the persed log

ref

known errors

以下参考資料