GCPメモ

"roles/errorreporting.writer" = [
      "serviceAccount:〜〜〜.iam.gserviceaccount.com",
    ]

gcloud auth activate-service-account --key-file=[キーファイルパス]

resource.type="k8s_cluster" severity>=WARNING
resource.type="k8s_cluster" resource.labels.location="asia-northeast1"
resource.labels.cluster_name="対象クラスター名"
jsonPayload.involvedObject.kind="CronJob"

      "filter" = <<-EOF
        severity>=WARNING
        httpRequest.status>=500
        resource.labels.backend_service_name=~""
      EOF

resource.type="cloudsql_database"
logname="{var.project_id}.cloudsql.googleapis.com%2Fmysql-slow.log

gcloud compute ssh --project=プロジェクト名 --zone=ゾーン名 インスタンス名

gcloud compute snapshots create  スナップショット名 \
    --source-disk ディスク名 \
    --source-disk-zone ゾーン名 \
    --project プロジェクト名 \
    --impersonate-service-account=〜〜@〜〜.com

gcloud compute snapshots list --project プロジェクト名 | grep 作成したスナップショット名

gcloud compute ssh 対象インスタンス名 \
　--zone 対象ゾーン名 \
　--project 対象プロジェクト名 \
　--impersonate-service-account=〜〜@〜〜.com

gcloud beta resource-config bulk-export --resource-format=terraform 
You do not currently have this command group installed.  Using it 
requires the installation of components: [beta]


Your current Google Cloud CLI version is: 379.0.0
Installing components from version: 379.0.0

┌─────────────────────────────────────────────┐
│     These components will be installed.     │
├──────────────────────┬────────────┬─────────┤
│         Name         │  Version   │   Size  │
├──────────────────────┼────────────┼─────────┤
│ gcloud Beta Commands │ 2022.03.25 │ < 1 MiB │
└──────────────────────┴────────────┴─────────┘

For the latest full release notes, please visit:
  https://cloud.google.com/sdk/release_notes

Do you want to continue (Y/n)?  Y

╔════════════════════════════════════════════════════════════╗
╠═ Creating update staging area                             ═╣
╠════════════════════════════════════════════════════════════╣
╠═ Installing: gcloud Beta Commands                         ═╣
╠════════════════════════════════════════════════════════════╣
╠═ Creating backup and activating new installation          ═╣
╚════════════════════════════════════════════════════════════╝

Performing post processing steps...done.                                                                                                                                                                   

Update done!

Restarting command:
  $ gcloud beta resource-config bulk-export --resource-format=terraform --path=./bulk_export/

Pausing command execution:

This command requires the `config-connector` binary to be installed to export GCP resource configurations. Would you like to install the`config-connector` binary to continue command execution? (Y/n)?  Y



Your current Google Cloud CLI version is: 379.0.0
Installing components from version: 379.0.0

┌───────────────────────────────────────────────────────────────────────────┐
│                    These components will be installed.                    │
├──────────────────────────────┬─────────────────────┬──────────────────────┤
│             Name             │       Version       │         Size         │
├──────────────────────────────┼─────────────────────┼──────────────────────┤
│ config-connector             │              1.78.0 │             51.0 MiB │
└──────────────────────────────┴─────────────────────┴──────────────────────┘

For the latest full release notes, please visit:
  https://cloud.google.com/sdk/release_notes

Do you want to continue (Y/n)?  Y

╔════════════════════════════════════════════════════════════╗
╠═ Creating update staging area                             ═╣
╠════════════════════════════════════════════════════════════╣
╠═ Installing: config-connector                             ═╣
╠════════════════════════════════════════════════════════════╣
╠═ Installing: config-connector                             ═╣
╠════════════════════════════════════════════════════════════╣
╠═ Creating backup and activating new installation          ═╣
╚════════════════════════════════════════════════════════════╝

Performing post processing steps...done.                                                                                                                                                                   

Update done!

ERROR: (gcloud.beta.resource-config.bulk-export) config-connector binary not installed

gcloud beta resource-config bulk-export \
>   --project #### \
>   --resource-format=terraform \
>   --path=./対象ディレクトリ
API [cloudasset.googleapis.com] is required to continue, but is not enabled on project [learngcp-335008]. Would you like to enable and retry (this will take a few minutes)? (y/N)?  y

Enabling service [cloudasset.googleapis.com] on project [learngcp-335008]...
Operation "operations/acat.p2-95270527167-ad5622f1-3a70-457b-940c-2a595fd4969e" finished successfully.
Path ./対象ディレクトリ does not exists. Do you want to create it?

Do you want to continue (Y/n)?  n

ERROR: (gcloud.beta.resource-config.bulk-export) Export aborted. No files written.

gcloud config set core/project プロジェクトID

gcloud beta resource-config bulk-export --project=プロジェクトID --resource-format=terraform --path=./対象ディレクトリ
Exporting resource configurations to [./対象ディレクトリ]...done.                                                                                                                                
Exported resource configuration(s) to [./対象ディレクトリ].

resource "google_project_service" "vpcaccess-api" {
  project = var.project_id # Replace this with your project ID in quotes
  service = "vpcaccess.googleapis.com"
}

# Cloud Run service
resource "google_cloud_run_service" "gcr_service" {
  name     = "mygcrservice"
  provider = google-beta
  location = "us-west1"

  template {
    spec {
      containers {
        image = "us-docker.pkg.dev/cloudrun/container/hello"
        resources {
          limits = {
            cpu = "1000m"
            memory = "512M"
          }
        }
      }
      # the service uses this SA to call other Google Cloud APIs
      # service_account_name = myservice_runtime_sa
    }

    metadata {
      annotations = {
        # Limit scale up to prevent any cost blow outs!
        "autoscaling.knative.dev/maxScale" = "5"
        # Use the VPC Connector
        "run.googleapis.com/vpc-access-connector" = google_vpc_access_connector.connector.name
        # all egress from the service should go through the VPC Connector
        "run.googleapis.com/vpc-access-egress" = "all"
      }
    }
  }
  autogenerate_revision_name = true
}

gcloud run deploy cloud run名 \
	--project 対象プロジェクトID \
	--region 対象リージョン \
  	--image 対象イメージID \
	--update-secrets=DB_PASSWORD=secret_manager名:バージョン

Deploying container to Cloud Run service [***] in project [***] region [***]
X Deploying... Google Cloud Run Service Agent must have permission to read the image, . Ensure tha
t the provided container image URL is correct and that the above account has permission to access the image. If you just enabled the Cl
oud Run API, the permissions might take a few minutes to propagate. Note that the image is from project [***], which is not the 
same as this project [***]. Permission must be granted to the Google Cloud Run Service Agent from this project.              
  X Creating Revision... Google Cloud Run Service Agent must have permission to read the image, ***. 
  Ensure that the provided container image URL is correct and that the above account has permission to access the image. If you just en
  abled the Cloud Run API, the permissions might take a few minutes to propagate. Note that the image is from project [***], which is not the same as this project [***]. Permission must be granted to the Google Cloud Run Service Agent from this projec
  t.                                                                                                                                   
  . Routing traffic...                                                                                                                 
Deployment failed                            

ERROR: (gcloud.run.deploy) Google Cloud Run Service Agent must have permission to read the image, ***. Ensure that the provided container image URL is correct and that the above account has permission to access the image. If you just enabled the Cloud Run API, the permissions might take a few minutes to propagate. Note that the image is from project [***], which is not the same as this project [***]. Permission must be granted to the Google Cloud Run Service Agent from this project.

echo -n "パスワード" | gcloud secrets create Sercret manger名 --data-file=- --impersonate-service-account=

-rwxr-xr-x    1   staff   3 15 10:52 cloud_sql_proxy*

mv cloud_sql_proxy /usr/local/bin/cloud_sql_proxy

cloud_sql_proxy -instances=***

gcloud beta run deploy **** \
--no-cpu-throttling \
--min-instances

  template {
    metadata {
      annotations = {"run.googleapis.com/cpu-throttling"= false}
    }
}

autoscaling.knative.dev/minScale // sets the minimum number of instances.
autoscaling.knative.dev/maxScale // sets the maximum number of instances.

Error: Forbidden
Your client does not have permission to get URL / from this server.

Ingress：内部トラフィックとCloud Load Balancingからのトラフィックを許可する
認証：未認証の呼び出しを許可

--allow-unauthenticated 
--ingress internal-and-cloud-load-balancing

  template {
    metadata {
      annotations = {
        "run.googleapis.com/ingress" = "internal-and-cloud-load-balancing"  // Ingressの設定
      }
    }

Google Cloud プロジェクトで 1 時間以内にエラーが 5 回以上再発した場合、
Error Reporting から最終通知が送られ、
今後 6 時間は通知が送信されないことが伝えられます。これにより、通知の量を制御します。

解決ステータス
解決ステータスが [解決済み] のエラーが再発した場合、
そのエラーがすでに削除されていても、解決ステータスが [対応待ち] に戻され、
Error Reporting から通知が送信されます。

[ミュート中] のエラーが再発した場合は、Error Reporting から通知は送信されません。

roles/errorreporting.writer
roles/logging.logWriter

--verbosity=debug

 gcloud --verbosity=debug run deploy *** \
  --project *** \
  --region *** \
  --allow-unauthenticated \
  --image asia.gcr.io/****:latest

openssl genrsa -out ****.co.jp.key 2048

cat <<'EOF' >openssl_file
[req]
default_bits              = 2048
req_extensions            = extension_requirements
distinguished_name        = dn_requirements

[extension_requirements]
basicConstraints          = CA:FALSE
keyUsage                  = nonRepudiation, digitalSignature, keyEncipherment
subjectAltName            = @sans_list

[dn_requirements]
countryName               = Country Name (2 letter code)
stateOrProvinceName       = State or Province Name (full name)
localityName              = Locality Name (eg, city)
0.organizationName        = Organization Name (eg, company)
organizationalUnitName    = Organizational Unit Name (eg, section)
commonName                = Common Name (e.g. server FQDN or YOUR name)
emailAddress              = Email Address

[sans_list]
DNS.1                     = ****.co.jp

EOF

openssl req -new -key ****.co.jp.key \
    -out *** \
    -config ***

openssl x509 -req \
    -signkey ***.co.jp.key \
    -in ***-csr \
    -out ***-certificate \
    -extfile *** \
    -extensions extension_requirements \
    -days 365

gcloud compute ssl-certificates create ****-cert \
  --certificate=***-certificate \
  --private-key=***.co.jp.key \
  --project=*** \
  --global

gcloud compute target-https-proxies update *** \
    --global \
    --ssl-certificates=*** \
    --global-ssl-certificates

module "sample_serveless_lb" {
  source                          = "GoogleCloudPlatform/lb-http/google//modules/serverless_negs"
  name                            = "sample_serveless_lb"
  project                         = var.project
  ssl                             = true
  managed_ssl_certificate_domains =  ["sample.co.jp"]
  ssl_certificates                = ["https://www.googleapis.com/compute/v1/projects/****/global/sslCertificates/***-co-jp-20990101"]
  create_address                  = false
  address                         = google_compute_global_address.sample_address.address
  https_redirect                  = false
  url_map                         = google_compute_url_map.sample_url_map.self_link
  create_url_map                  = false
  security_policy                 = google_compute_security_policy.sample_sp.name

  backends = {
    default = {
      groups = [
        {
          group = google_compute_region_network_endpoint_group.sample_serverless_neg.id
        }
      ]

      connection_draining_timeout_sec = 300
      description                     = null
      enable_cdn                      = false
      security_policy                 = null
      custom_request_headers          = null
      custom_response_headers         = null

      iap_config = {
        enable               = false
        oauth2_client_id     = ""
        oauth2_client_secret = ""
      }
      log_config = {
        enable      = true
        sample_rate = 1.0
      }
    }
  }
}

①Cloud RunサービスアカウントにDB用のroleをつける (roles/cloudsql.client)
②Cloud RunDB接続でenvにDB_PASSWORDを設定 (Secret Manager利用が良い)
③php artisan migrateコマンド用のshを作成
④DockerfileのENTRYPOINTでshを叩く
⑤CI/CDにgcloud run deployを組み込んでデプロイさせる

MYSQL_PWD=$(gcloud auth print-access-token  --impersonate-service-account=**user@****.iam.gserviceaccount.com) mysql -u **user -h 127.0.0.1 -P 3316 --enable-cleartext-plugin

　- サービスアカウント作成時に、roles/cloudsql.instanceUserをつけておく
　- roles/iam.serviceAccountTokenCreatorを対象グループ(DB用ML等)に付与する

# サービスアカウント作成, cloudsql userロール付与
module "db_user" {
  source  = "terraform-google-modules/service-accounts/google"
  version = "~> **"

  display_name = "db-user"
  description  = "db-user"
  project_id   = var.project_id
  prefix       = "db"
  names        = ["db-user"]
  project_roles = [
    "${var.project_id}=>roles/cloudsql.instanceUser"
  ]
}

# iam.serviceAccountTokenCreatorのroleを対象メンバーグループに付与
resource "google_service_account_iam_binding" "db_token_creater" {
  service_account_id = module.db_user[0].id
  role               = "roles/iam.serviceAccountTokenCreator"
  members = [
    "group:**@co.jp"
 ]
}

gcloud auth login

locals {
  now = timestamp()
  asia_tokyo_tz = timeadd(local.now, "9h")
  convert_cursorTimestamp = formatdate("YYYY-MM-DD hh:mm", local.asia_tokyo_tz)
  error_alert = {
    slack_post_message_document = <<-EOF
      エラーを検知しました。
      詳細の確認をお願いします。
      エラー発生日時：${local.convert_cursorTimestamp}
      https://console.cloud.google.com/logs/query;query=resource.type****
    EOF
 }
}

output "time" {
  value = {
    "now" = local.now,
    "asia_tokyo_tz" = local.asia_tokyo_tz,
    "convert_cursorTimestamp" = local.convert_cursorTimestamp
  }
}

resource "google_logging_metric" "test_error_log" {
  name     = "test-error-log"
  project  = var.project_id
  filter   = <<-EOF
    resource.type = "cloud_run_revision"
    resource.labels.location = "asia-northeast1"
    severity>=ERROR
  EOF

  metric_descriptor {
    metric_kind = "DELTA"
    value_type  = "INT64"
  }
}

data "google_monitoring_notification_channel" "test_alert_channel" {
  display_name = "test-error-log"
}

# DEV TEST エラー検知用
resource "google_monitoring_alert_policy" "test_error_alert" {
  project      = var.project_id
  combiner     = "OR"
  display_name = "TEST error-log"
  conditions {
    display_name = google_logging_metric.test_error_log.name

    condition_threshold {
      filter = <<-EOF
          metric.type="logging.googleapis.com/user/${google_logging_metric.test_error_log.name}"
          resource.type="cloud_run_revision"
      EOF
      comparison      = "COMPARISON_GT"
      duration        = "0s"
      threshold_value = "0" // 1件でも検知したら作動させる

      aggregations {
        alignment_period     = "60s"
        per_series_aligner   = "ALIGN_COUNT"
        cross_series_reducer = "REDUCE_COUNT"
      }

      trigger {
        count = 1
      }
    }
  }
  alert_strategy {
      auto_close = "1800s"
  }

  documentation {
    mime_type = "text/markdown"
    content =  local.error_alert.slack_post_message_document
  }

  notification_channels = [
    data.google_monitoring_notification_channel.test_alert_channel.name
  ]
}

Google Cloud Monitoring
アプリ  17:34
Incident #*** is ongoing
test-error-log
TEST error-log
logging/ Cloud Run Revision labels {project_id=***} is above the threshold of 0.000 with a value of 1.000.
Documentation
アプリケーションでエラーを検知しました。
詳細の確認をお願いします。
エラー発生日時：2022-**-** **:**

gcloud beta run jobs create test-job \
--image=asia.gcr.io/プロジェクト/test-container:latest \
--command=php \
--command=artisan \
--command=migrate \
--max-retries=0 \
--service-account=****@****.iam.gserviceaccount.com \
--set-secrets=DB_PASS=****:latest \
--vpc-connector=vpc-*** \
--vpc-egress=all-traffic \
--project=**** \
--region=asia-northeast1

gcloud beta run jobs add-iam-policy-binding ジョブ名 \
  --member="group:***@****.co.jp" \
  --role="roles/run.invoker" \
  --project ****

gcloud beta run jobs get-iam-policy ジョブ名 --project ****

gcloud functions deploy deploy-functions \
--trigger-http \
--runtime=php81 \
--entry-point=function \
--source=**** \
--allow-unauthenticated \
--region=asia-northeast1 \
--memory=256MB \
--timeout=540s \
--security-level=secure-always \
--vpc-connector=projects/****/locations/asia-northeast1/connectors/****

## cloud functionsで使う関数が設置されたパスを指定する
data "archive_file" "file" {
  type        = "zip"
  source_dir  = "${path.module}/scripts/functions"
  output_path = "${path.module}/functions.zip"
}

## terraformデプロイ時に必須の為構築 ソースはアプリ側のコードから反映
resource "google_storage_bucket_object" "functions_object" {
  name    = "functions${data.archive_file.file.output_md5}.zip"
# bucket作成は必須
  bucket  = google_storage_bucket.functions.name
  source  = data.archive_file.file.output_path
}

resource "google_storage_bucket" "functions_bucket" {
  name     = "functions-bucket"
  location = ****
  project  = ****
}

resource "google_cloudfunctions_function" "functions" {
  name                          = "functions"
  runtime                       = "php81"
  region                        = var.region
  timeout                       = 540
  available_memory_mb           = 256
  entry_point                   = "function"
  trigger_http                  = true
  source_archive_bucket         = google_storage_bucket.functions_bucket.name
  source_archive_object         = google_storage_bucket_object.functions_object.name
  vpc_connector                 = "projects/${var.project_id}/locations/${var.region}/connectors/${local.vpc_access_connector}"
  vpc_connector_egress_settings = "ALL_TRAFFIC"
  ingress_settings              = "ALLOW_INTERNAL_AND_GCLB"

  # MEMO：gcloudでのCI/CDデプロイ時にlabelsが更新される為無視する項目　
  lifecycle {
    ignore_changes = [
      source_archive_object,
      labels,
      ingress_settings,
      environment_variables
    ]
  }
}

## グローバルアドレス作成
resource "google_compute_global_address" "address" {
  name          = "****"
  address_type  = "EXTERNAL"
}

## cloud functionsと紐付け
resource "google_compute_region_network_endpoint_group" "serverless_neg" {
  name                  = "lb-functions-neg"
  network_endpoint_type = "SERVERLESS"
  project               = ****
  region                = ****
  cloud_function {
    function = google_cloudfunctions_function.****.name
  }
}

## url map作成
resource "google_compute_url_map" "url_map" {
  name            = "lb-url-map"
  default_service = module.***.backend_services["default"].self_link
  project         = ****
}

## moduleでLB作成
module "lb_serveless_lb" {
  source                          = "GoogleCloudPlatform/lb-http/google//modules/serverless_negs"
  name                            = "lb-functions"
  project                         = ****
  ssl                             　= true
  managed_ssl_certificate_domains = var.env == ["***"]
  create_address                  = false
  address                         = google_compute_global_address.****.address
  https_redirect                  = true
  url_map                         = google_compute_url_map.****.self_link
  create_url_map                  = false
  security_policy                 = ""

  backends = {
    default = {
      groups = [
        {
          group = google_compute_region_network_endpoint_group.serverless_neg.id
        }
      ]

      connection_draining_timeout_sec = 300
      description                     = null
      enable_cdn                      = false
      security_policy                 = null
      custom_request_headers          = null
      custom_response_headers         = null

      iap_config = {
        enable               = false
        oauth2_client_id     = ""
        oauth2_client_secret = ""
      }
      log_config = {
        enable      = true
        sample_rate = 1.0
      }
    }
  }
}

# /etc/google-cloud-ops-agent/config.yaml
data "template_file" "ops_agent" {
  template = file("config.yaml")
}

# index.htmlをindex.phpに上書き
data "template_file" "index1_php" {
  template = file("index1.php")
}

data "template_file" "index2_php" {
  template = file("index2.php")
}

resource "google_compute_instance" "inspection" {
  name         = "inspection-1"
  machine_type = "f1-micro"
  zone         = "asia-northeast1-a"
  project      = var.project

  boot_disk {
    initialize_params {
      image = "ubuntu-os-pro-cloud/ubuntu-pro-2204-jammy-v20221014"
    }
  }

  network_interface {
    network = "https://www.googleapis.com/compute/v1/projects/****/global/networks/default"
    access_config {
        nat_ip       = "**Route53で指定するIPアドレス**"
    }
  }

  metadata_startup_script = <<EOF
#!/bin/bash
sudo su
apt-get update
apt-get -y install apache2
systemctl start apache2
apt-get -y install php
curl -sSO https://dl.google.com/cloudagents/add-google-cloud-ops-agent-repo.sh
bash add-google-cloud-ops-agent-repo.sh --also-install
cp /etc/google-cloud-ops-agent/config.yaml /etc/google-cloud-ops-agent/config.yaml.bak
# config.yamlを上書き
echo "${data.template_file.ops_agent.rendered}" | tee /etc/google-cloud-ops-agent/config.yaml
cp /var/www/html/index.html /var/www/html/index.html.bak
mv /var/www/html/index.html /var/www/html/index.php
# index.htmlをindex.phpに変えて出力を変更
echo "${data.template_file.index_php.rendered}" | tee /var/www/html/index.php
service google-cloud-ops-agent restart
EOF

fetch https_lb_rule
| metric 'loadbalancing.googleapis.com/https/response_bytes_count'
| filter (metric.cache_result == 'HIT')
| group_by 1m,
    [value_response_bytes_count_aggregate:
       aggregate(value.response_bytes_count)]
| every 1m
| group_by [resource.backend_target_name, resource.backend_target_type],
    [value_response_bytes_count_aggregate_aggregate:
       aggregate(value_response_bytes_count_aggregate)]
| filter resource.backend_target_name == "****"
| filter resource.backend_target_type == "BACKEND_SERVICE"

gcloud artifacts repositories create $REGISTRY_NAME --repository-format=docker --location $REGION --description="Docker repository" --project $PROJECT

gcloud auth configure-docker 

gcloud auth configure-docker ** 対象region **-docker.pkg.dev

docker pull asia.gcr.io/project-id/image-name:latest
# 確認
docker image

# docker tag Container Registryのパス Artifact Registry:タグを入れる
docker tag asia.gcr.io/project-id/image-name region名-docker.pkg.dev/project-id/リポジトリ名/image-name:1.0

# 確認
docker image

# ↑でタグづけしたイメージをpush
docker push region名-docker.pkg.dev/project-id/リポジトリ名/image-name:1.0

ssh itd@IPアドレス番号

The authenticity of host '***' can't be established.
RSA key fingerprint is SHA256:***.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '****' (RSA) to the list of known hosts.
itd@****'s password: 

# brewでインストール
brew install nodebrew

# zsh用のセット
vim ~/.zshrc
export PATH=$HOME/.nodebrew/current/bin:$PATH
source ~/.zshrc

# 確認
nodebrew setup
nodebrew -v

# リモートのバージョン確認
nodebrew ls-remote

nodebrew install ***
nodebrew use ***
nodebrew -ls

gcloud artifacts repositories create REPOSITORY \
    --repository-format=npm \
    [--location=LOCATION] \
    [--description="DESCRIPTION"] \
    [--kms-key=KMS-KEY] \
    [--async] \

 gcloud artifacts print-settings npm \
    --project=**** \
    --repository=test-npm--repository \
    --location=asia-**** \
    --scope=@****

  "name": "@npm-test/private-repositry",
  "version": "***",

touch .npmrc
vim .npmrc
# 前の項目にあった2行を貼り付け
@SCOPE:registry=https://asia-northeast1-npm.pkg.dev/プロジェクトID/REPOSITRY/
//asia-northeast1-npm.pkg.dev/プロジェクトID/REPOSITRY/:always-auth=true

npm run artifactregistry-login

> @*****@*.**.0 artifactregistry-login
> npx google-artifactregistry-auth

Need to install the following packages:
  google-artifactregistry-auth@3.0.2
Ok to proceed? (y) y
Retrieving application default credentials...
Success!

npx google-artifactregistry-auth
npx: 47個のパッケージを4.278秒でインストールしました。
Retrieving application default credentials...
Success!

gcloud config unset run/region REGION