Closed1

zundonのHeroku Postgresのバージョンを上げる

zundazunda

https://devcenter.heroku.com/articles/upgrading-heroku-postgres-databases#standard-tier-and-higher に従う

フォロワの作成

$ date; heroku addons:create heroku-postgresql:standard-0 -a zundan-mastodon --follow HEROKU_POSTGRESQL_PURPLE_URL
Thu Oct 10 14:09:45 HST 2024
 ›   Warning: You’re using a deprecated syntax with the [--follow] flag.
 ›   Add a '--' (end of options) separator before the flags you’re passing through.
 ›   For example: heroku addons:create -a zundan-mastodon heroku-postgresql:standard-0 -- --follow HEROKU_POSTGRESQL_PURPLE_URL ›   See https://devcenter.heroku.com/changelog-items/2925 for more info.
Creating heroku-postgresql:standard-0 on ⬢ zundan-mastodon... ⣟
 ›   Warning: Release command executing: config vars set by this add-on will not be available until the command succeeds. Use `heroku releases:output` to view the log.
Creating heroku-postgresql:standard-0 on ⬢ zundan-mastodon... ~$0.069/hour (max $50/month)
The database should be available in 3-5 minutes.
Use `heroku pg:wait` to track status.
postgresql-fluffy-74001 is being created in the background. The app will restart when complete...
Use heroku addons:info postgresql-fluffy-74001 to check creation progress
Use heroku addons:docs heroku-postgresql to view documentation

5分ほどかかった。

$ heroku pg:wait postgresql-fluffy-74001 -a zundan-mastodon; date
Waiting for database postgresql-fluffy-74001... available
Thu Oct 10 14:14:35 HST 2024

現状

$ heroku pg:info -a zundan-mastodon
=== DATABASE_URL, HEROKU_POSTGRESQL_PURPLE_URL

Plan:                  Standard 0
Status:                Available
Data Size:             21.7 GB / 64 GB (33.87%)
Tables:                99
PG Version:            13.15, (deprecating)
Connections:           17/120
Connection Pooling:    Available
Credentials:           1
Fork/Follow:           Available
Rollback:              earliest from 2024-10-07 00:15 UTC
Created:               2022-04-15 20:29 
Region:                us
Data Encryption:       In Use
Continuous Protection: On
Enhanced Certificates: Off
Upgradable Extensions: Yes
Followers:             …
Maintenance:           not required
Maintenance window:    Thursdays 19:30 to 23:30 UTC
Warning:               Postgres version 13 is EOL on 2024-10-29. Upgrade Versions.
Add-on:                postgresql-tetrahedral-12223

=== HEROKU_POSTGRESQL_BLUE_URL

Plan:                  Standard 0
Status:                Available
Data Size:             21.7 GB / 64 GB (33.86%)
Tables:                99
PG Version:            13.16, (deprecating)
Connections:           8/120
Connection Pooling:    Available
Credentials:           1
Fork/Follow:           Unavailable on followers
Rollback:              earliest from 2024-10-11 00:09 
Created:               2024-10-11 00:09 
Region:                us
Data Encryption:       In Use
Continuous Protection: On (on leader)
Enhanced Certificates: Off
Upgradable Extensions: Yes
Following:             HEROKU_POSTGRESQL_PURPLE
Behind By:             24432 commits
Maintenance:           not required
Maintenance window:    Tuesdays 19:30 to 23:30 UTC
Warning:               Postgres version 13 is EOL on 2024-10-29. Upgrade Versions.
Add-on:                postgresql-fluffy-74001

45分ほど放置。追いついてきている。

$ heroku pg:info postgresql-fluffy-74001 -a zundan-mastodon; date
=== HEROKU_POSTGRESQL_BLUE_URL

Plan:                  Standard 0
Status:                Available
Data Size:             21.7 GB / 64 GB (33.87%)
Tables:                99
PG Version:            13.16, (deprecating)
Connections:           11/120
Connection Pooling:    Available
Credentials:           1
Fork/Follow:           Unavailable on followers
Rollback:              earliest from 2024-10-11 00:09 
Created:               2024-10-11 00:09 
Region:                us
Data Encryption:       In Use
Continuous Protection: On (on leader)
Enhanced Certificates: Off
Upgradable Extensions: Yes
Following:             HEROKU_POSTGRESQL_PURPLE
Behind By:             31 commits
Maintenance:           not required
Maintenance window:    Tuesdays 19:30 to 23:30 UTC
Warning:               Postgres version 13 is EOL on 2024-10-29. Upgrade Versions.
Add-on:                postgresql-fluffy-74001

Thu Oct 10 14:58:47 HST 2024

メンテナンス

メンテナンスモードにする

ストリーミングサーバを落とす。Sidekiqの一部も一緒に落ちる。

$ heroku ps:scale web=0 -a zundan-mastodon-streaming
Scaling dynos... done, now running web at 0:Eco

メンテナンスモードにしてPumaと最後のSidekiqを落とす。

$ date; heroku maintenance:on -a zundan-mastodon; heroku ps:scale web=0 -a zundan-mastodon
Fri Oct 11 09:13:17 HST 2024
Enabling maintenance mode for ⬢ zundan-mastodon... done
Scaling dynos... done, now running web at 0:Basic

フォロワの状態の確認。まだ追いつき中。

$ heroku pg:info postgresql-fluffy-74001 -a zundan-mastodon; date
=== HEROKU_POSTGRESQL_BLUE_URL

Plan:                  Standard 0
Status:                Available
Data Size:             21.7 GB / 64 GB (33.88%)
Tables:                99
PG Version:            13.16, (deprecating)
Connections:           11/120
Connection Pooling:    Available
Credentials:           1
Fork/Follow:           Unavailable on followers
Rollback:              earliest from 2024-10-11 00:09 
Created:               2024-10-11 00:09 
Region:                us
Data Encryption:       In Use
Continuous Protection: On (on leader)
Enhanced Certificates: Off
Upgradable Extensions: Yes
Following:             HEROKU_POSTGRESQL_PURPLE
Behind By:             11 commits
Maintenance:           not required
Maintenance window:    Tuesdays 19:30 to 23:30 UTC
Warning:               Postgres version 13 is EOL on 2024-10-29. Upgrade Versions.
Add-on:                postgresql-fluffy-74001

Fri Oct 11 09:13:51 HST 2024

30秒ほどで追いついた。

$ heroku pg:info postgresql-fluffy-74001 -a zundan-mastodon; date
  :
Following:             HEROKU_POSTGRESQL_PURPLE
Behind By:             0 commits
  :
Fri Oct 11 09:14:38 HST 2024

フォロワのupgrade

$ date; heroku pg:upgrade HEROKU_POSTGRESQL_BLUE -a zundan-mastodon; heroku pg:wait -a zundan-mastodon; date
Fri Oct 11 09:17:05 HST 2024
 ›   Warning: Destructive action
 ›   postgresql-fluffy-74001 will be upgraded to a newer PostgreSQL version, 
 ›   stop following HEROKU_POSTGRESQL_PURPLE, and become writable.
 ›
 ›   This cannot be undone.
 ›

To proceed, type zundan-mastodon or re-run this command with --confirm zundan-mastodon: zundan-mastodon
Starting upgrade of postgresql-fluffy-74001... Use heroku pg:wait to track status
Waiting for database postgresql-fluffy-74001... ⣟ upgrading
Waiting for database postgresql-fluffy-74001... ⣾ rotating credentials after upgrade
Waiting for database postgresql-fluffy-74001... performing final cleanup steps after upgrade
Fri Oct 11 09:23:06 HST 2024

16.4になりました。

$ heroku pg:info postgresql-fluffy-74001 -a zundan-mastodon
=== HEROKU_POSTGRESQL_BLUE_URL

Plan:                  Standard 0
Status:                Available
Data Size:             21.7 GB / 64 GB (33.87%)
Tables:                99
PG Version:            16.4
Connections:           13/120
Connection Pooling:    Available
Credentials:           1
Fork/Follow:           Temporarily Unavailable
Rollback:              Capturing Snapshot
Created:               2024-10-11 00:09 
Region:                us
Data Encryption:       In Use
Continuous Protection: On
Enhanced Certificates: Off
Upgradable Extensions: Yes
Forked From:           HEROKU_POSTGRESQL_PURPLE
Maintenance:           not required
Maintenance window:    Tuesdays 19:30 to 23:30 UTC
Add-on:                postgresql-fluffy-74001

フォロワのプライマリへの昇格

$ heroku pg:promote HEROKU_POSTGRESQL_BLUE -a zundan-mastodon
Ensuring an alternate alias for existing DATABASE_URL... HEROKU_POSTGRESQL_PURPLE_URL
Promoting postgresql-fluffy-74001 to DATABASE_URL on ⬢ zundan-mastodon... done
Checking release phase... pg:promote failed because Attach DATABASE (@ref:postgresql-fluffy-74001) release was unsuccessful. Your application is currently running with postgresql-tetrahedral-12223 attached as DATABASE_URL. Check your release phase logs for failure causes.

Release Phaseが落ちたぞ…

2024-10-11T19:24:55.528914+00:00 heroku[release.1781]: Process exited with status 1
2024-10-11T19:24:55.469783+00:00 app[release.1781]: rake aborted!
2024-10-11T19:24:55.469837+00:00 app[release.1781]: ActiveRecord::ConnectionNotEstablished: connection to server at "127.0.0.1", port 5432 failed: Connection refused (ActiveRecord::ConnectionNotEstablished)
2024-10-11T19:24:55.469841+00:00 app[release.1781]: Is the server running on that host and accepting TCP/IP connections?

再挑戦かな。

$ heroku pg:promote HEROKU_POSTGRESQL_BLUE -a zundan-mastodon
Ensuring an alternate alias for existing DATABASE_URL... !
 ›   Error: postgresql-fluffy-74001 is already promoted on ⬢ zundan-mastodon

リリースはつくられている。

$ heroku releases -a zundan-mastodon
=== zundan-mastodon Releases - Current: v2618

 v2618 …                        heroku-postgresql@addons.heroku.com 2024/10/11 09:25:26 -1000 (~ 57s ago) 
 v2617 … release command failed zundan@gmail.com                    2024/10/11 09:24:33 -1000 (~ 1m ago)  
 v2616 … release command failed zundan@gmail.com                    2024/10/11 09:24:33 -1000 (~ 1m ago)  
 v2615 …                        heroku-postgresql@addons.heroku.com 2024/10/10 14:42:21 -1000 (~ 18h ago) 
  :

コンソールで確認

$ heroku run bash -a zundan-mastodon
Running bash on ⬢ zundan-mastodon... up, run.8728
~ $ echo $DATABASE_URL 
(あたらしいものになっている)
irb(main):001> Account.find(1).username
=> "zundan"

ちゃんと読めてるな。

メンテナンスの終了

$ heroku ps:scale web=1 -a zundan-mastodon
Scaling dynos... done, now running web at 1:Basic

PumaとSidekiqの起動を確認して、

$ heroku maintenance:off -a zundan-mastodon; date
Disabling maintenance mode for ⬢ zundan-mastodon... done
Fri Oct 11 09:31:10 HST 2024

今回のメンテナンスは18分間ほどでした。ストリーミングも再開。

$ heroku maintenance:off -a zundan-mastodon-streaming; date
Disabling maintenance mode for ⬢ zundan-mastodon-streaming... done
Fri Oct 11 09:32:17 HST 2024

ロジカルバックアップ

旧版の書き込みの止まったデータベースから取得しておく。

$ date; heroku pg:backups:capture postgresql-tetrahedral-12223 -a zundan-mastodon; date
Fri Oct 11 09:41:38 HST 2024
 ›   Warning: Continuous protection is already enabled for this database. 
 ›   Logical backups of large databases are likely to fail.
 ›   Warning: See https://devcenter.heroku.com/articles/heroku-postgres-data-sa
 ›   fety-and-continuous-protection#physical-backups-on-heroku-postgres.
Starting backup of postgresql-tetrahedral-12223... done

Use Ctrl-C at any time to stop monitoring progress; the backup will continue running.
Use heroku pg:backups:info to check progress.
Stop a running backup with heroku pg:backups:cancel.

Backing up PURPLE to b553... done
Fri Oct 11 09:47:40 HST 2024

ダウンロード

$ heroku pg:backups:url b553 -a zundan-mastodon

で得られたURLから。

旧版のデータベースを削除

$ heroku addons:destroy postgresql-tetrahedral-12223 -a zundan-mastodon
 ›   Warning: WARNING: Destructive Action
 ›   This command will affect the app zundan-mastodon

To proceed, type zundan-mastodon or re-run this command with --confirm zundan-mastodon: zundan-mastodon
Destroying postgresql-tetrahedral-12223 on ⬢ zundan-mastodon... !
    Error: The add-on was unable to be destroyed: Resource 
    postgresql-tetrahedral-12223 must be detached from all other apps before 
    deprovisioning..

あと片づけ

ホームタイムラインの更新がおかしかった。ストリーミングには旧版のデータベースが共有されていた。上記に警告がありました。

$ heroku addons:detach postgresql-tetrahedral-12223 -a zundan-mastodon-streaming
Detaching DATABASE to postgresql-tetrahedral-12223 from zundan-mastodon-streaming... done
Unsetting DATABASE config vars and restarting zundan-mastodon-streaming... done, v2392
$ heroku addons:attach postgresql-fluffy-74001 -a zundan-mastodon-streaming
Attaching postgresql-fluffy-74001 to zundan-mastodon-streaming... done
Setting DATABASE config vars and restarting zundan-mastodon-streaming... done, v2393

旧版のデータベースを削除

$ heroku addons:destroy postgresql-tetrahedral-12223 -a zundan-mastodon
 ›   Warning: WARNING: Destructive Action
 ›   This command will affect the app zundan-mastodon

To proceed, type zundan-mastodon or re-run this command with --confirm zundan-mastodon: zundan-mastodon
Destroying postgresql-tetrahedral-12223 on ⬢ zundan-mastodon... done

タイムラインの矛盾を修正。

$ heroku run bash -a zundan-mastodon
Running bash on ⬢ zundan-mastodon... up, run.3468
~ $ tootctl feeds build zundan
OK 
~ $ 
exit

何分間かかかった。

このスクラップは1ヶ月前にクローズされました