Zenn
Closed8

MastodnのPsychを4にする旅 (未完)

Ruby
Mastodon
YAML
#
psych
zundazunda

MastodonのRubyを3.1にするためには、Ruby 3.1に添付の、読み込む型に制限の加わったPsych 4でYamlをパースできるのがうれしいです。

Mastodonでは、rails-settings-cached gem (0.6.6)がYAML.loadメソッド経由でPsychを使っています。

zundazunda

Psychの動作のバージョンによる違い

Psych 4でパースできないYamlは、例えばsettingsテーブルにあるようです。

> SELECT var,value FROM settings WHERE var='interactions';
     var      |                          value                          
--------------+---------------------------------------------------------
 interactions | --- !ruby/hash:ActiveSupport::HashWithIndifferentAccess+
              | must_be_follower: false                                +
              | must_be_following: false                               +
              | 
(1 row)

このようにRubyの型が指定されているYamlはPsych 4ではそのままでは読めません。

# test.rb
require 'yaml'

module ActiveSupport
  class HashWithIndifferentAccess < Hash
  end
end

yaml = <<_END
--- !ruby/hash:ActiveSupport::HashWithIndifferentAccess
must_be_follower: false
must_be_following: false
_END

puts "Psych::VERSION: #{Psych::VERSION}"
p YAML.load(yaml)

Ruby 3.0

$ rbenv local 3.0.4
$ ruby test.rb 
Psych::VERSION: 3.3.2
{"must_be_follower"=>false, "must_be_following"=>false}

Ruby 3.1

$ rbenv local 3.1.2
$ ruby test.rb 
Psych::VERSION: 4.0.3
/home/zunda/.rbenv/versions/3.1.2/lib/ruby/3.1.0/psych/class_loader.rb:99:in `find': Tried to load unspecified class: ActiveSupport::HashWithIndifferentAccess (Psych::DisallowedClass)
	from /home/zunda/.rbenv/versions/3.1.2/lib/ruby/3.1.0/psych/class_loader.rb:28:in `load'
	from /home/zunda/.rbenv/versions/3.1.2/lib/ruby/3.1.0/psych/visitors/to_ruby.rb:424:in `resolve_class'
	from /home/zunda/.rbenv/versions/3.1.2/lib/ruby/3.1.0/psych/visitors/to_ruby.rb:288:in `visit_Psych_Nodes_Mapping'
	from /home/zunda/.rbenv/versions/3.1.2/lib/ruby/3.1.0/psych/visitors/visitor.rb:30:in `visit'
	from /home/zunda/.rbenv/versions/3.1.2/lib/ruby/3.1.0/psych/visitors/visitor.rb:6:in `accept'
	from /home/zunda/.rbenv/versions/3.1.2/lib/ruby/3.1.0/psych/visitors/to_ruby.rb:35:in `accept'
	from /home/zunda/.rbenv/versions/3.1.2/lib/ruby/3.1.0/psych/visitors/to_ruby.rb:318:in `visit_Psych_Nodes_Document'
	from /home/zunda/.rbenv/versions/3.1.2/lib/ruby/3.1.0/psych/visitors/visitor.rb:30:in `visit'
	from /home/zunda/.rbenv/versions/3.1.2/lib/ruby/3.1.0/psych/visitors/visitor.rb:6:in `accept'
	from /home/zunda/.rbenv/versions/3.1.2/lib/ruby/3.1.0/psych/visitors/to_ruby.rb:35:in `accept'
	from /home/zunda/.rbenv/versions/3.1.2/lib/ruby/3.1.0/psych.rb:335:in `safe_load'
	from /home/zunda/.rbenv/versions/3.1.2/lib/ruby/3.1.0/psych.rb:370:in `load'
	from test.rb:15:in `<main>'
zundazunda

型の指定されたYamlはどこで生成されたのか

must_be_follower: falseがどこで扱われてるのか探してみます。

$ grep -rl must_be_follower | grep -v locales
config/settings.yml
spec/support/examples/lib/settings/scoped_settings.rb
spec/lib/user_settings_decorator_spec.rb
spec/controllers/settings/preferences/notifications_controller_spec.rb
log/test.log
coverage/index.html
app/services/notify_service.rb
app/views/settings/preferences/notifications/show.html.haml
app/controllers/settings/preferences_controller.rb

デフォルトの値はYamlのファイルに定義されているみたい。

# config/settings.yml
$ cat config/settings.yml 
# This file contains default values, and does not need to be edited. All
# important settings can be changed from the admin interface.

defaults: &defaults
  site_title: Mastodon
  site_short_description: ''
  site_description: ''
  site_extended_description: ''
  site_terms: ''
  site_contact_username: ''
  site_contact_email: ''
  registrations_mode: 'open'
  profile_directory: true
  closed_registrations_message: ''
  open_deletion: true
  min_invite_role: 'admin'
  timeline_preview: true
  show_staff_badge: true
  default_sensitive: false
  unfollow_modal: false
  boost_modal: false
  delete_modal: true
  auto_play_gif: false
  display_media: 'default'
  expand_spoilers: false
  preview_sensitive_media: false
  reduce_motion: false
  disable_swiping: false
  show_application: true
  system_font_ui: false
  noindex: false
  theme: 'default'
  aggregate_reblogs: true
  advanced_layout: false
  use_blurhash: true
  use_pending_items: false
  trends: true
  trendable_by_default: false
  crop_images: true
  notification_emails:
    follow: true
    reblog: false
    favourite: false
    mention: true
    follow_request: true
    digest: true
    report: true
    pending_account: true
    trending_tag: true
    appeal: true
  always_send_emails: false
  interactions:
    must_be_follower: false
    must_be_following: false
    must_be_following_dm: false
  reserved_usernames:
    - admin
    - support
    - help
    - root
    - webmaster
    - administrator
    - mod
    - moderator
  disallowed_hashtags: # space separated string or list of hashtags without the hash
  bootstrap_timeline_accounts: ''
  activity_api_enabled: true
  peers_api_enabled: true
  show_domain_blocks: 'disabled'
  show_domain_blocks_rationale: 'disabled'
  require_invite_text: false
  backups_retention_period: 7

development:
  <<: *defaults

test:
  <<: *defaults

production:
  <<: *defaults

/settings/preferences/notificationsで設定できる項目のようです。

<h4>その他の通知設定</h4>
<div class='fields-group'>
<div class="input with_label boolean optional user_interactions_must_be_follower"><div class="label_input"><label class="boolean optional" for="user_interactions_must_be_follower">フォロワー以外からの通知をブロック</label><div class="label_input__wrapper"><input value="0" autocomplete="off" type="hidden" name="user[interactions][must_be_follower]" /><label class="checkbox"><input class="boolean optional" type="checkbox" value="1" name="user[interactions][must_be_follower]" id="user_interactions_must_be_follower" /></label></div></div></div>
zundazunda

Specを書く

以前、branding_controller_spec.rbを参考にmedia_cache_retention_periodを整数にするSpecを書いた。似たものをユーザー用に書く必要がありそう。

# spec/controllers/admin/settings/content_retention_controller_spec.rb
# frozen_string_literal: true

require 'rails_helper'

RSpec.describe Admin::Settings::ContentRetentionController, type: :controller do
  render_views

  describe 'When signed in as an admin' do
    before do
      sign_in Fabricate(:user, role: UserRole.find_by(name: 'Admin')), scope: :user
    end

    describe 'GET #show' do
      it 'returns http success' do
        get :show

        expect(response).to have_http_status(200)
      end
    end

    describe 'PUT #update' do
      before do
        allow_any_instance_of(Form::AdminSettings).to receive(:valid?).and_return(true)
      end

      around do |example|
        before = Setting.media_cache_retention_period
        example.run
        Setting.media_cache_retention_period = before
      end

      it 'saves a settings value as an Integer' do
        patch :update, params: { form_admin_settings: { media_cache_retention_period: '42' } }

        expect(response).to redirect_to(admin_settings_content_retention_path)
        expect(Setting.media_cache_retention_period).to eq 42
      end

      it 'saves a blank value as a blank' do
        patch :update, params: { form_admin_settings: { media_cache_retention_period: '' } }

        expect(response).to redirect_to(admin_settings_content_retention_path)
        expect(Setting.media_cache_retention_period).to eq ''
      end
    end
  end
end

既存のspec/controllers/settings/preferences/notifications_controller_spec.rbが落ちてくれそうだ。

# spec/controllers/settings/preferences/notifications_controller_spec.rb
require 'rails_helper'

describe Settings::Preferences::NotificationsController do
  render_views

  let(:user) { Fabricate(:user) }

  before do
    sign_in user, scope: :user
  end

  describe 'GET #show' do
    it 'returns http success' do
      get :show
      expect(response).to have_http_status(200)
    end
  end

  describe 'PUT #update' do
    it 'updates notifications settings' do
      user.settings['notification_emails'] = user.settings['notification_emails'].merge('follow' => false)
      user.settings['interactions'] = user.settings['interactions'].merge('must_be_follower' => true)

      put :update, params: {
        user: {
          notification_emails: { follow: '1' },
          interactions: { must_be_follower: '0' },
        }
      }

      expect(response).to redirect_to(settings_preferences_notifications_path)
      user.reload
      expect(user.settings['notification_emails']['follow']).to be true
      expect(user.settings['interactions']['must_be_follower']).to be false
    end
  end
end
zundazunda

Rubyを更新してテストを落とす

$ vi Gemfile
$ git diff Gemfile
diff --git a/Gemfile b/Gemfile
index 07c300510..34123eac8 100644
--- a/Gemfile
+++ b/Gemfile
@@ -1,7 +1,7 @@
 # frozen_string_literal: true
 
 source 'https://rubygems.org'
-ruby '>= 2.6.0', '< 3.1.0'
+ruby '>= 2.6.0', '< 3.2.0'
 
 gem 'pkg-config', '~> 1.4'
 gem 'rexml', '~> 3.2'
@@ -12,6 +12,8 @@ gem 'sprockets', '~> 3.7.2'
 gem 'thor', '~> 1.2'
 gem 'rack', '~> 2.2.4'
 
+gem 'mail', git: 'https://github.com/Shopify/mail.git', branch: 'net-smtp-dependency'
+
 gem 'hamlit-rails', '~> 0.2'
 gem 'pg', '~> 1.4'
 gem 'makara', '~> 0.5'
$ rbenv local 3.1.2
$ bundle install --path=vendor/bundle
  :
Your bundle is locked to mail (2.7.1) from https://github.com/Shopify/mail.git (at net-smtp-dependency@651a59a), but that version
can no longer be found in that source. That means the author of mail (2.7.1) has removed it. You'll need to update your bundle to
a version other than mail (2.7.1) that hasn't been removed in order to install.

ぎゃー。なにか警告が出てる。

https://github.com/mastodon/mastodon/pull/21061 を参考に、Gemfileを下記のように編集した。

diff --git a/Gemfile b/Gemfile
index 07c300510..00459226e 100644
--- a/Gemfile
+++ b/Gemfile
@@ -1,7 +1,7 @@
 # frozen_string_literal: true
 
 source 'https://rubygems.org'
-ruby '>= 2.6.0', '< 3.1.0'
+ruby '>= 2.6.0', '< 3.2.0'
 
 gem 'pkg-config', '~> 1.4'
 gem 'rexml', '~> 3.2'
@@ -12,6 +12,9 @@ gem 'sprockets', '~> 3.7.2'
 gem 'thor', '~> 1.2'
 gem 'rack', '~> 2.2.4'
 
+gem 'net-smtp', require: false
+gem 'premailer-rails'
+
 gem 'hamlit-rails', '~> 0.2'
 gem 'pg', '~> 1.4'
 gem 'makara', '~> 0.5'

$ bundle install --path=vendor/bundle
$ bundle exec rspec spec/controllers/settings/preferences/notifications_controller_spec.rb
Calling `DidYouMean::SPELL_CHECKERS.merge!(error_name => spell_checker)' has been deprecated. Please call `DidYouMean.correct_error(error_name, spell_checker)' instead.

Randomized with seed 905
                                                                                                                                  
  1) Settings::Preferences::NotificationsController PUT #update updates notifications settings
     Failure/Error:
             val = Rails.cache.fetch(cache_key(key, nil)) do
               db_val = object(key)
       
               if db_val
                 default_value = default_settings[key]
       
                 return default_value.with_indifferent_access.merge!(db_val.value) if default_value.is_a?(Hash)
                 db_val.value
               else
                 default_settings[key]
     
     Psych::BadAlias:
       Unknown alias: defaults
     # ./app/models/setting.rb:26:in `[]'
     # ./app/validators/unreserved_username_validator.rb:21:in `reserved_username?'
     # ./app/validators/unreserved_username_validator.rb:9:in `validate'
     # ./spec/controllers/settings/preferences/notifications_controller_spec.rb:6:in `block (2 levels) in <top (required)>'
     # ./spec/controllers/settings/preferences/notifications_controller_spec.rb:9:in `block (2 levels) in <top (required)>'
     # ------------------
     # --- Caused by: ---
     # NoMethodError:
     #   undefined method `reserved_usernames' for Setting:Class
     #   Did you mean?  reset_sequence_name
     #   ./app/validators/unreserved_username_validator.rb:21:in `reserved_username?'

                                                                                                                                  
  2) Settings::Preferences::NotificationsController GET #show returns http success
     Failure/Error:
             val = Rails.cache.fetch(cache_key(key, nil)) do
               db_val = object(key)
       
               if db_val
                 default_value = default_settings[key]
       
                 return default_value.with_indifferent_access.merge!(db_val.value) if default_value.is_a?(Hash)
                 db_val.value
               else
                 default_settings[key]
     
     Psych::BadAlias:
       Unknown alias: defaults
     # ./app/models/setting.rb:26:in `[]'
     # ./app/validators/unreserved_username_validator.rb:21:in `reserved_username?'
     # ./app/validators/unreserved_username_validator.rb:9:in `validate'
     # ./spec/controllers/settings/preferences/notifications_controller_spec.rb:6:in `block (2 levels) in <top (required)>'
     # ./spec/controllers/settings/preferences/notifications_controller_spec.rb:9:in `block (2 levels) in <top (required)>'
     # ------------------
     # --- Caused by: ---
     # NoMethodError:
     #   undefined method `reserved_usernames' for Setting:Class
     #   Did you mean?  reset_sequence_name
     #   ./app/validators/unreserved_username_validator.rb:21:in `reserved_username?'

 2/2 |================================================== 100 ===================================================>| Time: 00:00:04 

Finished in 4.1 seconds (files took 3.62 seconds to load)
2 examples, 2 failures

Failed examples:

rspec ./spec/controllers/settings/preferences/notifications_controller_spec.rb:20 # Settings::Preferences::NotificationsController PUT #update updates notifications settings
rspec ./spec/controllers/settings/preferences/notifications_controller_spec.rb:13 # Settings::Preferences::NotificationsController GET #show returns http success

Randomized with seed 905
Coverage report generated for RSpec to /home/zunda/c/src/github.com/zunda/mastodon/coverage. 1397 / 35020 LOC (3.99%) covered.
Stopped processing SimpleCov as a previous error not related to SimpleCov has been detected

よしよし。

$ rbenv local 3.0.4
$ bundle install --path=vendor/bundle
$ bundle exec rspec spec/controllers/settings/preferences/notifications_controller_spec.rb

Randomized with seed 26161
 2/2 |================================================== 100 ===================================================>| Time: 00:00:04 

Finished in 4.61 seconds (files took 3.44 seconds to load)
2 examples, 0 failures

Randomized with seed 26161
Coverage report generated for RSpec to /home/zunda/c/src/github.com/zunda/mastodon/coverage. 2191 / 34521 LOC (6.35%) covered.
zunda@misoan:~/c/src/github.com/zunda/mastodon (update-rails-settings-cached *)

Ruby 3.0では通る。よしよし。

zundazunda

Ruby 3.1でテストを通す

下記の変更でテストが通るようになるが、Ruby 3.0では走らない。だが、rails-settings-cachedのみが問題であることはわかる。

diff -ur rails-settings-cached-0.6.6.orig/lib/rails-settings/default.rb rails-settings-cached-0.6.6/lib/rails-settings/default.rb
--- rails-settings-cached-0.6.6.orig/lib/rails-settings/default.rb	2022-11-20 11:16:19.242409196 -1000
+++ rails-settings-cached-0.6.6/lib/rails-settings/default.rb	2022-11-20 14:18:58.867986771 -1000
@@ -38,7 +38,7 @@
 
     def initialize
       content = open(self.class.source_path).read
-      hash = content.empty? ? {} : YAML.load(ERB.new(content).result).to_hash
+      hash = content.empty? ? {} : YAML.unsafe_load(ERB.new(content).result).to_hash
       hash = hash[Rails.env] || {}
       replace hash
     end
diff -ur rails-settings-cached-0.6.6.orig/lib/rails-settings/settings.rb rails-settings-cached-0.6.6/lib/rails-settings/settings.rb
--- rails-settings-cached-0.6.6.orig/lib/rails-settings/settings.rb	2022-11-20 11:16:19.242409196 -1000
+++ rails-settings-cached-0.6.6/lib/rails-settings/settings.rb	2022-11-20 14:19:07.748102237 -1000
@@ -8,7 +8,7 @@
 
     # get the value field, YAML decoded
     def value
-      YAML.load(self[:value]) if self[:value].present?
+      YAML.unsafe_load(self[:value]) if self[:value].present?
     end
 
     # set the value field, YAML encoded

パース対象をpするとRubyの型指定のあるものはなく、&defaultみたいな参照を扱えていないようだ。

zundazunda

rails-settings-cachedを更新する

https://github.com/huacnlee/rails-settings-cached/releases/tag/v2.0.0

  • New design release.
  • No more scope support (RailsSettings::Extend has removed);
  • No more YAML file.
  • Requuire Ruby 2.5+, Rails 5.0+
  • You must use field method to statement the setting keys before use.

例えばspec/controllers/settings/preferences/notifications_controller_spec.rbを通そうとするとconfig/settings.ymlの代わりに、

diff --git a/app/models/setting.rb b/app/models/setting.rb
index 4bcaa060f..6dd645c77 100644
--- a/app/models/setting.rb
+++ b/app/models/setting.rb
@@ -13,7 +13,23 @@
 #
 
 class Setting < RailsSettings::Base
-  source Rails.root.join('config', 'settings.yml')
+  field :interactions, type: :hash, default: {
+    must_be_follower: false,
+    must_be_following: false,
+    must_be_following_dm: false
+  }
+  field :reserved_usernames, type: :array, default: %w(
+    admin
+    support
+    help
+    root
+    webmaster
+    administrator
+    mod
+    moderator
+  )
+  field :require_invite_text, type: :boolean, default: false
+  field :registrations_mode, default: 'open'
 
   def to_param
     var

のような記述をすることになるようだ。

下記のような変更で設定にスコープを導入できるのだろうか。

diff --git a/app/lib/settings/scoped_settings.rb b/app/lib/settings/scoped_settings.rb
index 1e18d6d46..fd6da0d38 100644
--- a/app/lib/settings/scoped_settings.rb
+++ b/app/lib/settings/scoped_settings.rb
@@ -7,6 +7,15 @@ module Settings
       noindex
     ).freeze
 
+    # rails-settings-cached-0.6.6/lib/rails-settings/base.rb:26
+    def self.cache_key(var_name, scope_object)
+      key_parts = [Setting.cache_key]
+      key_parts << "#{scope_object.class.name}-#{scope_object.id}" if scope_object
+      key_parts << var_name.to_s
+      key_parts.join('/')
+    end
+
+
     def initialize(object)
       @object = object
     end
@@ -45,11 +54,11 @@ module Settings
       record = thing_scoped.find_or_initialize_by(var: key)
       record.update!(value: value)
 
-      Rails.cache.write(Setting.cache_key(key, @object), value)
+      Rails.cache.write(ScopedSettings.cache_key(key, @object), value)
     end
 
     def [](key)
-      Rails.cache.fetch(Setting.cache_key(key, @object)) do
+      Rails.cache.fetch(ScopedSettings.cache_key(key, @object)) do
         db_val = thing_scoped.find_by(var: key.to_s)
         if db_val
           default_value = ScopedSettings.default_settings[key]
diff --git a/app/models/setting.rb b/app/models/setting.rb
index 4bcaa060f..f46cea567 100644
--- a/app/models/setting.rb
+++ b/app/models/setting.rb
@@ -13,7 +13,23 @@
 #
 
 class Setting < RailsSettings::Base
-  source Rails.root.join('config', 'settings.yml')
+  field :interactions, type: :hash, default: {
+    must_be_follower: false,
+    must_be_following: false,
+    must_be_following_dm: false
+  }
+  field :reserved_usernames, type: :array, default: %w(
+    admin
+    support
+    help
+    root
+    webmaster
+    administrator
+    mod
+    moderator
+  )
+  field :require_invite_text, type: :boolean, default: false
+  field :registrations_mode, default: 'open'
 
   def to_param
     var
@@ -23,7 +39,7 @@ class Setting < RailsSettings::Base
     def [](key)
       return super(key) unless rails_initialized?
 
-      val = Rails.cache.fetch(cache_key(key, nil)) do
+      val = Rails.cache.fetch(Settings::ScopedSettings.cache_key(key, nil)) do
         db_val = object(key)
 
         if db_val
このスクラップは1ヶ月前にクローズされました