RailsのFeature specとSystem specは、内部的に何か違うのか？

Yusuke Iwaki 2021/07/25に更新

Feature Spec では、JavaScript が有効な Capybara ドライバー（Selenium や Poltergeist）を使った場合、テストは Rails とは別のプロセスで実行されます。その結果、テストプロセスと Rails プロセス間でデータベースのトランザクションが共有されないため、 RSpec 標準のデータベースロールバック機構を利用できず、代わりに Database Cleaner のような gem を使う必要がありました。 Rails 開発チームは System Test でこのような問題が発生しないように実装をしました。そのおかげで、System Spec では別の gem を利用することなく RSpec のロールバック機構を利用できます

これ。ほんとかな

→結論：うそだった。Rails 5.1以降であればFeature specもDatabaseCleanerいらなさそう。

（後述の調査もあるし、Stack Overflowにもそう書いてる。）

Yusuke Iwaki 2021/07/25

まずはこのアプリケーションで調査。（Feature specでもSystem specでもない）

spec/integration/example_spec.rb

  it 'can browse' do
    page.goto("#{base_url}/test")
    require 'pry'
    binding.pry
    page.wait_for_selector('input', visible: true)

スレッドダンプはsigdumpで調べる。

From: /Users/yusuke-iwaki/Desktop/railsfeature/spec/integration/example_spec.rb:185 :

    180:   let(:page) { @puppeteer_page }
    181: 
    182:   it 'can browse' do
    183:     page.goto("#{base_url}/test")
    184:     require 'pry'
 => 185:     binding.pry
    186:     page.wait_for_selector('input', visible: true)
    187:     page.type_text('input', 'hoge')
    188:     page.keyboard.press('Enter')
    189:     expect(page.eval_on_selector('#content', 'el => el.textContent')).to include('hoge')
    190:   end

[1] pry(#<RSpec::ExampleGroups::Example>)> Process.pid
=> 55322

kill -CONT 55322 すると /tmp/sigdump-55322.log が吐かれる。

/tmp/sigdump-55322.log

Sigdump at 2021-07-25 15:35:24 +0900 process 55322 (/Users/yusuke-iwaki/.rbenv/versions/3.0.0/bin/rspec)
  Thread #<Thread:0x00007fbace880140 run> status=run priority=0
      /Users/yusuke-iwaki/Desktop/railsfeature/spec/integration/example_spec.rb:55:in `backtrace'
      /Users/yusuke-iwaki/Desktop/railsfeature/spec/integration/example_spec.rb:55:in `dump_backtrace'
　　（略）

  Thread #<Thread:0x00007fbad5348388@puma reactor /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/puma-5.3.2/lib/puma/reactor.rb:37 sleep> status=sleep priority=0
      /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/puma-5.3.2/lib/puma/reactor.rb:75:in `select'
      /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/puma-5.3.2/lib/puma/reactor.rb:75:in `select_loop'
      /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/puma-5.3.2/lib/puma/reactor.rb:39:in `block in run'
  Thread #<Thread:0x00007fbad5343f40@puma threadpool reaper /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/puma-5.3.2/lib/puma/thread_pool.rb:305 sleep> status=sleep priority=0
      /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/puma-5.3.2/lib/puma/thread_pool.rb:309:in `sleep'
      /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/puma-5.3.2/lib/puma/thread_pool.rb:309:in `block in start!'
  Thread #<Thread:0x00007fbad5343b08@puma threadpool trimmer /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/puma-5.3.2/lib/puma/thread_pool.rb:305 sleep> status=sleep priority=0
      /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/puma-5.3.2/lib/puma/thread_pool.rb:309:in `sleep'
      /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/puma-5.3.2/lib/puma/thread_pool.rb:309:in `block in start!'
  Thread #<Thread:0x00007fbad5343888@puma server /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/puma-5.3.2/lib/puma/server.rb:257 sleep> status=sleep priority=0
      /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/puma-5.3.2/lib/puma/server.rb:328:in `select'
      /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/puma-5.3.2/lib/puma/server.rb:328:in `handle_servers'
      /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/puma-5.3.2/lib/puma/server.rb:259:in `block in run'
  Thread #<Thread:0x00007fbad53bb658@puma threadpool 001 /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/puma-5.3.2/lib/puma/thread_pool.rb:103 sleep_forever> status=sleep priority=0
      /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/puma-5.3.2/lib/puma/thread_pool.rb:131:in `sleep'
      /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/puma-5.3.2/lib/puma/thread_pool.rb:131:in `wait'
      /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/puma-5.3.2/lib/puma/thread_pool.rb:131:in `block (2 levels) in spawn_thread'
      /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/puma-5.3.2/lib/puma/thread_pool.rb:116:in `synchronize'
      /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/puma-5.3.2/lib/puma/thread_pool.rb:116:in `block in spawn_thread'

同一プロセスで、スレッドが分かれてRSpecとpumaが動いてる。

Yusuke Iwaki 2021/07/25

Feature spec

From: /Users/yusuke-iwaki/Desktop/railsfeature/spec/features/example_spec.rb:171 :

    166:   end
    167: 
    168:   it 'can browse' do
    169:     visit '/test'
    170:     require 'pry'
 => 171:     binding.pry
    172:     find('input').set('hoge')
    173:     send_keys(:enter)
    174:     expect(find('#content')).to have_text('hoge')
    175:   end
    176: end

[1] pry(#<RSpec::ExampleGroups::Example>)> Process.pid
=> 57865

/tmp/sigdump-57865.log

Sigdump at 2021-07-25 21:25:08 +0900 process 57865 (/Users/yusuke-iwaki/.rbenv/versions/3.0.0/bin/rspec)
  Thread #<Thread:0x00007f8ea8080140 run> status=run priority=0
      /Users/yusuke-iwaki/Desktop/railsfeature/spec/features/example_spec.rb:56:in `backtrace'
      /Users/yusuke-iwaki/Desktop/railsfeature/spec/features/example_spec.rb:56:in `dump_backtrace'
　　（略）
  Thread #<Thread:0x00007f8ea865d068@puma reactor /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/puma-5.3.2/lib/puma/reactor.rb:37 sleep> status=sleep priority=0
      /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/puma-5.3.2/lib/puma/reactor.rb:75:in `select'
      /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/puma-5.3.2/lib/puma/reactor.rb:75:in `select_loop'
      /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/puma-5.3.2/lib/puma/reactor.rb:39:in `block in run'
  Thread #<Thread:0x00007f8ea865cc30@puma threadpool reaper /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/puma-5.3.2/lib/puma/thread_pool.rb:305 sleep> status=sleep priority=0
      /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/puma-5.3.2/lib/puma/thread_pool.rb:309:in `sleep'
      /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/puma-5.3.2/lib/puma/thread_pool.rb:309:in `block in start!'
  Thread #<Thread:0x00007f8ea865c050@puma threadpool trimmer /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/puma-5.3.2/lib/puma/thread_pool.rb:305 sleep> status=sleep priority=0
      /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/puma-5.3.2/lib/puma/thread_pool.rb:309:in `sleep'
      /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/puma-5.3.2/lib/puma/thread_pool.rb:309:in `block in start!'
  Thread #<Thread:0x00007f8ea8656c18@puma server /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/puma-5.3.2/lib/puma/server.rb:257 sleep> status=sleep priority=0
      /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/puma-5.3.2/lib/puma/server.rb:328:in `select'
      /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/puma-5.3.2/lib/puma/server.rb:328:in `handle_servers'
      /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/puma-5.3.2/lib/puma/server.rb:259:in `block in run'
  Thread #<Thread:0x00007f8eae85ddd0@puma threadpool 001 /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/puma-5.3.2/lib/puma/thread_pool.rb:103 sleep_forever> status=sleep priority=0
      /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/puma-5.3.2/lib/puma/thread_pool.rb:131:in `sleep'
      /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/puma-5.3.2/lib/puma/thread_pool.rb:131:in `wait'
      /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/puma-5.3.2/lib/puma/thread_pool.rb:131:in `block (2 levels) in spawn_thread'
      /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/puma-5.3.2/lib/puma/thread_pool.rb:116:in `synchronize'
      /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/puma-5.3.2/lib/puma/thread_pool.rb:116:in `block in spawn_thread'
  Thread #<Process::Waiter:0x00007f8ea879ae30 sleep> status=sleep priority=0
  Thread #<Thread:0x00007f8ead95c9d0 /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/puppeteer-ruby-0.35.1/lib/puppeteer/web_socket.rb:62 sleep> status=sleep priority=0
      /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/puppeteer-ruby-0.35.1/lib/puppeteer/web_socket.rb:44:in `readpartial'
      /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/puppeteer-ruby-0.35.1/lib/puppeteer/web_socket.rb:44:in `readpartial'
      /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/puppeteer-ruby-0.35.1/lib/puppeteer/web_socket.rb:96:in `wait_for_data'
      /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/puppeteer-ruby-0.35.1/lib/puppeteer/web_socket.rb:63:in `block in initialize'
  Thread #<Thread:0x00007f8ead8bc638@io-worker-1 /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/concurrent-ruby-1.1.9/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:332 sleep_forever> status=sleep priority=0
      /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/concurrent-ruby-1.1.9/lib/concurrent-ruby/concurrent/promises.rb:775:in `sleep'
      /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/concurrent-ruby-1.1.9/lib/concurrent-ruby/concurrent/promises.rb:775:in `wait'
      /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/concurrent-ruby-1.1.9/lib/concurrent-ruby/concurrent/promises.rb:775:in `block in wait_until_resolved'
      /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/concurrent-ruby-1.1.9/lib/concurrent-ruby/concurrent/promises.rb:771:in `synchronize'
      /Users/yusuke-iwaki/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/concurrent-ruby-1.1.9/lib/concurrent-ruby/concurrent/promises.rb:771:in ```

やはり同一プロセスで、スレッドが分かれてRSpecとpumaが動いてる。

Yusuke Iwaki 2021/07/25に更新

System Test導入のPull Request

ActiveRecordのコネクション共有は↓ setup_fixtures, teardown_fixturesフックでスレッドを固定化/解除している。

Yusuke Iwaki 2021/07/25

そもそもFeature specとSystem specは誰が何を動かしているのか。

Rackサーバー定義

Feature spec

Capybaraに定義がある

Capybara.app = Rack::Builder.new do
  map '/' do
    run Rails.application
  end
end.to_app

これを、rspec-railsが引き込んで使っている

Capybara.serverはユーザが明示的に指定する。通常は Capybara.server = :puma

System spec

RailsのSystemTestCaseがやっている。

    def self.start_application # :nodoc:
      Capybara.app = Rack::Builder.new do
        map "/" do
          run Rails.application
        end
      end

      SystemTesting::Server.new.run
    end

Capybaraに

Capybara.server = :puma, { Silent: true }

を指定しているだけ。

Railsサーバー起動

これはFeature specもSystem specも共通。Capybara::Sessionの初期化時に起動する。

Capybara::Sessionは Capybara.current_session に初回参照時 に生成される。

Capybara DSLを使用している場合には大抵、最初にDSLを使用したときにサーバーが起動する。

Yusuke Iwaki 2021/07/25

実際に違うのは、システムテストの

    include SystemTesting::TestHelpers::SetupAndTeardown
    include SystemTesting::TestHelpers::ScreenshotHelper

ここくらいだろう。失敗時のスクリーンショット保存。

Yusuke Iwaki 2021/08/21に更新

rspec-railsの

feature spec

system spec

このスクラップは2021/07/25にクローズされました