Detect N+1 with prosopite gem and rspec

Post by Lưu Đại at 21-04-2023

1. Prosopite gem

  • Why didn't I use Bullet? 
  • That's because when I write this article bullet is not support Rails 7. 

Gem Load Error is: Bullet does not support active_record 7.0.3.1 yet
  • Gem prosopite has the same purpose as Bullet. It's used to find N+1 errors. Prosopite find N+1 base on project logs so if there're N+1 but the data only have 1 child record + 1 parent record prosopite won't fire error. 
  • Gem prosopite can be used to find N+1 when run Unit test, by the way in order to detect all of them we need to write unit test carefully. 

2. N+1 query

  • N+1 contains 2 parts: A query that follows by multiple N queries to fetch child records. By default Rails use lazy load that means whenever it needs data it will take exact that data from the database. When we loop through N child records it will take one records out at a time and results N queries to fetch N child records. 
  • Lazy load is not always bad. For example, when parent record have 3000 child records but we only need 10 of those. Event the number of queries reduce when we preload it the time execute 2 queries (one of them is tremendous) is longer than 11 small queries. In this case lazy load help us increase the performance. 

3. Find N+1 queries through prosipite gem and rspec

  • Prosopite gem has 2 mode. One of them is only warning when N+1 is detected and other will raise exception. Warning mode will only write the red warning to log. To switch between 2 modes, I use an environment variable STRICT_N_PLUS_ONE. When this variable has value it will raise exception when meet N+1, in the other hand it just warning.
  • In order to check N+1 for normal requests, I add before_action to each requests
  if Rails.env.development?
    before_action { Prosopite.scan }
    after_action { Prosopite.finish }
  end
  • Config warning mode / raise exception in config/enviroments/development.rb
  config.after_initialize do
    Prosopite.rails_logger = true
    Prosopite.raise = ENV['STRICT_N_PLUS_ONE'] || false
  end

  • Same config for test environment config/enviroments/test.rb
  config.after_initialize do
    Prosopite.rails_logger = true
    Prosopite.raise = ENV['STRICT_N_PLUS_ONE'] || false
  end
  • Config in spec/spec_helper.rb for all test cases.
  config.before(:each) do
    Prosopite.scan if ENV['STRICT_N_PLUS_ONE']
  end

  config.after(:each) do
    Prosopite.finish if ENV['STRICT_N_PLUS_ONE']
  end