Embedding

Starting in 7.0, Sidekiq can be embedded within another process so you do not need to run a separate Sidekiq process. This is called embedding.

Definition

The process will typically have some sort of lifecycle callbacks which you can use to create, start and stop the Sidekiq instance.

Puma

# config/puma.rb 
workers 2
threads 1, 3

# preloading the application is necessary to ensure
# the configuration in your initializer runs before
# the boot callback below.
preload_app!

x = nil
on_worker_boot do
  x = Sidekiq.configure_embed do |config|
    # config.logger.level = Logger::DEBUG
    config.queues = %w[critical default low]
    config.concurrency = 2
  end
  x.run
end

on_worker_shutdown do
  x&.stop
end

Embedding does not currently work with Puma's phased restart feature.

Passenger

# config/application.rb
if defined?(PhusionPassenger)
  x = nil
  PhusionPassenger.on_event(:starting_worker_process) do
    x = Sidekiq.configure_embed do |config|
      # config.logger.level = Logger::DEBUG
      config.queues = %w[critical default low]
      config.concurrency = 2
    end
    x.run
  end

  PhusionPassenger.on_event(:stopping_worker_process) do
    x&.stop
  end
end

I strongly recommend keeping your concurrency very low, i.e. 1-2. Embedding Sidekiq within an MRI process means that it shares one GVL with all other threads within the process. A good rule of thumb is that your puma threads + sidekiq concurrency should never be greater than 5. You'll create a puma process for every core so if your machine has 8 cores, you'll have 8*3 = 24 Puma threads and 8*2 = 16 Sidekiq threads. As with any rule of thumb, YMMV.

Notes

• The process owner owns any signal traps; the embedded Sidekiq instance does not directly respond to Ctrl-C, TERM or any other signal.
• The embedded instance does not look at config/sidekiq.yml or command line arguments. You can only configure it via Ruby.
• Sidekiq.server? will return false because this is not a Sidekiq process but configure_server callbacks will still be run so gems can install their middleware and extensions.
• Graceful restarts (a Sidekiq Enterprise feature) are not supported because we do not have full control of the process.