Attentive Sidekiq

Be aware of suddenly disappeared sidekiq jobs!

About

The case of disappearing sidekiq jobs was proved by github issues, stackoverflow questions and (sadly) personal experience. Attentive Sidekiq was made to protect your jobs and critical data from silent disappearing. In case there appears a job being started but not finished and not being processing at the moment, you will know this.

Usage

Attentive Sidekiq provides you with a couple of useful API methods.

To get a hash containing all information about jobs marked as disappeared:

AttentiveSidekiq::Disappeared.jobs

To get only JIDs of lost jobs:

AttentiveSidekiq::Disappeared.job_ids

To place a disappeared job back into queue:

AttentiveSidekiq::Disappeared.requeue(jid)

To remove a job from disappeared hash (e.g. after manual requeue):

AttentiveSidekiq::Disappeared.remove(jid)

Sidekiq Web integration

You may also watch info about disappeared jobs in a web UI. Simply make sure you have Sidekiq UI enabled, then head right to the Disappeared Jobs tab in the navbar. The Web UI uses the API exclusively: anything you can do in the UI can be scripted with the API.

Pre-requirements and notes

• Attentive Sidekiq assumes you've got Sidekiq installed already.
• You should make sure sidekiq process is started in order for disappeared jobs updater to work properly.
• It was tested with Sidekiq version 4. Seamless functionality with lower sidekiq versions is not guaranteed.

Installation

Add this line to your application's Gemfile:

gem 'attentive_sidekiq'

And then execute:

$ bundle

Configure your middleware chains, lookup Middleware usage on Sidekiq wiki for more info.

Sidekiq.configure_server do |config|
  config.server_middleware do |chain|
    chain.add AttentiveSidekiq::Middleware::Server::Attentionist
  end
end

After that you can use your jobs as usual.

Configuration

Specify desired configuration inside of sidekiq.yml file:

attentive:
  # Time in seconds between checks for disappeared jobs
  :execution_interval: 300  # default: 600
  # Time limit in seconds to perform disappeared jobs check
  :timeout_interval: 25     # default: 60

By default, Attentive Sidekiq uses Sidekiq.logger to log its work. You may change it in your initializer:

AttentiveSidekiq.logger = Logger.new("log/attentive_sidekiq.log")

Suggestions? Bugs?

If you've got a question, feature suggestion or found a bug please add an issue on GitHub or fork the project and send a pull request.