Start new orphan reaper process if orphan reaper is not running

[AlexFlint73]

If a sidekiq process that runs orphan reaper stops unexpectedly and the new process is up within reaper_interval, new orphan reaper won't start.

We've faced a rare issue, when the sidekiq process, that went down, was running both orphan reaper and a job, that needs to be unique. This led to leaving an orphaned lock and the job became stuck.

I believe, this PR can fix this. Each sidekiq process will run reaper ressurector, that starts a new orphan reaper if it looks like there is no reaper process running.

[mhenrixon]

This is brilliant, the only thing that might be improved would be that there seems to be a little duplication.

I haven't had time to properly review the code on my computer but I'll get to it as soon as I can.

It would be nice to avoid the additional thread but it could be considered an optional feature.

Thank you

[AlexFlint73]

Thanks a lot for your reply.

I've thought on how an additional thread can be avoided. However, it looks like a running thread is needed if a new reaper should be started on the fly.

Another possible solution can be running an instance of orphan reaper on each sidekiq process and use mutex in redis to make sure only one reaper checks locks during single iteration. Such a solution will cost 1 thread less.

As to duplication, I see what you mean. Will work on that.

[mhenrixon]

Some initial feedback for you :)