Releases: teamintricately/resque-unique_in_queue
Releases · teamintricately/resque-unique_in_queue
v2.0.2
Using SCAN instead of KEYS at remove_queue (#1) *** Why is this change necessary? If you have a big collection of keys to be fetched and deleted from Redis two problems will occur: 1. The server will be blocked due to the use of KEYS command since it takes a long time to return for big collections; 2. Using the splat ("*") operator for a large array raises a SystemStackError Ultimately the queue is removed but the keys remain there: ``` 2.3.8 :057 > Resque.size :test_queue => 300001 2.3.8 :058 > Resque.remove_queue :test_queue SystemStackError: stack level too deep from /root/project/vendor/bundle/ruby/2.3.0/gems/resque-unique_in_queue-2.0.1/lib/resque/uniqu e_in_queue/queue.rb:68:in `cleanup' from /root/project/vendor/bundle/ruby/2.3.0/gems/resque-unique_in_queue-2.0.1/lib/resque/uniqu e_in_queue/resque_ext/resque.rb:35:in `remove_queue_with_unique_in_queue_cleanup' from /root/project/lib/ext/hacks/resque.rb:115:in `remove_queue_rewrite' from (irb):58 from /root/project/vendor/bundle/ruby/2.3.0/gems/railties-4.2.11.1/lib/rails/commands/console. rb:110:in `start' from /root/project/vendor/bundle/ruby/2.3.0/gems/railties-4.2.11.1/lib/rails/commands/console. rb:9:in `start' from /root/project/vendor/bundle/ruby/2.3.0/gems/railties-4.2.11.1/lib/rails/commands/commands _tasks.rb:68:in `console' from /root/project/vendor/bundle/ruby/2.3.0/gems/railties-4.2.11.1/lib/rails/commands/commands _tasks.rb:39:in `run_command!' from /root/project/vendor/bundle/ruby/2.3.0/gems/railties-4.2.11.1/lib/rails/commands.rb:17:in `<top (required)>' from script/rails:9:in `require' from script/rails:9:in `<main>' 2.3.8 :059 > Resque.size :test_queue => 0 2.3.8 :060 > Resque.redis.keys("#{Resque::UniqueInQueue.configuration&.unique_in_queue_key_base}:queue:test_queue:job:*").count => 300001 ``` *** How does it address the issue? 1. Replacing KEYS by SCAN command 2. Removing splat operator. *** What side effects does this change have? The removal process for large collections will take a lot longer: ``` irb(main):001:0> Benchmark.measure { Resque.remove_queue :test_queue } => #<Benchmark::Tms:0x000055b793b3cd60 @label="", @real=167.7889306979996, @cstime=0.0, @cutime=0.0, @stime=0.7600000000000002, @utime=11.500000000000002, @total=12.260000000000002> ```