Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optionally fork before running each job #5

Closed
bkeepers opened this issue Sep 23, 2011 · 6 comments
Closed

Optionally fork before running each job #5

bkeepers opened this issue Sep 23, 2011 · 6 comments
Labels

Comments

@bkeepers
Copy link
Owner

This should be an option that is easy to turn on or off.

@johnae
Copy link

johnae commented Mar 29, 2012

Or threads for jruby?

@bkeepers
Copy link
Owner Author

Sure, I'm not opposed to that.

@DeTeam
Copy link

DeTeam commented Feb 5, 2013

Possible configuration option:

Qu.process_with = lambda do |job_closure|
  Thread.new { job_closure.call }
end


Qu.process_with = lambda do |job_closure|
  job_closure.call if fork
end

By default it's just .call on arg.
What do you think?

@bkeepers
Copy link
Owner Author

bkeepers commented Feb 5, 2013

I think I would rather see it wrapped up in a class so it can be tested and extended. I have a hunch the different strategies will be more complicated than you'd want to handle in a lambda.

class ThreadedProcessor
  def initializer(max_threads = 5)
    @max_threads = max_threads
  end

  def run(job)
    @threads << Thread.new { job.process }
    wait_until_a_thread_dies if at_max_threads?
  end

  # …
end

Qu.job_processor = ThreadedProcessor.new(10)

@DeTeam
Copy link

DeTeam commented Feb 5, 2013

Agree, strategy classes seem fine.

There's also an idea to move backend interface to a separate class with "implement me" warning.

mauricio added a commit to mauricio/qu that referenced this issue Feb 17, 2014
This is a proposal for bkeepers#5. Creates a separate
class to execute the job instead of doing the
execution inside the worker. This allows us to
have different execution mechanisms, from the
current "just run", to a forking runner and
in the future other threaded runners (that
would be better suited for JRuby and Rubinius).
@jnunemaker
Copy link
Collaborator

This is done. There is a runner that can be customized in #86

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants