Skip to content

Commit

Permalink
Update server-detection logic; remove :async_all and deprecation of…
Browse files Browse the repository at this point in the history
… `:async`
  • Loading branch information
bensheldon committed Apr 12, 2021
1 parent 2056fed commit bde582b
Show file tree
Hide file tree
Showing 7 changed files with 28 additions and 31 deletions.
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -227,10 +227,10 @@ Available configuration options are:
- `:inline` executes jobs immediately in whatever process queued them (usually the web server process). This should only be used in test and development environments.
- `:external` causes the adapter to enqueue jobs, but not execute them. When using this option (the default for production environments), you’ll need to use the command-line tool to actually execute your jobs.
- `:async_server` executes jobs in separate threads within the Rails webserver process (`bundle exec rails server`). It can be more economical for small workloads because you don’t need a separate machine or environment for running your jobs, but if your web server is under heavy load or your jobs require a lot of resources, you should choose `:external` instead. When not in the Rails webserver, jobs will execute in `:external` mode to ensure jobs are not executed within `rails console`, `rails db:migrate`, `rails assets:prepare`, etc.
- `:async_all` executes jobs in _all_ Rails processes. (Previously this configuration option was `:async`, which is deprecated. In future versions `:async` will behave like `:async_server`).
- `max_threads` (integer) sets the maximum number of threads to use when `execution_mode` is set to `:async_*`. You can also set this with the environment variable `GOOD_JOB_MAX_THREADS`.
- `queues` (string) determines which queues to execute jobs from when `execution_mode` is set to `:async_*`. See the description of `good_job start` for more details on the format of this string. You can also set this with the environment variable `GOOD_JOB_QUEUES`.
- `poll_interval` (integer) sets the number of seconds between polls for jobs when `execution_mode` is set to `:async_*`. You can also set this with the environment variable `GOOD_JOB_POLL_INTERVAL`.
- `:async` executes jobs in separate threads in _any_ Rails process.
- `max_threads` (integer) sets the maximum number of threads to use when `execution_mode` is set to `:async` or `:async_server`. You can also set this with the environment variable `GOOD_JOB_MAX_THREADS`.
- `queues` (string) determines which queues to execute jobs from when `execution_mode` is set to `:async` or `:async_server`. See the description of `good_job start` for more details on the format of this string. You can also set this with the environment variable `GOOD_JOB_QUEUES`.
- `poll_interval` (integer) sets the number of seconds between polls for jobs when `execution_mode` is set to `:async` or `:async_server`. You can also set this with the environment variable `GOOD_JOB_POLL_INTERVAL`.
- `max_cache` (integer) sets the maximum number of scheduled jobs that will be stored in memory to reduce execution latency when also polling for scheduled jobs. Caching 10,000 scheduled jobs uses approximately 20MB of memory. You can also set this with the environment variable `GOOD_JOB_MAX_CACHE`.
- `shutdown_timeout` (float) number of seconds to wait for jobs to finish when shutting down before stopping the thread. Defaults to forever: `-1`. You can also set this with the environment variable `GOOD_JOB_SHUTDOWN_TIMEOUT`.

Expand Down
21 changes: 16 additions & 5 deletions lib/good_job/adapter.rb
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,15 @@ module GoodJob
#
class Adapter
# Valid execution modes.
EXECUTION_MODES = [:async, :async_all, :async_server, :external, :inline].freeze
EXECUTION_MODES = [:async, :async_server, :external, :inline].freeze

# @param execution_mode [nil, Symbol] specifies how and where jobs should be executed. You can also set this with the environment variable +GOOD_JOB_EXECUTION_MODE+.
#
# - +:inline+ executes jobs immediately in whatever process queued them (usually the web server process). This should only be used in test and development environments.
# - +:external+ causes the adapter to enqueue jobs, but not execute them. When using this option (the default for production environments), you'll need to use the command-line tool to actually execute your jobs.
# - +:async_server+ executes jobs in separate threads within the Rails webserver process (`bundle exec rails server`). It can be more economical for small workloads because you don't need a separate machine or environment for running your jobs, but if your web server is under heavy load or your jobs require a lot of resources, you should choose +:external+ instead.
# When not in the Rails webserver, jobs will execute in +:external+ mode to ensure jobs are not executed within `rails console`, `rails db:migrate`, `rails assets:prepare`, etc.
# - +:async_all+ executes jobs in all Rails processes. (Previously this was named +:async+ mode, which is deprecated.)
# - +:async+ executes jobs in any Rails process.
#
# The default value depends on the Rails environment:
#
Expand Down Expand Up @@ -127,19 +127,30 @@ def shutdown(timeout: :default, wait: nil)

# Whether in +:async+ execution mode.
def execute_async?
@configuration.execution_mode == :async_all ||
@configuration.execution_mode == :async_server && Rails.const_defined?("Server")
@configuration.execution_mode == :async ||
@configuration.execution_mode == :async_server && in_server_process?
end

# Whether in +:external+ execution mode.
def execute_externally?
@configuration.execution_mode == :external ||
@configuration.execution_mode == :async_server && !Rails.const_defined?("Server")
@configuration.execution_mode == :async_server && !in_server_process?
end

# Whether in +:inline+ execution mode.
def execute_inline?
@configuration.execution_mode == :inline
end

private

# Whether running in a web server process.
def in_server_process?
return @_in_server_process if defined? @_in_server_process

@_in_server_process = Rails.const_defined?('Server') ||
caller.grep(%r{config.ru}).any? || # EXAMPLE: config.ru:3:in `block in <main>' OR config.ru:3:in `new_from_string'
(Concurrent.on_jruby? && caller.grep(%r{jruby/rack/rails_booter}).any?) # EXAMPLE: uri:classloader:/jruby/rack/rails_booter.rb:83:in `load_environment'
end
end
end
16 changes: 1 addition & 15 deletions lib/good_job/configuration.rb
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ module GoodJob
#
class Configuration
# Valid execution modes.
EXECUTION_MODES = [:async, :async_all, :async_server, :external, :inline].freeze
EXECUTION_MODES = [:async, :async_server, :external, :inline].freeze
# Default number of threads to use per {Scheduler}
DEFAULT_MAX_THREADS = 5
# Default number of seconds between polls for jobs
Expand Down Expand Up @@ -58,20 +58,6 @@ def execution_mode
end
mode = mode.to_sym if mode

if mode == :async
ActiveSupport::Deprecation.warn(<<~DEPRECATION)
GoodJob's execution mode configuration option `:async` has been deprecated. Replace with:
- `:async_server`: for async only within the web-process (`$ rails server`)
- `:async_all`: For async in all processes (equivalent to `async`).
In future versions, `:async` will behave like `:async_server. For more details:
https://github.com/bensheldon/good_job#configuration-options
DEPRECATION

mode = :async_all
end

@_execution_mode = if mode
mode
elsif Rails.env.development? || Rails.env.test?
Expand Down
4 changes: 2 additions & 2 deletions spec/integration/adapter_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ def perform(*_args, **_kwargs)

describe 'Async execution mode' do
context 'when Scheduler polling is disabled' do
let(:adapter) { GoodJob::Adapter.new execution_mode: :async_all, queues: 'mice:1', poll_interval: -1 }
let(:adapter) { GoodJob::Adapter.new execution_mode: :async, queues: 'mice:1', poll_interval: -1 }

it 'Jobs are directly handed to the performer, if they match the queues' do
elephant_ajob = ExampleJob.set(queue: 'elepehants').perform_later
Expand All @@ -81,7 +81,7 @@ def perform(*_args, **_kwargs)

it 'invokes the notifier if the job is not locally runnable', skip_if_java: true do
# Create another adapter but do not attach it
elephant_adapter = GoodJob::Adapter.new execution_mode: :async_all, queues: 'elephants:1', poll_interval: -1
elephant_adapter = GoodJob::Adapter.new execution_mode: :async, queues: 'elephants:1', poll_interval: -1
sleep_until { GoodJob::Notifier.instances.all?(&:listening?) }

elephant_ajob = ExampleJob.set(queue: 'elephants').perform_later
Expand Down
6 changes: 3 additions & 3 deletions spec/lib/good_job/adapter_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@
scheduler = instance_double(GoodJob::Scheduler, shutdown: nil, create_thread: nil)
allow(GoodJob::Scheduler).to receive(:new).and_return(scheduler)

adapter = described_class.new(execution_mode: :async_all, poll_interval: -1)
adapter = described_class.new(execution_mode: :async, poll_interval: -1)
adapter.enqueue(active_job)

expect(scheduler).to have_received(:create_thread)
Expand Down Expand Up @@ -79,7 +79,7 @@

describe '#execute_async?' do
context 'when execution mode async_all' do
let(:adapter) { described_class.new(execution_mode: :async_all) }
let(:adapter) { described_class.new(execution_mode: :async) }

it 'returns true' do
expect(adapter.execute_async?).to eq true
Expand All @@ -105,7 +105,7 @@
hide_const("Rails::Server")
end

it 'returns true' do
it 'returns false' do
expect(adapter.execute_async?).to eq false
expect(adapter.execute_externally?).to eq true
end
Expand Down
2 changes: 1 addition & 1 deletion spec/test_app/config/environments/development.rb
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@
config.good_job.execution_mode = :async_server
end

if config.good_job.execution_mode.in? [:async_all, :async_server, :async]
if config.good_job.execution_mode.in? [:async_server, :async]
config.good_job.poll_interval = 30
end
end
2 changes: 1 addition & 1 deletion spec/test_app/config/environments/production.rb
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@
config.good_job.execution_mode = :async_server
end

if config.good_job.execution_mode.in? [:async_all, :async_server, :async]
if config.good_job.execution_mode.in? [:async_server, :async]
config.good_job.poll_interval = 30
end

Expand Down

0 comments on commit bde582b

Please sign in to comment.