Add async_server option to run async only in Rails web server process

[bensheldon]

Closes #194.

• :async_server executes jobs in separate threads within the Rails webserver process (bundle exec rails server). It can be more economical for small workloads because you don’t need a separate machine or environment for running your jobs, but if your web server is under heavy load or your jobs require a lot of resources, you should choose :external instead. When not in the Rails webserver, jobs will execute in :external mode to ensure jobs are not executed within rails console, rails db:migrate, rails assets:prepare, etc.
• :async_all executes jobs in all Rails processes. (Previously this configuration option was :async, which is deprecated. In future versions :async will behave like :async_server). On second thought, async is not deprecated. This PR just adds a new option, and doesn't change the previous config behavior.

[bensheldon]

@tedhexaflow Could you please review this change to async? I wanted to confirm with you that when running Rails server with Java Tomcat/etc (as a WAR?) that Rails.const_defined?("Server") returns true.

I think this PR will be a good change by removing some of the rough edges of async mode running in unexpected places.

[tedt10e]

Hi @bensheldon,

Thanks for the heads up. Let me do a test and get back to you.

[tedt10e]

Hi @bensheldon

I've checked, and Rails.const_defined?("Server") will return false if we run the app on tomcat.

So if I need to continue using async mode on tomcat, I will have to use async_all, and async_server won't work.

Am I right?

[bensheldon]

@tedhexaflow Thank you for checking!

That's a bummer. I would really like to make this PR work for Tomcat/jRuby before moving forward.

Is there a blog post/instructions that you would recommend for me to simulate/recreate a jRuby/Tomcat server setup? Or could you list out the tools you're using? (I hate to ask you to do too much work).

[tedt10e]

Hi @bensheldon

In addition to JDK 15*, JRuby-9.2*,

We need warbler https://github.com/jruby/warbler to bundle the rails app into a war file.

Then download tomcat https://tomcat.apache.org/download-90.cgi and start the tomcat server by running bin/startup.sh. (the server will run on port 8080. you can change them in conf/server.xml)

Copy the war file to webapps/ROOT.war and open http://localhost:8080.

This link looks a little outdated, but I had a quick look, and we can still use it for a reference.
https://www.digitalocean.com/community/tutorials/how-to-use-jruby-to-run-a-rails-application-on-apache-tomcat-7-and-ubuntu-14-04

I wish I could have a docker image ready to provide you. Unfortunately, I didn't create one before.

In case setting up the tomcat wasting too much of your time and if you can wait a bit longer, I'll try to create a docker image for you next week.

Or you can try to set it up, and if you need help, please feel free to ask, and I will try my best to help.

Cheers!

[bensheldon]

@tedhexaflow that's all super helpful. I will try to get that running in a Docker file.

Really specifically, what I'm looking for is a way to determine, in the Ruby code, "Is the current ruby process running as a webserver or not?" With MRI it seems as simple as checking Rails.const_defined?("Server"). I'm looking for something similar for JRuby.

[tedt10e]

Hi @bensheldon

To clarify, Rails.const_defined?("Server") will return TRUE for both MRI and JRuby while running on Puma Server.

But running on Tomcat will not return TRUE.

[tedt10e]

Hi @bensheldon

I've also ask help from Charles from JRuby.
jruby/jruby#6657

[bensheldon]

@tedhexaflow thank you! I'll add some more details to jruby/jruby#6657

[bensheldon]

Having done some experimentation, I think the proper logic is:

def in_server_process?
  Rails.const_defined?('Server') || 
    caller.grep(%r{config.ru}).any? || # config.ru:3:in `block in <main>' OR config.ru:3:in `new_from_string'
    (Concurrent.on_jruby? && caller.grep(%r{jruby/rack/rails_booter}).any? # uri:classloader:/jruby/rack/rails_booter.rb:83:in `load_environment'
end

Some experimental information:

MRI

	Rails Server	Rack App	Rake Task	Generator
Rails.const_defined?('Server')	✅ True	⛔️ False	✅ False	✅ False
caller.grep(%r{config.ru}).any?	⛔️ False	✅ True	✅ False	✅ False

JRuby

	Warbler build	Web Server	Rake Task	Generator
Rails.const_defined?('Server')	✅ False	⛔️ False	✅ False	✅ False
JRuby.const_defined?('Rack')	⛔️ True	✅ True	⛔️ True	⛔️ True
caller.grep(%r{config.ru}).any?	✅ False	⛔️ False	✅ False	✅ False
caller.grep(%r{jruby/rack/rails_booter}).any?	✅ False	✅ True	✅ False	✅ False

What I did

I've added this code to config/application.rb:

puts "== DEBUGGING =="
puts "Rails::Server=#{Rails.const_defined?('Server')}"
puts "JRuby::Rack=#{defined?(JRuby) && JRuby.const_defined?('Rack')}"
puts "config.ru=#{caller.grep(%r{config.ru}).any?}"
puts "jruby/rack/rails_booter=#{caller.grep(%r{jruby/rack/rails_booter}).any?}" # uri:classloader:/jruby/rack/rails_booter.rb:83:in `load_environment'
# puts "CALLER: \n  #{caller.join("\n  ")}"
puts "== /DEBUGGING =="

MRI

# Running the Web Server
$ bin/rails server
# Rails::Server=true
# JRuby::Rack=
# config.ru=false
# jruby/rack/rails_booter=false

# Running a rack app directly
$ bundle exec puma -C config/puma.rb config.ru
# Rails::Server=false
# JRuby::Rack=
# config.ru=true
# jruby/rack/rails_booter=false

# Running a Rake Task
$ bin/rails db:create
# Rails::Server=false
# JRuby::Rack=
# config.ru=false
# jruby/rack/rails_booter=false

# Running a generator
$ bin/rails g migration ExampleTestMigration
# Rails::Server=false
# JRuby::Rack=
# config.ru=false
# jruby/rack/rails_booter=false

JRuby

# Building a WAR with Warbler
$ MAVEN_REPO=https://repo1.maven.org/maven2 bundle exec warble runnable executable war
# Rails::Server=false
# JRuby::Rack=true
# config.ru=false
# jruby/rack/rails_booter=false

# Running the Web Server
$ java -jar good_job_jruby_experiments.war
Rails::Server=false
JRuby::Rack=true
config.ru=false
jruby/rack/rails_booter=true

# Running a Rake Task
$ java -jar good_job_jruby_experiments.war -S rails db:create
# Rails::Server=false
# JRuby::Rack=true
# config.ru=false
# jruby/rack/rails_booter=false

# Running a generator
$ java -jar good_job_jruby_experiments.war -S rails g migration ExampleTestMigration
# Rails::Server=false
# JRuby::Rack=true
# config.ru=false
# jruby/rack/rails_booter=false