DNS resolution occasionally hangs

[jonhyman]

Hi,

We've been experimenting with using the HTTP gem with Celluloid::IO to make API calls to SendGrid. In testing this out with different pool sizes of HttpFetcher (taken from https://github.com/tarcieri/http/blob/master/examples/parallel_requests_with_celluloid.rb) and making a few hundred requests, getting future.value would hang occasionally, requiring a force kill of the program. We debugged this down to be the DNSResolver hanging, and were able to reproduce it with a simple example. The repro case is very sporadic, though, so it may take a few dozen times to run this code to get it to hang but we've been able to do it with this code.

Using MRI Ruby 2.1.2, Celluloid 0.15.2, Celluloid-IO 0.15.0

class Foo
  include Celluloid::IO
  def test
    Celluloid::IO::DNSResolver.new.resolve("api.sendgrid.com")
  end
end

f = Foo.pool(:size => 400)

futures = []
1000.times do |i|
  futures << f.future.test
end

# Occasionally this will hang on one of the iterations.
1000.times do |i|
  puts(i)
  futures[i].value
end

[tarcieri]

I'd suggest upgrading to the latest versions of celluloid/celluloid-io and see if your problems still persist

[jonhyman]

I'll give it a shot after 0.16.1, since we're stuck on 0.15.2 due to celluloid/celluloid#457.

[tarcieri]

Which problem in particular? I should probably break #457 down into more granular tickets

[jonhyman]

We were being affected by sidekiq/sidekiq#1929 and couldn't get our Sidekiq workers to shut down / restart. I didn't notice the other 2 that Mike talks about, but we caught that first one in staging the day we updated so we reverted quickly.

[tarcieri]

I'm still looking into that one myself. If you have any ideas, let me know.

[Asmod4n]

Btw, what about using https://code.google.com/p/libevhdns/ for crubies and whatever non blocking resolver Java has?