DNS failing when all the rest of the system seems to report it fine? #226

aronchick · 2015-05-13T06:14:08Z

I'm doing a simple get is my code which uses the internal DNS server for my system - it looks like this:

require 'sinatra/base'
require 'sinatra-websocket'
require 'celluloid'
require 'celluloid/autostart'
require 'celluloid/io'
require 'http'
require 'byebug'

class HttpFetcher
  include Celluloid::IO

  def scan(url)
    # puts "Request -> http://#{url}"
    puts "Requesting for: #{url}"
    return HTTP.get("http://" + url).to_s
  end
end

a = HttpFetcher.new
a.scan("client/json")

a.scan("google.com")

a.scan("nosuchdomainexistsintheworld.com")

When I execute this, I get "no such domain" for the first one. However, it definitely exists in my system - I can ping "client", I can curl "http://client" and when I go into Pry, and execute this, it works fine. Can you think of any reason that the way I'm executing it above would cause a problem? Is there something about the way Http.rb interacts with Celluloid that would cause a DNS issue?

The text was updated successfully, but these errors were encountered:

zanker · 2015-05-13T06:21:07Z

Does this work with just Celluloid or Http.rb without Celluloid IO?

Celluloid IO has some custom DNS code and it's more likely an issue in that then Http.rb.

Sent from my iPhone

On May 12, 2015, at 23:14, David Aronchick [email protected] wrote:

I'm doing a simple get is my code which uses the internal DNS server for my system - it looks like this:

require 'sinatra/base'
require 'sinatra-websocket'
require 'celluloid'
require 'celluloid/autostart'
require 'celluloid/io'
require 'http'
require 'byebug'

class HttpFetcher
include Celluloid::IO

def scan(url)
# puts "Request -> http://#{url}"
puts "Requesting for: #{url}"
return HTTP.get("http://" + url).to_s
end
end

a = HttpFetcher.new
a.scan("client/json")

a.scan("google.com")

a.scan("nosuchdomainexistsintheworld.com")
When I execute this, I get "no such domain" for the first one. However, it definitely exists in my system - I can ping "client", I can curl "http://client" and when I go into Pry, and execute this, it works fine. Can you think of any reason that the way I'm executing it above would cause a problem? Is there something about the way Http.rb interacts with Celluloid that would cause a DNS issue?

—
Reply to this email directly or view it on GitHub.

tarcieri · 2015-05-13T07:04:42Z

The given example isn't configuring http.rb to use the Celluloid backend, so it should be using the system DNS resolver. Specifically, DNS will be resolved here:

https://github.com/httprb/http.rb/blob/master/lib/http/connection.rb#L31

Per your example, it should be using ::TCPSocket.connect, and having Ruby resolve DNS.

@aronchick what Ruby interpreter are you using? Have you tried reproducing this issue with ::TCPSocket.connect? Alternatively you might try actually configuring the Celluloid::IO backend and seeing if that magically solves your problems.

See the example here:

https://github.com/httprb/http.rb/wiki/Parallel-requests-with-Celluloid%3A%3AIO

Note that we're considering a different API in future releases though (e.g. HTTP.use(:celluloid))

aronchick · 2015-05-13T13:44:37Z

So do you mean start with TCPConnect, and go from there? I actually have my real code closer to that example, I just stripped it down to repro here.

I'm trying to understand what you mean about configuring the Celluloid::IO backend - just let me know!

On May 13, 2015, at 00:04, Tony Arcieri [email protected] wrote:

The given example isn't configuring http.rb to use the Celluloid backend, so it should be using the system DNS resolver. Specifically, DNS will be resolved here:

https://github.com/httprb/http.rb/blob/master/lib/http/connection.rb#L31

Per that example, it should be using ::TCPSocket.connect.

@aronchick have you tried reproducing this issue with ::TCPSocket.connect? Alternatively you might try actually configuring the Celluloid::IO backend and seeing if that magically solves your problems.

See the example here:

https://github.com/httprb/http.rb/wiki/Parallel-requests-with-Celluloid%3A%3AIO

Note that we're considering a different API in future releases though (e.g. HTTP.use(:celluloid))

—
Reply to this email directly or view it on GitHub.

aronchick · 2015-05-13T14:02:20Z

Ah, I understand what you mean, but no joy. Here's what the sample method looks like now:

class HttpFetcher
  include Celluloid::IO
  def scan(url, options = {} )
      response = HTTP.get("http://" + url, {:socket_class => Celluloid::IO::TCPSocket})
      return response
  end
end

Here's the output:

root@dashboard-pyd9m:/home/app/dashapp# thin start -p 3000
Using rack adapter
I, [2015-05-13T13:58:30.941506 #1076]  INFO -- : Subscribing to topics.
Thin web server (v1.6.3 codename Protein Powder)
Maximum connections set to 1024
Listening on 0.0.0.0:3000, CTRL+C to stop
Connected to /ws.
Requesting for: client/json
DNS result has no information for client
[... repeats ...] 
Stopping ...
W, [2015-05-13T13:58:38.950896 #1076]  WARN -- : websocket closed
Client closed
D, [2015-05-13T13:58:38.951756 #1076] DEBUG -- : Terminating 58 actors...
root@dashboard-pyd9m:/home/app/dashapp# curl http://client/json
[Correct JSON]

aronchick · 2015-05-13T14:15:11Z

One more datapoint, I removed Celluloid::IO entirely as well as the pooling, and am now calling it directly, and it's still doing the same thing.

tarcieri · 2015-05-13T14:46:13Z

@aronchick yes, that's entirely expected as you aren't touching any Celluloid::IO codepaths. http.rb is using the native DNS resolution of whatever Ruby VM you're using.

Can you please tell us:

What Ruby VM you're using? (i.e. ruby -v)
What OS you're using?

After that, you should take the issue up with the Ruby VM in question as this is not an http.rb issue.

aronchick · 2015-05-13T15:03:46Z

The reason I thought it was associated with celluloid/http.rb was this same method (slightly altered to handle callbacks) worked fine when I was using em-http-client and moved over due to their lack of up keep.

I'm using:

root@dashboard-pyd9m:/home/app/dashapp# ruby -v
ruby 2.1.5p273 (2014-11-13 revision 48405) [x86_64-linux-gnu]
root@dashboard-pyd9m:/home/app/dashapp# uname -a
Linux dashboard-pyd9m 3.16.0-0.bpo.4-amd64 #1 SMP Debian 3.16.7-ckt4-3~bpo70+1 (2015-02-12) x86_64 x86_64 x86_64 GNU/Linux
root@dashboard-pyd9m:/home/app/dashapp#

aronchick · 2015-05-13T17:23:10Z

This appears to be an Celluloid::IO issue. After hacking away down to a minimum set of reproable steps, here's the issue:

_Without Celluloid::IO::TCPSocket:_

Code:

require 'celluloid'
require 'celluloid/autostart'
require 'celluloid/io'
require 'http'

class HttpFetcher
  include Celluloid::IO

  def scan(url, options = {} )
    # puts "Request -> http://#{url}"
      use_options = true
      if (use_options)
              options = { socket_class: Celluloid::IO::TCPSocket
                              }
      end
      response = HTTP.get(url, options)
  end
end

Testing:

[1] pry(main)> require 'http'
=> true
[2] pry(main)> HTTP.get("http://client")
=> #<HTTP::Response/1.1 200 OK {"Server"=>"nginx/1.6.2", "Date"=>"Wed, 13 May 2015 16:48:34 GMT", "Content-Type"=>"text/html;charset=utf-8", "Content-Length"=>"1131", "Connection"=>"close", "X-Xss-Protection"=>"1; mode=block", "X-Content-Type-Options"=>"nosniff", "X-Frame-Options"=>"SAMEORIGIN"}>
[3] pry(main)> require './test'
=> true
[4] pry(main)> HttpFetcher.new.scan("http://client")
Resolv::ResolvError: DNS result has no information for client
from /var/lib/gems/2.1.0/gems/celluloid-io-0.16.2/lib/celluloid/io/tcp_socket.rb:64:in `initialize'
[5] pry(main)> E, [2015-05-13T16:48:43.333658 #388] ERROR -- : Actor crashed!
Resolv::ResolvError: DNS result has no information for client
    /var/lib/gems/2.1.0/gems/celluloid-io-0.16.2/lib/celluloid/io/tcp_socket.rb:64:in `initialize'
    /var/lib/gems/2.1.0/gems/celluloid-io-0.16.2/lib/celluloid/io/tcp_socket.rb:16:in `new'
    /var/lib/gems/2.1.0/gems/celluloid-io-0.16.2/lib/celluloid/io/tcp_socket.rb:16:in `open'
    /var/lib/gems/2.1.0/gems/http-0.8.4/lib/http/timeout/null.rb:18:in `connect'
    /var/lib/gems/2.1.0/gems/http-0.8.4/lib/http/connection.rb:31:in `initialize'
    /var/lib/gems/2.1.0/gems/http-0.8.4/lib/http/client.rb:72:in `new'
    /var/lib/gems/2.1.0/gems/http-0.8.4/lib/http/client.rb:72:in `make_request'
    /var/lib/gems/2.1.0/gems/http-0.8.4/lib/http/client.rb:58:in `block in perform'
    /var/lib/gems/2.1.0/gems/http-0.8.4/lib/http/cache/null_cache.rb:9:in `perform'
    /var/lib/gems/2.1.0/gems/http-0.8.4/lib/http/client.rb:57:in `perform'
    /var/lib/gems/2.1.0/gems/http-0.8.4/lib/http/client.rb:46:in `request'
    /var/lib/gems/2.1.0/gems/http-0.8.4/lib/http/chainable.rb:72:in `request'
    /var/lib/gems/2.1.0/gems/http-0.8.4/lib/http/chainable.rb:16:in `get'
    /home/app/dashapp/test.rb:26:in `scan'
    /var/lib/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in `public_send'
    /var/lib/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in `dispatch'
    /var/lib/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:63:in `dispatch'
    /var/lib/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:60:in `block in invoke'
    /var/lib/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:71:in `block in task'
    /var/lib/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:357:in `block in task'
    /var/lib/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks.rb:57:in `block in initialize'
    /var/lib/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks/task_fiber.rb:15:in `block in create'

_Without Celluloid::IO::TCPSocket:_

Code:

require 'celluloid'
require 'celluloid/autostart'
require 'celluloid/io'
require 'http'

module Errors
  class NoWebResponse < StandardError; end
end

class HttpFetcher
  include Celluloid::IO

  def scan(url, options = {} )
    # puts "Request -> http://#{url}"
      use_options = false
      if (use_options)
          options = { socket_class: Celluloid::IO::TCPSocket
                              }
      end
      response = HTTP.get(url, options)
  end
end

Testing:

[1] pry(main)> require 'http'
=> true
[2] pry(main)> HTTP.get("http://client")
=> #<HTTP::Response/1.1 200 OK {"Server"=>"nginx/1.6.2", "Date"=>"Wed, 13 May 2015 16:59:26 GMT", "Content-Type"=>"text/html;charset=utf-8", "Content-Length"=>"1141", "Connection"=>"close", "X-Xss-Protection"=>"1; mode=block", "X-Content-Type-Options"=>"nosniff", "X-Frame-Options"=>"SAMEORIGIN"}>
[3] pry(main)> require './test'
=> true
[4] pry(main)> HttpFetcher.new.scan("http://client")
=> #<HTTP::Response/1.1 200 OK {"Server"=>"nginx/1.6.2", "Date"=>"Wed, 13 May 2015 16:59:42 GMT", "Content-Type"=>"text/html;charset=utf-8", "Content-Length"=>"1129", "Connection"=>"close", "X-Xss-Protection"=>"1; mode=block", "X-Content-Type-Options"=>"nosniff", "X-Frame-Options"=>"SAMEORIGIN"}>

aronchick · 2015-05-13T20:31:23Z

I'll move this over to Celluloid::IO since it seems pretty clearly in their camp.

digitalextremist mentioned this issue May 13, 2015

Celluloid DNS Issue? celluloid/celluloid#621

Closed

tarcieri closed this as completed May 13, 2015

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DNS failing when all the rest of the system seems to report it fine? #226

DNS failing when all the rest of the system seems to report it fine? #226

aronchick commented May 13, 2015

zanker commented May 13, 2015

tarcieri commented May 13, 2015

aronchick commented May 13, 2015

aronchick commented May 13, 2015

aronchick commented May 13, 2015

tarcieri commented May 13, 2015

aronchick commented May 13, 2015

aronchick commented May 13, 2015

aronchick commented May 13, 2015

DNS failing when all the rest of the system seems to report it fine? #226

DNS failing when all the rest of the system seems to report it fine? #226

Comments

aronchick commented May 13, 2015

zanker commented May 13, 2015

tarcieri commented May 13, 2015

aronchick commented May 13, 2015

aronchick commented May 13, 2015

aronchick commented May 13, 2015

tarcieri commented May 13, 2015

aronchick commented May 13, 2015

aronchick commented May 13, 2015

aronchick commented May 13, 2015