console RPC connection to resolvable name does not connect #16457

lazzarello · 2018-04-06T23:10:07Z

System information

Geth version:

$ geth version
Geth
Version: 1.8.4-unstable
Git Commit: 50dbe8e2444cfc171930cb82cc99017f6a0aadf2
Architecture: amd64
Protocol Versions: [63 62]
Network Id: 1
Go Version: go1.10.1
Operating System: darwin
GOPATH=/Users/leeazzarello/go
GOROOT=/usr/local/opt/go/libexec

OS & Version: macOS 10.13.3 (17D102)
Commit hash : 50dbe8e

Expected behaviour

Running geth attach http://2blockchains.local:8545 established a connection to the RPC interface and I can call API methods.

Actual behaviour

2blockchains:~ leeazzarello$ ping -c 1 2blockchains.local
PING 2blockchains.local (10.7.34.116): 56 data bytes
64 bytes from 10.7.34.116: icmp_seq=0 ttl=64 time=0.044 ms

--- 2blockchains.local ping statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.044/0.044/0.044/0.000 ms

2blockchains:~ leeazzarello$ geth attach http://2blockchains.local:8545
Fatal: Failed to start the JavaScript console: api modules: invalid character 'i' looking for beginning of value

2blockchains:~ leeazzarello$ geth attach http://10.7.34.116:8545
Welcome to the Geth JavaScript console!

instance: Geth/v1.8.2-stable/darwin-amd64/go1.10
 modules: eth:1.0 net:1.0 rpc:1.0 web3:1.0

> web3.eth.syncing
{
  currentBlock: 2952311,
  highestBlock: 2975628,
  knownStates: 36109445,
  pulledStates: 36109445,
  startingBlock: 2952311
}

Steps to reproduce the behaviour

Ensure the host has a resolvable Zeroconf/Bonjour name. Run geth with the --rpcaddr 0.0.0.0 parameter. Attach geth to the Zeroconf name.

Backtrace

I didn't get that far.

The text was updated successfully, but these errors were encountered:

lazzarello · 2018-04-07T22:43:28Z

I added some debug logging to trace this to a source function. I found where the error is thrown in my fork. Here's the debugging output. I'm not sure how to fix the problem. I'll confirm it's caused by Zeroconf name resolution.

2blockchains:go-ethereum leeazzarello$ $GOPATH/bin/geth --debug --pprof -verbosity 5 attach http://2blockchains.local:8545
DEBUG[04-07|15:34:46|geth/consolecmd.go:120] remote endpoint is http://2blockchains.local:8545 
DEBUG[04-07|15:34:46|rpc/client.go:171]      connecting to rpc url http://2blockchains.local:8545 
DEBUG[04-07|15:34:46|rpc/http.go:74]         connecting to HTTP endpoint http://2blockchains.local:8545 
INFO [04-07|15:34:46|internal/debug/flags.go:138] Starting pprof server                    addr=http://127.0.0.1:6060/debug/pprof
DEBUG[04-07|15:34:46|rpc/http.go:95]              Sending message {"jsonrpc":"2.0","id":1,"method":"rpc_modules","params":null} 
DEBUG[04-07|15:34:46|rpc/http.go:101]             Got response {    <nil> } 
DEBUG[04-07|15:34:46|rpc/http.go:103]             Error decoding JSON response: invalid character 'i' looking for beginning of value 
Fatal: Failed to start the remote JavaScript console: cannot create api client modules: invalid character 'i' looking for beginning of value

lazzarello · 2018-04-07T22:50:22Z

Okay, it's not Zeroconf name resolution. I added a static entry to /etc/hosts and got the same results

2blockchains:go-ethereum leeazzarello$ cat /etc/hosts | grep 2blockchains
192.168.1.90	2blockchains.geth.org
2blockchains:go-ethereum leeazzarello$ $GOPATH/bin/geth --debug --pprof -verbosity 5 attach http://2blockchains.geth.org:8545
DEBUG[04-07|15:49:36|geth/consolecmd.go:120] remote endpoint is http://2blockchains.geth.org:8545 
DEBUG[04-07|15:49:36|rpc/client.go:171]      connecting to rpc url http://2blockchains.geth.org:8545 
DEBUG[04-07|15:49:36|rpc/http.go:74]         connecting to HTTP endpoint http://2blockchains.geth.org:8545 
INFO [04-07|15:49:36|internal/debug/flags.go:138] Starting pprof server                    addr=http://127.0.0.1:6060/debug/pprof
DEBUG[04-07|15:49:36|rpc/http.go:95]              Sending message {"jsonrpc":"2.0","id":1,"method":"rpc_modules","params":null} 
DEBUG[04-07|15:49:36|rpc/http.go:101]             Got response {    <nil> } 
DEBUG[04-07|15:49:36|rpc/http.go:103]             Error decoding JSON response: invalid character 'i' looking for beginning of value 
Fatal: Failed to start the remote JavaScript console: cannot create api client modules: invalid character 'i' looking for beginning of value

karalabe · 2018-04-10T11:45:59Z

Just a hunch, but it might be the DNS rebind protection #15962. Please make sure you explicitly specify the DNS domains you want your node accessible from via --rpcvhosts=2blockchains.local

minzak · 2018-04-10T16:52:21Z

Have the similar problem.
ethereum/wiki#576

I try to use geth v1.8.3 with --rpcvhosts=mydomain - and still same error:
invalid host specified

lazzarello · 2018-04-10T22:00:12Z

Oh my! Indeed, it was #15962 . I would like to keep this issue open since the error output was...uh...less than helpful. The diff in the PR could probably help me understand where the string invalid character 'i' looking for beginning of value is generated. I'm new to both this codebase and the Go language, so it might take some time. What follows is my client session which confirm the solution.

2blockchains:~ leeazzarello$ geth --testnet attach http://2blockchains.local:8545
Fatal: Failed to start the JavaScript console: api modules: invalid character 'i' looking for beginning of value
2blockchains:~ leeazzarello$ geth --testnet attach http://2blockchains.local:8545
Welcome to the Geth JavaScript console!

instance: Geth/v1.8.4-unstable-50dbe8e2/darwin-amd64/go1.10.1
 modules: eth:1.0 net:1.0 rpc:1.0 web3:1.0

> eth.blockNumber
3013664
>

lazzarello · 2018-04-11T01:13:23Z

Yikes, this is way more confusing that I expected. Here's as far as I got. I still can't even find what is calling the function that outputs the actual contents of err but I did figure out how to get a stack trace. I added some logging that hints at what might be happening.

2blockchains:go-ethereum leeazzarello$ $GOPATH/bin/geth --verbosity 5 --debug --testnet attach http://2blockchains.local:8545
DEBUG[04-10|18:10:09|geth/consolecmd.go:120] remote endpoint is http://2blockchains.local:8545 
DEBUG[04-10|18:10:09|rpc/client.go:171]      connecting to rpc url http://2blockchains.local:8545 
DEBUG[04-10|18:10:09|rpc/http.go:75]         connecting to HTTP endpoint http://2blockchains.local:8545 
DEBUG[04-10|18:10:09|rpc/client.go:259]      Sending op &{[1] <nil> %!s(chan *rpc.jsonrpcMessage=0xc420cf8f00) %!s(*rpc.ClientSubscription=<nil>)} 
goroutine 1 [running]:
runtime/debug.Stack(0xc420d0f348, 0x1, 0x1)
	/usr/local/Cellar/go/1.10.1/libexec/src/runtime/debug/stack.go:24 +0xa7
runtime/debug.PrintStack()
	/usr/local/Cellar/go/1.10.1/libexec/src/runtime/debug/stack.go:16 +0x22
github.com/ethereum/go-ethereum/rpc.(*Client).sendHTTP(0xc420223ea0, 0x4cb4160, 0xc420cf8ea0, 0xc420276c80, 0x4a27a20, 0xc4202b7260, 0x0, 0x0)
	/Users/leeazzarello/go/src/github.com/ethereum/go-ethereum/rpc/http.go:103 +0x216
github.com/ethereum/go-ethereum/rpc.(*Client).CallContext(0xc420223ea0, 0x4cb4160, 0xc420cf8ea0, 0x4971860, 0xc4201d4060, 0x4b30876, 0xb, 0x0, 0x0, 0x0, ...)
	/Users/leeazzarello/go/src/github.com/ethereum/go-ethereum/rpc/client.go:261 +0x25b
github.com/ethereum/go-ethereum/rpc.(*Client).SupportedModules(0xc420223ea0, 0x0, 0x0, 0x0)
	/Users/leeazzarello/go/src/github.com/ethereum/go-ethereum/rpc/client.go:220 +0x113
github.com/ethereum/go-ethereum/console.(*Console).init(0xc42022a230, 0x0, 0x0, 0x0, 0x0, 0x6)
	/Users/leeazzarello/go/src/github.com/ethereum/go-ethereum/console/console.go:135 +0x863
github.com/ethereum/go-ethereum/console.New(0xc4202544e0, 0x2c, 0x4b27c15, 0x1, 0xc420223ea0, 0x4b27cff, 0x2, 0x4cb7c00, 0xc420293ec0, 0x4ca9ec0, ...)
	/Users/leeazzarello/go/src/github.com/ethereum/go-ethereum/console/console.go:99 +0x26a
main.remoteConsole(0xc4202569a0, 0x0, 0x0)
	/Users/leeazzarello/go/src/github.com/ethereum/go-ethereum/cmd/geth/consolecmd.go:146 +0x2fb
github.com/ethereum/go-ethereum/cmd/utils.MigrateFlags.func1(0xc4202569a0, 0x0, 0xc4202569a0)
	/Users/leeazzarello/go/src/github.com/ethereum/go-ethereum/cmd/utils/flags.go:1295 +0x103
github.com/ethereum/go-ethereum/vendor/gopkg.in/urfave/cli%2ev1.HandleAction(0x49af600, 0xc420336c30, 0xc4202569a0, 0xc420118600, 0x0)
	/Users/leeazzarello/go/src/github.com/ethereum/go-ethereum/vendor/gopkg.in/urfave/cli.v1/app.go:490 +0xc8
github.com/ethereum/go-ethereum/vendor/gopkg.in/urfave/cli%2ev1.Command.Run(0x4b2a7d7, 0x6, 0x0, 0x0, 0x0, 0x0, 0x0, 0x4b5915c, 0x3d, 0x0, ...)
	/Users/leeazzarello/go/src/github.com/ethereum/go-ethereum/vendor/gopkg.in/urfave/cli.v1/command.go:210 +0xa36
github.com/ethereum/go-ethereum/vendor/gopkg.in/urfave/cli%2ev1.(*App).Run(0xc4201ec340, 0xc4200da000, 0x7, 0x7, 0x0, 0x0)
	/Users/leeazzarello/go/src/github.com/ethereum/go-ethereum/vendor/gopkg.in/urfave/cli.v1/app.go:255 +0x6a0
main.main()
	/Users/leeazzarello/go/src/github.com/ethereum/go-ethereum/cmd/geth/main.go:206 +0x55
Fatal: Failed to start the remote JavaScript console: cannot create api client modules: Error creating HTTP connection to RPC socket. Did you set --rpcvhosts? Actual error is "invalid character 'i' looking for beginning of value"

adamschmideg · 2018-11-27T09:01:58Z

@lazzarello can you check if the problem persists

adamschmideg · 2019-11-07T08:52:39Z

Nobody seems to be affected by this issue, so I'm closing it. If you still experience the problem with a more recent version of Geth, please open a new issue.

lazzarello mentioned this issue Apr 13, 2018

Add logging for missing --rpcvhosts on server #16498

Closed

holiman mentioned this issue Apr 13, 2018

rpc: handle rpc (client) response error codes #16500

Merged

adamschmideg added the need:check-for-current-version label Nov 27, 2018

adamschmideg closed this as completed Nov 7, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

console RPC connection to resolvable name does not connect #16457

console RPC connection to resolvable name does not connect #16457

lazzarello commented Apr 6, 2018

lazzarello commented Apr 7, 2018

lazzarello commented Apr 7, 2018

karalabe commented Apr 10, 2018

minzak commented Apr 10, 2018

lazzarello commented Apr 10, 2018

lazzarello commented Apr 11, 2018

adamschmideg commented Nov 27, 2018

adamschmideg commented Nov 7, 2019

console RPC connection to resolvable name does not connect #16457

console RPC connection to resolvable name does not connect #16457

Comments

lazzarello commented Apr 6, 2018

System information

Expected behaviour

Actual behaviour

Steps to reproduce the behaviour

Backtrace

lazzarello commented Apr 7, 2018

lazzarello commented Apr 7, 2018

karalabe commented Apr 10, 2018

minzak commented Apr 10, 2018

lazzarello commented Apr 10, 2018

lazzarello commented Apr 11, 2018

adamschmideg commented Nov 27, 2018

adamschmideg commented Nov 7, 2019