Failure in environments without IPv6 support #664
Comments
|
Any way to get the stack trace out of your core dump? I'm kind of reluctant to spin a full FreeBSD VM to debug this one. |
|
Here it is the output from gdb Debug symbols not available? |
|
oh, great, no stacktrace visible... groan Can you try doing |
|
I tried this way and now I get some more output: |
|
Huh, that's... interesting. The uv_socket is NULL. Why... I guess we'll have to try and debug it... how did you get nodejs installed exactly? Was it a port or another method? |
|
The NodeJS port is from the official freebsd packages, I even tried the latest version, the same problem. |
|
On FreeBSD main host (outside jail) runs without any crash, it seems that grpc struggles getting an ip address inside a Jail... |
|
I think this may actually be a memory allocation failure, not a failure to get an address. If you can debug the running program, then you can break on |
|
Here's what I've tried: Why this is happening? That's so weird... |
|
@marcotuna but this is from the coredump itself, right? @murgatroid99 was talking about trying to run nodejs within gdb, placing a breakpoint, and getting the values there. |
|
I'm having issues setting the breakpoint because of the debug symbols are not present, I already installed with npm install --debug grpc_node.node seems to be the file causing the issues, starting it causes: So I tried the following: Probably I am doing something wrong Edit 1Running the gdb command as the last time, now prints more output Edit 2Edit 3 (Created new FreeBSD Jail) |
nicolasnoble
changed the title
|
It now looks like the actual failure here is libuv failing to initialize its handle object, which may be happening because you are not able to create sockets at all, or because you cannot create sockets with the given IP version. Since you have tried This has exposed a bug in how errors in that function are getting handled, but even after we fix that you will just see the same failure with a nicer message. |
|
If you can dtruss or dtrace the node process, and upload the resulting log somewhere, we'd be able to see which actual error code was returned by the socket creation code. |
|
@murgatroid99 I tried changing the IP Addresses in the configuration with IPV4 and/or IPV6 and no results so far... I tried so far enabling some features regarding socket's limitation on a FreeBSD Jail such as allow.raw_sockets and allow.socket_af No results so far Edit 1I was able to generate with truss and the results are interesting... (Attached) I tried disabling IPV6 and still, it keeps using PF_INET6, why?? Because it's compiled on Kernel maybe? Edit 2I added an IPV6 address to the Jail and GRPC starts! But what's weird is that I've set a static ip address to IPV4, i'm not even listening to the IPV6 ip. So, this leaves me to conclude that when GRPC detects that the kernel has IPV6 support, even without having an IPV6 address assigned to any network interface it will use PF_INET6 |
|
Right, I know what's going on here. We're supposed to be creating dual stack sockets first, and then fallback on IPv4 only if this failed. But since we're not checking the result, we never realize that the dual stack socket creation failed. This is a genuine bug. |
nicolasnoble
changed the title
|
It's actually simple to reproduce under plain Linux: |
|
I believe I was experiencing this exact same issue only recently realizing what was wrong last night with the help of a colleague. For me, code ran fine on my host machine and then failed in docker when ipv6 was not enabled. I did notice that forcing the package I was using on to [email protected] didn't suffer the same issue and my code finally ran as expected. |
|
When is there going to be a fix for this? |
|
@marcotuna have you tried 1.17.0-pre1? |
|
@nicolasnoble, yes I tried with that release. Even when I started this issue I had tested with that release version. I tried just now just to check if the problem still persist and it does. |
|
Ah, sorry. I was confused because of @fredantell's update. |
|
I'm still experiencing this bug on grpc-node v1.23.3. As with @fredantell , I'm experiencing this on a docker container (AWS ECS) with ipv6 disabled. I see the following from strace: The segfault happens due to As you can see, I'm specifying an ipv4 address, 192.168.13.4:6008. By the time we get to here the address now shows as [::ffff:192.168.13.4]:6008 |
|
It looks like all ipv4 interfaces get cast to ipv6 here: https://github.com/grpc/grpc/blob/master/src/core/lib/iomgr/tcp_server_custom.cc#L366 So it looks like it is impossible to start grpc-node and bind to a port on a system where IPv6 is disabled. Is that correct? |
|
We have the same issue at Centos7 when IPV6 disabled, if we try to create grpc server on IPV4 address we are getting error: server_chttp2.cc:40] {"created":"@1579610522.002391836","description":"No address added out of total 1 resolved","file":"../deps/grpc/src/core/ext/transport/chttp2/server/chttp2_server.cc","file_line":394,"referenced_errors":[{"created":"@1579610522.002385260","description":"Failed to add port to server","file":"../deps/grpc/src/core/lib/iomgr/tcp_server_custom.cc","file_line":404,"referenced_errors":[{"created":"@1579610522.002372812","description":"Failed to initialize UV tcp handle","file":"../deps/grpc/src/core/lib/iomgr/tcp_uv.cc","file_line":72,"grpc_status":14,"os_error":"address family not supported"}]}]} If we enable IPV6 support (kernel module + sysctl.conf) then it works |
|
We face the same issue with |
|
I wouldn't expect a fix in the near future. The |
|
The |
Problem description
Tried to implement GRPC on a project using FreeBSD 11.2 and it throw a core dump, then decided to try the demo project from the grpc repository, core dump, updated the packages to the latest versions, core dump... It compiles with node-gyp without errors apparently. The same steps on Linux works.
Reproduction steps
git clone https://github.com/grpc/grpc
cd grpc/examples/node
npm install
cd dynamic_codegen
node greeter_server.js
Environment
Additional context
The text was updated successfully, but these errors were encountered: