[Sparkswap] Error: 14 UNAVAILABLE: TCP Read failed

[dmichael]

Description

When placing an order using the Sparkswap broker, user occasionally will get a TCP Read failed from the gRPC call to LND. This issue does not seem to arise from Sparkswap's stock install on x86.

Environment

• Sparkswap Mini (ARM64)
• BlackboxOS v0.1.30
• Sparkswap Broker v0.6.2-beta

blackbox_sparkswap.1.srgiwkz3iwf7@sparkswap    | info: Request received: [OrderService:createBlockOrder] {"timestamp":"2019-05-16T14:20:16.861Z"}
blackbox_sparkswap.1.srgiwkz3iwf7@sparkswap    | info: Getting all block orders for market: BTC/LTC {"timestamp":"2019-05-16T14:20:16.869Z"}
blackbox_sparkswap.1.srgiwkz3iwf7@sparkswap    | info: Retrieving best priced from ASK up to 10000 {"timestamp":"2019-05-16T14:20:17.233Z"}
blackbox_sparkswap.1.srgiwkz3iwf7@sparkswap    | D0516 14:20:17.322003207      38 dns_resolver.cc:275]        Start resolving.
blackbox_sparkswap.1.srgiwkz3iwf7@sparkswap    | error: Error while handling request: [OrderService:createBlockOrder] {"timestamp":"2019-05-16T14:20:17.328Z"}
blackbox_sparkswap.1.srgiwkz3iwf7@sparkswap    | error: Error: 14 UNAVAILABLE: TCP Read failed
blackbox_sparkswap.1.srgiwkz3iwf7@sparkswap    |     at Object.exports.createStatusError (/home/app/node_modules/lnd-engine/node_modules/grpc/src/common.js:91:15)
blackbox_sparkswap.1.srgiwkz3iwf7@sparkswap    |     at Object.onReceiveStatus (/home/app/node_modules/lnd-engine/node_modules/grpc/src/client_interceptors.js:1204:28)
blackbox_sparkswap.1.srgiwkz3iwf7@sparkswap    |     at InterceptingListener._callNext (/home/app/node_modules/lnd-engine/node_modules/grpc/src/client_interceptors.js:568:42)
blackbox_sparkswap.1.srgiwkz3iwf7@sparkswap    |     at InterceptingListener.onReceiveStatus (/home/app/node_modules/lnd-engine/node_modules/grpc/src/client_interceptors.js:618:8)
blackbox_sparkswap.1.srgiwkz3iwf7@sparkswap    |     at callback (/home/app/node_modules/lnd-engine/node_modules/grpc/src/client_interceptors.js:845:24) {"timestamp":"2019-05-16T14:20:17.334Z"}
blackbox_sparkswap.1.srgiwkz3iwf7@sparkswap    | info: Request completed: [OrderService:createBlockOrder] {"timestamp":"2019-05-16T14:20:17.342Z"}

[dmichael]

Found similar issue grpc/grpc-node#692

[dmichael]

Possible lead on a solution. Check to see if the gRPC connections are being explicitly closed as per dgraph-io/dgraph#2514

[dmichael]

After a great deal of investigation, this issue is a Docker Swarm thing. The solution was to add deploy.endpoint_mode: dnsrr to the files. This solution has caused another quirk, which is that you can now not expose ports.

Summary: Docker Swarm should be considered dangerous and has been removed from the project.

[dmichael]

https://forums.docker.com/t/tcp-timeout-that-occurs-only-in-docker-swarm-not-simple-docker-run/58179/3

docker-archive/classicswarm#2855
moby/moby#35671
moby/moby#35548
moby/moby#36940