build fails after 2018-12-01 git pull (go 1.11.2)

[alevchuk]

Background

I git pull and re-build frequently. Build failures are unusual here. Upgraded go to 1.11.2 yet I still get the following build failure.

Your environment

• version of lnd - previous last working version was 0.5.1-beta commit=v0.5.1-beta-rc3, newly pulled code is f4b6e0b
• which operating system: Linux lnpi1 4.14.62+ [Test enhancement] Correlate integration test nodes with log output files #1134 Tue Aug 14 16:58:07 BST 2018 armv6l GNU/Linux
• version ofbitcoind:: Bitcoin Core Daemon version v0.17.99.0-134b42a40
• go version go1.11.2 linux/arm
• git version 2.11.0

Steps to reproduce

cd $GOPATH/src/github.com/lightningnetwork/lnd
make clean && make && make install

Expected behaviour

Build lnd

Actual behaviour

 $ make clean && make && make install
 Cleaning source.
rm -f ./lnd-debug ./lncli-debug
rm -f -r ./vendor .vendor-new
 Building debug lnd and lncli.
GO111MODULE=on go build -v -tags="dev" -o lnd-debug -ldflags "-X github.com/lightningnetwork/lnd/build.Commit=v0.5.1-beta-73-gf4b6e0b7755982fc571e2763e0a2ec93c8e89900" github.com/lightningnetwork/lnd
Fetching https://golang.org/x/net?go-get=1
Fetching https://cloud.google.com/go?go-get=1
Fetching https://golang.org/x/crypto?go-get=1
Fetching https://golang.org/x/sys?go-get=1
Fetching https://golang.org/x/time?go-get=1
Fetching https://golang.org/x/lint?go-get=1
Fetching https://golang.org/x/text?go-get=1
Parsing meta tags from https://golang.org/x/time?go-get=1 (status code 200)
get "golang.org/x/time": found meta tag get.metaImport{Prefix:"golang.org/x/time", VCS:"git", RepoRoot:"https://go.googlesource.com/time"} at https://golang.org/x/time?go-get=1
go: finding golang.org/x/time v0.0.0-20180412165947-fbb02b2291d2
Parsing meta tags from https://golang.org/x/crypto?go-get=1 (status code 200)
get "golang.org/x/crypto": found meta tag get.metaImport{Prefix:"golang.org/x/crypto", VCS:"git", RepoRoot:"https://go.googlesource.com/crypto"} at https://golang.org/x/crypto?go-get=1
go: finding golang.org/x/crypto v0.0.0-20181127143415-eb0de9b17e85
Parsing meta tags from https://golang.org/x/lint?go-get=1 (status code 200)
get "golang.org/x/lint": found meta tag get.metaImport{Prefix:"golang.org/x/lint", VCS:"git", RepoRoot:"https://go.googlesource.com/lint"} at https://golang.org/x/lint?go-get=1
go: finding golang.org/x/lint v0.0.0-20180702182130-06c8688daad7
Parsing meta tags from https://golang.org/x/sys?go-get=1 (status code 200)
get "golang.org/x/sys": found meta tag get.metaImport{Prefix:"golang.org/x/sys", VCS:"git", RepoRoot:"https://go.googlesource.com/sys"} at https://golang.org/x/sys?go-get=1
Parsing meta tags from https://golang.org/x/net?go-get=1 (status code 200)
Parsing meta tags from https://golang.org/x/text?go-get=1 (status code 200)
get "golang.org/x/net": found meta tag get.metaImport{Prefix:"golang.org/x/net", VCS:"git", RepoRoot:"https://go.googlesource.com/net"} at https://golang.org/x/net?go-get=1
get "golang.org/x/text": found meta tag get.metaImport{Prefix:"golang.org/x/text", VCS:"git", RepoRoot:"https://go.googlesource.com/text"} at https://golang.org/x/text?go-get=1
go: finding golang.org/x/sys v0.0.0-20180821140842-3b58ed4ad339
go: finding golang.org/x/sys v0.0.0-20181128092732-4ed8d59d0b35
go: finding golang.org/x/net v0.0.0-20181106065722-10aee1819953
go: finding golang.org/x/net v0.0.0-20180821023952-922f4815f713
go: finding golang.org/x/net v0.0.0-20180719180050-a680a1efc54d
go: finding golang.org/x/text v0.3.0
Parsing meta tags from https://cloud.google.com/go?go-get=1 (status code 200)
get "cloud.google.com/go": found meta tag get.metaImport{Prefix:"cloud.google.com/go", VCS:"git", RepoRoot:"https://code.googlesource.com/gocloud"} at https://cloud.google.com/go?go-get=1
go: finding cloud.google.com/go v0.26.0
go: finding golang.org/x/crypto v0.0.0-20170930174604-9419663f5a44
Fetching https://golang.org/x/sync?go-get=1
Parsing meta tags from https://golang.org/x/sync?go-get=1 (status code 200)
get "golang.org/x/sync": found meta tag get.metaImport{Prefix:"golang.org/x/sync", VCS:"git", RepoRoot:"https://go.googlesource.com/sync"} at https://golang.org/x/sync?go-get=1
go: finding golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f
go: golang.org/x/[email protected]: git fetch -f https://go.googlesource.com/sync refs/heads/*:refs/heads/* refs/tags/*:refs/tags/* in /home/lightning/gocode/pkg/mod/cache/vcs/55179c5d8c4db2eaed9fae4682d4c84a1fd3612df666b372bef3bbb997c9601f: exit status 128:
        fatal: unable to access 'https://go.googlesource.com/sync/': gnutls_handshake() failed: The TLS connection was non-properly terminated.
go: finding golang.org/x/sys v0.0.0-20180830151530-49385e6e1522
go: golang.org/x/[email protected]: git -c protocol.version=0 fetch --unshallow -f https://go.googlesource.com/crypto refs/heads/*:refs/heads/* refs/tags/*:refs/tags/* in /home/lightning/gocode/pkg/mod/cache/vcs/de5fd3af413a4f3f969455ae522b4002fcb7bb4c158f339396dfc77710c9007d: exit status 128:
        fatal: unable to access 'https://go.googlesource.com/crypto/': gnutls_handshake() failed: The TLS connection was non-properly terminated.
Fetching https://golang.org/x/oauth2?go-get=1
Parsing meta tags from https://golang.org/x/oauth2?go-get=1 (status code 200)
get "golang.org/x/oauth2": found meta tag get.metaImport{Prefix:"golang.org/x/oauth2", VCS:"git", RepoRoot:"https://go.googlesource.com/oauth2"} at https://golang.org/x/oauth2?go-get=1
go: finding golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be
go: finding golang.org/x/net v0.0.0-20180826012351-8a410e7b638d
Fetching https://gopkg.in/check.v1?go-get=1
go: finding golang.org/x/sync v0.0.0-20181108010431-42b317875d0f
go: golang.org/x/[email protected]: unknown revision 42b317875d0fa942474b76e1b46a6060d720ae6e
Parsing meta tags from https://gopkg.in/check.v1?go-get=1 (status code 200)
get "gopkg.in/check.v1": found meta tag get.metaImport{Prefix:"gopkg.in/check.v1", VCS:"git", RepoRoot:"https://gopkg.in/check.v1"} at https://gopkg.in/check.v1?go-get=1
go: finding gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405
go: error loading module requirements
Makefile:106: recipe for target 'build' failed
make: *** [build] Error 1

[cfromknecht]

@alevchuk what version of git are you running? others have reported seeing this when using older versions golang/go#26594

[alevchuk]

I have the stock git for Raspbian GNU/Linux 9
with latest update "git:armhf 1:2.11.0-3+deb9u4"
git version 2.11.0

[cfromknecht]

gotcha, i'd imagine that's recent enough. i'm running 2.17.2 fwiw

the failure seems to be:

go: finding golang.org/x/sync v0.0.0-20181108010431-42b317875d0f
go: golang.org/x/[email protected]: unknown revision 42b317875d0fa942474b76e1b46a6060d720ae6e

although I checked, and that is a valid commit in x/sync. Perhaps there was an issue updating the repo locally? Maybe try

GO111MODULE=on go get golang.org/x/sync@42b317875d0fa942474b76e1b46a6060d720ae6e

to ensure the package is updated locally, and then try running make again?

[alevchuk]

I was able to build.

I have a hypothesis why this happens. I had a large download, unrelated to LND, running in parrallel on my home network. After cancelling the download the LND build succeeded. "The TLS connection was non-properly terminated" errors likely were cause by the download (transient unavailability in the network). It appear that during fetch phase LND build proceeds even when some dependencies fails. This causes the build to fail.

It would be better to keep re-trying the fetch instead of proceeding and failing with a strange error later.

[JarenGlover]

fwiw, I just tried to install and got the same ltcd error twice

error message:
go: github.com/ltcsuite/[email protected]: unknown revision cdab10132e8c

I see cdab10132e8c exist via ltcsuite/ltcd@cdab10132e8c

The output of the command GO111MODULE=on go get github.com/ltcsuite/ltcd@cdab10132e8c is below

go: finding github.com/ltcsuite/ltcd v0.0.0-20180701083835-cdab10132e8c
go: github.com/ltcsuite/[email protected]: unknown revision cdab10132e8c
go: error loading module requirements

git version 2.17.2 (Apple Git-113)
go version go1.11.1 darwin/amd64

[alec-djinn]

I am having the same problem.

john@T800:/usr/lib$ cd $GOPATH/src/github.com/lightningnetwork/lnd
john@T800:~/gocode/src/github.com/lightningnetwork/lnd$ make
 Building debug lnd and lncli.
GO111MODULE=on go build -v -tags="dev" -o lnd-debug -ldflags "-X github.com/lightningnetwork/lnd/build.Commit=v0.5.1-beta-256-g089d461552ffc05d7b1c22d603af0fa82e788e85" github.com/lightningnetwork/lnd
go: finding github.com/ltcsuite/ltcd v0.0.0-20180701083835-cdab10132e8c
go: github.com/ltcsuite/[email protected]: unknown revision cdab10132e8c
go: error loading module requirements
Makefile:99: recipe for target 'build' failed
make: *** [build] Error 1
john@T800:~/gocode/src/github.com/lightningnetwork/lnd$ GO111MODULE=on go get golang.org/x/sync@42b317875d0fa942474b76e1b46a6060d720ae6e
go: finding github.com/ltcsuite/ltcd v0.0.0-20180701083835-cdab10132e8c
go: github.com/ltcsuite/[email protected]: unknown revision cdab10132e8c
go: error loading module requirements
john@T800:~/gocode/src/github.com/lightningnetwork/lnd$ git version
git version 2.17.1
john@T800:~/gocode/src/github.com/lightningnetwork/lnd$ go version
go version go1.11.4 linux/amd64

[alevchuk]

I made a proposal to solve this at the golang layer:
golang/go#29345

Yet, there maybe better solutions in other layers.