Update Open Telemetry version

[burdiyan]

Background

We use some pieces of LND in our own code, importing it as a library. Particularly zpay32 package.

By only importing this one package, we're bringing all of the LND's dependencies, including etcd, which is quite unfortunate, but wouldn't be a problem if not for one thing: the opentelemetry version used by LND is very old, and some other packages that we use require a newer version which is not compatible.

OpenTelemetry now have released v1 of their Go packages, and it's incompatible with v0, but since it has the same import path (because it's not v2), we can't import both of them.

Steps to reproduce

1. Create a new Go module.
2. Import the zpay32 package from LND v0.15.1-beta.
3. Import the latest version of open telemetry tracing package.

Expected behaviour

Everything should work.

Actual behaviour

Go tool complains because because it can't use open telemetry v1.10.0 while LND is using v0.32.0.

[burdiyan]

I looked more carefully into the go mod logs:

github.com/lightningnetwork/lnd/zpay32 imports
	github.com/lightningnetwork/lnd/lnwire imports
	github.com/lightningnetwork/lnd/input tested by
	github.com/lightningnetwork/lnd/input.test imports
	github.com/lightningnetwork/lnd/channeldb imports
	github.com/lightningnetwork/lnd/kvdb imports
	github.com/lightningnetwork/lnd/kvdb/etcd imports
	go.etcd.io/etcd/server/v3/embed imports
	go.opentelemetry.io/otel/exporters/otlp imports
	go.opentelemetry.io/otel/sdk/metric/controller/basic imports
	go.opentelemetry.io/otel/metric/registry: module go.opentelemetry.io/otel/metric@latest found (v0.32.0), but does not contain package go.opentelemetry.io/otel/metric/registry

It seems like the problem is inside etcd, not lnd, because open telemetry is only an indirect dependency of lnd. So, I guess there's little lnd can do about that, unless maybe making utility packages like lnwire and zpay32 not depend on etcd transitively by default.

The unfortunate thing in this particular case is that it's a testing package that depends on etcd in this case: github.com/lightningnetwork/lnd/input.test.

[guggero]

Thanks for the report. I think this can be solved by upgrading the etcd versions in the kvdb/go.mod file to v3.5.5 which seems to have bumped the opentelemetry libraries.

The dependency from input to channeldb (all the way down to etcd) is quite unfortunate. Though I think that requires a bigger refactor to fix.

[burdiyan]

For now, I solved this by using the exclude directive in our main go.mod file to exclude go.etcd.io/etcd/server/v3 entirely, and it seems to be doing the trick.

[l0k18]

I'm hitting this problem too just in trying to import lnwire so I can use the MilliSatoshi type as part of an interface to lnd payments for the app I'm developing. And it's not only the opentelemetry library that is broken, btcutil leads to attempting to access a repo that doesn't exist:

github.com/indra-labs/indra/pkg/node imports
        github.com/lightningnetwork/lnd/lnwire imports
        github.com/BitfuryLightning/tools/rt: cannot find module providing package github.com/BitfuryLightning/tools/rt: module github.com/BitfuryLightning/tools/rt: git ls-remote -q origin in /home/loki/pkg/mod/cache/vcs/84d5d689b315503bfc95e8ea3db36d0c8f7e3e0503eb905f712c6db975055774: exit status 128:
        fatal: could not read Username for 'https://github.com': terminal prompts disabled
Confirm the import path was entered correctly.
If this is a private repository, see https://golang.org/doc/faq#git_https for additional information.

@guggero you may recall my previous attempt to close the issue relating to removing btcd's dependencies from the generated RPC... I can't find a way to work around this issue and MilliSatoshi is such a trivial type... but I think I'm gonna wind up with this same mess if I try to import the RPC libraries as well for the invoice subscription/send functions I need.

It would be really nice if it was possible to just drop "github.com/lightningnetworks/lnd" into the import block and not have to make workarounds. The state of btcutil's location is really messy, and the sheer volume of indirect imports you end up with is kinda working against the benefits of writing this all in Go...

For now it looks like I just need to copy and paste, since most likely all the things involved really don't seem like they will change much.

I can understand how things get like this though, don't get me wrong. Go modules are pretty stable but sometimes still really confusing and opaque.

[guggero]

@l0k18 what exact version of lnd do you reference in your go.mod? We don't import github.com/BitfuryLightning/tools/rt anywhere I'm aware of... So something sounds weird.

There is an attempt to upgrade etcd to a more recent version which should solve the original issue, but that seems to have its own challenges.

[l0k18]

I was using the latest tagged version. Perhaps there is a recommended tag to use at the moment?

[guggero]

IIRC we fixed some lnd internal circular dependency. So I'd try with master (if you're using lnd as a library it should generally be fine to use the master branch, only when running a mainnet node you should only ever used tagged/released versions).

But I still wonder where the github.com/BitfuryLightning/tools/rt dependency in the error comes from. Is that something you added to your project?

[l0k18]

The error is very clear, and the imported version at the time that error printed was v0.15.5-beta. But the source could be the go modules system, and probably several levels deep related to btcutil and its forks and refactorings. There is an inordinate number of forks of that and it is plain that some mess may have got stuck in the Go mod proxies. My money is on a go modules bug.

[l0k18]

I found a tag that works, v0.14.4-beta. Newer ones in the v0.15.x range seem to be bringing in problems via etcd among other things. This eliminates the opentelemetry import problem, and even, it printed this after I set it right:

go: upgraded github.com/btcsuite/btcutil v1.0.2 => v1.0.3-0.20210527170813-e2ba6805a890
go: added go.opentelemetry.io/otel/internal/metric v0.27.0
go: upgraded go.opentelemetry.io/otel/metric v0.20.0 => v0.27.0

So there is something amiss with the 15 branches and the etcd and btcutil.

[l0k18]

This is the answer to your question I think:

go: finding module for package github.com/BitfuryLightning/tools/rt/graph
go: finding module for package github.com/roasbeef/btcd/btcec
go: finding module for package github.com/BitfuryLightning/tools/rt
go: finding module for package github.com/roasbeef/btcd/wire
go: finding module for package github.com/roasbeef/btcutil
go: finding module for package github.com/roasbeef/btcd/txscript
go: found github.com/roasbeef/btcd/btcec in github.com/roasbeef/btcd v0.0.0-20180418012700-a03db407e40d
go: found github.com/roasbeef/btcd/txscript in github.com/roasbeef/btcd v0.0.0-20180418012700-a03db407e40d
go: found github.com/roasbeef/btcd/wire in github.com/roasbeef/btcd v0.0.0-20180418012700-a03db407e40d
go: found github.com/roasbeef/btcutil in github.com/roasbeef/btcutil v0.0.0-20180406014609-dfb640c57141
go: finding module for package github.com/btcsuite/golangcrypto/ripemd160
go: finding module for package github.com/BitfuryLightning/tools/rt/graph
go: finding module for package github.com/Roasbeef/btcutil/bech32
go: finding module for package github.com/BitfuryLightning/tools/rt
go: found github.com/btcsuite/golangcrypto/ripemd160 in github.com/btcsuite/golangcrypto v0.0.0-20150304025918-53f62d9b43e8
go: found github.com/Roasbeef/btcutil/bech32 in github.com/Roasbeef/btcutil v0.0.0-20180406014609-dfb640c57141
go: finding module for package github.com/BitfuryLightning/tools/rt/graph
go: finding module for package github.com/BitfuryLightning/tools/rt
github.com/indra-labs/indra/pkg/node imports
        github.com/lightningnetwork/lnd/lnwire imports
        github.com/BitfuryLightning/tools/rt: cannot find module providing package github.com/BitfuryLightning/tools/rt: module github.com/BitfuryLightning/tools/rt: git ls-remote -q origin in /home/loki/pkg/mod/cache/vcs/84d5d689b315503bfc95e8ea3db36d0c8f7e3e0503eb905f712c6db975055774: exit status 128:
        fatal: could not read Username for 'https://github.com': terminal prompts disabled
Confirm the import path was entered correctly.

I think it has to do with go versions also.

[l0k18]

https://pkg.go.dev/github.com/lightningnetwork/lnd

godoc certainly doesn't seem to be seeing this entire repository since basically it was first uploaded october 2016!

[loki@pop-os indra]$ go mod tidy
go: finding module for package github.com/lightningnetwork/lnd/lnwire
go: downloading github.com/lightningnetwork/lnd v0.0.2
go: found github.com/lightningnetwork/lnd/lnwire in github.com/lightningnetwork/lnd v0.0.2

Exactly the same appears with go mod also.

edit:

It really looks like lnd simply has not been used in this way before and with the makefile a lot of standard go tools have been bypassed. I am just going to pick a version to use with our app and rip out lnwire and lnrpc at that tag and work from that.

[guggero]

It looks like you have a weird version in one of your go.mod. Unfortunately there was an ancient version (v0.0.2) published that didn't have the -beta suffix in its version, so all go mod tools automatically select that one if you don't specify an explicit one.
So things should work as expected if you don't use @latest or no exact version.

[l0k18]

I'm going to have a go at fixing this problem on a fork so that @latest doesn't break things with go mod tidy, I'll get back to you if I find a workaround that fully fixes it.

[guggero]

I don't think there is a "fix" other than releasing another version without the -beta tag. See #6860, though I don't think we're quite ready to declare lnd non-beta.

[l0k18]

My only solution is to fork and add my own tag so Go modules doesn't question it.

I think you should read the documentation on Semver and Go Modules and reevaluate your tag naming scheme.

[guggero]

Fixed by #7252.