v2 go module semantic versioning

[optiman]

Describe the bug
I'm using go client library (/pkg/promtail/client) instead of promtail to push log entries to Loki.
After updating Loki to v2.0.0 I cannot use go client library in projects with go.mod because Loki's module path does not contain /v2 suffix as required by the go command.

To preserve import compatibility, the go command requires that modules with major version v2 or later use a module path with that major version as the final element. For example, version v2.0.0 of example.com/m must instead use module path example.com/m/v2, and packages in that module would use that path as their import path prefix, as in example.com/m/v2/sub/pkg.

To Reproduce

• mkdir myproject && cd myproject
• go mod init myproject
• go get github.com/grafana/[email protected]

Expected behavior
Be able to use Loki v2 go client library in projects with go.mod file (module-aware mode).

Environment:
local, linux

Screenshots, Promtail config, or terminal output
go get github.com/grafana/[email protected]: github.com/grafana/[email protected]: invalid version: module contains a go.mod file, so major version must be compatible: should be v0 or v1, not v2

[slim-bean]

Yes this was expected and unfortunately is (IMO) a limitation and annoyance with Go mod. I don't think it's appropriate that Go mod scans repos and uses tags to assign versions, this really limits what we can do for versioning a project which was never built with the intention of being a library.

We made the decision to use the 2.0.0 tag knowing it would break go mod.

Sadly, your use case is completely acceptable and we want to encourage people to use things like the promtail client package in their code.

However, we want the ability to control the Loki version number as it feels appropriate to us and the feature set in the project, not as it pertains to it's use as a go library.

This creates a dilemma for us, and we chose to go the path of tying the version number to the project state and features, not to the use as a library.

We appreciate this issue, apologies for my long response but you were lucky to be the first to find this 😃 and I want to have something to reference when others find this problem too!

The workaround is to install Loki based on commit hashes now and not version tags.

The commit hash for the 2.0.0 tag is 6978ee5d73878d7406bda68296a3dd8ecda4d538

❯ go get -u github.com/grafana/loki@6978ee5d73878d7406bda68296a3dd8ecda4d538                                                                                
go: github.com/grafana/loki 6978ee5d73878d7406bda68296a3dd8ecda4d538 => v1.6.2-0.20201026154740-6978ee5d7387
go: downloading github.com/grafana/loki v1.6.2-0.20201026154740-6978ee5d7387

I know this is not ideal, and we are considering alternatives for something like the client package you are using maybe it makes sense to put this in a separate repo, we are still undecided here.

Thank you for reporting this, sorry if my answer is not what you were hoping for!

[optiman]

I completely agree that this is a limitation and major annoyance while working with go modules, but it seems that they have become de-facto standard nowadays.
So we should come up with a way to use (preferebly without workarounds) the go client without incompatible changes to the Loki project itself.
I think, moving the client to a separate repo with module-compatible versioning is a sensible compromise.

And thank you for the workaround, this should work for the time being.

[jeschkies]

I think, moving the client to a separate repo with module-compatible versioning is a sensible compromise.

This probably makes sense. If the overhead of managing multiple repositories is too big one could import the promtail repo as a submodule or subrepo.

[stale]

This issue has been automatically marked as stale because it has not had any activity in the past 30 days. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

[optiman]

Still relevant.

[stale]

This issue has been automatically marked as stale because it has not had any activity in the past 30 days. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

[realvasko]

If using the commit hash is the official workaround for now, is it possible to include it somewhere in the notes? I tried pulling v2.1.0 with 1b79df3754f60426459ea51b84fb8a387c4b9ff5, but go mod tells me it's getting v1.6.2-0.20201223215703-1b79df3754f6, which is odd.

[stale]

This issue has been automatically marked as stale because it has not had any activity in the past 30 days. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

[kavirajk]

Actually we don't have to create subdirectories or anything to make it module compatible.

The idea of subdirectory comes from the official blog (which is unfortunate). because during that time, creating subdirectory was the only way for old go tools chain to understand the go modules semantics.

If we don't care about that (no one shouldn't anymore, particularly Loki shouldn't), we can just change our module path in go.mod every time we release major version.

e.g: we just have to change our go.mod in v2 branch and add "/v2" in our module path(it becomes module github.com/grafana/loki/v2). Literally that's the only change needed.

This practice is covered in Go Wiki. https://github.com/golang/go/wiki/Modules#releasing-modules-v2-or-higher

Also as a proof of concept I created this repository with multiple major versions (with separate branch) and involving only change in module path (no subdirectories or anything)

https://github.com/kavirajk/v

And usage here https://play.golang.org/p/ZVumxvdSw1L

I can create a PR to see if anything else breaks!

// @grafana/loki-team @slim-bean

[optiman]

Was separate client idea abandoned? I see some work being done here, but not so much recently.
https://github.com/grafana/loki-client-go

Semantic versioning isn't the only problem when using go client.
There are a lot of replace directives in go.mod of this repository, not every one of them necessary for the client, only for Loki itself.
So when you use the same dependencies in your project, and then add github.com/grafana/loki to your go.mod just to use the client, you are now restricted by those versions, that the whole Loki project uses, otherwise compilation fails.
At least that was my experience.

[stale]

This issue has been automatically marked as stale because it has not had any activity in the past 30 days. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

[kavirajk]

[cross posting here for more context from #4352]

Even if we decide to go with supporting our module path with major version, I think this can happen only on next major release (say Loki v3). Purely because of technical reasons mentioned on the Go wiki

For example, if you are the author of foo, and the latest tag for the foo repository is v2.2.2, 
and foo has not yet adopted modules, then the best practice would be to use 
v3.0.0 for the first release of foo to adopt modules (and hence the first release of foo to contain a go.mod file).

[prochac]

Workaround:

go.mod

require github.com/grafana/loki v0.0.0-20231016134057-a17308db6f24 // v2.9.2

main.go

import _ "github.com/grafana/loki/pkg/loki" // github.com/grafana/loki is not a Go package

[drornir-velocity]

Releasing v3 would be amazing, ty

[chaudum]

Has been addressed with Loki 3.0 release