adding /v3/ to urls is a breaking change when building.

[hayres]

When building the project there is no such place as

```https://github.com/golang-migrate/migrate/v3/``

so the build cannot import its dependencies and the build fails.

Steps to Reproduce

Build the project as below

go build -tags 'postgres' -o ~/go/bin/migrate github.com/golang-migrate/migrate/cli

Expected Behavior
I expect the binary to be built and deployed to $GOPATH/bin/migrate

Migrate Version
v3.5.2

[dhui]

Hmmm, wonder if I should have use v4...

What version of go are you using? e.g. output of go version

[hayres]

The version of go is irrelevant. The URI used in the location does not exist, so building the project as a dependency means the build cannot find the location https://github.com/golang-migrate/migrate/v3 . or anything below it.

If you clone the repo and open it in an IDE the IDE is showing this bug.

If you try and browse to github.com/golang-migrate/migrate/v3/database you will find it does not exist. If you browse to github.com/golang-migrate/migrate/database you will find it does exist.

There are two versions of a fix.
Create the location https://github.com/golang-migrate/migrate/v3/* and populate it with the relevant code.
Or change the code to remove the /v3/

Your call.

We moved to download the tarball and use it but first, we experienced a breaking change. Developers don't like that too much.

[dhui]

The version of go is irrelevant.

The version of Go is quite relevant to this discussion.
The latest version of migrate (v3.5.2) added support for Go modules. This was documented in the release notes. Go modules were introduced in Go 1.11 and both Go 1.11 and Go 1.10.4 (latest version of Go 1.10.x) are module aware. As you can see, migrate v3.5.2 builds and passes tests in Go 1.11 and 1.10.4. The v3 you're seeing in the import path is handled by the versions of Go mentioned above.

However, all that being said, a goal of this project is to correctly follow SemVer so that it's obvious when a breaking change is made. So I may revert support for Go modules in v3 and support Go modules in v4, depending on the breakage that's being experienced.

Are you using anything to manage your dependencies? If so, for now, if you pin migrate to a version before v3.5.1, you should be able to build again.

[strindhaug]

How do you actually pin to a specific version?

I'm trying to fix a CircleCI script and suddenly the build now fails when trying to install migrate. (I didn't write the original project; I'm only updating the configuration for the build, so I only have a vague idea how this project is supposet to work. And I have no real experience with go programming.)

The current relevant part of the config that fails is:

apk add go glide musl-dev
go get github.com/lib/pq github.com/golang-migrate/migrate/cli
glide install
go build -tags 'postgres' -o migrate github.com/golang-migrate/migrate/cli

I tried pinning using the url for the v3.5.0 tag like this:
go get github.com/lib/pq github.com/golang-migrate/migrate/tree/v3.5.0/cli
But that didn't work.
And I tried to add it to the glide.yml instead:

package: github.com/mycompany/myproject
import:
- package: github.com/golang-migrate/migrate/cli
  version: v3.5.0
  # ...

But then the go build -tags 'postgres' -o migrate github.com/golang-migrate/migrate/cli line failed.

---

Also, I cannot find that v3 directory in any older tags either, e.g. https://github.com/golang-migrate/migrate/tree/v3.4.0/v3 is nonexistent
Oh. I see the the references to the v3 directory is in the code added in v3.5.2, but this tag does not add the v3 directory for some reason? How were you able to run the tests on v3.5.2, cause when I checkout this tag, and run dep ensure then make test-short i get screenfuls of:

migrate.go:13:2: cannot find package "github.com/golang-migrate/migrate/v3/database" in any of:
	/home/stein/go/src/github.com/golang-migrate/migrate/vendor/github.com/golang-migrate/migrate/v3/database (vendor tree)
	/usr/lib/go-1.10/src/github.com/golang-migrate/migrate/v3/database (from $GOROOT)
	/home/stein/go/src/github.com/golang-migrate/migrate/v3/database (from $GOPATH)
migrate.go:14:2: cannot find package "github.com/golang-migrate/migrate/v3/source" in any of:
	/home/stein/go/src/github.com/golang-migrate/migrate/vendor/github.com/golang-migrate/migrate/v3/source (vendor tree)
	/usr/lib/go-1.10/src/github.com/golang-migrate/migrate/v3/source (from $GOROOT)
	/home/stein/go/src/github.com/golang-migrate/migrate/v3/source (from $GOPATH)
Makefile:36: recipe for target 'test-with-flags' failed
... etc

(Which makes sense as there is no /usr/lib/go-1.10/src/github.com or /home/stein/go/src/github.com/golang-migrate/migrate/v3 directories on my computer after checking out and installing deps.)

Am I missing something?

[dhui]

@strindhaug What version of Go are you using? e.g. run go version
You'll need 1.9.7+, 1.10.3+, or 1.11

Per the Go module wiki:

To help with this, Go versions 1.9.7+, 1.10.3+ and 1.11 have been updated so that code built with those releases can properly consume v2+ modules without requiring modification of pre-existing code. (When relying on this updated mechanism, a package that has not yet opted in to modules would not include the major version in the import path for any imported v2+ modules. In contrast, a package that has opted in to modules must include the major version in the import path for any imported v2+ modules).

Migrate uses the first option to opt into modules with a v2+ module

[strindhaug]

So what you're saying is that even the latest beta build of docker:18.09.0-ce-beta1 ( https://hub.docker.com/r/library/docker/ ) is too old to use golang-migrate now?
Since it's "only" using alpine:3.8 where the latest version of go is 1.10.1-r0; I have to make my own "even-more-bleeding-edge" clone of docker:18.09.0-ce-beta1 that uses alpine:edge (i.e. rolling release) to get go version 1.11-r2 in order to run it?

[dhui]

@strindhaug
Docker has nothing to do with this. It doesn't matter if your app/service uses migrate in a Docker container as long as the version of migrate is compatible with the version of Go in the docker container.

The latest version of migrate clearly runs without any issues in a docker container:

$ docker pull migrate/migrate
Using default tag: latest
latest: Pulling from migrate/migrate
Digest: sha256:56f31ebe8cb7ae27eaf391865f0b3857767ee780ad8f93bd7a0668e718ecf76d
Status: Image is up to date for migrate/migrate:latest
$ docker run --rm migrate/migrate -version
v3.5.2
$

If you're running Go in a docker container, I suggest using one of docker's official Go image

[andrewpmartinez]

Seeing a similar issue when using govendor. I assume this means govendor doesn't support the new go modules?

[dhui]

@andrewpmartinez
I'm not familiar with govendor, but presumably, if you've pinned migrate to a version before v3.5.2 and govendor honors that, you won't have any issues.

If the version pin is not honored, you'll need to run Go 1.9.7+, 1.10.3+, or 1.11 and update your imports to include v3.

Honestly, it's pretty straight forward to run Go 1.11 and use Go modules. go mod init will automatically convert a bunch of existing dependency formats for you.
Spoiler alert: govendor's vendor/vendor.json is supported

[andrewpmartinez]

@dhui

govendor supports govendor add <repo> which vendors from $GOPATH so one can choose between pinned versions or fetching (govendor fetch <repo>) from a remote host.

Thanks for your help. I appreciate it!

[olivere]

Hmmm, wonder if I should have use v4...

What version of go are you using? e.g. output of go version

@dhui Just my 2 cents: I ran into the same issue with github.com/olivere/elastic. See olivere/elastic#877. In the end I delayed the move to Go modules until v7.

The only way I found (later) to make it work for all constellations was to move all code from $GOPATH/src/github.com/olivere/elastic to a newly added $GOPATH/src/github.com/olivere/elastic/v6 directory. That seems to make it work for all earlier versions of Go (including those that don't have minimal module awareness) plus all tooling like dep and (presumably) govendor. The problem is mostly tooling, not Go. E.g. dep doesn't have minimal module awareness as of now and therefore doesn't handle Go import paths with version numbers correctly (see golang/dep#1962). That's why I came here btw...

Notice that I don't blame dep for anything: It's a marvel. But I found that adding Go modules breaks things for my users, and they are more important to me than introducing Go modules.

BTW: If you decide to remove Go module support introduced in 3.5.2, you should also remove the go.mod and go.sum files. Otherwise, people who use Go modules with (the then non-Go-module version of) github.com/golang-migrate/migrate will not get the most up-to-date version (see olivere/elastic#903 (comment)).

[dhui]

@olivere Thanks for sharing! That info was really helpful.

It seems like the root cause for this issue is due the following:

1. Opting into Go modules
   • My incorrect usage of SemVer didn't help either. I thought that opting into Go modules would not be a breaking change but clearly, requiring v3 to be added to the import path is a breaking change.
2. Package/dependency managers not being module aware

migrate isn't quite ready for a v4 release yet as there are some breaking changes that haven't been developed yet that I'd like to introduce.
Regardless, I'm not sure that removing module support in v3 and adding module support in v4 would actually solve the problem for package mangers that aren't module aware since they'll hit the same problem they're having now.

This essentially leaves package maintainers who have packages released with versions >= v2 with the following options:

1. Support Go modules and force all users to update their imports
2. Don't support Go modules and have inconsistent builds
   • This is a problem for migrate which has a bunch of dependencies for DB and source drivers

Go modules are the future as all other package managers will be superseded by Go modules. Thus other package managers will not be supported by migrate. The question now becomes: when does migrate support Go modules?

Until this issue is resolved, I'd suggest that if you want to opt-out of Go modules, don't upgrade to migrate v3.5.2. The v3.5.2 release notes have been updated to mention the backwards incompatible change and provide a command to help update all of your imports.

Poll

Since there doesn't seem to be a clear correct solution with migrate being in a weird state with Go modules and version numbers, what would people prefer?
(vote using the emoji)

1. 😄 - No changes, and keep the status quo.
2. 🎉 - Back out support for Go modules in v3.5.3 and don't support Go modules. Anyone who has already updated their imports to use v3 would have to remove them.
3. 😕 - Back out support for Go modules in v3.5.3 and cut a new v3.6.0 release which supports Go modules. This is still incorrect usage of SemVer.
4. ❤️ - Back out support for Go modules in v3.5.3 and cut a new v4 release which supports Go modules. Anyone who has already updated their imports to use v3 would have to change their imports again to use v4. v4 may still break import paths with module unaware dependency managers.

[dhui]

As of 10/10/2018, there are 8 ❤️ s and no other votes/emoji, so we'll go w/ option 4:

Back out support for Go modules in v3.5.3 and cut a new v4 release which supports Go modules. Anyone who has already updated their imports to use v3 would have to change their imports again to use v4. v4 may still break import paths with module unaware dependency managers.

[dhui]

The change has been made. v3.5.3 (and v3.5.4 if you're using docker) no longer use Go modules. v4 has been release with Go module support.

Lemme know if you continue to have issues.

[dhui]

Issue should be resolved now