API Go module

[Joerger]

Changes:

• Initialize the api module and import it directly into the main teleport module.
• Update make update-vendor to delete api from vendor, and add a symlink in its place.

[Joerger]

@russjones @awly I'm not sure exactly what our vendoring strategy is. How do normally update dependencies, and are the upgrades here an issue?

[a-palchikov]

Regarding upgrades - these are better done intentionally, e.g. when specific issues are targeted. But regular upgrades are also possible, I guess with the caveat that things need to continue working afterwards.

[russjones]

@Joerger I'm not sure if we should vendor api here. @awly what do you think?

[awly]

See golang/go#33789 for the vendoring problem.
Can you try adding some post-processing steps in make update-vendor to remove the vendored copy of api/?

[awly]

See how consul does this: https://github.com/hashicorp/consul/blob/master/GNUmakefile#L225-L228

[Joerger]

See golang/go#33789 for the vendoring problem.
Can you try adding some post-processing steps in make update-vendor to remove the vendored copy of api/?

Post-processing does not seem to be possible in go 1.15, it leads to an "inconsistent vendoring error":

go: inconsistent vendoring in /home/bjoerger/gravitational/teleport:
	github.com/gravitational/teleport/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
	github.com/gravitational/teleport/api: is replaced in go.mod, but not marked as replaced in vendor/modules.txt

See how consul does this: https://github.com/hashicorp/consul/blob/master/GNUmakefile#L225-L228

If you upgrade Consol to go 1.14+ you get the same issue:

go: inconsistent vendoring in /home/bjoerger/examples/consul:
	github.com/hashicorp/consul/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
	github.com/hashicorp/consul/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
	github.com/hashicorp/consul/api: is replaced in go.mod, but not marked as replaced in vendor/modules.txt
	github.com/hashicorp/consul/sdk: is replaced in go.mod, but not marked as replaced in vendor/modules.txt
	github.com/gophercloud/[email protected]: is marked as explicit in vendor/modules.txt, but not explicitly required in go.mod

I spent a while trying to find a way around this in go 1.15, but it seems that partial vendoring of modules just isn't supported, and manual modification of vendor doesn't seem intended. I couldn't find any other way around vendoring api.

[awly]

@Joerger I got this working locally with a symlink:

.PHONY: update-vendor
update-vendor:
	go mod tidy
	go mod vendor
	rm -r vendor/github.com/gravitational/teleport/api
	ln -s $(shell readlink -f api) vendor/github.com/gravitational/teleport

The problem with keeping api vendored, is that the compiler looks at the version in vendor/. So if you make a change in api/ and forget to re-vendor, the change won't be picked up.

[Joerger]

@Joerger I got this working locally with a symlink

I changed this to a relative symlink to get it to work in docker without relinking.

[russjones]

Bot.