-
Notifications
You must be signed in to change notification settings - Fork 17.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
cmd/go: support vendoring in workspace mode #60056
Comments
The lack of vendor support makes go workspaces very difficult for Kubernetes to adopt. I don't care if it is exactly identical to classic vendor support, but we need something...
|
Change https://go.dev/cl/495801 mentions this issue: |
Change https://go.dev/cl/499977 mentions this issue: |
We might be introducing vendoring for workspace mode. Set GOWORK=off when running go mod vendor in a single module context to make sure it has the right behavior and doesn't return an error. For golang/go#60056 Change-Id: I703d74d579aec6e4dad86ca092e3651e0b2e4eb0 Reviewed-on: https://go-review.googlesource.com/c/tools/+/499977 TryBot-Result: Gopher Robot <[email protected]> Run-TryBot: Michael Matloob <[email protected]> gopls-CI: kokoro <[email protected]> Reviewed-by: Robert Findley <[email protected]>
This proposal has been added to the active column of the proposals project |
Based on the discussion above, this proposal seems like a likely accept. |
No change in consensus, so accepted. 🎉 |
In most cases this change removes assumptions that there is a single main module in vendor mode and iterates over the workspace modules when doing checks. The go mod vendor command will now, if in workspace mode, create a vendor directory in the same directory as the go.work file, containing the packages (and modules in modules.txt) loaded from the workspace. When reassembling the module graph from the vendor directory, an edges are added from each of the main modules to their requirements, plus additionally to a fake 'vendor/modules.txt' module with edges to all the modules listed in vendor/modules.txt. For #60056 Change-Id: I4a485bb39836e7ab35cdc7726229191c6599903e Reviewed-on: https://go-review.googlesource.com/c/go/+/495801 Reviewed-by: Bryan Mills <[email protected]> TryBot-Result: Gopher Robot <[email protected]> Run-TryBot: Michael Matloob <[email protected]>
Change https://go.dev/cl/513756 mentions this issue: |
modules.txt gets a new ## workspace line at the start of the file if it's generated in workspace mode. Then, when deciding whether the go command runs in mod=vendor, we only do so if we're in the same mode (workspace or not) as the modules.txt specifies. For #60056 Change-Id: If478a9891a7135614326fcb80c4c33a431e4e531 Reviewed-on: https://go-review.googlesource.com/c/go/+/513756 TryBot-Result: Gopher Robot <[email protected]> Reviewed-by: Bryan Mills <[email protected]> Run-TryBot: Bryan Mills <[email protected]>
Is your concern about "depending on the environment" regarding switching between workspace mode on vs. off? I believe all Go invocations in k8s will be done in workspace mode. @thockin knows best, but I've been following along and was trying to help at one point by implementing a hacky solution for a pseudo-vendor dir in workspace mode. That hack thankfully became obsolete with this feature here! |
Sigh, I really need to finish this work. Kubernetes has a top-level module and workspace. |
Per discussion with @rsc, we think this is done. |
Fine to close now. @thockin if you do run into any problems, please let us know. |
Yes, I was worried this change could interfere with some existing workspace mode users' current workflow who want to keep vendor in non-workspace mode while keeping their go.work files in the repo root. @rsc informed me that the latest change cl 513756 addressed it. The I tested with my existing workflow, and it seems ok so far. :-) |
Change https://go.dev/cl/548815 mentions this issue: |
For #61422. Updates #60056. Change-Id: Ie7c7e12acc173f1ff1644555016e51a52509bd6d Reviewed-on: https://go-review.googlesource.com/c/go/+/548815 Auto-Submit: Bryan Mills <[email protected]> Reviewed-by: Michael Matloob <[email protected]> LUCI-TryBot-Result: Go LUCI <[email protected]>
For golang#61422. Updates golang#60056. Change-Id: Ie7c7e12acc173f1ff1644555016e51a52509bd6d Reviewed-on: https://go-review.googlesource.com/c/go/+/548815 Auto-Submit: Bryan Mills <[email protected]> Reviewed-by: Michael Matloob <[email protected]> LUCI-TryBot-Result: Go LUCI <[email protected]>
If go.work is present and a vendor/ is present, then the module is using go workspaces with vendoring, recently introduced with Go 1.22 (see golang/go#60056 for details). When using Go workspaces, you must use `go work vendor` instead of `go mod vendor`.
The ability to vendor dependencies in projects that make use of workspaces was introduced in Go 1.22¹. Cachito currently does not support the use of "go work vendor", so this commit introduces an explicit check for that scenario and a clearer error message. Note that by design, Go does not allow the use of "go mod vendor" if workspaces are present, so the check assumes that the existence of a vendor folder in a project with workspaces means that workspace vendoring is being used. ¹ golang/go#60056 Signed-off-by: Bruno Pimentel <[email protected]>
The ability to vendor dependencies in projects that make use of workspaces was introduced in Go 1.22¹. Cachito currently does not support the use of "go work vendor", so this commit introduces an explicit check for that scenario and a clearer error message. Note that by design, Go does not allow the use of "go mod vendor" if workspaces are present, so the check assumes that the existence of a vendor folder in a project with workspaces means that workspace vendoring is being used. ¹ golang/go#60056 Signed-off-by: Bruno Pimentel <[email protected]>
The ability to vendor dependencies in projects that make use of workspaces was introduced in Go 1.22¹. Cachito currently does not support the use of "go work vendor", so this commit introduces an explicit check for that scenario and a clearer error message. Note that by design, Go does not allow the use of "go mod vendor" if workspaces are present, so the check assumes that the existence of a vendor folder in a project with workspaces means that workspace vendoring is being used. ¹ golang/go#60056 Signed-off-by: Bruno Pimentel <[email protected]>
The ability to vendor dependencies in projects that make use of workspaces was introduced in Go 1.22. Cachito currently does not support the use of "go work vendor", so this commit introduces an explicit check for that scenario and a clearer error message. Note that by design, Go does not allow the use of "go mod vendor" if workspaces are present, so the check assumes that the existence of a vendor folder in a project with workspaces means that workspace vendoring is being used. For more info, see: - https://go.dev/doc/go1.22#tools - golang/go#60056 Signed-off-by: Bruno Pimentel <[email protected]>
The ability to vendor dependencies in projects that make use of workspaces was introduced in Go 1.22. Cachito currently does not support the use of "go work vendor", so this commit introduces an explicit check for that scenario and a clearer error message. Note that by design, Go does not allow the use of "go mod vendor" if workspaces are present, so the check assumes that the existence of a vendor folder in a project with workspaces means that workspace vendoring is being used. For more info, see: - https://go.dev/doc/go1.22#tools - golang/go#60056 Signed-off-by: Bruno Pimentel <[email protected]>
The ability to vendor dependencies in projects that make use of workspaces was introduced in Go 1.22. Cachito currently does not support the use of "go work vendor", so this commit introduces an explicit check for that scenario and a clearer error message. Note that by design, Go does not allow the use of "go mod vendor" if workspaces are present, so the check assumes that the existence of a vendor folder in a project with workspaces means that workspace vendoring is being used. For more info, see: - https://go.dev/doc/go1.22#tools - golang/go#60056 Signed-off-by: Bruno Pimentel <[email protected]>
This is a proposal to support vendoring in workspace mode.
We originally didn't support vendoring of a workspace to simplify the implementation of workspaces, but there's recently been demand by users of vendoring who want to workspaces for that support, so now is a good time to consider adding that support in.
go work vendor
subcommandUnder this proposal, a new subcommand,
go work vendor
, will be added to thego
command.go work vendor
will check to make sure that workspace mode is active when it starts, (and return an error if it isn't) and if it is, it will write a vendor directory for the workspace. The vendor directory will be created in the same directory as thego.work
file. The vendor directory will contain packages taken from the modules in the build list of the workspace. vendor/modules.txt will contain the set of packages from each of the modules in the build list as usual, will list modules with# explicit
annotations for each module required by one of the workspace modules, and will annotate replacements for each of the effective replacements in the workspace (That is, if thego.work
file overrides replacements in workspace only the replacement in thego.work
file will be annotated.)go work vendor
otherwise behaves the same as and takes the same flags asgo mod vendor
.go mod vendor
subcommandgo mod vendor
will explicitly fail if ago.work
file is found andGOWORK
is not set tooff
. See discussion below.-mod=vendor
in workspace mode.Under this proposal, when workspace mode is active, if
-mod=vendor
is set explicitly, or it is set implicitly through the existence of a vendor directory in the same directory as thego.work
The go command will enter workspace mode and interpret the vendor directory as reflecting the workspace's modules. The go command will use the vendor directory in the same directory asgo.work
.vendor/modules.txt
is interpreted as reflecting the modules in the workspace. That interpretation is done through the vendoring consistency check, and the structure of the internal module graph built.The vendoring consistency check would check
vendor/modules.txt
against the workspace modules. It will check that all explicitly required modules appear invendor/modules.txt
with an# explicit
annotation and all modules invendor/modules.txt
with# explicit
are required by a workspace module. Similarly, all effective replacements in the workspace are expected to appear in vendor/modules.txt and all replacements in vendor/modules.txt are expected to appear in a workspace modulego.mod
or thego.work
file.The way the module graph is built is similar to to that of an unpruned module. The module graph contains the main modules as its roots, and each of the main modules would require their module requirements in addition to a fake
vendor/modules.txt
module that in turn requires the set of modules in thevendorList
(the set of modules in vendor/modules.txt` that provide packages.Otherwise, vendor mode with a workspace behaves the same as vendor mode with a module. The packages in the vendor directory are used to satisfy imports in the main modules when loading packages and doing builds.
Other considerations.
go work vendor
andgo mod vendor
.Under the proposal as it is,
go mod vendor
works to vendor the module it's in, but fails if there's ago.work
file present andGOWORK
is not set to off. If that wasn't the case, there could be confusion if ago.mod
andgo.work
files exist in the same directory. Thengo mod vendor
andgo work vendor
would create vendor directories in the same place, but with different contents. Since in workspace mode, the vendor directory is interpreted as a reflecting the workspace, ifgo mod vendor
is run before another command in workspace mode, the vendor consistency checks will likely fail. When they fail the error message will give the user the appropriate command to rungo mod vendor
orgo work vendor
depending on whethermodules.txt
was checked against a single module or a the workspace. Even so, the user might still get confused about why runninggo mod vendor
and immediately running another command might result in an error. To reduce the confusion, we require a user to not have ago.work
file present or haveGOWORK
set tooff
whengo mod vendor
is run, so that when a following command is run under the same conditions, it doesn't interpret the vendor directory as reflecting the workspace.The behavior might break already existing workflows where
go mod vendor
is run when ago.work
is present and module vendoring is being used, but it seems better to do so than to introduce the potential confusion of the user creating a vendor directory that fails consistency checks.The text was updated successfully, but these errors were encountered: