cmd/go/internal/modload: refactor global state into exported struct type #40775
Comments
jayconrod
added
NeedsFix
|
Another problem to add: |
|
Another problem: CL 237017 was a step in this direction, but we should confirm other commands like |
|
Change https://golang.org/cl/249398 mentions this issue: |
CL 237017 caused 'go list' to call modload.WriteGoMod once instead of twice, which changed the order of requirements in a temporary go.mod file created by gorelease. Previously, WriteGoMod was called once after the go version was added, then again after packages were loaded. The first call did not sort requirements, since packages hadn't been loaded. The second call didn't write go.mod since no requirements were missing and go.mod was not considered dirty. This change makes it so gorelease will only report a diagnostic when requirements are actually missing. the order of requirements no longer matters. Updates golang/go#40775 Change-Id: I5b42104207fbd88dd849b68fb2b5d1ab50ea48c7 Reviewed-on: https://go-review.googlesource.com/c/exp/+/249398 Reviewed-by: Bryan C. Mills <[email protected]>
|
Unfortunately, the In the meantime, it should be possible to address the other symptoms you report without eliminating the globals. (For example, |
|
Change https://golang.org/cl/255938 mentions this issue: |
This change replaces ImportPaths, ImportPathsQuiet, LoadALL, and LoadVendor with a single LoadPackages function, with a LoadOpts struct that more clearly documents the variations in behavior. It also eliminates the cmd/go/internal/load.ImportPaths function, which was undocumented and had only one call site (within its own package). The modload.LoadTests global variable is subsumed by a field in the new LoadOpts struct, and is no longer needed for callers that invoke LoadPackages directly. It has been (temporarily) replaced with a similar global variable, load.ModResolveTests, which can itself be converted to an explicit, local argument. For #37438 For #36460 Updates #40775 Fixes #26977 Change-Id: I4fb6086c01b04de829d98875db19cf0118d40f8c Reviewed-on: https://go-review.googlesource.com/c/go/+/255938 Trust: Bryan C. Mills <[email protected]> Trust: Jay Conrod <[email protected]> Run-TryBot: Bryan C. Mills <[email protected]> TryBot-Result: Go Bot <[email protected]> Reviewed-by: Michael Matloob <[email protected]> Reviewed-by: Jay Conrod <[email protected]>
|
Change https://golang.org/cl/261640 mentions this issue: |
…ttern The Query function allows the caller to specify the current version of the requested module, but the QueryPattern function is missing that parameter: instead, it always assumes that the current version is the one selected from the global build list. This change removes that assumption, instead adding a callback function to determine the current version. (The callback is currently invoked once per candidate module, regardless of whether that module exists, but in a future change we can refactor it to invoke the callback only when needed.) For #36460 For #40775 Change-Id: I001a4a8ab24f5b4fcc66a670d9bd305b47e948ba Reviewed-on: https://go-review.googlesource.com/c/go/+/261640 Trust: Bryan C. Mills <[email protected]> Run-TryBot: Bryan C. Mills <[email protected]> TryBot-Result: Go Bot <[email protected]> Reviewed-by: Jay Conrod <[email protected]> Reviewed-by: Michael Matloob <[email protected]>
|
Change https://golang.org/cl/272129 mentions this issue: |
The Reqs function returns an mvs.Reqs implemention for the global build list. The API that it presents assumes that the build list is globally consistent (problematic for #40775) and readily available (problematic for #36460). Fortunately, it is no longer used outside of the modload package. We can instead use individual instances of the unexported mvsReqs struct, making the dependency on the global build list more explicit. For #36460 For #40775 Change-Id: I8674442f2a86416b0bf9c3395cb591c1e724c9d2 Reviewed-on: https://go-review.googlesource.com/c/go/+/272129 Trust: Bryan C. Mills <[email protected]> Run-TryBot: Bryan C. Mills <[email protected]> TryBot-Result: Go Bot <[email protected]> Reviewed-by: Jay Conrod <[email protected]>
|
Change https://golang.org/cl/293689 mentions this issue: |
|
Change https://golang.org/cl/303869 mentions this issue: |
… requirements This is intended to be a pure-refactoring change, with very little observable change in the behavior of the 'go' command. A few error messages have prefixes changed (by virtue of being attached to packages or modules instead of the build list overall), and 'go list -m' (without arguments) no longer loads the complete module graph in order to provide the name of the (local) main module. The previous modload.buildList variable contained a flattened build list, from which the go.mod file was reconstructed using various heuristics and metadata cobbled together from the original go.mod file, the package loader (which was occasionally constructed without actually loading packages, for the sole purpose of populating otherwise-unrelated metadata!), and the updated build list. This change replaces that variable with a new package-level variable, named "requirements". The new variable is structured to match the structure of the go.mod file: it explicitly specifies the roots of the module graph, from which the complete module graph and complete build list can be reconstructed (and cached) on demand. Similarly, the "direct" markings on the go.mod requirements are now stored alongside the requirements themselves, rather than side-channeled through the loader. The requirements are now plumbed explicitly though the modload package, with accesses to the package-level variable occurring only within top-level exported functions. The structured requirements are logically immutable, so a new copy of the requirements is constructed whenever the requirements are changed, substantially reducing implicit communication-by-sharing in the package. For #36460 Updates #40775 Change-Id: I97bb0381708f9d3e42af385b5c88a7038e1f0556 Reviewed-on: https://go-review.googlesource.com/c/go/+/293689 Trust: Bryan C. Mills <[email protected]> Run-TryBot: Bryan C. Mills <[email protected]> TryBot-Result: Go Bot <[email protected]> Reviewed-by: Michael Matloob <[email protected]>
…porting package For #36460 Updates #40775 Change-Id: I833ee8ee733151aafcff508bf91d0e6e37c50032 Reviewed-on: https://go-review.googlesource.com/c/go/+/303869 Trust: Bryan C. Mills <[email protected]> Run-TryBot: Bryan C. Mills <[email protected]> TryBot-Result: Go Bot <[email protected]> Reviewed-by: Jay Conrod <[email protected]>
|
Change https://golang.org/cl/310089 mentions this issue: |
…rror The 'go mod tidy' hint was truncated due to a typo in CL 293689, and that particular case was not covered by any existing test. Updates #36460 Updates #40775 Change-Id: Ib6fa872a9dfdafc4e9a112e8add2ff5aecd2dbd0 Reviewed-on: https://go-review.googlesource.com/c/go/+/310089 Trust: Bryan C. Mills <[email protected]> Run-TryBot: Bryan C. Mills <[email protected]> Reviewed-by: David Chase <[email protected]> TryBot-Result: Go Bot <[email protected]>
|
Change https://golang.org/cl/310181 mentions this issue: |
|
Change https://golang.org/cl/310669 mentions this issue: |
PackageOpts is a new struct type accepted by package loading functions. It initially has two fields: IgnoreImports, and ModResolveTests. Previously, these were global variables set by clients. We'll add more to this in the future. For #40775 Change-Id: I6956e56502de836d3815ce788bdf16fc5f3e5338 Reviewed-on: https://go-review.googlesource.com/c/go/+/310669 Trust: Jay Conrod <[email protected]> Run-TryBot: Jay Conrod <[email protected]> TryBot-Result: Go Bot <[email protected]> Reviewed-by: Bryan C. Mills <[email protected]>
This eliminates some awkwardly-stateful outside calls to
modload.{Disallow,Allow,}WriteGoMod.
Perhaps more importantly, it gives the loader the opportunity to
reload packages and revise dependencies after the tidied requirements
are computed. With lazy loading, dropping an irrelevant requirement
from the main module's go.mod file may (rarely) cause other test
dependencies for packages outside the main module to become
unresolved, which may require the loader to re-resolve those
dependencies, which may in turn add new roots and increase the
selected versions of modules providing other packages.
This refactoring allows the loader to iterate between tidying the
build list and reloading packages as needed, making the exact
sequencing of loading and tidying an implementation detail of the
modload package.
For #36460
For #40775
Change-Id: Ib6da3672f32153d5bd7d653d85e3672ab96cbe36
Reviewed-on: https://go-review.googlesource.com/c/go/+/310181
Trust: Bryan C. Mills <[email protected]>
Run-TryBot: Bryan C. Mills <[email protected]>
TryBot-Result: Go Bot <[email protected]>
Reviewed-by: Michael Matloob <[email protected]>
Reviewed-by: Jay Conrod <[email protected]>
|
Change https://golang.org/cl/322369 mentions this issue: |
|
Change https://golang.org/cl/332571 mentions this issue: |
|
Change https://golang.org/cl/336151 mentions this issue: |
'go mod graph', 'go mod vendor', 'go mod verify', and 'go mod why' will no longer edit go.mod or go.sum. 'go mod graph', 'go mod verify', and 'go mod why' may still fetch files and look up packages as if they were able to update go.mod. They're useful for debugging and should still work when go.mod is a little broken. This is implemented in modload.setDefaultBuildMod based on command name for now. Super gross. Sorry. This should be fixed with a larger refactoring for #40775. Fixes #45551 Change-Id: If5f225937180d32e9a5dd252c78d988042bbdedf Reviewed-on: https://go-review.googlesource.com/c/go/+/336151 Trust: Jay Conrod <[email protected]> Run-TryBot: Jay Conrod <[email protected]> TryBot-Result: Go Bot <[email protected]> Reviewed-by: Michael Matloob <[email protected]>
|
Change https://golang.org/cl/339895 mentions this issue: |
|
Change https://golang.org/cl/341933 mentions this issue: |
'go mod graph', 'go mod vendor', 'go mod verify', and 'go mod why' will no longer edit go.mod or go.sum. 'go mod graph', 'go mod verify', and 'go mod why' may still fetch files and look up packages as if they were able to update go.mod. They're useful for debugging and should still work when go.mod is a little broken. This is implemented in modload.setDefaultBuildMod based on command name for now. Super gross. Sorry. This should be fixed with a larger refactoring for #40775. Fixes #45551 Change-Id: If5f225937180d32e9a5dd252c78d988042bbdedf Reviewed-on: https://go-review.googlesource.com/c/go/+/336151 Trust: Jay Conrod <[email protected]> Run-TryBot: Jay Conrod <[email protected]> TryBot-Result: Go Bot <[email protected]> Reviewed-by: Michael Matloob <[email protected]> Reviewed-on: https://go-review.googlesource.com/c/go/+/341933 Reviewed-by: Bryan C. Mills <[email protected]>
|
Change https://golang.org/cl/345172 mentions this issue: |
|
Change https://golang.org/cl/345169 mentions this issue: |
|
Change https://golang.org/cl/345170 mentions this issue: |
|
Change https://golang.org/cl/345171 mentions this issue: |
|
Change https://golang.org/cl/345173 mentions this issue: |
|
Change https://golang.org/cl/345169 mentions this issue: |
|
Change https://golang.org/cl/347155 mentions this issue: |
|
Change https://golang.org/cl/347153 mentions this issue: |
|
Change https://golang.org/cl/347157 mentions this issue: |
|
Change https://golang.org/cl/347156 mentions this issue: |
|
Change https://golang.org/cl/347154 mentions this issue: |
|
Change https://golang.org/cl/347158 mentions this issue: |
|
Change https://golang.org/cl/347230 mentions this issue: |
|
Change https://golang.org/cl/351468 mentions this issue: |
They should also not report an error if these files need to be updated. These commands are used for debugging, so it's important that they still work when go.mod and go.sum are incomplete. For #40775 Change-Id: I1b731599e5a4510f47827b9812525636a7402bf4 Reviewed-on: https://go-review.googlesource.com/c/go/+/351468 Trust: Jay Conrod <[email protected]> Reviewed-by: Bryan C. Mills <[email protected]>
These two functions together duplicated much of the functionality of modload.Lookup. Use that instead in modcmd.vendorPkg, and reduce the modload surface area. Updates #42504 Updates #40775 For #26904 Change-Id: Ib8aaac495d090178dd56971aef9e5aa44ffa818b Reviewed-on: https://go-review.googlesource.com/c/go/+/332571 Reviewed-by: Michael Matloob <[email protected]> Run-TryBot: Bryan Mills <[email protected]> Reviewed-by: David Chase <[email protected]> TryBot-Result: Gopher Robot <[email protected]> Reviewed-by: Jay Conrod <[email protected]> Auto-Submit: Bryan Mills <[email protected]>
The
modloadpackage has a lot of global state. In general, variables that are set once during initialization and global caches are fine. ButbuildListandloaderin particular cause problems, since most exported functions either read values out of them or replace and overwrite them. It's frequently not obvious when this is happening.This leads to some unfortunate situations. For example, see get.go in CL 228383. If
modload.ListModulesis called beforemodload.WriteGoMod, thego.modfile will contain spurious// indirectcomments. This happens becauseWriteGoModusesloaded.directto decide whether to mark a requirement as indirect, butListModulescallsLoadBuildList, which replacesloaded, so it appears that nothing is directly required.I think these side effects should be more obvious to callers, and the
loadedandbuildListvariables should be removed. Instead, theloaderstruct type should be exported, and exported functions that create, read, or write it should be converted to methods.The text was updated successfully, but these errors were encountered: