status: add project status #1367

darkowlzz · 2017-11-10T17:58:36Z

What does this do / why do we need it?

Adds status for project arguments.

dep status github.com/foo/bar

What should your reviewer look out for in this PR?

The printed output formatting and logic of fetching all the project details.

Do you need help or clarification on anything?

~~Need to figure out ways to fetch some attributes like SOURCE TYPE, UPSTREAM EXISTS, UPSTREAM VERSION EXISTS using gps.~~

sdboyer · 2017-11-14T00:56:03Z

internal/gps/source_manager.go

@@ -99,6 +99,12 @@ type SourceManager interface {
 	// repository root.
 	GetManifestAndLock(ProjectIdentifier, Version, ProjectAnalyzer) (Manifest, Lock, error)

+	// GetVcsType returns VCS Type for the provided ProjectIdentifier.
+	GetVcsType(ProjectIdentifier) (string, error)


s/Vcs/Source/

also, let's create the following to capture this, rather than using strings:

type SourceType uint8 const ( InvalidSource = iota // zero value is invalid; there is no "default" type VcsGit VcsHg VcsBzr VcsSvn ) func (st SourceType) String() string { switch st { case VcsGit: return "git" case VcsHg: return "hg" case VcsBzr: return "bzr" case VcsSvn: return "svn" default: return fmt.Sprintf("%v does not represent a known source type, this probably indicates a bug", st) } }

we'll then also want to update the source.sourceType() method to return this uint instead of a string.

sdboyer · 2017-11-14T01:07:10Z

internal/gps/source.go

@@ -388,6 +388,10 @@ func (sg *sourceGateway) getManifestAndLock(ctx context.Context, pr ProjectRoot,
 	return m, l, nil
 }

+func (sg *sourceGateway) getVcsType() string {


s/Vcs/Source/

also should return the SourceType type described in the other comment.

sdboyer · 2017-11-14T01:20:19Z

cmd/dep/status.go

+	SourceType            string
+	Packages              []string
+	ProjectImporters      map[string]string
+	PackageImporters      map[string]string


We've got project & package importers here, but not project and package imports?

ProjectImports will be able to be a []string, but PackageImports will need to be a map[string][]string. You'll want to use pkgtree.ListPackages(), TrimHiddenPackages() and then ToReachMap() to assemble the PackageImports information.

i'd also be fine with leaving this to a follow-up PR - it's more important that we get something basically functional then ship additional bits of data later on.

I was working on adding ProjectImports, which I realized could be obtained from collectConstraints(), so I made the necessary changes in #962 . But yeah, we can add it as a follow-up PR.

sdboyer · 2017-11-14T03:04:37Z

cmd/dep/status.go

+	Revision              string
+	LatestAllowed         string
+	SourceType            string
+	Packages              []string


The comment about ProjectImports and PackageImports made me realize that it would also be worth listing the ignores that are currently affecting this project - both what packages within it are ignored (that would otherwise actually be imported), and which of its imports are ignored.

This has the potential to be a little tricky, as if there's a hash mismatch (or if the user manually messed with Gopkg.lock), then it's possible that there may be some packages listed in Gopkg.lock that actually should be ignored. There's also a fair bit of work involved in actually pulling out the ignoreds, so i think it'll be fine for us to defer this to a later PR.

i've updated the spec with new lines and some comments explaining this, so we can discuss/dissect there; just noting it here.

sdboyer · 2017-11-14T03:15:03Z

cmd/dep/status.go

+
+		// Get the currently selected version from from lock.
+		for _, pl := range p.Lock.Projects() {
+			if pr == pl.Ident().ProjectRoot {


best practice here would be to minimize the indent level:

if pr != pl.Ident().ProjectRoot { continue }

then we can deindent the rest of the block, and the reader can immediately recognize that we're effectively just searching the slice for that single item.

sdboyer · 2017-11-14T03:16:09Z

cmd/dep/status.go

+			}
+		}
+
+		fmt.Println(projStatus)


print through the ctx.Out logger

sdboyer · 2017-11-14T03:19:44Z

cmd/dep/status.go

+
+				// Update local copy of the source and then fetch all the versions.
+				sm.SyncSourceFor(pl.Ident())
+				pvs, err := sm.ListVersions(pl.Ident())


in the current implementation, ListVersions() is already guaranteed to always return the latest set of versions, so the SyncSourceFor() call is unnecessary.

that may change when the caching work goes in, but we don't need to code defensively against that eventuality - we'll cross that bridge when we come to it.

sdboyer · 2017-11-14T03:21:14Z

cmd/dep/status.go

+
+				rev, _, _ := gps.VersionComponentStrings(pl.Version())
+				// REVISION
+				projStatus.Revision = rev


no need for the intermediate rev var, just assign directly to projStatus.Revision

darkowlzz · 2017-11-16T19:04:29Z

Almost there. Figured out how to gather PACKAGE IMPORTS & PROJECT IMPORTS (hopefully correct 😅). LATEST ALLOWED is the only thing left.
Once, we have all the data, I'll focus on the presentation.

darkowlzz · 2017-11-17T15:27:12Z

Made some optimizations like advanced collection of package reachmaps and package list, and single gps solve for all the projects' updates, to bring down the running time. Now, status of a single project or multiple projects take almost the same time (tried with 1 to 4 projects on dep itself, ~36s). It can be made concurrent to bring that down. Follow-up PR.

Sample output:

$ dep status github.com/Masterminds/vcs

PROJECT:			github.com/Masterminds/semver
VERSION:			parse-constraints-with-dash-in-pre
CONSTRAINTS:		[]
SOURCE:			github.com/Masterminds/semver
ALT SOURCE:		https://github.com/carolynvs/semver.git
PUB VERSION:		map[semver:[1.0.0 1.0.1 1.1.0 v1.1.1 v1.2.0 v1.2.1 v1.2.2 v1.2.3 v1.3.0 v1.3.1] branches:[2.x has-implied-caret master parse-constraints-with-dash-in-pre] nonsemvers:[]]
REVISION:		a93e51b5a57ef416dac8bb02d11407b6f55d8929
LATEST ALLOWED:		a93e51b5a57ef416dac8bb02d11407b6f55d8929
SOURCE TYPE:		git
PACKAGES:		github.com/Masterminds/semver
PROJECT IMPORTERS:	map[]
PACKAGE IMPORTERS:	map[]
UPSTREAM EXISTS:		true
UPSTREAM VERSION EXISTS:	true

PROJECT IMPORTERS and PACKAGE IMPORTERS are empty in this case because we don't have any dependency that uses semver project. An example for this for github.com/dgryski/go-metro would be:

PROJECT IMPORTERS:	map[github.com/jmank88/nuts:true]
PACKAGE IMPORTERS:	map[github.com/dgryski/go-metro:[github.com/jmank88/nuts/cmd/testpaths]]

CONSTRAINTS is empty for now because it depends on constraintsCollection(), which is implemented in #962 .

NEXT: Presentation improvements.

darkowlzz · 2017-11-18T20:17:59Z

Improved the output using tabwriter and added an integration test.

I'll add the json output in a follow-up PR, avoiding huge change-set.

Example output:

$ dep status github.com/sdboyer/deptest

PROJECT:                  github.com/sdboyer/deptest
VERSION:                  v1.0.0
CONSTRAINTS:              []
SOURCE:                   github.com/sdboyer/deptest
ALT SOURCE:
PUB VERSION:              branches: master
                          nonsemvers:
                          semvers: v0.8.0, v0.8.1, v1.0.0
REVISION:                 ff2948a2ac8f538c4ecd55962e919d1e13e74baf
LATEST ALLOWED:           ff2948a2ac8f538c4ecd55962e919d1e13e74baf
SOURCE TYPE:              git
PACKAGES:                 github.com/sdboyer/deptest
PROJECT IMPORTERS:        github.com/darkowlzz/deptest-project-status, github.com/sdboyer/deptestdos
PACKAGE IMPORTERS:        github.com/sdboyer/deptest
                            github.com/darkowlzz/deptest-project-status/pkgbar
                            github.com/darkowlzz/deptest-project-status/pkgfoo
                            github.com/sdboyer/deptestdos
UPSTREAM EXISTS:          yes
UPSTREAM VERSION EXISTS:  yes

Leaving CONSTRAINTS as it is for now since its String() output would depend on the data structure, which would be changed in #962 .

darkowlzz · 2017-11-29T11:07:05Z

Rebased and updated project status to print project constraints in proper table output.

CONSTRAINTS:              ^0.5.0(btb.com/x/y)
                          ^1.0.0(gh.com/f/b)

darkowlzz · 2017-11-29T14:30:58Z

cmd/dep/status.go

+	for _, pc := range pcs {
+		constList = append(constList, pc.String())
+	}
+	sort.Strings(constList)


This sorting would be simplified and improved with projectConstraint sort by Project implemented in #1413 .

darkowlzz · 2017-12-24T17:54:34Z

Rebased. The end result for CONSTRAINTS field would be improved by #1452, which would collect root project constraints too and #1409, which would collect applicable constraints only.

UPSTREAM VERSION EXISTS, PACKAGES, PUB VERSIONS, CONSTRAINTS

Perform solve at the end for all the target projects together and get the latest allowed revision for them.

mvdan · 2020-09-04T15:09:39Z

Dep was officially deprecated earlier this year, and the proposal to archive this repository was accepted. As such, I'm closing outstanding issues before archiving the repository. For any further comments, please use the proposal thread on the Go issue tracker. Thanks!

googlebot added the cla: yes label Nov 10, 2017

darkowlzz force-pushed the status-project branch from 63a5275 to ec0cb8a Compare November 11, 2017 14:13

darkowlzz requested a review from sdboyer as a code owner November 11, 2017 14:13

darkowlzz force-pushed the status-project branch from 2b6a198 to fd36ee2 Compare November 12, 2017 11:35

darkowlzz mentioned this pull request Nov 12, 2017

fix(status): add constraint for locked projects #962

Merged

sdboyer suggested changes Nov 14, 2017

View reviewed changes

darkowlzz force-pushed the status-project branch 4 times, most recently from 07a02c2 to 6157038 Compare November 16, 2017 19:00

darkowlzz force-pushed the status-project branch 2 times, most recently from e0098dc to 4e31fe6 Compare November 17, 2017 14:58

darkowlzz force-pushed the status-project branch 2 times, most recently from ba7a019 to a6ca4a8 Compare November 18, 2017 20:03

darkowlzz changed the title ~~[WIP] status: add project status~~ status: add project status Nov 18, 2017

darkowlzz added the area: status label Nov 18, 2017

darkowlzz force-pushed the status-project branch from de981c2 to ae10ff2 Compare November 20, 2017 13:45

sdboyer added this to the v0.4.0 milestone Nov 20, 2017

darkowlzz force-pushed the status-project branch from f760ff8 to 7de341c Compare November 29, 2017 08:59

darkowlzz commented Nov 29, 2017

View reviewed changes

sdboyer removed this from the v0.4.0 milestone Nov 29, 2017

darkowlzz force-pushed the status-project branch from fea4256 to 8d5d393 Compare December 6, 2017 16:58

darkowlzz force-pushed the status-project branch from 8d5d393 to 99aeaae Compare December 24, 2017 17:48

darkowlzz added 2 commits December 26, 2017 20:10

status: add project status

bbdfc49

gps: Add getSourceType in SourceManager

10a6372

darkowlzz added 12 commits December 26, 2017 20:10

gps: Add ExistsUpstream in SourceManager

19a451c

status: Add more attributes in project status

7ca608f

UPSTREAM VERSION EXISTS, PACKAGES, PUB VERSIONS, CONSTRAINTS

runProjectStatus: Add PACKAGE IMPORTS & PROJECT IMPORTS

644f0c2

runProjectStatus: Add LATEST ALLOWED

1b03d9f

runProjectStatus: perform single gps solve

d12d486

Perform solve at the end for all the target projects together and get the latest allowed revision for them.

runProjectStatus: Optimizations to avoid redundant computation

b73e137

projectStatus: Prettify output

a9ca322

test: Integration test for status project tableoutput

db77a19

test: Add TestProjectStatusString

436cab0

status: Exclude pkgs not in depgraph from project status

ec23702

status: project status error out when project is not in lock

05516b8

status: make project status compatible with collectConstraints

4b8290e

darkowlzz force-pushed the status-project branch from 99aeaae to 4b8290e Compare December 26, 2017 14:43

This was referenced Jan 19, 2018

[Proposal] Project Relationship Graphs #1547

Closed

status: remove -detailed flag from help #1550

Merged

sdboyer self-assigned this Feb 26, 2018

darkowlzz mentioned this pull request Apr 1, 2018

Support additional details with status command #1781

Merged

mvdan closed this Sep 4, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

status: add project status #1367

status: add project status #1367

darkowlzz commented Nov 10, 2017 •

edited

Loading

sdboyer Nov 14, 2017

sdboyer Nov 14, 2017

sdboyer Nov 14, 2017

sdboyer Nov 14, 2017

darkowlzz Nov 14, 2017

sdboyer Nov 14, 2017

sdboyer Nov 14, 2017

sdboyer Nov 14, 2017

sdboyer Nov 14, 2017

sdboyer Nov 14, 2017

darkowlzz commented Nov 16, 2017 •

edited

Loading

darkowlzz commented Nov 17, 2017 •

edited

Loading

darkowlzz commented Nov 18, 2017

darkowlzz commented Nov 29, 2017

darkowlzz Nov 29, 2017

darkowlzz commented Dec 24, 2017 •

edited

Loading

mvdan commented Sep 4, 2020

status: add project status #1367

status: add project status #1367

Conversation

darkowlzz commented Nov 10, 2017 • edited Loading

What does this do / why do we need it?

What should your reviewer look out for in this PR?

Do you need help or clarification on anything?

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

darkowlzz commented Nov 16, 2017 • edited Loading

darkowlzz commented Nov 17, 2017 • edited Loading

darkowlzz commented Nov 18, 2017

darkowlzz commented Nov 29, 2017

Choose a reason for hiding this comment

darkowlzz commented Dec 24, 2017 • edited Loading

mvdan commented Sep 4, 2020

darkowlzz commented Nov 10, 2017 •

edited

Loading

darkowlzz commented Nov 16, 2017 •

edited

Loading

darkowlzz commented Nov 17, 2017 •

edited

Loading

darkowlzz commented Dec 24, 2017 •

edited

Loading