Dep ensure results in exit status 128 for private repo

[nbosscher]

What version of dep are you using (dep version)?

dep:
 version     : v0.3.2
 build date  : 2017-10-19
 git hash    : 8ddfc8a
 go version  : go1.9
 go compiler : gc
 platform    : linux/amd64

What dep command did you run?

GOPATH="project-path" dep ensure

What did you expect to see?

Downloads and installs my dependancies in vendor

What did you see instead?

ensure Solve(): No versions of github.com/laddersoftware/http met constraints:
	master: unable to update checked out version: : command failed: [git checkout fb426ba7e2843046c0fa67beaf19890c3b31f62f]: exit status 128
	master: unable to update checked out version: : command failed: [git checkout fb426ba7e2843046c0fa67beaf19890c3b31f62f]: exit status 128

Extra

• I'm using git credential cache to temporarily store my credentials. The timeout is sufficiently long
• I can git clone github.com/laddersoftware/http with no issues.
• Attempting the same action on another machine with the same Gopkg.toml works fine.
• It's sporadically failing on different repositories in the project with the same error. Some sort of race I assume

[sdboyer]

hi, welcome! sorry you're running into this.

unfortunately, with the outstanding bug we have where we're missing the actual stderr/stdout git output, it's hard to know exactly what the problem is. however, i'm gonna guess that fb426ba7e2843046c0fa67beaf19890c3b31f62f isn't a revision that's actually present in the private repository - perhaps its only on your local GOPATH?

also, just a note that GOPATH="project-path" dep ensure may get you into some painful waters.

[nbosscher]

Hi, thanks for the response!

• fb426ba7e2843046c0fa67beaf19890c3b31f62f does actually exist on the repo (the remote repo).
• Is there anything I can do to get you better debug info?

also, just a note that GOPATH="project-path" dep ensure may get you into some painful waters.

Could you expand on what you mean by that or point me to someplace that explains that? I have several golang projects on the go, so setting a global GOPATH is a bit of a nuisance.

[sdboyer]

Is there anything I can do to get you better debug info?

i realized that our "bug" around this is actually pretty simple - modify https://github.com/golang/dep/blob/master/gps/cmd_unix.go#L81 to return b.Bytes(), err instead of return nil, err, compile dep with that change, and it should give you more context on what the actual git failure is.

Could you expand on what you mean by that

sure. a few things:

1. $GOPATH is not the root of where source code goes; $GOPATH/src is.
2. further, dep cannot operate at the $GOPATH/src root. there are several issues open on this, but Support vendor directory as $GOPATH/src/vendor #313 has my most detailed mechanical explanation at the end of it.
3. until we get allow cachedir override using env var #1234 in, dep stores local caches of upstream repos at $GOPATH/pkg/dep/sources. using different GOPATHs means dep will end up duplicating all those repos, which can make things quite slow.

GOPATH's requirements are a well-known pain point in the community, and they're something that we can make improvements on once dep makes its way into the toolchain. until we do, though, you'll continue to find ways, in addition to the above, where you're going against the grain by using GOPATH like this.

[nbosscher]

DEBUG

Made the change you suggested to https://github.com/golang/dep/blob/master/gps/cmd_unix.go#L81 and I get the following output.

Solving failure: No versions of github.com/laddersoftware/http met constraints:
	master: unable to update checked out version: fatal: reference is not a tree: fb426ba7e2843046c0fa67beaf19890c3b31f62f
: command failed: [git checkout fb426ba7e2843046c0fa67beaf19890c3b31f62f]: exit status 128
	master: unable to update checked out version: fatal: reference is not a tree: fb426ba7e2843046c0fa67beaf19890c3b31f62f
: command failed: [git checkout fb426ba7e2843046c0fa67beaf19890c3b31f62f]: exit status 128

Just to double check that I can checkout fb426ba7e2843046c0fa67beaf19890c3b31f62f with git normally:

$ git clone https://github.com/laddersoftware/http.git
Cloning into 'http'...
remote: Counting objects: 54, done.
remote: Compressing objects: 100% (23/23), done.
remote: Total 54 (delta 16), reused 53 (delta 15), pack-reused 0
$ cd http
$ git checkout fb426ba7e2843046c0fa67beaf19890c3b31f62f
Note: checking out 'fb426ba7e2843046c0fa67beaf19890c3b31f62f'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b new_branch_name

HEAD is now at fb426ba... <removed-comment>

GOPATH

Re GOPATH, I guess I wasn't very clear with my description, sorry about that. This is what I was doing. As far as I know that should work for all tools that require GOPATH, no?

$ pwd
/home/centos/repos/API
$ cd src/api
$ GOPATH="/home/centos/repos/API" dep ensure

[sdboyer]

re: GOPATH, yeah, it looks like that ought to work.

it looks like dep's failing to update its local cache repo. we haven't seen much of this recently, though, so it's surprising. i suspect it's some kind of error that we're not logging adequately (see #1375), possibly related to the credential cache.

have a look at GOPATH/dep/pkg/sources for a path (it's usually https---, but if you're using ssh, it won't be) that looks like it corresponds to this repo. cd into there - is the working tree dirty? does git show fb426ba7e2843046c0fa67beaf19890c3b31f62f error out? if you git fetch, does that rev get pulled in?

[nbosscher]

Ya, I noticed there isn't much logging with dep, would be nice to improve that for sure.

Took a look at GOPATH/dep/pkg/sources/https---github.com-laddersoftware-http

• git status was clean
• git fetch pulled in the missing rev (fb426ba7e2843046c0fa67beaf19890c3b31f62f)

Guess dep wasn't calling fetch for some reason...

[sdboyer]

ah, you were running v0.3.2 - that's slightly less worrisome. we've put in a few fixes since then related to making sure that these fetches actually happen. we're planning on a new release "soon", but i'm gonna guess that if you work from tip in the meantime, you won't have these issues anymore.

(closing in expectation of that, though please reopen if you do still experience this on tip)

[nbosscher]

Ok, thanks

[sudhirj]

@sdboyer This happens both on tip and on v0.3.2 for the AWS Go SDK (github.com/aws/aws-sdk-go). Looking at ~/go/pkg/dep/sources/https---github.com-aws-aws--sdk--go, I can see one of the files has been modified for some reason (whitespace changes only), and it looks like dep is the tool doing the modification. (I confirmed this by rm -rfing the source directory a few times). This results in the following error:

init failed: unable to write the manifest, lock and vendor directory to disk: error while writing out vendor tree: failed to write dep tree: failed to export github.com/aws/aws-sdk-go: : exit status 128

making it impossible to proceed further on any project that uses this package.

[sudhirj]

This seems to a problem with the AWS SDK, though, think there's a hook that's modifying files on checkout. Would be great if dep just reported getting stuck on a dirty git branch, but it doesn't do so at tip right now.

[sdboyer]

@sudhirj this is almost certainly a different problem than what this ticket is about, even though the git error message is the same - your issue appears to be dirty state, this issue is about missing revs from upstream. please open a new issue and we can discuss over there.

[sudhirj]

Will do.

[ashishkhatri]

Just wanted to comment that doing a manual git fetch before dep ensure fixed this problem for me as well.