-
Notifications
You must be signed in to change notification settings - Fork 17.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
cmd/go: get fails on gitlab subgroups due to go-import meta tags referring to nonexistent repos #34094
Comments
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Go 1.13 tries all module prefixes in parallel, and if another error other than "not-found" happens, the query fails. Of course the root cause is on gitlab side, but I think it's reasonable to ignore all errors if at least one attempt succeeds. |
EDIT: As a reminder, please note our https://github.com/golang/go/wiki/noplusone policy. Comments like "Me too" do not add anything new to the discussion. |
I could see doing that if the provided Git URL itself returned a 404 or 410 status code, but note that
|
I'm running into issues with GitLab subgroups as well, but I'm getting a different error:
It almost seems like it's assuming that the second part of the path must be the project. Going back to 1.12 fixes it. Edit: Nevermind, re-ran it with |
@bcmills I mean if a try on |
@leonshaw, maybe..? The problem with that is that it's asymmetric. If I have credentials for module The solution to that is to treat responses other than 404/410 as hard errors. Then I'll fail to fetch (More generally: if servers want to avoid leaking information to unauthenticated users, then it's more robust to deny the existence of every module rather than claim the existence of every module — especially now that #29888 is fixed.) |
I don't think this is possible, at least with giltab, as they don't have "directory-level permissions" but repository-level only. Probably the same for gihub. Even in case, if a user has credentials to every repo on the server,
|
Is this something more fundamental? (A module can't be uniquely located by an import path.) |
GitLab and GitHub are not the only ways to host a module. Ideally, the semantics of the |
A module has a unique path. The mapping of package paths to modules is not guaranteed to be 1:1, although the |
Yes, this is the problem. We need a way to avoid the collision, if possible. If it's simply treated as error, |
Probably, self-hosted giltab used by a majority of self-hosted git systems. However, this new behavior seems to conflict not with giltab only, but gitea as well and may conflict with any system surprised by parent discovery calls. I appreciate the generalized approach and concerns about asymmetrical behavior in some theoretical cases, but the current problem if very practical one. |
I have most of a solution coded up, but there is a bad interaction with proxy fallback that I still need to work out. Even then, my expectation is that a server that wishes to indicate “there is no repository here” (for a repository explicitly mentioned in a |
Change https://golang.org/cl/194560 mentions this issue: |
Change https://golang.org/cl/194561 mentions this issue: |
…Repo).Versions Updates #34094 Change-Id: Ifd10b51c2b4ebe77c4f8f68726e411f54c13b9c9 Reviewed-on: https://go-review.googlesource.com/c/go/+/194560 Run-TryBot: Bryan C. Mills <[email protected]> TryBot-Result: Gobot Gobot <[email protected]> Reviewed-by: Jay Conrod <[email protected]>
@gopherbot, please backport to 1.13: this is a regression, and prevents previously-working modules from being fetched. |
Backport issue(s) opened: #34215 (for 1.13). Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://golang.org/wiki/MinorReleases. |
…“not found” If a go-import directive refers to a nonexistent repository, today we treat that as an error fetching a module that actually exists. That makes the HTTP server responsible for determining which repositories do or do not exist, which may in general depend on the user's separately-stored credentials, and imposes significant complexity on such a server, which can otherwise be very simple. Instead, check the repository URL and/or error message to try to determine whether the repository exists at all. If the repo does not exist, treat its absence as a “not found” error — as if the server had not returned it in the first place. Updates #34094 Change-Id: I142619ff43b96d0de428cdd0b01cca828c9ba234 Reviewed-on: https://go-review.googlesource.com/c/go/+/194561 Reviewed-by: Jay Conrod <[email protected]>
I've implemented the approach described in #34094 (comment). That should solve the problem if GitLab is correctly serving 404 or 410 responses for nonexistent paths for signed-in users. However, I don't know whether they're doing that, so please help me confirm. @umputun (or one of the other folks with this problem): please try a |
Does this mean that this will get fixed for Go1.14, but will still not work for 1.13? |
@xavivars, this has already been fixed at head and backported to the 1.13 branch. It will be included in Go 1.13.3. |
I've just tried with Go 1.13.3 but I still failed in
P.S. My repository is a private repository. I don't know if it is a matter or not. |
@yookoala, you probably need to configure a |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
go version 1.13.3 fixed the problem. I have tried it against public (gitlab hosted) repo as well as self-hosted giltab. In both cases worked as expected. thx @bcmills and others helped to diagnose and address the problem. should I close the ticket? This was my first ticket here and I'm not sure what the rule. |
@umputun, thanks for confirming! I'm planning to leave the issue open for adding a more realistic regression test in 1.14 (probably during the freeze). |
My go version:
simple description:
So, have you tested private group or subgroup on gitlab? Thanks anyway! |
No, I didn't but tried it now with similar setup -
Not sure if important, but to access private repos from go I use insteadOf rewrite:
|
@umputun, thanks for the additional detail. This line in particular:
seems to indicate a bug or misconfiguration on the server side. If the repo is hosted at Did that repository work with Go 1.12? |
doesn't work with 1.12 either go version go1.12.12 linux/amd64, from
However with ~/.netrc everyting seems to work (1.13.3):
So, looks like .gitconfig insteadOf doesn't play well with go mods, but .netrc is fine. |
|
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
I didn't get to the more realistic regression test during the 1.14 cycle, and by now I don't remember which cases were remaining to be tested. 😞 I'm happy to review further improvements to GitLab integration testing, but I don't think we need to leave this issue open. |
GitLab Community Edition 12.3.5 self-hosted |
@Yi-Hunter, “this issue” as reported by its original author has been verified as fixed. If you're seeing a similar symptom, please open a new issue with details and steps to reproduce it. |
What version of Go are you using (
go version
)?Does this issue reproduce with the latest release?
yes
What operating system and processor architecture are you using (
go env
)?go env
OutputWhat did you do?
go get -v gitlab.com/umputuntests/sub/example
What did you expect to see?
with prev versions of go (1.12.x) it works file:
additional notes
I think this is both gitlab problem and go 1.13 change. Gitlab seems to return
go-import
tag on the parent, which doesn't exist and this seems to confuse "go get", trying to check module's parent.In fact, gitlab returns good-looking
go-import
on any random path, for example:The text was updated successfully, but these errors were encountered: