proxy.golang.org: module with OSL 3.0 license not cached

[bradfitz]

I've encountered many module versions that are in index.golang.org but serve errors from proxy.golang.org.

Is this expected? If so, why?

One example:

https://index.golang.org/index?since=2019-08-30T17:04:40.24986Z&limit=1

{"Path":"flamingo.me/flamingo-commerce-adapter-standalone","Version":"v0.0.1-beta","Timestamp":"2019-08-30T17:04:40.24986Z"}

$ curl https://proxy.golang.org/flamingo.me/flamingo-commerce-adapter-standalone/@v/list
v0.0.1-beta

$ curl -v https://proxy.golang.org/flamingo.me/flamingo-commerce-adapter-standalone/@v/v0.0.1-beta.zip
...
< HTTP/2 410 
< content-type: text/plain; charset=UTF-8
< date: Mon, 30 Sep 2019 02:03:56 GMT
< expires: Mon, 30 Sep 2019 02:33:56 GMT
< x-content-type-options: nosniff
< x-frame-options: SAMEORIGIN
< x-xss-protection: 0
< content-length: 300
< cache-control: public, max-age=1800
< age: 1637
< 
* Curl_http_done: called premature == 0
* Connection #0 to host proxy.golang.org left intact
not found: unrecognized import path "flamingo.me/flamingo-commerce-adapter-standalone" (parse https://flamingo.me/flamingo-commerce-adapter-standalone?go-get=1: no go-import meta tags (meta tag flamingo.me/flamingo-commerce did not match import path flamingo.me/flamingo-commerce-adapter-standalone))

Notably this error message from above:

not found: unrecognized import path "flamingo.me/flamingo-commerce-adapter-standalone" (parse https://flamingo.me/flamingo-commerce-adapter-standalone?go-get=1: no go-import meta tags (meta tag flamingo.me/flamingo-commerce did not match import path flamingo.me/flamingo-commerce-adapter-standalone))

[bradfitz]

/cc @hyangah

[aofei]

About @v/list printed unexpected content, I believe it's related to #34033.

And, it seems that index.golang.org only parses and caches the module's .info file at first, and only downloads and caches other data when someone requests its .zip file. (I think...)

/cc @katiehockman

[hyangah]

@bradfitz The module mirror successfully cached all .info, .zip, .mod a month ago but failed to detect a suitable license of the mentioned module. For modules with unrecognized licenses, the module mirror has to check against the origin of the module after the cached copy expires. The origin seems to be unreachable with the go import protocol since mid-September.

We are internally investigating why the license of the mentioned module version wasn't detected.

It's still related to #34033 as @aofei mentioned - @katiehockman is working on excluding versions from the list query output if the origin is not available and the mirror doesn't have cached copy.

Currently we do not have plan to remove this version from index.golang.org/index - this is the history of module versions observed (all .info, .mod, .zip) by the module mirror, regardless whether they are currently available or not.

[bcmills]

Since #34033 covers the @v/list behavior, retitling this issue to focus on the license detection.

[hyangah]

We are not able to add OSL-3.0 to the list of licenses that allow for permanent caching at this time.