repositories - Azure DevOps NuGet v2 endpoints don't provide metadata in Packages() queries - fails on "Authors is required." #1771
Comments
|
We're also perceiving the same issue in our deployments, it happens on our Azure DevOps deployed packages. |
|
We are also seeing this issue in our environment, which is also using Azure DevOps feeds. For some reason, the affected packages' nuspec files are empty (0 bytes) in lib\package_name when installing using v0.10.13, but are fine when using v0.10.11. |
|
@israelvaldez be careful with 0.10.12 due to the issue that caused an immediate bump to 0.10.13. Stick with 0.10.11 |
|
@AlexandreRatte do you have a log you can provide? This is very strange behavior. |
|
@bc3tech this is very strange, looking it over, knowing that it does have an authors bit in the nuspec - wondering if there was something that was changed for NuGet.Core between 0.10.11 and 0.10.12/0.10.13 (we made some slight changes to allow bad repos to be ignored, but that's the only known changes there). There was another change in logic on how we find the updated packages. Is anyone seeing this with install? Or is it only on upgrades? |
|
@ferventcoder I'm seeing the issue with both install and upgrades, the issue is that once the nuspec is downloaded the file gets saved with 0 bytes in the lib\package folder. |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
@AlexandreRatte We've seen this in the past when antivirus scanners or other utilities grabbing onto the files. It is a bit peculiar that everyone runs into this issue with the newer version of Chocolatey so that would be reaching at best. |
This comment has been minimized.
This comment has been minimized.
|
@ferventcoder I can swap between v0.10.11 that installs successfully and v0.10.13 that fails consistently. We're running Windows Server 2012 R2 |
|
@AlexandreRatte thanks for that. That widens this out a bit. Mostly sure it is something with 0.10.12/13 changes. Can you ensure cacheLocation ( |
|
Apologies, we are in the process of setting up something to attempt to reproduce this error. Until then we may need to ask folks experiencing this to help us troubleshoot this a bit more. |
|
I'm happy to assist. cacheLocation appears to be blank: |
|
I'm getting the issue in VMs with Windows Defender, with the default settings from Azure. |
|
Can you set cacheLocation to something like |
|
No effect.
Then ran: |
|
for me the failing OS is Win10 Enterprise 19h1 (insider); Defender enabled |
|
Okay, we did some research here. I always want to have my ducks in a row before we determine an issue is coming from something on the distant end. Caused by changes related to #1397. tl;drSo it appears the Azure DevOps repos (VSTS NuGet v2 endpoints) do not fill out all elements of the V2FeedPackage schema when you call the Unfortunately it is a bug in those Azure DevOps NuGet v2 endpoints and we'll either need to work around it and/or the Azure DevOps teams will need to get those fixed. We are following up with that team but don't have an ETA from their side at this time. WorkaroundIf you run into this problem, you will simply need to go back to 0.10.11 for now. You can do that with DetailsBack StoryFor Chocolatey v0.10.12+, we have converted to a much more efficient way of finding packages that allows us to reduce bandwidth and really speed things up. To do so, it means a change from The prior method Outdated Measurements
That's over 100% faster if I can math correctly. MetaData Url
These are the fields available with a v2 Repo and the methods you can call on them. This is purely informational at this stage. Also note the fields available with a V2FeedPackage. Old Method - FindPackagesById()So the old method we used to look for packages and install them is FindPackagesById() - it's inefficient. If you have 30 versions of a package, it will return all 30 in the response. This works fine as you can see here: Note the V2FeedPackage elements are all filled in with data. This is why the old method doesn't error (using Chocolatey v0.10.11 and below). New Method - Packages() with a LatestVersion filterIf we are working with a DataServicePackage Repository, then we should trust that the V2FeedPackage should have all of the elements filled out. Unfortunately that is not the case when we hit the Azure repository. This unfortunately returns a lot of things null as you can see here: This is where the errors occur because NuGet uses this information to write the package manifest (the nuspec file). Since all of those elements are null, it errors on the first victim "Authors" and we get this error message. Next StepsWe are working through our contacts to get in touch with the Azure DevOps teams that works in this area so we can get this hopefully resolved quickly. If need be, we'll need to make a special exception to handle their repository and issue a newer version of Chocolatey. What that means is that it will continue to have inefficient output, so our hope is we can get this resolved on the Azure side. |
|
Hi @alexmullans! Thanks for that heads up. Q2 is quite awhile out. It appears we are going to need to workaround it for now, because basically anyone wanting to migrate to the newer versions of Chocolatey is dead in the water if they want to use Azure DevOps repositories. Is there something uniquely identifiable about the Url that tells us we are looking at an azure devops repo? |
|
Uniquely identifiable and deterministic? |
|
Yep, sorry for the long timeframe. There's some structural issues in that API that we need to address. Yep! All URLs will look like either:
|
|
@alexmullans actually it appears we've found a simpler way to work around this issue by downloading the package first, then extracting the nuspec out of that. |
|
This has been fixed for 0.10.14. There will be a beta available soon for folks to test this out. |
ferventcoder
changed the title
Packages() url - Can't update to 0.10.13: "Authors is required."
ferventcoder
changed the title
Packages() url - Can't update to 0.10.13: "Authors is required."
ferventcoder
changed the title
ferventcoder
changed the title
Previously, when Chocolatey was downloading a nupkg, it would rely on the metadata information from an OData feed to be returned properly. Unfortunately, not all packaging repositories that support OData repositories return this data in a proper manner. Due to this aspect, download the nupkg first and extract the nuspec from that file instead or using the metedata returned from the `DataServicePackage` (which is simply an XML stream from a URL).
|
@ferventcoder sounds good, glad you got it working. |
|
@bc3tech @israelvaldez @AlexandreRatte a beta has been pushed out to Chocolatey.org for this. It should be available in about an hour or two. If you can't wait that long, you can always grab it from https://chocolatey.box.com/shared/static/z1fr740vag1bbwy4wfby0s8t0s0w2vad.nupkg |
|
np man i'm cool waiting. (that's my way of saying i'll wait for it to hit prod before i try it out ;) ) |
|
@bc3tech we see it working, just want to verify it works for you as well. You can always go back to 0.10.11 afterwards. |
|
Thanks @pauby for getting something set up so we could get a quick turnaround on this! |
|
@ferventcoder it works*!, thanks for the help. `PS>choco upgrade chocolatey --pre --force You have chocolatey v0.10.13 installed. Version 0.10.14-beta-20190321 is available based on your source(s). Chocolatey upgraded 0/1 packages. 1 packages failed. Failures
|
|
Yes, that is a known issue #717 (it applies to nuspecs as well I'm seeing). Just manually remove them. |
|
Downgrading Chocolatey to v0.10.11, and then installing Chocolatey v0.10.14-beta-7-gb30427d worked for me |
|
Any news on that to support v3 and not only v2 ? |
|
@DiscoveringWorld rather than posting the same question in multiple places, let's keep the conversation in once place. The correct place to continue discussing this is in this issue: #508 |
What You Are Seeing?
What is Expected?
I'd expect choco 0.10.12 to update to 0.10.13 w/o issue
How Did You Get This To Happen? (Steps to Reproduce)
See log output above
Output Log
Full Log Output
NOTE: Added by admin
Caused by changes in #1397. Related to #1770.
Release Notes
This is fixed in v0.10.14+ - hopefully there is a beta available to try out.
The text was updated successfully, but these errors were encountered: