Upgrading from Chocolatey CLI v2.0.0 to v2.1.0 take a long time to complete when certain other packages are installed #3237
Comments
Can you provide the full set of steps to allow us to reproduce this? I upgraded to 2.1.0 from 2.0.0 yesterday and it worked, using that command. |
|
I just ran "choco upgrade chocolatey --verbose". I guess it's some kind of bug that occurs when you have chromium installed. So the steps would be:
|
MajorGlory86
changed the title
|
@MajorGlory86 thanks for the tip but it didn't work for me. What did work was running: Thanks to everyone else for helping get this sorted. It's been more than a tad frustrating for a while now to hit the brick wall. |
|
I had this issue upgrading from 2.0.0 to the 2.1.0 alpha and from the alpha to the 2.1.0 release. I reproduced it by:
The This does appear to hang, but it does complete eventually, after maybe a half hour (and a gigantic number of requests). |
|
@TheCakeIsNaOH thanks for that detail. |
|
I managed to reproduce it in a virtual machine with these steps:
And yes, after waiting long enough it will continue but it shows the following warning. 2023-07-03 08:45:44,451 5344 [WARN ] - [NuGet] One or more unresolved package dependency constraints detected in the Chocolatey lib folder. All dependency constraints must be resolved to add or update packages. If these packages are being updated this message may be ignored, if not the following error(s) may be blocking the current package operation: 'chromium 114.0.5735.199 constraint: chocolatey (>= 0.10.5)' |
|
@MajorGlory86 thank you for the reproduction steps. I can confirm that I am also seeing an issue on a fresh machine when attempting these steps. The command does eventually finish, but for me, there were over 1300 outgoing requests before the upgrade of the package is completed. This is certainly not what is intended, and we will need to investigate, and correct this problem. Thank you all for highlighting this problem! |
|
So, I'm pretty sure the problem is here: choco/src/chocolatey/infrastructure.app/services/NugetService.cs Lines 1212 to 1218 in d836138 It just blindly gets all package versions of all "parent" packages (aka packages that take a dependency on the package being upgraded). When there are a large number of package versions of one of these parent packages (aka chromium taking a dependency on chocolatey) then getting this information takes a long time. It probably would make sense to optimize this by initially only getting say the latest version of each parent package, then trying to resolve dependencies, and if that fails, then get all package versions and retry dependency resolution. This can be solved long term by implementing Package Indices |
Yes, I believe that you are correct that this is the problem area. I hadn't linked to it earlier, but I had looked this code before you posted. Need to have some discussions about the correct approach going forward on this one, to ensure that we don't break something elsewhere in the codebase. |
When attempting to figure out what packages need to be verified/checked during a package upgrade operation, we were previously fetching information about all package versions for all "parent packages" (i.e. packages that include a dependency on the package being acted on). This causes a problem when doing something like `choco upgrade chocolatey` when you have the chromium package installed. When in this situation, due to the high number of package versions for chromium and the fact that it takes a dependency on Chocolatey, the number of queries that are emitted for Chocolatey CLI is very large. This makes it appear as though the operation has stalled, when in actual fact it is just working its way through the queries (this can be verified by looking at the requests through fiddler when this command is running). This commit attempt to address this problem by changing to only query for the latest package version (and/or the requested package version) when attempting the look up for the parent package dependencies. This greatly reduces the number of outgoing queries, but it also means that it is possible for things to not work correctly due to not all package information being available. To guard against this, a fallback has been put in place such that if the initial attempt at resolving the solution fails, it goes back and fetches information about all package versions for the parent packages, and attempt to gather the resolve the solution again. This change has been made in a way that there is some code duplication, however, based on some discussion within the team, the way that this method is written needs to be overhauled, and now is not the right time to do this. Instead, we are accepting the duplication, until some effort can be put in to overhaul this method completely.
When attempting to figure out what packages need to be verified/checked during a package upgrade operation, we were previously fetching information about all package versions for all "parent packages" (i.e. packages that include a dependency on the package being acted on). This causes a problem when doing something like `choco upgrade chocolatey` when you have the chromium package installed. When in this situation, due to the high number of package versions for chromium and the fact that it takes a dependency on Chocolatey, the number of queries that are emitted for Chocolatey CLI is very large. This makes it appear as though the operation has stalled, when in actual fact it is just working its way through the queries (this can be verified by looking at the requests through fiddler when this command is running). This commit attempt to address this problem by changing to only query for the latest package version (and/or the requested package version) when attempting the look up for the parent package dependencies. This greatly reduces the number of outgoing queries, but it also means that it is possible for things to not work correctly due to not all package information being available. To guard against this, a fallback has been put in place such that if the initial attempt at resolving the solution fails, it goes back and fetches information about all package versions for the parent packages, and attempt to gather the resolve the solution again. This change has been made in a way that there is some code duplication, however, based on some discussion within the team, the way that this method is written needs to be overhauled, and now is not the right time to do this. Instead, we are accepting the duplication, until some effort can be put in to overhaul this method completely.
When attempting to figure out what packages need to be verified/checked during a package upgrade operation, we were previously fetching information about all package versions for all "parent packages" (i.e. packages that include a dependency on the package being acted on). This causes a problem when doing something like `choco upgrade chocolatey` when you have the chromium package installed. When in this situation, due to the high number of package versions for chromium and the fact that it takes a dependency on Chocolatey, the number of queries that are emitted for Chocolatey CLI is very large. This makes it appear as though the operation has stalled, when in actual fact it is just working its way through the queries (this can be verified by looking at the requests through fiddler when this command is running). This commit attempt to address this problem by changing to only query for the latest package version (and/or the requested package version) when attempting the look up for the parent package dependencies. This greatly reduces the number of outgoing queries, but it also means that it is possible for things to not work correctly due to not all package information being available. To guard against this, a fallback has been put in place such that if the initial attempt at resolving the solution fails, it goes back and fetches information about all package versions for the parent packages, and attempt to gather the resolve the solution again. This change has been made in a way that there is some code duplication, however, based on some discussion within the team, the way that this method is written needs to be overhauled, and now is not the right time to do this. Instead, we are accepting the duplication, until some effort can be put in to overhaul this method completely.
(#3237) Reduce number of queries for dependencies
gep13
changed the title
gep13
changed the title
* release/2.2.0: (21 commits) (doc) Update to indicate new package version used (maint) Add helper to split on max line lengths (#3281) Add validation for cache folder permissions (#3264) Update to latest Chocolatey.NuGet.Client (#3264) Ignore lock folders in cache directories (#3186) Remove easter egg (doc) Improve error message for defaultPushSource (tests) Clear HTTP Cache before getting packages (#3258) Expand logging for nuget resources errors (maint) Set file encoding to include BOM (#3237) Reduce number of queries for dependencies (#3231) Add tests to ensure package listing (maint) Remove unnecessary using statements (#3231) Don't refresh local package info during upgrade no-ops (build) Update to latest recipe package (doc) Minor corrections to wording (#3242) Add a script to run Authenticated tests (#3242) Attempt default credentials for sources (maint) Fix incorrect naming style uses (doc) Apply scripting best practices to output ...
|
🎉 This issue has been resolved in version 2.2.0 🎉 The release is available on: Your GitReleaseManager bot 📦🚀 |
|
So, eh, how do I update from 2.1.0 to 2.2.0? Am I missing the solution path? I'm stuck because of the Chromium thing, too.. |
You can upgrade. It just takes a while to finish. Like 5 minutes or so. |
The last time I had a look at my 2 devices without anything that happened was at around 30 minutes. But it worked after all.^^ |
|
Hopefully people have managed to upgrade at this point, but I know of three solutions:
|
Checklist
What You Are Seeing?
When upgrading to v2.1.0 from v2.0.0 the update process hangs and seems to be searching for a wrong package on the API (in this case it was chromium).
What is Expected?
Upgrading to v2.1.0 should be the result of this action.
How Did You Get This To Happen?
choco upgrade chocolatey --verbose
System Details
Installed Packages
Output Log
Additional Context
As a workaround I temporarily moved the chromium folder from the lib directory to another location. After the update to v2.1.0 I moved it back.
The text was updated successfully, but these errors were encountered: