Choco pack is extremely slow on Windows Environment

[jetersen]

Description
Choco pack is extremely slow on the windows virtual environments not sure why...

Area for Triage:

Question, Bug, or Feature?:

Virtual environments affected

• Ubuntu 16.04
• Ubuntu 18.04
• Ubuntu 20.04
• macOS 10.15
• macOS 11.0
• Windows Server 2016 R2
• Windows Server 2019

Image version

Current runner version: '2.277.1'
Operating System
  Microsoft Windows Server 2019
  10.0.17763
  Datacenter
Virtual Environment
  Environment: windows-2019
  Version: 20210316.1
  Included Software: actions/virtual-environments@win19/20210316.1/images/win/Windows2019-Readme.md
  Image Release: actions/virtual-environments@win19%2F20210316.1 (release)

Expected behavior
Choco pack is relatively fast to pack.
Locally it builds in under a seconds.

Actual behavior
Choco pack on Windows when calling "choco pack" directly takes roughly over two minutes.
And calling choco pack via CakeBuild takes over one minutes to pack.

Repro steps
https://github.com/jetersen/test-choco-pack-github-action

1. Create a nuget spec
2. and call choco pack

[jetersen]

I created a repro:
https://github.com/jetersen/test-choco-pack-github-action

Even when I forgot to checkout the files, it takes over a minute to figure out that the nuspec file is missing.
https://github.com/jetersen/test-choco-pack-github-action/runs/2176728690?check_suite_focus=true

[jetersen]

After testing it seems that the initial call to choco is the one being extremely slow...
Also seems 2016 is a bit faster to do the initial call to choco.

[al-cheb]

Hello, @jetersen
I can reproduce the same behavior. We pre-install a lot of chocolatey packages that's why searching operation for the first time takes soo much time:

2021-03-25 11:57:42,834 5836 [DEBUG] - RemovePendingPackagesTask is now ready and waiting for PreRunMessage.
2021-03-25 11:57:42,840 5836 [DEBUG] - Sending message 'PreRunMessage' out if there are subscribers...
2021-03-25 11:57:42,847 5836 [DEBUG] - [Pending] Removing all pending packages that should not be considered installed...
2021-03-25 11:59:08,041 5836 [DEBUG] - Performing validation checks.

Workaround(temporary rename lib folder) :

- name: DotNet Tool Restore
  run:  dotnet tool restore
- name: rename lib folder
  run: |
       rename-item "C:\ProgramData\chocolatey\lib" "C:\ProgramData\chocolatey\lib1"
       $null = mkdir C:\ProgramData\chocolatey\lib
- name: Choco Pack
  run: choco.exe pack .\helloworld\helloworld.nuspec
- name: Cake Pack
  run: dotnet cake

Looks like it's well-known issue:
remove pending packages task - search only top level level package folders to clean up #1866
choco command takes time to initialise. How to skip RemovePendingPackagesTask? #2186
(GH-1866) Restrict pending package removal to top level #1941

cc: @gep13 @ferventcoder

[jetersen]

@al-cheb I wonder if you recreate these VM every time would it not be better to clean the choco lib folder after all the install of choco packages is done?

[al-cheb]

@al-cheb I wonder if you recreate these VM every time would it not be better to clean the choco lib folder after all the install of choco packages is done?

If we clean the lib folder we will break choco packages.

https://chocolatey.org/courses/getting-started/installation-in-detail

Chocolatey packages are installed to ChocolateyInstall\lib, but the software could go to various locations, depending on how the package maintainer created the package.

Some packages are installed under ChocolateyInstall\lib, others - especially packages that are based on Windows installers (.msi files) - install to the default path of the original installer (which is most likely within Program Files).

[jetersen]

Right forgot that some packages live in the lib folder 😅

[al-cheb]

Close as external.

[gep13]

A nee version of Chocolatey v0.11.1 has been released that should address this issue. If we could get it installed on the build agent machine and test to ensure if it is fixed, that would be great. Thanks!