Validate chef-client packages downloaded into the chef-workstation cache

[stuartpreston]

Description

Invalid chef-client packages (e.g. because of an interrupted download) should not be transferred to target nodes for installation when no client exists.

Chef Apply Version

0.2.2

Platform Version

Linux/Ubuntu 18.04-LTS

Target Platform Version

Windows Server 2016

Replication Case

0. rm -rf ~/chef-workstation/cache
1. Against a target node that does not have Chef Client installed yet:
2. Perform any valid chef-run command, e.g. chef-run winrm://mymachine --user user --password password user stuart action=create
3. Interrupt the download that populates the cache.
4. Re-run the original command.
5. Observe the package being transferred to the remote machine then failing to install (e.g. MSI 1,620 error on Windows)

Client Output

During a chef-run on a clean machine, the download got interrupted for some reason:

spreston@Azure:~$ chef-run winrm://51.144.88.152 --user azure --password MYPASSWORDHERE user z1 action=create
[✔] Packaging cookbook... done!
[✔] Generating local policyfile... exporting... done!
[-] Applying user[z1] from resource to target.
└── [-] [51.144.88.152] Downloading Chef client installer into local cache.
^CINTERNAL ERROR
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Message:

Backtrace:
/opt/chef-workstation/embedded/lib/ruby/gems/2.5.0/gems/tty-spinner-0.8.0/lib/tty/spinner/multi.rb:133:in `join'
/opt/chef-workstation/embedded/lib/ruby/gems/2.5.0/gems/tty-spinner-0.8.0/lib/tty/spinner/multi.rb:133:in `each'
/opt/chef-workstation/embedded/lib/ruby/gems/2.5.0/gems/tty-spinner-0.8.0/lib/tty/spinner/multi.rb:133:in `auto_spin'
/opt/chef-workstation/embedded/lib/ruby/gems/2.5.0/gems/chef-apply-0.1.21/lib/chef_apply/ui/terminal.rb:75:in `render_parallel_jobs'
/opt/chef-workstation/embedded/lib/ruby/gems/2.5.0/gems/chef-apply-0.1.21/lib/chef_apply/cli.rb:151:in `render_converge'
/opt/chef-workstation/embedded/lib/ruby/gems/2.5.0/gems/chef-apply-0.1.21/lib/chef_apply/cli.rb:111:in `perform_run'
/opt/chef-workstation/embedded/lib/ruby/gems/2.5.0/gems/chef-apply-0.1.21/lib/chef_apply/cli.rb:71:in `block in run'
/opt/chef-workstation/embedded/lib/ruby/gems/2.5.0/gems/chef-apply-0.1.21/lib/chef_apply/telemeter.rb:85:in `block in timed_capture'
/opt/chef-workstation/embedded/lib/ruby/2.5.0/benchmark.rb:293:in `measure'
/opt/chef-workstation/embedded/lib/ruby/gems/2.5.0/gems/chef-apply-0.1.21/lib/chef_apply/telemeter.rb:85:in `timed_capture'
/opt/chef-workstation/embedded/lib/ruby/gems/2.5.0/gems/chef-apply-0.1.21/lib/chef_apply/telemeter.rb:74:in `timed_run_capture'
/opt/chef-workstation/embedded/lib/ruby/gems/2.5.0/gems/chef-apply-0.1.21/lib/chef_apply/cli.rb:69:in `run'
/opt/chef-workstation/embedded/lib/ruby/gems/2.5.0/gems/chef-apply-0.1.21/lib/chef_apply/startup.rb:174:in `start_chef_apply'
/opt/chef-workstation/embedded/lib/ruby/gems/2.5.0/gems/chef-apply-0.1.21/lib/chef_apply/startup.rb:61:in `run'
/opt/chef-workstation/embedded/lib/ruby/gems/2.5.0/gems/chef-apply-0.1.21/bin/chef-run:23:in `<top (required)>'
/usr/bin/chef-run:288:in `load'
/usr/bin/chef-run:288:in `<main>'

I re-ran the command but got an error at install time:

spreston@Azure:~$ chef-run winrm://51.144.88.152 --user azure --password MYPASSWORDHERE user z1 action=create
✔] Packaging cookbook... done!
[✔] Generating local policyfile... exporting... done!
[✖] Applying user[z1] from resource to target.
└── [✖] [51.144.88.152] The command 'cmd /c msiexec /package C:\Users\azure\AppData\Local\Temp\chef-installer\chef-client-14.7.17-1-x64.
msi /quiet' exited with return code '1,620' on '51.144.88.152'.
CHEFRMT001

The command 'cmd /c msiexec /package C:\Users\azure\AppData\Local\Temp\chef-installer\chef-client-14.7.17-1-x64.msi /quiet' exited withreturn code '1,620' on '51.144.88.152'.

The following error was reported:

  This installation package could not be opened.  Contact the application vendor to verify that this is a valid Windows Installer package.




If you are not able to resolve this issue, please contact Chef support
at [email protected]

On closer inspection of the cache, the file size was not correct. (why don’t we show the file size on the download site?). I moved the file to .msi.backup and retried the chef-run command:

spreston@Azure:~/.chef-workstation/cache$ ls -l
total 368948
-rw-r--r-- 1 spreston spreston  54162504 Oct 13 15:44 chef_14.5.33-1_amd64.deb
-rw-r--r-- 1 spreston spreston 189489152 Nov 15 14:26 chef-client-14.7.17-1-x64.msi
-rw-r--r-- 1 spreston spreston 133761080 Nov 15 14:17 chef-client-14.7.17-1-x64.msi.backup
spreston@Azure:~/.chef-workstation/cache$ sha256sum chef-client-14.7.17-1-x64.msi
17e570f1bce16d3bb090addbbfdb97fb78fe6d96da88ff20c456c5e2567828d4  chef-client-14.7.17-1-x64.msi
spreston@Azure:~/.chef-workstation/cache$ sha256sum chef-client-14.7.17-1-x64.msi.backup
7a17b0cf1310fb9f30a3054ca872fb09e030d818797d192a6c546e84a7dc4b23  chef-client-14.7.17-1-x64.msi.backup

Requested Enhancement

1. Maybe perform a two stage download of the .msi into ~/.chef-workstation/cache - first with a . extension then once the download is complete, verify the sha256sum matches the expected value, move it to the correct filename so it’s ready for use.
2. Show the size of the Chef Client downloads on the download site (to help diagnose whether the package size is in the right vicinity) - I’ll raise an issue elsewhere for this!

[tyler-ball]

Thanks for the great bug @stuartpreston and for the proposed enhacements!