Add a synced folder to persist chef omnibus packages

[afiune]

Setting up the new chef_directory interface within the Driver. Then with this directory we will allow the the instance to have a cache directory where omnitruck will download the artifacts, so we can cache the packages locally and avoid re-pulling down artifacts that you already have on your workstation.

Signed-off-by: Salim Afiune [email protected]

[tduffield]

extra # inline here

[afiune]

Yup, thanks! 😄

[tduffield]

I'm not sure this format will work on Windows. We might need to do something a little more Windows friendly?

[thommay]

Astonishingly, this actually does the right thing on windows:

irb(main):002:0> File.expand_path '~/.chef'
=> "C:/Users/thom/.chef"

[tduffield]

Oh cool!

[coderanger]

It's actually a bigger problem on Unix systems:

$ env HOME=/tmp ruby -e 'puts File.expand_path("~/foo")'
/tmp/foo

When running in a jenkins job or similar, it's easy for $HOME to not be set correctly since that's usually handled by the login shell. Here be dragons. Might be better to explicitly get the current user's homedir.

[mwrock]

The unfortunate issue with this path is that it makes this cache cookbook local. A user may test many cookbooks (very common) and each one will need to maintain their own cache in its local .kitchen folder. Would be nice to move it to a temp directory of the host.

[afiune]

@mwrock Apologize Matt but I couldn't understand your concern about the cache directory in the scope of testing multiple cookbooks, the cache is uniq per user, that is why we are storing it inside the home directory, that means that it will be shared across suites, instances and even projects (different repositories).

Another important thing to mention is that this cache is for chef packages and not cookbooks, maybe that cache that you mention is the actual berkshelf cache which is different.

Let me know if I am helping with this comment or confusing you more. 😄

[mwrock]

argh. nevermind, I was confusing local/remote locations.

[tduffield]

Also, would love to see some tests! :)

[smurawski]

[mwrock]

This PR looks good but after testing this I think we have a problem in Test-Kitchen that is going to severely limit the reach of this feature. That is that it only "kicks in" when provisioner options include chef_omnibus_install_options: -P chef and as far as I have seen, only our own internal CI processes use that. We should really address this in TK so that this caching gets used when the provisioner options are simply:

provisioner:
  name: chef_zero

or something similarly minimal.

[afiune]

@mwrock that is so weird because the code explicitly expect chef_omnibus_install_options to be nil (code) - But I wonder if the problem is that it is not nil and instead it is empty ""... 🤔

[mwrock]

I was mainly looking at

      def install_options
        add_omnibus_directory_option if instance.driver.cache_directory
        project = /\s*-P (\w+)\s*/.match(config[:chef_omnibus_install_options])
        {
          :omnibus_url => config[:chef_omnibus_url],
          :project => project.nil? ? nil : project[1],
          :install_flags => config[:chef_omnibus_install_options],
          :sudo_command => sudo_command
        }.tap do |opts|
          opts[:root] = config[:chef_omnibus_root] if config.key? :chef_omnibus_root
          [:install_msi_url, :http_proxy, :https_proxy].each do |key|
            opts[key] = config[key] if config.key? key
          end
        end
      end

Only the first case will use chef_omnibus_install_options

[mwrock]

when I added product to my options (which no one does) it started caching

[vinyar]

So, what's the status of this feature?

It's loosely documented and does it work as intended? @mwrock