with_chef_local_server can't find vendored cookbook

[dvlscm]

I created a brand new cookbook called stage with Berkshelf. My directory structure looks like this:

project_root
  cookbooks
    stage
      recipes
        base.rb
      metadata.rb
  vendored
    apt

I already created two machines with a metal recipe. My metal recipe to converge looks like so:

require 'chef_metal'
with_driver 'fog:AWS', :compute_options => { :aws_access_key_id => '...', :aws_secret_access_key => '...' }
with_chef_local_server :chef_repo_path => '/Users/me/project_root', :cookbook_path => ['/Users/me/project_root/cookbooks','/Users/me/project_root/vendored']
machine_batch do
action :converge
machine 'box1' do
recipe 'stage::base'
end
end

When I run it, I get:

            Starting Chef Client, version 11.14.2
    [test2] resolving cookbooks for run list: ["stage::base"]
===========================================================================
            Error Resolving Cookbooks for Run List:
===========================================================================
            Missing Cookbooks:
            ------------------
            No such cookbook: apt
            Expanded Run List:
            ------------------
            * stage::base

Based on what I've learned from other issues, I think the :cookbook_path in with_chef_local_server is correct. All of the directories it points to exist.

If I run the converge recipe with the apt cookbook in project_root/cookbooks, it runs successfully. If I run with the apt cookbook only in project_root/vendored, I get the error. This seems like a bug. Please correct me if I've still got something wrong in my setup.

[jkeiser]

Yep, what I'm hearing is the most recent cheffish broke that: chef/cheffish#22 . I won't be able to look into it for a couple of hours, but ... is the "stage" cookbook in your vendored directory too? Can you set your chef_repo_path there for now?

[jkeiser]

i.e. just have one path, "vendored"

[dvlscm]

I think I'll give that a try and wait for your fix, whatever that ends up being. Unless I clean up the vendored directory, yes, the stage cookbook is in there, along with a Berksfile.lock. So in theory, if I have a repo path set to that directory, it should work. Actually, I'll create a directory with a better name and stick stuff in there. I'll report back with my results.

[dvlscm]

All right, I gave it a go. Didn't work, but I'm not confident I'm doing this right. Here's the rundown...

I created a directory structure like this:

project_root
  pipe

Then I vendored by cookbooks into a vendored directory inside pipe, with this result

project_root
  pipe
    vendored
      apt
      stage
        recipes
          base.rb
      Berksfile.lock

Then I ran my converge recipe from inside the pipe directory:

require 'chef_metal'
with_driver 'fog:AWS', :compute_options => { :aws_access_key_id => '...', :aws_secret_access_key => '...' }
with_chef_local_server :chef_repo_path => '/Users/me/project_root/pipe', :cookbook_path => '/Users/me/project_root/pipe/vendored'
machine_batch do
action :converge
machine 'box1' do
recipe 'stage::base'
end
end

When I ran that, I got json files in pipe/nodes and pipe/clients (and a json file for my local machine in project_root/nodes, not sure why that ended up there) but I also got this error:

Starting Chef Client, version 11.14.2
resolving cookbooks for run list: ["stage::base"]                                 ===========================================================================
Error Resolving Cookbooks for Run List:
===========================================================================
Missing Cookbooks:
------------------
No such cookbook: stage
Expanded Run List:
------------------
 * stage::base

The pipe/vendored directory does have the stage cookbook, which does indeed contain recipes/base.rb, so it's a little mysterious to me why metal can't find things. Unfortunately the stacktrace doesn't make things any clearer. Am I doing something obviously wrong?

The more I think about it, I'm happy to structure my stuff as in this experiment, if I can just get it to work. Not having to specify multiple cookbook paths is fine by me.

[dvlscm]

I can confirm after running all day today that I can run with my cookbook dependencies loaded into the cookbooks directory in my project root, like so:

project_root
  clients
  cookbooks
  nodes

If I set :chef_repo_path for with_chef_local_server to project_root, things run successfully. That's not what I want, of course, but it does work.

What I'm after, and what I thought with_chef_local_server would let me do, is to specify a :chef_repo_path inside 'project_root, and a:cookbook_path` inside that repo path, like so:

project_root
  cookbooks
  pipe
    clients
    nodes
    vendored
      [cookbooks in here]

Vendoring to the project_root/cookbooks directory with Berkshelf won't work, since you can't vendor to an existing directory. So I have to copy cookbooks into there manually. I'd rather not :) I'd rather vendor them to project_root/pipe/vendored, which I can delete and refill at will.

If I'm after something impossible, let me know, but I really got the impression that what I want to do is perfectly acceptable.

[johnewart]

Setting cookbook paths was broken in the latest cheffish, that's fixed along with multiple cookbook paths in master - should be released soon. Closing.

[dvlscm]

Well, that's good news. I await the fix anxiously :) Thanks.