Initial commit of 2.0.0_wip version

- Remove attributes that were repeated across different resources all working on the same Elasticsearch instance. Add methods in Helpers that do these lookups. We default to 'elasticsearch' and 'default', fail if there's more than one match, but also allow the user to override with `instance_name`. - All providers and resources: - move class names outside of Chef namespace - add `provides :...` to all resources and providers - all resources that depend on another resource now look up that resource for values, e.g. `elasticsearch_plugin` will run commands with the user specificed in `elasticsearch_user`, fixes #373 - add `instance_name` to point every provider to the related resources - elasticsearch_install: - deprecate String for type parameter, and only use symbols - drop :tar in favor of :tarball. We don't need to say tarball two ways. - ensure the default recipe uses the attribute for install type - elasticsearch_configure: - add additional paths `path_pid, path_plugins, path_bin` - change all path parameters to use a hash with two keys -- :package and :tarball - elasticsearch_plugin: - ensure plugins directory exists and has correct ownership, to address differences in ES 1.x and 2.0.0, fixes #382 - change `-install` on plugin command to `install`, fixes #383 - warn if the user is set to 'root' and ES version < 2.0.0 - elasticsearch_service: - drop `node_name` parameter, and look it up from elasticsearch_configure - drop pid file parameter, use a sanitized node name for that now too - Update Berkshelf in Gemfile to Berkshelf ~> 4.0.0. - Testing changes: - drop testing and support for Chef 11.x - add Chef 12.5.1 testing and support - split the 'override' testing into separate package and tarball recipes - chefspec tests now try to identify which type of install happens, and test for those specifically
sous-chefs · Oct 20, 2015 · dc77bc5 · dc77bc5
1 parent dc13aeb
commit dc77bc5
Show file tree

Hide file tree

Showing 27 changed files with 843 additions and 619 deletions.
diff --git a/.kitchen.yml b/.kitchen.yml
@@ -42,19 +42,26 @@ suites:
     require_chef_omnibus: 12.5.1
     attributes:
       elasticsearch:
-        install_type: 'package'
+        install_type: package
 
     run_list:
       - recipe[java]
       - recipe[elasticsearch_test::default_with_plugins]
 
-  - name: elasticsearch_test # the override-everything use case
+  - name: override_default # the override-everything use case
     driver_config:
       require_chef_omnibus: 12.5.1
     run_list:
       - recipe[java]
       - recipe[elasticsearch_test::default]
 
+  - name: override_package # the override-everything use case
+    driver_config:
+      require_chef_omnibus: 12.5.1
+    run_list:
+      - recipe[java]
+      - recipe[elasticsearch_test::package]
+
   - name: chef-12.4.3
     driver_config:
       require_chef_omnibus: 12.4.3
@@ -90,12 +97,3 @@ suites:
     run_list:
       - recipe[java]
       - recipe[elasticsearch_test::default_with_plugins]
-
-  - name: chef-11.18.12
-    driver_config:
-      require_chef_omnibus: 11.18.12
-    includes:
-      - ubuntu-14.04
-    run_list:
-      - recipe[java]
-      - recipe[elasticsearch_test::default_with_plugins]
diff --git a/Gemfile b/Gemfile
@@ -6,7 +6,7 @@ group :lint do
 end
 
 group :unit do
-  gem 'berkshelf', '~> 3'
+  gem 'berkshelf', '~> 4.0.0'
   gem 'chefspec', '>= 4.2'
   gem 'chef-sugar'
 end

diff --git a/README.md b/README.md
@@ -1,34 +1,59 @@
 # Elasticsearch Chef Cookbook
 
 This cookbook has been converted into a library cookbook as of version 1.0.0,
-and supports Chef 12.4.1, 12.3.0, 12.2.1, and higher. It implements support for
+and supports Chef 12.5.1, 12.4.3, 12.3.0, 12.2.1, 12.1.2, and higher. It implements support for
 CI as well as more modern testing with chefspec and test-kitchen. It no longer
 supports some of the more extraneous features such as discovery (use [chef search](http://docs.chef.io/chef_search.html) in your wrapper cookbook) or EBS
 device creation (use [the aws cookbook](https://github.com/opscode-cookbooks/aws)).
 
 The previous version of this cookbook may be found in the [0.3.x branch](https://github.com/elastic/cookbook-elasticsearch/tree/0.3.x).
 
 ## Default version, download URLs, and checksums
-Please consult [attributes/default.rb](attributes/default.rb) for these values.
-Both the recipes and resources/providers here source their default values for
-Elasticsearch version, download URL, and Checksum from `attributes/default.rb`.
+
+
+
+## Attributes
+
+Please consult [attributes/default.rb](attributes/default.rb) for a large list
+of checksums for many different archives and package files of different
+elasticsearch versions. Both recipes and resources/providers here use those
+default values.
 
 Please take note that you may use `%s` in your URL and this cookbook will use
 sprintf/format to insert the version parameter as a string into your
 download_url.
 
-You may adjust the node attributes to force this cookbook to use different
-default values for all three settings.
+|Name|Default|Other values|
+|----|-------|------------|
+|`default['elasticsearch']['version']`|`'1.7.3'`|[See list](attributes/default.rb).|
+|`default['elasticsearch']['install_type']`|`:tarball`|`:package`|
+|`default['elasticsearch']['download_urls']['debian']`|`'https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-%s.deb'`|`%s` will be replaced with the version attribute above|
+|`default['elasticsearch']['download_urls']['rhel']`|`'https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-%s.noarch.rpm'`|`%s` will be replaced with the version attribute above|
+|`default['elasticsearch']['download_urls']['tar']`|`'https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-%s.tar.gz'`|`%s` will be replaced with the version attribute above|
 
 ## Recipes
 
+Resources are the intended way to consume this cookbook, however we have
+provided a single recipe that configures Elasticsearch by downloading an archive
+containing a distribution of Elasticsearch, and extracting that into /usr/local.
+
+See the attributes section above to for what defaults you can adjust.
+
 ### default
 
 The default recipe creates an elasticsearch user and group with the default
 options.
 
 ## Resources
 
+You should be aware that potentially all resources in this cookbook look each
+other up in the resource collection. By default, they will look for appropriate
+resources named 'default' or 'elasticsearch', as in the examples below. For
+example, if you use `elasticsearch_install 'default'`, `elasticsearch_config`
+will use the first resource to determine how you installed Elasticsearch. If you
+need to override this behavior, all resources accept `instance_name` as an
+additional parameter, to be used for matching.
+
 ### elasticsearch_user
 Actions: `:create`, `:remove`
 
@@ -58,19 +83,19 @@ end
 Actions: `:install`, `:remove`
 
 Downloads the elasticsearch software, and unpacks it on the system. There are
-currently two ways to install -- `package`, which downloads the appropriate
+currently two ways to install -- `:package`, which downloads the appropriate
 package from elasticsearch.org and uses the package manager to install it, and
-`tarball` which downloads a tarball from elasticsearch.org and unpacks it in
+`:tarball` which downloads a tarball from elasticsearch.org and unpacks it in
 /usr/local on the system. The resource name is not used for anything in
 particular. This resource also comes with a `:remove` action which will remove
 the package or directory elasticsearch was unpacked into.
 
 You may always specify a download_url and/or download_checksum, and you may
 include `%s` which will be replaced by the version parameter you supply.
 
-Please be sure to consult the above section 'Default version, download URLs,
-and checksums' as that controls how Elasticsearch version, download URL and
-checksum are determined if you omit them.
+Please be sure to consult the above attribute section as that controls how
+Elasticsearch version, download URL and checksum are determined if you omit
+them.
 
 Examples:
 
@@ -83,9 +108,6 @@ elasticsearch_install 'my_es_installation' do
   type :tarball # type of install
   dir '/usr/local' # where to install
 
-  owner 'elasticsearch' # user and group to install under
-  group 'elasticsearch'
-
   download_url "https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.7.2.tar.gz"
   # sha256
   download_checksum "6f81935e270c403681e120ec4395c28b2ddc87e659ff7784608b86beb5223dd2"
@@ -108,6 +130,7 @@ elasticsearch_install 'my_es_installation' do
   download_url "https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.7.2.deb"
   # sha256
   download_checksum "791fb9f2131be2cf8c1f86ca35e0b912d7155a53f89c2df67467ca2105e77ec2"
+  instance_name 'elasticsearch'
   action :install # could be :remove as well
 end
 ```
@@ -151,12 +174,13 @@ More complicated -
 elasticsearch_configure 'my_elasticsearch' do
   # if you override one of these, you probably want to override all
   dir '/usr/local/awesome'
-  path_conf "/usr/local/awesome/etc/elasticsearch"
-  path_data "/usr/local/awesome/var/data/elasticsearch"
-  path_logs "/usr/local/awesome/var/log/elasticsearch"
+  path_conf     tarball: "/usr/local/awesome/etc/elasticsearch"
+  path_data     tarball: "/usr/local/awesome/var/data/elasticsearch"
+  path_logs     tarball: "/usr/local/awesome/var/log/elasticsearch"
+  path_pid      tarball: "/usr/local/awesome/var/run/elasticsearch"
+  path_plugins  tarball: "/usr/local/elasticsearch/plugins"
+  path_bin      tarball: "/usr/local/bin"
 
-  user 'foo'
-  group 'bar'
   logging({:"action" => 'INFO'})
 
   allocated_memory '123m'
@@ -194,10 +218,6 @@ elasticsearch_service 'elasticsearch'
 ```
 elasticsearch_service 'elasticsearch-crazy' do
   node_name 'crazy'
-  path_conf '/usr/local/awesome/etc/elasticsearch'
-  pid_path '/usr/local/awesome/var/run'
-  user 'foo'
-  group 'bar'
 end
 ```
 
@@ -224,9 +244,7 @@ To run multiple instances per machine, an explicit `plugin_dir` location
 has to be provided:
 
 ```
-elasticsearch_plugin 'mobz/elasticsearch-head' do
-  plugin_dir '/usr/local/awesome/elasticsearch-1.7.2/plugins'
-end
+elasticsearch_plugin 'mobz/elasticsearch-head'
 ```
 NB: You [may encounter issues on certain distros](http://blog.backslasher.net/java-ssl-crash.html) with NSS 3.16.1 and OpenJDK 7.x.
 

diff --git a/attributes/default.rb b/attributes/default.rb
@@ -1,6 +1,6 @@
 # elasticsearch version & install type
 default['elasticsearch']['version'] = '1.7.3'
-default['elasticsearch']['install_type'] = 'tarball'
+default['elasticsearch']['install_type'] = :tarball
 
 # platform_family keyed download URLs
 default['elasticsearch']['download_urls']['debian'] = 'https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-%s.deb'

diff --git a/libraries/helpers.rb b/libraries/helpers.rb
@@ -1,6 +1,67 @@
 module ElasticsearchCookbook
   # Helper methods included by various providers and passed to the template engine
   module Helpers
+
+    def find_es_resource(run_context, resource_type, resource)
+      resource_name = resource.name
+      instance_name = resource.instance_name
+
+      # if we are truly given a specific name to find
+      name_match = find_exact_resource(run_context, resource_type, resource_name) rescue nil
+      return name_match if name_match
+
+      # first try by instance name attribute
+      name_instance = find_instance_name_resource(run_context, resource_type, instance_name) rescue nil
+      return name_instance if name_instance
+
+      # otherwise try the defaults
+      name_default = find_exact_resource(run_context, resource_type, 'default') rescue nil
+      name_elasticsearch = find_exact_resource(run_context, resource_type, 'elasticsearch') rescue nil
+
+      # if we found exactly one default name that matched
+      return name_default if name_default && !name_elasticsearch
+      return name_elasticsearch if name_elasticsearch && !name_default
+
+      raise "Could not find exactly one #{resource_type} resource, and no specific resource or instance name was given"
+    end
+
+    # find exactly the resource name and type, but raise if there's multiple matches
+    # see https://github.com/chef/chef/blob/master/lib/chef/resource_collection/resource_set.rb#L80
+    def find_exact_resource(run_context, resource_type, resource_name)
+      rc = run_context.resource_collection
+      result = rc.find(resource_type => resource_name)
+
+      if result && result.kind_of?(Array)
+        str = ''
+        str << "more than one #{resource_type} was found, "
+        str << "you must specify a precise resource name"
+        raise str
+      end
+
+      return result
+    end
+
+    def find_instance_name_resource(run_context, resource_type, instance_name)
+      results = []
+      rc = run_context.resource_collection
+
+      rc.each do |r|
+        next unless r.resource_name == resource_type && r.instance_name == instance_name
+        results << r
+      end
+
+      if !results.empty? && results.length > 1
+        str = ''
+        str << "more than one #{resource_type} was found, "
+        str << "you must specify a precise instance name"
+        raise str
+      elsif !results.empty?
+        return results.first
+      end
+
+      return nil # falsey
+    end
+
     def determine_version(new_resource, node)
       if new_resource.version
         new_resource.version.to_s
@@ -56,14 +117,6 @@ def determine_download_checksum(new_resource, node)
       end
     end
 
-    def get_tarball_home_dir(new_resource, node)
-      new_resource.dir || node['ark']['prefix_home']
-    end
-
-    def get_tarball_root_dir(new_resource, node)
-      new_resource.dir || node['ark']['prefix_root']
-    end
-
     # This method takes a hash, but will convert to mash
     def print_value(data, key, options = {})
       separator = options[:separator] || ': '