Update mongodb to e4c2f0809a7c44c1c940d50f8108d1a448d79dfb

e4c2f0809a7c44c1c940d50f8108d1a448d79dfb Merge pull request redhat-openstack#182 from hunner/antonlindstrom-feature/arbiter-replicaset 384889b09f98d17e05189ea4328773040a42c59a Add a stub 027af73f5ed85ae945ad3a55953f83b2619ec714 Convert specs to modern format 9fc086d98ffb74e14cb5da098925b5e43ec0b17c add arbiter configuration 5412c0c27d143453a570745ec5e6fd9ecf3e1367 Merge pull request redhat-openstack#180 from hunner/unique_name bfa02b1ca0d9f52818749b37f7bb1bb5bed319b3 Merge pull request redhat-openstack#181 from hunner/danehans-keyfile f8b5f2bbc2507736e39f70bcb518e66d17acd6f0 Change keyfile management 7bbcff8024152b4ec9ce53dbba00024523707162 Adds Support for Keyfile Key b981126a87fb853076c007db5e1cbcd176f80fc2 Allow for non-conflicting users in mongodb::db 5a35da665d7fb1067174c9a3a86920f0a51b9486 Merge pull request redhat-openstack#178 from enovance/sane_default_el7_mongos d6f2c0266045d7a2c29b4fd65cd5d2a9438ef320 mongos: Enable sane default for EL7 to work out of the box 4f5ca99fcba4339a8093d2cb329f5498866a8c8c Merge pull request redhat-openstack#176 from enovance/pin_rspec_core 7df82cc8f330f4d8b129c09dc66267ee1a720d6e Gemfile: Pin to rspec-core 3.1.7 due to some bug with ruby 1.8.7 cb8db6aafa7c55729adb7b35db02315f784ff2ec Merge pull request redhat-openstack#175 from enovance/various_26_fix 191a08831bf76d77fc1e0790892b4ff8a650d13a conn_validator: Run on 127.0.0.1 when bind_ip is set at 0.0.0.0 d6a9e4b00922b0bc35819a750bf7c3173520ec43 acceptance: use proper command exit status a70e7dd3d7d34d30e5f9fa50314319c2498db3d9 Allow mongo client to connect on conf spec port abfc625bdfb9587bb41b36d1125a4c91692f442a Uniformize the way mongos works in 2.4 and 2.6 eaff55ca631a832972ca9032984f62d7d1fddc66 Enable configsvr and shardsvr mode in 2.6 accf317263e8416391b7d51f9445fb863998a6d3 rs.conf() connects to the proper IP with 2.6 80650e0899c6d946e48a3281358ed4a05c01bbad Merge pull request redhat-openstack#173 from Cloudlynx/replsetfix befb6954f5b0c276d53d1f7cfe9c19d0cfd9c4b9 More fixes to debug messages d37b97c60b923ebe3dfefc1d0180e0d45f8f9eb3 Fixed a few debug statements 76682cffe8ce16b255f329af33083ccde7845271 Added code so the case of multiple IP adresses on bind_ip is correctly handled. 8e9ceced8e53f42152e1be8e3b18ba2c2e1e20c1 Merge pull request redhat-openstack#171 from Mylezeem/fix_lint 75baf2f221c8750f2e8227d957e6b42d6bc60d8e Fix some lint error to reach 5.0 on Code Quality def1a3dd04fa1e8170b2fd07a62acede14e66683 Merge pull request redhat-openstack#170 from cmurphy/release 6ec0139ca4d6de3eb69fc84b57d6f2f64e1482b1 Merge pull request redhat-openstack#168 from cmurphy/fix_port 2d38fff73c28c19137485d3c8a68b87e65c1df2d Prepare 0.10.0 Release 8644e55bcaf59bfd75800327a2aec6a9e85818d2 Merge pull request redhat-openstack#169 from cmurphy/fix_150 14208c294108a161edd22c5b5e359932b3a56546 Add acceptance tests for mongodb user 10d8f22323a4bda128661ebe8216dea2594013dd Fix port in 2.6 mongo.conf template 16991924e1e90ca76a9123daae03e95d58951fc9 Fix unit tests for mongodb_user provider changes 572ae1b0a6128b0faf97ea1bf09a85dc32e0974d db.runCommand with manually-joint json command d3121775932be5ac8e286628ed601e8e99b2bdb7 Fix the bug that new user's password_hash will be rehashed 218238c24fe7c0bc1a5d895b307eb064acc4c2ea Merge pull request redhat-openstack#164 from cmurphy/fix_137 e65d2dc41f565ef15f7cd7eab9eb7739eeef8033 Add validation to ssl parameters f24a1cc8b4004f170bb3069d787cf43cf5a83b11 Merge pull request redhat-openstack#167 from cmurphy/fix_165 bd579b79c9945801e4447d6e24fba8cda5296a04 Adding documentation for ssl support. 3d4bd2d51654080f9f686a40430f24bf31865976 Adding rudimentary support for ssl configuration options. ab2540076abdd4b66689a972a42f205039785aed Fix non-default port support and add tests 7aea8c07eb70847ebe4b37dc0f86372e863c7b3f Merge pull request redhat-openstack#151 from enovance/sharding dc5d6effbad7ca766ebf4dadd4b085cdf78aaa5c Implement Sharding Support a37d826b118fec9f939abb9887c34db8a058f3b7 Add support for mongos configuration 92632dc823f2468e8498bf6030c97d09005c4e32 Ensure rs.conf() is run on the proper mongod process 4a43ab4d07374f172aa41f55c99597889fdc0c99 Make port definition optional 1bb1995e8f4067a69a59fb335f1e27e601383af6 Add support for shardsvr 6cd09d1bbb304b2137f6819bdadd682da712882f Add support for configsvr 67bdd17956991e99d888cf282ce6bfcaecabba37 Fix the bug mongo js client fail to run under non-default port 51c6edb7b71ac67326c36e452a755178262f7a5d Merge pull request redhat-openstack#140 from crawlik/add_enterprise_support a3991bfeccd1d7ea8796f2c055ecf3b6275e76b5 Merge pull request redhat-openstack#159 from cmurphy/revert_rhel7 da347a00e0650d444df83947694b5da044c9962e Fix against mongodb 2.6.5 from epel 96c78e7b7c71ea88c00b76cd13f505e0298d000b Merge pull request redhat-openstack#155 from cmurphy/fix_epel_tests c10880221a94170007cdc2297189f20696d52455 Add centos 7 nodesets and fix spec helper to setup iptables on all hosts 799d22b3ad1dd7999221d8950a7057e510ded7aa Merge pull request redhat-openstack#154 from cmurphy/rhel7 bb6aff8fee032a2f71c4cd5cbb7d851af634dccb Merge pull request redhat-openstack#153 from cmurphy/fix_tests a45104593779e8c7acf46a3585fe18f43b8c8715 Add support for RHEL 7 f50e9c03e1b948ea49f6059ab5eca8600d857d77 Fix server acceptance tests a2fb53c7b58ea0c8fce1537cb75eaece581c8a44 Merge pull request redhat-openstack#152 from cmurphy/fix_beaker 37f237a66c3753f8bb077957cae7f0dc3f8ae4d2 Update Gemfile and .travis.yml 6805321418e7f4d6359e9b8e5f1233f20b5e1c65 Use boolean to check enterprise flag 640e18d20e40535c1ee6212a1efba839c7bd56ad Add enterprise repo support
xbezdick · Mar 3, 2015 · fcc23f0 · fcc23f0
1 parent 4797337
commit fcc23f0
Show file tree

Hide file tree

Showing 55 changed files with 2,126 additions and 156 deletions.
diff --git a/Puppetfile b/Puppetfile
@@ -91,7 +91,7 @@ mod 'module-data',
   :git => 'https://github.com/ripienaar/puppet-module-data.git'
 
 mod 'mongodb',
-  :commit => 'fe562b86f388a6d107bb1f3cb3b32a3978f59c2a',
+  :commit => 'e4c2f0809a7c44c1c940d50f8108d1a448d79dfb',
   :git => 'https://github.com/puppetlabs/puppetlabs-mongodb.git'
 
 mod 'mysql',

diff --git a/mongodb/.travis.yml b/mongodb/.travis.yml
@@ -1,33 +1,17 @@
-branches:
-  only:
-    - master
+---
 language: ruby
-before_install:
-  - gem update bundler
-  - bundle --version
-  - gem update --system 2.1.11
-  - gem --version
-bundler_args: --without development
-script: "bundle exec rake spec SPEC_OPTS='--format documentation'"
-after_success:
-  - git clone -q git://github.com/puppetlabs/ghpublisher.git .forge-release
-  - .forge-release/publish
-rvm:
-  - 1.8.7
-  - 1.9.3
-  - 2.0.0
-env:
-  matrix:
-    - PUPPET_GEM_VERSION="~> 2.7.0"
-    - PUPPET_GEM_VERSION="~> 3.3.0"
-  global:
-    - PUBLISHER_LOGIN=puppetlabs
-    - secure: "iUYpjvk33JffZB9lVRqjuwRWesvcvmTknh908xnf60rUOA0QbGEPXxQY+LsQJEIimVsMA22fV6vp9BcqMEjO7OfK2MvAWsEWU/lG+kisFqhWDRf96sADE7k/RvPWJeB2xe+lWXK7Eh26jgctNfk4NptX1X1MjGmdzEvH7Aq79/w="
+bundler_args: --without system_tests
+script: "bundle exec rake validate && bundle exec rake lint && bundle exec rake spec SPEC_OPTS='--format documentation'"
 matrix:
-  exclude:
-    - rvm: 1.9.3
-      env: PUPPET_GEM_VERSION="~> 2.7.0"
-    - rvm: 2.0.0
-      env: PUPPET_GEM_VERSION="~> 2.7.0"
+  fast_finish: true
+  include:
+  - rvm: 1.8.7
+    env: PUPPET_GEM_VERSION="~> 2.7.0" FACTER_GEM_VERSION="~> 1.6.0"
+  - rvm: 1.8.7
+    env: PUPPET_GEM_VERSION="~> 2.7.0" FACTER_GEM_VERSION="~> 1.7.0"
+  - rvm: 1.9.3
+    env: PUPPET_GEM_VERSION="~> 3.0"
+  - rvm: 2.0.0
+    env: PUPPET_GEM_VERSION="~> 3.0"
 notifications:
   email: false
diff --git a/mongodb/CHANGELOG.md b/mongodb/CHANGELOG.md
@@ -1,3 +1,18 @@
+##2015-01-13 - Release 0.10.0
+###Summary
+
+This release adds a number of significant features and several bug fixes.
+
+####Features
+- Adds support for sharding
+- Adds support for RHEL 7
+- Adds rudimentary support for SSL configuration
+- Adds support for the enterprise repository
+
+####Bugfixes
+- Fixes support for running on non-default ports
+- Fixes the idempotency of password setting (for mongo 2.6)
+
 ##2014-11-25 - Release 0.9.0
 ###Summary
 

diff --git a/mongodb/Gemfile b/mongodb/Gemfile
@@ -1,19 +1,31 @@
-source ENV['GEM_SOURCE'] || 'https://rubygems.org'
+source ENV['GEM_SOURCE'] || "https://rubygems.org"
 
-group :test, :development do
-  gem 'rspec-puppet',            :require => false
+group :development, :unit_tests do
   gem 'rake',                    :require => false
+  gem 'rspec-puppet', '~> 1.0',  :require => false
   gem 'puppetlabs_spec_helper',  :require => false
-  gem 'serverspec',              :require => false
   gem 'puppet-lint',             :require => false
-  gem 'pry',                     :require => false
   gem 'simplecov',               :require => false
-  gem 'beaker',                  :require => false
-  gem 'beaker-rspec',            :require => false
+  gem 'puppet_facts',            :require => false
+  gem 'json',                    :require => false
+  gem 'rspec-core', '3.1.7'
 end
 
-if puppetversion = ENV['PUPPET_VERSION']
+group :system_tests do
+  gem 'beaker-rspec',  :require => false
+  gem 'serverspec',    :require => false
+end
+
+if facterversion = ENV['FACTER_GEM_VERSION']
+  gem 'facter', facterversion, :require => false
+else
+  gem 'facter', :require => false
+end
+
+if puppetversion = ENV['PUPPET_GEM_VERSION']
   gem 'puppet', puppetversion, :require => false
 else
   gem 'puppet', :require => false
 end
+
+# vim:ft=ruby
diff --git a/mongodb/README.md b/mongodb/README.md
@@ -19,9 +19,9 @@ Installs MongoDB on RHEL/Ubuntu/Debian from OS repo, or alternatively from
 
 ### Deprecation Warning ###
 
-This release is a major refactoring of the module which means that the API may
-have changed in backwards incompatible ways. If your project depends on the old API,
-please pin your dependencies to 0.3 version to ensure your environments don't break.
+This module is still in beta which means the API is subject to change in
+backwards incompatible ways. If your project depends on an old API, please pin
+your dependencies to the necessary version to ensure your environments don't break.
 
 The current module design is undergoing review for potential 1.0 release. We welcome
 any feedback with regard to the APIs and patterns used in this release.
@@ -45,6 +45,7 @@ For the 0.6 release, the MongoDB module now supports basic replicaset features
 * MongoDB configuration files.
 * MongoDB service.
 * MongoDB client.
+* MongoDB sharding support (mongos)
 * 10gen/mongodb apt/yum repository.
 
 ###Beginning with MongoDB
@@ -69,6 +70,15 @@ class {'::mongodb::client':}
 Note that for Debian/Ubuntu family systems the client is installed with the 
 server. Using the client class will by default install the server.
 
+If one plans to configure sharding for a Mongo deployment, the module offer
+the `mongos` installation. `mongos` can be installed the following way :
+
+```puppet
+class {'::mongodb::mongos' :
+  configdb => ['configsvr1.example.com:27018'],
+}
+```
+
 Although most distros come with a prepacked MongoDB server we recommend to
 use the 10gen/MongoDB software repository, because most of the current OS
 packages are outdated and not appropriate for a production environment.
@@ -116,6 +126,7 @@ Unsafe plain text password could be used with 'password' parameter instead of 'p
 * `mongodb::server`: Installs and configure MongoDB
 * `mongodb::client`: Installs the MongoDB client shell (for Red Hat family systems)
 * `mongodb::globals`: Configure main settings in a global way
+* `mongodb::mongos`: Installs and configure Mongos server (for sharding support)
 
 ####Private classes
 * `mongodb::repo`: Manage 10gen/MongoDB software repository
@@ -125,6 +136,9 @@ Unsafe plain text password could be used with 'password' parameter instead of 'p
 * `mongodb::server::install`: Install MongoDB software packages
 * `mongodb::server::service`: Manages service
 * `mongodb::client::install`: Installs the MongoDB client software package
+* `mongodb::mongos::config`: Configures Mongos configuration files
+* `mongodb::mongos::install`: Install Mongos software packages
+* `mongodb::mongos::service`: Manages Mongos service
 
 ####Class: mongodb::globals
 *Note:* most server specific defaults should be overridden in the `mongodb::server`
@@ -328,6 +342,12 @@ MMS identifier for mms monitoring. Default: None
 #####`mms_interval`
 MMS interval for mms monitoring. Default: None
 
+#####`configsvr`
+Use this setting to enable config server mode for mongod.
+
+#####`shardsvr`
+Use this setting to enable shard server mode for mongod.
+
 #####`replset`
 Use this setting to configure replication with replica sets. Specify a replica
 set name as an argument to this set. All hosts must have the same set name.
@@ -351,6 +371,10 @@ Default: 100 ms
 Specify the path to a key file to store authentication information. This option
 is only useful for the connection between replica set members. Default: None
 
+#####'key'
+Specify the key contained within the keyfile. This option
+is only useful for the connection between replica set members. Default: None
+
 #####`master`
 Set to true to configure the current instance to act as master instance in a
 replication configuration. Default: False  *Note*: deprecated – use replica sets
@@ -379,6 +403,64 @@ Used with the slave setting to specify the master instance from which
 this slave instance will replicate. Default: <>
 *Note*: deprecated – use replica sets
 
+#####`ssl`
+Set to true to enable ssl. Default: <>
+*Important*: You need to have ssl_key and ssl_ca set as well and files
+need to pre-exist on node.
+
+#####`ssl_key`
+Default: <>
+
+#####`ssl_ca`
+Default: <>
+
+####Class: mongodb::mongos
+class. This class should only be used if you want to implement sharding within
+your mongodb deployment.
+
+This class allows you to configure the mongos daemon (responsible for routing)
+on your platform.
+
+#####`ensure`
+Used to ensure that the package is installed and the service is running, or that the package is absent/purged and the service is stopped. Valid values are true/false/present/absent/purged.
+
+#####`config`
+Path of the config file. If not specified, the module will use the default
+for your OS distro.
+
+#####`config_content`
+Path to the config template if the default doesn't match one needs.
+
+#####`configdb`
+Array of the config servers IP addresses the mongos should connect to.
+
+#####`service_name`
+This setting can be used to override the default Mongos service name. If not
+specified, the module will use whatever service name is the default for your OS distro.
+
+#####`service_provider`
+This setting can be used to override the default Mongos service provider. If
+not specified, the module will use whatever service provider is the default for
+your OS distro.
+
+#####`service_status`
+This setting can be used to override the default status check command for
+your Mongos service. If not specified, the module will use whatever service
+name is the default for your OS distro.
+
+#####`service_enable`
+This setting can be used to specify if the service should be enable at boot
+
+#####`service_ensure`
+This setting can be used to specify if the service should be running
+
+#####`package_ensure`
+This setting can be used to specify if puppet should install the package or not
+
+#####`package_name`
+This setting can be used to specify the name of the package that should be installed.
+If not specified, the module will use whatever service name is the default for your OS distro.
+
 ### Definitions
 
 #### Definition: mongodb:db
@@ -464,6 +546,28 @@ Array of 'host:port' of the replicaset members.
 
 It currently only adds members without options.
 
+#### Provider: mongodb_shard
+'mongodb_shard' can be used to create and manage MongoDB shards.
+*Note:* Removing a shard is not yet supported. Shard can only be added.
+
+```puppet
+mongodb_shard { 'rsmain':
+  member => 'rsmain/host1:27017',
+  keys   => [{'rsmain.foo' => {'name' => 1}}],
+}
+```
+#####`member`
+Member of the shard in the form;
+
+* [hostname]
+* [hostname]:[port]
+* [replica-set-name]/[hostname]
+* [replica-set-name]/[hostname]:port
+
+#####`keys`
+Sharding keys for a specific database. This variable should be an array
+of sharding keys.
+
 ## Limitations
 
 This module has been tested on:
@@ -472,8 +576,8 @@ This module has been tested on:
 * Debian 6.* (squeeze)
 * Ubuntu 12.04.2 (precise)
 * Ubuntu 10.04.4 LTS (lucid)
-* RHEL 5/6
-* CentOS 5/6
+* RHEL 5/6/7
+* CentOS 5/6/7
 
 For a full list of tested operating systems please have a look at the [.nodeset.xml](https://github.com/puppetlabs/puppetlabs-mongodb/blob/master/.nodeset.yml) definition.
 

diff --git a/mongodb/lib/puppet/provider/mongodb.rb b/mongodb/lib/puppet/provider/mongodb.rb
@@ -1,3 +1,4 @@
+require 'yaml'
 class Puppet::Provider::Mongodb < Puppet::Provider
 
   # Without initvars commands won't work.
@@ -17,13 +18,69 @@ def mongorc_file
     self.class.mongorc_file
   end
 
+  def self.get_mongod_conf_file
+    if File.exists? '/etc/mongod.conf'
+      file = '/etc/mongod.conf'
+    else
+      file = '/etc/mongodb.conf'
+    end
+    file
+  end
+
+  def self.get_conn_string
+    file = get_mongod_conf_file
+    # The mongo conf is probably a key-value store, even though 2.6 is
+    # supposed to use YAML, because the config template is applied
+    # based on $::mongodb::globals::version which is the user will not
+    # necessarily set. This attempts to get the port from both types of
+    # config files.
+    config = YAML.load_file(file)
+    if config.kind_of?(Hash) # Using a valid YAML file for mongo 2.6
+      bindip = config['net.bindIp']
+      port = config['net.port']
+      shardsvr = config['sharding.clusterRole']
+      confsvr = config['sharding.clusterRole']
+    else # It has to be a key-value config file
+      config = {}
+      File.readlines(file).collect do |line|
+         k,v = line.split('=')
+         config[k.rstrip] = v.lstrip.chomp if k and v
+      end
+      bindip = config['bind_ip']
+      port = config['port']
+      shardsvr = config['shardsvr']
+      confsvr = config['confsvr']
+    end
+
+    if bindip
+      first_ip_in_list = bindip.split(',').first
+      if first_ip_in_list.eql? '0.0.0.0'
+        ip_real = '127.0.0.1'
+      else
+        ip_real = first_ip_in_list
+      end
+    end
+
+    if port
+      port_real = port
+    elsif !port and (confsvr.eql? 'configsvr' or confsvr.eql? 'true')
+      port_real = 27019
+    elsif !port and (shardsvr.eql? 'shardsvr' or shardsvr.eql? 'true')
+      port_real = 27018
+    else
+      port_real = 27017
+    end
+
+    "#{ip_real}:#{port_real}"
+  end
+
   # Mongo Command Wrapper
   def self.mongo_eval(cmd, db = 'admin')
     if mongorc_file
         cmd = mongorc_file + cmd
     end
 
-    out = mongo([db, '--quiet', '--eval', cmd])
+    out = mongo([db, '--quiet', '--host', get_conn_string, '--eval', cmd])
 
     out.gsub!(/ObjectId\(([^)]*)\)/, '\1')
     out