(BKR-1342) Import ci rake tasks and install utils #26

melissa · 2018-02-02T18:34:05Z

This commit moves all ci acceptance rake tasks and install/setup
utilities out of the puppet repo and into the beaker-puppet repo. This
will allow us to reuse all of this code across multiple components of
puppet-agent. The goal is to reduce the number of places we need to
duplicate code across different code bases. With this change, any future
updates or overhauls will be consistent across all our projects.

melissa · 2018-02-02T18:34:41Z

This is still blocked on a few updated to puppet, so it's not yet ready to be merged

melissa · 2018-02-02T22:07:44Z

This PR is based on changes made in puppetlabs/puppet#6550

melissa · 2018-02-02T22:09:23Z

lib/beaker-puppet/helpers/setup/common.rb

+            return true
+          end
+
+          def ruby_command(host)


I don't really like these here, but I was unsure where to drop them. They are utilities that are only used by tests defined in common.rb, gem.rb, aio.rb, etc. They are not referenced by a component when running tests, as are the rest of the methods in this file. I think it would be good to move these to a different file, but I wasn't sure the best place for that

melissa · 2018-02-02T22:38:53Z

I also want to note that I did not bring along any of the pe-specific utilities, specifically these ones I left out:
https://github.com/puppetlabs/puppet/blob/44e1a85194583bd350c8a79c3c4e0494fc4b144f/acceptance/setup/common/pre-suite/110_SetPEPuppetService.rb#L1

https://github.com/puppetlabs/puppet/blob/44e1a85194583bd350c8a79c3c4e0494fc4b144f/acceptance/setup/pe/pre-suite/000_Install.rb#L1

https://github.com/puppetlabs/puppet/blob/44e1a85194583bd350c8a79c3c4e0494fc4b144f/acceptance/setup/pe/pre-suite/010_UpdatePkg.rb#L1

melissa · 2018-02-05T21:33:35Z

@kevpl I just realized you already have a lot of this code in the acceptance directory of beaker-puppet (https://github.com/puppetlabs/beaker-puppet/tree/master/acceptance). Was that intended to be used by projects like puppet and hiera to run acceptance tests?

melissa · 2018-02-06T19:19:35Z

lib/beaker-puppet/helpers/setup/aio.rb

+          def install_cumulus_modules
+            test_name "Install Cumulus Modules" do
+              platforms = hosts.map{|val| val[:platform]}
+              skip_test "No cumulus hosts present" unless platforms.any? { |val| /cumulus/ =~ val }


@joshcooper it looks like this skip is causing the top level test (https://github.com/puppetlabs/puppet/pull/6602/files#diff-62e97df027f65c15385cb6a34498e2acR1) to skip, unless I'm missing something? Is this something that happens with nested tests?

Install Cumulus Modules No cumulus hosts present Begin teardown End teardown Warning: setup/aio.rb skipped in 91.94 seconds Test Suite: pre_suite @ 2018-02-06 11:10:42 -0800 - Host Configuration Summary - - Test Case Summary for suite 'pre_suite' - Total Suite Time: 91.94 seconds Average Test Time: 91.94 seconds Attempted: 1 Passed: 0 Failed: 0 Errored: 0 Skipped: 1 Pending: 0 Total: 1 - Specific Test Case Status - Failed Tests Cases: Errored Tests Cases: Skipped Tests Cases: Test Case setup/aio.rb Pending Tests Cases: No tests to run for suite 'tests' No tests to run for suite 'post_suite' No tests to run for suite 'pre_cleanup' Cleanup: cleaning up after successful run

alright, I got this figured out

kevpl · 2018-02-08T16:55:35Z

@melissa to this question:

@kevpl I just realized you already have a lot of this code in
the acceptance directory of beaker-puppet
(https://github.com/puppetlabs/beaker-puppet/tree/master/acceptance).
Was that intended to be used by projects like puppet and hiera to
run acceptance tests?

No, it wasn't, it was put there just to help beaker-puppet do its own acceptance testing. However, we'd much rather have a "true" (meaning not in beaker-puppet's acceptance helpers) DSL method that both you & we can use in testing. So my recommendation would be that we switch beaker-puppet's testing to use your versions of the methods & delete beaker-puppet-acceptance's copy of them.

I understand that this is additional work for what is already potentially a big glob of work, so I don't think this needs to necessarily be done before this PR is merged. If you'd like, we can ticket this work for BKR & hopefully be able to schedule this soon with a goalie. @melissa would you like to create the ticket since your head is in this right now, or should I do that?

melissa · 2018-02-08T20:34:38Z

@kevpl created https://tickets.puppetlabs.com/browse/BKR-1424 for that work. Thank you!

Also, this is blocked because of a bug we found where beaker subcommands don't work well when passing in additional options. BKR-1422 was created to cover this particular bug.

melissa · 2018-02-16T00:57:45Z

tasks/ci.rake

+  sh('beaker', command.to_s, *argv)
+end
+
+def beaker_suite(type)


@joshcooper this is what I have so far. Not necessarily a final version, but at least it's proof this is working!

joshcooper · 2018-02-23T06:12:34Z

lib/beaker-puppet/helpers/rake_helpers.rb

+      # Methods that help you interact with rake during ci setup
+      module RakeHelpers
+        class << self
+          def load_tasks(beaker_root = '/Users/melissa/beaker-puppet')


can we make it a required parameter?

oh shoot, I meant to fix this, but I totally forgot. Thank you! I don't think we need to make it required, since we should be able to introspect where beaker-puppet is being run. I do that for the setup tests

melissa · 2018-02-23T18:05:11Z

lib/beaker-puppet/helpers/rake_helpers.rb

+      # Methods that help you interact with rake during ci setup
+      module RakeHelpers
+        class << self
+          def load_tasks(beaker_root = File.dirname(__dir__))


@joshcooper fixed!

melissa · 2018-03-12T21:55:25Z

I tested aio, gem, and git. Aio and gem both succeeded. Git failed, but in an expected way and should not be a blocker.

kevpl

@joshcooper any concerns you'd like to raise, or can I proceed with merging & releasing these changes?

joshcooper · 2018-03-15T23:09:27Z

tasks/ci.rake

+    task :quick => ['ci:check_env', 'ci:gen_hosts'] do
+      ENV['TESTS'] = get_test_sample.join(",")
+      Rake::Task["ci:test:aio"].invoke
+    end


The ci:test:quick task is specific to the puppet component, e.g. the get_test_sample method returns tests that only live in puppet. How about keeping this task (and related helper methods) in puppet's Rakefile?

This commit moves all ci acceptance rake tasks and install/setup utilities out of the puppet repo and into the beaker-puppet repo. This will allow us to reuse all of this code across multiple components of puppet-agent. The goal is to reduce the number of places we need to duplicate code across different code bases. With this change, any future updates or overhauls will be consistent across all our projects.

melissa · 2018-03-16T17:27:52Z

@joshcooper I removed the ci:test:quick task and associated utilities

@ericldelaney did you want me to add a check for if --preserve-hosts=always is present in OPTIONS and skip beaker destroy if it is?

ericldelaney · 2018-03-16T17:41:36Z

@melissa it would be nice if at some point we honored --preserve-hosts correctly. It would save me from having to create a failing test to run against so that it leaves the machines. (shrug)

joshcooper

Let's file a ticket about adding back preserve hosts.

melissa · 2018-03-16T18:15:17Z

ticket filed https://tickets.puppetlabs.com/browse/BKR-1443 feel free to add more detail

melissa added enhancement help wanted question blocked labels Feb 2, 2018

melissa requested a review from joshcooper February 2, 2018 18:34

melissa force-pushed the ticket/master/BKR-1342-puppet-acceptance-utilities branch from 7d7fbe3 to 7a51054 Compare February 2, 2018 22:03

melissa removed blocked question labels Feb 2, 2018

melissa changed the title ~~[WIP] (BKR-1342) Import ci rake tasks and install utils~~ (BKR-1342) Import ci rake tasks and install utils Feb 2, 2018

melissa requested review from tvpartytonight and kevpl February 2, 2018 22:07

melissa commented Feb 2, 2018

View reviewed changes

melissa mentioned this pull request Feb 2, 2018

(PUP-8416) Export ci rake tasks and install utiles puppetlabs/puppet#6602

Merged

melissa force-pushed the ticket/master/BKR-1342-puppet-acceptance-utilities branch from 7a51054 to e4202af Compare February 2, 2018 22:34

melissa commented Feb 6, 2018

View reviewed changes

melissa force-pushed the ticket/master/BKR-1342-puppet-acceptance-utilities branch from e4202af to dd7dc91 Compare February 7, 2018 01:31

melissa removed enhancement help wanted labels Feb 7, 2018

melissa added the blocked label Feb 8, 2018

melissa commented Feb 16, 2018

View reviewed changes

melissa force-pushed the ticket/master/BKR-1342-puppet-acceptance-utilities branch from 1878517 to 3e383f0 Compare February 16, 2018 01:09

joshcooper reviewed Feb 23, 2018

View reviewed changes

melissa force-pushed the ticket/master/BKR-1342-puppet-acceptance-utilities branch from 3e383f0 to 182e1c3 Compare February 23, 2018 18:04

melissa commented Feb 23, 2018

View reviewed changes

melissa force-pushed the ticket/master/BKR-1342-puppet-acceptance-utilities branch from 182e1c3 to 429a9aa Compare March 12, 2018 21:50

melissa removed the blocked label Mar 12, 2018

kevpl approved these changes Mar 15, 2018

View reviewed changes

joshcooper reviewed Mar 15, 2018

View reviewed changes

melissa force-pushed the ticket/master/BKR-1342-puppet-acceptance-utilities branch from 429a9aa to eb2a9a5 Compare March 16, 2018 17:25

joshcooper approved these changes Mar 16, 2018

View reviewed changes

joshcooper merged commit adf1547 into puppetlabs:master Mar 16, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

(BKR-1342) Import ci rake tasks and install utils #26

(BKR-1342) Import ci rake tasks and install utils #26

melissa commented Feb 2, 2018 •

edited

Loading

melissa commented Feb 2, 2018

melissa commented Feb 2, 2018

melissa Feb 2, 2018

melissa commented Feb 2, 2018

melissa commented Feb 5, 2018

melissa Feb 6, 2018

melissa Feb 7, 2018

kevpl commented Feb 8, 2018

melissa commented Feb 8, 2018

melissa Feb 16, 2018

joshcooper Feb 23, 2018

melissa Feb 23, 2018

melissa Feb 23, 2018

melissa commented Mar 12, 2018

kevpl left a comment

joshcooper Mar 15, 2018

melissa commented Mar 16, 2018

ericldelaney commented Mar 16, 2018

joshcooper left a comment

melissa commented Mar 16, 2018

(BKR-1342) Import ci rake tasks and install utils #26

(BKR-1342) Import ci rake tasks and install utils #26

Conversation

melissa commented Feb 2, 2018 • edited Loading

melissa commented Feb 2, 2018

melissa commented Feb 2, 2018

Choose a reason for hiding this comment

melissa commented Feb 2, 2018

melissa commented Feb 5, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kevpl commented Feb 8, 2018

melissa commented Feb 8, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

melissa commented Mar 12, 2018

kevpl left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

melissa commented Mar 16, 2018

ericldelaney commented Mar 16, 2018

joshcooper left a comment

Choose a reason for hiding this comment

melissa commented Mar 16, 2018

melissa commented Feb 2, 2018 •

edited

Loading