update goss verify.yml to allow to pass host to run on #1646

loomsen · 2019-01-02T17:10:09Z

This PR allows for the goss verifier to run different tests on certain hosts.
Example: have a side-effect playbook, which makes keepalived move an IP to a different host in an active-active scenario. The verifier should verify that the IP actually moved to a different host, and that the side-effect playbook killed the supervised process.

Basically what you can do with testinfra by specifying the host in .get_hosts

testinfra_hosts = testinfra.utils.ansible_runner.AnsibleRunner(
    os.environ['MOLECULE_INVENTORY_FILE'])\
    .get_hosts('instance1')

You can do this for goss by prepending the test file with test_host_$hostname_as_given_in_molecule.yml.

PR Type

Feature Pull Request

decentral1se

Thanks for the PR! I'm finding it hard to understand your use case. Does this change cover a general case? Can you explain further? Are there any tests you can provide here?

.../verifier/goss/{{cookiecutter.molecule_directory}}/{{cookiecutter.scenario_name}}/verify.yml

test/scenarios/verifier/molecule/goss/verify.yml

loomsen · 2019-01-16T10:35:57Z

Hi, thank you for reviewing my PR :)

I set up a stripped down, real world example of the roles we actually use in production. It's dumbed down to a default vhost returning 502, but it shows the purpose of this PR.

https://github.com/loomsen/ansible-nginx-ha-cluster-pr-example

Another scenario would be a master-slave databse setup, where you want to setup the replication with ansible, and have a side effect playbook run to make sure the failover is working.
Without this PR, there is no way to test that, because there is only one common goss test file for all hosts.

I hope this enlightens the dark. Does it?

Signed-off-by: Norbert Varzariu <[email protected]>

loomsen · 2019-01-16T15:27:51Z

OK I changed ignore_errors: true to failed_when: false, which seems a little prettier.
Pls review.

loomsen · 2019-01-16T18:54:08Z

Politely asking travis for a rebuild, the failure seems kind of random to me.

loomsen · 2019-01-16T23:26:18Z

Yeah, those seem like random failures, now it failed at a different step.

decentral1se · 2019-01-22T13:36:08Z

Sorry that you're PR is sitting so long here @loomsen. Thanks for the further example.

I've given some more time on this. If I understand it correctly, this PR solves two issues:

Fixes the SHA for the goss binary as reported in Bad Goss sha256sum in generated verify.yml #1618.
Allows to specify hosts which goss will run on (via test_host_MYHOST.yml files).

So yes, this definitely feels like something we need then. This PR is missing documentation for this change that would definitely make it more visible for other users. Also, can me perhaps use the more readable find module to replace the shell: ... pattern matching?

And finally, how to test this? It will be hard to have any sense of sureness without that. If there isn't something already available to help you write tests, please let us know and we better get to making that happen too!

.../verifier/goss/{{cookiecutter.molecule_directory}}/{{cookiecutter.scenario_name}}/verify.yml

Signed-off-by: Norbert Varzariu <[email protected]>

loomsen · 2019-01-23T19:26:23Z

depends on #1695

loomsen · 2019-01-23T19:43:53Z

Only linter fails, which is addressed in #1695

decentral1se

This is looking great now @loomsen!

.../verifier/goss/{{cookiecutter.molecule_directory}}/{{cookiecutter.scenario_name}}/verify.yml

Signed-off-by: Norbert Varzariu <[email protected]>

loomsen · 2019-01-25T09:52:33Z

I don't know why the builds fail.
https://travis-ci.com/loomsen/molecule/builds/98478819
That's the build from my fork, which is fine (except the lint errors).

After fetching latest master into my develop branch, some jobs also start failing.

https://travis-ci.com/loomsen/molecule/builds/98581045

Hence I think the failures are unrelated to my changes.
What do you guys think?

decentral1se · 2019-01-28T12:13:10Z

Added to agenda in ansible/community#427. Please join if you have time!

decentral1se · 2019-02-21T14:05:19Z

Thanks for your patience @loomsen. Our CI is back in shape. It reported:

https://travis-ci.com/ansible/molecule/jobs/179293976

Warning, treated as error:
/home/travis/build/ansible/molecule/molecule/verifier/goss.py:docstring of molecule.verifier.goss.Goss:51:Error in "code-block" directive:
maximum 1 argument(s) allowed, 18 supplied.
.. code-block::
    test_default.yml (will run on all hosts)
    test_host_instance1.yml (will run only on instance1)
    test_host_instance2.yml (will run only on instance2)
ERROR: InvocationError for command '/home/travis/build/ansible/molecule/.tox/doc/bin/python setup.py build_sphinx -n -W --builder=html' (exited with code 1)

decentral1se · 2019-03-11T17:19:15Z

Just missing the --signoff DCO for the commit 😰

loomsen · 2019-03-11T17:21:44Z

Gah, every single time.

- add trailing dot to goss bin shasum to follow current style Signed-off-by: Norbert Varzariu <[email protected]>

Signed-off-by: Norbert Varzariu <[email protected]>

loomsen · 2019-03-12T16:54:39Z

Woohoo, finally :)

decentral1se · 2019-04-01T11:25:13Z

Oh no .... we still didn't merge this ....

@loomsen, can you resolve the final merge conflicts? (sorry 🙏).

We're waiting on one final issue wrt. packaging and then we can merge this (for real).

😅

loomsen · 2019-04-09T17:16:53Z

OK, so I pulled in the latest changes. Code style check fails, which shouldn't be related to this PR, and also a test randomly fails at destroy stage, maybe one of you could re-run just this build, if possible?

@decentral1se thanks for bearing with me :)

decentral1se · 2019-04-09T20:59:28Z

No, thank you!!!

decentral1se · 2019-04-09T21:00:25Z

It appears to be related to #1911!

I'm re-running the CI and will merge if it passes 👍

decentral1se · 2019-04-09T21:31:21Z

loomsen force-pushed the master branch 2 times, most recently from b456261 to cd49513 Compare January 2, 2019 18:11

loomsen closed this Jan 3, 2019

loomsen reopened this Jan 3, 2019

decentral1se reviewed Jan 15, 2019

View reviewed changes

decentral1se mentioned this pull request Jan 15, 2019

Bad Goss sha256sum in generated verify.yml #1618

Closed

loomsen pushed a commit to loomsen/ansible-nginx-ha-cluster-pr-example that referenced this pull request Jan 16, 2019

create example repo for ansible/molecule#1646

41f2b4b

Norbert Varzariu added 6 commits January 16, 2019 16:26

update goss verify.yml to allow to pass host to run on

313db82

Signed-off-by: Norbert Varzariu <[email protected]>

update goss version

7f79d7f

Signed-off-by: Norbert Varzariu <[email protected]>

fix sha sum for goss binary

da642c3

Signed-off-by: Norbert Varzariu <[email protected]>

fix test verify.yml

30629e0

Signed-off-by: Norbert Varzariu <[email protected]>

do not require root for local find

3e2c779

Signed-off-by: Norbert Varzariu <[email protected]>

use failed_when instead of ignore_errors

b73ff42

Signed-off-by: Norbert Varzariu <[email protected]>

loomsen force-pushed the master branch from de1d6ef to b73ff42 Compare January 16, 2019 15:26

loomsen closed this Jan 16, 2019

loomsen reopened this Jan 16, 2019

loomsen closed this Jan 16, 2019

loomsen reopened this Jan 16, 2019

loomsen closed this Jan 18, 2019

loomsen reopened this Jan 18, 2019

Merge remote-tracking branch 'upstream/master'

7222254

decentral1se reviewed Jan 22, 2019

View reviewed changes

.../verifier/goss/{{cookiecutter.molecule_directory}}/{{cookiecutter.scenario_name}}/verify.yml Outdated Show resolved Hide resolved

Norbert Varzariu added 2 commits January 23, 2019 17:59

Merge remote-tracking branch 'upstream/master'

e476c2f

add docs, fix travis tests, add test scenario to test our changes

183506f

Signed-off-by: Norbert Varzariu <[email protected]>

loomsen force-pushed the master branch from d1576e7 to 183506f Compare January 23, 2019 17:34

themr0c approved these changes Jan 24, 2019

View reviewed changes

decentral1se approved these changes Jan 24, 2019

View reviewed changes

webknjaz reviewed Jan 24, 2019

View reviewed changes

.../verifier/goss/{{cookiecutter.molecule_directory}}/{{cookiecutter.scenario_name}}/verify.yml Show resolved Hide resolved

remove trailing dot from goss sha256sum

b3ae53e

Signed-off-by: Norbert Varzariu <[email protected]>

decentral1se mentioned this pull request Jan 28, 2019

Molecule Working Group Agenda ansible/community#427

Closed

themr0c added the enhancement label Jan 31, 2019

themr0c added this to the v.2.21 milestone Jan 31, 2019

decentral1se modified the milestone: v.2.21 Jan 31, 2019

decentral1se removed this from the v.2.21 milestone Feb 27, 2019

Merge remote-tracking branch 'upstream/master'

a204917

- fix documentation

d8b695b

- add trailing dot to goss bin shasum to follow current style Signed-off-by: Norbert Varzariu <[email protected]>

loomsen force-pushed the master branch from 445baf7 to d8b695b Compare March 11, 2019 17:22

Norbert Varzariu added 2 commits March 11, 2019 21:05

fix doc (add missing empty line to code block)

068f013

Signed-off-by: Norbert Varzariu <[email protected]>

make linter happy

675730f

Signed-off-by: Norbert Varzariu <[email protected]>

Merge remote-tracking branch 'upstream/master' into merge_upstream

433a31d

decentral1se merged commit 9ffa4ea into ansible:master Apr 9, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

update goss verify.yml to allow to pass host to run on #1646

update goss verify.yml to allow to pass host to run on #1646

loomsen commented Jan 2, 2019 •

edited

Loading

decentral1se left a comment

loomsen commented Jan 16, 2019

loomsen commented Jan 16, 2019

loomsen commented Jan 16, 2019

loomsen commented Jan 16, 2019 •

edited

Loading

decentral1se commented Jan 22, 2019

loomsen commented Jan 23, 2019

loomsen commented Jan 23, 2019

decentral1se left a comment

loomsen commented Jan 25, 2019

decentral1se commented Jan 28, 2019

decentral1se commented Feb 21, 2019

decentral1se commented Mar 11, 2019

loomsen commented Mar 11, 2019

loomsen commented Mar 12, 2019

decentral1se commented Apr 1, 2019

loomsen commented Apr 9, 2019

decentral1se commented Apr 9, 2019

decentral1se commented Apr 9, 2019 •

edited

Loading

decentral1se commented Apr 9, 2019

update goss verify.yml to allow to pass host to run on #1646

update goss verify.yml to allow to pass host to run on #1646

Conversation

loomsen commented Jan 2, 2019 • edited Loading

PR Type

decentral1se left a comment

Choose a reason for hiding this comment

loomsen commented Jan 16, 2019

loomsen commented Jan 16, 2019

loomsen commented Jan 16, 2019

loomsen commented Jan 16, 2019 • edited Loading

decentral1se commented Jan 22, 2019

loomsen commented Jan 23, 2019

loomsen commented Jan 23, 2019

decentral1se left a comment

Choose a reason for hiding this comment

loomsen commented Jan 25, 2019

decentral1se commented Jan 28, 2019

decentral1se commented Feb 21, 2019

decentral1se commented Mar 11, 2019

loomsen commented Mar 11, 2019

loomsen commented Mar 12, 2019

decentral1se commented Apr 1, 2019

loomsen commented Apr 9, 2019

decentral1se commented Apr 9, 2019

decentral1se commented Apr 9, 2019 • edited Loading

decentral1se commented Apr 9, 2019

loomsen commented Jan 2, 2019 •

edited

Loading

loomsen commented Jan 16, 2019 •

edited

Loading

decentral1se commented Apr 9, 2019 •

edited

Loading