(PDK-381) Ensure spec fixtures are cleaned up, even if the test fails

[rodjek]

Currently if the rspec tests fail, the spec_clean task is not invoked as rake aborts. By moving the spec_clean task into an ensure block, it will get run and clean things up even when the rspec tests fail.

[puppetcla]

CLA signed by all contributors.

[ekohl]

I've noticed this and relied on this behaviour. It bit me a few times, for example when I was developing in the airport on slow wifi and accidentally ran (out of custom) rake spec rather than rspec on the failing test.

Is it possible to revert this or make it configurable?

[DavidS]

The previous behaviour can be achieved by running rake spec_prep spec_standalone. Admittedly, that won't help you, if you inadvertently run rake spec, or do anything else that poisons your fixtures.

[ekohl]

@DavidS you're right that I can achieve it, but as a user I was surprised that in a patch release this changed.

[ekohl]

Also I realise this is a bit of https://xkcd.com/1172/

[DavidS]

Thanks for your understanding, and apologies for breaking your workflow.

[sean797]

ugh, this broke some of my workflows too. #211 should fix mine, not sure if it will also fix your problem @ekohl

[ekohl]

Another side-effect is that when you use parallel_spec and cancel tests (ctrl+c) that it starts cleaning up fixtures while some tests are still running. This leads to unexpected failures like:

       Evaluation Error: Error while evaluating a Resource Statement, Evaluation Error: Error while evaluating a Function Call, Could not find class ::foreman::params for donald.ens.kohlvanwijngaarden.nl at /home/ekohl/dev/puppet-foreman_proxy/spec/fixtures/modules/foreman/manifests/repos.pp:6:3 at /home/ekohl/dev/puppet-foreman_proxy/spec/fixtures/modules/foreman_proxy/manifests/install.pp:4 on HOSTNAME
       Could not find parent resource type '::dhcp::params' of type hostclass in rp_env at /home/ekohl/dev/puppet-foreman_proxy/spec/fixtures/modules/dhcp/manifests/init.pp:1 on node HOSTNAME
       Evaluation Error: Error while evaluating a Function Call, Could not autoload puppet/parser/functions/cache_data: No such file or directory @ realpath_rec - /home/ekohl/dev/puppet-foreman_proxy/spec/fixtures/modules/extlib/lib/puppet/parser/functions/cache_data.rb at /home/ekohl/dev/puppet-foreman_proxy/spec/fixtures/modules/foreman_proxy/manifests/params.pp:349:28 on node HOSTNAME

You don't see these when you let the tests continue to run. This is a system with 8 CPUs.

[DavidS]

@ekohl that sounds like an issue with the parallel_tests library rather than our usage.

[ekohl]

@DavidS possibly, but in the end I'm using the spec helper and this is the experience it provides me.

[DavidS]

grosser/parallel_tests#523

[ekohl]

Thanks, that does explain it.