before & after hooks for a fixture

[vutenkov]

Are you requesting a feature or reporting a bug?

Requesting a feature.

What is the current behavior?

There is no ability to run some setup/tear down code before/after all tests in a fixture.

What is the expected behavior?

To have such the ability.

Provide the test code and the tested page URL (if applicable)

Test code

fixture('My cool fixture')
    .before(() => {
        // do something that needs to be done once _before_ all tests
    })
    .after(() => {
        // do something that needs to be done once _after_ all tests
    })

[inikulin]

I was hoping to implement this feature in current sprint, however once I've got my hands on it, it appeared that it's not as simple as it seems. Most problems are related to consistency with fixture.beforeEach, test.after, etc. hooks. Those hooks are run in each browser for each test run for the test, also they have a test controller object t provided. It seems that it's not a case for fixture.before and fixture.after hooks. I can't figure out a scenario there we'll need to run it on page. While fixture.beforeEach hooks are usable e.g. to perform authentication, for fixture.before it doesn't makes sense, since test state is not shared between tests. More likely fixture.before will be sued to deal with database or website itself and should be run before any test page is loaded, so made action will take effect on loaded page. Also, unlike e.g. test.before such hook should be run only once. With all those things in mind, I propose the following:

• Call new hooks fixture.setup, fixture.teardown, test.setup, test.teardown to make them distinguishable from after... and before.. hooks.

• Those hooks will be run once for each test or fixture, and in case of setup hooks, before tested page is loaded for the first time.

• Each new hook will accept ctx object that can be used to share data between hooks. ctx object passed to before... and after... hooks will have setup ctx as a prototype, so you'll be able to access and modify properties created by setup hook in tests, but properties added in test will not be seen e.g. by teardown hook.

\cc @DevExpress/testcafe @vladimirutenkov

[helen-dikareva]

Setup and teardown will executed once for fixture/test and before.../after... will executed each time in quarantine mode , is that right?

[inikulin]

@helen-dikareva If we have a test that run in 3 browsers, before/after will be executed 3 times, once per each browser. setup/teardown will be executed only once unrelated to the number of browsers specified for the task.

[helen-dikareva]

ok, I see

[kirovboris]

The teardown handler will be run after a fixture or test passed/failed in all browsers, right?
+1 in this case. In my opinion it's still not clear, that it will be run once, e.g. if I write test.setup first time, i'll expect it will execute several times, because I'll met the famous term setup and will be think, that it's occur for every test run, in other words for each browser. If I met term e.g. setupOnce instead of setup, I'd would tried to know what does it do.

[inikulin]

@kirovboris How about fixture.beforeAll and fixture.afterAll? But test.setup/teardown doesn't fit in this paradigm then.

[kirovboris]

From these two options (beforeAll/afterAll and setup/teardown) I prefer the second one. We already have beforeEach ...test, so beforeAll will be associated with tests for me as well, not browsers.

[inikulin]

We already have beforeEach ...test, so beforeAll will be associated with tests for me as well, not browsers.

Indeed, it has nothing to do with browsers, so it will be correct association

[inikulin]

TODO

• Basic functionality
• Skipped tests
• Shared context object

[lock]

This thread has been automatically locked since it is closed and there has not been any recent activity. Please open a new issue for related bugs or feature requests. We recommend you ask TestCafe API, usage and configuration inquiries on StackOverflow.