Introduce @BeforeSuite and @AfterSuite annotations

[ondrejlerch]

I suggest that @BeforeSuite and @AfterSuite annotations should be added to JUnit 5.

Obviously they should execute before and after the whole suite.

Or is there some other mechanism to accomplish that?

I suggest that there should not be static restriction, i.e. also non-static methods could be annotated with @BeforeSuite and @AfterSuite.

I personally prefer non-static as we use Spring in our tests and want to use injected beans in @BeforeSuite method.

[smoyer64]

Related to #163.

There's not always an easy way to control what's considered a "Suite". If you look at the Maven plugin, it controls the inclusion/exclusion of tests via XML. The JUnitPlatform test runner allows a suite to be defined declaratively but it's not honored by the Maven plugin.

If there was always a declarative way to run a suite, I'd say that @BeforeAll/@AfterAll could just be reused on the suite declaration. As things stand and in the name of unambiguous behavior, I prefer your annotations.

[ttddyy]

Related to #19
(e.g.: @BeforeAllTests/@AfterAllTests)

[ondrejlerch]

In my consideration "Suite" can be defined via:

• Maven/Gradle plugin inclusion/exclusion
• JUnitPlatform declarative declaration
• IDE when I click Run on the whole package of tests

In all those cases I would expect that @BeforeSuite/@AfterSuite would be executed just once before/after the whole bunch of tests, for example to perform time consuming operation such as database cleanup.

[sebersole]

I agree with everything @ondrejlerch brings up

[marcphilipp]

@ondrejlerch How would JUnit find @BeforeSuite/@AfterSuite annotated methods? For example, when you run a test class from an IDE would it only look at the test class or someplace else?

The Store abstraction along with the root ExtensionContext provides a means to prepare something once and share it between multiple test classes. However, there's no way to clean it up at the moment (that depends on #742).

[sebersole]

@marcphilipp you could consider a solution such as Jandex, which provides you an index over all annotations

[sormuras]

https://github.com/wildfly/jandex ?

[marcphilipp]

Well, in any case, it has to be obvious for test authors where JUnit will actually look for these annotations, i.e. if they will be applied when they execute a test class.

[sormuras]

environment (shell/build tool/IDE/...)  (1)
  platform/launcher (DefaultLauncher)   (2)
    engine a (Jupiter)                 (3a)
    ...
    engine x                           (3x)
    engine z                           (3z)

I see an easy way to implement an extension point for 3a -- which frames the all tests executed by the Jupiter engine with a "before the first" and "after the last" callback.

Pseudo-code for JupiterTestEngine:

@Override
public final void execute(ExecutionRequest request) {
	// TODO context = createExecutionContext(request);
	// try {
	// TODO callBeforeExecutionCallbacks(context);
	new HierarchicalTestExecutor<>(request, context).execute();
	// } finally {
	// TODO callAfterExecutionCallbacks(context);
	// }
}

[sebersole]

For the record, I actually would not use the annotation specifically. I am more interested in the corresponding Extension (especially AfterSuiteCallback).

I was just answering the general question of "finding annotations". That is what Jandex excels at, whether you want to use it or not...

[marcphilipp]

I’m fine with adding an extension point that uses the existing registration mechanisms.

That should work for everything except Maven Surefire... see #1113.

[sebersole]

As I would never use Maven ever again, I can live with that ;)

But yes, that is exactly what would work best for us:

@ExtendWith( MyAfterSuiteCallback.class )
...

[sormuras]

Started working on a PR that introduces BeforeExecutionCallback and AfterExecutionCallback...

[ondrejlerch]

@marcphilipp
To your question: How would JUnit find @BeforeSuite/@AfterSuite annotated methods? For example, when you run a test class from an IDE would it only look at the test class or someplace else?
I suggest answer: JUnit should search among all executed test classes, not someplace else. So if you execute just one test class (via IDE or Maven or Gradle) it should look only there. However, if you execute multiple test classes it should look at all of them and and run @BeforeSuite first.

If we use JUnit 5 with Spring then Spring Context should be initialized first, then @BeforeSuite should be executed, so that we can use Spring in @BeforeSuite. Obviously @AfterSuite should be executed before Spring Context is closed for the same reason.

BeforeExecutionCallback and AfterExecutionCallback sound also good.

Solution should work with IDE, Maven and Gradle as those are most typical tools for test execution. I agree with @sebersole that Maven is not cutting edge, however, if Maven Surefire would not work then please suggest what would work in Maven as it is widely used.

Thanks!

[sebersole]

@sormuras Thanks so much for starting on this.

[sbrannen]

I'm also fine with the idea of introducing new BeforeExecutionCallback and AfterExecutionCallback extension APIs.

But... we'll have to work on the names. 😉

That and the implementation and semantics of course.

[marcphilipp]

Team decision: Investigate the feasibility of adding @BeforeSuite and @AfterSuite support in @Suite classes.

[YomnaZidan]

Hi @marcphilipp ,

Please feedback what is the alternative of BeforeSuite and AfterSuite , in order to create 1 consolidated Extend Report across all Test Classes .
Same Issue Described by @praveenreddynarala

Regards ,
Yomna Zidan

[kishkk84]

@YomnaZidan @praveenreddynarala I am facing same issue as you both with extent report generation. Did you guys find any alternatives, if so can you please share. That would be a great help. Thanks 🙏

[Furrial]

@kishkk84 @YomnaZidan you can take a look at my solution implemented based on workaround using ClosableResource (see 456 comment)

[difflaender]

We have the same requirement. In JUnit 4, we were using an ExternalResource in a class that was running with a custom Suite implementation in order to set up a database structure for all tests that were going to be executed in the suite and to clean up the database afterwards.
Knowing which classes are going to be executed by the Suite when these BeforeSuite and AfterSuite hooks are called would be pretty important to us, since we ask each of the executed test classes about what database contents the test requires. This allows us to set up the database just once in the beginning, instead of individually for each test, which results in a lot of saved time if multiple test classes have the same database requirements.

[nagkumar]

Related to #163.

There's not always an easy way to control what's considered a "Suite". If you look at the Maven plugin, it controls the inclusion/exclusion of tests via XML. The JUnitPlatform test runner allows a suite to be defined declaratively but it's not honored by the Maven plugin.

If there was always a declarative way to run a suite, I'd say that @BeforeAll/@AfterAll could just be reused on the suite declaration. As things stand and in the name of unambiguous behavior, I prefer your annotations.

Can we not follow what the definition of suite is as per TestNG.. https://testng.org/doc/documentation-main.html

@BeforeSuite
@AfterSuite

[MaudeVanhengel]

@marcphilipp Is there any update to this?
I'm using Junit 5 in combo with Cucumber and Spring Boot.
My Suite class looks like this:

@Suite
@SelectPackages("features")
@ConfigurationParameter(key = GLUE_PROPERTY_NAME, value = "stepdefs")
public class FrontofficeSuiteRunner {

}

I've tried a solution of creating a custom extension class with BeforeAllCallback and then extending my suite with it, but no success.
I'm finding no workaround that works for my use case.

The Suite runs all feature files in my features package and I need the BeforeAll method to do database cleanup before running the features..

[RainerGanss]

@MaudeVanhengel What we endet up doing is a

• spring bean
• with @PostConstruct and @PreDestroy methods for the BeforeSuite- and AfterSuite code
• has a test profile

We add the profile to the test execution and write our "BeforeAllCallback"-Code in the annoated methods. You could also use the InitializingBean or listen for spring application events (e.g. ContextStartedEvent) and execute code once the context is fully started.

The problem with the Callbacks is, that you don't have access to the spring application context to for example initialize the database using spring repositories.

WARNING:
This solutions does not exactly the same as the here proposed BeforeSuite and AfterSuite annotations. The code in the bean is executed every time, the application context gets created/destroyed. So if for example a test method is annotated with springs @DirtiesContext annotation, the code will be executed again. In our case, that is actually exactly what we want. But it might not be the correct thing in your case. Also mocking the context with @MockBean will destroy/re-create the context. So use this with caution.