Introduce an extension point to allow test container creation

[smoyer64]

Overview

• (X) Feature request.

When implementing a JUnit 5 extension to provide @Suite, @BeforeSuite and @AfterSuite, I had to create a new test engine that recreates a lot of the discovery mechanisms already provided by the Jupiter test engine. Ultimately the test execution is delegated to a specified engine, so all this engine really does is alter the way discovery works.

There are currently no Extension points in the Jupiter API that interact with the discovery process - they're all focused on the contextual execution of the already discovered hierarchy. I'd like to propose the following new Jupiter API types (but I'd be happy with the equivalent functionality in any form):

• @TestContainer
• ContainerDescriptorFactory

I'm picturing the @TestContainer annotation being used like this:

@TestContainer(MyContainerDescriptorFactory)

And the @Suite example I used as the basis for this request becomes a composed annotation like this:

@Retention( ... )
@Target( ... )
@TestContainer(SuiteFactory.class)
public @interface Suite

The TestDescriptorFactory would be a functional interface with the following method:

    void addTestContainer(TestDescriptor parent);

When called, this method would add one or more children to the parent TestDescriptor. There are a couple of beautiful (in my opinion) features of this design:

1. A child TestDesciptor created by the factory can itself have an arbitrary test hierarchy, perhaps even by invoking other @TestContainers to help build the structure.
2. At an arbitrary depth, @BeforeSuite/@AfterSuite become meaningless. Every @TestContainer can simply execute @BeforeAll/@AfterAll callbacks at their own level in the test hierarchy and the set-up and tear-down is automatically nested correctly.
3. It's not as inherently dangerous as the DiscoveryCallback described in RFC: Please critique this working PoC for an AfterDiscoveryCallback. #354 and implemented in Adds DiscoveryCallbacks #577.
4. Building this into the existing HierarchicalTestEngine is trivial as it simply delegates the addition of child containers (and their contents) to the factory.

This also leaves me with a few questions:

1. What happens in non-hierarchical test engines?
2. How do engines that execute test concurrently (Surefire's default) make sure they don't multi-thread a hierarchy?
3. Should @TestContainer be marked @Testable? I can argue both ways but I can see IDEs that inspect the source getting confused.

Deliverables (this is a long-shot at being correct on the first pass)

• Add the @TestContainer annotation
• Create the ContainerDescriptorFactory
• Move the TestDescriptor interface to the junit-jupiter-api project

Also relates to #19.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. Given the limited bandwidth of the team, it will be automatically closed if no further activity occurs. Thank you for your contribution.

[stale]

This issue has been automatically closed due to inactivity. If you have a good use case for this feature, please feel free to reopen the issue.