Introduce extension API for executing test in user-defined thread #157
Comments
|
Your first point falls into the broader category of dynamic test creation As for test execution in a different thread, we haven't discussed that yet
2016-02-15 14:23 GMT+01:00 Jens Schauder [email protected]:
|
Not really, because all Swing Component manipulation must be in the EDT. Anyway these are just two examples where the approach of Rules came in really handy: you get a Statement, you return a Statement, and that gets executed as a Test. I'd really like to have a similar abstraction right in the heart of JUnit 5. Right now Extensions seem to be either rather limited, or bound to the JUnit 5 Engine JUnit 5 Execution Context, because they depend on things being there like TestClasses / or methods, which might not exist for other engines. |
|
The current design tries to balance the power of extensions, composability and the guarantees the JUnit 5 engine can give. JUnit4's statement approach was one of the things that made composition difficult. I'm not saying we made the best calls or that we won't change it. But I do believe there's no solution that will give us only benefits and no drawbacks.
|
|
The JUnit5 engine was designed in that way, that the test execution is always guaranteed by the engine and cannot be changed. In this way, we can guarantee the execution flow and allow extension providers to build extension upon this contract. For altering the test execution we do not yet provide a mechanism. One would be the resolver based discovery approach (see pull request #134) that would allow extension writers to contribute they own To me, all of your examples seem to be very specific use-cases which might not need to be composable and, therefore, are perfect candidates for @schauder : Please have a look and contact me if you have further questions. I am looking forward to getting your feedback. 😃 |
|
Composing (nesting) statements only works if all statements do the exact right thing as of exception handling, ordering, calling before/after behaviour etc.; the difference between correct and incorrect can be subtle. The framework has no means to influence things from the outside. Thus it's easy to compose statements if you control all of them, but it becomes erratic as you compose unknown ones from different sources. One could of course introduce an extension point that takes an Executable of the test method only and returns a different Executable. My fear is that it would be abused for all kind of before/after/exception-handling stuff - just because it's so easy to do. And then we'd be in the same place as we were with JUnit4.
|
|
@jlink I'm probably missing something doing the "right thing" shouldn't be to difficult. The contract of statement in JUnit4 seemed to be rather simple and easy to follow. When filtering and ordering enter the mix I can understand your concerns, but as long as we follow a modified Mad Max principle: One statement goes in one statement goes out everything should be fine. Filtering and ordering should be done by a separate part of the system to avoid any problems.
That was actually the reason why Rules are so important to me: I can put all my reusable before/after/stuff in single code point and get rid of tons of ugly reuse-by-inheritance abdominations. An appropriate substitute for this is essential for adoption of JUnit5 for me. |
|
I don't think that the Mad Max principle is enough since you can do so many I'm absolutely willing though, to rethink out extension mechanism. Let's 2016-02-22 8:26 GMT+01:00 Jens Schauder [email protected]:
|
|
Regarding execution in a different thread, we've discussed two alternatives this morning in a team call: (1) Add a
|
|
As far as I can tell right now both variants should work for what I do with Rules. I kind of agree with your statement about dynamic tests. |
|
Running Moreover, a working contract would require that returning from 2016-02-29 13:21 GMT+01:00 Jens Schauder [email protected]:
|
@jlink, I couldn't have phrased it better myself! I share the exact same concerns and have for a long time: these are in fact the reasons that we opted not to implement extensions in JUnit 5 using the Chain of Responsibility pattern like JUnit 4's rules. However, having said that, I do understand that some extensions may need to control/spawn the thread in which a test executes. To that end, something like a But... at the very least we need to guide developers to use the standard extensions and not abuse these executor extensions, and as Johannes pointed out: we need to be very careful about the ramifications of introducing such an extension. |
|
Alternative design: interface UserCodeExecutor {
default void execute(ExtensionContext context, UserCodeType userCodeType, Executable executable) throws Exception {
executable.execute();
}
}
enum UserCodeType {
TEST_CLASS_CONSTRUCTOR,
BEFORE_ALL_METHOD,
...
} |
|
Hello Marc Thanks a lot for the detailed analysis and your proposals! I am sorry my reply comes a bit late, but I was on holiday until today. However, we really like your first proposal (TestExecutor interface) and prefer that solution to the version that uses enums. This allows us to wrap each executable in a Scout RunContext/Subject again, as we have done it with JUnit 4. I guess this solution would also work for thread-starter @schauder who wanted to execute a test in the Swing thread. Can you make any predictions in which JUnit release this will be implemented (if at all)? |
|
The suggested proposal would work for us as well, I like |
|
Slightly modified proposal to allow multiple registered interface ExecutionInterceptor extends Extension {
void createTestInstance(ExecutionContext context, Executable executable);
void executeBeforeAllMethod(ExecutionContext context, Executable executable);
void executeBeforeEachMethod(ExecutionContext context, Executable executable);
void executeExtensionMethod(ExecutionContext context, Executable executable);
void executeTestMethod(ExecutionContext context, Executable executable);
void executeAfterEachMethod(ExecutionContext context, Executable executable);
void executeAfterAllMethod(ExecutionContext context, Executable executable);
}
interface ExecutionContext {
ExtensionContext getExtensionContext();
Optional<Object> getTarget(); // empty for static methods
Method getMethod();
List<Object> getArguments();
} |
|
Another iteration that allows to inspect invocation results: interface InvocationInterceptor extends Extension {
default <T> T createTestInstance(Invocation<T> invocation) throws Throwable {
return invocation.proceed();
}
default void executeBeforeAllMethod(Invocation<Void> invocation) throws Throwable {
invocation.proceed();
}
default void executeBeforeEachMethod(Invocation<Void> invocation) throws Throwable {
invocation.proceed();
}
default <T> T executeExtensionMethod(Invocation<T> invocation) throws Throwable {
return invocation.proceed();
}
default void executeTestMethod(Invocation<Void> invocation) throws Throwable {
invocation.proceed();
}
default <T> T executeTestFactoryMethod(Invocation<T> invocation) throws Throwable {
return invocation.proceed();
}
default void executeTestTemplateMethod(Invocation<Void> invocation) throws Throwable {
invocation.proceed();
}
default void executeAfterEachMethod(Invocation<Void> invocation) throws Throwable {
invocation.proceed();
}
default void executeAfterAllMethod(Invocation<Void> invocation) throws Throwable {
invocation.proceed();
}
interface Invocation<T> {
T proceed() throws Throwable;
ExtensionContext getExtensionContext();
Class<?> getTargetClass();
Optional<Object> getTarget(); // empty for static methods
Executable getExecutable();
List<Object> getArguments();
}
} |
|
Updated once again to add support for intercepting dynamic tests and to increase consistency with other import java.lang.reflect.Executable;
import java.util.List;
import java.util.Optional;
interface InvocationInterceptor extends Extension {
default <T> T createTestInstance(ReflectiveInvocation<T> invocation, ExtensionContext extensionContext) throws Throwable {
return invocation.proceed();
}
default void executeBeforeAllMethod(ReflectiveInvocation<Void> invocation, ExtensionContext extensionContext) throws Throwable {
invocation.proceed();
}
default void executeBeforeEachMethod(ReflectiveInvocation<Void> invocation, ExtensionContext extensionContext) throws Throwable {
invocation.proceed();
}
default <T> T executeExtensionMethod(ReflectiveInvocation<T> invocation, ExtensionContext extensionContext) throws Throwable {
return invocation.proceed();
}
default void executeTestMethod(ReflectiveInvocation<Void> invocation, ExtensionContext extensionContext) throws Throwable {
invocation.proceed();
}
default <T> T executeTestFactoryMethod(ReflectiveInvocation<T> invocation, ExtensionContext extensionContext) throws Throwable {
return invocation.proceed();
}
default void executeTestTemplateMethod(ReflectiveInvocation<Void> invocation, ExtensionContext extensionContext) throws Throwable {
invocation.proceed();
}
default void executeDynamicTest(Invocation<Void> invocation, ExtensionContext extensionContext) throws Throwable {
invocation.proceed();
}
default void executeAfterEachMethod(ReflectiveInvocation<Void> invocation, ExtensionContext extensionContext) throws Throwable {
invocation.proceed();
}
default void executeAfterAllMethod(ReflectiveInvocation<Void> invocation, ExtensionContext extensionContext) throws Throwable {
invocation.proceed();
}
interface Invocation<T> {
T proceed() throws Throwable;
}
interface ReflectiveInvocation<T> extends Invocation<T> {
Class<?> getTargetClass();
Optional<Object> getTarget(); // empty for static methods
Executable getExecutable();
List<Object> getArguments();
}
} |
|
Starting to remind me of AspectJ's |
The new extension API allows intercepting the invocation of test class constructors, lifecycle methods, testable methods, and dynamic tests. It validates that an invocation is asked to proceed exactly once. The user guide is updated with an example that executes all test methods in Swing's EDT. Resolves #157.
The new extension API allows intercepting the invocation of test class constructors, lifecycle methods, testable methods, and dynamic tests. It validates that an invocation is asked to proceed exactly once. The user guide is updated with an example that executes all test methods in Swing's EDT. Resolves #157.
Overview
I couldn't find a way to write an extension that changes the behavior of test execution.
Examples of what I would like to do are:
Such use cases were easy to implement with JUnit 4 Rules, and I used especially the second a lot. So I think it would be bad to lose this option in JUnit Jupiter.
Proposals
Related Issues
The text was updated successfully, but these errors were encountered: