Provide dedicated Spring Boot starter for JUnit 5 #14716
Comments
spring-projects-issues
added
the
status: waiting-for-triage
|
@olivergierke and I discussed this on Slack last week. A couple of thoughts from me from that discussion:
One additional thought: The example above makes things look one dependency worse then they actually are. |
|
I concluded with the following:
|
|
Thanks for the API JAR catch, I've removed that one. Regarding the naming, I thought that aligning with |
|
While I understand there is slightly more work than expected to use JUnit Jupiter, I am not keen to branch our current starter (or any starter for that matter). I agree with @wilkinsona that this adds complexity here for a problem that's in JUnit. |
Hm, isn't basically every Spring Boot starter an attempt to solve that problem for some third party library? I.e. what else than an attempt to solve exactly that problem are starters in the first place? How do you mean branch? Wouldn't it be a new starter (i.e. a single additional POM file) containing just a slightly different setup of JUnit libraries? It's not that new major versions of JUnit pop up every other week, is it? If you think it's too complex, fine. I just thought it felt weird to have to declare 5 explicit dependencies instead of 1, one of which even needs an explict version number if I wanted to use the current version of the most commonly used Java testing framework. That's quite at odds with the rest of the Spring Boot dependency management experience. |
What I mean is having a "v4" and "v5" starter of the same feature. Regardless of how we name the new one, it'll become irrelevant once we decide to switch the current starter to JUnit 5. That's an odd situation I'd like to avoid if we can. |
It may be the current version as in the latest, but it isn't the most widely used, even for greenfield development. I'm pretty sure that honour still belongs to JUnit 4.
Our goal is to make common use cases as easy as possible without preventing more complex or less typical scenarios. In this situation the common use case is to use JUnit 4 and the less typical scenario is the use of JUnit 5. As you've demonstrated above, using JUnit 5 is possible. It's more complex than you'd like but that's almost entirely due to JUnit 5's structure and the rest of the ecosystem arguably not being JUnit 5-ready just yet. The only way that I can think of to simplify the situation for those who want to use JUnit 5 is to introduce another starter. That could either be two separate top-level starters, or it could be a lower-level starter that's swapped in much as we do with |
|
I'd argue this is a chicken and egg problem. If we only provide It feels like the team is not eager to add support for JUnit 5 right now. Let's use this ticket to see whether other parties are interested and gather further feedback. |
I think there is. It's things like IDE support (largely there now, but will require adoption of new IDE versions) and build system support (likewise), in addition to things like Hamcrest not assuming the use of JUnit 4. |
|
I’ve come across the same problem (Boot lacking OOTB JUnit 5 support), with one difference. I used AssertJ, not Hamcrest, so I think not automatically bringing in Hamcrest and leaving that choice to the user might be a good thing. The way I see it, neither Hamcrest nor AssertJ are necessary to write the tests; in fact, JUnit 5 has some new assertions by itself |
|
Regarding Hamcrest: I consider |
I am not entirely certain that the honour belongs to JUnit 4. All recent projects that I have recently started are on JUnit 5 (with all the hoops one needs to do to support that in Spring Boot). I would agree with @olivergierke that having a
How can you measure JUnit 5 adoption for Spring Boot? The IDE support is already there and I think that users are already on the new IDE versions. IntelliJ has support since 2016.2 which is more than 2 years now and eclipse since Oxygen 1.a which is more than a year now. Since the 2.22.0 version of the What exactly is the complexity for the Spring Boot team for this? |
|
I'm personally not keen to add an additional JUnit 5 starter because I think at some point (probably 2.2) we'll be looking to upgrade the existing test starter to JUnit 5 (probably whilst also trying to support JUnit 4 based tests). Once we do that, it would be nice if we didn't also have the additional job of deprecating a Looking again at the original issues that @olivergierke raised, he wants to add the following: junit-jupiter-paramsI'm quite surprised this one is a module at all, given that it appears to be very commonly used. I wonder if the JUnit team have considered publishing a POM themselves that pulls in org.mockito:mockito-junit-jupiterThis jar is super light so we could add it to the existing starter and exclude org.hamcrest:hamcrest-junitAs @marcphilipp mentioned above, using |
|
I've raised junit-team/junit5#1629. If that's accepted and we pull in |
philwebb
added
the
for: team-attention
junit-jupiter-paramsSee junit-team/junit5#858 for why we didn't merge junit-jupiter-engine
Maven doesn't provide such scopes ootb, and Surefire is not smart enough, yet, to auto-resolve the matching engine here. This junit-platform-maven-plugin does already do the auto-resolve "magic". See this picture for an architecture overview. |
|
For those reacting with a 👍, can you please comment with suggestions on how to deal with the added complexity? We need to consider both the desire to use JUnit 5 and a desire not to make things harder or more complex for those who want to stick with JUnit 4. We also need to factor in Phil's concern above. Without an answer to those, I can't see a way forward other than waiting until we flip over to JUnit 5 by default. |
It's not really complexity for the Boot team but complexity for users. For example, if we offered If moving |
👍 Nice side-effect: it gives build tool authors more time to improve their JUnit 5 integration. |
|
@philwebb said:
If you are considering adding JUnit 5 test starter. I would definitely not mix it with JUnit 4. This will also lead to people needing to exclude JUnit 4. If people want to use JUnit 4 (with JUnit 5) they just need to pull the Regarding the concern for the extra complexity for users. I don't think it would be confusing. JUnit 4 has been the default engine in Regarding the 2.2 if you want to switch the default then you will write that in the migration notes and you can add |
What if a
Are you saying you consider to upgrade the If we accept your statement that JUnit 4 is still the predominant version used in Boot projects, why would that change before Boot has more first-class support for 5? Why produce that cliff for ourselves if we could just switch to support both generations (effectively two different testing frameworks) so that users can switch at their own pace and us being able to deprecate and phase out the JUnit 4 support at some point. |
|
I wanted to leave one important note: The list of dependencies I listed was caused by very specific upgrade requirements: I wanted to get a JUnit 4, Hamcrest based project to run on JUnit 5. I accepted a few package import changes but anything but that. I am not suggesting that this is proposed content for a new starter. I am also not suggesting to move IMO, the new starter should only contain dependencies that you'd use in an idiomatic JUnit 5 setup. Just like |
No. That's not the proposal. This is what @philwebb said above:
Perhaps that's not possible. We don't know yet, but we need to know the answer before anything can be done here.
Unfortunately, that makes it a poor example for the benefits that the starter would bring as it overstates the benefits. With Spring Boot 2.0.x, I can write and run JUnit tests in Eclipse (Photon) with the following dependencies in my project's pom: <dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<exclusions>
<exclusion>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
</exclusion>
</exclusions>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
<scope>test</scope>
</dependency> |
|
Yes, you can. However you wouldn't get the same rough featureset you got on JUnit 4. It's mostly the parameterized tests extension and Mockito. And yes, if I find myself missing these annotations I can go browse the JUnit documentation, find the pointers to the artifacts and add those artifacts to my POM. That's still significantly more work than just using a dedicated starter that brings those around. To summarize it's: <dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-params</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-junit-jupiter</artifactId>
<scope>test</scope>
</dependency>VS. <dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test-…</artifactId>
<scope>test</scope>
</dependency> |
|
I'm taking an executive decision here that we're not going to add a JUnit 5 starter at this time. I've raised #14736 instead to look at what's going to be involved with upgrading applications. I think we need to answer that question before we can consider anything else. I've also labeled a few related issues with theme: testing to help us track things. Thanks for the input that everyone has given on this issue, it will be very helpful when we look at #14736. Depending on the outcome of #14736 we may also end up reopening this one, but I really want to answer the upgrade question before jumping into a new starter. |
philwebb
added
status: declined
|
IMO, the new versions of starter should get rid of JUnit4. For a user that migrates to newer versions of Boot, there is an expectation on having to update/fix things, but the newcomer wants the latest, not the vintage, transparently. Anyway, the Spring Boot ecosystem mostly consists of developers eager to embrace change, we should aim for the ultimate standard in doubt. Just my 2c as user |
|
@nightswimmings Yes, it's odd to me as well that Spring Boot Starter Test 2.3.0 still contains JUnit 4 dependency |
|
I've opened #21625 to consider when we should remove JUnit 4. |
When trying to port a Spring Boot project that has previously used JUnit 4 to JUnit 5 one needs to add quite a few additional dependencies to basically get back to the very same functionality that one got with the standard test starter. This is due to the fact that in JUnit 5, a lot of functionality that had been included in JUnit itself was extracted into extensions that now live in third party projects that require explicit inclusion.
Most notabily it are the following additional libraries:
org.junit.jupiter:junit-jupiter-paramsfor parameterized testsorg.mockito:mockito-junit-jupiterfor@Mockorg.hamcrest:hamcrest-junitforassertThat(…)Especially the latter is quite nasty as it in turn depends on JUnit 4.12, so that an explicit exclude is needed.
Be sure to read the note on that list below.
It would be cool if that could be reduced to only declare a
spring-boot-starter-test-junit5(name tbd) instead ofspring-boot-starter-test.The text was updated successfully, but these errors were encountered: