Karate tests should be integration tests in Maven

[atehrani]

Maven has lifecycle events dedicated for unit tests and integration tests, which are fundamentally different types of tests, hence the different lifecycles. In fact there are different Maven plugins for the respective lifecycles; the maven-surefire-plugin is dedicated for unit tests and the maven-failsafe-plugin is dedicated for integration tests, this is clearly stated in the documentation.

Karate tests, are by nature integration tests, not unit tests and should be treated as such. Currently Karate leverages the maven-surefire-plugin which is used for unit tests, this is incorrect. Because of this, it makes it very difficult to separate running unit tests vs integration tests, as Karate breaks the Maven convention (in many ways). Also the Maven lifecycle for integration tests has a pre-integration-test and post-integration-test lifecycle, which is handy to startup common utilities used by integration tests (such as WireMock, SQLite, ActiveMQ...etc).

Are there examples to show how to use Karate using the maven-failsafe-plugin, is this even possible?

[ptrthomas]

@atehrani personally I am not in favor of trying to force tests into any particular "bucket". FWIW karate has been in the wild for 2.5 years and this is the first time we've gotten feedback that Karate tests being treated like unit-tests is an "issue". Do note that Karate can flex to even do unit-tests - here is an example: https://twitter.com/ptrthomas/status/1132515667310047233

Yes, all the existing examples and demos use the surefire-plugin, and plenty of teams all across the world seem to be doing fine and getting work done. We simply use @BeforeClass to do "setup stuff" and it is quite simple, even using Karate for API mocks just works. Using tags or different JUnit runner classes (to model "suites") is more than sufficient to solve for "segregation".

That said, Karate does not "force" you into anything, you are free to use failsafe and maybe you can blog about it or something - and we will be happy to point to this from the documentation. closing as won't fix.

[ptrthomas]

@atehrani just one more thing, here is a stack overflow answer that I think directly answers your question: https://stackoverflow.com/a/48522439/143475

[ptrthomas]

an update for others looking for this solution, pasting a comment from a Stack Overflow answer below - https://stackoverflow.com/a/59084632/143475

Turns out that I cannot be done and it is a limitation of Maven, not Karate. https://stackoverflow.com/questions/10138559/howto-add-another-test-source-folder-to-maven-and-compile-it-to-a-separate-folde/40917384 - Here is my test project to prove it out: https://github.com/djangofan/spring-boot-hello - Thanks for leading me down what appears to have been the correct path to discover the limitation. Using Gradle would likely solve my issue but that is not an option on my project. If I use Karate for "separated integration tests", I need a separate mvn test module.