https://github.com/italktothewind/karate-test-run-issue

[italktothewind]

Hi,

I followed the recommended folder structure in: https://github.com/intuit/karate#folder-structure

One of the feature files (common.feature) was created to be called by other feature files (like example.feature), but it is being counted as a test ("Tests run: 2")

I uploaded a minimum working example here (just run mvn clean install):
https://github.com/italktothewind/karate-test-run-issue

I don't know if it is a bug or am I doing something wrong, thanks!

[ptrthomas]

your pom is not a simple example. also you still haven't followed the instructions, for e.g. what is the command you run, it is missing

but 3 options

• move common.feature so that your test-suite runner does not pick it up see docs below
• use an @ignore tag
• specify names of features to run explicitly in your "suite" runner

please follow instructions clearly next time

[italktothewind]

The pom is minimal. I'm using Failsafe instead of Surefire because they are integration tests so there are not *Test but *IT. Also, in the issue it was mentioned what command has to be used: mvn clean install.

About the options:

1. I have a test suite, just for including the runners. It is not related with Karate but with Failsafe. My runner is in a folder that it is a sibling (not a child) of common.feature. Runner folder: it/src/java/example. Common.feature folder: it/src/java/common. Do you know why is running also features from sibling folders?
2. I don't like to ignore something that is not a test. Somebody can think that has to be unignored.
3. I think that the annotation is very practical to avoid configuration so I will keep it.

[italktothewind]

The issue was that I was running the suite with @RunWith(Karate.class) instead of @RunWith(Suite.class)

[italktothewind]

Hmmm, I have to said, however, that when I change:

<karate.version>0.9.0.RC5</karate.version>

with a newer one:

<karate.version>0.9.2</karate.version>

Again I have:

Tests run: 2, Failures: 0, Errors: 0, Skipped: 0

[italktothewind]

I tried with 0.9.5 (the latest) and now I have:

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

Probably there was some issue with version 0.9.2

[ptrthomas]

my bad, I missed that you actually had mvn clean run, because I looked for it on a new line.

and sorry, I don't agree that the pom is minimal. I will update the instructions, so that people don't do what you did in the future :)

we recommend that Karate tests be normal tests, if you go down the IT route, you are on your own, see this comment: #911 (comment)

[italktothewind]

It is very reasonable to think of Karate tests as integration tests. In fact, most of the times you need to setup some environment before running them, at least something that listens to http requests. A good idea is to start the environment in pre-integration tests and tear it down in post-integration tests. So, you guess it! The best place to run Karate tests is in integration-tests maven phase and these kind of tests use IT convention.

[ptrthomas]

@italktothewind I guess you haven't even read the link in my comment :) all this is good in theory, the reality is quite different, but if you end up with a good sample of integration tests, let me know and I will add it to the documentation as a link / reference.