@Ignore and @Category with Request.filterWith() behave differently

[noel-yap]

PASSES:

@Ignore
public class NotReallyATest {
}

FAILS with "No runnable methods" initializationError (using Request.filterWith to filter out tests Categorized as Manual):

@Category(Manual.class)
public class NotReallyATest {
}

[Tibor17]

@noel-yap

this would help if you provide a test in your repo.
it would simplify reproducing the identical test.

your experiences in categories is essential!
would you join us in the code review in #142 and contribute?
thx,
Tibor

[Tibor17]

@noel-yap
I was facing very similar issues, now fixed in my repository implementing features #336 and #354.
When no test is written in test case, as you described, the runner should fail even with categories.
Similar if just only one test in test case appears in code but the particular test is @Ignore-d, or if empty suite.

Your test can be found in the test nothingRunnable3()
and similar variants in tests noTestsCategoriesSuite(), emptyCategoriesSuite().
https://github.com/Tibor17/junit/blob/master/src/test/java/org/junit/tests/experimental/categories/CategoryTest.java

you may check it out.

[dsaff]

@Tibor17, @noel-yap, is this one fixed?

[Tibor17]

@dsaff
I agree that this is behavior of categories is different from @ignore.
It sounds strange to me when someone has an empty test with a kind of category on the test-class and trying to execute it. I think such test does not make sense. Even if the developer thinks about commenting out all test methods, I guess he would rather @ignore the class itself. This means:

@Ignore @Category(Manual.class) public class NotReallyATest { }

which should be ignored test as well as :

@Ignore public class NotReallyATest { }

IMHO this is relevant scenario to check.

[kcooney]

If this only happens for classes that do not have any test methods, then I'm inclined to say that this is working as intended.

[noel-yap]

IIRC, the issue lies in the difference between the implementations of
@ignore and @category; one doesn't remove methods while the other one does
so even if the test had methods, if they're filtered out, the exception is
thrown.

On Tue, Dec 24, 2013 at 9:50 AM, Kevin Cooney [email protected]:

If this only happens for classes that do not have any test methods, then
I'm inclined to say that this is working as intended.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/381#issuecomment-31180348
.

[kcooney]

@noel-yap Could you provide a test for an exception being thrown if a test class annotated with @Category that has methods annotated with @Test throws an exception if the tests are filtered out?