Is there a way for the report to ONLY contain the results of those using the @XrayTest, @Requirement ?

[Khanivorous]

I'm trying to run tests for a spring app and I have integration tests and "true unit" tests. I want the xray junit report to only contain details of the integration tests. I tagged the integration test methods with XrayReport but I still get new xray tests generated automatically, because the junit report file contains the results for all tests.

[bitcoder]

There's currently no logic for that. Maybe it could make sense... need additional thoughts on that.
Meanwhile, maybe running maven multiple times - one for unit and another for integration tests - can be an option?

[Khanivorous]

Hi @bitcoder , thanks :). Yeah I'm currently working with is using gradle tasks; different test tasks for integration and unit. I guess maven would use -Dgroups. So it's not an impossible issue, but would be a great enhancement!

[bitcoder]

Ill prepare a PR whenever I have some more time. I have a proof-of-concept but requires some additional code

[Khanivorous]

Thanks again! In the meanwhile I have an example project to show how I'm using it, with the workaround in gradle tasks:
https://github.com/Khanivorous/TodoClient

[rfelgent]

Hi @Khanivorous and @bitcoder ,

today I stumbled accross the same problem: filtering xray relevant test results.

My 2 cents:

• @XrayTest and @Requirement are (currently) optional and therefore imho not suitable for identification
• intoduction of another annotation/marker interface might make sense. Its single purpose is: show xray export relevance

I have a proof-of-concept but requires some additional code

What kind of PoC are you talking about? Is the merge request/code pulic available ? I would like to contribute in order to speed up things, as my company/team requires the "xray export relevance" feature

[bitcoder]

I just pushed the branch and created a PR with it.
This has some old code, needs to be reviewed with care.
I'm not fully convinced that the feature should be added at this point though; for the initial reason I shared at start: maybe in this case, it would make more sense to run these 2 types of tests separately. Another option would be to filter this out, post-processing the report, using a tool such as this junit-processor; I made that tool btw, and it should be easy to implement there a rule for it (I can do that). I'm inclined to forward that path, as it removes complexity from here and right now I don't see many users having this demand.
Feel free to share your thoughts!

[bitcoder]

I released junit-processor v0.0.6 with patch 4 that can be used to post-process the JUnit XML report.

npm install -g junit-processor
junit-processor -p 4 some_junit_report.xml

[rfelgent]

Hi @bitcoder ,

thank you for your code and initial thoughts about the ideas.

Please let me give you my 2 cents:

• My assumption was to find a library, that turns a (existing) test into a XRay honored test. I wanted to find a library, that is annotation driven: for me @XRayTest was the way to go. Now I understand, that this annotation is currently not required as a XRay mark - it just provides additional information when writing the XRay-Unit test report. Still, other people have the same assumption/requirement like me (e.g. @Khanivorous). But in the end, I do not know whether the majority of the users share that demand or not.

• If your library has the assumption that all tests under /src/main/test are of interest for the XRay test management, you move the requirement of filtering tests to some other place. Therefore, you move (code) complexity for filtering of XRay honored tests to some other place. I think, that the total amount of effort for the externalized filtering (not part of the library) compared to the internal filtering (part of the library) is eventually higher. Let me explain further:

... maybe in this case, it would make more sense to run these 2 types of tests separately...

"2 Types" means some kind of distinction and this more or less happening by introducing another annotation/interface (or using the existent @XRayTest): mark the test as "XRay-honored-Test".
Of course, the "2 Types idea" could also be implemented by other ways like using JUNIT5 @Tags or by introducing some kind of file/test naming strategy or whatever. In my situation, this means more discussions with other (java) developers of (other) teams.

... Another option would be to filter this out, post-processing the report...

By introducing a post-processing, you increase complexity because you add more steps to a build life cycle (e.g. other maven plugin). In this specific case the https://github.com/bitcoder/junit-processor is not an option, because it introduces another (programming) language for the project.

Bottom line: The one library solution would increases acceptance among teams/developers by supporting all requirements "out-of-the box".

Fun Fact: the issue #10 is also on my company's wish list. But that is a another discussion :)

I am looking forward hearing from you @bitcoder

[rfelgent]

@bitcoder how do we proceed here ?

[bitcoder]

I added the report_only_annotated_tests configuration option; if in enabled only test methods annotated with @Xraytest or @requirement, the two annotations provided by the plugin, will be on the JUnit XML report. It will be part of the 0.7.0 release.

[rfelgent]

thank you very much @bitcoder

as far as I can see, the release is not yet published on maven central ?! When will that happen ?