Include stdout for tests that pass

[pete-thompson]

When a test passes there is no way to include the messages for that test in the element. It would be helpful if there were a configuration option to enable verbose messaging on successful tests.

[Siphonophora]

@pete-thompson Hi, are you not seeing the messages in std output here https://github.com/spekt/junit.testlogger/blob/master/docs/assets/TestResults.xml#L98-L102 ?

Some of the GUIDs from this example file came from a passing test https://github.com/spekt/junit.testlogger/blob/master/test/assets/JUnit.Xml.TestLogger.NetCore.Tests/UnitTest1.cs#L14-L15 so I would expect std out to be there.

In the junit spec, there isn't anywhere to put standard output per test case. the nunit spec, for example, does allow for output per test case.

Let me know what you are seeing and we can discuss further.

[pete-thompson]

Thanks @Siphonophora - I can see the stdout messages in the main section, but would ideally see them in the individual test cases. As far as I can tell the and elements are valid for any element regardless of whether the test passes or fails. It's hard to tell where the Junit XML specification is, but I found this XML schema which seems to allow for system-out: https://github.com/junit-team/junit5/blob/main/platform-tests/src/test/resources/jenkins-junit.xsd. I also found this, which likewise makes no distinction between passing or failing tests: https://llg.cubic.org/docs/junit/#:~:text=The%20JUnit%20testing%20framework%20has,display%20results%20of%20the%20tests.&text=The%20files%20shown%20on%20this,CC0%201.0)%20Public%20Domain%20Dedication.

[Siphonophora]

@pete-thompson Based on the ticket above, it seems like the jenkins schema is the most official one.

@AnashOommen I know its been a while but I recall you were doing a strict validation of the output of our reports against the Ant unit schema. Are you in a position to change that schema to the jenkins one linked above, or would adopting the different schema break your system?

So, this will take a little thought. Hopefully the current report passes validation against both of these schemas, and its just the stdout we need to think about as a possible breaking change.

[mclayton7]

@Siphonophora would you accept community contributions for putting stdout into tags for each test case a la @pete-thompson's suggestions above?

[Siphonophora]

Hi @mclayton7,

Yes we would be happy to. I do think the right answer here is to swap out https://github.com/spekt/junit.testlogger/blob/master/test/assets/JUnit.xsd for the jenkins version and then go ahead with the changes.

Note our Nunit logger does this sort of logging at the test level, see https://github.com/spekt/nunit.testlogger/blob/7d302a3391f0eae699277dd678daa0dffb8b14d7/src/NUnit.Xml.TestLogger/NUnitXmlSerializer.cs#L262

One other note. The acceptance tests for this library run using the most recent packaged version of the library you have locally. Run build.ps1 in order to build and package any changes you make, so the acceptance tests can run with your updates.

Let me know if you have any questions after you get started.

[mclayton7]

@Siphonophora thanks! We use the NUnitXmlTestLogger for our CI setup, but I'd like to switch to the JUnit logger for better GitLab integration. What's holding us back is that we have some scripts parsing stdout from individual tests with the NUnit version, this will let us switch.

I'll try to make the change stay within the spec and match what the NUnit logger is doing.