NUnit support

[guwirth]

update sonar-dotnet-tests-library to 1.2
use "sonar.cxx.nunit.reportsPaths" for NUnit reports
rename derived classes from MSTestXxx to CxxUnitTestXxx
clean-up folder structure (xunit => tests/xunit and tests/dotnet)
close #722

[guwirth]

@Bertk maybe you can test if the MSTest part is still working for you? I had to change the logic of the unit tests (isRoot true/false). Looking to https://github.com/SonarSource/sonar-dotnet-tests-library/blob/master/src/main/java/org/sonar/plugins/dotnet/tests/UnitTestResultsImportSensor.java analyze V1.2 is only working this way.

[Bertk]

I will test it tomorrow but my test environment changed and the MS-tests are somehow limited.

[jmecosta]

@guwirth i can test also with our c++/cli.

[Bertk]

There is still on feature missing which aggregates test results from multiple sources e.g. xunit (nunit, gtest) and MS-test similar to https://github.com/SonarSource/sonar-dotnet-tests-library/blob/master/src/main/java/org/sonar/plugins/dotnet/tests/UnitTestResultsAggregator.java
The sonar-dotnet-tests-library supports aggregation of nunit and MS-test reports (BatchExtension)

  public boolean hasUnitTestResultsProperty() {
    return hasVisualStudioTestResultsFile() || hasNUnitTestResultsFile();
  }

  private boolean hasVisualStudioTestResultsFile() {
    return settings.hasKey(unitTestConf.visualStudioTestResultsFilePropertyKey());
  }

  private boolean hasNUnitTestResultsFile() {
    return settings.hasKey(unitTestConf.nunitTestResultsFilePropertyKey());
  }

And we have now:

sonar.cxx.xunit.reportPath
sonar.cxx.vstest.reportsPaths
sonar.cxx.nunit.reportsPaths

[jmecosta]

Indeed this was what I wanted to test, so we don't have the possibility to
aggregate those yet?

On Tue, Jan 5, 2016, 17:59 Bert [email protected] wrote:

There is still on feature missing which aggregates test results from
multiple sources e.g. xunit (nunit, gtest) and MS-test similar to
https://github.com/SonarSource/sonar-dotnet-tests-library/blob/master/src/main/java/org/sonar/plugins/dotnet/tests/UnitTestResultsAggregator.java
The sonar-dotnet-tests-library supports aggregation of nunit and MS-test
reports (BatchExtension)

public boolean hasUnitTestResultsProperty() {
return hasVisualStudioTestResultsFile() || hasNUnitTestResultsFile();
}

private boolean hasVisualStudioTestResultsFile() {
return settings.hasKey(unitTestConf.visualStudioTestResultsFilePropertyKey());
}

private boolean hasNUnitTestResultsFile() {
return settings.hasKey(unitTestConf.nunitTestResultsFilePropertyKey());
}

And we have now:

sonar.cxx.xunit.reportPath
sonar.cxx.vstest.reportsPaths
sonar.cxx.nunit.reportsPaths

—
Reply to this email directly or view it on GitHub
#742 (comment)
.

[guwirth]

@Bertk why do we have to aggregate the results? Thought SQ is doing this?

[Bertk]

The values for the metric is set to zero (see CxxXunitSensor simpleMode) and without aggregation for the sensor results only the results of last sensor for the project will be uploaded to SQ.
The C++ plugin should support the following use cases

• Multiple test result files for one project consumed by single sensor e.g. running tests for different platforms (x86, x64, Windows, Linux)
• Test results from different tools gtest, cppunit, … and MS-Test for one component consumed by different sensors (migration of tests from one framework to another one)

[Bertk]

I checked the MS-test results on a single project and got identical results before and after applying the PR.

[guwirth]

Ok what I understand:

• using only one kind of unit test sensor for the whole project works: using only xunit, vstest or nunit?
• dotnet has an aggregator so using vstest and nunit together in one project should also work?
• xunit is not working in parallel with vstest/nunit in simpleMode because measures are set on project level?
• xunit is working in parallel with vstest/nunit in detailedMode because measures are set on file level?

Gotcha?

[Bertk]

Exactly 👍

[jmecosta]

there are a few more things i notice, if 2 plugins import tests it will fail with measure already added. see this: https://github.com/SonarSource/sonar-dotnet-tests-library/pull/14

the question here seems that core does not support more than one language plugin importing reports. if a single language is involved, then the aggregation should be ok.

but i see all this in need of support from core, or at least a explanation how this should work

[guwirth]

Wrote a comment in the SQ newsgroup: https://groups.google.com/forum/#!topic/sonarqube/LgC3sqcdx4A

[guwirth]

Limitations has to be documented.