Multi-module project simplification

[jeubank12]

Description

Multi-module projects (for example Visual Studio solution file) seem to be doing a Scan of each project and then the whole solution. The documentation seems to suggest that this doesn't need to happen or shouldn't happen, but is not clear enough to indicate how to tell sonar-cxx to only the solution.

Steps to reproduce the problem

Solution file with multiple vcxproj in subdirectories.
Jenkins pipeline configuration

def sqScannerMsBuildHome = tool 'Scanner for MSBuild 4.0'
withSonarQubeEnv {
    bat "${sqScannerMsBuildHome}/SonarQube.Scanner.MSBuild.exe begin /n:OrionCollectors /k:orion_collectors_pipeline /v:${componentVersion}.${build_branch}.${BUILD_NUMBER} /d:sonar.host.url=http://jenkins.secure.root9b.com:9000/ /d:sonar.exclusions=Library/include"

    bat '"C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\BuildTools\\MSBuild\\15.0\\Bin\\msbuild.exe" Project.sln /p:Configuration=Release /p:Platform=x86 /m /v:m /fl /fileLoggerParameters:WarningsOnly;Verbosity=detailed;Encoding=UTF-8 /nologo /t:Rebuild'

    bat '"C:\\Program Files\\Cppcheck\\cppcheck.exe" --enable=all --xml --xml-version=2 -iLibrary/include . 2> cppcheck.xml'

    bat "${sqScannerMsBuildHome}/SonarQube.Scanner.MSBuild.exe end"
}

Expected behavior

Analyze only from the root folder, parse the msbuild.log only once, Include all the files of all the subprojects and only require one set of relative paths

Actual behavior

Executes all steps and report consumers for each project, then does the same thing for the overall solution.
Because of Jenkins workspaces and unknown absolute paths, I have to specify multiple layers of include paths (../Library/include,Library/include,../../Library/include) and I get many warning messages when files are not in the correct location.

Known workarounds

For msbuild.log parsing, keep it in the root folder and specify the path as 'msbuild.log' and not '../msbuild.log'. This ensures that at least the log consumption process (about 2 minutes a go) does not repeat.

LOG file

N/A, normal execution, just multiple times

Related information

• cxx plugin version 0.9.9
• SonarQube version 6.7 LTS

[jeubank12]

I may be seeing side effects from this #1325

[guwirth]

Hello @jeubank12,

This multi-module feature is very seldom in use and looking back there were many changes. Each SonarQube version and plugin version behaves a little bit different. What you can try is to turn debug info on and have a look to the LOG file what the core/plugin are doing internally: https://github.com/SonarOpenCommunity/sonar-cxx/wiki/Get-debug-information

Anyway I think the easier way is to:

• split it into several sub-projects
• or forward it as one big project

In https://github.com/SonarOpenCommunity/sonar-cxx/tree/master/integration-tests/testdata/cpp-multimodule-project you find a sample how to put all report into one central folder below. Thats the way we are doing it.

Regards,

[jeubank12]

I found the easiest for this project was to use sonar-runner rather than the msbuild tool. This cut out a majority of the duplicate processing.

…

On Thu, Feb 15, 2018, 4:27 AM Günter Wirth ***@***.***> wrote: Hello @jeubank12 <https://github.com/jeubank12>, This multi-module feature is very seldom in use and looking back there were many changes. Each SonarQube version and plugin version behaves a little bit different. What you can try is to turn debug info on and have a look to the LOG file what the core/plugin are doing internally: https://github.com/SonarOpenCommunity/sonar-cxx/wiki/Get-debug-information Anyway I think the easier way is to: - split it into several sub-projects - or forward it as one big project In https://github.com/SonarOpenCommunity/sonar-cxx/tree/master/integration-tests/testdata/cpp-multimodule-project you find a sample how to put all report into one central folder below. Thats the way we are doing it. Regards, — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1423 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ALXDK4aazR4Z0yT07hP0XbioK_bVGr2Tks5tVAangaJpZM4SDFor> .

[Bertk]

Hello @jeubank12,

your observation is correct and some report files are parsed multiple times. Until now the sensors are activated with every project regardless it has a parent or not.

The following excerpt works nicely with Microsoft Visual Studio solutions and MSBuild scanner but I am not sure whether this works for other scanners as well.

excerpt from C# UnitTestResultsImportSensor.java

  @Override
  public void execute(SensorContext context) {
    if (!unitTestResultsAggregator.hasUnitTestResultsProperty()) {
      LOG.debug("No unit test results property. Skip Sensor");
      return;
    }
    if (projectDef.getParent() == null) {
      analyze(context, new UnitTestResults());
    }
  }

The Cxx plugin uses the visual studio build log to extract include paths and preprocessor definitions before the AST is created and compiler warnings are transformed to SonarQube issues.
I use an enhanced regular expression to limit the log file scanning and handle only warning for the current visual studio project.

If you do not want to preserve the multi-module definition from the Visual Studio solution file, using the sonar-runner will be a good choice.

[guwirth]

Until now the sensors are activated with every project regardless it has a parent or not.

@Bertk what do you mean with this?

[guwirth]

@jeubank12 how is your setup?

• Inside of the .SLN there are several .VCXPROJ?
• Is each .VCXPROJ in an own folder?

[Bertk]

@guwirth

The C# UnitTestResultsImportSensor checks the module context and execute an analysis only for the parent (Visual Studio Solution).

  @Override
  public void execute(SensorContext context) {
    if (!unitTestResultsAggregator.hasUnitTestResultsProperty()) {
      LOG.debug("No unit test results property. Skip Sensor");
      return;
    }
    if (projectDef.getParent() == null) {
      analyze(context, new UnitTestResults());
    }
  }

[ivangalkin]

Hi All,

I can only confirm, that in this regard the behavior of sonar-cxx matches the standard behaviour of other plugins and (especially) sensors: Each sensor is invoked for each (sub-)module separately. That means some exemplary ImportHugeXMLSensor will most probably skip a large number of its entries with the message

Cannot find the file <path>', skipping violations

The same message could appear multiple time, since <path> is part of module0 only, but not of the module1, module2, ..., moduleN etc.

I believe, that forcing such sensor to be executed on the root module only (if it's possible at all) would not solve the redundancy. AFAIK root module does not contain <path>, from the formal point of view it belongs to module0.

In any way from my experience the reported behavior seems to be standard.