Projects created automatically are broken

[felipecrs]

Describe the bug
If I run mvn sonar:sonar with the sonar.pullrequest parameters for a project that does not already exist in Sonarqube, the project gets created automatically.

However, in that case, the project that gets created in Sonarqube is apparently broken because when I try to open it from the UI this shows up:

Digging a bit further, this issue is apparently related to the branch name of the project. I think the project does not get created by the plugin with the correct branch name set as the main branch.

If I create the project manually first in SonarQube, and set its main branch to the correct branch, then mvn sonar:sonar works as expected.

To Reproduce

1. Run mvn sonar:sonar with the sonar.pullrequest parameters for a project that does not exist already in Sonarqube.
2. After the analysis runs, go to Sonarqube and try to open the project from its UI. It should fail with the same error as the screenshot of earlier.

Expected behavior
To have the project created with the target branch sonar.pullrequest.base set as the default branch.

Screenshots
See above.

Software Versions

• SonarQube Version: 9.9 (build 65466)
• Plugin Version: 1.14.0

Additional context
This causes the Sonar Gerrit Jenkins plugin to fail for some weird reasons like:

SonarQube analysis 'AYaAYSId_x56GYGzmtJ8' failed with message: Could not find target branch 'null' in project.

The Sonarqube logs shows things like:

ERROR ce[AYaAYSId_x56GYGzmtJ8][o.s.c.t.s.ComputationStepExecutor] Execution of listener failed
java.lang.IllegalStateException: Branch has not been set
java.lang.IllegalStateException: Could not find target branch 'null' in project
ERROR web[AYZ/fJ+hFk6S4/ySAAUs][o.s.s.w.WebServiceEngine] Fail to process request http://<host>/sonarqube/api/project_pull_requests/list?project=<project>%3Aspring-app-parent
java.lang.NullPointerException: Pull request data should be available for branch type PULL_REQUEST

[mc1arke]

I don't think this can work the way you want. Sonarqube needs to have the analysis of the target branch in place before it can perform a Pull Request analysis, so dynamically creating a project during pull request submission isn't something that would feasibly allow this since the target branch couldn't already exist in Sonarqube and Sonarqube would therefore find no baseline to work against. The error messages could probably do with some clarification here, but I'm closing this as invalid given Sonarqube core can't support this way of working

[felipecrs]

Well, shouldn't at least the scan fail more gracefully in this case?

For example, instead of having the project created brokenly in Sonarqube, requiring some administrator to delete the broken project and recreate it manually, it could simply fail during the first scan saying "please create the project in Sonarqube before issuing the first scan".

[felipecrs]

Thanks for the answer by the way.

[felipecrs]

Sonarqube needs to have the analysis of the target branch in place before it can perform a Pull Request analysis

I don't think this is the case, not according to my tests. As long as the branch is also fetched by git during the pull request analysis, the Sonarqube analyses it just fine. I did a few tests for it (without having Sonarqube scanned the branch prior to the PR analysis).

so dynamically creating a project during pull request submission isn't something that would feasibly allow this since the target branch couldn't already exist in Sonarqube

But this is exactly what happens today: the project gets created automatically but without the branch, and I'm reporting that, as part of the automatic creation of the project, I'd say the branch had to be created as well.

[MDullak]

Hi,

I upgraded today from 8.9 LTS with plugin 1.8.1 to LTS with plugin 1.14.0.
Everything looked OK up to the point where I now have the same issue: same screen + same log message.
My branches were OK, unless something got broken during the migration (which I am not aware of).

Now I can not do anything. Not even manually go to sonarqube.xxx.com/project/branches?id=Project_Name to see what I have and what I am missing.

I am running branch to which the PRs were merging now (our main development branch) and will report if that fixed the problem.

[MDullak]

So the moment that analysis was scheduled (it is still in progress) everything is back OK.
Strange thing is that main branch was already analyzed according to log.
We will see what will happen once current analysis in completed.

[Wihrt]

I have looked into this problem.

For the following explanation, we will run an analysis for a non-existant branch with directly a merge request.

My analysis is the following :

• Analysis ▶️ Fails because the target branch of the merge request does not exists. It is totally normal.
• Database ▶️ Writes in database a row with the merge request and the missing info. It is strange
• Web ▶️ Throws an error because of the merge request with the missing info. It is normal.

The brutal solution would be to destroy the project, then re-create it properly.

The clean solution is to remove only the errors.
To do that, you would need the database and the API.

Database
Run the following Query :
SELECT project_uuid, kee FROM public.project_branches WHERE branch_type = 'PULL_REQUEST' and merge_branch_uuid is null

You can run a small query to get the project key. It will be needed for the API

API
Run the following command:
curl -u <user>:<password> https://sonar/api/project_pull_requests/delete?project=<project_key>&pullRequest=<pullRequestId>

Once you have done that, your project is now available.

If you are interested, I have developed a small Python script to perform that :)

[MDullak]

Hi, I got it back up and running by manually performing analysis of the target branch (my main branch).
This resolved the issue and I have all the data visible now.
Maybe I should have opened new issue not commenting on this one since my issue was not on a new project, it was after upgrade.

[felipecrs]

Anyway, it's not necessary to perform an analysis of the target branch. The only requirement is that the target branch exists in SonarQube (it does not need to be analyzed).

This is why I think a pull request execution should also create the target branch automatically.

[mc1arke]

The only requirement is that the target branch exists in SonarQube (it does not need to be analyzed).

You can't have a branch in Sonarqube without an analysis, and having a branch without an up-to-date analysis would render the PR analysis as invalid since the baseline wouldn't have all the findings to perform a comparison against

This is why I think a pull request execution should also create the target branch automatically.

This isn't something that would be reasonable to try and support since it would give potentially misleading results. The scanner component that submits details for analysis also does not support submitting more than the currently checked-out branch, so this operation wouldn't be feasible.

[felipecrs]

You can't have a branch in Sonarqube without an analysis, and having a branch without an up-to-date analysis would render the PR analysis as invalid since the baseline wouldn't have all the findings to perform a comparison against

According to my tests you are wrong in this statement. However, let's assume you are right.

So, someone is trying to add the sonarqube scan to their project. The way how the scan is introduced in the CI workflow has to be through a PR, then, after PR is merged, Sonarqube is finally able to scan the branch for the first time and produce meaningful results for the following PR scans.

However, with the current behavior, the user would be stopped to even merge the PR to begin with.

What's your proposal to that?

In my opinion, even if you were right about the scan producing misleading results, it should be acceptable as a first time scan (otherwise the person can't even merge the PR which adds the scan and thus the scan will never run for the branch).

The scanner component that submits details for analysis also does not support submitting more than the currently checked-out branch, so this operation wouldn't be feasible.

But I'm not talking about publishing an analysis of the branch while in pull request mode. I'm talking about creating the branch for the project (without publishing any analysis for the branch), that's a different thing.

[mustafaglr]

Does your branch has special charachters? If there is could you try to put your branch name in single quotes?
(E.g. -Dsonar.pullrequest.branch='hotfix/1.1')

[felipecrs]

Mine hasn't (branch name is "develop").

[mjaggard]

Database Run the following Query : SELECT project_uuid, kee FROM public.project_branches WHERE branch_type = 'PULL_REQUEST' and merge_branch_uuid is null

You can run a small query to get the project key. It will be needed for the API

API Run the following command: curl -u <user>:<password> https://sonar/api/project_pull_requests/delete?project=<project_key>&pullRequest=<pullRequestId>

Once you have done that, your project is now available.

Doing this means that I now get no results in the SQL query but the project is still unavailable

[Wihrt]

Database Run the following Query : SELECT project_uuid, kee FROM public.project_branches WHERE branch_type = 'PULL_REQUEST' and merge_branch_uuid is null
You can run a small query to get the project key. It will be needed for the API
API Run the following command: curl -u <user>:<password> https://sonar/api/project_pull_requests/delete?project=<project_key>&pullRequest=<pullRequestId>
Once you have done that, your project is now available.

Doing this means that I now get no results in the SQL query but the project is still unavailable

Honestly, I have tested this procedure in SonarQube 9.8.
Since I don't know on which version you are running your Sonarqube.
Quite difficult to help you without other informations ...

[felipecrs]

In my case, it's SonarQube 9.9.1 LTS.

[mjaggard]

9.9.1.69595 for me. Any idea what else I can try to look into?

[mjaggard]

@Wihrt apologies that did work for me. I have no idea why the SQL request didn't return the problem after the first time I tried. Maybe I misread or changed the command somehow.

Anyway, I did find one problem which was that you have to POST to that URL in order for the delete to work. For example:
curl -vvv -d '' -u api_key: 'https://sonarhost/api/project_pull_requests/delete?project=projectkey&pullRequest=1336'

[mjaggard]

@mc1arke I think there is a bug here. If you create a pull request from one branch to another that is not scanned, it completely crashes the whole project in Sonarqube. In my case I created a new branch from a colleagues' slightly longer-running branch and it broke everyone's use of Sonar. I'm happy for my PR to not work but it shouldn't break the whole project

[mwallace72]

Would it be possible to get this change for v1.14, to provide this for Sonarqube 9.9 LTS? I'm not opposed to making the PR, assuming it's just the same code as the fix in v1.17, but this would be my first contribution so wanted to ask first.

[felipecrs]

That would be super nice for me as well.

[felipecrs]

@mc1arke is it very hard to release a newer version of the plugin for SonarQube 9.9 LTS?