Specify a prefix for limiting the list of Gerrit Projects #391
Conversation
The GerritServer thread executes an unlimited gerrit ls-projects at startup which has two major issues: 1. May take a very long time, because the list of projects could be huge, even hundreds of thousands of repositories. 2. May cause trouble to the Gerrit Server that has to load *ALL* projects in memory to access their configuration and evaluate the ACLs. By introducing a projects prefix pattern, the GerritServer startup speed could be increased by 20x or even 100x, depending on the Gerrit server that is connecting to.
| * @param projectPrefix prefix of the projects to list from the Gerrit Server. | ||
| */ | ||
| @Exported | ||
| public void setProjectPrefix(String projectPrefix) { |
There was a problem hiding this comment.
Maybe I'm missing something, but where does this get called?
If it's not called from anywhere, the projectPrefix is uninitialized which results in the full list being returned, i.e. the same behavior as before.
There was a problem hiding this comment.
For backward compatibility, it will work as before.
Need to find a place to hook into it, do you have any suggestions? I’m not very familiar with this codebase :-(
There was a problem hiding this comment.
Yes this uses an outdated way of doing Jenkins form submission. See public void doConfigSubmit(StaplerRequest req, StaplerResponse rsp) throws ServletException... in the decriptor.
| @@ -54,8 +54,9 @@ | |||
| /** | |||
| * The command for fetching projects. | |||
| */ | |||
| public static final String GERRIT_LS_PROJECTS = "gerrit ls-projects"; | |||
| public static final String GERRIT_LS_PROJECTS = "gerrit ls-projects --prefix "; | |||
There was a problem hiding this comment.
Note: The --prefix option is not documented on the ls-projects command for 2.14, 2.15 or 2.16.
There was a problem hiding this comment.
True, this needs to be fixed in Gerrit though.
With regards to the support in Gerrit < 2.13, needs to be documented in this plugin and checked as soon as the version is queried.
There was a problem hiding this comment.
I've posted the documentation for review https://gerrit-review.googlesource.com/c/gerrit/+/212012
| @@ -239,7 +242,8 @@ private void tryLoadProjectList() { | |||
| activeConfig.getGerritProxy(), | |||
| activeConfig.getGerritAuthentication() | |||
| ); | |||
| List<String> projects = readProjects(sshConnection.executeCommandReader(GERRIT_LS_PROJECTS)); | |||
| List<String> projects = readProjects(sshConnection | |||
| .executeCommandReader(GERRIT_LS_PROJECTS + projectPrefix)); | |||
There was a problem hiding this comment.
Should we assume that htis works if projectPrefix is an empty string?
In that case I guess it won't work when it's null
The GerritServer thread executes an unlimited gerrit ls-projects
at startup which has two major issues:
May take a very long time, because the list of projects could
be huge, even hundreds of thousands of repositories.
May cause trouble to the Gerrit Server that has to load ALL
projects in memory to access their configuration and evaluate
the ACLs.
By introducing a projects prefix pattern, the GerritServer startup
speed could be increased by 20x or even 100x, depending on the
Gerrit server that is connecting to.