Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Very Long gitlabMergeRequestDescription environment variable breaks builds #1724

Open
Reamer opened this issue Nov 12, 2024 · 5 comments
Open

Very Long gitlabMergeRequestDescription environment variable breaks builds #1724

Reamer opened this issue Nov 12, 2024 · 5 comments

Comments

@Reamer
Copy link

Reamer commented Nov 12, 2024

Jenkins and plugins versions report

Environment 
Jenkins: 2.479.1
OS: Linux - 5.15.0-125-generic
Java: 21.0.5 - Ubuntu (OpenJDK 64-Bit Server VM)
---
antisamy-markup-formatter:162.v0e6ec0fcfcf6
apache-httpcomponents-client-4-api:4.5.14-208.v438351942757
apache-httpcomponents-client-5-api:5.4-124.v31e2987e48f4
asm-api:9.7.1-97.v4cc844130d97
authentication-tokens:1.119.v50285141b_7e1
authorize-project:1.8.1
blueocean-commons:1.27.16
blueocean-rest:1.27.16
bootstrap5-api:5.3.3-1
bouncycastle-api:2.30.1.78.1-248.ve27176eb_46cb_
branch-api:2.1197.vfa_d0c47c267d
build-monitor-plugin:1.14-925.v95b_9089a_4c7f
build-timeout:1.33
caffeine-api:3.1.8-133.v17b_1ff2e0599
checks-api:2.2.1
cloud-stats:336.v788e4055508b_
cloudbees-folder:6.955.v81e2a_35c08d3
command-launcher:115.vd8b_301cc15d0
commons-compress-api:1.26.1-2
commons-lang3-api:3.17.0-84.vb_b_938040b_078
commons-text-api:1.12.0-129.v99a_50df237f7
conditional-buildstep:1.4.3
config-file-provider:980.v88956a_a_5d6a_d
continuum-avm:1.10
cors-filter:1.1
credentials:1389.vd7a_b_f5fa_50a_2
credentials-binding:681.vf91669a_32e45
data-tables-api:2.1.8-1
display-url-api:2.209.v582ed814ff2f
docker-commons:443.v921729d5611d
docker-java-api:3.4.0-94.v65ced49b_a_7d5
docker-plugin:1.7.0
docker-workflow:580.vc0c340686b_54
durable-task:577.v2a_8a_4b_7c0247
echarts-api:5.5.1-4
eddsa-api:0.3.0-4.v84c6f0f4969e
extensible-choice-parameter:1.8.1
external-monitor-job:215.v2e88e894db_f8
font-awesome-api:6.6.0-2
git:5.6.0
git-client:6.1.0
git-parameter:0.10.0
git-server:126.v0d945d8d2b_39
gitlab-api:5.6.0-97.v6603a_83f8690
gitlab-branch-source:711.v41fb_c8b_2b_7a_e
gitlab-plugin:1.9.7-SNAPSHOT (private-422a2a46-pdallig)
google-oauth-plugin:1.330.vf5e86021cb_ec
gson-api:2.11.0-85.v1f4e87273c33
handy-uri-templates-2-api:2.1.8-30.v7e777411b_148
hashicorp-vault-plugin:371.v884a_4dd60fb_6
htmlpublisher:1.36
instance-identity:201.vd2a_b_5a_468a_a_6
ionicons-api:74.v93d5eb_813d5f
jackson2-api:2.17.0-379.v02de8ec9f64c
jakarta-activation-api:2.1.3-1
jakarta-mail-api:2.1.3-1
javadoc:280.v050b_5c849f69
javax-activation-api:1.2.0-7
javax-mail-api:1.6.2-10
jaxb:2.3.9-1
jdk-tool:80.v8a_dee33ed6f0
jersey2-api:2.44-151.v6df377fff741
jnr-posix-api:3.1.19-2
job-dsl:1.89
joda-time-api:2.13.0-93.v9934da_29b_a_e9
jquery:1.12.4-3
jquery3-api:3.7.1-2
jsch:0.2.16-86.v42e010d9484b_
json-api:20240303-101.v7a_8666713110
json-path-api:2.9.0-118.v7f23ed82a_8b_8
junit:1307.vdd5b_2646279e
kubernetes:4295.v7fa_01b_309c95
kubernetes-client-api:6.10.0-240.v57880ce8b_0b_2
kubernetes-credentials:190.v03c305394deb_
ldap:725.v3cb_b_711b_1a_ef
lockable-resources:1327.ved786b_a_197e0
m2release:0.16.4
mailer:488.v0c9639c1a_eb_3
matrix-auth:3.2.3
matrix-project:840.v812f627cb_578
maven-artifact-choicelistprovider:1.17
maven-plugin:3.24
metrics:4.2.21-458.vcf496cb_839e4
mina-sshd-api-common:2.14.0-133.vcc091215a_358
mina-sshd-api-core:2.14.0-133.vcc091215a_358
mission-control-view:0.9.16
oauth-credentials:0.653.v14cf2088e950
oic-auth:4.418.vccc7061f5b_6d
okhttp-api:4.11.0-172.vda_da_1feeb_c6e
openshift-client:1.1.0.424.v829cb_ccf8798
pam-auth:1.11
parameterized-trigger:806.vf6fff3e28c3e
pipeline-aggregator-view:104.v94a_e5f6cdb_c3
pipeline-build-step:540.vb_e8849e1a_b_d8
pipeline-graph-analysis:216.vfd8b_ece330ca_
pipeline-groovy-lib:744.v5b_556ee7c253
pipeline-input-step:495.ve9c153f6067b_
pipeline-milestone-step:119.vdfdc43fc3b_9a_
pipeline-model-api:2.2214.vb_b_34b_2ea_9b_83
pipeline-model-definition:2.2214.vb_b_34b_2ea_9b_83
pipeline-model-extensions:2.2214.vb_b_34b_2ea_9b_83
pipeline-rest-api:2.34
pipeline-stage-step:312.v8cd10304c27a_
pipeline-stage-tags-metadata:2.2214.vb_b_34b_2ea_9b_83
pipeline-stage-view:2.34
pipeline-utility-steps:2.18.0
plain-credentials:183.va_de8f1dd5a_2b_
plugin-util-api:5.1.0
run-condition:1.7
scm-api:698.v8e3b_c788f0a_6
script-security:1367.vdf2fc45f229c
snakeyaml-api:2.3-123.v13484c65210a_
sonar:2.17.2
ssh-agent:376.v8933585c69d3
ssh-credentials:343.v884f71d78167
ssh-slaves:2.973.v0fa_8c0dea_f9f
sshd:3.330.vc866a_8389b_58
structs:338.v848422169819
token-macro:400.v35420b_922dcb_
trilead-api:2.147.vb_73cc728a_32e
variant:60.v7290fc0eb_b_cd
workflow-aggregator:600.vb_57cdd26fdd7
workflow-api:1336.vee415d95c521
workflow-basic-steps:1058.vcb_fc1e3a_21a_9
workflow-cps:3990.vd281dd77a_388
workflow-durable-task-step:1371.vb_7cec8f3b_95e
workflow-job:1468.vcf4f5ee92395
workflow-multibranch:795.ve0cb_1f45ca_9a_
workflow-scm-step:427.v4ca_6512e7df1
workflow-step-api:678.v3ee58b_469476
workflow-support:930.vf51d22b_ce488

What Operating System are you using (both controller, and any agents involved in the problem)?

Ubuntu 22.04

Reproduction steps

  1. Create a Merge-Request with a really huge description
  2. Trigger a Jenkins Pipeline-Job via Webhook-Push

Expected Results

The job runs smoothly

Actual Results

git execution fails

Triggered by GitLab Merge Request #21: misc/renovate/version.quarkus => master
Checking out git [email protected]:My-Team/misc/ci-reference.git into /avm/data/jenkins/jobs/misc/jobs/ci-reference-master/workspace@script/f2973eb2a4b1f1753a9f45b0d858da66788cb040b63b64c7c628dd40ee584815 to read Jenkinsfile
The recommended git tool is: NONE
using credential jenkins-sv-01
 > git rev-parse --resolve-git-dir /avm/data/jenkins/jobs/misc/jobs/ci-reference-master/workspace@script/f2973eb2a4b1f1753a9f45b0d858da66788cb040b63b64c7c628dd40ee584815/.git # timeout=10
ERROR: Workspace has a .git repository, but it appears to be corrupt.
hudson.plugins.git.GitException: Error performing git command: git rev-parse --resolve-git-dir /avm/data/jenkins/jobs/misc/jobs/ci-reference-master/workspace@script/f2973eb2a4b1f1753a9f45b0d858da66788cb040b63b64c7c628dd40ee584815/.git
	at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2862)
	at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2766)
	at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2761)
	at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:2051)
	at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:2063)
	at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAPIImpl.hasGitRepo(CliGitAPIImpl.java:405)
	at PluginClassLoader for git-client//hudson.plugins.git.GitAPI.hasGitRepo(GitAPI.java:281)
	at PluginClassLoader for git//hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1205)
	at PluginClassLoader for git//hudson.plugins.git.GitSCM._checkout(GitSCM.java:1310)
	at PluginClassLoader for git//hudson.plugins.git.GitSCM.checkout(GitSCM.java:1277)
	at PluginClassLoader for workflow-scm-step//org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:136)
	at PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:167)
	at PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:73)
	at PluginClassLoader for workflow-job//org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:311)
	at hudson.model.ResourceController.execute(ResourceController.java:101)
	at hudson.model.Executor.run(Executor.java:445)
Caused by: java.io.IOException: Cannot run program "git" (in directory "/avm/data/jenkins/jobs/misc/jobs/ci-reference-master/workspace@script/f2973eb2a4b1f1753a9f45b0d858da66788cb040b63b64c7c628dd40ee584815"): error=7, Argument list too long
	at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1170)
	at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1089)
	at hudson.Proc$LocalProc.<init>(Proc.java:252)
	at hudson.Proc$LocalProc.<init>(Proc.java:221)
	at hudson.Launcher$LocalLauncher.launch(Launcher.java:994)
	at hudson.Launcher$ProcStarter.start(Launcher.java:506)
	at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2839)
	... 15 more
Caused by: java.io.IOException: error=7, Argument list too long
	at java.base/java.lang.ProcessImpl.forkAndExec(Native Method)
	at java.base/java.lang.ProcessImpl.<init>(ProcessImpl.java:295)
	at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:225)
	at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1126)
	... 21 more
Cloning the remote Git repository
Cloning repository [email protected]:My-Team/misc/ci-reference.git
 > git init /avm/data/jenkins/jobs/misc/jobs/ci-reference-master/workspace@script/f2973eb2a4b1f1753a9f45b0d858da66788cb040b63b64c7c628dd40ee584815 # timeout=10
ERROR: Error cloning remote repo 'origin'
hudson.plugins.git.GitException: Could not init /avm/data/jenkins/jobs/misc/jobs/ci-reference-master/workspace@script/f2973eb2a4b1f1753a9f45b0d858da66788cb040b63b64c7c628dd40ee584815
	at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAPIImpl$5.execute(CliGitAPIImpl.java:1073)
	at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAPIImpl$2.execute(CliGitAPIImpl.java:819)
	at PluginClassLoader for git//hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1220)
	at PluginClassLoader for git//hudson.plugins.git.GitSCM._checkout(GitSCM.java:1310)
	at PluginClassLoader for git//hudson.plugins.git.GitSCM.checkout(GitSCM.java:1277)
	at PluginClassLoader for workflow-scm-step//org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:136)
	at PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:167)
	at PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:73)
	at PluginClassLoader for workflow-job//org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:311)
	at hudson.model.ResourceController.execute(ResourceController.java:101)
	at hudson.model.Executor.run(Executor.java:445)
Caused by: hudson.plugins.git.GitException: Error performing git command: git init /avm/data/jenkins/jobs/misc/jobs/ci-reference-master/workspace@script/f2973eb2a4b1f1753a9f45b0d858da66788cb040b63b64c7c628dd40ee584815
	at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2862)
	at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2766)
	at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2761)
	at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:2051)
	at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAPIImpl$5.execute(CliGitAPIImpl.java:1071)
	... 10 more
Caused by: java.io.IOException: Cannot run program "git" (in directory "/avm/data/jenkins/jobs/misc/jobs/ci-reference-master/workspace@script/f2973eb2a4b1f1753a9f45b0d858da66788cb040b63b64c7c628dd40ee584815"): error=7, Argument list too long
	at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1170)
	at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1089)
	at hudson.Proc$LocalProc.<init>(Proc.java:252)
	at hudson.Proc$LocalProc.<init>(Proc.java:221)
	at hudson.Launcher$LocalLauncher.launch(Launcher.java:994)
	at hudson.Launcher$ProcStarter.start(Launcher.java:506)
	at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2839)
	... 14 more
Caused by: java.io.IOException: error=7, Argument list too long
	at java.base/java.lang.ProcessImpl.forkAndExec(Native Method)
	at java.base/java.lang.ProcessImpl.<init>(ProcessImpl.java:295)
	at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:225)
	at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1126)
	... 20 more
ERROR: Error cloning remote repo 'origin'
ERROR: Maximum checkout retry attempts reached, aborting
ERROR: Error cloning remote repo 'origin'
ERROR: Maximum checkout retry attempts reached, aborting
Finished: FAILURE

Anything else?

I only found out by luck that it was due to the long description of the merge request. The background is that I use Renovate. Renovate stores the change log of the libraries in the merge request description. Change logs can become very long.

To solve the problem, I have made the following code change. I know it's a little hack. The 1000 has no deeper meaning except to not overload the environment variable. I would prefer to remove the environment variable completely, but in my opinion this should be configurable. Perhaps some users evaluate the variable.

diff --git a/src/main/java/com/dabsquared/gitlabjenkins/cause/CauseData.java b/src/main/java/com/dabsquared/gitlabjenkins/cause/CauseData.java
index eff3950..b04b58b 100644
--- a/src/main/java/com/dabsquared/gitlabjenkins/cause/CauseData.java
+++ b/src/main/java/com/dabsquared/gitlabjenkins/cause/CauseData.java
@@ -174,7 +174,7 @@ public final class CauseData {
         variables.put("gitlabSourceRepoHttpUrl", sourceRepoHttpUrl);
         variables.putIfNotNull("gitlabMergeCommitSha", mergeCommitSha);
         variables.put("gitlabMergeRequestTitle", mergeRequestTitle);
-        variables.put("gitlabMergeRequestDescription", mergeRequestDescription);
+        variables.put("gitlabMergeRequestDescription", mergeRequestDescription.substring(0, Math.min(mergeRequestDescription.length(), 1000)));
         variables.put("gitlabMergeRequestId", mergeRequestId == null ? "" : mergeRequestId.toString());
         variables.put("gitlabMergeRequestIid", mergeRequestIid == null ? "" : mergeRequestIid.toString());
         variables.put(

Are you interested in contributing a fix?

How would you solve the problem? I would be happy to try to submit a pull request.
@krisstern
Copy link
Member

Hi @Reamer, thank you for reporting this bug! We would love to have you submit a patch for this via a PR.

@Reamer
Copy link
Author

Reamer commented Nov 13, 2024

What kind of pull request should I aim for?

  1. The little hack to limit the size to 1000 or an other quite small value
  2. Drop gitlabMergeRequestDescription from the env variables
  3. Drop gitlabMergeRequestDescription from env variables, which a additional plugin configuration

@krisstern
Copy link
Member

Hi @Reamer my apologies for the delayed response! I personally prefer option 3: "Drop gitlabMergeRequestDescription from env variables, with an additional plugin configuration". But from your personal experiences what would be your preference? What works best in your use case?

@Reamer
Copy link
Author

Reamer commented Nov 24, 2024

From my point of view, the description should not be passed on as an environment variable, so I would simply drop it. Option 2.
I don't think it makes sense to base a program logic on this, but I will try option three.

@krisstern
Copy link
Member

Hi @Reamer,

Sure, sounds good to me!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Reamer @krisstern