Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use JGit 7.0.0, require Java 17 and Jenkins 2.463 or newer #1170

Merged
merged 2 commits into from
Sep 12, 2024

Conversation

MarkEWaite
Copy link
Contributor

@MarkEWaite MarkEWaite commented Aug 8, 2024

Use JGit 7.0.0, require Java 17 and Jenkins 2.463 or newer

JGit 7.0.0 requires Java 17. Jenkins 2.463 requires Java 17.

Use the plugin bill of materials from 2.462.x because it is the closest we have to 2.463. The plugin is expected to work with any release 2.463 or later.

Checklist

  • I have read the CONTRIBUTING doc
  • I have referenced the Jira issue related to my changes in one or more commit messages
  • I have added tests that verify my changes
  • Unit tests pass locally with my changes
  • I have added documentation as necessary
  • No Javadoc warnings were introduced with my changes
  • No spotbugs warnings were introduced with my changes
  • I have interactively tested my changes

Types of changes

What types of changes does your code introduce? Put an x in the boxes that apply

  • New feature (non-breaking change which adds functionality)

@MarkEWaite MarkEWaite added the rfe Improvement or new feature label Aug 8, 2024
@MarkEWaite MarkEWaite force-pushed the test-jgit-7.0.0 branch 2 times, most recently from 3b8f18e to b97ab86 Compare August 9, 2024 20:23
@MarkEWaite MarkEWaite force-pushed the test-jgit-7.0.0 branch 3 times, most recently from 6bfa95c to c14430c Compare August 17, 2024 14:04
MarkEWaite added a commit to MarkEWaite/docker-lfs that referenced this pull request Aug 23, 2024
jenkinsci/git-client-plugin#1170
Test JGit 7.0.0 with Java 17

JGit 7.0.0.202408202050-m3
MarkEWaite added a commit to MarkEWaite/docker-lfs that referenced this pull request Sep 10, 2024
JGit 7.0.0 requires Java 17.  Jenkins 2.463 requires Java 17.

Use the plugin bill of materials from 2.462.x because it is the closest
we have to 2.463.  The plugin is expected to work with any release 2.463
or later.

Adapt to removed API's in JGit 7.0.0 by replacing calls to removed API's
with the recommended equivalents.

Tested in my development environment and no issues detected.  Tests run
from a container based Jenkins controller with Windows, Linux, and FreeBSD
agents.  Tests ran on multiple weekly releases of Jenkins since 2.463.
The Linux agents include:

* Alpine
* Debian Linux 11, 12, testing, and unstable
* openSUSE
* Red Hat Enterprise Linux 8
* Rocky Linux 9
* Ubuntu 20.04, 22.04, and 24.04
@MarkEWaite MarkEWaite changed the title Test JGit 7.0.0 with Java 17 Use JGit 7.0.0 and require Java 17 Sep 12, 2024
@MarkEWaite MarkEWaite marked this pull request as ready for review September 12, 2024 18:16
@MarkEWaite MarkEWaite requested a review from a team as a code owner September 12, 2024 18:16
@MarkEWaite
Copy link
Contributor Author

@jenkinsci/git-client-plugin-developers could you review and comment as necessary on this pull request?

I would like to merge and release this change by the end of this week. I've been testing JGit 7.0.0 pre-releases for a long time. Now that JGit 7.0.0 is released, I'd prefer to include it in the git client plugin.

The release with JGit 7.0.0 won't be shown to LTS users until the 30 Oct 2024 LTS release that requires Java 17.

@MarkEWaite MarkEWaite changed the title Use JGit 7.0.0 and require Java 17 Use JGit 7.0.0, require Java 17 and Jenkins 2.463 or newer Sep 12, 2024
@MarkEWaite MarkEWaite self-assigned this Sep 12, 2024
Copy link
Member

@olamy olamy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM Go for it!
But bear in mind Apache Mina SSHD will be released soon (before end of September)
I have done some testing the last few weeks and it was working fine with SNAPSHOT but just in case we need modification after the final release/

@MarkEWaite MarkEWaite merged commit c1c888d into master Sep 12, 2024
17 checks passed
@MarkEWaite MarkEWaite deleted the test-jgit-7.0.0 branch September 12, 2024 22:06
@MarkEWaite
Copy link
Contributor Author

MarkEWaite commented Sep 16, 2024

This change should have been preceded by a thorough review of the consumers of the JGit API within the Jenkins plugins. I've run a preliminary review using usage in plugins with the following signatures:

org.eclipse.jgit.api.CheckoutCommand#setForce
org.eclipse.jgit.api.RemoteRemoveCommand#setName
org.eclipse.jgit.api.RemoteSetUrlCommand#setName
org.eclipse.jgit.api.RemoteSetUrlCommand#setPush
org.eclipse.jgit.api.RemoteSetUrlCommand#setUri
org.eclipse.jgit.api.StashApplyCommand#setApplyIndex
org.eclipse.jgit.api.StashApplyCommand#setApplyUntracked
org.eclipse.jgit.dircache.DirCacheCheckout#checkoutEntry
org.eclipse.jgit.dircache.DirCacheEntry#getLastModified
org.eclipse.jgit.dircache.DirCacheEntry#mightBeRacilyClean
org.eclipse.jgit.dircache.DirCacheEntry#setLastModified
org.eclipse.jgit.internal.storage.file.FileSnapshot#lastModified
org.eclipse.jgit.internal.storage.file.FileSnapshot#save
org.eclipse.jgit.internal.storage.file.LockFile#getCommmitLastModified
org.eclipse.jgit.lib.CommitBuilder#setEncoding
org.eclipse.jgit.lib.CoreConfig#isLogAllRefUpdates
org.eclipse.jgit.lib.RefDatabase#getRef
org.eclipse.jgit.lib.Repository#hasObject
org.eclipse.jgit.lib.Repository#peel
org.eclipse.jgit.lib.TagBuilder#toByteArray
org.eclipse.jgit.merge.MergeFormatter#formatMerge
org.eclipse.jgit.pgm.ReceivePack#setEchoCommandFailures
org.eclipse.jgit.revwalk.ReachabilityChecker#areAllReachable
org.eclipse.jgit.revwalk.RevWalk#createReachabilityChecker
org.eclipse.jgit.src.org.eclipse.jgit.merge.MergeMessageFormatterformatWithConflicts
org.eclipse.jgit.src.org.eclipse.jgit.patch.PatchApplierapplyPatch
org.eclipse.jgit.storage.file.WindowCacheStats#getOpenBytes
org.eclipse.jgit.storage.file.WindowCacheStats#getOpenFiles
org.eclipse.jgit.transport.ReceivePack.FirstLine
org.eclipse.jgit.transport.RefAdvertiser#send
org.eclipse.jgit.transport.ssh.jsch.JschSession#getSftpChannel
org.eclipse.jgit.transport.Transport#getFilterBlobLimit
org.eclipse.jgit.transport.UploadPack#getFilterBlobLimit
org.eclipse.jgit.transport.UserAgent#getAgent
org.eclipse.jgit.treewalk.FileTreeIterator.Entry#getEntryLastModified
org.eclipse.jgit.treewalk.WorkingTreeIterator.Entry#getEntryLastModified
org.eclipse.jgit.treewalk.WorkingTreeIterator#getEntryLastModified
org.eclipse.jgit.util.FileUtils#lastModified
org.eclipse.jgit.util.FileUtils#setLastModified
org.eclipse.jgit.util.FS#createNewFile
org.eclipse.jgit.util.FS#getLastModifiedTime
org.eclipse.jgit.util.FS#lastModified
org.eclipse.jgit.util.FS#setAsyncFileStoreAttributes
org.eclipse.jgit.util.FS#setLastModified
org.eclipse.jgit.util.io.AutoLFInputStream

That set of removed method signatures leads to the following detected issues and their matching fixes:

The change also broke the git-changelog plugin but the break is not detected by usage in plugins because the git-changelog plugin uses a library that references JGit 6.x methods. Git changelog issue is:

Users of the git changelog plugin will need to remain with git client 5.0.0 and JGit 6.10.0 until they upgrade to a Jenkins version that requires Java 17 or newer. The git changelog plugin will require Java 17 or newer in an upcoming release because it depends on two libraries (Eclipse JGit 7.0.0 and Java Handlebars) that require Java 17.

MarkEWaite added a commit to MarkEWaite/last-changes-plugin that referenced this pull request Sep 16, 2024
https://issues.jenkins.io/browse/JENKINS-73781 notes that JGit
deprecated peel() in favor of getRefDatabase().peel() a long time ago.
The deprecated method was removed from JGit 7.0.0.  This uses the
replacement method that has been available since the deprecation.

jenkinsci/git-client-plugin#1170 is the git
client plugin pull request that introduced JGit 7.0.0.

https://github.com/jenkinsci/git-client-plugin/releases/tag/git-client-6.0.0
is the git client plugin release that introduced JGit 7.0.0.
@railsharipov
Copy link

Looks like these changes break Git Changelog plugin:

java.lang.IllegalAccessError: class se.bjurr.gitchangelog.internal.git.GitRepo tried to access private method 'org.eclipse.jgit.lib.Ref org.eclipse.jgit.lib.Repository.peel(org.eclipse.jgit.lib.Ref)' (se.bjurr.gitchangelog.internal.git.GitRepo is in unnamed module of loader 'PluginClassLoader for git-changelog' @568f4cfa; org.eclipse.jgit.lib.Repository is in unnamed module of loader 'PluginClassLoader for git-client' @479a864c)
 	at PluginClassLoader for git-changelog//se.bjurr.gitchangelog.internal.git.GitRepo.getPeeledObjectId(GitRepo.java:171)
 	at PluginClassLoader for git-changelog//se.bjurr.gitchangelog.internal.git.GitRepo.findRef(GitRepo.java:164)
 	at PluginClassLoader for git-changelog//se.bjurr.gitchangelog.internal.git.GitRepo.getRef(GitRepo.java:149)
 	at PluginClassLoader for git-changelog//se.bjurr.gitchangelog.internal.git.GitRepo.firstCommit(GitRepo.java:242)
 	at PluginClassLoader for git-changelog//se.bjurr.gitchangelog.internal.git.GitRepo.getCommit(GitRepo.java:104)
 	at PluginClassLoader for git-changelog//se.bjurr.gitchangelog.api.GitChangelogApi.getFrom(GitChangelogApi.java:776)
 	at PluginClassLoader for git-changelog//se.bjurr.gitchangelog.api.GitChangelogApi.getChangelog(GitChangelogApi.java:717)
 	at PluginClassLoader for git-changelog//se.bjurr.gitchangelog.api.GitChangelogApi.getChangelog(GitChangelogApi.java:90)
 	at PluginClassLoader for git-changelog//se.bjurr.gitchangelog.api.GitChangelogApi.getChangelog(GitChangelogApi.java:84)
 	at PluginClassLoader for git-changelog//org.jenkinsci.plugins.gitchangelog.steps.GitChangelogStep.perform(GitChangelogStep.java:451)
 	at PluginClassLoader for git-changelog//org.jenkinsci.plugins.gitchangelog.steps.GitChangelogStep$1$1.call(GitChangelogStep.java:353)
 	at hudson.remoting.UserRequest.perform(UserRequest.java:225)
 	at hudson.remoting.UserRequest.perform(UserRequest.java:50)
 	at hudson.remoting.Request$2.run(Request.java:391)
 	at hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:81)
 	at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
 	at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:140)
 	at java.base/java.lang.Thread.run(Unknown Source)
 Also:   org.jenkinsci.plugins.workflow.actions.ErrorAction$ErrorId: dc8858c2-7af4-492c-afea-45b0b9b25d4b
 Caused: java.io.IOException: Remote call on JNLP4-connect connection from ip-10-0-102-77.ec2.internal/10.0.102.77:56808 failed
 	at hudson.remoting.Channel.call(Channel.java:1116)
 	at hudson.FilePath.act(FilePath.java:1319)
 	at PluginClassLoader for git-changelog//org.jenkinsci.plugins.gitchangelog.steps.GitChangelogStep$1.run(GitChangelogStep.java:357)
 	at PluginClassLoader for workflow-step-api//org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
 	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
 	at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
 	at java.base/java.lang.Thread.run(Unknown Source)

@MarkEWaite
Copy link
Contributor Author

MarkEWaite commented Sep 16, 2024

Looks like these changes break Git Changelog plugin:

java.lang.IllegalAccessError: class se.bjurr.gitchangelog.internal.git.GitRepo tried to access private method 'org.eclipse.jgit.lib.Ref org.eclipse.jgit.lib.Repository.peel(org.eclipse.jgit.lib.Ref)' (se.bjurr.gitchangelog.internal.git.GitRepo is in unnamed module of loader 'PluginClassLoader for git-changelog' @568f4cfa; org.eclipse.jgit.lib.Repository is in unnamed module of loader 'PluginClassLoader for git-client' @479a864c)
 	at PluginClassLoader for git-changelog//se.bjurr.gitchangelog.internal.git.GitRepo.getPeeledObjectId(GitRepo.java:171)
 	at PluginClassLoader for git-changelog//se.bjurr.gitchangelog.internal.git.GitRepo.findRef(GitRepo.java:164)
 	at PluginClassLoader for git-changelog//se.bjurr.gitchangelog.internal.git.GitRepo.getRef(GitRepo.java:149)
 	at PluginClassLoader for git-changelog//se.bjurr.gitchangelog.internal.git.GitRepo.firstCommit(GitRepo.java:242)
 	at PluginClassLoader for git-changelog//se.bjurr.gitchangelog.internal.git.GitRepo.getCommit(GitRepo.java:104)
 	at PluginClassLoader for git-changelog//se.bjurr.gitchangelog.api.GitChangelogApi.getFrom(GitChangelogApi.java:776)
 	at PluginClassLoader for git-changelog//se.bjurr.gitchangelog.api.GitChangelogApi.getChangelog(GitChangelogApi.java:717)
 	at PluginClassLoader for git-changelog//se.bjurr.gitchangelog.api.GitChangelogApi.getChangelog(GitChangelogApi.java:90)
 	at PluginClassLoader for git-changelog//se.bjurr.gitchangelog.api.GitChangelogApi.getChangelog(GitChangelogApi.java:84)
 	at PluginClassLoader for git-changelog//org.jenkinsci.plugins.gitchangelog.steps.GitChangelogStep.perform(GitChangelogStep.java:451)
 	at PluginClassLoader for git-changelog//org.jenkinsci.plugins.gitchangelog.steps.GitChangelogStep$1$1.call(GitChangelogStep.java:353)
 	at hudson.remoting.UserRequest.perform(UserRequest.java:225)
 	at hudson.remoting.UserRequest.perform(UserRequest.java:50)
 	at hudson.remoting.Request$2.run(Request.java:391)
 	at hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:81)
 	at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
 	at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:140)
 	at java.base/java.lang.Thread.run(Unknown Source)
 Also:   org.jenkinsci.plugins.workflow.actions.ErrorAction$ErrorId: dc8858c2-7af4-492c-afea-45b0b9b25d4b
 Caused: java.io.IOException: Remote call on JNLP4-connect connection from ip-10-0-102-77.ec2.internal/10.0.102.77:56808 failed
 	at hudson.remoting.Channel.call(Channel.java:1116)
 	at hudson.FilePath.act(FilePath.java:1319)
 	at PluginClassLoader for git-changelog//org.jenkinsci.plugins.gitchangelog.steps.GitChangelogStep$1.run(GitChangelogStep.java:357)
 	at PluginClassLoader for workflow-step-api//org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
 	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
 	at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
 	at java.base/java.lang.Thread.run(Unknown Source)

Unfortunately, there is not much that I can do about that issue. The class se.bjurr.gitchangelog.internal.git.GitRepo in the se.bjurr.gitchangelog.git-changelog-lib library has made the same (flawed) decision that the git client plugin made. It has ignored the deprecation warnings from earlier JGit releases and will now need to be modified to adapt to the APIs that have been removed from JGit 7.0.0.

The changes in the git client plugin to adapt to JGit 7.0.0 were not large, but they did require changes.

The most direct workaround is to remain with Git client plugin 5.0.0 and not upgrade to Git client plugin 6.0.0 until a new version of git changelog plugin is released that has been adapted to replace the deprecated use of Repository.peel(Ref) with Repository.getRefDatabase().peel(Ref).

@MarkEWaite
Copy link
Contributor Author

MarkEWaite commented Sep 18, 2024

Looks like these changes break Git Changelog plugin:

Git changelog plugin has updated the dependent library and released a new version. Users running a recent Jenkins weekly can upgrade to the new git changelog plugin version. Users running Jenkins LTS 2.462.x won't see the issue because they are not offered git client plugin 6.0.0.

@railsharipov
Copy link

I confirm that issue with Git Changelog plugin has been resolved. Thank you, @MarkEWaite !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
rfe Improvement or new feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants