Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Truffle TCK libraryDependencies to runtime-test-instruments. #8766

Merged
merged 1 commit into from
Jan 16, 2024

Conversation

Akirathan
Copy link
Member

Pull Request Description

Since #8685, there is the following error (warning) message from JPMS plugin when building runtime-test-instruments:

[error] Returned (10): Vector(/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/sdk/nativeimage/23.1.0/nativeimage-23.1.0.jar, /home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/sdk/word/23.1.0/word-23.1.0.jar, /home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/sdk/jniutils/23.1.0/jniutils-23.1.0.jar, /home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/sdk/collections/23.1.0/collections-23.1.0.jar, /home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/polyglot/polyglot/23.1.0/polyglot-23.1.0.jar, /home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/truffle/truffle-api/23.1.0/truffle-api-23.1.0.jar, /home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/truffle/truffle-runtime/23.1.0/truffle-runtime-23.1.0.jar, /home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/truffle/truffle-compiler/23.1.0/truffle-compiler-23.1.0.jar, /home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/sdk/polyglot-tck/23.1.0/polyglot-tck-23.1.0.jar, /home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/netbeans/api/org-openide-util-lookup/RELEASE180/org-openide-util-lookup-RELEASE180.jar)
[error] Expected: (13): List(org.graalvm.sdk:nativeimage:23.1.0, org.graalvm.sdk:word:23.1.0, org.graalvm.sdk:jniutils:23.1.0, org.graalvm.sdk:collections:23.1.0, org.graalvm.polyglot:polyglot:23.1.0, org.graalvm.truffle:truffle-api:23.1.0, org.graalvm.truffle:truffle-runtime:23.1.0, org.graalvm.truffle:truffle-compiler:23.1.0, org.graalvm.sdk:polyglot-tck:23.1.0, org.graalvm.truffle:truffle-tck:23.1.0, org.graalvm.truffle:truffle-tck-common:23.1.0, org.graalvm.truffle:truffle-tck-tests:23.1.0, org.netbeans.api:org-openide-util-lookup:RELEASE180)

This PR removes this error message by providing appropriate libraryDependencies to runtime-test-instruments.

Important Notes

Checklist

Please ensure that the following checklist has been satisfied before submitting the PR:

  • Make sure that the error message is no longer present in the job output.

Thi removes the error message that JPMS plugin cannot find these modules from the update report.
@Akirathan Akirathan self-assigned this Jan 15, 2024
@Akirathan Akirathan added the CI: No changelog needed Do not require a changelog entry for this PR. label Jan 15, 2024
Copy link
Collaborator

@hubertp hubertp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

"org.graalvm.sdk" % "polyglot-tck" % graalMavenPackagesVersion,
"org.netbeans.api" % "org-openide-util-lookup" % netbeansApiVersion % "provided"
"org.graalvm.sdk" % "polyglot-tck" % graalMavenPackagesVersion,
"org.graalvm.truffle" % "truffle-tck" % graalMavenPackagesVersion,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still think all of those should be "provided"

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe we don't have to care in this case, because runtime-test-instruments project is used exclusively in tests.

@Akirathan Akirathan added the CI: Ready to merge This PR is eligible for automatic merge label Jan 15, 2024
Copy link
Member

@JaroslavTulach JaroslavTulach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for fixing my mistake. I knew not everything is right, but I had no idea how to fix it.

appropriate libraryDependencies to runtime-test-instruments.

Can you elaborate and describe what does it mean appropriate library dependencies? Does there have to be a transitive closure of dependencies?

At the end everything runs - e.g. the libraryDependencies cannot be completely incorrect, right?

@Akirathan
Copy link
Member Author

Can you elaborate and describe what does it mean appropriate library dependencies?

@JaroslavTulach The error message mentioned in the description comes from the JPMSUtils. More specifically, from this method

/** Filters all the requested modules from the given [[UpdateReport]].
*
* @param updateReport The update report to filter. This is the result of `update.value`.
* @param modules The modules to filter from the update report. Can be duplicated.
* @param log The logger to use for logging.
* @param shouldContainAll If true, the method will log an error if not all modules were found.
* @return The list of files (Jar archives, directories, etc.) that were found in the update report.
*/
def filterModulesFromUpdate(
updateReport: UpdateReport,
modules: Seq[ModuleID],
log: sbt.util.Logger,
shouldContainAll: Boolean = false
): Seq[File] = {
val distinctModules = modules.distinct
def shouldFilterModule(module: ModuleID): Boolean = {
distinctModules.exists(m =>
m.organization == module.organization &&
m.name == module.name &&
m.revision == module.revision
)
}
val foundFiles = updateReport.select(
module = shouldFilterModule
)
if (shouldContainAll) {
if (foundFiles.size < distinctModules.size) {
log.error("Not all modules from update were found")
log.error(s"Returned (${foundFiles.size}): $foundFiles")
log.error(s"Expected: (${distinctModules.size}): $distinctModules")
}
}
foundFiles
}

When you set modulePath to a list of modules, JPMSPlugin then tries to find these modules on the disk. There is no simple way how to tell the sbt to download a module (moduleID, for example org.graalvm.truffle % truffle-api % "23.1.0") if necessary. JPMSPlugin looks at every module in modulePath, tries to find it on the disk, and assembles the --module-path option. It expects that all the modules are on the disk. The only way to ensure that the modules are on the disk, is to put them inside libraryDependencies as well.

There may be some method in the sbt-librarymanagement API that does exactly that, but I was unable to find that. Hence, this workaround, and this error message.

Does there have to be a transitive closure of dependencies?

As mentioned in the comment here

/** Manually maintained GraalVM languages and their dependencies. Optimally,
* we would use 'org.graalvm.polyglot:js-community' or 'org.graavm.polyglot:python-community'
* maven artifacts and all their transitive dependencies, but we have to copy all these artifacts
* into engine distribution build, so we have to maintain these manually.
*/
we cannot use simply org.graalvm.polyglot % js-community % "23.1.0", which contains a parent POM that has transitive dependencies for js language, because, once again, I was unable to find an appropriate method in the sbt-librarymanagement API that would solve this issue for me. So for now, we need to look at all the transitive dependencies of these modules and manually maintain them.

@mergify mergify bot merged commit 04b0d26 into develop Jan 16, 2024
26 of 28 checks passed
@mergify mergify bot deleted the wip/akirathan/fix-jpms-error-msg branch January 16, 2024 10:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI: No changelog needed Do not require a changelog entry for this PR. CI: Ready to merge This PR is eligible for automatic merge
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants