Polyglot Sampler doesn't work with GraalVM for JDK 21 #537

JaroslavTulach · 2023-11-30T08:15:15Z

Describe the bug

I seem to be unable to use polyglot sampler with GraalVM 23.1 for JDK 21. My application puts Truffle as well as some libraries on --module-path, VisualVM 2.1.6 or 2.1.7 can open the application and shows "Polyglot Sampler" tab. But when switching to it, no button is enabled. I see following exception on console of my application:

[error] Nov 30, 2023 8:55:44 AM org.graalvm.visualvm.sampler.truffle.stagent.TruffleJMX agentmain
[error] SEVERE: null
[error] java.lang.ClassNotFoundException: com.oracle.graalvm.locator.GraalVMLocator
[error]         at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
[error]         at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
[error]         at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)
[error]         at org.graalvm.visualvm.sampler.truffle.stagent.TruffleClassLoader.getClass(TruffleClassLoader.java:74)
[error]         at org.graalvm.visualvm.sampler.truffle.stagent.TruffleClassLoader.getGraalVMLocatorLoaders(TruffleClassLoader.java:91)
[error]         at org.graalvm.visualvm.sampler.truffle.stagent.TruffleClassLoader.<init>(TruffleClassLoader.java:51)
[error]         at org.graalvm.visualvm.sampler.truffle.stagent.TruffleJMX.getSamplerClassLoader(TruffleJMX.java:143)
[error]         at org.graalvm.visualvm.sampler.truffle.stagent.TruffleJMX.agentmain(TruffleJMX.java:68)
[error]         at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
[error]         at java.base/java.lang.reflect.Method.invoke(Method.java:580)
[error]         at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:560)
[error]         at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndCallAgentmain(InstrumentationImpl.java:582)

I guess that's the cause. Btw. looks like that GraalVMLocator is gone in GraalVM for JDK 21. The class probably no longer exists.

To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Expected behavior

It'd be nice if polyglot sampling worked again with latest GraalVM.

Desktop (please complete the following information):

OS: Linux
GraalVM 23.1 for JDK 21
Version 2.1.6

The text was updated successfully, but these errors were encountered:

thurka · 2023-11-30T08:21:33Z

Please provide steps to reproduce. How do I run python or javascript?

JaroslavTulach · 2023-11-30T08:28:54Z

I guess this can be reproduced with any Truffle application. Originally I tried to use the sampler with Enso, but I can also reproduce it with my graal.js websocket repository:

graaljs-websocket$ git log | head -n1
commit 71f3187ad845517c0e6aa3c502ec52b85cc2b873
graaljs-websocket$ JAVA_HOME=~/bin/graalvm-community-openjdk-21+35.1/ mvn package exec:exec -DskipTests

that launches a JVM app with Graal.js. Then I connect to it via VisualVM and get the above mentioned exception. Here is even simpler application:

diff --git pom.xml pom.xml
new file mode 100644
index 0000000..94316f3
--- /dev/null
+++ pom.xml
@@ -0,0 +1,28 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+    <modelVersion>4.0.0</modelVersion>
+    <groupId>cz.xelfi.demo</groupId>
+    <artifactId>jsdemo</artifactId>
+    <version>1.0-SNAPSHOT</version>
+    <packaging>jar</packaging>
+    <properties>
+        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
+        <maven.compiler.source>11</maven.compiler.source>
+        <maven.compiler.target>11</maven.compiler.target>
+        <exec.mainClass>cz.xelfi.demo.jsdemo.Jsdemo</exec.mainClass>
+        <graalvm.version>23.1.1</graalvm.version>
+    </properties>
+    <dependencies>
+        <dependency>
+            <groupId>org.graalvm.polyglot</groupId>
+            <artifactId>polyglot</artifactId>
+            <version>${graalvm.version}</version>
+        </dependency>
+        <dependency>
+            <groupId>org.graalvm.polyglot</groupId>
+            <artifactId>js</artifactId>
+            <version>${graalvm.version}</version>
+            <type>pom</type>
+        </dependency>        
+    </dependencies>
+</project>
\ No newline at end of file
diff --git src/main/java/cz/xelfi/demo/jsdemo/Jsdemo.java src/main/java/cz/xelfi/demo/jsdemo/Jsdemo.java
new file mode 100644
index 0000000..a5f840c
--- /dev/null
+++ src/main/java/cz/xelfi/demo/jsdemo/Jsdemo.java
@@ -0,0 +1,14 @@
+package cz.xelfi.demo.jsdemo;
+
+import org.graalvm.polyglot.Context;
+
+public class Jsdemo {
+
+    public static void main(String[] args) throws Exception {
+        var ctx = Context.create("js");
+        for (var i = 0; ; i++) {
+            ctx.eval("js", "print('Hi " + i + " times');");
+            Thread.sleep(100);
+        }
+    }
+}

thurka · 2023-11-30T09:08:21Z

FYI: JVM Standalone from https://github.com/oracle/graaljs/releases/tag/graal-23.1.1 works fine.

JaroslavTulach · 2023-12-01T11:03:48Z

FYI: JVM Standalone from https://github.com/oracle/graaljs/releases/tag/graal-23.1.1 works fine.

Probably because there is modules/locator.jar in the standalone distribution. However (as pointed out by @Akirathan) there is no Maven distribution of that JAR. We haven't found any and there seems to be Maven : False setting in the build script - e.g. it is not easy for us to include that JAR in the Enso release.

thurka · 2023-12-06T09:33:57Z

It looks to me that the above test application is missing some dependencies, since I am not able to find com.oracle.truffle.tools.profiler.CPUSampler class in any of the maven jars. Is CPUSampler available somewhere?

JaroslavTulach · 2023-12-06T10:16:14Z

Do you mean this CPUSampler? That one seems to be available in org.graalvm.tool:profiler-tool package.

the above test application is missing some dependencies

Do you mean that without the com.oracle.truffle.tools.profiler.CPUSampler & co. classes VisualVM cannot "polyglot sample" a JVM? I have just tried to add the org.graalvm.tool:profiler-tool:23.1.1 dependency, but I am still getting the original java.lang.ClassNotFoundException: com.oracle.graalvm.locator.GraalVMLocator error (this time with VisualVM 2.1.5)...

thurka · 2023-12-06T13:08:06Z

Do you mean that without the com.oracle.truffle.tools.profiler.CPUSampler & co. classes VisualVM cannot "polyglot sample" a JVM?

Right.

I am still getting the original CNFE

Yes, this needs to be fixed, but the fix for CNFE will not help if CPUSampler is not available.

thurka · 2023-12-06T13:10:09Z

That one seems to be available in org.graalvm.tool:profiler-tool package.

Please update the testcase above. The tescase will never work without this dependency.

JaroslavTulach · 2023-12-06T14:29:44Z

Here are the updated steps, Tomáši. To reproduce the steps on a Mac OS X computer copy the following "diff" into clipboard:

diff --git a/pom.xml b/pom.xml
new file mode 100644
index 0000000..71649fc
--- /dev/null
+++ b/pom.xml
@@ -0,0 +1,34 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+    <modelVersion>4.0.0</modelVersion>
+    <groupId>cz.xelfi.demo</groupId>
+    <artifactId>jsdemo</artifactId>
+    <version>1.0-SNAPSHOT</version>
+    <packaging>jar</packaging>
+    <properties>
+        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
+        <maven.compiler.source>11</maven.compiler.source>
+        <maven.compiler.target>11</maven.compiler.target>
+        <exec.mainClass>cz.xelfi.demo.jsdemo.Jsdemo</exec.mainClass>
+        <graalvm.version>23.1.1</graalvm.version>
+    </properties>
+    <dependencies>
+        <dependency>
+            <groupId>org.graalvm.polyglot</groupId>
+            <artifactId>polyglot</artifactId>
+            <version>${graalvm.version}</version>
+        </dependency>
+        <dependency>
+            <groupId>org.graalvm.polyglot</groupId>
+            <artifactId>js</artifactId>
+            <version>${graalvm.version}</version>
+            <type>pom</type>
+        </dependency>        
+        <dependency>
+            <groupId>org.graalvm.tools</groupId>
+            <artifactId>profiler-tool</artifactId>
+            <version>${graalvm.version}</version>
+            <type>jar</type>
+        </dependency>        
+    </dependencies>
+</project>
\ No newline at end of file
diff --git a/src/main/java/cz/xelfi/demo/jsdemo/Jsdemo.java b/src/main/java/cz/xelfi/demo/jsdemo/Jsdemo.java
new file mode 100644
index 0000000..a5f840c
--- /dev/null
+++ b/src/main/java/cz/xelfi/demo/jsdemo/Jsdemo.java
@@ -0,0 +1,14 @@
+package cz.xelfi.demo.jsdemo;
+
+import org.graalvm.polyglot.Context;
+
+public class Jsdemo {
+
+    public static void main(String[] args) throws Exception {
+        var ctx = Context.create("js");
+        for (var i = 0; ; i++) {
+            ctx.eval("js", "print('Hi " + i + " times');");
+            Thread.sleep(100);
+        }
+    }
+}

and in a fresh directory:

$ patch -p1
<...paste the code and Ctrl+D...>
$ JAVA_HOME=/graalvm-community-openjdk-21.0.1+12.1/Contents/Home/ mvn package -DskipTests
...
[INFO] BUILD SUCCESS

The above builds the project without any dependency problems. Then I can execute it (from NetBeans) or via CLI:

$ JAVA_HOME=/graalvm-community-openjdk-21.0.1+12.1/Contents/Home/ mvn -Dexec.args="-classpath %classpath cz.xelfi.demo.jsdemo.Jsdemo" -Dexec.executable=/graalvm-community-openjdk-21.0.1+12.1/Contents/Home/bin/java exec:exec

Once the application is running, connect to it with VisualVM and you'll see the error:

pro 06, 2023 10:56:03 DOP. org.graalvm.visualvm.sampler.truffle.stagent.TruffleJMX agentmain
SEVERE: null
java.lang.ClassNotFoundException: com.oracle.graalvm.locator.GraalVMLocator
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)

The CPUSampler class shall however be available. At least its instrument seems to be available in the process:

thurka · 2023-12-08T08:14:52Z

Here are the updated steps, Tomáši.

Thanks. This looks OK, however there is still some classloader mismatch. :-(

thurka · 2023-12-08T12:59:35Z

It looks like there is clash between classpath and module-path. Engine class is loaded by two classloaders. One is loaded by module classloader and the second is loaded by system classloader. This also affects the CPUSampler class. The code below:

            Instrument sampler = ctx.getEngine().getInstruments().get("cpusampler");
            System.out.println("Sampler "+sampler.getName());
            System.out.println("CPUSampler "+sampler.lookup(CPUSampler.class));

prints

Sampler CPU Sampler
CPUSampler null

The solution would be to use only JPMS modules, since all maven dependecies are in fact JPMS modules.

JaroslavTulach · 2023-12-08T15:34:27Z

There are two ways to use Truffle Unchained. --module-path and --class-path see https://medium.com/graalvm/truffle-unchained-13887b77b62c - it'd be good if there was a how to use VisualVM with Truffle Unchained provided by OracleLabs.

solution would be to use only JPMS modules

It'd be ideal to support both: --class-path as well as --module-path. However Enso has already migrated to module-path, so if there is a way to get VisualVM working in --module-path mode, with published Maven artifacts (e.g. no locator), I'll be fine with that.

Just tell me what version/commit of VisualVM I need and what modules to include. Thank you.

thurka · 2023-12-11T08:34:15Z

AFAICT --class-path implementation looks suspisious. I was able to modify command-line for above example to use --module-path and the truffle sampler works using attached custom build of VisualVM.
BTW: How does one change pom.xml to use --module-path instead of classpath?

visualvm_GH-537.zip

JaroslavTulach · 2023-12-14T07:46:16Z

Hello Tomáši. I can confirm I can debug latest version of Enso with your custom VisualVM build. Thank you.

I've noticed that there are strange messages being printed on console when the VisualVM is attached:

enso.Range.fold<arg-2>(enso-dev/lib/Standard/Base/0.0.0-dev/src/Data/Range.enso:308) 2 true
enso.Standard.Base.Data.Range::Standard.Base.Data.Range.Range::fold(enso-dev/lib/Standard/Base/0.0.0-dev/src/Data/Range.enso:307) 2 true
enso.Standard.Base.Data.Vector::Standard.Base.Data.Vector.Vector::fold(enso-dev/lib/Standard/Base/0.0.0-dev/src/Data/Vector.enso:257) 2 true
enso.case_branch(enso-dev/lib/Standard/Test/0.0.0-dev/src/Bench.enso:134) 2 true
enso.Standard.Test.Bench::Standard.Test.Bench.Bench::fold(enso-dev/lib/Standard/Test/0.0.0-dev/src/Bench.enso:133) 2 true
enso.Standard.Test.Bench::Standard.Test.Bench.Bench::run_main(enso-dev/lib/Standard/Test/0.0.0-dev/src/Bench.enso:158) 2 true

is that expected in the custom build?

thurka · 2023-12-15T08:59:05Z

Hello Tomáši. I can confirm I can debug latest version of Enso with your custom VisualVM build.

OK. I need to do some more testing and I will fix it in master shortly.

is that expected in the custom build?

Yes, this is part of the debug output.

…Locator is not present

thurka · 2024-01-18T09:47:44Z

Classpath case should be fixed by just released GraalVM 23.1.2.

JaroslavTulach · 2024-01-29T08:57:45Z

What am I supposed to do to verify the fix, Tomáši?

Should I use some new version of VisualVM? Where should I download it from?
Should I upgrade Enso to GraalVM 23.1.2?

Thank you for your guidance. I tried enso-org/enso#8883 with VisualVM 2.1.6 but I don't think the polyglot sampler is working. The exception is the same #537 (comment) as originally reported.

thurka · 2024-02-01T11:39:51Z

What am I supposed to do to verify the fix, Tomáši?

You already verified it. Here: #537 (comment) .
Since you are using --module-path, you don't need to update to GraalVM 23.1.2. GraalVM 23.1.2 is needed for --class-path usecase.

JaroslavTulach added the bug label Nov 30, 2023

enso-bot bot mentioned this issue Nov 30, 2023

Unused arguments cause buildEngineDistribution to generate invalid caches and prevent tests requiring the affected file from running enso-org/enso#8384

Closed

enso-bot bot mentioned this issue Dec 7, 2023

Inconsistent type reported by Meta for [] enso-org/enso#8255

Closed

thurka added a commit that referenced this issue Dec 18, 2023

GH-537 improve lookup of contextImpl, fallback to defult when GraalVM…

5c8989e

…Locator is not present

thurka closed this as completed Jan 18, 2024

thurka self-assigned this Mar 13, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Polyglot Sampler doesn't work with GraalVM for JDK 21 #537

Polyglot Sampler doesn't work with GraalVM for JDK 21 #537

JaroslavTulach commented Nov 30, 2023

thurka commented Nov 30, 2023

JaroslavTulach commented Nov 30, 2023 •

edited

Loading

thurka commented Nov 30, 2023

JaroslavTulach commented Dec 1, 2023

thurka commented Dec 6, 2023

JaroslavTulach commented Dec 6, 2023

thurka commented Dec 6, 2023

thurka commented Dec 6, 2023

JaroslavTulach commented Dec 6, 2023 •

edited

Loading

thurka commented Dec 8, 2023

thurka commented Dec 8, 2023

JaroslavTulach commented Dec 8, 2023

thurka commented Dec 11, 2023

JaroslavTulach commented Dec 14, 2023

thurka commented Dec 15, 2023

thurka commented Jan 18, 2024

JaroslavTulach commented Jan 29, 2024

thurka commented Feb 1, 2024 •

edited

Loading

Polyglot Sampler doesn't work with GraalVM for JDK 21 #537

Polyglot Sampler doesn't work with GraalVM for JDK 21 #537

Comments

JaroslavTulach commented Nov 30, 2023

thurka commented Nov 30, 2023

JaroslavTulach commented Nov 30, 2023 • edited Loading

thurka commented Nov 30, 2023

JaroslavTulach commented Dec 1, 2023

thurka commented Dec 6, 2023

JaroslavTulach commented Dec 6, 2023

thurka commented Dec 6, 2023

thurka commented Dec 6, 2023

JaroslavTulach commented Dec 6, 2023 • edited Loading

thurka commented Dec 8, 2023

thurka commented Dec 8, 2023

JaroslavTulach commented Dec 8, 2023

thurka commented Dec 11, 2023

JaroslavTulach commented Dec 14, 2023

thurka commented Dec 15, 2023

thurka commented Jan 18, 2024

JaroslavTulach commented Jan 29, 2024

thurka commented Feb 1, 2024 • edited Loading

JaroslavTulach commented Nov 30, 2023 •

edited

Loading

JaroslavTulach commented Dec 6, 2023 •

edited

Loading

thurka commented Feb 1, 2024 •

edited

Loading