Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

J9 (1.8.0_275-b01) crashed while J9 (9.0.4) and J9 (11.0.9+11) did not. #11683

Closed
fuzzy000 opened this issue Jan 19, 2021 · 15 comments · Fixed by #11753
Closed

J9 (1.8.0_275-b01) crashed while J9 (9.0.4) and J9 (11.0.9+11) did not. #11683

fuzzy000 opened this issue Jan 19, 2021 · 15 comments · Fixed by #11753

Comments

@fuzzy000
Copy link

Java -version output

We used the following versions of J9:

openjdk version "1.8.0_275"
OpenJDK Runtime Environment (build 1.8.0_275-b01)
Eclipse OpenJ9 VM (build openj9-0.23.0, JRE 1.8.0 Linux amd64-64-Bit Compressed References 20201110_845 (JIT enabled, AOT enabled)
OpenJ9   - 0394ef754
OMR      - 582366ae5
JCL      - b52d2ff7ee based on jdk8u275-b01)
openjdk version "9.0.4-adoptopenjdk"
OpenJDK Runtime Environment (build 9.0.4-adoptopenjdk+12)
Eclipse OpenJ9 VM (build openj9-0.9.0, JRE 9 Linux amd64-64-Bit Compressed References 20180814_248 (JIT enabled, AOT enabled)
OpenJ9   - 24e53631
OMR      - fad6bf6e
JCL      - feec4d2ae based on jdk-9.0.4+12)
openjdk version "11.0.9" 2020-10-20
OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.9+11)
Eclipse OpenJ9 VM AdoptOpenJDK (build openj9-0.23.0, JRE 11 Linux amd64-64-Bit Compressed References 20201022_810 (JIT enabled, AOT enabled)
OpenJ9   - 0394ef754
OMR      - 582366ae5
JCL      - 3b09cfd7e9 based on jdk-11.0.9+11)

Summary of problem

We made some changes to a class file in the JUnit project, and let several JVMs execute the corresponding JUnit test. We found that there are 6 classes caused the J9 (1.8.0_275-b01) to crash. For each crash, four files appears: core.xxx.dmp, javacore.xxx.txt, jitdump.xxx.dmp and Snap.xxx.trc.

Diagnostic files

j9-8-crash.zip

Steps to reproduce the behavior:

  1. extract the j9-8-crash.zip
  2. In directory j9-8-crash, run command cp Checks1.class sootOutput/junit-junit/org/junit/internal/Checks.class. You can also cp Checks2, Checks3, etc.
  3. java -cp sootOutput/junit-junit/:hamcrest-all-1.3.jar:junit-4.12.jar org.junit.runner.JUnitCore org.junit.internal.ChecksTest

Execution environment

  • OS and version: Ubuntu 16.04.6 LTS
  • CPU model: Intel(R) Xeon(R) CPU E5-4610 v4 @ 1.80GHz
  • Number of CPU cores: 4CPUs, each has 10 cores
  • Size of physical memory: 16384 MB * 20
@JasonFengJ9
Copy link
Member

Run the testcase against JDK8 build 1.8.0_275-b01 (at Linux x64 & Mac OS), but got following output:

JUnit version 4.13
.<org.junit.internal.Checks: java.lang.Object notNull(java.lang.Object)> **** Executed Line: **** 1 **** if r0 != null goto return r0
E...<org.junit.internal.Checks: java.lang.Object notNull(java.lang.Object,java.lang.String)> **** Executed Line: **** 2 **** if r0 != null goto return r0
.<org.junit.internal.Checks: java.lang.Object notNull(java.lang.Object,java.lang.String)> **** Executed Line: **** 2 **** if r0 != null goto return r0
E
Time: 0.047
There were 2 failures:
1) notNullShouldThrowOnNullValues(org.junit.internal.ChecksTest)
java.lang.AssertionError: NullPointerException expected
	at org.junit.Assert.fail(Assert.java:89)
	at org.junit.internal.ChecksTest.notNullShouldThrowOnNullValues(ChecksTest.java:26)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:61)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.junit.runners.Suite.runChild(Suite.java:128)
	at org.junit.runners.Suite.runChild(Suite.java:27)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
	at org.junit.runner.JUnitCore.runMain(JUnitCore.java:77)
	at org.junit.runner.JUnitCore.main(JUnitCore.java:36)
2) notNullWithMessageShouldThrowOnNullValues(org.junit.internal.ChecksTest)
java.lang.AssertionError: message does not match expected:<woops> but was:<null>
	at org.junit.Assert.fail(Assert.java:89)
	at org.junit.Assert.failNotEquals(Assert.java:835)
	at org.junit.Assert.assertEquals(Assert.java:120)
	at org.junit.internal.ChecksTest.notNullWithMessageShouldThrowOnNullValues(ChecksTest.java:47)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:61)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.junit.runners.Suite.runChild(Suite.java:128)
	at org.junit.runners.Suite.runChild(Suite.java:27)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
	at org.junit.runner.JUnitCore.runMain(JUnitCore.java:77)
	at org.junit.runner.JUnitCore.main(JUnitCore.java:36)

FAILURES!!!
Tests run: 5,  Failures: 2

There was no crash and no core files generated.

@fuzzy000 anything else required to reproduce the crash?

@fuzzy000
Copy link
Author

fuzzy000 commented Jan 19, 2021

@JasonFengJ9
We tried this on another server it also did not crash. But on the original server, the crash can be reproduced. We are thinking about if it is related to the Linux version. So we checked the Linux versions of those two server:

On the original server (which can reproduce the crash):

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.04.7 LTS
Release:        16.04
Codename:       xenial

On the server that can not reproduce:

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.4 LTS
Release:        18.04
Codename:       bionic

Could you please try it on the same Ubuntu version? If you need any additional information, please feel free to let me know.

Also, here inside the zip is the core.xxx.dmp, javacore.xxx.txt, jitdump.xxx.dmp and Snap.xxx.trc. I hope it useful for you.
jitdump.zip

@JasonFengJ9
Copy link
Member

> !gpinfo
Failing Thread: !j9vmthread 0x45d00
Failing Thread ID: 0x9abd (39613)
gpInfo:
J9Generic_Signal_Number=00000018 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000001
Handler1=00007FF6A2A1A3F0 Handler2=00007FF6A22FEF90 InaccessibleAddress=000000004809EE18
RDI=00000000000E0003 RSI=00000000000A1100 RAX=000000004809EE00 RBX=00007FF6A0A6C3F2
RCX=00000000000A1100 RDX=00000000000A1100 R8=00000007FFFB4828 R9=00007FF69C194DC0
R10=0000000000000000 R11=000000004809EE00 R12=0000000000093C80 R13=00000000FFFB4810
R14=0000000000000001 R15=00007FF6A3957870
RIP=00007FF69BB2F1C4 GS=0000 FS=0000 RSP=0000000000112B98
EFlags=0000000000010246 CS=0033 RBP=0000000000045D00 ERR=0000000000000004
TRAPNO=000000000000000E OLDMASK=0000000000000000 CR2=000000004809EE18
xmm0 0000000000000005 (f: 5.000000, d: 2.470328e-323)
xmm1 6c2f6176616a4c28 (f: 1634356224.000000, d: 1.320535e+213)
xmm2 00007ff6a39576d0 (f: 2744481536.000000, d: 6.951369e-310)
xmm3 76616a286c6c754e (f: 1819047296.000000, d: 1.713648e+262)
xmm4 616a2c7463656a62 (f: 1667590784.000000, d: 1.839891e+161)
xmm5 3e29676e69727453 (f: 1769108608.000000, d: 2.957418e-09)
xmm6 0000000700000001 (f: 1.000000, d: 1.485397e-313)
xmm7 00000007fff1f408 (f: 4294046720.000000, d: 1.697551e-313)
xmm8 6a203a736b636568 (f: 1801676160.000000, d: 1.590013e+203)
xmm9 207463656a624f2e (f: 1784827648.000000, d: 2.433002e-152)
xmm10 6e616c2e6176616a (f: 1635148160.000000, d: 5.038246e+223)
xmm11 616c2e6176616a2c (f: 1986095616.000000, d: 1.981011e+161)
xmm12 2a2a2a2a203e2967 (f: 540944768.000000, d: 1.426026e-105)
xmm13 203a656e694c2064 (f: 1766596736.000000, d: 1.968720e-153)
xmm14 72206669202a2a2a (f: 539634240.000000, d: 5.467786e+241)
xmm15 72206f746f67206c (f: 1869029504.000000, d: 5.479564e+241)
Module=/Volume2/mingyuanwu/JVMs/j9-8u275-b01/jre/lib/amd64/compressedrefs/libj9jit29.so
Module_base_address=00007FF69B1D2000

It was a crash at libj9jit29.so.

@pshipton pshipton added this to the Release 0.25 (Java 16) milestone Jan 19, 2021
@pshipton
Copy link
Member

@fuzzy000
Copy link
Author

@pshipton We also got a crash for the j9-8u282-b07 (0.24 release), here are the dump files: j9-8u282-dmp.zip

We think this crash is possibly related to the server load. Our server that got those crashes was running lots of CPU tasks at the same time.

@pshipton
Copy link
Member

@gacholio do you have cycles to look at the latest core?

@gacholio
Copy link
Contributor

gacholio commented Jan 21, 2021

Reproduced locally on an internal build:

rsi            0xd6dca500	3604784384

(gdb) x/20i 0x00007FFFDB2077C0
   0x7fffdb2077c0:	mov    0x8(%rsp),%rax
   0x7fffdb2077c5:	cmp    0x50(%rbp),%rsp
   0x7fffdb2077c9:	jbe    0x7fffdb207803
   0x7fffdb2077cb:	sub    $0x18,%rsp
   0x7fffdb2077cf:	mov    %r9,0x10(%rsp)
   0x7fffdb2077d4:	mov    %rax,0x20(%rsp)
   0x7fffdb2077d9:	mov    %rax,%rsi
=> 0x7fffdb2077dc:	mov    (%rsi),%ecx
   0x7fffdb2077de:	and    $0xffffffffffffff00,%rcx
 
> J9JITExceptionTable at 0x00007FFFEB2C8938 {
    struct J9UTF8* className = !j9utf8 0x00007FFFEB50455E   // java/lang/Object
    struct J9UTF8* methodName = !j9utf8 0x00007FFFEB504400   // <init>
    struct J9UTF8* methodSignature = !j9utf8 0x00007FFFEB504408   // ()V
    struct J9ConstantPool* constantPool = !j9constantpool 0x000000000003B460 
    struct J9Method* ramMethod = !j9method 0x000000000003B2C0   // java/lang/Object.<init>()V
    UDATA parm.startPC = 0x00007FFFDB2077C0;
    UDATA parm.endWarmPC = 0x00007FFFDB20780F;
    UDATA parm.startColdPC = 0x0000000000000000;
    UDATA parm.endPC = 0x00007FFFDB20780F;
    UDATA parm.totalFrameSize = 0x0000000000000003;
    I_16 parm.slots = 0x0001;
    I_16 parm.scalarTempSlots = 0x0000;
    I_16 parm.objectTempSlots = 0x0000;
    U_16 parm.prologuePushes = 0x0001;
    I_16 parm.tempOffset = 0x0000;
    U_16 parm.numExcptionRanges = 0x0000;
    I_32 parm.size = 0x00000117;
    UDATA parm.registerSaveDescription = 0x0000000000000200;
    void* gcStackAtlas = !void 0x00007FFFEB2C8A10 
    void* inlinedCalls = !void 0x0000000000000000 
    void* bodyInfo = !void 0x00007FFFED7231A0 
    struct J9JITExceptionTable* nextMethod = !j9jitexceptiontable 0x00007FFFEB2C87F8 
    struct J9JITExceptionTable* prevMethod = !j9jitexceptiontable 0x00007FFFEB2C8A78 
    void* debugSlot1 = !void 0x0000000000000000 
    void* debugSlot2 = !void 0x0000000000000000 
    void* osrInfo = !void 0x0000000000000000 
    void* runtimeAssumptionList = !void 0x00007FFFED7230D0 
    I_32 parm.hotness = 0x00000001;
    UDATA parm.codeCacheAlloc = 0x00007FFFDB207790;
    void* gpuCode = !void 0x0000000000000000 
    void* riData = !void 0x0000000000000000 
}

Looks like the receiver to Object.init<>()V is bad.

@gacholio
Copy link
Contributor

> !jitstackslots 0x14e00,0xf1e40,0x7fffdb2077dc
<14e00> *** BEGIN STACK WALK, flags = 01400001 walkThread = 0x0000000000014E00 ***
<14e00> 	START_AT_JIT_FRAME
<14e00> 	ITERATE_O_SLOTS
<14e00> 	RECORD_BYTECODE_PC_OFFSET
<14e00> Initial values: walkSP = 0x00000000000F1E40, PC = 0x00007FFFDB2077DC, literals = 0x0000000000000000, A0 = 0x0000000000000000, j2iFrame = 0x0000000000000000, ELS = 0x00007FFFF6CA59D0, decomp = 0x0000000000000000
<14e00> JIT frame: bp = 0x00000000000F1E58, pc = 0x00007FFFDB2077DC, unwindSP = 0x00000000000F1E40, cp = 0x000000000003B460, arg0EA = 0x00000000000F1E60, jitInfo = 0x00007FFFEB2C8938
<14e00> 	Method: java/lang/Object.<init>()V !j9method 0x000000000003B2C0
<14e00> 	Bytecode index = -1, inlineDepth = 0, PC offset = 0x000000000000001C
Jan 21, 2021 12:19:46 PM com.ibm.j9ddr.vm29.events.DefaultEventListener corruptData
WARNING: CorruptData encountered iterating o-slots. walkThread = 0x0000000000014E00
com.ibm.j9ddr.AddressedCorruptDataException: Unable to locate JIT stack map
	at com.ibm.j9ddr.vm29.j9.stackwalker.JITStackWalker$JITStackWalker_29_V0.jitWalkFrame(JITStackWalker.java:1077)
	at com.ibm.j9ddr.vm29.j9.stackwalker.JITStackWalker$JITStackWalker_29_V0.jitWalkStackFrames(JITStackWalker.java:257)
	at com.ibm.j9ddr.vm29.j9.stackwalker.JITStackWalker.jitWalkStackFrames(JITStackWalker.java:101)
	at com.ibm.j9ddr.vm29.j9.stackwalker.StackWalker$StackWalker_29_V0.walkStackFrames(StackWalker.java:488)
	at com.ibm.j9ddr.vm29.j9.stackwalker.StackWalker.walkStackFrames(StackWalker.java:103)
	at com.ibm.j9ddr.vm29.tools.ddrinteractive.commands.JitstackCommand.run(JitstackCommand.java:118)
	at com.ibm.j9ddr.tools.ddrinteractive.Context.tryCommand(Context.java:229)
	at com.ibm.j9ddr.tools.ddrinteractive.Context.execute(Context.java:202)
	at com.ibm.j9ddr.tools.ddrinteractive.DDRInteractive.execute(DDRInteractive.java:356)
	at com.ibm.j9ddr.command.CommandReader.processLine(CommandReader.java:79)
	at com.ibm.j9ddr.tools.ddrinteractive.DDRInteractive.processLine(DDRInteractive.java:331)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:90)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
	at java.lang.reflect.Method.invoke(Method.java:508)
	at com.ibm.jvm.dtfjview.CombinedContext.executeDDRInteractiveCommand(CombinedContext.java:258)
	at com.ibm.jvm.dtfjview.CombinedContext.execute(CombinedContext.java:169)
	at com.ibm.jvm.dtfjview.Session.execute(Session.java:811)
	at com.ibm.jvm.dtfjview.Session.execute(Session.java:765)
	at com.ibm.jvm.dtfjview.tools.ToolsRegistry.executeJdmpviewCommand(ToolsRegistry.java:183)
	at com.ibm.jvm.dtfjview.tools.ToolsRegistry.execute(ToolsRegistry.java:107)
	at com.ibm.jvm.dtfjview.tools.ToolsRegistry.execute(ToolsRegistry.java:88)
	at com.ibm.jvm.dtfjview.tools.ToolsRegistry.recordAndExecute(ToolsRegistry.java:72)
	at com.ibm.jvm.dtfjview.Session.runInteractive(Session.java:739)
	at com.ibm.jvm.dtfjview.Session.run(Session.java:663)
	at com.ibm.jvm.dtfjview.DTFJView.launch(DTFJView.java:52)
	at com.ibm.jvm.dtfjview.DTFJView.main(DTFJView.java:46)

<14e00> 	JIT-Frame-RegisterMap[0x00000000000F1E50] = UDATA(0x00000000FFFE7C30) (jit_r9)
<14e00> I2J values: PC = 0x00007FFFF023A191, A0 = 0x00000000000F1E80, walkSP = 0x00000000000F1E68, literals = 0x0000000000060C70, JIT PC = 0x00007FFFEF9B94B0, pcAddress = 0x00007FFFF6CA59F8, decomp = 0x0000000000000000
<14e00> Bytecode frame: bp = 0x00000000000F1E78, sp = 0x00000000000F1E68, pc = 0x00007FFFF023A191, cp = 0x00000000000604E0, arg0EA = 0x00000000000F1E80, flags = 0x0000000000000000
<14e00> 	Method: java/lang/Throwable.<init>()V !j9method 0x0000000000060C70
<14e00> 	Bytecode index = 1
<14e00> 	Using local mapper
<14e00> 	Locals starting at 0x00000000000F1E80 for 0x0000000000000001 slots
<14e00> 		O-Slot: a0[0x00000000000F1E80] = 0x00000000D6DCA500
<14e00> Bytecode frame: bp = 0x00000000000F1E98, sp = 0x00000000000F1E88, pc = 0x00007FFFF023A1D5, cp = 0x00000000000604E0, arg0EA = 0x00000000000F1EA8, flags = 0x0000000000000000
<14e00> 	Method: java/lang/Throwable.<init>(Ljava/lang/String;)V !j9method 0x0000000000060C90
<14e00> 	Bytecode index = 1
<14e00> 	Using local mapper
<14e00> 	Locals starting at 0x00000000000F1EA8 for 0x0000000000000002 slots
<14e00> 		O-Slot: a0[0x00000000000F1EA8] = 0x00000000D6DCA500
<14e00> 		O-Slot: a1[0x00000000000F1EA0] = 0x0000000000000000
<14e00> Bytecode frame: bp = 0x00000000000F1EC0, sp = 0x00000000000F1EB0, pc = 0x00007FFFF023C0F6, cp = 0x0000000000062420, arg0EA = 0x00000000000F1ED0, flags = 0x0000000000000000
<14e00> 	Method: java/lang/Exception.<init>(Ljava/lang/String;)V !j9method 0x000000000005E2C8
<14e00> 	Bytecode index = 2
<14e00> 	Using local mapper
<14e00> 	Locals starting at 0x00000000000F1ED0 for 0x0000000000000002 slots
<14e00> 		I-Slot: a0[0x00000000000F1ED0] = 0x00000000D6DCA500
<14e00> 		I-Slot: a1[0x00000000000F1EC8] = 0x0000000000000000
<14e00> Bytecode frame: bp = 0x00000000000F1EE8, sp = 0x00000000000F1ED8, pc = 0x00007FFFF023C7AE, cp = 0x0000000000063820, arg0EA = 0x00000000000F1EF8, flags = 0x0000000000000000
<14e00> 	Method: java/lang/RuntimeException.<init>(Ljava/lang/String;)V !j9method 0x00000000000631C8
<14e00> 	Bytecode index = 2
<14e00> 	Using local mapper
<14e00> 	Locals starting at 0x00000000000F1EF8 for 0x0000000000000002 slots
<14e00> 		I-Slot: a0[0x00000000000F1EF8] = 0x00000000D6DCA500
<14e00> 		I-Slot: a1[0x00000000000F1EF0] = 0x0000000000000000
<14e00> Bytecode frame: bp = 0x00000000000F1F10, sp = 0x00000000000F1F00, pc = 0x00007FFFF023C616, cp = 0x00000000000638E0, arg0EA = 0x00000000000F1F20, flags = 0x0000000000000000
<14e00> 	Method: java/lang/NullPointerException.<init>(Ljava/lang/String;)V !j9method 0x00000000000638B8
<14e00> 	Bytecode index = 2
<14e00> 	Using local mapper
<14e00> 	Locals starting at 0x00000000000F1F20 for 0x0000000000000002 slots
<14e00> 		I-Slot: a0[0x00000000000F1F20] = 0x00000000D6DCA500
<14e00> 		I-Slot: a1[0x00000000000F1F18] = 0x0000000000000000
<14e00> Bytecode frame: bp = 0x00000000000F1F38, sp = 0x00000000000F1F28, pc = 0x00007FFFF0480684, cp = 0x00000000001895B0, arg0EA = 0x00000000000F1F70, flags = 0x0000000000000000
<14e00> 	Method: org/junit/internal/Checks.notNull(Ljava/lang/Object;Ljava/lang/String;)Ljava/lang/Object; !j9method 0x0000000000189370
<14e00> 	Bytecode index = 68
<14e00> 	Using local mapper
<14e00> 	Locals starting at 0x00000000000F1F70 for 0x0000000000000007 slots
<14e00> 		I-Slot: a0[0x00000000000F1F70] = 0x0000000000000000
<14e00> 		I-Slot: a1[0x00000000000F1F68] = 0x0000000000000000
<14e00> 		I-Slot: t2[0x00000000000F1F60] = 0x00007FFF00000005
<14e00> 		I-Slot: t3[0x00000000000F1F58] = 0x00007FFF00000005
<14e00> 		I-Slot: t4[0x00000000000F1F50] = 0x00007FFF00000001
<14e00> 		O-Slot: t5[0x00000000000F1F48] = 0x00000000D6DCA500
<14e00> 		I-Slot: t6[0x00000000000F1F40] = 0x00007FFF00000004
<14e00> Bytecode frame: bp = 0x00000000000F1F88, sp = 0x00000000000F1F78, pc = 0x00007FFFF0467D16, cp = 0x000000000016A8E0, arg0EA = 0x00000000000F1F98, flags = 0x0000000000000000
<14e00> 	Method: org/junit/internal/ChecksTest.notNullWithNullMessageShouldThrowOnNullValues()V !j9method 0x000000000016AAF8
<14e00> 	Bytecode index = 2
<14e00> 	Using local mapper
<14e00> 	Locals starting at 0x00000000000F1F98 for 0x0000000000000002 slots
<14e00> 		I-Slot: a0[0x00000000000F1F98] = 0x00000000FFFE7918
<14e00> 		I-Slot: t1[0x00000000000F1F90] = 0x00000000000CFA20
<14e00> JNI call-in frame: bp = 0x00000000000F1FC0, sp = 0x00000000000F1FA0, pc = 0x00007FFFF5E030BD, cp = 0x0000000000000000, arg0EA = 0x00000000000F1FC0, flags = 0x0000000000030000
<14e00> 	New ELS = 0x00007FFFF6CA5CB0
<14e00> JIT JNI call-out frame: bp = 0x00000000000F1FE8, sp = 0x00000000000F1FC8, pc = 0x0000000000000006, cp = 0x00000000000F7AC0, arg0EA = 0x00000000000F2090, flags = 0x0000000020000000
<14e00> 	Method: sun/reflect/NativeMethodAccessorImpl.invoke0(Ljava/lang/reflect/Method;Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; !j9method 0x00000000000F7D08
<14e00> JIT frame: bp = 0x00000000000F2028, pc = 0x00007FFFDB20202D, unwindSP = 0x00000000000F1FF0, cp = 0x00000000000F7AC0, arg0EA = 0x00000000000F2040, jitInfo = 0x00007FFFEB2C3DB8
<14e00> 	Method: sun/reflect/NativeMethodAccessorImpl.invoke0(Ljava/lang/reflect/Method;Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; !j9method 0x00000000000F7D08
<14e00> 	Bytecode index = -1, inlineDepth = 0, PC offset = 0x00000000000000F1
<14e00> 	stackMap=0x00007FFFEB2C3EB1, slots=I16(0x0003) parmBaseOffset=I16(0x0008), parmSlots=U16(0x0003), localBaseOffset=I16(0x0000)
<14e00> 	Described JIT args starting at 0x00000000000F2030 for U16(0x0003) slots
<14e00> 		O-Slot: : a2[0x00000000000F2030] = 0x00000000FFFE7C30
<14e00> 		O-Slot: : a1[0x00000000000F2038] = 0x00000000FFFE7918
<14e00> 		O-Slot: : a0[0x00000000000F2040] = 0x00000000FFF12A50
<14e00> 	Described JIT temps starting at 0x00000000000F2028 for IDATA(0x0000000000000000) slots
<14e00> 	JIT-RegisterMap = UDATA(0x0000000000000000)
<14e00> 	JIT-Frame-RegisterMap[0x00000000000F2018] = UDATA(0x00000000FFFC2978) (jit_rbx)
<14e00> 	JIT-Frame-RegisterMap[0x00000000000F2020] = UDATA(0x0000000000000000) (jit_r9)
<14e00> I2J values: PC = 0x00007FFFF032BC3E, A0 = 0x00000000000F2090, walkSP = 0x00000000000F2050, literals = 0x00000000000F7CC8, JIT PC = 0x00007FFFEF9B9D80, pcAddress = 0x00007FFFF6CA5CD8, decomp = 0x0000000000000000
<14e00> Bytecode frame: bp = 0x00000000000F2060, sp = 0x00000000000F2050, pc = 0x00007FFFF032BC3E, cp = 0x00000000000F7AC0, arg0EA = 0x00000000000F2090, flags = 0x0000000000000000
<14e00> 	Method: sun/reflect/NativeMethodAccessorImpl.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; !j9method 0x00000000000F7CC8
<14e00> 	Bytecode index = 134
<14e00> 	Using local mapper
<14e00> 	Locals starting at 0x00000000000F2090 for 0x0000000000000006 slots
<14e00> 		I-Slot: a0[0x00000000000F2090] = 0x00000000FFFE7C60
<14e00> 		I-Slot: a1[0x00000000000F2088] = 0x00000000FFFE7918
<14e00> 		I-Slot: a2[0x00000000000F2080] = 0x00000000FFFE7C30
<14e00> 		I-Slot: t3[0x00000000000F2078] = 0x0000000000000001
<14e00> 		I-Slot: t4[0x00000000000F2070] = 0x00000000FFFE7C78
<14e00> 		I-Slot: t5[0x00000000000F2068] = 0x00007FFFF024AD80
<14e00> Bytecode frame: bp = 0x00000000000F20A8, sp = 0x00000000000F2098, pc = 0x00007FFFF032C1EE, cp = 0x00000000000F7D30, arg0EA = 0x00000000000F20C0, flags = 0x0000000000000000
<14e00> 	Method: sun/reflect/DelegatingMethodAccessorImpl.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; !j9method 0x00000000000F7FE8
<14e00> 	Bytecode index = 6
<14e00> 	Using local mapper
<14e00> 	Locals starting at 0x00000000000F20C0 for 0x0000000000000003 slots
<14e00> 		I-Slot: a0[0x00000000000F20C0] = 0x00000000FFFE7C78
<14e00> 		I-Slot: a1[0x00000000000F20B8] = 0x00000000FFFE7918
<14e00> 		I-Slot: a2[0x00000000000F20B0] = 0x00000000FFFE7C30
<14e00> Bytecode frame: bp = 0x00000000000F20D8, sp = 0x00000000000F20C8, pc = 0x00007FFFF024ABFA, cp = 0x000000000006D140, arg0EA = 0x00000000000F20F8, flags = 0x0000000000000000
<14e00> 	Method: java/lang/reflect/Method.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; !j9method 0x000000000006E208
<14e00> 	Bytecode index = 58
<14e00> 	Using local mapper
<14e00> 	Locals starting at 0x00000000000F20F8 for 0x0000000000000004 slots
<14e00> 		I-Slot: a0[0x00000000000F20F8] = 0x00000000FFF12A50
<14e00> 		I-Slot: a1[0x00000000000F20F0] = 0x00000000FFFE7918
<14e00> 		I-Slot: a2[0x00000000000F20E8] = 0x00000000FFFE7C30
<14e00> 		I-Slot: t3[0x00000000000F20E0] = 0x00000000FFFE7C78
<14e00> Bytecode frame: bp = 0x00000000000F2110, sp = 0x00000000000F2100, pc = 0x00007FFFF04EE6C7, cp = 0x00000000001BC7C0, arg0EA = 0x00000000000F2118, flags = 0x0000000000000000
<14e00> 	Method: org/junit/runners/model/FrameworkMethod$1.runReflectiveCall()Ljava/lang/Object; !j9method 0x00000000001BC898
<14e00> 	Bytecode index = 15
<14e00> 	Using local mapper
<14e00> 	Locals starting at 0x00000000000F2118 for 0x0000000000000001 slots
<14e00> 		I-Slot: a0[0x00000000000F2118] = 0x00000000FFFE7C48
<14e00> Bytecode frame: bp = 0x00000000000F2130, sp = 0x00000000000F2120, pc = 0x00007FFFF04EDFC9, cp = 0x00000000001BBBC0, arg0EA = 0x00000000000F2140, flags = 0x0000000000000000
<14e00> 	Method: org/junit/internal/runners/model/ReflectiveCallable.run()Ljava/lang/Object; !j9method 0x00000000001BBC58
<14e00> 	Bytecode index = 1
<14e00> 	Using local mapper
<14e00> 	Locals starting at 0x00000000000F2140 for 0x0000000000000002 slots
<14e00> 		I-Slot: a0[0x00000000000F2140] = 0x00000000FFFE7C48
<14e00> 		I-Slot: t1[0x00000000000F2138] = 0x00000000FFFE7C48
<14e00> Bytecode frame: bp = 0x00000000000F2158, sp = 0x00000000000F2148, pc = 0x00007FFFF047D776, cp = 0x0000000000185410, arg0EA = 0x00000000000F2170, flags = 0x0000000000000000
<14e00> 	Method: org/junit/runners/model/FrameworkMethod.invokeExplosively(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; !j9method 0x0000000000185BA8
<14e00> 	Bytecode index = 10
<14e00> 	Using local mapper
<14e00> 	Locals starting at 0x00000000000F2170 for 0x0000000000000003 slots
<14e00> 		I-Slot: a0[0x00000000000F2170] = 0x00000000FFF629B0
<14e00> 		I-Slot: a1[0x00000000000F2168] = 0x00000000FFFE7918
<14e00> 		I-Slot: a2[0x00000000000F2160] = 0x00000000FFFE7C30
<14e00> Bytecode frame: bp = 0x00000000000F2188, sp = 0x00000000000F2178, pc = 0x00007FFFF04793D0, cp = 0x0000000000182AC0, arg0EA = 0x00000000000F2190, flags = 0x0000000000000000
<14e00> 	Method: org/junit/internal/runners/statements/InvokeMethod.evaluate()V !j9method 0x0000000000182B78
<14e00> 	Bytecode index = 12
<14e00> 	Using local mapper
<14e00> 	Locals starting at 0x00000000000F2190 for 0x0000000000000001 slots
<14e00> 		I-Slot: a0[0x00000000000F2190] = 0x00000000FFFE7928
<14e00> Bytecode frame: bp = 0x00000000000F21A8, sp = 0x00000000000F2198, pc = 0x00007FFFF04709A4, cp = 0x0000000000171E00, arg0EA = 0x00000000000F21B8, flags = 0x0000000000000000
<14e00> 	Method: org/junit/runners/ParentRunner$3.evaluate()V !j9method 0x00000000001720E0
<14e00> 	Bytecode index = 4
<14e00> 	Using local mapper
<14e00> 	Locals starting at 0x00000000000F21B8 for 0x0000000000000002 slots
<14e00> 		I-Slot: a0[0x00000000000F21B8] = 0x00000000FFFE7C20
<14e00> 		I-Slot: t1[0x00000000000F21B0] = 0x00000000FFF629B0
<14e00> Bytecode frame: bp = 0x00000000000F21D0, sp = 0x00000000000F21C0, pc = 0x00007FFFF0479033, cp = 0x0000000000182300, arg0EA = 0x00000000000F21D8, flags = 0x0000000000000000
<14e00> 	Method: org/junit/runners/BlockJUnit4ClassRunner$1.evaluate()V !j9method 0x0000000000182040
<14e00> 	Bytecode index = 11
<14e00> 	Using local mapper
<14e00> 	Locals starting at 0x00000000000F21D8 for 0x0000000000000001 slots
<14e00> 		I-Slot: a0[0x00000000000F21D8] = 0x00000000FFFE77A8
<14e00> Bytecode frame: bp = 0x00000000000F21F0, sp = 0x00000000000F21E0, pc = 0x00007FFFF046ADAD, cp = 0x000000000016BAE0, arg0EA = 0x00000000000F2228, flags = 0x0000000000000000
<14e00> 	Method: org/junit/runners/ParentRunner.runLeaf(Lorg/junit/runners/model/Statement;Lorg/junit/runner/Description;Lorg/junit/runner/notification/RunNotifier;)V !j9method 0x000000000016C6F8
<14e00> 	Bytecode index = 17
<14e00> 	Using local mapper
<14e00> 	Locals starting at 0x00000000000F2228 for 0x0000000000000007 slots
<14e00> 		I-Slot: a0[0x00000000000F2228] = 0x00000000FFF116C0
<14e00> 		I-Slot: a1[0x00000000000F2220] = 0x00000000FFFE77A8
<14e00> 		I-Slot: a2[0x00000000000F2218] = 0x00000000FFFA7EA8
<14e00> 		I-Slot: a3[0x00000000000F2210] = 0x00000000FFF12BB0
<14e00> 		O-Slot: t4[0x00000000000F2208] = 0x00000000FFFE77B8
<14e00> 		I-Slot: t5[0x00000000000F2200] = 0x00007FFFF0475024
<14e00> 		I-Slot: t6[0x00000000000F21F8] = 0x000000000017F788
<14e00> Bytecode frame: bp = 0x00000000000F2240, sp = 0x00000000000F2230, pc = 0x00007FFFF047502E, cp = 0x000000000017E8B0, arg0EA = 0x00000000000F2268, flags = 0x0000000000000000
<14e00> 	Method: org/junit/runners/BlockJUnit4ClassRunner.runChild(Lorg/junit/runners/model/FrameworkMethod;Lorg/junit/runner/notification/RunNotifier;)V !j9method 0x000000000017F788
<14e00> 	Bytecode index = 38
<14e00> 	Using local mapper
<14e00> 	Locals starting at 0x00000000000F2268 for 0x0000000000000005 slots
<14e00> 		I-Slot: a0[0x00000000000F2268] = 0x00000000FFF116C0
<14e00> 		I-Slot: a1[0x00000000000F2260] = 0x00000000FFF629B0
<14e00> 		I-Slot: a2[0x00000000000F2258] = 0x00000000FFF12BB0
<14e00> 		I-Slot: t3[0x00000000000F2250] = 0x00000000FFFA7EA8
<14e00> 		I-Slot: t4[0x00000000000F2248] = 0x00000000FFFE77A8
<14e00> Bytecode frame: bp = 0x00000000000F2280, sp = 0x00000000000F2270, pc = 0x00007FFFF047597E, cp = 0x000000000017E8B0, arg0EA = 0x00000000000F2298, flags = 0x0000000000000000
<14e00> 	Method: org/junit/runners/BlockJUnit4ClassRunner.runChild(Ljava/lang/Object;Lorg/junit/runner/notification/RunNotifier;)V !j9method 0x000000000017FB68
<14e00> 	Bytecode index = 6
<14e00> 	Using local mapper
<14e00> 	Locals starting at 0x00000000000F2298 for 0x0000000000000003 slots
<14e00> 		I-Slot: a0[0x00000000000F2298] = 0x00000000FFF116C0
<14e00> 		I-Slot: a1[0x00000000000F2290] = 0x00000000FFF629B0
<14e00> 		I-Slot: a2[0x00000000000F2288] = 0x00000000FFF12BB0
<14e00> Bytecode frame: bp = 0x00000000000F22B0, sp = 0x00000000000F22A0, pc = 0x00007FFFF04ED778, cp = 0x00000000001BB5C0, arg0EA = 0x00000000000F22B8, flags = 0x0000000000000000
<14e00> 	Method: org/junit/runners/ParentRunner$4.run()V !j9method 0x00000000001BAB78
<14e00> 	Bytecode index = 12
<14e00> 	Using local mapper
<14e00> 	Locals starting at 0x00000000000F22B8 for 0x0000000000000001 slots
<14e00> 		I-Slot: a0[0x00000000000F22B8] = 0x00000000FFFE7790
<14e00> Bytecode frame: bp = 0x00000000000F22D0, sp = 0x00000000000F22C0, pc = 0x00007FFFF04802DF, cp = 0x0000000000188FF0, arg0EA = 0x00000000000F22E0, flags = 0x0000000000000000
<14e00> 	Method: org/junit/runners/ParentRunner$1.schedule(Ljava/lang/Runnable;)V !j9method 0x00000000001892E8
<14e00> 	Bytecode index = 3
<14e00> 	Using local mapper
<14e00> 	Locals starting at 0x00000000000F22E0 for 0x0000000000000002 slots
<14e00> 		I-Slot: a0[0x00000000000F22E0] = 0x00000000FFF67B08
<14e00> 		I-Slot: a1[0x00000000000F22D8] = 0x00000000FFFE7790
<14e00> Bytecode frame: bp = 0x00000000000F22F8, sp = 0x00000000000F22E8, pc = 0x00007FFFF046ACD2, cp = 0x000000000016BAE0, arg0EA = 0x00000000000F2328, flags = 0x0000000000000000
<14e00> 	Method: org/junit/runners/ParentRunner.runChildren(Lorg/junit/runner/notification/RunNotifier;)V !j9method 0x000000000016C698
<14e00> 	Bytecode index = 46
<14e00> 	Using local mapper
<14e00> 	Locals starting at 0x00000000000F2328 for 0x0000000000000006 slots
<14e00> 		O-Slot: a0[0x00000000000F2328] = 0x00000000FFF116C0
<14e00> 		O-Slot: a1[0x00000000000F2320] = 0x00000000FFF12BB0
<14e00> 		O-Slot: t2[0x00000000000F2318] = 0x00000000FFF67B08
<14e00> 		O-Slot: t3[0x00000000000F2310] = 0x00000000FFFD4170
<14e00> 		I-Slot: t4[0x00000000000F2308] = 0x00000000FFF629B0
<14e00> 		I-Slot: t5[0x00000000000F2300] = 0x00000000FFFD4160
<14e00> Bytecode frame: bp = 0x00000000000F2340, sp = 0x00000000000F2330, pc = 0x00007FFFF046B506, cp = 0x000000000016BAE0, arg0EA = 0x00000000000F2350, flags = 0x0000000000000000
<14e00> 	Method: org/junit/runners/ParentRunner.access$100(Lorg/junit/runners/ParentRunner;Lorg/junit/runner/notification/RunNotifier;)V !j9method 0x000000000016C898
<14e00> 	Bytecode index = 2
<14e00> 	Using local mapper
<14e00> 	Locals starting at 0x00000000000F2350 for 0x0000000000000002 slots
<14e00> 		I-Slot: a0[0x00000000000F2350] = 0x00000000FFF116C0
<14e00> 		I-Slot: a1[0x00000000000F2348] = 0x00000000FFF12BB0
<14e00> Bytecode frame: bp = 0x00000000000F2368, sp = 0x00000000000F2358, pc = 0x00007FFFF04707C0, cp = 0x0000000000171970, arg0EA = 0x00000000000F2370, flags = 0x0000000000000000
<14e00> 	Method: org/junit/runners/ParentRunner$2.evaluate()V !j9method 0x0000000000171DE0
<14e00> 	Bytecode index = 8
<14e00> 	Using local mapper
<14e00> 	Locals starting at 0x00000000000F2370 for 0x0000000000000001 slots
<14e00> 		I-Slot: a0[0x00000000000F2370] = 0x00000000FFFD4070
<14e00> Bytecode frame: bp = 0x00000000000F2388, sp = 0x00000000000F2378, pc = 0x00007FFFF04709A4, cp = 0x0000000000171E00, arg0EA = 0x00000000000F2398, flags = 0x0000000000000000
<14e00> 	Method: org/junit/runners/ParentRunner$3.evaluate()V !j9method 0x00000000001720E0
<14e00> 	Bytecode index = 4
<14e00> 	Using local mapper
<14e00> 	Locals starting at 0x00000000000F2398 for 0x0000000000000002 slots
<14e00> 		I-Slot: a0[0x00000000000F2398] = 0x00000000FFFD4160
<14e00> 		I-Slot: t1[0x00000000000F2390] = 0x00000000FFF12BB0
<14e00> Bytecode frame: bp = 0x00000000000F23B0, sp = 0x00000000000F23A0, pc = 0x00007FFFF046AF50, cp = 0x000000000016BAE0, arg0EA = 0x00000000000F23D8, flags = 0x0000000000000000
<14e00> 	Method: org/junit/runners/ParentRunner.run(Lorg/junit/runner/notification/RunNotifier;)V !j9method 0x000000000016C758
<14e00> 	Bytecode index = 24
<14e00> 	Using local mapper
<14e00> 	Locals starting at 0x00000000000F23D8 for 0x0000000000000005 slots
<14e00> 		I-Slot: a0[0x00000000000F23D8] = 0x00000000FFF116C0
<14e00> 		I-Slot: a1[0x00000000000F23D0] = 0x00000000FFF12BB0
<14e00> 		O-Slot: t2[0x00000000000F23C8] = 0x00000000FFFD3F20
<14e00> 		I-Slot: t3[0x00000000000F23C0] = 0x00000000FFFD4160
<14e00> 		I-Slot: t4[0x00000000000F23B8] = 0x000000000016D4B0
<14e00> Bytecode frame: bp = 0x00000000000F23F0, sp = 0x00000000000F23E0, pc = 0x00007FFFF0469942, cp = 0x000000000016CCE0, arg0EA = 0x00000000000F2408, flags = 0x0000000000000000
<14e00> 	Method: org/junit/runners/Suite.runChild(Lorg/junit/runner/Runner;Lorg/junit/runner/notification/RunNotifier;)V !j9method 0x000000000016D4B0
<14e00> 	Bytecode index = 2
<14e00> 	Using local mapper
<14e00> 	Locals starting at 0x00000000000F2408 for 0x0000000000000003 slots
<14e00> 		I-Slot: a0[0x00000000000F2408] = 0x00000000FFF11750
<14e00> 		I-Slot: a1[0x00000000000F2400] = 0x00000000FFF116C0
<14e00> 		I-Slot: a2[0x00000000000F23F8] = 0x00000000FFF12BB0
<14e00> Bytecode frame: bp = 0x00000000000F2420, sp = 0x00000000000F2410, pc = 0x00007FFFF0469972, cp = 0x000000000016CCE0, arg0EA = 0x00000000000F2438, flags = 0x0000000000000000
<14e00> 	Method: org/junit/runners/Suite.runChild(Ljava/lang/Object;Lorg/junit/runner/notification/RunNotifier;)V !j9method 0x000000000016D4D0
<14e00> 	Bytecode index = 6
<14e00> 	Using local mapper
<14e00> 	Locals starting at 0x00000000000F2438 for 0x0000000000000003 slots
<14e00> 		I-Slot: a0[0x00000000000F2438] = 0x00000000FFF11750
<14e00> 		I-Slot: a1[0x00000000000F2430] = 0x00000000FFF116C0
<14e00> 		I-Slot: a2[0x00000000000F2428] = 0x00000000FFF12BB0
<14e00> Bytecode frame: bp = 0x00000000000F2450, sp = 0x00000000000F2440, pc = 0x00007FFFF04ED778, cp = 0x00000000001BB5C0, arg0EA = 0x00000000000F2458, flags = 0x0000000000000000
<14e00> 	Method: org/junit/runners/ParentRunner$4.run()V !j9method 0x00000000001BAB78
<14e00> 	Bytecode index = 12
<14e00> 	Using local mapper
<14e00> 	Locals starting at 0x00000000000F2458 for 0x0000000000000001 slots
<14e00> 		I-Slot: a0[0x00000000000F2458] = 0x00000000FFFD3F08
<14e00> Bytecode frame: bp = 0x00000000000F2470, sp = 0x00000000000F2460, pc = 0x00007FFFF04802DF, cp = 0x0000000000188FF0, arg0EA = 0x00000000000F2480, flags = 0x0000000000000000
<14e00> 	Method: org/junit/runners/ParentRunner$1.schedule(Ljava/lang/Runnable;)V !j9method 0x00000000001892E8
<14e00> 	Bytecode index = 3
<14e00> 	Using local mapper
<14e00> 	Locals starting at 0x00000000000F2480 for 0x0000000000000002 slots
<14e00> 		I-Slot: a0[0x00000000000F2480] = 0x00000000FFFA9AD8
<14e00> 		I-Slot: a1[0x00000000000F2478] = 0x00000000FFFD3F08
<14e00> Bytecode frame: bp = 0x00000000000F2498, sp = 0x00000000000F2488, pc = 0x00007FFFF046ACD2, cp = 0x000000000016BAE0, arg0EA = 0x00000000000F24C8, flags = 0x0000000000000000
<14e00> 	Method: org/junit/runners/ParentRunner.runChildren(Lorg/junit/runner/notification/RunNotifier;)V !j9method 0x000000000016C698
<14e00> 	Bytecode index = 46
<14e00> 	Using local mapper
<14e00> 	Locals starting at 0x00000000000F24C8 for 0x0000000000000006 slots
<14e00> 		O-Slot: a0[0x00000000000F24C8] = 0x00000000FFF11750
<14e00> 		O-Slot: a1[0x00000000000F24C0] = 0x00000000FFF12BB0
<14e00> 		O-Slot: t2[0x00000000000F24B8] = 0x00000000FFFA9AD8
<14e00> 		O-Slot: t3[0x00000000000F24B0] = 0x00000000FFFD2820
<14e00> 		I-Slot: t4[0x00000000000F24A8] = 0x00000000FFF116C0
<14e00> 		I-Slot: t5[0x00000000000F24A0] = 0x0000000000000000
<14e00> Bytecode frame: bp = 0x00000000000F24E0, sp = 0x00000000000F24D0, pc = 0x00007FFFF046B506, cp = 0x000000000016BAE0, arg0EA = 0x00000000000F24F0, flags = 0x0000000000000000
<14e00> 	Method: org/junit/runners/ParentRunner.access$100(Lorg/junit/runners/ParentRunner;Lorg/junit/runner/notification/RunNotifier;)V !j9method 0x000000000016C898
<14e00> 	Bytecode index = 2
<14e00> 	Using local mapper
<14e00> 	Locals starting at 0x00000000000F24F0 for 0x0000000000000002 slots
<14e00> 		I-Slot: a0[0x00000000000F24F0] = 0x00000000FFF11750
<14e00> 		I-Slot: a1[0x00000000000F24E8] = 0x00000000FFF12BB0
<14e00> Bytecode frame: bp = 0x00000000000F2508, sp = 0x00000000000F24F8, pc = 0x00007FFFF04707C0, cp = 0x0000000000171970, arg0EA = 0x00000000000F2510, flags = 0x0000000000000000
<14e00> 	Method: org/junit/runners/ParentRunner$2.evaluate()V !j9method 0x0000000000171DE0
<14e00> 	Bytecode index = 8
<14e00> 	Using local mapper
<14e00> 	Locals starting at 0x00000000000F2510 for 0x0000000000000001 slots
<14e00> 		I-Slot: a0[0x00000000000F2510] = 0x00000000FFFCB0F0
<14e00> Bytecode frame: bp = 0x00000000000F2528, sp = 0x00000000000F2518, pc = 0x00007FFFF04709A4, cp = 0x0000000000171E00, arg0EA = 0x00000000000F2538, flags = 0x0000000000000000
<14e00> 	Method: org/junit/runners/ParentRunner$3.evaluate()V !j9method 0x00000000001720E0
<14e00> 	Bytecode index = 4
<14e00> 	Using local mapper
<14e00> 	Locals starting at 0x00000000000F2538 for 0x0000000000000002 slots
<14e00> 		I-Slot: a0[0x00000000000F2538] = 0x00000000FFFD2810
<14e00> 		I-Slot: t1[0x00000000000F2530] = 0x00000000000F257A
<14e00> Bytecode frame: bp = 0x00000000000F2550, sp = 0x00000000000F2540, pc = 0x00007FFFF046AF50, cp = 0x000000000016BAE0, arg0EA = 0x00000000000F2578, flags = 0x0000000000000000
<14e00> 	Method: org/junit/runners/ParentRunner.run(Lorg/junit/runner/notification/RunNotifier;)V !j9method 0x000000000016C758
<14e00> 	Bytecode index = 24
<14e00> 	Using local mapper
<14e00> 	Locals starting at 0x00000000000F2578 for 0x0000000000000005 slots
<14e00> 		I-Slot: a0[0x00000000000F2578] = 0x00000000FFF11750
<14e00> 		I-Slot: a1[0x00000000000F2570] = 0x00000000FFF12BB0
<14e00> 		O-Slot: t2[0x00000000000F2568] = 0x00000000FFFC96B0
<14e00> 		I-Slot: t3[0x00000000000F2560] = 0x00000000FFFD2810
<14e00> 		I-Slot: t4[0x00000000000F2558] = 0x0000000000165748
<14e00> Bytecode frame: bp = 0x00000000000F2590, sp = 0x00000000000F2580, pc = 0x00007FFFF0461F65, cp = 0x0000000000164F80, arg0EA = 0x00000000000F25B8, flags = 0x0000000000000000
<14e00> 	Method: org/junit/runner/JUnitCore.run(Lorg/junit/runner/Runner;)Lorg/junit/runner/Result; !j9method 0x0000000000165748
<14e00> 	Bytecode index = 37
<14e00> 	Using local mapper
<14e00> 	Locals starting at 0x00000000000F25B8 for 0x0000000000000005 slots
<14e00> 		O-Slot: a0[0x00000000000F25B8] = 0x00000000FFF11788
<14e00> 		I-Slot: a1[0x00000000000F25B0] = 0x00000000FFF11750
<14e00> 		O-Slot: t2[0x00000000000F25A8] = 0x00000000FFFB8CA8
<14e00> 		O-Slot: t3[0x00000000000F25A0] = 0x00000000FFFB8E78
<14e00> 		I-Slot: t4[0x00000000000F2598] = 0x0000000000165708
<14e00> Bytecode frame: bp = 0x00000000000F25D0, sp = 0x00000000000F25C0, pc = 0x00007FFFF0461EE1, cp = 0x0000000000164F80, arg0EA = 0x00000000000F25E0, flags = 0x0000000000000000
<14e00> 	Method: org/junit/runner/JUnitCore.run(Lorg/junit/runner/Request;)Lorg/junit/runner/Result; !j9method 0x0000000000165708
<14e00> 	Bytecode index = 5
<14e00> 	Using local mapper
<14e00> 	Locals starting at 0x00000000000F25E0 for 0x0000000000000002 slots
<14e00> 		I-Slot: a0[0x00000000000F25E0] = 0x00000000FFF11788
<14e00> 		I-Slot: a1[0x00000000000F25D8] = 0x00000000FFFA9D48
<14e00> Bytecode frame: bp = 0x00000000000F25F8, sp = 0x00000000000F25E8, pc = 0x00007FFFF0461E1F, cp = 0x0000000000164F80, arg0EA = 0x00000000000F2620, flags = 0x0000000000000000
<14e00> 	Method: org/junit/runner/JUnitCore.runMain(Lorg/junit/internal/JUnitSystem;[Ljava/lang/String;)Lorg/junit/runner/Result; !j9method 0x0000000000165688
<14e00> 	Bytecode index = 59
<14e00> 	Using local mapper
<14e00> 	Locals starting at 0x00000000000F2620 for 0x0000000000000005 slots
<14e00> 		I-Slot: a0[0x00000000000F2620] = 0x00000000FFFD0968
<14e00> 		I-Slot: a1[0x00000000000F2618] = 0x00000000FFFD95A8
<14e00> 		I-Slot: a2[0x00000000000F2610] = 0x00000000FFFD08C0
<14e00> 		I-Slot: t3[0x00000000000F2608] = 0x00000000FFFE15A0
<14e00> 		I-Slot: t4[0x00000000000F2600] = 0x00000000FFFE48A0
<14e00> Bytecode frame: bp = 0x00000000000F2638, sp = 0x00000000000F2628, pc = 0x00007FFFF0461D43, cp = 0x0000000000164F80, arg0EA = 0x00000000000F2648, flags = 0x0000000000000000
<14e00> 	Method: org/junit/runner/JUnitCore.main([Ljava/lang/String;)V !j9method 0x0000000000165628
<14e00> 	Bytecode index = 15
<14e00> 	Using local mapper
<14e00> 	Locals starting at 0x00000000000F2648 for 0x0000000000000002 slots
<14e00> 		I-Slot: a0[0x00000000000F2648] = 0x00000000FFFD08C0
<14e00> 		I-Slot: t1[0x00000000000F2640] = 0x00000000FFFD0908
<14e00> JNI call-in frame: bp = 0x00000000000F2670, sp = 0x00000000000F2650, pc = 0x00007FFFF5E030BD, cp = 0x0000000000000000, arg0EA = 0x00000000000F2670, flags = 0x0000000000020000
<14e00> 	New ELS = 0x0000000000000000
<14e00> JNI native method frame: bp = 0x00000000000F2708, sp = 0x00000000000F2678, pc = 0x0000000000000007, cp = 0x0000000000000000, arg0EA = 0x00000000000F2708, flags = 0x000000000000000E
<14e00> 	Object pushes starting at 0x00000000000F2678 for 14 slots
<14e00> 		Push[0x00000000000F2678] = 0x0000000000000000
<14e00> 		Push[0x00000000000F2680] = 0x0000000000000000
<14e00> 		Push[0x00000000000F2688] = 0x00000000FFF11798
<14e00> 		Push[0x00000000000F2690] = 0x00000000E0000910
<14e00> 		Push[0x00000000000F2698] = 0x00000000E00175D8
<14e00> 		Push[0x00000000000F26A0] = 0x00000000E00175D8
<14e00> 		Push[0x00000000000F26A8] = 0x0000000000000000
<14e00> 		Push[0x00000000000F26B0] = 0x0000000000000000
<14e00> 		Push[0x00000000000F26B8] = 0x00000000E00121E8
<14e00> 		Push[0x00000000000F26C0] = 0x00000000E0003708
<14e00> 		Push[0x00000000000F26C8] = 0x00000000E0003668
<14e00> 		Push[0x00000000000F26D0] = 0x00000000E0000280
<14e00> 		Push[0x00000000000F26D8] = 0x00000000E0001410
<14e00> 		Push[0x00000000000F26E0] = 0x00000000E0000000
<14e00> <end of stack>
<14e00> *** END STACK WALK (rc = NONE) ***

@gacholio
Copy link
Contributor

> !bytecodes 0x0000000000189370
  Name: notNull
  Signature: (Ljava/lang/Object;Ljava/lang/String;)Ljava/lang/Object;
  Access Flags (2040009): public static 
  Max Stack: 2
  Argument Count: 2
  Temp Count: 5

    0 iconst5 
    1 istore2 
    2 iconst5 
    3 istore3 
    4 iconst1 
    5 istore 4
    7 iconst5 
    8 istore 6
   10 aload0 
   11 ifnonnull 79
   14 ldc 5 (java.lang.String) <org.junit.internal.Checks: java.lang.Object notNull(java.lang.Object,java.lang.String)> **** Executed Line: **** 2 **** if r0 != null goto return r0
   16 invokestatic 11 Print.logPrint(Ljava/lang/String;)V
   19 iinc 6 -1
   22 iload 6
   24 ifge 64
   27 iload 4
   29 iconstm1 
   30 iadd 
   31 istore 5
   33 iload 5
   35 ifle 79
   38 new 12 java/lang/NullPointerException
   41 astore 5
   43 ldc 1 (java.lang.String) <org.junit.internal.Checks: java.lang.Object notNull(java.lang.Object,java.lang.String)> **** Executed Line: **** 3 **** $r1 = new java.lang.NullPointerException
   45 invokestatic 11 Print.logPrint(Ljava/lang/String;)V
   48 iload3 
   49 iconstm1 
   50 iadd 
   51 istore0 
   52 iload0 
   53 ifge 65
   56 iload2 
   57 iconstm1 
   58 iadd 
   59 istore0 
   60 iload0 
   61 ifge 65
   64 nop 
   65 aload 5
   67 aload1 
   68 invokespecial 14 java/lang/NullPointerException.<init>(Ljava/lang/String;)V
   71 ldc 6 (java.lang.String) <org.junit.internal.Checks: java.lang.Object notNull(java.lang.Object,java.lang.String)> **** Executed Line: **** 4 **** specialinvoke $r1.<java.lang.NullPointerException: void <init>(java.lang.String)>(r2)
   73 invokestatic 11 Print.logPrint(Ljava/lang/String;)V
   76 aload 5
   78 athrow 
   79 aload0 
   80 return1 

@gacholio
Copy link
Contributor

gacholio commented Jan 21, 2021

The only odd thing in the bytecodes is that the logger is called in between the creation of the NPE and the construction of it. There's also some junk code between the logger call and the constructor call.

@gacholio
Copy link
Contributor

The failure I'm looking at occurs with -Xint.

@gacholio
Copy link
Contributor

Running with -XX:+DebugInterpreter seems not to crash, implying maybe a missed vmThread root update somewhere.

@gacholio
Copy link
Contributor

Looking at the bytecodes above, this should fail bytecode verification.

Bytecode 24 may branch to bytecode 64, bypassing any write to local 5 which is read at bytecode 65. The end result would be random data being passed up the NPE constructor chain eventually resulting in the garbage being read in the finalizer test in the object constructor.

@gacholio
Copy link
Contributor

@ChengJin01 Any thoughts as to why this doesn't fail verification? It does not appear to be exempt:

	0x18: UDATA classDepthAndFlags = 0x00000000000E0001 (917505)
	0x20: U32 classDepthWithFlags = 0x00000000 (0)
	0x24: U32 classFlags = 0x00000000 (0)

Class Name: org/junit/internal/Checks
Superclass Name: java/lang/Object
Source File Name: Checks.java
Basic Access Flags (0x31): public final super 
J9 Access Flags (0x8000000): 

@ChengJin01
Copy link

ChengJin01 commented Jan 21, 2021

@gacholio , this is the same issue as #11684 and #11685 which was triggered by the fix at #9419 (explained at #11684 (comment)).

The bytecode above was intentionally modified to trigger verification errors. I already created a new fix at ChengJin01@dbed590 to capture the verification error here

j9-8-crash$ ../jdk11_openj9_fix_v0/bin/java  -cp sootOutput/junit-junit/:hamcrest-all-1.3.jar:junit-4.12.jar 
org.junit.runner.JUnitCore org.junit.internal.ChecksTest
JUnit version 4.13

verifyBytecodes: Class.Method.Sig = org/junit/internal/Checks.notNull.(Ljava/lang/Object;Ljava/lang/String;)Ljava/lang/Object;, ***** verifyData->errorDetailCode: = 0
.E
Time: 0.004
There was 1 failure:
1) initializationError(org.junit.internal.ChecksTest)
java.lang.VerifyError: JVMVRFY012 stack shape inconsistent; class=org/junit/internal/Checks, method=notNull(Ljava/lang/Object;Ljava/lang/String;)Ljava/lang/Object;, pc=65
Exception Details:
  Location:
    org/junit/internal/Checks.notNull(Ljava/lang/Object;Ljava/lang/String;)Ljava/lang/Object; @65: JBaload
  Reason:
    Type top (current frame, locals[5]) is not assignable to 'reference' type
  Current Frame:
    bci: @65
    flags: { }
    locals: { top, 'java/lang/String', integer, integer, integer, top, integer }
    stack: { }
	at org.junit.runners.ParentRunner.<init>(ParentRunner.java:101)
	at org.junit.runners.BlockJUnit4ClassRunner.<init>(BlockJUnit4ClassRunner.java:84)
	at org.junit.runners.JUnit4.<init>(JUnit4.java:23)
	at org.junit.internal.builders.JUnit4Builder.runnerForClass(JUnit4Builder.java:10)
	at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:70)
	at org.junit.internal.builders.AllDefaultPossibilitiesBuilder.runnerForClass(AllDefaultPossibilitiesBuilder.java:37)
	at org.junit.runner.Computer.getRunner(Computer.java:50)
	at org.junit.runner.Computer$1.runnerForClass(Computer.java:31)
	at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:70)
	at org.junit.runners.model.RunnerBuilder.runners(RunnerBuilder.java:125)
	at org.junit.runners.model.RunnerBuilder.runners(RunnerBuilder.java:111)
	at org.junit.runners.Suite.<init>(Suite.java:81)
	at org.junit.runner.Computer$2.<init>(Computer.java:33)
	at org.junit.runner.Computer.getSuite(Computer.java:28)
	at org.junit.runner.Request.classes(Request.java:77)
	at org.junit.runner.JUnitCommandLineParseResult.createRequest(JUnitCommandLineParseResult.java:116)
	at org.junit.runner.JUnitCore.runMain(JUnitCore.java:77)
	at org.junit.runner.JUnitCore.main(JUnitCore.java:36)

FAILURES!!!
Tests run: 1,  Failures: 1

against the same RI's behaviour at

j9-8-crash$ ../jdk11_hotspot/bin/java  -cp sootOutput/junit-junit/:hamcrest-all-1.3.jar:junit-4.12.jar 
org.junit.runner.JUnitCore org.junit.internal.ChecksTest
JUnit version 4.13
.E
Time: 0.004
There was 1 failure:
1) initializationError(org.junit.internal.ChecksTest)
java.lang.VerifyError: (class: org/junit/internal/Checks, method: notNull signature: 
(Ljava/lang/Object;Ljava/lang/String;)Ljava/lang/Object;) Register 5 contains wrong type
	at org.junit.runners.ParentRunner.<init>(ParentRunner.java:101)
	at org.junit.runners.BlockJUnit4ClassRunner.<init>(BlockJUnit4ClassRunner.java:84)
	at org.junit.runners.JUnit4.<init>(JUnit4.java:23)
	at org.junit.internal.builders.JUnit4Builder.runnerForClass(JUnit4Builder.java:10)
	at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:70)
	at org.junit.internal.builders.AllDefaultPossibilitiesBuilder.runnerForClass(AllDefaultPossibilitiesBuilder.java:37)
	at org.junit.runner.Computer.getRunner(Computer.java:50)
	at org.junit.runner.Computer$1.runnerForClass(Computer.java:31)
	at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:70)
	at org.junit.runners.model.RunnerBuilder.runners(RunnerBuilder.java:125)
	at org.junit.runners.model.RunnerBuilder.runners(RunnerBuilder.java:111)
	at org.junit.runners.Suite.<init>(Suite.java:81)
	at org.junit.runner.Computer$2.<init>(Computer.java:33)
	at org.junit.runner.Computer.getSuite(Computer.java:28)
	at org.junit.runner.Request.classes(Request.java:77)
	at org.junit.runner.JUnitCommandLineParseResult.createRequest(JUnitCommandLineParseResult.java:116)
	at org.junit.runner.JUnitCore.runMain(JUnitCore.java:77)
	at org.junit.runner.JUnitCore.main(JUnitCore.java:36)

FAILURES!!!
Tests run: 1,  Failures: 1

Will launch personal builds and verify the internal tests to ensure there is no break in the existing test suites.

ChengJin01 pushed a commit to ChengJin01/openj9 that referenced this issue Jan 22, 2021
The change is to reinforce the initial intention that only
a target slot with BCV_SPECIAL_INIT is exempted from being
set with top given setInitializedThisStatus() needs to check
BCV_SPECIAL_INIT to flag the uninitialized_this object.

Fixes: eclipse-openj9#11683,eclipse-openj9#11684,eclipse-openj9#11685

Signed-off-by: Cheng Jin <[email protected]>
@fjeremic fjeremic added comp:vm and removed comp:jit labels Apr 8, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants