Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

JTReg Test Failure - serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorArrayAllSampledTest.java #5969

Closed
JasonFengJ9 opened this issue May 31, 2019 · 7 comments
Closed

JTReg Test Failure - serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorArrayAllSampledTest.java #5969

JasonFengJ9 opened this issue May 31, 2019 · 7 comments
Labels
comp:gc test failure

Comments

@JasonFengJ9
Copy link
Member

JasonFengJ9 commented May 31, 2019 •
edited
Loading

Failure link

N/A - local run with #5968

Optional info

  • intermittent failure (yes)
  • new feature test

Failure output

There are two failures:

J9Generic_Signal_Number=00000004 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000001
Handler1=00007F9DBEF3C480 Handler2=00007F9DBE839B30 InaccessibleAddress=0000000000000000
RDI=00007F9DBC075490 RSI=00007F9DBC074030 RAX=0000000000000001 RBX=00007F9DB8044AF0
RCX=00007F9DBC075488 RDX=0000000000000400 R8=0000000099669966 R9=00000000800B0001
R10=0000000000000400 R11=00007F9DB733A4C8 R12=00007F9D60001A68 R13=00000000FF9C0000
R14=0000000000000001 R15=0000000000000000
RIP=00007F9DBC70922E GS=0000 FS=0000 RSP=00007F9D9CF29B10
EFlags=0000000000010246 CS=0033 RBP=00007F9D9CF29B40 ERR=0000000000000004
TRAPNO=000000000000000E OLDMASK=0000000000000000 CR2=0000000000000000
xmm0 00000000e00935c0 (f: 3758700032.000000, d: 1.857045e-314)
xmm1 00007f9dbcaa1288 (f: 3165262592.000000, d: 6.932504e-310)
xmm2 0000000000000000 (f: 0.000000, d: 0.000000e+00)
xmm3 00007f9dbcaa1208 (f: 3165262336.000000, d: 6.932504e-310)
xmm4 0000000000000000 (f: 0.000000, d: 0.000000e+00)
xmm5 0000000000000ae3 (f: 2787.000000, d: 1.376961e-320)
xmm6 0000000000000ae2 (f: 2786.000000, d: 1.376467e-320)
xmm7 0000000000000000 (f: 0.000000, d: 0.000000e+00)
xmm8 0000000000000000 (f: 0.000000, d: 0.000000e+00)
xmm9 fff14210fff141f8 (f: 4294001152.000000, d: -nan)
xmm10 fff14240fff14228 (f: 4294001152.000000, d: -nan)
xmm11 fff1428800000000 (f: 0.000000, d: -nan)
xmm12 fff142d0fff142b8 (f: 4294001408.000000, d: -nan)
xmm13 fff1431800000000 (f: 0.000000, d: -nan)
xmm14 0000000000000000 (f: 0.000000, d: 0.000000e+00)
xmm15 fff1436000000000 (f: 0.000000, d: -nan)
Module=/team/2019-openj9-builds/xa-docker/jdk11/jdk11-0526/openj9-openjdk-jdk11/build/linux-x86_64-normal-server-release/images/jdk/lib/compressedrefs/libj9gc29.so
Module_base_address=00007F9DBC5DD000

Diagnose files are at /team/triage/gitissue5969/gc-core-0x00020002.

J9Generic_Signal_Number=00000004 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000001
Handler1=00007F93BDF7D480 Handler2=00007F93BD87AB30 InaccessibleAddress=0000000000000058
RDI=0000000000000008 RSI=0000000000000010 RAX=0000000000000000 RBX=0000000001115D18
RCX=00000000E007AD28 RDX=00007F93BE26F920 R8=0000000000FF8E00 R9=0000000000000000
R10=0000000000000000 R11=0000000000000000 R12=00000000E005F688 R13=00000000E007AD28
R14=0000000000000000 R15=0000000001115DF0
RIP=00007F93BDF0CCA4 GS=0000 FS=0000 RSP=00007F936FEB7480
EFlags=0000000000010246 CS=0033 RBP=00000000FFCE0000 ERR=0000000000000004
TRAPNO=000000000000000E OLDMASK=0000000000000000 CR2=0000000000000058
xmm0 00007f9350006e60 (f: 1342205568.000000, d: 6.930292e-310)
xmm1 00007f9350006e38 (f: 1342205440.000000, d: 6.930292e-310)
xmm2 00007f936feb7840 (f: 1877702656.000000, d: 6.930319e-310)
xmm3 0000000000000000 (f: 0.000000, d: 0.000000e+00)
xmm4 0000000000000000 (f: 0.000000, d: 0.000000e+00)
xmm5 00000000000000ff (f: 255.000000, d: 1.259867e-321)
xmm6 00007f9350006e60 (f: 1342205568.000000, d: 6.930292e-310)
xmm7 00007f9350006e38 (f: 1342205440.000000, d: 6.930292e-310)
xmm8 3bbcc86800000000 (f: 0.000000, d: 6.095003e-21)
xmm9 3fd2d11e51a9b782 (f: 1370077056.000000, d: 2.940136e-01)
xmm10 3d74500e2dd03677 (f: 768620160.000000, d: 1.154644e-12)
xmm11 402db340e135b73c (f: 3778393856.000000, d: 1.485010e+01)
xmm12 3ca0000000000000 (f: 0.000000, d: 1.110223e-16)
xmm13 bfd2d11e51a9b780 (f: 1370077056.000000, d: -2.940136e-01)
xmm14 402d1cb7eea86980 (f: 4004014592.000000, d: 1.455609e+01)
xmm15 bc790b2aa541283b (f: 2772510720.000000, d: -2.172188e-17)
Module=/team/2019-openj9-builds/xa-docker/jdk11/jdk11-0526/openj9-openjdk-jdk11/build/linux-x86_64-normal-server-release/images/jdk/lib/compressedrefs/libj9vm29.so
Module_base_address=00007F93BDEDE000

#12 <signal handler called>
#13 0x00007fd17bc8fca4 in VM_BytecodeInterpreter::objectArrayStoreAllowed (storeValue=<optimized out>, array=<optimized out>, this=<optimized out>) at BytecodeInterpreter.hpp:1424
#14 VM_BytecodeInterpreter::aastore (_pc=<optimized out>, _sp=<optimized out>, this=<optimized out>) at BytecodeInterpreter.hpp:5818
#15 VM_BytecodeInterpreter::run (this=0x8, this@entry=0x7fd159e0a810, vmThread=0x0) at BytecodeInterpreter.hpp:9818
#16 0x00007fd17bc7a120 in bytecodeLoop (currentThread=<optimized out>) at BytecodeInterpreter.cpp:109
#17 0x00007fd17bd3b672 in c_cInterpreter () at xcinterp.s:160
#18 0x00007fd17bceceb0 in runCallInMethod (env=0x7fd159e0a920, receiver=0x12b3df0, clazz=0x0, methodID=0x7fd17c406038, args=0x7fd159e0ab08) at callin.cpp:1073
#19 0x00007fd17bd053cb in gpProtectedRunCallInMethod (entryArg=0x7fd159e0aab0) at jnicsup.cpp:258
#20 gpCheckCallin (env=env@entry=0x1267500, receiver=<optimized out>, cls=cls@entry=0x0, methodID=0x7fd17c406038, args=<optimized out>) at jnicsup.cpp:439
#21 0x00007fd17bd02aa1 in callVirtualIntMethodV (env=0x1267500, receiver=<optimized out>, methodID=<optimized out>, va=<optimized out>) at jnicgen.c:90
#22 0x00007fd1793ca7a6 in JNIEnv_::CallBooleanMethod (this=this@entry=0x1267500, obj=obj@entry=0x12b3df0, methodID=methodID@entry=0x7fd17c406038) at ../include/jni.h:472
#23 0x00007fd1793c8e64 in process_reference (refMID=0x7fd17c406038, reference=<optimized out>, vmThread=0x1267500) at FinalizerSupport.cpp:421
#24 process (finalizeJob=<optimized out>, finalizeJob=<optimized out>, referenceEnqueueImplMID=0x7fd17c406038, runFinalizeMID=0x7fd17c406010, j9VMInternalsClass=0x7fd17c083e70, vmThread=0x1267500)
    at FinalizerSupport.cpp:455
#25 FinalizeSlaveThread (arg=0x7fd1140009b0, arg@entry=0x12b3df0) at FinalizerSupport.cpp:607
#26 FinalizeSlaveThreadGlue (portLib=portLib@entry=0x7fd180636980 <j9portLibrary>, userData=userData@entry=0x7fd1140009b0) at FinalizerSupport.cpp:807
#27 0x00007fd17b5fe7d8 in omrsig_protect (portLibrary=0x7fd180636980 <j9portLibrary>, fn=0x7fd1793c8b10 <FinalizeSlaveThreadGlue(J9PortLibrary*, void*)>, fn_arg=0x7fd1140009b0, 
    handler=0x7fd17bd00480 <structuredSignalHandlerVM>, handler_arg=0x7fd17c00d720, flags=126, result=0x7fd159e0ae28) at ../../omr/port/unix/omrsignal.c:396
#28 0x00007fd1793c8585 in gpProtectedFinalizeSlaveThread (entryArg=<optimized out>) at FinalizerSupport.cpp:817
#29 0x00007fd17ba552a2 in thread_wrapper (arg=0x7fd17c3711e8) at ../omr/thread/common/omrthread.c:1605
#30 0x00007fd181b3e6ba in start_thread (arg=0x7fd159e0d700) at pthread_create.c:333
#31 0x00007fd18146041d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

Diagnose files are at /team/triage/gitissue5969/vm-core.

> !gccheck objectheap
Starting GC Check
Checking HEAP...  <gc check (1): from debugger: HEAP: Object slot e004e540(e004e548) -> ff9c0000: class pointer is null>
done (1741 ms).
Done (1743ms)
> !j9object 0xe004e540
!J9Object 0x00000000E004E540 {
      struct J9Class* clazz = !j9class 0x88E700   // java/lang/ref/ReferenceQueue
      Object flags = 0x00000010;
      I lockword = 0x00961500 (offset=0) (java/lang/Object) <hidden>
      [Ljava/lang/ref/Reference; references = !fj9object 0xff9c0000 (offset=4) (java/lang/ref/ReferenceQueue) <--- bad object
      I head = 0x00000000 (offset=8) (java/lang/ref/ReferenceQueue)
      I tail = 0x00000000 (offset=12) (java/lang/ref/ReferenceQueue)
      Z empty = 0x00000001 (offset=16) (java/lang/ref/ReferenceQueue)
}

> !fj9object 0xff9c0000
<can not read RAM class address>

> !j9x 0xff9c0000
0xFF9C0000 :  0000000000000001 0000000000320000 [ ..........2..... ]
0xFF9C0010 :  0000000000000000 0000000000000000 [ ................ ]

https://github.com/eclipse/openj9/blob/79bb39a30f3e175d4b5794a1d7a20a56ee6ccdf0/jcl/src/java.base/share/classes/java/lang/ref/ReferenceQueue.java#L193-L206
This shows that java.lang.ref.ReferenceQueue.references was initialized but later becomes invalid. I would ask GC team to help investigation.
fyi @dmitripivkine
@JasonFengJ9 JasonFengJ9 mentioned this issue May 31, 2019
Merged
@dmitripivkine
Copy link
Contributor

Would you please try to use -Xgcpolicy:optthruput and in case it crashed in GC provide system core?

@dmitripivkine
Copy link
Contributor

Also I would like to see all system cores collected if any, particularly for crashes in GC.

@JasonFengJ9
Copy link
Member Author

JasonFengJ9 commented May 31, 2019 •
edited
Loading

Would you please try to use -Xgcpolicy:optthruput and in case it crashed in GC provide system core

Interesting, the failing rate was ~1/5 for default -Xgcpolicy:gencon, but no Segmentation error observed in a 50x run with -Xgcpolicy:optthruput.

@dmitripivkine
Copy link
Contributor

Also would you please try -Xint?

@JasonFengJ9
Copy link
Member Author

Tried -Xint before, similar results.

@dmitripivkine
Copy link
Contributor

this might help: #5993

@JasonFengJ9
Copy link
Member Author

#5993 does fix this issue. Thanks @charliegracie

@JasonFengJ9 JasonFengJ9 mentioned this issue Jun 5, 2019
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
comp:gc test failure
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@DanHeidinga @JasonFengJ9 @dmitripivkine