jdk20 MiniMix bigdecimal

[pshipton]

The following tests failed on OpenJ9 jdk20. Seems to me we had a similar issue in jdk19 and the tests were removed.

https://openj9-jenkins.osuosl.org/job/Test_openjdk20_j9_extended.system_aarch64_mac_Nightly/1/tapResults/

LT  testFailure: testItem_0839(net.adoptopenjdk.test.bigdecimal.TestSuite057): expected:<-9[E+18]> but was:<-9[000000000000000000]>
LT  org.junit.ComparisonFailure: expected:<-9[E+18]> but was:<-9[000000000000000000]>
LT  	at org.junit.Assert.assertEquals(Assert.java:115)
LT  	at org.junit.Assert.assertEquals(Assert.java:144)
LT  	at net.adoptopenjdk.test.bigdecimal.TestSuite057.testItem_0839(TestSuite057.java:5778)

https://openj9-jenkins.osuosl.org/job/Test_openjdk20_j9_extended.system_aarch64_linux_Nightly/1/tapResults/

LT  testFailure: testItem_0159(net.adoptopenjdk.test.bigdecimal.TestSuite019): expected:<-4[E+1]> but was:<-4[0]>
LT  org.junit.ComparisonFailure: expected:<-4[E+1]> but was:<-4[0]>
LT  	at org.junit.Assert.assertEquals(Assert.java:115)
LT  	at org.junit.Assert.assertEquals(Assert.java:144)
LT  	at net.adoptopenjdk.test.bigdecimal.TestSuite019.testItem_0159(TestSuite019.java:1064)

LT  testFailure: testItem_0113(net.adoptopenjdk.test.bigdecimal.TestSuite048): expected:<3[.2E+2]> but was:<3[20]>
LT  org.junit.ComparisonFailure: expected:<3[.2E+2]> but was:<3[20]>
LT  	at org.junit.Assert.assertEquals(Assert.java:115)
LT  	at org.junit.Assert.assertEquals(Assert.java:144)
LT  	at net.adoptopenjdk.test.bigdecimal.TestSuite048.testItem_0113(TestSuite048.java:763)

LT  testFailure: testItem_0205(net.adoptopenjdk.test.bigdecimal.TestSuite048): expected:<-3[E+2]> but was:<-3[00]>
LT  org.junit.ComparisonFailure: expected:<-3[E+2]> but was:<-3[00]>
LT  	at org.junit.Assert.assertEquals(Assert.java:115)
LT  	at org.junit.Assert.assertEquals(Assert.java:144)
LT  	at net.adoptopenjdk.test.bigdecimal.TestSuite048.testItem_0205(TestSuite048.java:1429)

LT  testFailure: testItem_0282(net.adoptopenjdk.test.bigdecimal.TestSuite024): expected:<2[E+308]> but was:<2[00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000]>
LT  org.junit.ComparisonFailure: expected:<2[E+308]> but was:<2[00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000]>
LT  	at org.junit.Assert.assertEquals(Assert.java:115)
LT  	at org.junit.Assert.assertEquals(Assert.java:144)
LT  	at net.adoptopenjdk.test.bigdecimal.TestSuite024.testItem_0282(TestSuite024.java:2018)

https://openj9-jenkins.osuosl.org/job/Test_openjdk20_j9_extended.system_ppc64_aix_Nightly/1/tapResults/

LT  testFailure: testItem_0328(net.adoptopenjdk.test.bigdecimal.TestSuite033): expected:<6[E+2]> but was:<6[00]>
LT  org.junit.ComparisonFailure: expected:<6[E+2]> but was:<6[00]>
LT  	at org.junit.Assert.assertEquals(Assert.java:115)
LT  	at org.junit.Assert.assertEquals(Assert.java:144)
LT  	at net.adoptopenjdk.test.bigdecimal.TestSuite033.testItem_0328(TestSuite033.java:2373)

LT  testFailure: testItem_0017(net.adoptopenjdk.test.bigdecimal.TestSuite015): expected:<0[E+2147483646]> but was:<0[]>
LT  org.junit.ComparisonFailure: expected:<0[E+2147483646]> but was:<0[]>
LT  	at org.junit.Assert.assertEquals(Assert.java:115)
LT  	at org.junit.Assert.assertEquals(Assert.java:144)
LT  	at net.adoptopenjdk.test.bigdecimal.TestSuite015.testItem_0017(TestSuite015.java:150)

There are more in the extended.system testing for other platforms.
https://openj9-jenkins.osuosl.org/job/Pipeline-Build-Test-JDK20/5/

[pshipton]

@llxia @Mesbah-Alam pls take a look.

[Mesbah-Alam]

Hi @pshipton ,

net.adoptopenjdk.test.bigdecimal.TestSuite057 is not excluded in JDK 19, it's actually working normally on 19 : https://openj9-jenkins.osuosl.org/job/Test_openjdk19_j9_extended.system_aarch64_mac_Nightly_testList_1/117/consoleFull.

org.junit.ComparisonFailure: expected:<-9[E+18]> but was:<-9[000000000000000000]>

Is this failure on JDK20 aarch64 Linux indicating a real issue?

Similar failure is not seen on JDK20 AIX : https://openj9-jenkins.osuosl.org/job/Test_openjdk20_j9_extended.system_ppc64_aix_Nightly/1/tapResults/.

[pshipton]

If you look at the build https://openj9-jenkins.osuosl.org/job/Pipeline-Build-Test-JDK20/5/ you can see that different platforms are failing different tests. Can you pls try the tests on Adoptium and see if they similarly fail. OpenJ9 didn't change any related behavior for jdk20 afaik.

[pshipton]

I tried some grinders on the latest and the failures are still occurring with OpenJ9.
https://openj9-jenkins.osuosl.org/job/Grinder/2087/
https://openj9-jenkins.osuosl.org/job/Grinder/2088/

[pshipton]

Running on an Adoptium jdk20 build also fails.
https://openj9-jenkins.osuosl.org/view/Test/job/Grinder/2089/

17:06:05  openjdk version "20-beta" 2023-03-21
17:06:05  OpenJDK Runtime Environment Temurin-20+36-202303170331 (build 20-beta+36-202303170331)
17:06:05  OpenJDK 64-Bit Server VM Temurin-20+36-202303170331 (build 20-beta+36-202303170331, mixed mode, sharing)

[Mesbah-Alam]

From the readme: "The Math suite contains simple tests which are designed to exercise the Math apis".

Are these math APIs supposed to generate different results on different platforms?

Looking at a failed test method on AIX in https://openj9-jenkins.osuosl.org/job/Test_openjdk20_j9_extended.system_ppc64_aix_Nightly/1/tapResults/ :

 testItem_0328(net.adoptopenjdk.test.bigdecimal.TestSuite033): expected:<6[E+2]> but was:<6[00]>

The same test passes on Aarch64 Linux in Adoptiom JDK20 :

17:09:57  LT  21:09:56.909 -   33 JUnit[net.adoptopenjdk.test.bigdecimal.TestSuite033]  Weighting=1 

Similarly, it passes on Aarch64 Linux on OpenJ9 : https://openj9-jenkins.osuosl.org/job/Test_openjdk20_j9_extended.system_aarch64_linux_Nightly/1/tapResults/.

[pshipton]

Even on the same platform, different tests fail between OpenJ9 and Adoptium, and different tests fail from run to run. There is something else going on. Perhaps a random sub-set of tests is being run instead of a consistent set.

I looked at all of the particular failures below, and they all have checks for JavaSpecVersionChecker.isJDK13OrNewer() but the behavior has reverted to the pre Java 13 behavior. I'm guessing if you update isJDK13OrNewer() to return false for jdk20 then the tests will pass.

OpenJ9:
https://openj9-jenkins.osuosl.org/job/Test_openjdk20_j9_extended.system_aarch64_linux_Nightly/1/tapResults/
https://openj9-jenkins.osuosl.org/job/Grinder/2088/tapResults/
Adoptium:
https://openj9-jenkins.osuosl.org/view/Test/job/Grinder/2089/tapResults/

[smlambert]

related: adoptium/aqa-systemtest#482

[Mesbah-Alam]

I'm guessing if you update isJDK13OrNewer() to return false for jdk20 then the tests will pass.

Yes, confirming that that is indeed the case.

Also, following adoptium/aqa-systemtest#482, updated the test to check for 12 < version < 20, instead of version > 12 and it passes in jdk20. Grinder links in following PR : adoptium/aqa-systemtest#483.

[Mesbah-Alam]

The MiniMix_5m test is now passing on JDK 20 https://openj9-jenkins.osuosl.org/job/Test_openjdk20_j9_extended.system_aarch64_mac_Nightly/8/tapResults/.

Can this issue be closed?