[CI][Packaging] Java jars job is currently failing on arm64 #41577
Comments
|
@danepitkin @vibhatha @lidavidm any idea what is the issue on the java-jars? This is currently blocking 16.1.0, if someone can take a look. |
|
Though we didn't cherry-pick the suspected commit so something else is going on? |
|
ok, let me cherry-pick: 2fd3fcc manually and validate |
|
I thought that commit didn't fix the issue? |
|
I'm saying that linked issue suspects 9090e67 as the cause. But that's not in 16.1.0, so there's some other cause. |
|
The release was failing on a slightly different point which might be solved by only adding the other commit I mentioned. I assumed (probably wrongly) that this would also be required for the release. I am testing at the moment, see: #41437 (comment) |
|
Yes that commit didn't fix it as far as I understand. I got the same issue in another PR today. |
|
Sure @lidavidm, I can. |
|
It sounds like either: we can downgrade maven-source-plugin, or we can see if java_full_build.sh is specifying any duplicate phases |
|
I looked into the solution, I will probably start testing the ideas within the hour. |
|
Might also fix: #41571 |
|
I don't think that's related, I would suspect bad usage of modules on our side or theirs |
|
I am testing a few ideas here: #41586 |
|
Adding the following to the maint-16.1.0 branch: $ git log -p b28633cfe5263ac977598f058b53d76f95d3d1fc
commit b28633cfe5263ac977598f058b53d76f95d3d1fc
Author: Raúl Cumplido <[email protected]>
Date: Wed May 8 10:15:17 2024 +0200
MINOR: [Release] Manually commit fix to avoid cherry-pick conflict for changing runner on java-jars
diff --git a/dev/tasks/java-jars/github.yml b/dev/tasks/java-jars/github.yml
index 0437ee7..eb9478e 100644
--- a/dev/tasks/java-jars/github.yml
+++ b/dev/tasks/java-jars/github.yml
@@ -80,7 +80,7 @@ jobs:
fail-fast: false
matrix:
platform:
- - { runs_on: ["macos-latest"], arch: "x86_64"}
+ - { runs_on: ["macos-13"], arch: "x86_64"}
- { runs_on: ["macos-14"], arch: "aarch_64" }
env:
MACOSX_DEPLOYMENT_TARGET: "10.15"
commit ac8af222dd2e5800e2c6cbf45a5bdbb2309deed4
Author: Raúl Cumplido <[email protected]>
fixes the initial issue but fails somewhere else: I've retried 3 times to see if it was a fluke but seems to be pretty consistent: |
|
Meaning the issue on main is not reproducible on the 16.1.0 release but there's something else to be fixed. |
|
I mean if that is the case, the documentation describes the solution https://arrow.apache.org/docs/java/install.html#java-compatibility java --add-opens=java.base/java.nio=org.apache.arrow.memory.core,ALL-UNNAMED -jar ...
env _JAVA_OPTIONS="--add-opens=java.base/java.nio=org.apache.arrow.memory.core,ALL-UNNAMED" java -jar ...cc @lidavidm |
|
https://github.com/apache/arrow/blob/main/java/dataset/pom.xml#L204 Could this solve? |
This doesn't seem to have duplicate phases, right? # build the entire project
mvn clean \
install \
assembly:single \
source:jar \
javadoc:jar \
-Papache-release \
-Parrow-c-data \
-Parrow-jni \
-Darrow.cpp.build.dir=$dist_dir \
-Darrow.c.jni.dist.dir=$dist_dir \
-DdescriptorId=source-release |
It does solve indeed. Thanks @vibhatha |
|
That's great. I am working on a parallel fix on main. Still I couldn't figure out it. But these findings could help to narrow it down, I guess. |
|
Ah, how about using |
|
@raulcd Is apache/arrow-java#92 a blocker for 16.1.0 release? |
|
Sorry for missing this...we can add it to main too. It should be OK. |
|
I assume it would fail without that once we fix the current java-jars issue on main anyways. (I believe @jbonofre has been looking at that, it sounds like this has been an issue for multiple Apache projects.) |
|
@lidavidm I tested a few ideas here But the Also I am not sure if we have multiple phases on the build script either. Sorry I am still trying to figure out a solution for this. |
|
We can let JB look at it then, I thought it was causing problems for the 16.1.0 branch. I assumed we could just override the parent pom and downgrade the plugin. It would be nice to reproduce it locally and see if we can avoid the duplicate artifact in the first place. Of course, the other solution might be to drop Java 8 and drop the plugin, and hope the rest of the build works. |
|
I need to see if we can bypass the parent pom using a particular version of a dependency it is using, like we do for regular dependencies. Not very familiar with this part, but I will keep looking in parallel. |
|
I am going to close this as the 16.1.0 release has been solved. The issue for main can be tracked on the original: apache/arrow-java#92 |
Describe the bug, including details regarding any error messages, version, and platform.
The java-jars jobs is failing with:
This seems to be failing since
macos-latestmoved to arm64.Component(s)
Continuous Integration, Packaging
The text was updated successfully, but these errors were encountered: