Skip to content

Commit

Permalink
[SPARK-49117][K8S] Fix docker-image-tool.sh to be up-to-date
Browse files Browse the repository at this point in the history
### What changes were proposed in this pull request?

This PR aims to fix `docker-image-tool.sh` to be up-to-date.

### Why are the changes needed?

Apache Spark 4 dropped Java 11 support. So, we should fix the following.
- #43005
```
-  - Build and push Java11-based image with tag "v3.4.0" to docker.io/myrepo
+  - Build and push Java17-based image with tag "v4.0.0" to docker.io/myrepo
```

Apache Spark 4 requires JDK instead of JRE. So, we should fix the following.
- #45761
```
-    $0 -r docker.io/myrepo -t v3.4.0 -b java_image_tag=11-jre build
+    $0 -r docker.io/myrepo -t v4.0.0 -b java_image_tag=17 build
```

Lastly, `3.4.0` is too old because it's released on April 13, 2023. We had better use v4.0.0.
```
-    $0 -r docker.io/myrepo -t v3.4.0 -p kubernetes/dockerfiles/spark/bindings/python/Dockerfile build
+    $0 -r docker.io/myrepo -t v4.0.0 -p kubernetes/dockerfiles/spark/bindings/python/Dockerfile build
```

### Does this PR introduce _any_ user-facing change?

No functional change because this is a usage message.

### How was this patch tested?

Manual review.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes #47618 from dongjoon-hyun/SPARK-49117.

Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
  • Loading branch information
dongjoon-hyun authored and HyukjinKwon committed Aug 6, 2024
1 parent 6d32472 commit d9c5902
Showing 1 changed file with 8 additions and 8 deletions.
16 changes: 8 additions & 8 deletions bin/docker-image-tool.sh
Original file line number Diff line number Diff line change
Expand Up @@ -261,18 +261,18 @@ Examples:
$0 -m -t testing build
- Build PySpark docker image
$0 -r docker.io/myrepo -t v3.4.0 -p kubernetes/dockerfiles/spark/bindings/python/Dockerfile build
$0 -r docker.io/myrepo -t v4.0.0 -p kubernetes/dockerfiles/spark/bindings/python/Dockerfile build
- Build and push image with tag "v3.4.0" to docker.io/myrepo
$0 -r docker.io/myrepo -t v3.4.0 build
$0 -r docker.io/myrepo -t v3.4.0 push
- Build and push image with tag "v4.0.0" to docker.io/myrepo
$0 -r docker.io/myrepo -t v4.0.0 build
$0 -r docker.io/myrepo -t v4.0.0 push
- Build and push Java11-based image with tag "v3.4.0" to docker.io/myrepo
$0 -r docker.io/myrepo -t v3.4.0 -b java_image_tag=11-jre build
$0 -r docker.io/myrepo -t v3.4.0 push
- Build and push Java17-based image with tag "v4.0.0" to docker.io/myrepo
$0 -r docker.io/myrepo -t v4.0.0 -b java_image_tag=17 build
$0 -r docker.io/myrepo -t v4.0.0 push
- Build and push image for multiple archs to docker.io/myrepo
$0 -r docker.io/myrepo -t v3.4.0 -X build
$0 -r docker.io/myrepo -t v4.0.0 -X build
# Note: buildx, which does cross building, needs to do the push during build
# So there is no separate push step with -X
Expand Down

0 comments on commit d9c5902

Please sign in to comment.