Skip to content

Commit

Permalink
[apache#5349] improvement(docker): Update Gravitino dockerfile to put…
Browse files Browse the repository at this point in the history
… cloud bundles into the docker image. (apache#5358)

### What changes were proposed in this pull request?

Copy cloud storage bundles jars to folder
`$GRAVITINO_HOME/catalogs/hadoop/libs` when building Gravitino docker
image to free from manually doing it.

### Why are the changes needed?

We aim to make Gravitino user-friendly for managing cloud storage. 

Fix: apache#5349 

### Does this PR introduce _any_ user-facing change?

N/A.

### How was this patch tested?

Manually verify locally.
  • Loading branch information
yuqi1129 authored Oct 31, 2024
1 parent 1cbaaf7 commit a2f7093
Show file tree
Hide file tree
Showing 2 changed files with 15 additions and 2 deletions.
11 changes: 10 additions & 1 deletion dev/docker/gravitino/gravitino-dependency.sh
Original file line number Diff line number Diff line change
Expand Up @@ -22,15 +22,24 @@ gravitino_dir="$(dirname "${BASH_SOURCE-$0}")"
gravitino_dir="$(cd "${gravitino_dir}">/dev/null; pwd)"
gravitino_home="$(cd "${gravitino_dir}/../../..">/dev/null; pwd)"

# Build the Gravitino project
${gravitino_home}/gradlew clean build -x test

rm -rf ${gravitino_home}/distribution
# Prepare compile Gravitino packages
${gravitino_home}/gradlew clean compileDistribution -x test
${gravitino_home}/gradlew compileDistribution -x test

# Removed old packages, Avoid multiple re-executions using the wrong file
rm -rf "${gravitino_dir}/packages"
mkdir -p "${gravitino_dir}/packages"

cp -r "${gravitino_home}/distribution/package" "${gravitino_dir}/packages/gravitino"

# Copy the Aliyun, AWS, and GCP bundles to the Hadoop catalog libs
cp ${gravitino_home}/bundles/aliyun-bundle/build/libs/*.jar "${gravitino_dir}/packages/gravitino/catalogs/hadoop/libs"
cp ${gravitino_home}/bundles/aws-bundle/build/libs/*.jar "${gravitino_dir}/packages/gravitino/catalogs/hadoop/libs"
cp ${gravitino_home}/bundles/gcp-bundle/build/libs/*.jar "${gravitino_dir}/packages/gravitino/catalogs/hadoop/libs"

# Keeping the container running at all times
cat <<EOF >> "${gravitino_dir}/packages/gravitino/bin/gravitino.sh"
Expand Down
6 changes: 5 additions & 1 deletion docs/docker-image-details.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,15 @@ You can deploy the service with the Gravitino Docker image.
Container startup commands

```shell
docker run --rm -d -p 8090:8090 -p 9001:9001 apache/gravitino:0.6.1-incubating
docker run --rm -d -p 8090:8090 -p 9001:9001 apache/gravitino:0.7.0-incubating
```

Changelog

- apache/gravitino:0.7.0-incubating
- Based on Gravitino 0.7.0-incubating, you can know more information from 0.7.0-incubating [release notes](https://github.com/apache/gravitino/releases/tag/v0.7.0-incubating).
- Place bundle jars (gravitino-aws-bundle.jar, gravitino-gcp-bundle.jar, gravitino-aliyun-bundle.jar) in the `${GRAVITINO_HOME}/catalogs/hadoop/libs` folder to support the cloud storage catalog without manually adding the jars to the classpath.

- apache/gravitino:0.6.1-incubating
- Based on Gravitino 0.6.1-incubating, you can know more information from 0.6.1-incubating release notes.

Expand Down

0 comments on commit a2f7093

Please sign in to comment.