Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(helm-chart): remove helm-chart support #110

Merged
merged 26 commits into from
Dec 16, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
c3f3302
feat(helm-chart): remove helm-chart support
unknowntpo Nov 25, 2024
c7e41ea
feat(playground.sh): revert helm related code
unknowntpo Nov 26, 2024
9b8b8fc
feat(init/common/init_metalake_catalog.sh): revert init_metalake_cata…
unknowntpo Nov 26, 2024
2353aae
feat(healthcheck/mysql-healthcheck.sh): revert
unknowntpo Nov 26, 2024
1d1ea10
feat(healthcheck/gravitino-healthcheck.sh): revert
unknowntpo Nov 26, 2024
6c7a6f1
feat(healthcheck/hive-healthcheck.sh): revert
unknowntpo Nov 26, 2024
23e1e74
feat(healthcheck/trino-healthcheck.sh): revert
unknowntpo Nov 26, 2024
5fc913a
feat(docker-compose.yaml): revert
unknowntpo Nov 26, 2024
2d8a533
feat(init/jupyter/gravitino-trino-example.ipynb): revert
unknowntpo Nov 26, 2024
f214024
feat(init/jupyter/gravitino_llamaIndex_demo.ipynb): revert
unknowntpo Nov 26, 2024
752bedb
feat(init/jupyter/gravitino-fileset-example.ipynb): revert
unknowntpo Nov 26, 2024
57a91ae
feat(init/jupyter/gravitino-spark-trino-example.ipynb): revert
unknowntpo Nov 26, 2024
261e5eb
feat(README.md): revert
unknowntpo Nov 26, 2024
0e3b4c9
feat(playground.sh): revert helm related code
unknowntpo Nov 26, 2024
25c6507
fix: revert init.sh
unknowntpo Nov 26, 2024
9faba33
refactor(README.md): revert helm-chart related doc
unknowntpo Nov 27, 2024
1262b21
Update playground.sh
unknowntpo Nov 27, 2024
353d1cf
refactor(playground.sh): tweak --enable-ranger option
unknowntpo Nov 27, 2024
e8b70a3
refactor(hive/init.sh): should use sed to replace __REPLACE__HOST_NAME
unknowntpo Dec 4, 2024
1640e8f
refactor(healthcheck/trino-healthcheck.sh): use localhost as hostname
unknowntpo Dec 5, 2024
af41f3f
refactor(init/spark/init.sh): no need to substitute host ip info
unknowntpo Dec 5, 2024
59e18da
refactor(init/gravitino/gravitino.conf): hard code docker service name
unknowntpo Dec 5, 2024
f4404e6
refactor(healthcheck/gravitino-healthcheck.sh): use 127.0.0.1 as host ip
unknowntpo Dec 5, 2024
badb490
refactor(hive/init.sh): no need to replace hive host name
unknowntpo Dec 5, 2024
ed6c5b5
refactor(hive/init.sh): delete last line of start.sh if it is tail -f…
unknowntpo Dec 10, 2024
7ed9dae
refactor(init/hive): use sed to replace tail -f command
unknowntpo Dec 11, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 3 additions & 64 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,6 @@ Depending on your network and computer, startup time may take 3-5 minutes. Once
## Prerequisites

Install Git (optional), Docker, Docker Compose.
Docker Desktop (or Orbstack) with Kubernetes enabled and helm CLI is required if you use helm-chart to deploy services.

## System Resource Requirements

Expand Down Expand Up @@ -63,81 +62,21 @@ git clone [email protected]:apache/gravitino-playground.git
cd gravitino-playground
```

#### Docker

##### Start
### Start

```
./playground.sh docker start
```

##### Check status
### Check status
```shell
./playground.sh docker status
```
##### Stop playground
### Stop playground
```shell
./playground.sh docker stop
```

#### Kubernetes

Enable Kubernetes in Docker Desktop or Orbstack.

In the project root directory, execute this command:

```
helm upgrade --install gravitino-playground ./helm-chart/ --create-namespace --namespace gravitino-playground --set projectRoot=$(pwd)
```

##### Start

```
./playground.sh k8s start
```

##### Check status
```shell
./playground.sh k8s status
```

##### Port Forwarding

To access the pods or services at `localhost`, you need to do these steps:

1. Log in to the Gravitino playground Trino pod using the following command:

```
TRINO_POD=$(kubectl get pods --namespace gravitino-playground -l app=trino -o jsonpath="{.items[0].metadata.name}")
kubectl exec $TRINO_POD -n gravitino-playground -it -- /bin/bash
```
2. Log in to the Gravitino playground Spark pod using the following command:

```
SPARK_POD=$(kubectl get pods --namespace gravitino-playground -l app=spark -o jsonpath="{.items[0].metadata.name}")
kubectl exec $SPARK_POD -n gravitino-playground -it -- /bin/bash
```

3. Port-forward the Gravitino service to access it at `localhost:8090`.

```
kubectl port-forward svc/gravitino -n gravitino-playground 8090:8090
```

4. Port-forward the Jupyter Notebook service to access it at `localhost:8888`.

```
kubectl port-forward svc/jupyternotebook -n gravitino-playground 8888:8888
```

##### Stop playground
```shell
./playground.sh k8s stop
```




## Experiencing Apache Gravitino with Trino SQL

### Using Trino CLI in Docker Container
Expand Down
12 changes: 0 additions & 12 deletions docker-compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -65,9 +65,6 @@ services:
- "8090:8090"
- "9001:9001"
container_name: playground-gravitino
environment:
- MYSQL_HOST_IP=mysql
- HIVE_HOST_IP=hive
depends_on:
ranger :
condition: service_healthy
Expand Down Expand Up @@ -160,10 +157,6 @@ services:
entrypoint: /bin/bash /tmp/spark/init.sh
environment:
- HADOOP_USER_NAME=root
- GRAVITINO_HOST_IP=gravitino
- GRAVITINO_HOST_PORT=8090
- HIVE_HOST_IP=hive
- TRINO_HOST_IP=trino
ports:
- "14040:4040"
volumes:
Expand All @@ -173,11 +166,6 @@ services:
jupyter:
image: jupyter/pyspark-notebook:spark-3.4.1
container_name: playground-jupyter
environment:
- GRAVITINO_HOST_IP=gravitino
- HIVE_HOST_IP=hive
- TRINO_HOST_IP=trino
- POSTGRES_HOST_IP=postgresql
ports:
- "18888:8888"
volumes:
Expand Down
4 changes: 1 addition & 3 deletions healthcheck/gravitino-healthcheck.sh
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,8 @@ max_attempts=3
attempt=0
success=false

HOST_IP=${GRAVITINO_HOST_IP:-localhost}

while [ $attempt -lt $max_attempts ]; do
response=$(curl -X GET -H "Content-Type: application/json" http://${HOST_IP}:8090/api/version)
response=$(curl -X GET -H "Content-Type: application/json" http://127.0.0.1:8090/api/version)

if echo "$response" | grep -q "\"code\":0"; then
success=true
Expand Down
6 changes: 1 addition & 5 deletions healthcheck/hive-healthcheck.sh
Original file line number Diff line number Diff line change
Expand Up @@ -19,12 +19,8 @@
#
set -ex

# Set Hive connection details
HOST_IP=${HIVE_HOST_IP:-localhost}
HIVE_PORT="10000"

# Attempt to connect to Hive using curl
curl -s -o /dev/null -w "%{http_code}" http://${HOST_IP}:${HIVE_PORT}
curl -s -o /dev/null -w "%{http_code}" http://localhost:10000

# Check the HTTP status code
if [ $? -eq 0 ]; then
Expand Down
3 changes: 1 addition & 2 deletions healthcheck/mysql-healthcheck.sh
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,7 @@
#
set -ex

HOST_IP=${MYSQL_HOST_IP:-localhost}
mysqladmin ping -h ${HOST_IP} -p${MYSQL_ROOT_PASSWORD}
mysqladmin ping -h localhost -p${MYSQL_ROOT_PASSWORD}
if [ $? -eq 0 ]; then
echo "MySQL container started successfully."
exit 0
Expand Down
2 changes: 1 addition & 1 deletion healthcheck/trino-healthcheck.sh
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
set -ex

# Because trino-connector must first synchronize a default metalake from the Gravitino server
response=$(trino --server ${TRINO_HOST_IP}:8080 --execute "SHOW CATALOGS LIKE 'catalog_hive'")
response=$(trino --server localhost:8080 --execute "SHOW CATALOGS LIKE 'catalog_hive'")
if echo "$response" | grep -q catalog_hive; then
echo "Gravitino Trino connector has finished synchronizing metadata"
else
Expand Down
29 changes: 0 additions & 29 deletions helm-chart/.helmignore

This file was deleted.

9 changes: 0 additions & 9 deletions helm-chart/Chart.yaml

This file was deleted.

1 change: 0 additions & 1 deletion helm-chart/healthcheck

This file was deleted.

1 change: 0 additions & 1 deletion helm-chart/init

This file was deleted.

24 changes: 0 additions & 24 deletions helm-chart/templates/NOTES.txt

This file was deleted.

62 changes: 0 additions & 62 deletions helm-chart/templates/_helpers.tpl

This file was deleted.

84 changes: 0 additions & 84 deletions helm-chart/templates/gravitino.yaml

This file was deleted.

Loading