Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[#2874] improvement(CI): remove SSH service from Hive container #2871

Merged
merged 2 commits into from
Apr 11, 2024
Merged

[#2874] improvement(CI): remove SSH service from Hive container #2871

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
590e555
remove ssh
mchades Apr 10, 2024
d7288ab
fix trino it
mchades Apr 10, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion catalogs/catalog-hadoop/build.gradle.kts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ tasks.test {
dependsOn(tasks.jar)

doFirst {
environment("GRAVITINO_CI_HIVE_DOCKER_IMAGE", "datastrato/gravitino-ci-hive:0.1.9")
environment("GRAVITINO_CI_HIVE_DOCKER_IMAGE", "datastrato/gravitino-ci-hive:0.1.10")
}

val init = project.extra.get("initIntegrationTest") as (Test) -> Unit
Expand Down
2 changes: 1 addition & 1 deletion catalogs/catalog-hive/build.gradle.kts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -164,7 +164,7 @@ tasks.test {
dependsOn(tasks.jar)

doFirst {
environment("GRAVITINO_CI_HIVE_DOCKER_IMAGE", "datastrato/gravitino-ci-hive:0.1.9")
environment("GRAVITINO_CI_HIVE_DOCKER_IMAGE", "datastrato/gravitino-ci-hive:0.1.10")
}

val init = project.extra.get("initIntegrationTest") as (Test) -> Unit
Expand Down
2 changes: 1 addition & 1 deletion catalogs/catalog-lakehouse-iceberg/build.gradle.kts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -162,7 +162,7 @@ tasks.test {
dependsOn(tasks.jar)

doFirst {
environment("GRAVITINO_CI_HIVE_DOCKER_IMAGE", "datastrato/gravitino-ci-hive:0.1.9")
environment("GRAVITINO_CI_HIVE_DOCKER_IMAGE", "datastrato/gravitino-ci-hive:0.1.10")
}

val init = project.extra.get("initIntegrationTest") as (Test) -> Unit
Expand Down
14 changes: 8 additions & 6 deletions dev/docker/hive/start.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,16 @@
# This software is licensed under the Apache License version 2.
#

# start ssh
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mchades Do we need also remove apt-get install openssh-server from `hive/Dockerfile/?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if removing this service will cause other issues, this requires more testing.

I suggest keeping it for now and handling it in a separate PR if necessary.

service ssh start
ssh-keyscan localhost > /root/.ssh/known_hosts
ssh-keyscan 0.0.0.0 >> /root/.ssh/known_hosts

# start hdfs
${HADOOP_HOME}/sbin/start-dfs.sh
echo "Starting HDFS..."
echo "Starting NameNode..."
${HADOOP_HOME}/sbin/hadoop-daemon.sh start namenode

echo "Starting DataNode..."
${HADOOP_HOME}/sbin/hadoop-daemon.sh start datanode

# start mysql and create databases/users for hive
echo "Starting MySQL..."
chown -R mysql:mysql /var/lib/mysql
usermod -d /var/lib/mysql/ mysql
service mysql start
Expand All @@ -28,6 +29,7 @@ echo """
""" | mysql --user=root --password=${MYSQL_PWD}

# start hive
echo "Starting Hive..."
${HIVE_HOME}/bin/schematool -initSchema -dbType mysql
${HIVE_HOME}/bin/hive --service hiveserver2 > /dev/null 2>&1 &
${HIVE_HOME}/bin/hive --service metastore > /dev/null 2>&1 &
Expand Down
4 changes: 4 additions & 0 deletions docs/docker-image-details.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,10 @@ You can use this kind of image to test the catalog of Apache Hive.

Changelog

- gravitino-ci-hive:0.1.10
- Remove SSH service from the startup script.
- Use `hadoop-daemon.sh` to start HDFS services.

- gravitino-ci-hive:0.1.9
- Remove cache after installing packages.

Expand Down
2 changes: 1 addition & 1 deletion ...mon/src/test/java/com/datastrato/gravitino/integration/test/container/ContainerSuite.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@ public void startTrinoContainer(
TrinoContainer.builder()
.withEnvVars(
ImmutableMap.<String, String>builder()
.put("HADOOP_USER_NAME", "root")
.put("HADOOP_USER_NAME", "datastrato")
.put("GRAVITINO_HOST_IP", "host.docker.internal")
.put("GRAVITINO_HOST_PORT", String.valueOf(gravitinoServerPort))
.put("GRAVITINO_METALAKE_NAME", metalakeName)
Expand Down
2 changes: 1 addition & 1 deletion integration-test/build.gradle.kts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -153,7 +153,7 @@ tasks.test {

doFirst {
// Gravitino CI Docker image
environment("GRAVITINO_CI_HIVE_DOCKER_IMAGE", "datastrato/gravitino-ci-hive:0.1.9")
environment("GRAVITINO_CI_HIVE_DOCKER_IMAGE", "datastrato/gravitino-ci-hive:0.1.10")
environment("GRAVITINO_CI_TRINO_DOCKER_IMAGE", "datastrato/gravitino-ci-trino:0.1.5")

copy {
Expand Down
2 changes: 1 addition & 1 deletion integration-test/trino-it/docker-compose.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ version: '3.0'
services:

hive:
image: datastrato/gravitino-ci-hive:0.1.9
image: datastrato/gravitino-ci-hive:0.1.10
networks:
- trino-net
container_name: trino-ci-hive
Expand Down
Loading