Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support Cassandra with docker-compose in server #2307

Merged
merged 10 commits into from
Oct 23, 2023
14 changes: 10 additions & 4 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,8 @@ COPY --from=build /pkg/apache-hugegraph-incubating-$version/ /hugegraph
LABEL maintainer="HugeGraph Docker Maintainers <[email protected]>"

# TODO: use g1gc or zgc as default
ENV JAVA_OPTS="-XX:+UnlockExperimentalVMOptions -XX:+UseContainerSupport -XX:MaxRAMPercentage=50 -XshowSettings:vm"
ENV JAVA_OPTS="-XX:+UnlockExperimentalVMOptions -XX:+UseContainerSupport -XX:MaxRAMPercentage=50 -XshowSettings:vm" \
HUGEGRAPH_HOME="hugegraph"

#COPY . /hugegraph/hugegraph-server
WORKDIR /hugegraph/
Expand All @@ -50,11 +51,16 @@ RUN set -x \
# 2. Init HugeGraph Sever
RUN set -e \
&& pwd && cd /hugegraph/ \
&& sed -i "s/^restserver.url.*$/restserver.url=http:\/\/0.0.0.0:8080/g" ./conf/rest-server.properties \
&& ./bin/init-store.sh
&& sed -i "s/^restserver.url.*$/restserver.url=http:\/\/0.0.0.0:8080/g" ./conf/rest-server.properties

# 3. Init docker script
COPY hugegraph-dist/docker/scripts/remote-connect.groovy ./scripts
COPY hugegraph-dist/docker/scripts/detect-storage.groovy ./scripts
COPY hugegraph-dist/docker/docker-entrypoint.sh .
RUN chmod 755 ./docker-entrypoint.sh

EXPOSE 8080
VOLUME /hugegraph

ENTRYPOINT ["/usr/bin/dumb-init", "--"]
CMD ["./bin/start-hugegraph.sh", "-d false -j $JAVA_OPTS -g zgc"]
CMD ["./docker-entrypoint.sh"]
24 changes: 24 additions & 0 deletions hugegraph-dist/docker/docker-entrypoint.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
#!/bin/bash
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with this
# work for additional information regarding copyright ownership. The ASF
# licenses this file to You under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
#


./bin/wait-storage.sh

./bin/init-store.sh

./bin/start-hugegraph.sh -d false -j "$JAVA_OPTS" -g zgc
61 changes: 61 additions & 0 deletions hugegraph-dist/docker/example/docker-compose-cassandra.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

version: "3"

services:
graph:
image: hugegraph/hugegraph
container_name: cas-graph
ports:
- 18080:8080
environment:
hugegraph.backend: cassandra
hugegraph.serializer: cassandra
hugegraph.cassandra.host: cas-cassandra
hugegraph.cassandra.port: 9042
networks:
- ca-network
depends_on:
- cassandra
healthcheck:
test: ["CMD", "bin/gremlin-console.sh", "--" ,"-e", "scripts/remote-connect.groovy"]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we still need -e?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://tinkerpop.apache.org/docs/3.6.2-SNAPSHOT/reference/#execution-mode
-e is for the gremlin console to use the execution mode to execute scripts.

interval: 10s
timeout: 30s
retries: 3

cassandra:
image: cassandra:4
container_name: cas-cassandra
ports:
- 7000:7000
- 9042:9042
security_opt:
- seccomp:unconfined
networks:
- ca-network
healthcheck:
test: ["CMD", "cqlsh", "--execute", "describe keyspaces;"]
interval: 10s
timeout: 30s
retries: 5

networks:
ca-network:

volumes:
hugegraph-data:
31 changes: 31 additions & 0 deletions hugegraph-dist/docker/scripts/detect-storage.groovy
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/

import org.apache.hugegraph.HugeFactory
import org.apache.hugegraph.dist.RegisterUtil

// register all the backend to avoid changes if docker needs to support othre backend
RegisterUtil.registerPlugins()
RegisterUtil.registerRocksDB()
RegisterUtil.registerCassandra()
RegisterUtil.registerScyllaDB()
RegisterUtil.registerHBase()
RegisterUtil.registerMysql()
RegisterUtil.registerPalo()
RegisterUtil.registerPostgresql()

graph = HugeFactory.open('./conf/graphs/hugegraph.properties')
19 changes: 19 additions & 0 deletions hugegraph-dist/docker/scripts/remote-connect.groovy
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/

:remote connect tinkerpop.server conf/remote.yaml
:> hugegraph
24 changes: 24 additions & 0 deletions hugegraph-dist/src/assembly/static/bin/docker-entrypoint.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
#!/bin/bash
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with this
# work for additional information regarding copyright ownership. The ASF
# licenses this file to You under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
#


./bin/wait-storage.sh

./bin/init-store.sh

./bin/start-hugegraph.sh -d false -j "$JAVA_OPTS" -g zgc
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why the params here? could we get the value of $JAVA_OPTS?

Copy link
Contributor Author

@aroundabout aroundabout Sep 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

JAVA_OPTS is an env var which is set in dockerfile. And this shell is only used in dockerfile, hence I think this way is available.

10 changes: 3 additions & 7 deletions hugegraph-dist/src/assembly/static/bin/gremlin-console.sh
Original file line number Diff line number Diff line change
Expand Up @@ -84,14 +84,10 @@ PROFILING_ENABLED=false

# Process options
MAIN_CLASS=org.apache.tinkerpop.gremlin.console.Console
while getopts "elpv" opt; do
while getopts "lpv" opt; do
case "$opt" in
e) MAIN_CLASS=org.apache.tinkerpop.gremlin.groovy.jsr223.ScriptExecutor
# Stop processing gremlin-console.sh arguments as soon as the -e switch
# is seen; everything following -e becomes arguments to the
# ScriptExecutor main class. This maintains compatibility with
# older deployments.
break;;
# class ScriptExecutor has been Deprecated.
# reference https://tinkerpop.apache.org/javadocs/3.2.3/full/org/apache/tinkerpop/gremlin/groovy/jsr223/ScriptExecutor.html
l) eval GREMLIN_LOG_LEVEL=\$$OPTIND
OPTIND="$(( $OPTIND + 1 ))"
if [ "$GREMLIN_LOG_LEVEL" = "TRACE" -o \
Expand Down
55 changes: 55 additions & 0 deletions hugegraph-dist/src/assembly/static/bin/wait-storage.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
#!/bin/bash
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with this
# work for additional information regarding copyright ownership. The ASF
# licenses this file to You under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
#

function abs_path() {
SOURCE="${BASH_SOURCE[0]}"
while [[ -h "$SOURCE" ]]; do
DIR="$(cd -P "$(dirname "$SOURCE")" && pwd)"
SOURCE="$(readlink "$SOURCE")"
[[ $SOURCE != /* ]] && SOURCE="$DIR/$SOURCE"
done
cd -P "$(dirname "$SOURCE")" && pwd
}

BIN=$(abs_path)
TOP="$(cd "$BIN"/../ && pwd)"
GRAPH_PROP="$TOP/conf/graphs/hugegraph.properties"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

prefer GRAPH_CONF

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes have been applied on the var name

HUGE_STORAGE_TIMEOUT_S=120
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

prefer WAIT_STORAGE_TIMEOUT_S

TRY_STORAGE="$TOP/scripts/detect-storage.groovy"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also update TRY_STORAGE


. "$BIN"/util.sh

# apply config from env
while IFS=' ' read -r envvar_key envvar_val; do
if [[ "${envvar_key}" =~ hugegraph\. ]] && [[ ! -z ${envvar_val} ]]; then
envvar_key=${envvar_key#"hugegraph."}
if grep -q -E "^\s*${envvar_key}\s*=\.*" ${GRAPH_PROP}; then
sed -ri "s#^(\s*${envvar_key}\s*=).*#\\1${envvar_val}#" ${GRAPH_PROP}
else
echo "${envvar_key}=${envvar_val}" >> ${GRAPH_PROP}
fi
else
continue
fi
done < <(env | sort -r | awk -F= '{ st = index($0, "="); print $1 " " substr($0, st+1) }')

# wait for storage
if ! [ -z "${HUGE_STORAGE_TIMEOUT_S:-}" ]; then
timeout "${HUGE_STORAGE_TIMEOUT_S}s" bash -c \
"until bin/gremlin-console.sh -- -e $TRY_STORAGE > /dev/null 2>&1; do echo \"waiting for storage...\"; sleep 5; done"
fi
Loading