forked from deephaven/deephaven-core
-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add scripts to fully automate debezium/perf testing. (deephaven#2109)
* Basic scripting support for debezium/perf runs. * Small tweaks. * More tweaks. * More scripts. * Added .gitignore. * Tweaks. * Fixed a typo. * Updated debezium/perf/README.md; improved output of run_experiment.sh.
- Loading branch information
1 parent
e8252bc
commit f4a67c6
Showing
17 changed files
with
505 additions
and
8 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -10,6 +10,7 @@ services: | |
| service: server | ||
| volumes: | ||
| - ../scripts:/scripts | ||
| - ./logs:/logs | ||
|
|
||
| web: | ||
| extends: | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| logs |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| #!/bin/bash | ||
|
|
||
| set -eu | ||
|
|
||
| exec ../../java-client/session-examples/build/install/java-client-session-examples/bin/execute-script --python ../scripts/demo.py |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| #!/bin/bash | ||
|
|
||
| set -eu | ||
|
|
||
| exec ../../java-client/session-examples/build/install/java-client-session-examples/bin/execute-script --python ../scripts/sample_dt.py |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| #!/bin/bash | ||
|
|
||
| set -eu | ||
|
|
||
| exec docker-compose run -T mzcli -f /scripts/demo.sql |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| #!/bin/bash | ||
|
|
||
| set -eu | ||
|
|
||
| if [ -z "$PERF_TAG" ]; then | ||
| echo "$0: PERF_TAG environment variable is not defined, aborting." 1>&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| DATA_TAG="mz_sample_dt" | ||
| OUT=logs/${PERF_TAG}/${DATA_TAG}.log | ||
|
|
||
| SCRIPT=$(cat <<'EOF' | ||
| while true; do | ||
| DATE_TAG=$(date -u '+%Y-%m-%d %H:%M:%S%z') | ||
| echo -n "$DATE_TAG|" | ||
| psql --csv -A -t -f /scripts/sample_dt.sql -U materialize -h materialized -p 6875 | ||
| sleep 1 | ||
| done | ||
| EOF | ||
| ) | ||
|
|
||
| (nohup docker-compose run -T --entrypoint /bin/bash mzcli -c "$SCRIPT" < /dev/null >& $OUT &) | ||
|
|
||
| exit 0 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,55 @@ | ||
| import argparse | ||
| import datetime as dt | ||
| import os | ||
| import re | ||
| import subprocess | ||
| import sys | ||
|
|
||
| parser = argparse.ArgumentParser(description='Match process command line regex to pid') | ||
| parser.add_argument( | ||
| 'proc_specs_strs', | ||
| metavar='PROCSPEC', | ||
| type=str, nargs='+', | ||
| help='a string of the form "name:regex" where regex should only match one process in `ps -o command` output') | ||
|
|
||
| args = parser.parse_args() | ||
|
|
||
| proc_specs = {} | ||
| for proc_spec_str in args.proc_specs_strs: | ||
| name, regex_str = proc_spec_str.split(':', maxsplit=1) | ||
| proc_specs[name] = re.compile(regex_str) | ||
|
|
||
| ps_lines = subprocess.run( | ||
| ['ps', '-ahxww', '-o', 'pid,command' ], | ||
| stdout=subprocess.PIPE).stdout.decode('utf-8').splitlines() | ||
|
|
||
| matches = {} | ||
| nmatches = 0 | ||
| my_pid = f'{os.getpid()}' | ||
|
|
||
| for ps_line in ps_lines: | ||
| pid, cmd = ps_line.split(maxsplit=1) | ||
| if pid == my_pid: | ||
| continue | ||
| for name, regex in proc_specs.items(): | ||
| if re.search(regex, cmd) is not None: | ||
| prev = matches.get(name, None) | ||
| if prev is not None: | ||
| print(f"{sys.argv[0]}: found more than one match for '{name}': {prev}, {pid}, aborting", | ||
| file=sys.stderr) | ||
| sys.exit(1) | ||
| matches[name] = pid | ||
|
|
||
| for name in proc_specs.keys(): | ||
| if matches.get(name, None) is None: | ||
| print(f"{sys.argv[0]}: couldn't find a match for {name}, aborting", file=sys.stderr) | ||
| sys.exit(1) | ||
|
|
||
| first = True | ||
| for name, pid in matches.items(): | ||
| s = f'{name}:{pid}' | ||
| if not first: | ||
| s = ' ' + s | ||
| print(s, end='') | ||
| first = False | ||
| print() |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,66 @@ | ||
| #!/bin/sh | ||
|
|
||
| set -eu | ||
|
|
||
| if [ $# -ne 5 -o \( "$1" != 'dh' -a "$1" != 'mz' \) ]; then | ||
| echo "Usage: $0 dh|mz per_second_rate wait_seconds top_samples top_delay_seconds" 1>&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| engine="$1" | ||
| rate_per_s="$2" | ||
| wait_s="$3" | ||
| top_samples="$4" | ||
| top_delay="$5" | ||
|
|
||
| echo "About to run an experiment for ${engine} with ${rate_per_s} pageviews/s." | ||
| echo | ||
| echo "Actions that will be performed in this run:" | ||
| echo "1. Start compose services required for for ${engine}." | ||
| echo "2. Execute demo in ${engine} and setup update delay logging." | ||
| echo "3. Set ${rate_per_s} pageviews per second rate." | ||
| echo "4. Wait ${wait_s} seconds." | ||
| echo "5. Take ${top_samples} samples for mem and CPU utilization, ${top_delay} seconds between samples." | ||
| echo "6. Stop and 'reset' (down) compose." | ||
| echo | ||
| echo "Running experiment." | ||
| echo | ||
| echo "1. Starting compose." | ||
| export PERF_TAG=$(./start_perf_run.sh "$engine" "$rate_per_s") | ||
| echo "PERF_TAG=${PERF_TAG}" | ||
| echo | ||
| echo "Logs are being saved to logs/$PERF_TAG." | ||
| echo | ||
|
|
||
| echo "2. Running demo in ${engine} and sampling delays." | ||
| if [ "$engine" = "mz" ]; then | ||
| ./mz_run_demo.sh | ||
| ./mz_sample_dt.sh | ||
| elif [ "$engine" = "dh" ]; then | ||
| ./dh_run_demo.sh | ||
| ./dh_sample_dt.sh | ||
| else | ||
| echo "$0: Internal error, aborting." 1>&2 | ||
| exit 1 | ||
| fi | ||
| echo | ||
|
|
||
| echo "3. Setting pageviews per second" | ||
| ./set_pageviews_per_second.sh $rate_per_s | ||
| echo | ||
|
|
||
| echo "4. Waiting for $wait_s seconds." | ||
| sleep "$wait_s" | ||
| echo | ||
|
|
||
| echo "5. Sampling top." | ||
| ./sample_top.sh "$engine" "$top_samples" "$top_delay" | ||
| echo | ||
|
|
||
| echo "6. Stopping and 'reset' (down) compose." | ||
|
|
||
| ./stop_all.sh | ||
| echo | ||
| echo "Experiment finished." | ||
|
|
||
| exit 0 |
Oops, something went wrong.