Skip to content

Commit

Permalink
Add utility to validate the run-file
Browse files Browse the repository at this point in the history
And update doc
  • Loading branch information
rafaelfolco committed Oct 4, 2023
1 parent 75467b8 commit 4c0e256
Show file tree
Hide file tree
Showing 3 changed files with 84 additions and 13 deletions.
37 changes: 25 additions & 12 deletions util/README.md
Original file line number Diff line number Diff line change
@@ -1,29 +1,42 @@

### JSON Run File
# Crucible run-file

The "all-in-one" JSON run file enables test configuration by using a single
JSON file. Users can specify multi-value parameters, tools, tags, endpoint,
and passthru arguments in JSON format, everything in the same file.
The Crucible run-file, a.k.a. "all-in-one" JSON, is the configuration file
used for running benchmarks. It consolidates all kinds of the settings from
the test run without the need of creating separated files for different
aspects of the test.

When a run-file JSON is used by Crucible command line, it enables test
configuration by using a single JSON file. Users can specify multi-value
parameters, tools, tags, endpoints, and run parameters, all using an unified
JSON format, in one single place.

For more details on the supported format, refer to the JSON [schema](JSON/schema.json).

To validate a run-file, run:
```
python3 validate.py --json <run-file-json>
```

To use a run-file JSON in Crucible, run:
```
crucible run --from-file <run-file-json>
```

## blockbreaker.py utility

The "blockbreaker.py" utility extracts a configuration block from the
"all-in-one" JSON run file and transforms into a single JSON block or stream.
run-file JSON and transforms into a JSON block or stream output.

## Usage
Usage:
```
# python3 blockbreaker.py --json <json-run-file> --config <config>
python3 blockbreaker.py --json <run-file-json> --config <config>
```
Example:
```
# python3 blockbreaker.py --json run-file.json --config mv-params
python3 blockbreaker.py --json run-file.json --config mv-params
```

For more options and usage run:
```
# python3 blockbreaker.py --help
python3 blockbreaker.py --help
```


2 changes: 1 addition & 1 deletion util/blockbreaker.py
Original file line number Diff line number Diff line change
Expand Up @@ -245,7 +245,7 @@ def get_mv_params(input_json, name):
return benchmark_blk

def main():
"""Main function of get-json-config.py tool"""
"""Main function of blockbreaker.py tool"""

global args

Expand Down
58 changes: 58 additions & 0 deletions util/validate.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
#!/usr/bin/python3

'''Utility to validate crucible run file JSON'''

from blockbreaker import load_json_file,validate_schema
import argparse

def process_options():
"""Handle the CLI argument parsing options"""

parser = argparse.ArgumentParser(description = "Get a config block from crucible run file")

parser.add_argument("--json",
dest = "json_file",
help = "Crucible run-file JSON",
required = True,
type = str)

args = parser.parse_args()
return args

def main():
"""Main function of the validate.py utility"""

global args
err_msg=None

input_json = load_json_file(args.json_file)
if input_json is None:
err_msg=f"{ err }: Failed to load run-file JSON { args.json_file }"
rc=1

if not validate_schema(input_json):
err_msg=(
f"{ err }: Failed to validate run-file JSON "
f"{ args.json_file } against schema."
)
rc=2

for json_blk in input_json["endpoints"]:
endpoint_type = json_blk["type"]
if not validate_schema(json_blk, "schema-" + endpoint_type + ".json"):
err_msg=(
f"{ err }: Failed to validate the 'endpoints' block from "
f" the JSON run-file { args.json_file } against the "
f"{ endpoint_type }'s schema."
)
rc=3

if err_msg is not None:
print(f"ERROR: { err_msg }")
return rc

print(f"[ OK ] run-file JSON { args.json_file } validated!")

if __name__ == "__main__":
args = process_options()
exit(main())

0 comments on commit 4c0e256

Please sign in to comment.