Comparing Benchmarks

Use pbench cmp to compare the files of two directories and generate unified diffs for matching files.

Note: pbench cmp is an experimental feature that is not included in the default builds in Releases.

Run ./pbench cmp --help to see the online help for pbench cmp.

Compare two query result directories

Usage:
  pbench cmp [flags] [directory 1] [directory 2]

Flags:
  -r, --file-id-regex string regex to extract file id from file names in two directories to find matching files to compare (default ".*(query_\d{2})(?:_c0)?\.output")
  -h, --help help for cmp
  -o, --output-path string diff output path (default "./diff")

-r, --file-id-regex string

When specifying a custom regex with -r, you must include one capture group that is written to match the files in the directories that you are comparing.

For more information, visit Using pbench cmp.

-h, --help

Display the help for pbench cmp.

-o, --output-path string

The output directory to write to, relative to the current directory that pbench cmp is run from.

If -o is not used, the default output directory is ./diff.

Using pbench cmp

Use pbench cmp to compare the files of two directories and generate unified diffs for matching files, based on a regex expression and a capture group.

• The regex expression is used to filter and extract relevant parts of the filenames.

• A capture group is a part of a regular expression that is enclosed in parentheses, and is used in pbench cmp to match files.

Default Regex and Capture Group

The default regex expression is .*(query_\d{2})(?:_c0)?\.output.

The default capture group is  (query_\d{2}).

(?:_c0) is a non-capturing group, and is not used to match files. The ? makes this group optional, so it matches zero or one occurrence of _c0.

Example

The process follows these steps:

1. In directory1, if the file name matches the regex, the capture group is extracted and stored.
2. In directory2, if the file name matches the regex, the capture group is extracted and is used to search for a match in directory1.
3. If there is a match, the two files are compared with a diff.  

For example, ds_power_query_01_c0.output matches the default regex expression.  

query_01 is extracted, and is used to look in the other directory to match files who also have query_01 in the capture group and match the regex expression.