Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ci] add generator for templated flag testdata #13150

Merged
merged 3 commits into from
Jun 5, 2023
Merged

[ci] add generator for templated flag testdata #13150

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
f6a49cf
add generator for templated flag testdata
May 25, 2023
65c4b25
Apply suggestions from code review
ajm188 May 25, 2023
4d5998b
while loop + variable directory
May 25, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ export REWRITER=go/vt/sqlparser/rewriter.go
# Since we are not using this Makefile for compilation, limiting parallelism will not increase build time.
.NOTPARALLEL:

.PHONY: all build install test clean unit_test unit_test_cover unit_test_race integration_test proto proto_banner site_test site_integration_test docker_bootstrap docker_test docker_unit_test java_test reshard_tests e2e_test e2e_test_race minimaltools tools generate_ci_workflows
.PHONY: all build install test clean unit_test unit_test_cover unit_test_race integration_test proto proto_banner site_test site_integration_test docker_bootstrap docker_test docker_unit_test java_test reshard_tests e2e_test e2e_test_race minimaltools tools generate_ci_workflows generate-flag-testdata

all: build

Expand Down Expand Up @@ -457,6 +457,9 @@ vtadmin_authz_testgen:
generate_ci_workflows:
cd test && go run ci_workflow_gen.go && cd ..

generate-flag-testdata:
./tools/generate_flag_testdata.sh

release-notes:
go run ./go/tools/release-notes --from "$(FROM)" --to "$(TO)" --version "$(VERSION)" --summary "$(SUMMARY)"

Expand Down
2 changes: 1 addition & 1 deletion go/flags/endtoend/mysqlctl.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ Global flags:
--config-file string Full path of the config file (with extension) to use. If set, --config-path, --config-type, and --config-name are ignored.
--config-file-not-found-handling ConfigFileNotFoundHandling Behavior when a config file is not found. (Options: error, exit, ignore, warn) (default warn)
--config-name string Name of the config file (without extension) to search for. (default "vtconfig")
--config-path strings Paths to search for config files in. (default [{{.Workdir}}])
--config-path strings Paths to search for config files in. (default [{{ .Workdir }}])
--config-persistence-min-interval duration minimum interval between persisting dynamic config changes back to disk (if no change has occurred, nothing is done). (default 1s)
--config-type string Config file type (omit to infer config type from file extension).
--db-credentials-file string db credentials file; send SIGHUP to reload this file
Expand Down
2 changes: 1 addition & 1 deletion go/flags/endtoend/mysqlctld.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Usage of mysqlctld:
--config-file string Full path of the config file (with extension) to use. If set, --config-path, --config-type, and --config-name are ignored.
--config-file-not-found-handling ConfigFileNotFoundHandling Behavior when a config file is not found. (Options: error, exit, ignore, warn) (default warn)
--config-name string Name of the config file (without extension) to search for. (default "vtconfig")
--config-path strings Paths to search for config files in. (default [{{.Workdir}}])
--config-path strings Paths to search for config files in. (default [{{ .Workdir }}])
--config-persistence-min-interval duration minimum interval between persisting dynamic config changes back to disk (if no change has occurred, nothing is done). (default 1s)
--config-type string Config file type (omit to infer config type from file extension).
--db-credentials-file string db credentials file; send SIGHUP to reload this file
Expand Down
2 changes: 1 addition & 1 deletion go/flags/endtoend/vtaclcheck.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Usage of vtaclcheck:
--config-file string Full path of the config file (with extension) to use. If set, --config-path, --config-type, and --config-name are ignored.
--config-file-not-found-handling ConfigFileNotFoundHandling Behavior when a config file is not found. (Options: error, exit, ignore, warn) (default warn)
--config-name string Name of the config file (without extension) to search for. (default "vtconfig")
--config-path strings Paths to search for config files in. (default [{{.Workdir}}])
--config-path strings Paths to search for config files in. (default [{{ .Workdir }}])
--config-persistence-min-interval duration minimum interval between persisting dynamic config changes back to disk (if no change has occurred, nothing is done). (default 1s)
--config-type string Config file type (omit to infer config type from file extension).
-h, --help display usage and exit
Expand Down
2 changes: 1 addition & 1 deletion go/flags/endtoend/vtbackup.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Usage of vtbackup:
--config-file string Full path of the config file (with extension) to use. If set, --config-path, --config-type, and --config-name are ignored.
--config-file-not-found-handling ConfigFileNotFoundHandling Behavior when a config file is not found. (Options: error, exit, ignore, warn) (default warn)
--config-name string Name of the config file (without extension) to search for. (default "vtconfig")
--config-path strings Paths to search for config files in. (default [{{.Workdir}}])
--config-path strings Paths to search for config files in. (default [{{ .Workdir }}])
--config-persistence-min-interval duration minimum interval between persisting dynamic config changes back to disk (if no change has occurred, nothing is done). (default 1s)
--config-type string Config file type (omit to infer config type from file extension).
--consul_auth_static_file string JSON File to read the topos/tokens from.
Expand Down
2 changes: 1 addition & 1 deletion go/flags/endtoend/vtctlclient.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Usage of vtctlclient:
--config-file string Full path of the config file (with extension) to use. If set, --config-path, --config-type, and --config-name are ignored.
--config-file-not-found-handling ConfigFileNotFoundHandling Behavior when a config file is not found. (Options: error, exit, ignore, warn) (default warn)
--config-name string Name of the config file (without extension) to search for. (default "vtconfig")
--config-path strings Paths to search for config files in. (default [{{.Workdir}}])
--config-path strings Paths to search for config files in. (default [{{ .Workdir }}])
--config-persistence-min-interval duration minimum interval between persisting dynamic config changes back to disk (if no change has occurred, nothing is done). (default 1s)
--config-type string Config file type (omit to infer config type from file extension).
--datadog-agent-host string host to send spans to. if empty, no tracing will be done
Expand Down
2 changes: 1 addition & 1 deletion go/flags/endtoend/vtctld.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ Usage of vtctld:
--config-file string Full path of the config file (with extension) to use. If set, --config-path, --config-type, and --config-name are ignored.
--config-file-not-found-handling ConfigFileNotFoundHandling Behavior when a config file is not found. (Options: error, exit, ignore, warn) (default warn)
--config-name string Name of the config file (without extension) to search for. (default "vtconfig")
--config-path strings Paths to search for config files in. (default [{{.Workdir}}])
--config-path strings Paths to search for config files in. (default [{{ .Workdir }}])
--config-persistence-min-interval duration minimum interval between persisting dynamic config changes back to disk (if no change has occurred, nothing is done). (default 1s)
--config-type string Config file type (omit to infer config type from file extension).
--consul_auth_static_file string JSON File to read the topos/tokens from.
Expand Down
2 changes: 1 addition & 1 deletion go/flags/endtoend/vtexplain.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Usage of vtexplain:
--config-file string Full path of the config file (with extension) to use. If set, --config-path, --config-type, and --config-name are ignored.
--config-file-not-found-handling ConfigFileNotFoundHandling Behavior when a config file is not found. (Options: error, exit, ignore, warn) (default warn)
--config-name string Name of the config file (without extension) to search for. (default "vtconfig")
--config-path strings Paths to search for config files in. (default [{{.Workdir}}])
--config-path strings Paths to search for config files in. (default [{{ .Workdir }}])
--config-persistence-min-interval duration minimum interval between persisting dynamic config changes back to disk (if no change has occurred, nothing is done). (default 1s)
--config-type string Config file type (omit to infer config type from file extension).
--dbname string Optional database target to override normal routing
Expand Down
2 changes: 1 addition & 1 deletion go/flags/endtoend/vtgate.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Usage of vtgate:
--config-file string Full path of the config file (with extension) to use. If set, --config-path, --config-type, and --config-name are ignored.
--config-file-not-found-handling ConfigFileNotFoundHandling Behavior when a config file is not found. (Options: error, exit, ignore, warn) (default warn)
--config-name string Name of the config file (without extension) to search for. (default "vtconfig")
--config-path strings Paths to search for config files in. (default [{{.Workdir}}])
--config-path strings Paths to search for config files in. (default [{{ .Workdir }}])
--config-persistence-min-interval duration minimum interval between persisting dynamic config changes back to disk (if no change has occurred, nothing is done). (default 1s)
--config-type string Config file type (omit to infer config type from file extension).
--consul_auth_static_file string JSON File to read the topos/tokens from.
Expand Down
2 changes: 1 addition & 1 deletion go/flags/endtoend/vtgr.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ Usage of vtgr:
--config-file string Full path of the config file (with extension) to use. If set, --config-path, --config-type, and --config-name are ignored.
--config-file-not-found-handling ConfigFileNotFoundHandling Behavior when a config file is not found. (Options: error, exit, ignore, warn) (default warn)
--config-name string Name of the config file (without extension) to search for. (default "vtconfig")
--config-path strings Paths to search for config files in. (default [{{.Workdir}}])
--config-path strings Paths to search for config files in. (default [{{ .Workdir }}])
--config-persistence-min-interval duration minimum interval between persisting dynamic config changes back to disk (if no change has occurred, nothing is done). (default 1s)
--config-type string Config file type (omit to infer config type from file extension).
--consul_auth_static_file string JSON File to read the topos/tokens from.
Expand Down
2 changes: 1 addition & 1 deletion go/flags/endtoend/vtorc.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Usage of vtorc:
--config-file string Full path of the config file (with extension) to use. If set, --config-path, --config-type, and --config-name are ignored.
--config-file-not-found-handling ConfigFileNotFoundHandling Behavior when a config file is not found. (Options: error, exit, ignore, warn) (default warn)
--config-name string Name of the config file (without extension) to search for. (default "vtconfig")
--config-path strings Paths to search for config files in. (default [{{.Workdir}}])
--config-path strings Paths to search for config files in. (default [{{ .Workdir }}])
--config-persistence-min-interval duration minimum interval between persisting dynamic config changes back to disk (if no change has occurred, nothing is done). (default 1s)
--config-type string Config file type (omit to infer config type from file extension).
--consul_auth_static_file string JSON File to read the topos/tokens from.
Expand Down
2 changes: 1 addition & 1 deletion go/flags/endtoend/vttablet.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ Usage of vttablet:
--config-file string Full path of the config file (with extension) to use. If set, --config-path, --config-type, and --config-name are ignored.
--config-file-not-found-handling ConfigFileNotFoundHandling Behavior when a config file is not found. (Options: error, exit, ignore, warn) (default warn)
--config-name string Name of the config file (without extension) to search for. (default "vtconfig")
--config-path strings Paths to search for config files in. (default [{{.Workdir}}])
--config-path strings Paths to search for config files in. (default [{{ .Workdir }}])
--config-persistence-min-interval duration minimum interval between persisting dynamic config changes back to disk (if no change has occurred, nothing is done). (default 1s)
--config-type string Config file type (omit to infer config type from file extension).
--consolidator-stream-query-size int Configure the stream consolidator query size in bytes. Setting to 0 disables the stream consolidator. (default 2097152)
Expand Down
2 changes: 1 addition & 1 deletion go/flags/endtoend/vttestserver.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ Usage of vttestserver:
--config-file string Full path of the config file (with extension) to use. If set, --config-path, --config-type, and --config-name are ignored.
--config-file-not-found-handling ConfigFileNotFoundHandling Behavior when a config file is not found. (Options: error, exit, ignore, warn) (default warn)
--config-name string Name of the config file (without extension) to search for. (default "vtconfig")
--config-path strings Paths to search for config files in. (default [{{.Workdir}}])
--config-path strings Paths to search for config files in. (default [{{ .Workdir }}])
--config-persistence-min-interval duration minimum interval between persisting dynamic config changes back to disk (if no change has occurred, nothing is done). (default 1s)
--config-type string Config file type (omit to infer config type from file extension).
--consul_auth_static_file string JSON File to read the topos/tokens from.
Expand Down
2 changes: 1 addition & 1 deletion go/flags/endtoend/zkctl.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ Usage of zkctl:
--config-file string Full path of the config file (with extension) to use. If set, --config-path, --config-type, and --config-name are ignored.
--config-file-not-found-handling ConfigFileNotFoundHandling Behavior when a config file is not found. (Options: error, exit, ignore, warn) (default warn)
--config-name string Name of the config file (without extension) to search for. (default "vtconfig")
--config-path strings Paths to search for config files in. (default [{{.Workdir}}])
--config-path strings Paths to search for config files in. (default [{{ .Workdir }}])
--config-persistence-min-interval duration minimum interval between persisting dynamic config changes back to disk (if no change has occurred, nothing is done). (default 1s)
--config-type string Config file type (omit to infer config type from file extension).
-h, --help display usage and exit
Expand Down
2 changes: 1 addition & 1 deletion go/flags/endtoend/zkctld.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ Usage of zkctld:
--config-file string Full path of the config file (with extension) to use. If set, --config-path, --config-type, and --config-name are ignored.
--config-file-not-found-handling ConfigFileNotFoundHandling Behavior when a config file is not found. (Options: error, exit, ignore, warn) (default warn)
--config-name string Name of the config file (without extension) to search for. (default "vtconfig")
--config-path strings Paths to search for config files in. (default [{{.Workdir}}])
--config-path strings Paths to search for config files in. (default [{{ .Workdir }}])
--config-persistence-min-interval duration minimum interval between persisting dynamic config changes back to disk (if no change has occurred, nothing is done). (default 1s)
--config-type string Config file type (omit to infer config type from file extension).
-h, --help display usage and exit
Expand Down
17 changes: 17 additions & 0 deletions tools/generate_flag_testdata.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
#!/bin/bash

ajm188 marked this conversation as resolved.
Show resolved Hide resolved
templatize_help_text() {
$1 --help 2>&1 | \
sed 's/{{/{{ "{{/' | \
sed 's/}}/}}" }}/' | \
sed 's/Paths to search for config files in. (default .*)/Paths to search for config files in. (default [{{ .Workdir }}])/'
ajm188 marked this conversation as resolved.
Show resolved Hide resolved
return 0
}

export -f templatize_help_text

find go/flags/endtoend -iname '*.txt' | \
xargs basename | \
cut -d. -f1 | \
xargs -I{} \
bash -c 'templatize_help_text "{}" >go/flags/endtoend/{}.txt'
Copy link
Contributor

@mattlord mattlord May 25, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, this is better IMO:

#!/bin/bash

set -euo pipefail

DIR="go/flags/endtoend"

templatize_help_text() {
    $1 --help 2>&1 | \
        sed -e 's/{{/{{ "{{/' \
        -e 's/}}/}}" }}/' \
        -e 's/Paths to search for config files in. \(default .*\)/Paths to search for config files in. (default [{{ .Workdir }}])/'
    return 0
}

while read -r line; do
	file="${line##*/}"
	binary="${file%.*}"
	templatize_help_text "${binary}" > "${line}"
done < <(find "${DIR}" -iname '*.txt')

exit 0