-
Notifications
You must be signed in to change notification settings - Fork 53
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: add a CLI tool to validate generation configuration #2691
Changes from 3 commits
26d4888
58c05eb
8130ead
a3652b3
8b0b8e1
1674469
8aac838
f2dc50f
1cd4b95
8dca4ed
bd53ac4
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -46,6 +46,7 @@ def __init__( | |
self.libraries = libraries | ||
self.grpc_version = grpc_version | ||
self.protoc_version = protoc_version | ||
self.__validate() | ||
|
||
def get_proto_path_to_library_name(self) -> dict[str, str]: | ||
""" | ||
|
@@ -62,6 +63,17 @@ def get_proto_path_to_library_name(self) -> dict[str, str]: | |
def is_monorepo(self) -> bool: | ||
return len(self.libraries) > 1 | ||
|
||
def __validate(self) -> None: | ||
seen_library_names = dict() | ||
for library in self.libraries: | ||
library_name = library.get_library_name() | ||
if library_name in seen_library_names: | ||
raise ValueError( | ||
f"{library.name_pretty} has the same library name with " | ||
f"{seen_library_names.get(library_name)}" | ||
) | ||
seen_library_names[library_name] = library.name_pretty | ||
|
||
|
||
def from_yaml(path_to_yaml: str) -> GenerationConfig: | ||
""" | ||
|
@@ -130,7 +142,9 @@ def from_yaml(path_to_yaml: str) -> GenerationConfig: | |
|
||
def __required(config: Dict, key: str): | ||
if key not in config: | ||
raise ValueError(f"required key {key} not found in yaml") | ||
raise ValueError( | ||
f"required key {key} not found in {config} " f"when parsing yaml" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is {config} going to print out the whole config? If yes, I don't think we want to do that. Something like
should be good enough. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Separately, there are a few library level required field, and I don't see any validation for them yet. They can be addressed in a separate PR. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I remove
Library level required fields are tests in There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I see that library level config is validated in the same way as repo level config now. However, it makes the error message not clear for library level config. e.g. If |
||
) | ||
return config[key] | ||
|
||
|
||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,51 @@ | ||
gapic_generator_version: 2.34.0 | ||
protoc_version: 25.2 | ||
googleapis_commitish: 1a45bf7393b52407188c82e63101db7dc9c72026 | ||
libraries_bom_version: 26.37.0 | ||
owlbot_cli_image: sha256:623647ee79ac605858d09e60c1382a716c125fb776f69301b72de1cd35d49409 | ||
synthtool_commitish: 6612ab8f3afcd5e292aecd647f0fa68812c9f5b5 | ||
template_excludes: | ||
- ".github/*" | ||
- ".kokoro/*" | ||
- "samples/*" | ||
- "CODE_OF_CONDUCT.md" | ||
- "CONTRIBUTING.md" | ||
- "LICENSE" | ||
- "SECURITY.md" | ||
- "java.header" | ||
- "license-checks.xml" | ||
- "renovate.json" | ||
- ".gitignore" | ||
libraries: | ||
- api_shortname: cloudasset | ||
name_pretty: Cloud Asset | ||
product_documentation: "https://cloud.google.com/resource-manager/docs/cloud-asset-inventory/overview" | ||
api_description: "provides inventory services based on a time series database." | ||
library_name: "asset" | ||
client_documentation: "https://cloud.google.com/java/docs/reference/google-cloud-asset/latest/overview" | ||
distribution_name: "com.google.cloud:google-cloud-asset" | ||
release_level: "stable" | ||
issue_tracker: "https://issuetracker.google.com/issues/new?component=187210&template=0" | ||
api_reference: "https://cloud.google.com/resource-manager/docs/cloud-asset-inventory/overview" | ||
codeowner_team: "@googleapis/analytics-dpe" | ||
excluded_poms: proto-google-iam-v1-bom,google-iam-policy,proto-google-iam-v1 | ||
excluded_dependencies: google-iam-policy | ||
GAPICs: | ||
- proto_path: google/cloud/asset/v1 | ||
- proto_path: google/cloud/asset/v1p1beta1 | ||
- proto_path: google/cloud/asset/v1p2beta1 | ||
- proto_path: google/cloud/asset/v1p5beta1 | ||
- proto_path: google/cloud/asset/v1p7beta1 | ||
|
||
- api_shortname: another-cloudasset | ||
name_pretty: Cloud Asset Inventory | ||
product_documentation: "https://cloud.google.com/resource-manager/docs/cloud-asset-inventory/overview" | ||
api_description: "provides inventory services based on a time series database." | ||
library_name: "asset" | ||
client_documentation: "https://cloud.google.com/java/docs/reference/google-cloud-asset/latest/overview" | ||
distribution_name: "com.google.cloud:google-cloud-asset" | ||
release_level: "stable" | ||
issue_tracker: "https://issuetracker.google.com/issues/new?component=187210&template=0" | ||
api_reference: "https://cloud.google.com/resource-manager/docs/cloud-asset-inventory/overview" | ||
GAPICs: | ||
- proto_path: google/cloud/asset/v1 |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -130,10 +130,6 @@ def prepare_repo( | |
# use absolute path because docker requires absolute path | ||
# in volume name. | ||
absolute_library_path = str(Path(library_path).resolve()) | ||
if absolute_library_path in libraries: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We don't need this check because the Generation object is validated when reaches this point. |
||
# check whether the java_library is unique among all libraries | ||
# because two libraries should not go to the same destination. | ||
raise ValueError(f"{absolute_library_path} already exists.") | ||
libraries[absolute_library_path] = library | ||
# remove existing .repo-metadata.json | ||
json_name = ".repo-metadata.json" | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we make the error message more verbose and actionable? Something like
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Another edge case is that two libraries may have the same name_pretty, but I think the error message is already good enough, so we may not have to consider it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.