adding location and doc schema #8371

purvii-n · 2023-07-18T07:41:48Z

Adding a new service, Document AI Warehouse to Terraform. Added 2 resources, DocumentSchema and Location.
Location is not passing the acceptance tests, please provide some assistance

If this PR is for Terraform, I acknowledge that I have:

Searched through the issue tracker for an open issue that this either resolves or contributes to, commented on it to claim it, and written "fixes {url}" or "part of {url}" in this PR description. If there were no relevant open issues, I opened one and commented that I would like to work on it (not necessary for very small changes).
Ensured that all new fields I added that can be set by a user appear in at least one example (for generated resources) or third_party test (for handwritten resources or update tests).
Generated Terraform providers, and ran make test and make lint in the generated providers to ensure it passes unit and linter tests.
Ran relevant acceptance tests using my own Google Cloud project and credentials (If the acceptance tests do not yet pass or you are unable to run them, please let your reviewer know).
Read Write release notes before writing my release note below.

Release Note Template for Downstream PRs (will be copied)

`google_document_ai_warehouse_document_schema`

`google_document_ai_warehouse_location`

modular-magician · 2023-07-18T07:42:05Z

Oops! It looks like you're using an unknown release-note type in your changelog entries:

Adding a new service (Document AI Warehouse) with 2 resources (DocumentSchema, Location)

Please only use the types listed in https://github.com/GoogleCloudPlatform/magic-modules/blob/master/.ci/RELEASE_NOTES_GUIDE.md.

modular-magician · 2023-07-18T07:42:14Z

Hello! I am a robot who works on Magic Modules PRs.

I've detected that you're a community contributor. @zli82016, a repository maintainer, has been assigned to assist you and help review your changes.

❓ First time contributing? Click here for more details

Your assigned reviewer will help review your code by:

Ensuring it's backwards compatible, covers common error cases, etc.
Summarizing the change into a user-facing changelog note.
Passes tests, either our "VCR" suite, a set of presubmit tests, or with manual test runs.

You can help make sure that review is quick by running local tests and ensuring they're passing in between each push you make to your PR's branch. Also, try to leave a comment with each push you make, as pushes generally don't generate emails.

If your reviewer doesn't get back to you within a week after your most recent change, please feel free to leave a comment on the issue asking them to take a look! In the absence of a dedicated review dashboard most maintainers manage their pending reviews through email, and those will sometimes get lost in their inbox.

modular-magician · 2023-07-18T07:58:12Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 15 files changed, 2531 insertions(+), 2 deletions(-))
Terraform Beta: Diff ( 15 files changed, 2531 insertions(+), 2 deletions(-))
TF Conversion: Diff ( 6 files changed, 736 insertions(+), 3 deletions(-))
TF OiCS: Diff ( 8 files changed, 214 insertions(+))

Missing test report

Your PR includes resource fields which are not covered by any test.

Resource: google_document_ai_warehouse_document_schema (1 total tests)
Untested fields: document_is_folder, property_definitions.is_required, property_definitions.schema_sources.processor_type, property_definitions.schema_sources.name, property_definitions.timestamp_type_options.dummy, property_definitions.map_type_options.dummy, property_definitions.float_type_options.dummy, property_definitions.enum_type_options.possible_values, property_definitions.enum_type_options.validation_check_disabled, property_definitions.retrieval_importance, property_definitions.integer_type_options.dummy, property_definitions.date_time_type_options.dummy, property_definitions.is_repeatable, property_definitions.property_type_options.property_definitions.display_name, property_definitions.property_type_options.property_definitions.is_filterable, property_definitions.property_type_options.property_definitions.is_metadata, property_definitions.property_type_options.property_definitions.is_repeatable, property_definitions.property_type_options.property_definitions.is_required, property_definitions.property_type_options.property_definitions.is_searchable, property_definitions.property_type_options.property_definitions.name, property_definitions.property_type_options.property_definitions.text_type_options.dummy, property_definitions.text_type_options.dummy, property_definitions.is_metadata, property_definitions.is_searchable, property_definitions.is_filterable

Please add acceptance tests which include these fields.
Resource: google_document_ai_warehouse_location (1 total tests)
Untested fields: database_type, access_control_mode

Please add acceptance tests which include these fields.

modular-magician · 2023-07-18T09:28:18Z

Tests analytics

Total tests: 2865
Passed tests 2562
Skipped tests: 299
Affected tests: 4

Action taken

Found 4 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccDocumentAIWarehouseLocation_documentAiWarehouseLocationExample|TestAccDocumentAIWarehouseDocumentSchema_documentAiWarehouseDocumentSchemaExample|TestAccContainerAwsNodePool_BetaBasicHandWritten|TestAccComputeFirewallPolicyRule_multipleRules

Get to know how VCR tests work

modular-magician · 2023-07-18T09:31:06Z

$\textcolor{red}{\textsf{Tests failed during RECORDING mode:}}$
TestAccDocumentAIWarehouseLocation_documentAiWarehouseLocationExample[Error message] [Debug log]
TestAccDocumentAIWarehouseDocumentSchema_documentAiWarehouseDocumentSchemaExample[Error message] [Debug log]
TestAccContainerAwsNodePool_BetaBasicHandWritten[Error message] [Debug log]
TestAccComputeFirewallPolicyRule_multipleRules[Error message] [Debug log]

$\textcolor{red}{\textsf{Please fix these to complete your PR.}}$
View the build log or the debug log for each test

zli82016 · 2023-07-18T17:04:03Z

/gcbrun

zli82016 · 2023-07-18T17:07:04Z

Can you please add the copyright to the following files? Thanks.

Files missing (or outdated) copyright:
- /workspace/mmv1/products/documentaiwarehouse/DocumentSchema.yaml
- /workspace/mmv1/products/documentaiwarehouse/Location.yaml
- /workspace/mmv1/products/documentaiwarehouse/product.yaml

purvii-n · 2023-07-18T17:13:31Z

Hello, I have added the copyright to the mentioned files

modular-magician · 2023-07-18T17:27:31Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 15 files changed, 2531 insertions(+), 2 deletions(-))
Terraform Beta: Diff ( 15 files changed, 2531 insertions(+), 2 deletions(-))
TF Conversion: Diff ( 6 files changed, 736 insertions(+), 3 deletions(-))
TF OiCS: Diff ( 8 files changed, 214 insertions(+))

Missing test report

Your PR includes resource fields which are not covered by any test.

Resource: google_document_ai_warehouse_document_schema (1 total tests)
Untested fields: document_is_folder, property_definitions.is_metadata, property_definitions.is_searchable, property_definitions.float_type_options.dummy, property_definitions.is_repeatable, property_definitions.retrieval_importance, property_definitions.schema_sources.processor_type, property_definitions.schema_sources.name, property_definitions.integer_type_options.dummy, property_definitions.text_type_options.dummy, property_definitions.is_filterable, property_definitions.property_type_options.property_definitions.is_repeatable, property_definitions.property_type_options.property_definitions.is_required, property_definitions.property_type_options.property_definitions.is_searchable, property_definitions.property_type_options.property_definitions.name, property_definitions.property_type_options.property_definitions.text_type_options.dummy, property_definitions.property_type_options.property_definitions.display_name, property_definitions.property_type_options.property_definitions.is_filterable, property_definitions.property_type_options.property_definitions.is_metadata, property_definitions.map_type_options.dummy, property_definitions.date_time_type_options.dummy, property_definitions.is_required, property_definitions.enum_type_options.possible_values, property_definitions.enum_type_options.validation_check_disabled, property_definitions.timestamp_type_options.dummy

Please add acceptance tests which include these fields.
Resource: google_document_ai_warehouse_location (1 total tests)
Untested fields: access_control_mode, database_type

Please add acceptance tests which include these fields.

mmv1/products/documentaiwarehouse/Location.yaml

zli82016 · 2023-07-18T18:25:01Z

You mentioned that Location is not passing the acceptance tests. Do you see any errors?

purvii-n · 2023-07-18T18:32:54Z

You mentioned that Location is not passing the acceptance tests. Do you see any errors?

Yes.

test_working_directory=/var/folders/xj/cq2c4_317375yxbsz3xv8fn401054d/T/plugintest2904110445 test_name=TestAccDocumentAIWarehouseLocation_documentAiWarehouseLocationExample
    vcr_utils.go:152: Step 1/2 error: After applying this test step, the plan was not empty.
        stdout:
        
        
        Terraform used the selected providers to generate the following execution
        plan. Resource actions are indicated with the following symbols:
          ~ update in-place
        
        Terraform will perform the following actions:
        
          # google_document_ai_warehouse_location.example will be updated in-place
          ~ resource "google_document_ai_warehouse_location" "example" {
              + access_control_mode = "ACL_MODE_DOCUMENT_LEVEL_ACCESS_CONTROL_GCI"
              + database_type       = "DB_INFRA_SPANNER"
                id                  = "projects/557511864805/locations/us/operations/7984bd91-51f5-4317-a0ed-ffadc810285c"
              + location            = "us"
                name                = "projects/557511864805/locations/us/operations/7984bd91-51f5-4317-a0ed-ffadc810285c"
                # (1 unchanged attribute hidden)
            }
        
        Plan: 0 to add, 1 to change, 0 to destroy.

modular-magician · 2023-07-18T19:02:31Z

Tests analytics

Total tests: 2865
Passed tests 2562
Skipped tests: 299
Affected tests: 4

Action taken

Found 4 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccDocumentAIWarehouseLocation_documentAiWarehouseLocationExample|TestAccDocumentAIWarehouseDocumentSchema_documentAiWarehouseDocumentSchemaExample|TestAccContainerAwsNodePool_BetaBasicHandWritten|TestAccComputeFirewallPolicyRule_multipleRules

Get to know how VCR tests work

modular-magician · 2023-07-18T19:05:07Z

$\textcolor{red}{\textsf{Tests failed during RECORDING mode:}}$
TestAccDocumentAIWarehouseLocation_documentAiWarehouseLocationExample[Error message] [Debug log]
TestAccDocumentAIWarehouseDocumentSchema_documentAiWarehouseDocumentSchemaExample[Error message] [Debug log]
TestAccContainerAwsNodePool_BetaBasicHandWritten[Error message] [Debug log]
TestAccComputeFirewallPolicyRule_multipleRules[Error message] [Debug log]

$\textcolor{red}{\textsf{Please fix these to complete your PR.}}$
View the build log or the debug log for each test

zli82016 · 2023-07-18T19:38:53Z

Two tests failed with the different error message Invalid project number 'ci-test-project-188019'. Can you please try to fix it?

TestAccDocumentAIWarehouseLocation_documentAiWarehouseLocationExample
TestAccDocumentAIWarehouseDocumentSchema_documentAiWarehouseDocumentSchemaExample

mmv1/products/documentaiwarehouse/Location.yaml

mmv1/templates/terraform/examples/document_ai_warehouse_location.tf.erb

mmv1/products/documentaiwarehouse/Location.yaml

…sync, async in Location

modular-magician · 2023-07-19T17:07:53Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 16 files changed, 2672 insertions(+), 2 deletions(-))
Terraform Beta: Diff ( 16 files changed, 2672 insertions(+), 2 deletions(-))
TF Conversion: Diff ( 6 files changed, 746 insertions(+), 3 deletions(-))
TF OiCS: Diff ( 8 files changed, 220 insertions(+))

Missing test report

Your PR includes resource fields which are not covered by any test.

Resource: google_document_ai_warehouse_document_schema (1 total tests)
Untested fields: property_definitions.is_searchable, property_definitions.property_type_options.property_definitions.is_repeatable, property_definitions.property_type_options.property_definitions.is_required, property_definitions.property_type_options.property_definitions.is_searchable, property_definitions.property_type_options.property_definitions.name, property_definitions.property_type_options.property_definitions.text_type_options.dummy, property_definitions.property_type_options.property_definitions.display_name, property_definitions.property_type_options.property_definitions.is_filterable, property_definitions.property_type_options.property_definitions.is_metadata, property_definitions.retrieval_importance, property_definitions.schema_sources.processor_type, property_definitions.schema_sources.name, property_definitions.float_type_options.dummy, property_definitions.integer_type_options.dummy, property_definitions.is_required, property_definitions.map_type_options.dummy, property_definitions.is_filterable, property_definitions.date_time_type_options.dummy, property_definitions.text_type_options.dummy, property_definitions.timestamp_type_options.dummy, property_definitions.is_metadata, property_definitions.enum_type_options.validation_check_disabled, property_definitions.enum_type_options.possible_values, property_definitions.is_repeatable, document_is_folder

Please add acceptance tests which include these fields.
Resource: google_document_ai_warehouse_location (1 total tests)
Untested fields: document_creator_default_role, kms_key

Please add acceptance tests which include these fields.

mmv1/products/documentaiwarehouse/DocumentSchema.yaml

zli82016 · 2023-07-26T16:17:48Z

Thanks so much for making the changes. Can you please add a separate test file in the tests folder to test the [PATCH operation]? (https://cloud.google.com/document-warehouse/docs/reference/rest/v1/projects.locations.documentSchemas/patch)

Please check the documentation how to write the update tests. If you have any questions, please let me know. https://googlecloudplatform.github.io/magic-modules/develop/add-handwritten-test/#update-tests

purvii-n · 2023-07-27T14:33:47Z

Thanks so much for making the changes. Can you please add a separate test file in the tests folder to test the [PATCH operation]? (https://cloud.google.com/document-warehouse/docs/reference/rest/v1/projects.locations.documentSchemas/patch)

Please check the documentation how to write the update tests. If you have any questions, please let me know. https://googlecloudplatform.github.io/magic-modules/develop/add-handwritten-test/#update-tests

I have separated the tests into files, but I'm unsure of how to write the update tests properly. Also, considering that updating Document Schemas is not a customer requirement right now, can I add it later?

modular-magician · 2023-07-27T14:45:20Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 16 files changed, 3535 insertions(+), 2 deletions(-))
Terraform Beta: Diff ( 16 files changed, 3535 insertions(+), 2 deletions(-))
TF Conversion: Diff ( 6 files changed, 861 insertions(+), 3 deletions(-))
TF OiCS: Diff ( 40 files changed, 1285 insertions(+))

modular-magician · 2023-07-27T16:21:05Z

Tests analytics

Total tests: 2899
Passed tests 2587
Skipped tests: 302
Affected tests: 10

Action taken

Found 10 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccDocumentAIWarehouseDocumentSchema_documentAiWarehouseDocumentSchemaPropertyExample|TestAccDocumentAIWarehouseDocumentSchema_documentAiWarehouseDocumentSchemaFloatExample|TestAccDocumentAIWarehouseDocumentSchema_documentAiWarehouseDocumentSchemaIntegerExample|TestAccDocumentAIWarehouseDocumentSchema_documentAiWarehouseDocumentSchemaTextExample|TestAccDocumentAIWarehouseDocumentSchema_documentAiWarehouseDocumentSchemaTimestampExample|TestAccDocumentAIWarehouseDocumentSchema_documentAiWarehouseDocumentSchemaDatetimeExample|TestAccDocumentAIWarehouseDocumentSchema_documentAiWarehouseDocumentSchemaMapExample|TestAccDocumentAIWarehouseDocumentSchema_documentAiWarehouseDocumentSchemaEnumExample|TestAccDocumentAIWarehouseDocumentSchema_documentAiWarehouseDocumentSchemaPropertyEnumExample|TestAccContainerAwsNodePool_BetaBasicHandWritten

Get to know how VCR tests work

modular-magician · 2023-07-27T16:28:10Z

$\textcolor{green}{\textsf{Tests passed during RECORDING mode:}}$
TestAccDocumentAIWarehouseDocumentSchema_documentAiWarehouseDocumentSchemaPropertyExample[Debug log]
TestAccDocumentAIWarehouseDocumentSchema_documentAiWarehouseDocumentSchemaFloatExample[Debug log]
TestAccDocumentAIWarehouseDocumentSchema_documentAiWarehouseDocumentSchemaIntegerExample[Debug log]
TestAccDocumentAIWarehouseDocumentSchema_documentAiWarehouseDocumentSchemaTextExample[Debug log]
TestAccDocumentAIWarehouseDocumentSchema_documentAiWarehouseDocumentSchemaTimestampExample[Debug log]
TestAccDocumentAIWarehouseDocumentSchema_documentAiWarehouseDocumentSchemaDatetimeExample[Debug log]
TestAccDocumentAIWarehouseDocumentSchema_documentAiWarehouseDocumentSchemaMapExample[Debug log]
TestAccDocumentAIWarehouseDocumentSchema_documentAiWarehouseDocumentSchemaEnumExample[Debug log]
TestAccDocumentAIWarehouseDocumentSchema_documentAiWarehouseDocumentSchemaPropertyEnumExample[Debug log]

Rerun these tests in REPLAYING mode to catch issues

$\textcolor{green}{\textsf{No issues found for passed tests after REPLAYING rerun.}}$

$\textcolor{red}{\textsf{Tests failed during RECORDING mode:}}$
TestAccContainerAwsNodePool_BetaBasicHandWritten[Error message] [Debug log]

$\textcolor{red}{\textsf{Please fix these to complete your PR.}}$
View the build log or the debug log for each test

zli82016 · 2023-07-27T16:41:22Z

Thanks so much for making the changes. Can you please add a separate test file in the tests folder to test the [PATCH operation]? (https://cloud.google.com/document-warehouse/docs/reference/rest/v1/projects.locations.documentSchemas/patch)
Please check the documentation how to write the update tests. If you have any questions, please let me know. https://googlecloudplatform.github.io/magic-modules/develop/add-handwritten-test/#update-tests

I have separated the tests into files, but I'm unsure of how to write the update tests properly. Also, considering that updating Document Schemas is not a customer requirement right now, can I add it later?

This is an example to write updating test. https://github.com/GoogleCloudPlatform/magic-modules/blob/main/mmv1/third_party/terraform/tests/resource_cloud_iot_device_update_test.go

Also, do you know when updating Document Schemas will be a requirement for the users?

mmv1/products/documentaiwarehouse/DocumentSchema.yaml

modular-magician · 2023-07-28T04:46:18Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 16 files changed, 3517 insertions(+), 2 deletions(-))
Terraform Beta: Diff ( 16 files changed, 3517 insertions(+), 2 deletions(-))
TF Conversion: Diff ( 6 files changed, 861 insertions(+), 3 deletions(-))
TF OiCS: Diff ( 40 files changed, 1285 insertions(+))

modular-magician · 2023-07-28T06:30:18Z

Tests analytics

Total tests: 2900
Passed tests 2597
Skipped tests: 302
Affected tests: 1

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccContainerAwsNodePool_BetaBasicHandWritten

Get to know how VCR tests work

modular-magician · 2023-07-28T06:31:08Z

$\textcolor{red}{\textsf{Tests failed during RECORDING mode:}}$
TestAccContainerAwsNodePool_BetaBasicHandWritten[Error message] [Debug log]

$\textcolor{red}{\textsf{Please fix these to complete your PR.}}$
View the build log or the debug log for each test

zli82016

LGTM. Thanks.

* adding location and doc schema * added copyright * resolved required field, url_param_only field, description, autogen_async, async in Location * resolved required field, url_param_only field, description, autogen_async, async in Location * resolved hardcoded location, few fields in location, removed unnecessary enum values * resolved project_number, added fields in location example * DocumentSchema final changes * added timeouts in Location * adding missing fields from test report * added DocumentSchema tests * removed required field from processorType * changed dummy_values * removed fields under properties * added new line * added enum tests * separated tests into separate files * made Document Schema immutable

adding location and doc schema

b9f3b3e

modular-magician requested a review from zli82016 July 18, 2023 07:42

added copyright

c2f4c1c