-
Notifications
You must be signed in to change notification settings - Fork 1.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Added job_notification_emails and deidentify actions in google_data_loss_prevention_job_trigger #7687
Added job_notification_emails and deidentify actions in google_data_loss_prevention_job_trigger #7687
Changes from 1 commit
ab01534
b06a74c
d3f606a
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -371,6 +371,8 @@ properties: | |
- pub_sub | ||
- publish_findings_to_cloud_data_catalog | ||
- publish_summary_to_cscc | ||
- job_notification_emails | ||
- deidentify | ||
description: | | ||
If set, the detailed findings will be persisted to the specified OutputStorageConfig. Only a single instance of this action can be specified. Compatible with: Inspect, Risk | ||
properties: | ||
|
@@ -425,6 +427,8 @@ properties: | |
- pub_sub | ||
- publish_findings_to_cloud_data_catalog | ||
- publish_summary_to_cscc | ||
- job_notification_emails | ||
- deidentify | ||
description: | | ||
Publish a message into a given Pub/Sub topic when the job completes. | ||
properties: | ||
|
@@ -440,6 +444,8 @@ properties: | |
- pub_sub | ||
- publish_findings_to_cloud_data_catalog | ||
- publish_summary_to_cscc | ||
- job_notification_emails | ||
- deidentify | ||
allow_empty_object: true | ||
send_empty_value: true | ||
properties: [] | ||
|
@@ -452,8 +458,109 @@ properties: | |
- pub_sub | ||
- publish_findings_to_cloud_data_catalog | ||
- publish_summary_to_cscc | ||
- job_notification_emails | ||
- deidentify | ||
allow_empty_object: true | ||
send_empty_value: true | ||
properties: [] | ||
description: | | ||
Publish findings of a DlpJob to Data Catalog. | ||
- !ruby/object:Api::Type::NestedObject | ||
name: 'jobNotificationEmails' | ||
exactly_one_of: | ||
- save_findings | ||
- pub_sub | ||
- publish_findings_to_cloud_data_catalog | ||
- publish_summary_to_cscc | ||
- job_notification_emails | ||
- deidentify | ||
allow_empty_object: true | ||
send_empty_value: true | ||
properties: [] | ||
description: | | ||
Sends an email when the job completes. The email goes to IAM project owners and technical Essential Contacts. | ||
- !ruby/object:Api::Type::NestedObject | ||
name: 'deidentify' | ||
exactly_one_of: | ||
- save_findings | ||
- pub_sub | ||
- publish_findings_to_cloud_data_catalog | ||
- publish_summary_to_cscc | ||
- job_notification_emails | ||
- deidentify | ||
description: | | ||
Create a de-identified copy of the requested table or files. | ||
properties: | ||
- !ruby/object:Api::Type::String | ||
name: 'cloudStorageOutput' | ||
required: true | ||
description: | | ||
User settable Cloud Storage bucket and folders to store de-identified files. | ||
|
||
This field must be set for cloud storage deidentification. | ||
|
||
The output Cloud Storage bucket must be different from the input bucket. | ||
|
||
De-identified files will overwrite files in the output path. | ||
|
||
Form of: gs://bucket/folder/ or gs://bucket | ||
- !ruby/object:Api::Type::Array | ||
name: 'fileTypesToTransform' | ||
description: | | ||
List of user-specified file type groups to transform. If specified, only the files with these filetypes will be transformed. | ||
|
||
If empty, all supported files will be transformed. Supported types may be automatically added over time. | ||
|
||
If a file type is set in this field that isn't supported by the Deidentify action then the job will fail and will not be successfully created/started. | ||
item_type: !ruby/object:Api::Type::Enum | ||
name: 'undefined' | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The name should be fileType? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Thanks for pointing that out. I've updated the name to There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You can click the link in Diff report to check the generated resource in the provider.
|
||
description: | | ||
This field only has a name and description because of MM | ||
limitations. It should not appear in downstreams. | ||
values: | ||
- :IMAGES | ||
- :TEXT_FILES | ||
- :CSV | ||
- :TSV | ||
- !ruby/object:Api::Type::NestedObject | ||
name: 'transformationConfig' | ||
description: | | ||
User specified deidentify templates and configs for structured, unstructured, and image files. | ||
properties: | ||
- !ruby/object:Api::Type::String | ||
name: 'deidentifyTemplate' | ||
description: | | ||
If this template is specified, it will serve as the default de-identify template. | ||
- !ruby/object:Api::Type::String | ||
name: 'structuredDeidentifyTemplate' | ||
description: | | ||
If this template is specified, it will serve as the de-identify template for structured content such as delimited files and tables. | ||
- !ruby/object:Api::Type::String | ||
name: 'imageRedactTemplate' | ||
description: | | ||
If this template is specified, it will serve as the de-identify template for images. | ||
- !ruby/object:Api::Type::NestedObject | ||
name: 'transformationDetailsStorageConfig' | ||
description: | | ||
Config for storing transformation details. | ||
properties: | ||
- !ruby/object:Api::Type::NestedObject | ||
name: 'table' | ||
required: true | ||
description: | | ||
The BigQuery table in which to store the output. | ||
properties: | ||
- !ruby/object:Api::Type::String | ||
name: 'datasetId' | ||
description: The ID of the dataset containing this table. | ||
required: true | ||
- !ruby/object:Api::Type::String | ||
name: 'projectId' | ||
description: The ID of the project containing this table. | ||
required: true | ||
- !ruby/object:Api::Type::String | ||
name: 'tableId' | ||
description: | | ||
The ID of the table. The ID must contain only letters (a-z, | ||
A-Z), numbers (0-9), or underscores (_). The maximum length | ||
is 1,024 characters. |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -101,6 +101,57 @@ func TestAccDataLossPreventionJobTrigger_dlpJobTriggerPubsub(t *testing.T) { | |
}) | ||
} | ||
|
||
func TestAccDataLossPreventionJobTrigger_dlpJobTriggerJobNotificationEmails(t *testing.T) { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can you please move these two new tests to examples in JobTrigger.yaml file? https://github.com/GoogleCloudPlatform/magic-modules/blob/main/mmv1/products/dlp/JobTrigger.yaml#L28 In this file mmv1/third_party/terraform/tests/resource_data_loss_prevention_job_trigger_test.go, can you please add tests for updating field? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I've moved these tests to examples and also added the update tests. |
||
t.Parallel() | ||
|
||
context := map[string]interface{}{ | ||
"project": GetTestProjectFromEnv(), | ||
} | ||
|
||
VcrTest(t, resource.TestCase{ | ||
PreCheck: func() { AccTestPreCheck(t) }, | ||
ProtoV5ProviderFactories: ProtoV5ProviderFactories(t), | ||
CheckDestroy: testAccCheckDataLossPreventionJobTriggerDestroyProducer(t), | ||
Steps: []resource.TestStep{ | ||
{ | ||
Config: testAccDataLossPreventionJobTrigger_dlpJobTriggerJobNotificationEmails(context), | ||
}, | ||
{ | ||
ResourceName: "google_data_loss_prevention_job_trigger.job_notification_emails", | ||
ImportState: true, | ||
ImportStateVerify: true, | ||
ImportStateVerifyIgnore: []string{"parent"}, | ||
}, | ||
}, | ||
}) | ||
} | ||
|
||
func TestAccDataLossPreventionJobTrigger_dlpJobTriggerDeidentify(t *testing.T) { | ||
t.Parallel() | ||
|
||
context := map[string]interface{}{ | ||
"project": GetTestProjectFromEnv(), | ||
"random_suffix": RandString(t, 10), | ||
} | ||
|
||
VcrTest(t, resource.TestCase{ | ||
PreCheck: func() { AccTestPreCheck(t) }, | ||
ProtoV5ProviderFactories: ProtoV5ProviderFactories(t), | ||
CheckDestroy: testAccCheckDataLossPreventionJobTriggerDestroyProducer(t), | ||
Steps: []resource.TestStep{ | ||
{ | ||
Config: testAccDataLossPreventionJobTrigger_dlpJobTriggerDeidentify(context), | ||
}, | ||
{ | ||
ResourceName: "google_data_loss_prevention_job_trigger.deidentify", | ||
ImportState: true, | ||
ImportStateVerify: true, | ||
ImportStateVerifyIgnore: []string{"parent"}, | ||
}, | ||
}, | ||
}) | ||
} | ||
|
||
func testAccDataLossPreventionJobTrigger_dlpJobTriggerBasic(context map[string]interface{}) string { | ||
return Nprintf(` | ||
resource "google_data_loss_prevention_job_trigger" "basic" { | ||
|
@@ -294,3 +345,121 @@ resource "google_data_loss_prevention_job_trigger" "pubsub" { | |
} | ||
`, context) | ||
} | ||
|
||
func testAccDataLossPreventionJobTrigger_dlpJobTriggerJobNotificationEmails(context map[string]interface{}) string { | ||
return Nprintf(` | ||
resource "google_data_loss_prevention_job_trigger" "job_notification_emails" { | ||
parent = "projects/%{project}" | ||
description = "Description for the job_trigger created by terraform" | ||
display_name = "TerraformDisplayName" | ||
|
||
triggers { | ||
schedule { | ||
recurrence_period_duration = "86400s" | ||
} | ||
} | ||
|
||
inspect_job { | ||
inspect_template_name = "sample-inspect-template" | ||
actions { | ||
job_notification_emails {} | ||
} | ||
storage_config { | ||
cloud_storage_options { | ||
file_set { | ||
url = "gs://mybucket/directory/" | ||
} | ||
} | ||
} | ||
} | ||
} | ||
`, context) | ||
} | ||
|
||
func testAccDataLossPreventionJobTrigger_dlpJobTriggerDeidentify(context map[string]interface{}) string { | ||
return Nprintf(` | ||
resource "google_data_loss_prevention_job_trigger" "deidentify" { | ||
parent = "projects/%{project}" | ||
description = "Description for the job_trigger created by terraform" | ||
display_name = "TerraformDisplayName" | ||
|
||
triggers { | ||
schedule { | ||
recurrence_period_duration = "86400s" | ||
} | ||
} | ||
|
||
inspect_job { | ||
inspect_template_name = "sample-inspect-template" | ||
actions { | ||
deidentify { | ||
cloud_storage_output = "gs://samplebucket/dir/" | ||
file_types_to_transform = ["CSV", "TSV"] | ||
transformation_details_storage_config { | ||
table { | ||
project_id = "%{project}" | ||
dataset_id = google_bigquery_dataset.default.dataset_id | ||
table_id = google_bigquery_table.default.table_id | ||
} | ||
} | ||
transformation_config { | ||
deidentify_template = "sample-deidentify-template" | ||
image_redact_template = "sample-image-redact-template" | ||
structured_deidentify_template = "sample-structured-deidentify-template" | ||
} | ||
} | ||
} | ||
storage_config { | ||
cloud_storage_options { | ||
file_set { | ||
url = "gs://mybucket/directory/" | ||
} | ||
} | ||
} | ||
} | ||
} | ||
|
||
resource "google_bigquery_dataset" "default" { | ||
dataset_id = "tf_test_%{random_suffix}" | ||
friendly_name = "terraform-test" | ||
description = "Description for the dataset created by terraform" | ||
location = "US" | ||
default_table_expiration_ms = 3600000 | ||
|
||
labels = { | ||
env = "default" | ||
} | ||
} | ||
|
||
resource "google_bigquery_table" "default" { | ||
dataset_id = google_bigquery_dataset.default.dataset_id | ||
table_id = "tf_test_%{random_suffix}" | ||
deletion_protection = false | ||
|
||
time_partitioning { | ||
type = "DAY" | ||
} | ||
|
||
labels = { | ||
env = "default" | ||
} | ||
|
||
schema = <<EOF | ||
[ | ||
{ | ||
"name": "quantity", | ||
"type": "NUMERIC", | ||
"mode": "NULLABLE", | ||
"description": "The quantity" | ||
}, | ||
{ | ||
"name": "name", | ||
"type": "STRING", | ||
"mode": "NULLABLE", | ||
"description": "Name of the object" | ||
} | ||
] | ||
EOF | ||
} | ||
`, context) | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just want to double check that the value of jobNotificationEmails should be {} ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In the API docs, there is no information about the object of
jobNotificationEmails
. Moreover, I've tried sendingjobNotificationEmails
as {} and it's working fine. I've also checked this with the API explorer and it is not suggesting any fields insidejobNotificationEmails
.Following is the payload I've used to configure
jobTriggers
withjobNotificationEmails
action:There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I cannot find the doc for this field, either. I asked the service team and am waiting for their response. Sorry for holding the PR a little bit.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I checked with the service team and they confirmed that this field should be an empty json object.