Add support for providing additional experiments to Dataflow job

[wazim]

An example use case is as-per: https://cloud.google.com/dataflow/docs/guides/using-cloud-monitoring#receive_worker_vm_metrics_from_the_agent. It is needed that an additional experiment is configured: enable_stackdriver_agent_metrics.

An example looks like:

resource "google_dataflow_job" "with_additional_experiments" {
  name = "dataflow-job"

  additional_experiments = ["enable_stackdriver_agent_metrics"]

  template_gcs_path = "gs://testing/template"
  temp_gcs_location = "gs://testing/tmp"
  parameters = {
    inputFile = "gs://testing/input"
    output    = "gs://testing/output"
  }
  on_delete = "cancel"
}

[danawillow]

Is the order here meaningful? If not, I'd recommend using a TypeSet instead.

[wazim]

Good idea! I've updated it to TypeSet. Thanks!

[danawillow]

Want to add a few more values to this list just for extra confidence?

[wazim]

I've added shuffle_mode=service to the list as per https://cloud.google.com/blog/products/gcp/introducing-cloud-dataflow-shuffle-for-up-to-5x-performance-improvement-in-data-analytic-pipelines. I can add some more if you think there is value in it? Thanks!

[danawillow]

Looks good, thanks @wazim!

Since the source of truth for our resources is at https://github.com/GoogleCloudPlatform/magic-modules, I'll take care of upstreaming this change there.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 [email protected]. Thanks!