Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add event_filters to event triggers in cloudfunctions2 resource #114

Conversation

modular-magician
Copy link
Collaborator

Description

Closes hashicorp/terraform-provider-google#12021

This PR adds the ability to set event filters within event triggers defined when creating a 2nd generation cloud function. Previously users could make 2nd gen functions be triggered by events sent from an 'event provider' (i.e. a Google API) within Eventarc, but now they can filter those events based on the event's attributes.

Valid attributes that can be used in filters differ by the event provider - here's the relevant part of the docs that describes a gcloud command to learn more about the events and attributes for a given event provider.

E.g. All events related to the storage.googleapis.com event provider have attributes type and bucket - and bucket wouldn't be an attribute on non-GCS events

$ gcloud eventarc providers describe storage.googleapis.com --location=us-central1

displayName: Cloud Storage
eventTypes:
- description: The live version of an object has become a noncurrent version.
  filteringAttributes:
  - attribute: bucket
    description: The bucket name being watched.
    required: true
  - attribute: type
    required: true
  type: google.cloud.storage.object.v1.archived

#...omitted other events in list...

name: projects/[PROJECT_ID]/locations/us-central1/providers/storage.googleapis.com

Things to note

  1. How should operator be used? I couldn't work it out for an acceptance test - addressed in fb04576 - docs are here and path patterns aren't appropriate for all attributes

  2. The type attribute cannot be used in the new event_filters block because it is already set by the event_type field inside the event_trigger block.

A configuration similar to the one below causes this error

The request was invalid: duplicated declaration of attribute \"type\" in trigger.event_filters
resource "google_cloudfunctions2_function" "terraform-test" {
 
  ...

  event_trigger {
    trigger_region                = "us-central1"
    event_type                    = "google.cloud.storage.object.v1.finalized"
    retry_policy                  = "RETRY_POLICY_RETRY"
    service_account_email = google_service_account.account.email
    event_filters {
      attribute                        = "bucket"
      value                            = google_storage_bucket.trigger-bucket.name
    }
    event_filters {
      attribute                        = "type" # This clashes with event_type and triggers an API error
      value                            = "google.cloud.storage.object.v1.finalized"
    }
  }
}

If this PR is for Terraform, I acknowledge that I have:

  • Searched through the issue tracker for an open issue that this either resolves or contributes to, commented on it to claim it, and written "fixes {url}" or "part of {url}" in this PR description. If there were no relevant open issues, I opened one and commented that I would like to work on it (not necessary for very small changes).
  • Generated Terraform, and ran make test and make lint to ensure it passes unit and linter tests.
  • Ensured that all new fields I added that can be set by a user appear in at least one example (for generated resources) or third_party test (for handwritten resources or update tests).
  • Ran relevant acceptance tests (If the acceptance tests do not yet pass or you are unable to run them, please let your reviewer know).
  • Read the Release Notes Guide before writing my release note below.

Release Note Template for Downstream PRs (will be copied)

cloudfunctions2: added field `event_filters` to resource `google_cloudfunctions2_function` (beta)

Derived from GoogleCloudPlatform/magic-modules#6278

…278)

* Add `eventFilters` in cloudfunctions2 API definition

* Remove trailing whitespace

* Start adding cloudfunctions2 eventarc acceptance test that uses GCS

* Add missing IAM bindings to prevent API errors during `FullGcsExample` acceptance test

* Make `eventTrigger.pubsubTopic` default from the API to handle topics made by Eventarc

* Update field description for `attribute` - info to learn more about possible values

* Remove unneeded attribute from `google_storage_bucket_object`

* Add API service enabling to new project in test, fix provider used to make project

* Fix inbuilt function call in acceptance test

* Add explicit dependencies to try make API calls happen after the API is turned on

* Remove generated acceptance test and add handwritten test with 2 stages

* Remove `depends_on` attributes from example config

* Fix bug in test's TF config, make project ID and name the same

* Replace use of `for_each` in acceptance test

There is an open issue about how for_each loops don't work : hashicorp/terraform-plugin-sdk#536

* Add `run.googleapis.com` to project

* Change `disable_dependent_services` to `false` for eventarc API in acceptance tests

Change example config to match, too

* Enable `artifactregistry.googleapis.com` in acceptance test & example

* Add missing IAM binding to cloud function service account - `roles/artifactregistry.reader`

Odd that this is needed, as it wasn't required when I created a similar cloud function in an existing GCP project (where APIs were already activated etc)

* Add explicit dependency between cloud function resource and IAM bindings for its service account

Address "Unable to retrieve the repository metadata" error that suggests service account doesn't have permissions in place

* Enable `cloudbuild.googleapis.com` API in acceptance test and example

* Enable `pubsub.googleapis.com` API in acceptance test and example

* Fix test so step 1 of test enables all the APIs

* Replace zipped cloud function code with zip made by `archive_file` Terraform resource

* Add repo-level IAM binding (in addition to project level) using inferred repo ID

* Fix missing quotation mark in test

:(

* Split project into 3 stages and remove repo-specific IAM binding

This is after running the test locally and it succeeding in 3 steps but not in 2 steps

* Fix problem where TF couldn't delete resources at end of test

Couldn't delete the cloud function because the API had been disabled

* Update test to get project ID from ENV instead of project created by the test

Update test and example block with needed variables

Remove handwritten test

* Fix API spec to make required fields required

* Update event_filter fields' descriptions for clarity

* Change GCS event_filter test to basic as it can't use all fields

* Add example test that shows the operator field in use

* Fix whitespace

* Add test for updating `event_filters` blocks

* Reuse generated cloudfunction2 test functions in handwritten test

Make code homogenous between the two tests to faciliatate this

Signed-off-by: Modular Magician <[email protected]>
@modular-magician modular-magician merged commit f8ca788 into terraform-google-modules:main Aug 2, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

google_cloudfunctions2_function - support event_filters
1 participant