Added pubsubConfig and webhookConfig support to the cloud build resource.

[iamsumit]

Google have API documentation to create a pub/sub and webhook trigger in the cloud build resource, but, the terraform google provider doesn't. I have updated the terraform resource to add that support.

Fixes hashicorp/terraform-provider-google#8692, hashicorp/terraform-provider-google#9189

If this PR is for Terraform, I acknowledge that I have:

• Searched through the issue tracker for an open issue that this either resolves or contributes to, commented on it to claim it, and written "fixes {url}" or "part of {url}" in this PR description. If there were no relevant open issues, I opened one and commented that I would like to work on it (not necessary for very small changes).
• Generated Terraform, and ran make test and make lint to ensure it passes unit and linter tests.
• Ensured that all new fields I added that can be set by a user appear in at least one example (for generated resources) or third_party test (for handwritten resources or update tests).
• Ran relevant acceptance tests (If the acceptance tests do not yet pass or you are unable to run them, please let your reviewer know).
• Read the Release Notes Guide before writing my release note below.

Release Note Template for Downstream PRs (will be copied)

cloudbuild: Added `pubsub_config` and `webhook_config` parameter to `google_cloudbuild_trigger`.

[google-cla]

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.

---

What to do if you already signed the CLA

Individual signers

• It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.

Corporate signers

• Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot (Public version).
• The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
• The email used to register you as an authorized contributor must also be attached to your GitHub account.

ℹ️ Googlers: Go here for more info.

[modular-magician]

Hello! I am a robot who works on Magic Modules PRs.

I have detected that you are a community contributor, so your PR will be assigned to someone with a commit-bit on this repo for initial review.

Thanks for your contribution! A human will be with you soon.

@c2thorn, please review this PR or find an appropriate assignee.

[iamsumit]

@googlebot I signed it!

[google-cla]

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors. If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and then comment @googlebot I fixed it.. If the bot doesn't comment, it means it doesn't think anything has changed.

ℹ️ Googlers: Go here for more info.

[iamsumit]

@googlebot I fixed it.

[c2thorn]

@iamsumit, sorry about this, but could you rebase off of the latest master commit? We changed our CI and unfortunately it means our tests won't run without doing so.

[iamsumit]

@c2thorn I have rebased it. Thank you.

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 3 files changed, 532 insertions(+), 5 deletions(-))
Terraform Beta: Diff ( 4 files changed, 533 insertions(+), 6 deletions(-))
Ansible: Diff ( 2 files changed, 190 insertions(+))
TF Conversion: Diff ( 1 file changed, 102 insertions(+))
Inspec: Diff ( 6 files changed, 108 insertions(+))

[iamsumit]

@c2thorn Is there anything I have to do for the failing test?

[c2thorn]

Looks solid, just requesting a few schema changes. Thanks for adding the update tests in there as well!

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 3 files changed, 529 insertions(+), 5 deletions(-))
Terraform Beta: Diff ( 5 files changed, 531 insertions(+), 8 deletions(-))
Ansible: Diff ( 2 files changed, 157 insertions(+))
TF Conversion: Diff ( 1 file changed, 102 insertions(+))
Inspec: Diff ( 6 files changed, 108 insertions(+))

[iamsumit]

@c2thorn A gentle reminder to review this PR. Please do let me know if there is anything I am missing?

[c2thorn]

/gcbrun

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 3 files changed, 529 insertions(+), 5 deletions(-))
Terraform Beta: Diff ( 3 files changed, 529 insertions(+), 5 deletions(-))
Ansible: Diff ( 2 files changed, 157 insertions(+))
TF Conversion: Diff ( 1 file changed, 102 insertions(+))
Inspec: Diff ( 6 files changed, 108 insertions(+))

[modular-magician]

I have triggered VCR tests in RECORDING mode for the following tests that failed during VCR: TestAccBigQueryDataTable_jsonEquivalency|TestAccCloudBuildTrigger_pubsub_config|TestAccCloudBuildTrigger_webhook_config|TestAccComputeInstance_updateRunning_desiredStatusNotSet_notAllowStoppingForUpdate|TestAccComputeInstance_updateRunning_desiredStatusRunning_notAllowStoppingForUpdate|TestAccContainerNodePool_withGPU|TestAccPrivatecaCaPool_privatecaCapoolUpdate|TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthorityBasicExample|TestAccPrivatecaCertificate_privatecaCertificateConfigExample|TestAccPrivatecaCertificate_privatecaCertificateNoAuthorityExample You can view the result here: https://ci-oss.hashicorp.engineering/viewQueued.html?itemId=195858

[c2thorn]

Relevant tests have passed

[modular-magician]

Tests failed during RECORDING mode: TestAccPrivatecaCaPool_privatecaCapoolUpdate|TestAccPrivatecaCertificate_privatecaCertificateNoAuthorityExample|TestAccContainerNodePool_withGPU Please fix these to complete your PR

[iamsumit]

@c2thorn So, I checked that the 3.75 includes the changes but when I used that on my local, it doesn't work. Did we miss anything?

hashicorp/terraform-provider-google@v3.75.0...master

[c2thorn]

@iamsumit the release was cut before this change. https://github.com/hashicorp/terraform-provider-google/commits/release-3.75.0 shows the commits in 3.75, I believe your link shows the differences between v3.75.0 and master

This should be released next week on Monday

[iamsumit]

@c2thorn Aah. My bad. Thank you for confirming.

[ffjeremy]

I've updated to version v3.77.0, but it appears that this feature is still missing some functionality. I do not see a way to specify the repo/branch info. These fields are required when creating a pubsub trigger from the GCP console. Did I miss something in the docs, or is feature not complete yet?

[iamsumit]

I've updated to version v3.77.0, but it appears that this feature is still missing some functionality. I do not see a way to specify the repo/branch info. These fields are required when creating a pubsub trigger from the GCP console. Did I miss something in the docs, or is feature not complete yet?

There is no parameter in Google API documentation to specify this information. I know it is required in the UI, but it seems like a bug, and there is no need to specify those in the terraform or over the rest API.

[ffjeremy]

@iamsumit It seems that pubsubConfig should not be mutually exclusive with github|triggerTemplate. Perhaps that is the missing piece. Otherwise there is no way to actually run any code.

[c2thorn]

I would suggest submitting a new Github issue outlining the problem here so it can get assigned and tracked.

[tuananhnguyen-ct]

I have the same issue using webhookConfig being unable to specify a source repo

[albrechtflo-hg]

Did anyone already create an issue for the missing repo information? If yes, please link the issue here.

FYI @iamsumit The fact that the repo could not / had not to be passed via REST API was most likely a bug in the Google API. We reported a related bug to them within our support subscription, and that has been fixed about four weeks ago (previously, it was not even possible to create a Pub/Sub trigger with a repository reference using either REST API or gcloud command line tool - see https://issuetracker.google.com/issues/191394513).

But, if you refer to their gcloud docs (https://cloud.google.com/sdk/gcloud/reference/alpha/builds/triggers/create/pubsub), you will see that there is a --repo and a --branch flag, which is even mandatory as soon as you specify a topic (the only alternative would be to specify a trigger config file, which of course would also contain repo / branch information). I can confirm that this gcloud command currently works as specified.

So, this feature here seems indeed completely unusable ATM (also because of the missing filter feature which is required for Pub/Sub based build triggers to avoid uncontrolled number of builds, see hashicorp/terraform-provider-google#9755).

If no one responds regarding a created bug issue, I will create one next week.

[iamsumit]

@albrechtflo-hg I was aware of the fact that this was not part of the API. We actually talked to Google folks about the missing rest API as you said the rest API was not documented earlier. They added it after our discussion only. Our primary need was to use the webhook trigger only so I didn't pursue it further.

It's not just about the issue of missing repo key in the API but also I noticed that the configuration form was not showing the pub/sub button checked and the the rest API doesn't need the repo. I did told them but didn't raise any issue in the google issue tracker. I wonder why the repo is required in pub/sub trigger. It should be optional just like it is in the webhook trigger.

[albrechtflo-hg]

@iamsumit But if you do not specify a repo, what shall the Cloud Build build?

[iamsumit]

@albrechtflo-hg The requirement is quite complicated. Let's say the code is on GitHub's private repository and cloud build needs to access it, it requires authorisation using manual actions. What if we don't want automate the process? So, the idea is to give cloud build an access token to access the private repository and then do its thing. I hope this make sense.

[ffjeremy]

Did anyone already create an issue for the missing repo information? If yes, please link the issue here.

FYI @iamsumit The fact that the repo could not / had not to be passed via REST API was most likely a bug in the Google API. We reported a related bug to them within our support subscription, and that has been fixed about four weeks ago (previously, it was not even possible to create a Pub/Sub trigger with a repository reference using either REST API or gcloud command line tool - see https://issuetracker.google.com/issues/191394513).

But, if you refer to their gcloud docs (https://cloud.google.com/sdk/gcloud/reference/alpha/builds/triggers/create/pubsub), you will see that there is a --repo and a --branch flag, which is even mandatory as soon as you specify a topic (the only alternative would be to specify a trigger config file, which of course would also contain repo / branch information). I can confirm that this gcloud command currently works as specified.

So, this feature here seems indeed completely unusable ATM (also because of the missing filter feature which is required for Pub/Sub based build triggers to avoid uncontrolled number of builds, see hashicorp/terraform-provider-google#9755).

If no one responds regarding a created bug issue, I will create one next week.

I had meant to create an issue but didn't get around to it before I went on vacation, and then I forgot about it. Please go ahead since it sounds like you have more details about the issue than I currently have/remember.

[albrechtflo-hg]

@albrechtflo-hg The requirement is quite complicated. Let's say the code is on GitHub's private repository and cloud build needs to access it, it requires authorisation using manual actions. What if we don't want automate the process? So, the idea is to give cloud build an access token to access the private repository and then do its thing. I hope this make sense.

To be honest, no, it does not make sense to me - I do not understand that use case. How should the build know what to do with the GitHub repository? And why should there be manual steps involved in a Pub/Sub based trigger? (There is a manual trigger for this, isn't it?)

I now have created a bug issue for this: hashicorp/terraform-provider-google#9936

Perhaps you could add some .tf examples there for this use case. In any case, discussions should be continued there.

[bbhoss]

I guess this currently only allows you to trigger cloudbuilds defined inline without any associated repository? It seems like at the very least you could allow trigger_template with pubsub_config or webhook_config. That way someone could still manually create a cloud source repo and enable the integration/sync using the GUI. Not ideal at all considering that via the GCP UI you can create pubsub and webhook or even manually triggered builds using the github app without selecting push/tag/etc trigger. But it would be better than totally blocking with no workarounds.