serviceusage: retry on 403's for operation calls

[ScottSuarez]

closes hashicorp/terraform-provider-google#9489

Release Note Template for Downstream PRs (will be copied)

serviceusage: enabled the service api to retry on failed operation calls in anticipation of transient errors that occur when first enabling the service.

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 1 file changed, 22 insertions(+), 3 deletions(-))
Terraform Beta: Diff ( 1 file changed, 22 insertions(+), 3 deletions(-))

[ScottSuarez]

/gcbrun

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 1 file changed, 22 insertions(+), 3 deletions(-))
Terraform Beta: Diff ( 2 files changed, 23 insertions(+), 4 deletions(-))

[rileykarson]

Won't this just get reset on every iteration? I don't think the code allows us to retry n times as written (and the change to do so would be messy- not sure we want to do it to the common op code)

[ScottSuarez]

umm I'm confused? This is specific to three times. We use w.retryCount to monitor the count.

If you look at the downstream code it should make more sense.

[modular-magician]

I have triggered VCR tests in RECORDING mode for the following tests that failed during VCR: TestAccComputeServiceAttachment_serviceAttachmentBasicExample You can view the result here: https://ci-oss.hashicorp.engineering/viewQueued.html?itemId=207248

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 1 file changed, 22 insertions(+), 3 deletions(-))
Terraform Beta: Diff ( 2 files changed, 23 insertions(+), 4 deletions(-))

[modular-magician]

I have triggered VCR tests in RECORDING mode for the following tests that failed during VCR: TestAccComputeServiceAttachment_serviceAttachmentBasicExampleUpdate|TestAccNetworkServicesEdgeCacheOrigin_networkServicesEdgeCacheOriginBasicExample|TestAccNetworkServicesEdgeCacheOrigin_networkServicesEdgeCacheOriginAdvancedExample|TestAccNetworkServicesEdgeCacheService_networkServicesEdgeCacheServiceAdvancedExample You can view the result here: https://ci-oss.hashicorp.engineering/viewQueued.html?itemId=207362

[rileykarson]

Maybe specify how it was fixed in the changelog entry- we aren't 100% certain it worked because you had no repro or test case, right?

[ScottSuarez]

Maybe specify how it was fixed in the changelog entry- we aren't 100% certain it worked because you had no repro or test case, right?

correct, I could not reproduce on my end. Presumably this should fix the issue for the team. We will have to monitor if the issue still occurs.