vcr: use go-vcr package to setup vcr recorder

[gemmahou]

Change description

In this PR, I implemented the same thing as in #1345, with a different approach. I incorporated go-vcr package and used its recorder.

Tests you have done

• Run make ready-pr to ensure this PR is ready for review.
• Perform necessary E2E testing for changed resources.

[justinsb]

A few nit but LGTM... can we add a VCR test as a GitHub actions presubmit (maybe just running one subtest to start with?)

[gemmahou]

A few nit but LGTM... can we add a VCR test as a GitHub actions presubmit (maybe just running one subtest to start with?)

I'm thinking about that too, I'll have that setup in a future PR and see how it works. I'm also thinking about if we can migrate VCR test into our prow job so that we can save cassette files in GCS private storage, similar to artifacts http logs.

[maqiuyujoyce]

Looks like it's the diff base for #1349 though I've already left some non-blocking comments there. /lgtm

[maqiuyujoyce]

/approve

[google-oss-prow]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: maqiuyujoyce

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [maqiuyujoyce]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment