fix: address issues with concurrent BigQuery tests

[shollyman]

This PR changes how testing is done for the BQ tests which now run multiple instances concurrently.

Basic premise is we inject new values into the code under test, and use the template syntax to elide the replacement from the embedded representations. the test harness takes care of deleting resources.

[leahecole]

I'm here for it, but I tend to have less strong opinions about this than some of the other folks. @tmatsuo has done a LOT of refactoring tests recently, so I want his eyes too to double check

[tmatsuo]

Looks good overall.

If the client library doesn't automatically retry transient failures, I'd retry few times upon such exceptions.
I found that backoff is a decent library for this purpose.

The code looks like:

    @backoff.on_exception(backoff.expo, max_time=60)
    def delete_dataset():
        try:
            client.delete_dataset(item, delete_contents=True)
        except NotFound as e:
            # We ignore this case.                                                                                                                                 
            print("The dataset doesn't exist: detail: {}".format(str(e)))
    delete_dataset()

If the client library already retries transient errors, just ignore my comment. Also, I'm open to do that research after merging this PR.

[tmatsuo]

Can you do the following?

import uuid

from google.cloud import bigquery
import pytest

import authorized_view_tutorial

Usually imports should be 1) standard libs 2) 3rd party libs and 3) package local libs, separated by an empty line.

[shollyman]

Looks good overall.

If the client library doesn't automatically retry transient failures, I'd retry few times upon such exceptions.
I found that backoff is a decent library for this purpose.

The code looks like:

    @backoff.on_exception(backoff.expo, max_time=60)
    def delete_dataset():
        try:
            client.delete_dataset(item, delete_contents=True)
        except NotFound as e:
            # We ignore this case.                                                                                                                                 
            print("The dataset doesn't exist: detail: {}".format(str(e)))
    delete_dataset()

If the client library already retries transient errors, just ignore my comment.

Yeah, BQ uses a default retry policy.

https://googleapis.dev/python/bigquery/latest/generated/google.cloud.bigquery.retry.DEFAULT_RETRY.html

[tmatsuo]

Thanks