Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature/Addition] Add Default Timeout to BigQuery Database Connections #910

Open
dexchan opened this issue Oct 24, 2023 · 0 comments · May be fixed by #1105
Open

[Feature/Addition] Add Default Timeout to BigQuery Database Connections #910

dexchan opened this issue Oct 24, 2023 · 0 comments · May be fixed by #1105
Labels
connector update Work type - additions or changes to the functions of an existing Parsons connector enhancement Impact - something should be added to or changed about Parsons that isn't causing a current breakage high priority Priority - addressing this is an urgent need for a broad swath of Parsons users

Comments

@dexchan
Copy link
Contributor

dexchan commented Oct 24, 2023

Set a default timeout parameter for BigQuery queries.

Detailed Description

Per Google BQ query job documentation, it's possible to pass a default timeout parameter to the synchronous call that awaits a response.

This change would require choosing a reasonable maximum default (30s?), setting that up as a configurable parameter, and passing that parameter into load_job.result.

Context

Generally, it's best practice to establish a timeout for database connections, to prevent code from indefinitely waiting for a response under unreliable network conditions.

Per #452, there was existing low-priority work to set this up with the built-in SQL connector. Having a well-chosen default timeout and good notifications around errors will make everything dependent on parsons more robust.

Possible Implementation

Should be as easy as changing

load_job.result()

and

load_job.result()

to load_job.result(timeout=MAX_TIMEOUT)

Priority

High -- this is a very low lift now, but becomes much harder to implement and test once more code becomes dependent on the new BQ connector.

@dexchan dexchan added the enhancement Impact - something should be added to or changed about Parsons that isn't causing a current breakage label Oct 24, 2023
@shaunagm shaunagm added high priority Priority - addressing this is an urgent need for a broad swath of Parsons users connector update Work type - additions or changes to the functions of an existing Parsons connector labels Dec 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
connector update Work type - additions or changes to the functions of an existing Parsons connector enhancement Impact - something should be added to or changed about Parsons that isn't causing a current breakage high priority Priority - addressing this is an urgent need for a broad swath of Parsons users
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants