[Feature/Addition] Add Default Timeout to BigQuery Database Connections #910
Labels
connector update
Work type - additions or changes to the functions of an existing Parsons connector
enhancement
Impact - something should be added to or changed about Parsons that isn't causing a current breakage
high priority
Priority - addressing this is an urgent need for a broad swath of Parsons users
Set a default
timeout
parameter for BigQuery queries.Detailed Description
Per Google BQ query job documentation, it's possible to pass a default timeout parameter to the synchronous call that awaits a response.
This change would require choosing a reasonable maximum default (30s?), setting that up as a configurable parameter, and passing that parameter into
load_job.result
.Context
Generally, it's best practice to establish a timeout for database connections, to prevent code from indefinitely waiting for a response under unreliable network conditions.
Per #452, there was existing low-priority work to set this up with the built-in SQL connector. Having a well-chosen default timeout and good notifications around errors will make everything dependent on parsons more robust.
Possible Implementation
Should be as easy as changing
parsons/parsons/google/google_bigquery.py
Line 417 in 55d3717
and
parsons/parsons/google/google_bigquery.py
Line 572 in 55d3717
to
load_job.result(timeout=MAX_TIMEOUT)
Priority
High -- this is a very low lift now, but becomes much harder to implement and test once more code becomes dependent on the new BQ connector.
The text was updated successfully, but these errors were encountered: