diff --git a/Rakefile b/Rakefile index 968a4a9f..1934b37d 100644 --- a/Rakefile +++ b/Rakefile @@ -82,13 +82,8 @@ task :update_bigquery_tables do config.version = `bundle exec ruby -e 'puts DfE::ReferenceData::VERSION'`.chomp config.commit = `git rev-parse HEAD`.chomp - # Suffix table names with the major version number, so multiple release - # branches can coexist peacefully - version_parts = config.version.split('.') - major_version = version_parts[0] - minor_version = version_parts[1] || '0' - - config.table_name_suffix = "_v#{major_version}_#{minor_version}" + # Suffix table names with '_latest' to push to the same table + config.table_name_suffix = '_latest' puts "Updating #{config.project}.#{config.dataset} with version #{config.version}:" end diff --git a/docs/bigquery.md b/docs/bigquery.md index e1359edc..882a05e6 100644 --- a/docs/bigquery.md +++ b/docs/bigquery.md @@ -13,9 +13,7 @@ The following lists are, on each release, automatically written into BigQuery un Edit `BIGQUERY_TABLES` in the Rakefile to change this list. The table name is made by taking the base table name from the table above, and -appending a version based on the major version of the release, for instance -`_v1`. This is to allow existing users of older releases to continue operating, -rather than changing the data structure out underneath them. +appending `_latest`. This ensures that the data is always pushed to the same table, providing a consistent and up-to-date reference. ### Internals @@ -38,5 +36,5 @@ If you have a `dfe-reference-data_bigquery_api_key.json` file or the test of the BigQuery importer will be run. This will create a randomly-named table in the `cross-teacher-services.dfe_reference_data_dev` dataset and, hopefully, delete it afterwards. The project and dataset used for the test can -be overriden with the `BIGQUERY_QA_PROJECT` and `BIGQUERY_QA_DATASET` +be overridden with the `BIGQUERY_QA_PROJECT` and `BIGQUERY_QA_DATASET` environment variables, respectively. diff --git a/docs/lists_qualifications.md b/docs/lists_qualifications.md index d5afe35b..bfb58893 100644 --- a/docs/lists_qualifications.md +++ b/docs/lists_qualifications.md @@ -28,4 +28,3 @@ This list is [autocomplete compatible](autocomplete_compatability.md). | `match_synonyms` | string array | A list of common alternative names that are equivalent to this type. An answer matching a match synonym can be safely matched to this type.| | `level` | enumerated string | The level of the qualification. `entry`, `1`, `2`, `3`, `4`, `5`, `6`, or `7`. | | `degree` | optional enumerated symbol | If the qualification type is a degree, the level of degree - `foundation`, `bachelor`, `master` or `doctor` | - diff --git a/lib/dfe/reference_data/bigquery/importer.rb b/lib/dfe/reference_data/bigquery/importer.rb index 1f31809a..558182db 100644 --- a/lib/dfe/reference_data/bigquery/importer.rb +++ b/lib/dfe/reference_data/bigquery/importer.rb @@ -21,7 +21,7 @@ class Config self.max_retry_sleep = 60 # seconds self.bigquery_retries = 10 self.bigquery_timeout = 10 - self.table_name_suffix = '' + self.table_name_suffix = '_latest' # Default suffix def self.configure yield(self) @@ -236,7 +236,7 @@ def convert_list_to_bigquery_format(list) end def update_reference_list_into_bigquery_table(dataset, table_name, list) - table = dataset.table table_name + table = dataset.table(table_name) # NOTE: TO FUTURE SELF: If we start to deal with very large reference # lists we might start to hit BigQuery API limits documented here: