Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update WorldBank WDI scripts #963

Draft
wants to merge 5 commits into
base: master
Choose a base branch
from

remove properties not used in SVObs from tmcf,csv

7d8604f
Select commit
Loading
Failed to load commit list.
Draft

Update WorldBank WDI scripts #963

remove properties not used in SVObs from tmcf,csv
7d8604f
Select commit
Loading
Failed to load commit list.
Google Cloud Build / data-pull-request-py (datcom-ci) succeeded Jan 9, 2024 in 6m 29s

Summary

Build Information

Trigger data-pull-request-py
Build 5821c49c-10ab-43a3-a951-830f99bb99dd
Start 2024-01-09T02:45:06-08:00
Duration 6m26.593s
Status SUCCESS

Steps

Step Status Duration
python_install SUCCESS 1m35.087s
python_test SUCCESS 4m25.384s
python_format_check SUCCESS 2m40.796s

Details

starting build "5821c49c-10ab-43a3-a951-830f99bb99dd"

FETCHSOURCE
hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint: 
hint: 	git config --global init.defaultBranch <name>
hint: 
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint: 
hint: 	git branch -m <name>
Initialized empty Git repository in /workspace/.git/
From https://github.com/datacommonsorg/data
 * branch            7d8604f8bac116801ebf6ad4caf17a8942c2a94a -> FETCH_HEAD
Updating files:  27% (838/3032)
Updating files:  28% (849/3032)
Updating files:  29% (880/3032)
Updating files:  29% (894/3032)
Updating files:  30% (910/3032)
Updating files:  31% (940/3032)
Updating files:  32% (971/3032)
Updating files:  33% (1001/3032)
Updating files:  34% (1031/3032)
Updating files:  35% (1062/3032)
Updating files:  36% (1092/3032)
Updating files:  37% (1122/3032)
Updating files:  38% (1153/3032)
Updating files:  39% (1183/3032)
Updating files:  40% (1213/3032)
Updating files:  41% (1244/3032)
Updating files:  42% (1274/3032)
Updating files:  43% (1304/3032)
Updating files:  44% (1335/3032)
Updating files:  45% (1365/3032)
Updating files:  46% (1395/3032)
Updating files:  47% (1426/3032)
Updating files:  48% (1456/3032)
Updating files:  49% (1486/3032)
Updating files:  49% (1515/3032)
Updating files:  50% (1516/3032)
Updating files:  51% (1547/3032)
Updating files:  52% (1577/3032)
Updating files:  53% (1607/3032)
Updating files:  54% (1638/3032)
Updating files:  55% (1668/3032)
Updating files:  56% (1698/3032)
Updating files:  57% (1729/3032)
Updating files:  58% (1759/3032)
Updating files:  59% (1789/3032)
Updating files:  60% (1820/3032)
Updating files:  61% (1850/3032)
Updating files:  62% (1880/3032)
Updating files:  63% (1911/3032)
Updating files:  64% (1941/3032)
Updating files:  65% (1971/3032)
Updating files:  66% (2002/3032)
Updating files:  67% (2032/3032)
Updating files:  68% (2062/3032)
Updating files:  69% (2093/3032)
Updating files:  70% (2123/3032)
Updating files:  71% (2153/3032)
Updating files:  72% (2184/3032)
Updating files:  73% (2214/3032)
Updating files:  74% (2244/3032)
Updating files:  75% (2274/3032)
Updating files:  76% (2305/3032)
Updating files:  77% (2335/3032)
Updating files:  78% (2365/3032)
Updating files:  78% (2371/3032)
Updating files:  79% (2396/3032)
Updating files:  80% (2426/3032)
Updating files:  81% (2456/3032)
Updating files:  82% (2487/3032)
Updating files:  83% (2517/3032)
Updating files:  84% (2547/3032)
Updating files:  85% (2578/3032)
Updating files:  86% (2608/3032)
Updating files:  87% (2638/3032)
Updating files:  88% (2669/3032)
Updating files:  89% (2699/3032)
Updating files:  90% (2729/3032)
Updating files:  91% (2760/3032)
Updating files:  92% (2790/3032)
Updating files:  93% (2820/3032)
Updating files:  94% (2851/3032)
Updating files:  95% (2881/3032)
Updating files:  96% (2911/3032)
Updating files:  97% (2942/3032)
Updating files:  98% (2972/3032)
Updating files:  99% (3002/3032)
Updating files: 100% (3032/3032)
Updating files: 100% (3032/3032), done.
HEAD is now at 7d8604f remove properties not used in SVObs from tmcf,csv
BUILD
Starting Step #0 - "python_install"
Step #0 - "python_install": Pulling image: python:3.7
Step #0 - "python_install": 3.7: Pulling from library/python
Step #0 - "python_install": 167b8a53ca45: Pulling fs layer
Step #0 - "python_install": b47a222d28fa: Pulling fs layer
Step #0 - "python_install": debce5f9f3a9: Pulling fs layer
Step #0 - "python_install": 1d7ca7cd2e06: Pulling fs layer
Step #0 - "python_install": ff3119008f58: Pulling fs layer
Step #0 - "python_install": c2423a76a32b: Pulling fs layer
Step #0 - "python_install": e1c98ca4926a: Pulling fs layer
Step #0 - "python_install": 3b62c8e1d79b: Pulling fs layer
Step #0 - "python_install": 1d7ca7cd2e06: Waiting
Step #0 - "python_install": ff3119008f58: Waiting
Step #0 - "python_install": c2423a76a32b: Waiting
Step #0 - "python_install": e1c98ca4926a: Waiting
Step #0 - "python_install": 3b62c8e1d79b: Waiting
Step #0 - "python_install": b47a222d28fa: Verifying Checksum
Step #0 - "python_install": b47a222d28fa: Download complete
Step #0 - "python_install": 167b8a53ca45: Verifying Checksum
Step #0 - "python_install": 167b8a53ca45: Download complete
Step #0 - "python_install": debce5f9f3a9: Verifying Checksum
Step #0 - "python_install": debce5f9f3a9: Download complete
Step #0 - "python_install": ff3119008f58: Verifying Checksum
Step #0 - "python_install": ff3119008f58: Download complete
Step #0 - "python_install": c2423a76a32b: Verifying Checksum
Step #0 - "python_install": c2423a76a32b: Download complete
Step #0 - "python_install": e1c98ca4926a: Verifying Checksum
Step #0 - "python_install": e1c98ca4926a: Download complete
Step #0 - "python_install": 3b62c8e1d79b: Verifying Checksum
Step #0 - "python_install": 3b62c8e1d79b: Download complete
Step #0 - "python_install": 1d7ca7cd2e06: Verifying Checksum
Step #0 - "python_install": 1d7ca7cd2e06: Download complete
Step #0 - "python_install": 167b8a53ca45: Pull complete
Step #0 - "python_install": b47a222d28fa: Pull complete
Step #0 - "python_install": debce5f9f3a9: Pull complete
Step #0 - "python_install": 1d7ca7cd2e06: Pull complete
Step #0 - "python_install": ff3119008f58: Pull complete
Step #0 - "python_install": c2423a76a32b: Pull complete
Step #0 - "python_install": e1c98ca4926a: Pull complete
Step #0 - "python_install": 3b62c8e1d79b: Pull complete
Step #0 - "python_install": Digest: sha256:eedf63967cdb57d8214db38ce21f105003ed4e4d0358f02bedc057341bcf92a0
Step #0 - "python_install": Status: Downloaded newer image for python:3.7
Step #0 - "python_install": docker.io/library/python:3.7
Step #0 - "python_install": ### Installing Python requirements
Step #0 - "python_install": Installing Python requirements
Step #0 - "python_install":   DEPRECATION: ratelimit is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at https://github.com/pypa/pip/issues/8559
Step #0 - "python_install":   DEPRECATION: func-timeout is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at https://github.com/pypa/pip/issues/8559
Step #0 - "python_install":   DEPRECATION: frozendict is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at https://github.com/pypa/pip/issues/8559
Step #0 - "python_install":   DEPRECATION: easydict is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at https://github.com/pypa/pip/issues/8559
Step #0 - "python_install":   DEPRECATION: future is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at https://github.com/pypa/pip/issues/8559
Step #0 - "python_install":   DEPRECATION: rdp is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at https://github.com/pypa/pip/issues/8559
Step #0 - "python_install": 
Step #0 - "python_install": [notice] A new release of pip is available: 23.0.1 -> 23.3.2
Step #0 - "python_install": [notice] To update, run: pip install --upgrade pip
Finished Step #0 - "python_install"
Starting Step #1 - "python_test"
Starting Step #2 - "python_format_check"
Step #2 - "python_format_check": Already have image (with digest): python:3.7
Step #1 - "python_test": Already have image (with digest): python:3.7
Step #1 - "python_test": ### Running Python tests in util/
Step #2 - "python_format_check": ### Testing lint
Step #1 - "python_test": Installing Python requirements
Step #2 - "python_format_check": Installing Python requirements
Step #1 - "python_test": 
Step #1 - "python_test": [notice] A new release of pip is available: 23.0.1 -> 23.3.2
Step #1 - "python_test": [notice] To update, run: pip install --upgrade pip
Step #1 - "python_test": #### Testing Python code in util/
Step #2 - "python_format_check": 
Step #2 - "python_format_check": [notice] A new release of pip is available: 23.0.1 -> 23.3.2
Step #2 - "python_format_check": [notice] To update, run: pip install --upgrade pip
Step #2 - "python_format_check": #### Testing Python lint
Step #1 - "python_test": test_aggregate_dict (aggregation_util_test.AggregationUtilTest) ... ok
Step #1 - "python_test": test_aggregate_value (aggregation_util_test.AggregationUtilTest) ... ok
Step #1 - "python_test": test_config_map_with_override (config_map_test.TestConfigMap) ... ok
Step #1 - "python_test": test_load_config_file (config_map_test.TestConfigMap)
Step #1 - "python_test": Test loading of config dictionary from a file. ... ok
Step #1 - "python_test": test_set_config (config_map_test.TestConfigMap) ... ok
Step #1 - "python_test": test_update_config (config_map_test.TestConfigMap) ... ok
Step #1 - "python_test": test_add_counter (counters_test.TestCounters)
Step #1 - "python_test": Verify increment and decrement counters. ... Counters:
Step #1 - "python_test":                                        test_inputs =         10
Step #1 - "python_test":                          test_process_elapsed_time =       0.00
Step #1 - "python_test":                                     test_processed =          0
Step #1 - "python_test":                                    test_start_time =     207.54
Step #1 - "python_test": ok
Step #1 - "python_test": test_counter_dict (counters_test.TestCounters)
Step #1 - "python_test": Verify counter dict is shared across counters. ... Counters:
Step #1 - "python_test":                               process_elapsed_time =       0.00
Step #1 - "python_test":                                          processed =          0
Step #1 - "python_test":                                         start_time =     207.54
Step #1 - "python_test":                                           test_ctr =          1
Step #1 - "python_test": ok
Step #1 - "python_test": test_debug_counters (counters_test.TestCounters)
Step #1 - "python_test": Verify counters with debug string suffixes. ... Counters:
Step #1 - "python_test":                                       test3_inputs =         10
Step #1 - "python_test":                           test3_inputs_test-case-2 =         10
Step #1 - "python_test":                         test3_process_elapsed_time =       0.00
Step #1 - "python_test":                                    test3_processed =          0
Step #1 - "python_test":                                   test3_start_time =     207.54
Step #1 - "python_test": ok
Step #1 - "python_test": test_set_counter (counters_test.TestCounters)
Step #1 - "python_test": Verify set_counter overrides current value. ... Counters:
Step #1 - "python_test":                                        test2_lines =          1
Step #1 - "python_test":                                  test2_lines_file1 =          1
Step #1 - "python_test":                         test2_process_elapsed_time =       0.00
Step #1 - "python_test":                                    test2_processed =          0
Step #1 - "python_test":                                   test2_start_time =     207.54
Step #1 - "python_test": Counters:
Step #1 - "python_test":                                        test2_lines =         11
Step #1 - "python_test":                                  test2_lines_file1 =         11
Step #1 - "python_test":                         test2_process_elapsed_time =       0.00
Step #1 - "python_test":                                    test2_processed =          0
Step #1 - "python_test":                                   test2_start_time =     207.54
Step #1 - "python_test": ok
Step #1 - "python_test": test_show_counters (counters_test.TestCounters) ... Counters:
Step #1 - "python_test":                                     test-file-rows =        100
Step #1 - "python_test":                          test-process_elapsed_time =       0.00
Step #1 - "python_test":                        test-process_remaining_time = 1000000.00
Step #1 - "python_test":                                     test-read-rows =          0
Step #1 - "python_test":                                    test-start_time =     207.54
Step #1 - "python_test": Counters:
Step #1 - "python_test":                                     test-file-rows =        100
Step #1 - "python_test":                          test-process_elapsed_time =       0.00
Step #1 - "python_test":                        test-process_remaining_time =       0.00
Step #1 - "python_test":                               test-processing_rate =   53056.59
Step #1 - "python_test":                                     test-read-rows =         10
Step #1 - "python_test":                                    test-start_time =     207.54
Step #1 - "python_test": ok
Step #1 - "python_test": test_dc_api_batched_wrapper (dc_api_wrapper_test.TestDCAPIWrapper)
Step #1 - "python_test": Test DC API wrapper for batched calls. ... ok
Step #1 - "python_test": test_dc_api_is_defined_dcid (dc_api_wrapper_test.TestDCAPIWrapper)
Step #1 - "python_test": Test API wrapper for defined DCIDs. ... ok
Step #1 - "python_test": test_dc_api_wrapper (dc_api_wrapper_test.TestDCAPIWrapper)
Step #1 - "python_test": Test the wrapper for DC API. ... ok
Step #1 - "python_test": test_dc_get_node_property_values (dc_api_wrapper_test.TestDCAPIWrapper)
Step #1 - "python_test": Test API wrapper to get all property:values for a node. ... ok
Step #1 - "python_test": test_download_file (download_util_test.TestCounters) ... ok
Step #1 - "python_test": test_prefilled_url (download_util_test.TestCounters) ... ok
Step #1 - "python_test": test_request_url (download_util_test.TestCounters) ... ok
Step #1 - "python_test": test_read_write (file_util_test.FileIOTest) ... ok
Step #1 - "python_test": test_file_get_estimate_num_rows (file_util_test.FileUtilsTest) ... ok
Step #1 - "python_test": test_file_get_matching (file_util_test.FileUtilsTest) ... ok
Step #1 - "python_test": test_file_load_csv_dict (file_util_test.FileUtilsTest) ... ok
Step #1 - "python_test": test_file_type (file_util_test.FileUtilsTest) ... ok
Step #1 - "python_test": test_file_write_load_py_dict (file_util_test.FileUtilsTest) ... ok
Step #1 - "python_test": test_aa2 (latlng2place_mapsapi_test.Latlng2PlaceMapsAPITest) ... ok
Step #1 - "python_test": test_country (latlng2place_mapsapi_test.Latlng2PlaceMapsAPITest) ... ok
Step #1 - "python_test": test_main (latlng_recon_geojson_test.LatlngReconGeojsonTest) ... ok
Step #1 - "python_test": test_basic (latlng_recon_service_test.LatlngReconServiceTest) ... /usr/local/lib/python3.7/concurrent/futures/thread.py:57: ResourceWarning: unclosed <ssl.SSLSocket fd=10, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('192.168.10.3', 44988), raddr=('35.244.133.155', 443)>
Step #1 - "python_test":   result = self.fn(*self.args, **self.kwargs)
Step #1 - "python_test": ResourceWarning: Enable tracemalloc to get the object allocation traceback
Step #1 - "python_test": ok
Step #1 - "python_test": test_filter (latlng_recon_service_test.LatlngReconServiceTest) ... /usr/local/lib/python3.7/concurrent/futures/thread.py:57: ResourceWarning: unclosed <ssl.SSLSocket fd=10, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('192.168.10.3', 42218), raddr=('35.244.133.155', 443)>
Step #1 - "python_test":   result = self.fn(*self.args, **self.kwargs)
Step #1 - "python_test": ResourceWarning: Enable tracemalloc to get the object allocation traceback
Step #1 - "python_test": ok
Step #1 - "python_test": test_dict_list_to_mcf_str (mcf_dict_util_test.TestMCFDict) ... ok
Step #1 - "python_test": test_drop_nodes (mcf_dict_util_test.TestMCFDict) ... ok
Step #1 - "python_test": test_get_dcid_node (mcf_dict_util_test.TestMCFDict) ... ok
Step #1 - "python_test": test_mcf_dict_rename_namespace (mcf_dict_util_test.TestMCFDict) ... ok
Step #1 - "python_test": test_mcf_dict_rename_prop (mcf_dict_util_test.TestMCFDict) ... ok
Step #1 - "python_test": test_mcf_dict_rename_prop_value (mcf_dict_util_test.TestMCFDict) ... ok
Step #1 - "python_test": test_mcf_to_dict_list (mcf_dict_util_test.TestMCFDict) ... ok
Step #1 - "python_test": test_node_list_check_existence_dc (mcf_dict_util_test.TestMCFDict) ... ok
Step #1 - "python_test": test_node_list_check_existence_node_list (mcf_dict_util_test.TestMCFDict) ... ok
Step #1 - "python_test": test_example_usage (mcf_template_filler_test.MCFTemplateFillerTest) ... ok
Step #1 - "python_test": test_pop_and_2_obs_with_all_pv (mcf_template_filler_test.MCFTemplateFillerTest)
Step #1 - "python_test": Use separate templates for Pop Obs, and use Obs template repeatedly. ... ok
Step #1 - "python_test": test_pop_with_missing_req_pv (mcf_template_filler_test.MCFTemplateFillerTest) ... ok
Step #1 - "python_test": test_require_node_name (mcf_template_filler_test.MCFTemplateFillerTest) ... ok
Step #1 - "python_test": test_unified_pop_obs_with_missing_optional_pv (mcf_template_filler_test.MCFTemplateFillerTest) ... ok
Step #1 - "python_test": test_place_id_resolution_by_name (state_division_to_dcid_test.PlaceMapTest) ... ok
Step #1 - "python_test": test_boolean_naming (statvar_dcid_generator_test.TestStatVarDcidGenerator) ... ok
Step #1 - "python_test": test_double_underscore (statvar_dcid_generator_test.TestStatVarDcidGenerator) ... ok
Step #1 - "python_test": test_ignore_props (statvar_dcid_generator_test.TestStatVarDcidGenerator) ... ok
Step #1 - "python_test": test_legacy_mapping (statvar_dcid_generator_test.TestStatVarDcidGenerator) ... ok
Step #1 - "python_test": test_measured_property (statvar_dcid_generator_test.TestStatVarDcidGenerator) ... ok
Step #1 - "python_test": test_measurement_constraint_removal (statvar_dcid_generator_test.TestStatVarDcidGenerator) ... ok
Step #1 - "python_test": test_measurement_denominator (statvar_dcid_generator_test.TestStatVarDcidGenerator) ... ok
Step #1 - "python_test": test_measurement_qualifier (statvar_dcid_generator_test.TestStatVarDcidGenerator) ... ok
Step #1 - "python_test": test_naics_name_generation (statvar_dcid_generator_test.TestStatVarDcidGenerator) ... ok
Step #1 - "python_test": test_namespace_removal (statvar_dcid_generator_test.TestStatVarDcidGenerator) ... ok
Step #1 - "python_test": test_prepend_append_replace (statvar_dcid_generator_test.TestStatVarDcidGenerator) ... ok
Step #1 - "python_test": test_quantity_name_generation (statvar_dcid_generator_test.TestStatVarDcidGenerator) ... ok
Step #1 - "python_test": test_quantity_range_name_generation (statvar_dcid_generator_test.TestStatVarDcidGenerator) ... ok
Step #1 - "python_test": test_soc_map (statvar_dcid_generator_test.TestStatVarDcidGenerator) ... ok
Step #1 - "python_test": test_soc_name_generation (statvar_dcid_generator_test.TestStatVarDcidGenerator) ... ok
Step #1 - "python_test": test_sorted_constraints (statvar_dcid_generator_test.TestStatVarDcidGenerator) ... ok
Step #1 - "python_test": test_stat_type (statvar_dcid_generator_test.TestStatVarDcidGenerator) ... ok
Step #1 - "python_test": 
Step #1 - "python_test": ----------------------------------------------------------------------
Step #1 - "python_test": Ran 61 tests in 4.857s
Step #1 - "python_test": 
Step #1 - "python_test": OK
Step #1 - "python_test": ### Running Python tests in import-automation/executor
Step #1 - "python_test": Installing Python requirements
Step #1 - "python_test": 
Step #1 - "python_test": [notice] A new release of pip is available: 23.0.1 -> 23.3.2
Step #1 - "python_test": [notice] To update, run: pip install --upgrade pip
Step #1 - "python_test": #### Testing Python code in import-automation/executor
Step #1 - "python_test": test_appengine_job_request (test.cloud_scheduler_test.CloudSchedulerTest) ... ok
Step #1 - "python_test": test_http_job_request (test.cloud_scheduler_test.CloudSchedulerTest) ... ok
Step #1 - "python_test": test_levels (test.dashboard_api_test.DashboardAPITest)
Step #1 - "python_test": Tests that the convenient logging functions set the right ... ok
Step #1 - "python_test": test_log_helper (test.dashboard_api_test.DashboardAPITest) ... ok
Step #1 - "python_test": test_log_helper_http (test.dashboard_api_test.DashboardAPITest)
Step #1 - "python_test": Tests that an exception is thrown is the HTTP request fails. ... ok
Step #1 - "python_test": test_log_helper_id (test.dashboard_api_test.DashboardAPITest)
Step #1 - "python_test": Tests that at least one of run_id and attempt_id ... ok
Step #1 - "python_test": test_log_helper_time (test.dashboard_api_test.DashboardAPITest)
Step #1 - "python_test": Tests that time_logged is generated if not supplied. ... ok
Step #1 - "python_test": test_update_attempt (test.dashboard_api_test.DashboardAPITest) ... ok
Step #1 - "python_test": test_update_run (test.dashboard_api_test.DashboardAPITest) ... ok
Step #1 - "python_test": test.file_uploader_test (unittest.loader._FailedTest) ... ERROR
Step #1 - "python_test": test.github_api_test (unittest.loader._FailedTest) ... ERROR
Step #1 - "python_test": test_clean_time (test.import_executor_test.ImportExecutorTest) ... ok
Step #1 - "python_test": test_construct_process_message (test.import_executor_test.ImportExecutorTest) ... ok
Step #1 - "python_test": test_construct_process_message_no_output (test.import_executor_test.ImportExecutorTest)
Step #1 - "python_test": Tests that _construct_process_message does not append ... ok
Step #1 - "python_test": test_create_venv (test.import_executor_test.ImportExecutorTest) ... ok
Step #1 - "python_test": test_run_and_handle_exception (test.import_executor_test.ImportExecutorTest) ... ERROR:root:An unexpected exception was thrown
Step #1 - "python_test": Traceback (most recent call last):
Step #1 - "python_test":   File "/workspace/import-automation/executor/app/executor/import_executor.py", line 604, in run_and_handle_exception
Step #1 - "python_test":     return exec_func(*args)
Step #1 - "python_test":   File "/workspace/import-automation/executor/test/import_executor_test.py", line 64, in raise_exception
Step #1 - "python_test":     raise Exception
Step #1 - "python_test": Exception
Step #1 - "python_test": ok
Step #1 - "python_test": test_run_with_timeout (test.import_executor_test.ImportExecutorTest) ... ERROR:root:An unexpected exception was thrown: Command '['sleep', '5']' timed out after 0.1 seconds when running ['sleep', '5']: Traceback (most recent call last):
Step #1 - "python_test":   File "/workspace/import-automation/executor/app/executor/import_executor.py", line 712, in _run_with_timeout
Step #1 - "python_test":     env=env)
Step #1 - "python_test":   File "/usr/local/lib/python3.7/subprocess.py", line 490, in run
Step #1 - "python_test":     stdout, stderr = process.communicate(input, timeout=timeout)
Step #1 - "python_test":   File "/usr/local/lib/python3.7/subprocess.py", line 964, in communicate
Step #1 - "python_test":     stdout, stderr = self._communicate(input, endtime, timeout)
Step #1 - "python_test":   File "/usr/local/lib/python3.7/subprocess.py", line 1732, in _communicate
Step #1 - "python_test":     self._check_timeout(endtime, orig_timeout, stdout, stderr)
Step #1 - "python_test":   File "/usr/local/lib/python3.7/subprocess.py", line 1011, in _check_timeout
Step #1 - "python_test":     stderr=b''.join(stderr_seq) if stderr_seq else None)
Step #1 - "python_test": subprocess.TimeoutExpired: Command '['sleep', '5']' timed out after 0.1 seconds
Step #1 - "python_test": Traceback (most recent call last):
Step #1 - "python_test":   File "/workspace/import-automation/executor/app/executor/import_executor.py", line 712, in _run_with_timeout
Step #1 - "python_test":     env=env)
Step #1 - "python_test":   File "/usr/local/lib/python3.7/subprocess.py", line 490, in run
Step #1 - "python_test":     stdout, stderr = process.communicate(input, timeout=timeout)
Step #1 - "python_test":   File "/usr/local/lib/python3.7/subprocess.py", line 964, in communicate
Step #1 - "python_test":     stdout, stderr = self._communicate(input, endtime, timeout)
Step #1 - "python_test":   File "/usr/local/lib/python3.7/subprocess.py", line 1732, in _communicate
Step #1 - "python_test":     self._check_timeout(endtime, orig_timeout, stdout, stderr)
Step #1 - "python_test":   File "/usr/local/lib/python3.7/subprocess.py", line 1011, in _check_timeout
Step #1 - "python_test":     stderr=b''.join(stderr_seq) if stderr_seq else None)
Step #1 - "python_test": subprocess.TimeoutExpired: Command '['sleep', '5']' timed out after 0.1 seconds
Step #1 - "python_test": FAIL
Step #1 - "python_test": test_are_imports_finished (test.import_service_test.ImportServiceTest) ... ok
Step #1 - "python_test": test_block_on_import (test.import_service_test.ImportServiceTest) ... ok
Step #1 - "python_test": test_fix_input_path (test.import_service_test.ImportServiceTest) ... ok
Step #1 - "python_test": test_format_import_info (test.import_service_test.ImportServiceTest) ... ok
Step #1 - "python_test": test_get_fixed_absolute_import_name (test.import_service_test.ImportServiceTest) ... ok
Step #1 - "python_test": test_get_import_id (test.import_service_test.ImportServiceTest) ... ok
Step #1 - "python_test": test_smart_import (test.import_service_test.ImportServiceTest) ... ok
Step #1 - "python_test": test_absolute_import_name (test.import_target_test.ImportTargetTest) ... ok
Step #1 - "python_test": test_is_import_targetted_by_commit (test.import_target_test.ImportTargetTest) ... ok
Step #1 - "python_test": test_parse_commit_message_targets (test.import_target_test.ImportTargetTest) ... ok
Step #1 - "python_test": test.integration_test (unittest.loader._FailedTest) ... ERROR
Step #1 - "python_test": test_download_file (test.utils_test.AppUtilsTest)
Step #1 - "python_test": Response does not have a Content-Disposition header. ... ok
Step #1 - "python_test": test_download_file_timeout (test.utils_test.AppUtilsTest)
Step #1 - "python_test": Raises requests.Timeout exception. ... ok
Step #1 - "python_test": test_get_filename (test.utils_test.AppUtilsTest) ... ok
Step #1 - "python_test": test_get_filename_raise (test.utils_test.AppUtilsTest) ... ok
Step #1 - "python_test": test_pacific_time_to_datetime (test.utils_test.AppUtilsTest)
Step #1 - "python_test": Tests that the string returned by pacific_time can be converted to ... ok
Step #1 - "python_test": test_pacific_time_to_datetime_then_back (test.utils_test.AppUtilsTest)
Step #1 - "python_test": Tests that the string returned by pacific_time can be converted to ... ok
Step #1 - "python_test": test_compare_lines (test.utils_test.TestUtilsTest) ... ok
Step #1 - "python_test": test_import_spec_valid (test.validation_test.ValidationTest) ... ok
Step #1 - "python_test": test_import_spec_valid_fields_absent (test.validation_test.ValidationTest) ... ok
Step #1 - "python_test": test_import_spec_valid_script_not_exist (test.validation_test.ValidationTest) ... ok
Step #1 - "python_test": test_import_targets_valid_absolute_names (test.validation_test.ValidationTest) ... ok
Step #1 - "python_test": test_import_targets_valid_manifest_not_exist (test.validation_test.ValidationTest) ... ok
Step #1 - "python_test": test_import_targets_valid_name_not_exist (test.validation_test.ValidationTest) ... ok
Step #1 - "python_test": test_import_targets_valid_relative_names (test.validation_test.ValidationTest) ... ok
Step #1 - "python_test": test_import_targets_valid_relative_names_multiple_dirs (test.validation_test.ValidationTest) ... ok
Step #1 - "python_test": test_manifest_valid_fields_absent (test.validation_test.ValidationTest) ... ok
Step #1 - "python_test": 
Step #1 - "python_test": ======================================================================
Step #1 - "python_test": ERROR: test.file_uploader_test (unittest.loader._FailedTest)
Step #1 - "python_test": ----------------------------------------------------------------------
Step #1 - "python_test": ImportError: Failed to import test module: test.file_uploader_test
Step #1 - "python_test": Traceback (most recent call last):
Step #1 - "python_test":   File "/usr/local/lib/python3.7/unittest/loader.py", line 436, in _find_test_path
Step #1 - "python_test":     module = self._get_module_from_name(name)
Step #1 - "python_test":   File "/usr/local/lib/python3.7/unittest/loader.py", line 377, in _get_module_from_name
Step #1 - "python_test":     __import__(name)
Step #1 - "python_test":   File "/workspace/import-automation/executor/test/file_uploader_test.py", line 22, in <module>
Step #1 - "python_test":     from test import integration_test
Step #1 - "python_test":   File "/workspace/import-automation/executor/test/integration_test.py", line 27, in <module>
Step #1 - "python_test":     'github_repo_owner_username': os.environ['GITHUB_AUTH_USERNAME'],
Step #1 - "python_test":   File "/usr/local/lib/python3.7/os.py", line 681, in __getitem__
Step #1 - "python_test":     raise KeyError(key) from None
Step #1 - "python_test": KeyError: 'GITHUB_AUTH_USERNAME'
Step #1 - "python_test": 
Step #1 - "python_test": 
Step #1 - "python_test": ======================================================================
Step #1 - "python_test": ERROR: test.github_api_test (unittest.loader._FailedTest)
Step #1 - "python_test": ----------------------------------------------------------------------
Step #1 - "python_test": ImportError: Failed to import test module: test.github_api_test
Step #1 - "python_test": Traceback (most recent call last):
Step #1 - "python_test":   File "/usr/local/lib/python3.7/unittest/loader.py", line 436, in _find_test_path
Step #1 - "python_test":     module = self._get_module_from_name(name)
Step #1 - "python_test":   File "/usr/local/lib/python3.7/unittest/loader.py", line 377, in _get_module_from
...
[Logs truncated due to log size limitations. For full logs, see https://console.cloud.google.com/cloud-build/builds/5821c49c-10ab-43a3-a951-830f99bb99dd?project=879489846695.]
...
dcid', 'MetricTon')])
Step #1 - "python_test": [ 2024-01-09 10:49:41.620727 ]  Error:  warning_ignored_stat_var_duplicate_AL Invalid statVar {'typeOf': 'dcs:StatisticalVariable', 'measurementQualifier': 'dcs:Annual', 'populationType': 'dcs:Energy', 'statType': 'dcs:measuredValue'} for row OrderedDict([('Commodity Code', 'ZG'), ('Country or Area Code', '276'), ('Country or Area', 'Germany'), ('Transaction Code', '121'), ('Commodity - Transaction Code', 'ZG121'), ('Commodity - Transaction', 'Of which: biogasoline - Consumption by manufacturing, construction and non-fuel mining industry'), ('Year', '2018'), ('Unit', 'Metric tons,  thousand'), ('Quantity', '5000'), ('Quantity Footnotes', ''), ('_File', '/workspace/scripts/un/energy/test_data/un_energy_input.csv'), ('_Row', 2834), ('Country_dcid', 'dcs:country/DEU'), ('Unit_dcid', 'MetricTon')])
Step #1 - "python_test": [ 2024-01-09 10:49:41.620836 ]  Error:  warning_ignored_stat_var_duplicate_AL Invalid statVar {'typeOf': 'dcs:StatisticalVariable', 'measurementQualifier': 'dcs:Annual', 'populationType': 'dcs:Energy', 'statType': 'dcs:measuredValue'} for row OrderedDict([('Commodity Code', 'ZG'), ('Country or Area Code', '32'), ('Country or Area', 'Argentina'), ('Transaction Code', '022'), ('Commodity - Transaction Code', 'ZG022'), ('Commodity - Transaction', 'Of which: biogasoline - Receipts from other sources'), ('Year', '2019'), ('Unit', 'Metric tons,  thousand'), ('Quantity', '850000'), ('Quantity Footnotes', ''), ('_File', '/workspace/scripts/un/energy/test_data/un_energy_input.csv'), ('_Row', 2835), ('Country_dcid', 'dcs:country/ARG'), ('Unit_dcid', 'MetricTon')])
Step #1 - "python_test": [ 2024-01-09 10:49:41.620938 ]  Error:  warning_ignored_stat_var_duplicate_AL Invalid statVar {'typeOf': 'dcs:StatisticalVariable', 'measurementQualifier': 'dcs:Annual', 'populationType': 'dcs:Energy', 'statType': 'dcs:measuredValue'} for row OrderedDict([('Commodity Code', 'ZG'), ('Country or Area Code', '32'), ('Country or Area', 'Argentina'), ('Transaction Code', '06'), ('Commodity - Transaction Code', 'ZG06'), ('Commodity - Transaction', 'Of which: biogasoline - Stock changes'), ('Year', '2013'), ('Unit', 'Metric tons,  thousand'), ('Quantity', '-2000'), ('Quantity Footnotes', ''), ('_File', '/workspace/scripts/un/energy/test_data/un_energy_input.csv'), ('_Row', 2836), ('Country_dcid', 'dcs:country/ARG'), ('Unit_dcid', 'MetricTon')])
Step #1 - "python_test": [ 2024-01-09 10:49:41.621174 ]  Error:  warning_ignored_stat_var_duplicate_AL Invalid statVar {'typeOf': 'dcs:StatisticalVariable', 'measurementQualifier': 'dcs:Annual', 'populationType': 'dcs:Energy', 'statType': 'dcs:measuredValue'} for row OrderedDict([('Commodity Code', 'ZG'), ('Country or Area Code', '32'), ('Country or Area', 'Argentina'), ('Transaction Code', '1221'), ('Commodity - Transaction Code', 'ZG1221'), ('Commodity - Transaction', 'Of which: biogasoline - Consumption by road'), ('Year', '2019'), ('Unit', 'Metric tons,  thousand'), ('Quantity', '842000'), ('Quantity Footnotes', ''), ('_File', '/workspace/scripts/un/energy/test_data/un_energy_input.csv'), ('_Row', 2837), ('Country_dcid', 'dcs:country/ARG'), ('Unit_dcid', 'MetricTon')])
Step #1 - "python_test": [ 2024-01-09 10:49:41.621303 ]  Error:  warning_ignored_stat_var_duplicate_AL Invalid statVar {'typeOf': 'dcs:StatisticalVariable', 'measurementQualifier': 'dcs:Annual', 'populationType': 'dcs:Energy', 'statType': 'dcs:measuredValue'} for row OrderedDict([('Commodity Code', 'ZG'), ('Country or Area Code', '32'), ('Country or Area', 'Argentina'), ('Transaction Code', '122'), ('Commodity - Transaction Code', 'ZG122'), ('Commodity - Transaction', 'Of which: biogasoline - Consumption in transport'), ('Year', '2019'), ('Unit', 'Metric tons,  thousand'), ('Quantity', '842000'), ('Quantity Footnotes', ''), ('_File', '/workspace/scripts/un/energy/test_data/un_energy_input.csv'), ('_Row', 2838), ('Country_dcid', 'dcs:country/ARG'), ('Unit_dcid', 'MetricTon')])
Step #1 - "python_test": [ 2024-01-09 10:49:41.621415 ]  Error:  warning_ignored_stat_var_duplicate_AL Invalid statVar {'typeOf': 'dcs:StatisticalVariable', 'measurementQualifier': 'dcs:Annual', 'populationType': 'dcs:Energy', 'statType': 'dcs:measuredValue'} for row OrderedDict([('Commodity Code', 'ZG'), ('Country or Area Code', '32'), ('Country or Area', 'Argentina'), ('Transaction Code', '12'), ('Commodity - Transaction Code', 'ZG12'), ('Commodity - Transaction', 'Of which: biogasoline - Final energy consumption'), ('Year', '2019'), ('Unit', 'Metric tons,  thousand'), ('Quantity', '842000'), ('Quantity Footnotes', ''), ('_File', '/workspace/scripts/un/energy/test_data/un_energy_input.csv'), ('_Row', 2839), ('Country_dcid', 'dcs:country/ARG'), ('Unit_dcid', 'MetricTon')])
Step #1 - "python_test": [ 2024-01-09 10:49:41.621545 ]  Error:  warning_ignored_stat_var_duplicate_AL Invalid statVar {'typeOf': 'dcs:StatisticalVariable', 'measurementQualifier': 'dcs:Annual', 'populationType': 'dcs:Energy', 'statType': 'dcs:measuredValue'} for row OrderedDict([('Commodity Code', 'ZG'), ('Country or Area Code', '32'), ('Country or Area', 'Argentina'), ('Transaction Code', 'GA'), ('Commodity - Transaction Code', 'ZGGA'), ('Commodity - Transaction', 'Of which: biogasoline - Total energy supply'), ('Year', '2019'), ('Unit', 'Metric tons,  thousand'), ('Quantity', '850000'), ('Quantity Footnotes', ''), ('_File', '/workspace/scripts/un/energy/test_data/un_energy_input.csv'), ('_Row', 2840), ('Country_dcid', 'dcs:country/ARG'), ('Unit_dcid', 'MetricTon')])
Step #1 - "python_test": [ 2024-01-09 10:49:41.621650 ]  Error:  warning_ignored_stat_var_duplicate_AL Invalid statVar {'typeOf': 'dcs:StatisticalVariable', 'measurementQualifier': 'dcs:Annual', 'populationType': 'dcs:Energy', 'statType': 'dcs:measuredValue'} for row OrderedDict([('Commodity Code', 'ZG'), ('Country or Area Code', '32'), ('Country or Area', 'Argentina'), ('Transaction Code', 'NA'), ('Commodity - Transaction Code', 'ZGNA'), ('Commodity - Transaction', 'Of which: biogasoline - Final consumption'), ('Year', '2019'), ('Unit', 'Metric tons,  thousand'), ('Quantity', '842000'), ('Quantity Footnotes', ''), ('_File', '/workspace/scripts/un/energy/test_data/un_energy_input.csv'), ('_Row', 2841), ('Country_dcid', 'dcs:country/ARG'), ('Unit_dcid', 'MetricTon')])
Step #1 - "python_test": [ 2024-01-09 10:49:41.621752 ]  Error:  warning_ignored_stat_var_duplicate_AL Invalid statVar {'typeOf': 'dcs:StatisticalVariable', 'measurementQualifier': 'dcs:Annual', 'populationType': 'dcs:Energy', 'statType': 'dcs:measuredValue'} for row OrderedDict([('Commodity Code', 'ZG'), ('Country or Area Code', '40'), ('Country or Area', 'Austria'), ('Transaction Code', '03'), ('Commodity - Transaction Code', 'ZG03'), ('Commodity - Transaction', 'Of which: biogasoline - Imports'), ('Year', '2019'), ('Unit', 'Metric tons,  thousand'), ('Quantity', '34000'), ('Quantity Footnotes', ''), ('_File', '/workspace/scripts/un/energy/test_data/un_energy_input.csv'), ('_Row', 2842), ('Country_dcid', 'dcs:country/AUT'), ('Unit_dcid', 'MetricTon')])
Step #1 - "python_test": [ 2024-01-09 10:49:41.621896 ]  Error:  warning_ignored_stat_var_duplicate_AL Invalid statVar {'typeOf': 'dcs:StatisticalVariable', 'measurementQualifier': 'dcs:Annual', 'populationType': 'dcs:Energy', 'statType': 'dcs:measuredValue'} for row OrderedDict([('Commodity Code', 'ZG'), ('Country or Area Code', '40'), ('Country or Area', 'Austria'), ('Transaction Code', '04'), ('Commodity - Transaction Code', 'ZG04'), ('Commodity - Transaction', 'Of which: biogasoline - Exports'), ('Year', '2019'), ('Unit', 'Metric tons,  thousand'), ('Quantity', '49000'), ('Quantity Footnotes', ''), ('_File', '/workspace/scripts/un/energy/test_data/un_energy_input.csv'), ('_Row', 2843), ('Country_dcid', 'dcs:country/AUT'), ('Unit_dcid', 'MetricTon')])
Step #1 - "python_test": [ 2024-01-09 10:49:41.622002 ]  Error:  warning_ignored_stat_var_duplicate_AL Invalid statVar {'typeOf': 'dcs:StatisticalVariable', 'measurementQualifier': 'dcs:Annual', 'populationType': 'dcs:Energy', 'statType': 'dcs:measuredValue'} for row OrderedDict([('Commodity Code', 'ZG'), ('Country or Area Code', '604'), ('Country or Area', 'Peru'), ('Transaction Code', '07'), ('Commodity - Transaction Code', 'ZG07'), ('Commodity - Transaction', 'Of which: biogasoline - Transfers and recycled products'), ('Year', '2012'), ('Unit', 'Metric tons,  thousand'), ('Quantity', '-6000'), ('Quantity Footnotes', ''), ('_File', '/workspace/scripts/un/energy/test_data/un_energy_input.csv'), ('_Row', 2844), ('Country_dcid', 'dcs:country/PER'), ('Unit_dcid', 'MetricTon')])
Step #1 - "python_test": [ 2024-01-09 10:49:41.622098 ]  Error:  warning_ignored_stat_var_duplicate_AL Invalid statVar {'typeOf': 'dcs:StatisticalVariable', 'measurementQualifier': 'dcs:Annual', 'populationType': 'dcs:Energy', 'statType': 'dcs:measuredValue'} for row OrderedDict([('Commodity Code', 'ZG'), ('Country or Area Code', '604'), ('Country or Area', 'Peru'), ('Transaction Code', '1214e'), ('Commodity - Transaction Code', 'ZG1214e'), ('Commodity - Transaction', 'Of which: biogasoline - Consumption by mining and quarrying '), ('Year', '2018'), ('Unit', 'Metric tons,  thousand'), ('Quantity', '0'), ('Quantity Footnotes', ''), ('_File', '/workspace/scripts/un/energy/test_data/un_energy_input.csv'), ('_Row', 2845), ('Country_dcid', 'dcs:country/PER'), ('Unit_dcid', 'MetricTon')])
Step #1 - "python_test": [ 2024-01-09 10:49:41.622195 ]  Error:  warning_ignored_stat_var_duplicate_AL Invalid statVar {'typeOf': 'dcs:StatisticalVariable', 'measurementQualifier': 'dcs:Annual', 'populationType': 'dcs:Energy', 'statType': 'dcs:measuredValue'} for row OrderedDict([('Commodity Code', 'ZG'), ('Country or Area Code', '604'), ('Country or Area', 'Peru'), ('Transaction Code', '1214o'), ('Commodity - Transaction Code', 'ZG1214o'), ('Commodity - Transaction', 'Of which: biogasoline - Consumption not elsewhere specified (industry)'), ('Year', '2012'), ('Unit', 'Metric tons,  thousand'), ('Quantity', '1000'), ('Quantity Footnotes', ''), ('_File', '/workspace/scripts/un/energy/test_data/un_energy_input.csv'), ('_Row', 2846), ('Country_dcid', 'dcs:country/PER'), ('Unit_dcid', 'MetricTon')])
Step #1 - "python_test": [ 2024-01-09 10:49:41.622288 ]  Error:  warning_ignored_stat_var_duplicate_AL Invalid statVar {'typeOf': 'dcs:StatisticalVariable', 'measurementQualifier': 'dcs:Annual', 'populationType': 'dcs:Energy', 'statType': 'dcs:measuredValue'} for row OrderedDict([('Commodity Code', 'ZG'), ('Country or Area Code', '604'), ('Country or Area', 'Peru'), ('Transaction Code', '1225'), ('Commodity - Transaction Code', 'ZG1225'), ('Commodity - Transaction', 'Of which: biogasoline - Consumption not elsewhere specified (transport)'), ('Year', '2012'), ('Unit', 'Metric tons,  thousand'), ('Quantity', '12000'), ('Quantity Footnotes', ''), ('_File', '/workspace/scripts/un/energy/test_data/un_energy_input.csv'), ('_Row', 2847), ('Country_dcid', 'dcs:country/PER'), ('Unit_dcid', 'MetricTon')])
Step #1 - "python_test": [ 2024-01-09 10:49:41.622529 ]  Error:  warning_ignored_stat_var_duplicate_AL Invalid statVar {'typeOf': 'dcs:StatisticalVariable', 'measurementQualifier': 'dcs:Annual', 'populationType': 'dcs:Energy', 'statType': 'dcs:measuredValue'} for row OrderedDict([('Commodity Code', 'ZG'), ('Country or Area Code', '858'), ('Country or Area', 'Uruguay'), ('Transaction Code', '101'), ('Commodity - Transaction Code', 'ZG101'), ('Commodity - Transaction', 'Of which: biogasoline - Losses'), ('Year', '2013'), ('Unit', 'Metric tons,  thousand'), ('Quantity', '600.0'), ('Quantity Footnotes', ''), ('_File', '/workspace/scripts/un/energy/test_data/un_energy_input.csv'), ('_Row', 2848), ('Country_dcid', 'dcs:country/URY'), ('Unit_dcid', 'MetricTon')])
Step #1 - "python_test": [ 2024-01-09 10:49:41.622672 ]  Error:  warning_ignored_stat_var_duplicate_BJ Invalid statVar {'typeOf': 'dcs:StatisticalVariable', 'measurementQualifier': 'dcs:Annual', 'populationType': 'dcs:Energy', 'statType': 'dcs:measuredValue'} for row OrderedDict([('Commodity Code', 'ZJ'), ('Country or Area Code', '246'), ('Country or Area', 'Finland'), ('Transaction Code', '04'), ('Commodity - Transaction Code', 'ZJ04'), ('Commodity - Transaction', 'Of which: bio jet kerosene - Exports'), ('Year', '2015'), ('Unit', 'Metric tons,  thousand'), ('Quantity', '1000'), ('Quantity Footnotes', ''), ('_File', '/workspace/scripts/un/energy/test_data/un_energy_input.csv'), ('_Row', 2849), ('Country_dcid', 'dcs:country/FIN'), ('Unit_dcid', 'MetricTon')])
Step #1 - "python_test": [ 2024-01-09 10:49:41.622808 ]  Error:  warning_ignored_stat_var_duplicate_BJ Invalid statVar {'typeOf': 'dcs:StatisticalVariable', 'measurementQualifier': 'dcs:Annual', 'populationType': 'dcs:Energy', 'statType': 'dcs:measuredValue'} for row OrderedDict([('Commodity Code', 'ZJ'), ('Country or Area Code', '246'), ('Country or Area', 'Finland'), ('Transaction Code', '07'), ('Commodity - Transaction Code', 'ZJ07'), ('Commodity - Transaction', 'Of which: bio jet kerosene - Transfers and recycled products'), ('Year', '2015'), ('Unit', 'Metric tons,  thousand'), ('Quantity', '-1000'), ('Quantity Footnotes', ''), ('_File', '/workspace/scripts/un/energy/test_data/un_energy_input.csv'), ('_Row', 2850), ('Country_dcid', 'dcs:country/FIN'), ('Unit_dcid', 'MetricTon')])
Step #1 - "python_test": [ 2024-01-09 10:49:41.622916 ]  Error:  warning_ignored_stat_var_duplicate_BJ Invalid statVar {'typeOf': 'dcs:StatisticalVariable', 'measurementQualifier': 'dcs:Annual', 'populationType': 'dcs:Energy', 'statType': 'dcs:measuredValue'} for row OrderedDict([('Commodity Code', 'ZJ'), ('Country or Area Code', '246'), ('Country or Area', 'Finland'), ('Transaction Code', 'GA'), ('Commodity - Transaction Code', 'ZJGA'), ('Commodity - Transaction', 'Of which: bio jet kerosene - Total energy supply'), ('Year', '2015'), ('Unit', 'Metric tons,  thousand'), ('Quantity', '-1000'), ('Quantity Footnotes', ''), ('_File', '/workspace/scripts/un/energy/test_data/un_energy_input.csv'), ('_Row', 2851), ('Country_dcid', 'dcs:country/FIN'), ('Unit_dcid', 'MetricTon')])
Step #1 - "python_test": ok
Step #1 - "python_test": test_process_containment (un.sdg.geography_test.GeographyTest) ... ok
Step #1 - "python_test": test_should_include_containment (un.sdg.geography_test.GeographyTest) ... ok
Step #1 - "python_test": test_write_place_mappings (un.sdg.geography_test.GeographyTest) ... ok
Step #1 - "python_test": test_write_un_containment (un.sdg.geography_test.GeographyTest) ... ok
Step #1 - "python_test": test_write_un_places (un.sdg.geography_test.GeographyTest) ... ok
Step #1 - "python_test": test_drop_null (un.sdg.process_test.ProcessTest) ... ok
Step #1 - "python_test": test_drop_special (un.sdg.process_test.ProcessTest) ... ok
Step #1 - "python_test": test_fix_encoding (un.sdg.process_test.ProcessTest) ... ok
Step #1 - "python_test": test_get_geography (un.sdg.process_test.ProcessTest) ... ok
Step #1 - "python_test": test_get_measurement_method (un.sdg.process_test.ProcessTest) ... ok
Step #1 - "python_test": test_process (un.sdg.process_test.ProcessTest) ... ok
Step #1 - "python_test": test_curate_pvs (un.sdg.util_test.UtilTest) ... ok
Step #1 - "python_test": test_format_description (un.sdg.util_test.UtilTest) ... ok
Step #1 - "python_test": test_format_property (un.sdg.util_test.UtilTest) ... ok
Step #1 - "python_test": test_format_title (un.sdg.util_test.UtilTest) ... ok
Step #1 - "python_test": test_format_variable_code (un.sdg.util_test.UtilTest) ... ok
Step #1 - "python_test": test_format_variable_description (un.sdg.util_test.UtilTest) ... ok
Step #1 - "python_test": test_is_float (un.sdg.util_test.UtilTest) ... ok
Step #1 - "python_test": test_is_valid (un.sdg.util_test.UtilTest) ... ok
Step #1 - "python_test": test_data_processing_small (us_bea.states_gdp.import_data_test.USStateQuarterlyGDPImportTest)
Step #1 - "python_test": Tests end-to-end data cleaning on a small example. ... ok
Step #1 - "python_test": test_data_processing_tiny (us_bea.states_gdp.import_data_test.USStateQuarterlyGDPImportTest)
Step #1 - "python_test": Tests end-to-end data cleaning on a tiny example. ... ok
Step #1 - "python_test": test_date_converter (us_bea.states_gdp.import_data_test.USStateQuarterlyGDPImportTest)
Step #1 - "python_test": Tests the date converter function used to process raw data. ... ok
Step #1 - "python_test": test_geoid_converter (us_bea.states_gdp.import_data_test.USStateQuarterlyGDPImportTest)
Step #1 - "python_test": Tests the geoid converter function used to process raw data. ... ok
Step #1 - "python_test": test_data_processing_tiny (us_bea.states_gdp.import_data_test.USStateQuarterlyPerIndustryImportTest)
Step #1 - "python_test": Tests end-to-end data cleaning on a tiny example. ... ok
Step #1 - "python_test": test_industry_class (us_bea.states_gdp.import_data_test.USStateQuarterlyPerIndustryImportTest)
Step #1 - "python_test": Tests industry class converter function that cleans out empty ... ok
Step #1 - "python_test": test_value_converter (us_bea.states_gdp.import_data_test.USStateQuarterlyPerIndustryImportTest)
Step #1 - "python_test": Tests value converter function that cleans out empty datapoints. ... ok
Step #1 - "python_test": test_preprocess (us_bjs.nps.import_data_test.TestPreprocess) ... /workspace/scripts/us_bjs/nps/preprocess_data.py:21: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead.  To get a de-fragmented frame, use `newframe = frame.copy()`
Step #1 - "python_test":   df["PVINF_Temp"] = df["PVINF"].apply(convert_nan_for_calculation)
Step #1 - "python_test": /workspace/scripts/us_bjs/nps/preprocess_data.py:22: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead.  To get a de-fragmented frame, use `newframe = frame.copy()`
Step #1 - "python_test":   df["PVOTHF_Temp"] = df["PVOTHF"].apply(convert_nan_for_calculation)
Step #1 - "python_test": /workspace/scripts/us_bjs/nps/preprocess_data.py:23: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead.  To get a de-fragmented frame, use `newframe = frame.copy()`
Step #1 - "python_test":   df["PVINM_Temp"] = df["PVINM"].apply(convert_nan_for_calculation)
Step #1 - "python_test": /workspace/scripts/us_bjs/nps/preprocess_data.py:24: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead.  To get a de-fragmented frame, use `newframe = frame.copy()`
Step #1 - "python_test":   df["PVOTHM_Temp"] = df["PVOTHM"].apply(convert_nan_for_calculation)
Step #1 - "python_test": ok
Step #1 - "python_test": test_filter_series (us_bls.cpi.generate_csv_mcf_test.TestGenerateCSVMCF) ... ok
Step #1 - "python_test": test_generate_statvar (us_bls.cpi.generate_csv_mcf_test.TestGenerateCSVMCF) ... ok
Step #1 - "python_test": test_invalid_series_id (us_bls.cpi.generate_csv_mcf_test.TestGenerateCSVMCF) ... ok
Step #1 - "python_test": test_parse_series_id (us_bls.cpi.generate_csv_mcf_test.TestGenerateCSVMCF) ... ok
Step #1 - "python_test": test_valid_series_id (us_bls.cpi.generate_csv_mcf_test.TestGenerateCSVMCF) ... ok
Step #1 - "python_test": test_clean_cdc_places_data (us_cdc.500_places.parse_cdc_places_test.TestParseCDCPlaces)
Step #1 - "python_test": Tests the clean_cdc_places_data function. ... ok
Step #1 - "python_test": test_brfss_asthma_extracted_data (us_cdc.brfss_aggregated_asthma_2016_2018.brfss_asthma_import_test.ProcessTest) ... ok
Step #1 - "python_test": test_clean_air_quality_data (us_cdc.environmental_health_toxicology.parse_air_quality_test.TestParseAirQuality)
Step #1 - "python_test": Tests the clean_air_quality_data function. ... ok
Step #1 - "python_test": test_clean_precipitation_data (us_cdc.environmental_health_toxicology.parse_precipitation_index_test.TestParsePrecipitationData)
Step #1 - "python_test": Tests the clean_precipitation_data function. ... ok
Step #1 - "python_test": test_spec_generation (us_census.acs5yr.subject_tables.common.acs_spec_generator_test.TestSpecGenerator) ... /workspace/scripts/us_census/acs5yr/subject_tables/common/datacommons_api_wrappers/datacommons_wrappers.py:147: ResourceWarning: unclosed file <_io.TextIOWrapper name='/workspace/scripts/us_census/acs5yr/subject_tables/common/datacommons_api_wrappers/prefetched_outputs/Person_dc_props.json' mode='r' encoding='UTF-8'>
Step #1 - "python_test":   open(os.path.join(cache_path, f'{dcid}_dc_props.json'), 'r'))
Step #1 - "python_test": ResourceWarning: Enable tracemalloc to get the object allocation traceback
Step #1 - "python_test": /workspace/scripts/us_census/acs5yr/subject_tables/common/datacommons_api_wrappers/datacommons_wrappers.py:172: ResourceWarning: unclosed file <_io.TextIOWrapper name='/workspace/scripts/us_census/acs5yr/subject_tables/common/datacommons_api_wrappers/prefetched_outputs/Person_dc_props_types.json' mode='r' encoding='UTF-8'>
Step #1 - "python_test":   open(os.path.join(cache_path, f'{dcid}_dc_props_types.json'), 'r'))
Step #1 - "python_test": ResourceWarning: Enable tracemalloc to get the object allocation traceback
Step #1 - "python_test": /workspace/scripts/us_census/acs5yr/subject_tables/common/datacommons_api_wrappers/datacommons_wrappers.py:200: ResourceWarning: unclosed file <_io.TextIOWrapper name='/workspace/scripts/us_census/acs5yr/subject_tables/common/datacommons_api_wrappers/prefetched_outputs/Person_dc_props_enum_values.json' mode='r' encoding='UTF-8'>
Step #1 - "python_test":   'r'))
Step #1 - "python_test": ResourceWarning: Enable tracemalloc to get the object allocation traceback
Step #1 - "python_test": ok
Step #1 - "python_test": test_find_columns_with_no_properties (us_census.acs5yr.subject_tables.common.acs_spec_validator_test.TestSpecValidator) ... ok
Step #1 - "python_test": test_find_extra_inferred_properties (us_census.acs5yr.subject_tables.common.acs_spec_validator_test.TestSpecValidator) ... ok
Step #1 - "python_test": test_find_extra_tokens (us_census.acs5yr.subject_tables.common.acs_spec_validator_test.TestSpecValidator) ... ok
Step #1 - "python_test": test_find_ignore_conflicts (us_census.acs5yr.subject_tables.common.acs_spec_validator_test.TestSpecValidator) ... ok
Step #1 - "python_test": test_find_missing_denominator_total_column (us_census.acs5yr.subject_tables.common.acs_spec_validator_test.TestSpecValidator) ... ok
Step #1 - "python_test": test_find_missing_denominators (us_census.acs5yr.subject_tables.common.acs_spec_validator_test.TestSpecValidator) ... ok
Step #1 - "python_test": test_find_multiple_measurement (us_census.acs5yr.subject_tables.common.acs_spec_validator_test.TestSpecValidator) ... ok
Step #1 - "python_test": test_find_multiple_population (us_census.acs5yr.subject_tables.common.acs_spec_validator_test.TestSpecValidator) ... ok
Step #1 - "python_test": test_find_repeating_denominators (us_census.acs5yr.subject_tables.common.acs_spec_validator_test.TestSpecValidator) ... ok
Step #1 - "python_test": test_check_column_map (us_census.acs5yr.subject_tables.common.column_map_validator_test.TestColumnMapValidator) ... ok
Step #1 - "python_test": test_column_ignore (us_census.acs5yr.subject_tables.common.common_util_test.TestCommonUtil) ... ok
Step #1 - "python_test": test_columns_from_CSVreader (us_census.acs5yr.subject_tables.common.common_util_test.TestCommonUtil) ... ok
Step #1 - "python_test": test_find_missing_tokens (us_census.acs5yr.subject_tables.common.common_util_test.TestCommonUtil) ... ok
Step #1 - "python_test": test_get_spec_token_list (us_census.acs5yr.subject_tables.common.common_util_test.TestCommonUtil) ... ok
Step #1 - "python_test": test_token_in_list (us_census.acs5yr.subject_tables.common.common_util_test.TestCommonUtil) ... ok
Step #1 - "python_test": test_tokens_from_column_list (us_census.acs5yr.subject_tables.common.common_util_test.TestCommonUtil) ... ok
Step #1 - "python_test": test_csv_file_input (us_census.acs5yr.subject_tables.common.data_loader_test.DataLoaderBaseTest) ... ok
Finished Step #2 - "python_format_check"
Step #1 - "python_test": test_zip_file_input (us_census.acs5yr.subject_tables.common.data_loader_test.DataLoaderBaseTest) ... ok
Step #1 - "python_test": test_generating_column_map_from_csv (us_census.acs5yr.subject_tables.common.generate_col_map_test.GenerateColMapTest) ... ok
Step #1 - "python_test": test_generating_column_map_from_zip (us_census.acs5yr.subject_tables.common.generate_col_map_test.GenerateColMapTest) ... ok
Step #1 - "python_test": test_geoIds_at_all_summary_levels (us_census.acs5yr.subject_tables.common.resolve_geo_id_test.ResolveCensusGeoIdTest) ... ok
Step #1 - "python_test": test_convert_column_to_stat_var (us_census.acs5yr.subject_tables.s2201.process_test.ProcessTest) ... ok
Step #1 - "python_test": test_create_csv (us_census.acs5yr.subject_tables.s2201.process_test.ProcessTest) ... ok
Step #1 - "python_test": test_create_tmcf (us_census.acs5yr.subject_tables.s2201.process_test.ProcessTest) ... ok
Step #1 - "python_test": test_csv_mcf_column_map (us_census.acs5yr.subject_tables.subject_table_test.TestSubjectTable) ... ok
Step #1 - "python_test": test_e2e (us_census.decennial.process_test.ProcessTest) ... ok
Step #1 - "python_test": test_bad_tmcf_variable_measured_two_equals_exception (us_census.enhanced_tmcf.process_etmcf_test.Process_ETMCF_Test) ... ok
Step #1 - "python_test": test_bad_tmcf_variable_measured_two_question_marks_exception (us_census.enhanced_tmcf.process_etmcf_test.Process_ETMCF_Test) ... ok
Step #1 - "python_test": test_csv_file_not_found_exception (us_census.enhanced_tmcf.process_etmcf_test.Process_ETMCF_Test) ... ok
Step #1 - "python_test": test_process_enhanced_tmcf_medium_success (us_census.enhanced_tmcf.process_etmcf_test.Process_ETMCF_Test) ... ok
Step #1 - "python_test": test_simple_opaque_success (us_census.enhanced_tmcf.process_etmcf_test.Process_ETMCF_Test) ... ok
Step #1 - "python_test": test_simple_success (us_census.enhanced_tmcf.process_etmcf_test.Process_ETMCF_Test) ... ok
Step #1 - "python_test": test_tmcf_file_not_found_exception (us_census.enhanced_tmcf.process_etmcf_test.Process_ETMCF_Test) ... ok
Step #1 - "python_test": test_process (us_eia.eia_860.main_test.TestProcess) ... ok
Step #1 - "python_test": test_cleanup_name (us_eia.opendata.process.common_test.TestProcess) ... ok
Step #1 - "python_test": test_process (us_eia.opendata.process.common_test.TestProcess) ... ok
Step #1 - "python_test": test_write_csv_county (us_epa.airdata.air_quality_aggregate_test.TestCriteriaGasesTest) ... ok
Step #1 - "python_test": test_write_csv_csba (us_epa.airdata.air_quality_aggregate_test.TestCriteriaGasesTest) ... ok
Step #1 - "python_test": test_write_tmcf (us_epa.airdata.air_quality_aggregate_test.TestCriteriaGasesTest) ... ok
Step #1 - "python_test": test_write_csv (us_epa.airdata.air_quality_test.TestCriteriaGasesTest) ... ok
Step #1 - "python_test": test_write_tmcf (us_epa.airdata.air_quality_test.TestCriteriaGasesTest) ... ok
Step #1 - "python_test": test_write_csv (us_epa.ejscreen.ejscreen_test.TestEjscreen) ... ok
Step #1 - "python_test": test_e2e (us_epa.facility.process_facility_test.ProcessTest) ... ok
Step #1 - "python_test": test_name_to_dcid (us_epa.ghgrp.gas_test.GasTest) ... ok
Step #1 - "python_test": test_process_direct_emitters (us_epa.ghgrp.process_test.ProcessTest) ... ok
Step #1 - "python_test": test_name_to_dcid (us_epa.ghgrp.sources_test.SourcesTest) ... ok
Step #1 - "python_test": test_parent_companies_e2e (us_epa.parent_company.process_parent_company_test.ProcessTest) ... ok
Step #1 - "python_test": test_svobs_e2e (us_epa.parent_company.process_parent_company_test.ProcessTest) ... ok
Step #1 - "python_test": test_e2e_superfund_funding_status (us_epa.superfund.site_and_funding_status.process_sites_fundingStatus_test.ProcessTest) ... ok
Step #1 - "python_test": test_e2e_superfund_sites (us_epa.superfund.site_and_funding_status.process_sites_test.ProcessTest) ... /workspace/scripts/us_epa/superfund/site_and_funding_status/process_sites.py:144: FutureWarning: The default value of regex will change from True to False in a future version.
Step #1 - "python_test":   site_csv['siteName'] = site_csv['siteName'].str.replace(', Inc.', ' Inc.')
Step #1 - "python_test": ok
Step #1 - "python_test": test_e2e_superfund_site_contamination (us_epa.superfund.site_contamination.process_sites_contamination_test.ProcessTest) ... ok
Step #1 - "python_test": test_e2e (us_epa.superfund.site_hazards.process_sites_hazards_test.ProcessTest) ... ok
Step #1 - "python_test": test_e2e (us_epa.superfund.sites.measurement_sites.generate_measurement_site_mcf_test.ProcessTest) ... ok
Step #1 - "python_test": test_e2e (us_epa.superfund.sites.tar_creek.process_contaminants_test.ProcessTest) ... ok
Step #1 - "python_test": test_main (us_fema.national_risk_index.generate_schema_and_tmcf_test.ProcessFemaNriFileTest) ... ok
Step #1 - "python_test": test_county_missing_trailing_zero (us_fema.national_risk_index.process_data_test.FormatGeoIDTest) ... ok
Step #1 - "python_test": test_county_no_change_needed (us_fema.national_risk_index.process_data_test.FormatGeoIDTest) ... ok
Step #1 - "python_test": test_tract_missing_trailing_zero (us_fema.national_risk_index.process_data_test.FormatGeoIDTest) ... ok
Step #1 - "python_test": test_tract_no_change_needed (us_fema.national_risk_index.process_data_test.FormatGeoIDTest) ... ok
Step #1 - "python_test": test_process_county_file (us_fema.national_risk_index.process_data_test.ProcessFemaNriFileTest) ... ok
Step #1 - "python_test": test_process_tract_file (us_fema.national_risk_index.process_data_test.ProcessFemaNriFileTest) ... ok
Step #1 - "python_test": test_preprocess (us_gs.earthquake.preprocess_test.USGSEarthquakePreprocessTest) ... ok
Step #1 - "python_test": test_compute_150 (us_hud.income.process_test.ProcessTest) ... ok
Step #1 - "python_test": test_get_url (us_hud.income.process_test.ProcessTest) ... ok
Step #1 - "python_test": test_process (us_hud.income.process_test.ProcessTest) ... ok
Step #1 - "python_test": test_output_mcf (world_bank.boundaries.country_boundaries_mcf_generator_test.CountyBoundariesMcfGeneratorTest) ... ok
Step #1 - "python_test": 
Step #1 - "python_test": ----------------------------------------------------------------------
Step #1 - "python_test": Ran 172 tests in 205.903s
Step #1 - "python_test": 
Step #1 - "python_test": OK
Finished Step #1 - "python_test"
PUSH
DONE

Build Log: https://console.cloud.google.com/cloud-build/builds/5821c49c-10ab-43a3-a951-830f99bb99dd?project=879489846695