Options arg in read_config_user_file now optional (#709)

* Remove utils section (#697) Moved the last script from esmvalcore/utils to ESMValTool * Fixed bug in time weights calculation (#695) * Fixed calculation of time weights * Fixed failing FLAKE8 test * Added more test for time weighting and fixed cube dimensions in test * Avoid pytest version that crashes (#707) * Suggested Documentation changes (#690) Update documentation on relative diagnostics paths and preprocessor order. Co-authored-by: Bouwe Andela <[email protected]> * Options arg in read_config_user_file now optional * Fix codacy warning Co-authored-by: Bouwe Andela <[email protected]> Co-authored-by: Manuel Schlund <[email protected]> Co-authored-by: Steve Smith <[email protected]> Co-authored-by: Bouwe Andela <[email protected]>
ESMValGroup · Jul 13, 2020 · eb78c24 · eb78c24
1 parent e67f5b9
commit eb78c24
Show file tree

Hide file tree

Showing 9 changed files with 16 additions and 116 deletions.
diff --git a/doc/gensidebar.py b/doc/gensidebar.py
@@ -76,7 +76,6 @@ def _header(project, text):
     _write("esmvalcore", "Diagnostic script interfaces", "interfaces")
     _write("esmvalcore", "Development", "develop/index")
     _write("esmvalcore", "Contributing", "contributing")
-    _write("esmvalcore", "Utilities", "utils")
     _write("esmvalcore", "ESMValCore API Reference", "api/esmvalcore")
     _write("esmvalcore", "Changelog", "changelog")
     _endl()

diff --git a/doc/quickstart/find_data.rst b/doc/quickstart/find_data.rst
@@ -79,7 +79,7 @@ structures.
 
 Explaining ``config-user/drs: CMIP5:`` or ``config-user/drs: CMIP6:``
 ---------------------------------------------------------------------
-Whreas ESMValTool will **always** use the CMOR standard for file naming (please
+Whereas ESMValTool will **always** use the CMOR standard for file naming (please
 refer above), by setting the ``drs`` parameter the user tells the tool what
 type of root paths they need the data from, e.g.:
 

diff --git a/doc/recipe/overview.rst b/doc/recipe/overview.rst
@@ -179,6 +179,11 @@ arguments):
    data, applying CMOR checks and fixes (:ref:`CMOR check and dataset-specific
    fixes`) and saving the data to disk.
 
+Preprocessor operations will be applied using the default order
+as listed in :ref:`preprocessor_functions`.
+Preprocessor tasks can be set to run in the order they are listed in the recipe
+by adding ``custom_order: true`` to the preprocessor definition.
+
 .. _Diagnostics:
 
 Recipe section: ``diagnostics``
@@ -237,6 +242,10 @@ The path to the script provided in the ``script`` option should be
 either the absolute path to the script, or the path relative to the
 ``esmvaltool/diag_scripts`` directory.
 
+Depending on the installation configuration, you may get an error of
+"file does not exist" when the system tries to run the diagnostic script
+using relative paths. If this happens, use an absolute path instead.
+
 Ancestor tasks
 --------------
 Some tasks require the result of other tasks to be ready before they can start,

diff --git a/doc/recipe/preprocessor.rst b/doc/recipe/preprocessor.rst
@@ -59,7 +59,8 @@ Overview
 The ESMValTool preprocessor can be used to perform a broad range of operations
 on the input data before diagnostics or metrics are applied. The preprocessor
 performs these operations in a centralized, documented and efficient way, thus
-reducing the data processing load on the diagnostics side.
+reducing the data processing load on the diagnostics side.  For an overview of
+the preprocessor structure see the :ref:`Preprocessors`.
 
 Each of the preprocessor operations is written in a dedicated python module and
 all of them receive and return an Iris `cube

diff --git a/doc/utils.rst b/doc/utils.rst
diff --git a/esmvalcore/_config.py b/esmvalcore/_config.py
@@ -31,7 +31,7 @@ def find_diagnostics():
 DIAGNOSTICS_PATH = find_diagnostics()
 
 
-def read_config_user_file(config_file, folder_name, options):
+def read_config_user_file(config_file, folder_name, options=None):
     """Read config user file and store settings in a dictionary."""
     config_file = os.path.abspath(
         os.path.expandvars(os.path.expanduser(config_file)))
@@ -42,6 +42,8 @@ def read_config_user_file(config_file, folder_name, options):
     with open(config_file, 'r') as file:
         cfg = yaml.safe_load(file)
 
+    if options is None:
+        options = dict()
     for key, value in options.items():
         cfg[key] = value
 

diff --git a/esmvalcore/utils/__init__.py b/esmvalcore/utils/__init__.py
diff --git a/esmvalcore/utils/draft_release_notes.py b/esmvalcore/utils/draft_release_notes.py
diff --git a/setup.py b/setup.py
@@ -48,7 +48,7 @@
     # Test dependencies
     # Execute 'python setup.py test' to run tests
     'test': [
-        'pytest>=3.9',
+        'pytest>=3.9,!=6.0.0rc1',
         'pytest-cov',
         'pytest-env',
         'pytest-flake8',