Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiple pytests fail for ERA5 and SARAH cutouts merging (update_feature_test). #147

Closed
euronion opened this issue May 27, 2021 · 0 comments · Fixed by #152
Closed

Multiple pytests fail for ERA5 and SARAH cutouts merging (update_feature_test). #147

euronion opened this issue May 27, 2021 · 0 comments · Fixed by #152

Comments

@euronion
Copy link
Collaborator

Description

pytest fails for tests in test/test_preparation_and_conversion.py (ERA5 and SARAH).

Expected Behavior

No fail.

Actual Behavior

See error message.

Error Message

test/test_preparation_and_conversion.py:301: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
test/test_preparation_and_conversion.py:39: in update_feature_test
    red.prepare('influx', overwrite=True)
atlite/data.py:94: in wrapper
    res = func(*args, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

cutout = <Cutout "era5">
 x = -4.00 ⟷ 1.50, dx = 0.25
 y = 56.00 ⟷ 61.00, dy = 0.25
 time = 2013-01-01 ⟷ 2013-01-01, dt = H
 module = era5
 prepared_features = ['height', 'wind', 'influx', 'temperature', 'runoff'], features = array(['influx'], dtype='<U6'), tmpdir = '/tmp/tmpthpj0_pq', overwrite = True

    @maybe_remove_tmpdir
    def cutout_prepare(cutout, features=None, tmpdir=None, overwrite=False):
        """
        Prepare all or a selection of features in a cutout.
    
        This function loads the feature data of a cutout, e.g. influx or runoff.
        When not specifying the `feature` argument, all available features will be
        loaded. The function compares the variables which are already included in
        the cutout with the available variables of the modules specified by the
        cutout. It detects missing variables and stores them into the netcdf file
        of the cutout.
    
    
        Parameters
        ----------
        cutout : atlite.Cutout
        features : str/list, optional
            Feature(s) to be prepared. The default slice(None) results in all
            available features.
        tmpdir : str/Path, optional
            Directory in which temporary files (for example retrieved ERA5 netcdf
            files) are stored. If set, the directory will not be deleted and the
            intermediate files can be examined.
        overwrite : bool, optional
            Whether to overwrite variables which are already included in the
            cutout. The default is False.
    
        Returns
        -------
        cutout : atlite.Cutout
            Cutout with prepared data. The variables are stored in `cutout.data`.
    
        """
        if cutout.prepared and not overwrite:
            logger.info('Cutout already prepared.')
            return cutout
    
        logger.info(f'Storing temporary files in {tmpdir}')
    
        modules = atleast_1d(cutout.module)
        features = atleast_1d(features) if features else slice(None)
        prepared = set(atleast_1d(cutout.data.attrs['prepared_features']))
    
        # target is series of all available variables for given module and features
        target = available_features(modules).loc[:, features].drop_duplicates()
    
        for module in target.index.unique('module'):
            missing_vars = target[module]
            if not overwrite:
                missing_vars = missing_vars[lambda v: ~v.isin(cutout.data)]
            if missing_vars.empty:
                continue
            logger.info(f'Calculating and writing with module {module}:')
            missing_features = missing_vars.index.unique('feature')
            ds = get_features(cutout, module, missing_features, tmpdir=tmpdir)
            prepared |= set(missing_features)
    
            cutout.data.attrs.update(dict(prepared_features=list(prepared)))
>           ds = (cutout.data.merge(ds[missing_vars.values])
                  .assign_attrs(**non_bool_dict(cutout.data.attrs), **ds.attrs))
E           TypeError: xarray.core.common.DataWithCoords.assign_attrs() got multiple values for keyword argument 'Conventions'

atlite/data.py:159: TypeError

Your Environment

  • The atlite version used: master / update and sync dependencies #144
  • How you installed atlite (conda, pip or github): pip install -e . in conda env based on environment.yaml
  • Operating System: Linux
  • My environment: -
FabianHofmann added a commit that referenced this issue Jun 7, 2021
@FabianHofmann FabianHofmann mentioned this issue Jun 7, 2021
9 tasks
FabianHofmann pushed a commit that referenced this issue Jun 7, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant