Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unclear default preprocessor order for time related preprocessor functions #971

Closed
bouweandela opened this issue Jan 31, 2021 · 1 comment · Fixed by #973
Closed

Unclear default preprocessor order for time related preprocessor functions #971

bouweandela opened this issue Jan 31, 2021 · 1 comment · Fixed by #973
Assignees
Labels
bug Something isn't working
Milestone

Comments

@bouweandela
Copy link
Member

It looks like there has been a mistake when fixing a merge conflict in #696 and now the default order in which the preprocessor functions are applied is no longer clear because several preprocessor functions are mentioned twice in the list that defines the order:

# Extract years given by dataset keys (start_year and end_year)
'clip_start_end_year',
# Time extraction
'extract_time',
'extract_season',
'extract_month',
'resample_hours',
'resample_time',
# Data reformatting/CMORization
'fix_data',
'cmor_check_data',
# Time extraction (as defined in the preprocessor section)
'extract_time',
'extract_season',
'extract_month',

I think that all time related preprocessor functions would need to be moved after cmor_check_data, but can you please check @schlunma and @jvegasbsc?

@bouweandela bouweandela added the bug Something isn't working label Jan 31, 2021
@bouweandela bouweandela added this to the v2.2.0 milestone Jan 31, 2021
@schlunma
Copy link
Contributor

schlunma commented Feb 1, 2021

Yes, all non-default preprocessors that can be specified by the user in the recipe should be moved after cmor_check_data so that custom_order: true works with them (see #762). Thus, the order should be:

    ...
    # Concatenate all cubes in one
    'concatenate',
    'cmor_check_metadata',
    # Extract years given by dataset keys (start_year and end_year)
    'clip_start_end_year',
    # Data reformatting/CMORization
    'fix_data',
    'cmor_check_data',
    # Time extraction (as defined in the preprocessor section)
    'extract_time',
    'extract_season',
    'extract_month',
    'resample_hours',
    'resample_time',
    # Level extraction
    'extract_levels',
    # Weighting
    'weighting_landsea_fraction',
    ...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants