Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Core functionality for fx variables handling in volume and area statistics in light of PR 439 (PR 439 child 2) #511

Closed
wants to merge 39 commits into from
Closed
Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
4b21a4c
functionality for fx vars data to be preprocessed part of area and vo…
valeriupredoi Feb 21, 2020
8159f59
slight change for special fx vars case
valeriupredoi Feb 21, 2020
d0cb3f9
comprehensive testing for fx vars preprocessing hadling for area and …
valeriupredoi Feb 21, 2020
17d03ef
Merge branch 'master' into PR439_child2_main_functionality
valeriupredoi Feb 21, 2020
4c3202f
Klaus suggestion
valeriupredoi Feb 24, 2020
5dadda1
Klaus suggestion
valeriupredoi Feb 24, 2020
887fe65
Klaus suggestion
valeriupredoi Feb 24, 2020
c4f01e6
batch suggestions by Klaus
valeriupredoi Feb 25, 2020
d485ffe
comments from Klaus
valeriupredoi Feb 25, 2020
bfe3611
more Klaus suggestions and a change of flag name
valeriupredoi Feb 25, 2020
e1c197c
moar Klaus suggestions
valeriupredoi Feb 25, 2020
085bf1f
moar Klaus suggestions
valeriupredoi Feb 25, 2020
a35ac22
last bit of Klaus suggestion
valeriupredoi Feb 25, 2020
39fbd19
changed test in light of recipe change
valeriupredoi Feb 25, 2020
bb40b37
reafctored get_preprocessor_task and added a separate function for fx…
valeriupredoi Feb 25, 2020
a293aca
put in nested def so to eliminate code duplication
valeriupredoi Feb 25, 2020
facad13
Refactor fx settings
Feb 26, 2020
b009768
Merge branch 'master' into PR439_child2_main_functionality
valeriupredoi Feb 26, 2020
cc0ca0c
last suggestion from hero Klaus
valeriupredoi Feb 26, 2020
6a5cf43
Refactor fx_files settings updating
Feb 26, 2020
ca68238
fixed the fx var handling for spatial stats
valeriupredoi Feb 27, 2020
e7b25fa
Merge branch 'PR439_child2_main_functionality' into PR439_child2_main…
valeriupredoi Feb 27, 2020
2afe12f
slight reformatting of logger message
valeriupredoi Feb 27, 2020
9cb7b23
Merge pull request #522 from ESMValGroup/PR439_child2_main_functional…
valeriupredoi Mar 2, 2020
2af0853
Merge branch 'master' into PR439_child2_main_functionality
valeriupredoi Mar 5, 2020
5bd033e
renamed function to more appropriate name
valeriupredoi Mar 5, 2020
2194cb6
unified functions and made it possible for user to define their own f…
valeriupredoi Mar 5, 2020
e32ba97
removed duplicated redundancy
valeriupredoi Mar 5, 2020
e99344b
let mask take dicts only for fx files
valeriupredoi Mar 5, 2020
4e62f1f
make mask work with dicts for fx files
valeriupredoi Mar 5, 2020
997d010
corrected test for dicts fx files
valeriupredoi Mar 5, 2020
dac52c4
corrected for dicts all over
valeriupredoi Mar 5, 2020
2aab695
cancelled stupid complication with lists of lists
valeriupredoi Mar 5, 2020
9cfdd3a
corrected for simpler approach
valeriupredoi Mar 5, 2020
b5a4fc4
account for empty list of keys
valeriupredoi Mar 5, 2020
87fe74f
dict comprehensione
valeriupredoi Mar 5, 2020
36010e1
simplified even more and optimized handling
valeriupredoi Mar 5, 2020
4e14f0e
added test for user specified vfx variable for landsea mask
valeriupredoi Mar 5, 2020
b0fa35d
added one more test and fixed flake shtuff
valeriupredoi Mar 5, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions esmvalcore/_data_finder.py
Original file line number Diff line number Diff line change
Expand Up @@ -240,7 +240,7 @@ def get_input_filelist(variable, rootpath, drs):
return (files, dirnames, filenames)


def get_output_file(variable, preproc_dir):
def get_output_file(variable, preproc_dir, fx_var_alias=None):
"""Return the full path to the output (preprocessed) file."""
cfg = get_project_config(variable['project'])

Expand All @@ -249,11 +249,12 @@ def get_output_file(variable, preproc_dir):
variable = dict(variable)
variable['exp'] = '-'.join(variable['exp'])

tag_source = variable if fx_var_alias is None else fx_var_alias
outfile = os.path.join(
preproc_dir,
variable['diagnostic'],
variable['variable_group'],
_replace_tags(cfg['output_file'], variable)[0],
_replace_tags(cfg['output_file'], tag_source)[0],
)
if variable['frequency'] != 'fx':
outfile += '_{start_year}-{end_year}'.format(**variable)
Expand Down
Loading