Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Only list necessary parameters in .settings files #210

Open
forsyth2 opened this issue Mar 9, 2022 · 1 comment
Open

Only list necessary parameters in .settings files #210

forsyth2 opened this issue Mar 9, 2022 · 1 comment
Assignees
Labels
priority: low Low priority task

Comments

@forsyth2
Copy link
Collaborator

forsyth2 commented Mar 9, 2022

Currently, parameters defined high up in the inheritance hierarchy will show up in .settings files for tasks they aren't used in. For instance, www is often defined in [default] but [ts] doesn't actually use www. This isn't a huge problem but it can confuse users and developers.

@forsyth2 forsyth2 added the priority: low Low priority task label Mar 9, 2022
@forsyth2 forsyth2 self-assigned this Mar 9, 2022
@forsyth2
Copy link
Collaborator Author

forsyth2 commented Mar 9, 2022

Started looking into this as part of #204, but it is a bit tedious.

In zppy/utils.py:


# -----------------------------------------------------------------------------                                                                                          
# Condense directory to only have necessary settings                                                                                                                     
# To find all used parameters, run:                                                                                                                                      
# `git grep "c\[" zppy/<task.py` and `git grep "{{" zppy/templates/<task>.bash`                                                                                          

def condense_settings(settings_dict, used_parameters):
    keys = list(settings_dict.keys())
    used_parameters = set(used_parameters)
    slurm_header_parameters = ["nodes", "partition", "prefix", "qos", "scriptDir", "walltime"]
    used_parameters = used_parameters.union(slurm_header_parameters)
    for key in keys:
        if key not in used_parameters:
            settings_dict.pop(key)
    return settings_dict

In zppy/tc_analysis.py, for instance, the following would be added

            used_parameters = ["atm_name", "case", "debug", "dry_run", "environment_commands", "input_subdir", "input_files", "output", "prefix", "scratch", "scriptDir", "year1", "year2", "years"]
            c = condense_settings(c, used_parameters)

above these lines:

            # Create script                                                                                                                                              
            with open(scriptFile, "w") as f:
                f.write(template.render(**c))

            with open(settingsFile, "w") as sf:
                p = pprint.PrettyPrinter(indent=2, stream=sf)
                p.pprint(c)
                p.pprint(s)

Would also need to account for parameters in zppy/__main__.py.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
priority: low Low priority task
Projects
None yet
Development

No branches or pull requests

1 participant