Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better structure tests between zppy & zppy-interfaces #5

Open
forsyth2 opened this issue Nov 22, 2024 · 3 comments
Open

Better structure tests between zppy & zppy-interfaces #5

forsyth2 opened this issue Nov 22, 2024 · 3 comments
Assignees
Labels
Testing Files in `tests` modified

Comments

@forsyth2
Copy link
Collaborator

Rethink how testing is done in zppy & zppy-interfaces. zppy should simply test that it can invoke zppy-interfaces global-time-series. zppy-interfaces however should test that Global Time Series plots actually generate correctly.

Maybe we could have zppy test that zppy is passing in the right parameters and then have zppy-interfaces actually do the image comparisons of the plots, using pre-generated ts and mpas_analysis output.

@forsyth2 forsyth2 added the Testing Files in `tests` modified label Nov 22, 2024
@forsyth2 forsyth2 self-assigned this Nov 22, 2024
@forsyth2
Copy link
Collaborator Author

Also see E3SM-Project/zppy#537

@forsyth2
Copy link
Collaborator Author

forsyth2 commented Nov 25, 2024

Maybe we could have zppy test that zppy is passing in the right parameters and then have zppy-interfaces actually do the image comparisons of the plots, using pre-generated ts and mpas_analysis output.

I think this is the best path forward, but there are two concerns:

  1. This works fine for packages like e3sm_diags and zppy_interfaces where we can move the image comparison checking to those packages (indeed e3sm_diags already has that implemented). But what about mpas_analysis and ILAMB?
  2. It's one thing to know the arguments are correct, but it's another to know the input data is available. Part of zppy's purpose is setting up the necessary input data for each package it calls. (This is the reason for the job dependency chaining). So yes, we could test the command line arguments or generated cfg (for diags) are correct, but how would we test that we've generated the correct input data and put it in the right place?

@forsyth2
Copy link
Collaborator Author

Note that E3SM-Project/zppy#520 is to automate the zppy weekly tests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Testing Files in `tests` modified
Projects
None yet
Development

No branches or pull requests

1 participant