compass python package (compass v1.0) #28
Conversation
Instructions for setting up and running the Baroclinic Channels test caseSetting up and activating the conda environemntInstall Miniconda3 if you haven't already. Activate your base environment if you haven't already, e.g.: Don't try to add a new environment to the shared conda environments (where E3SM-Unified is found). Hopefully, no one but me has permission to do this anyway. Create a conda environment: Each time you want to run, you should first do: Then, load whatever modules you need to build and run MPAS-Ocean Setting up the test casesFirst, build MPAS-Ocean as normal. You can use the submodule at Once the model is built, set up a config file similar to this one ( If you're on one of the supported machines, you can specify the machine name and you won't need to specify the mesh database, etc. or any of the parallel options because that's all known. The example above is more for testing on a personal machine. To list test cases: To list the available machines, do: To set up test cases: To set up the 10-km test cases: Running the test casesChange to the work directory, where you will see the directory structure indicated above. This should run the full test case. You can also #!/usr/bin/env python
import pickle
import configparser
from compass.ocean.tests.baroclinic_channel.initial_state import run as run
def main():
with open('initial_state.pickle', 'rb') as handle:
test = pickle.load(handle)
config = configparser.ConfigParser(
interpolation=configparser.ExtendedInterpolation())
config.read('default.cfg')
run(test, config)
if __name__ == '__main__':
main()In this case, we're running the function Don't worry about the All the |
|
I'm trying to test this, but getting errors on the conda environment above. @xylar could you check that it works for you? details
|
|
The ones causing problems are our own packages: I'm guessing I need to give OK, I tried |
|
@mark-petersen, the first steps you need to do when you install Miniconda3 are: This should become a habit as part of installing Miniconda3 but I will try to be explicit about it. I added these two steps above. We get all of the package we need for |
|
@xylar I tested on grizzly, and it worked beautifully! I ran the test cases, and then within the run directory A few comments:
|
No, unfortunately, that would be a dangerous thing to do because it could be imported incorrectly as a module with
Yes, I haven't yet put in support for validation checks in the prototype. It wasn't my top priority. But it won't be hard. Similarly, the baseline flag doesn't actually do anything right now, nor the flag for not downloading. This is beyond the level of review that the prototype is up for at this point but I'm glad it runs for you! |
xylar
force-pushed
the
compass_1.0
branch
2 times, most recently
from
f49cd5f to
fd40a95
Compare
|
Hello @xylar! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:
Comment last updated at 2021-04-23 20:49:09 UTC |
xylar
force-pushed
the
compass_1.0
branch
6 times, most recently
from
851a65e to
c889398
Compare
|
@xylar, I was excited to see your progress here, as I'll be working on test cases this month, and would like to add to this system if possible. I gave this PR a try again, but ran into an error. I see above you moved to mpas_tools 0.1.0., so I used: but I always get an error: Are there other library versions that should be updated in the |
|
Yes, I made changes in MPAS-Tools that are required here. There is not a release of that package yet because I didn't think anyone else needed it. So this branch is not available for use by anyone but me at the moment. I'll let you know when you can try it out again, with updated instructions. |
xylar
force-pushed
the
compass_1.0
branch
4 times, most recently
from
65d1988 to
9746b41
Compare
|
@mark-petersen, @vanroekel, @matthewhoffman and anyone else that's following along, just a quick note to say that I think I'm done with developing the prototype, other than minor changes and any bug fixes that emerge from testing and review. I have successfully:
The additional flexibility of the new framework is paying off beyond what I had envisioned! It is easy, for example, to set up a full initialization and spin-up of any supported mesh with or without ice-shelf cavities, with or without BGC, with the PHC or EN4_1900 initial condition, and with either split-explicit or RK4 time stepping. The same applies to all the shorter test cases (performance, restart, threads, cores, analysis, etc.). I've currently gone a bit overboard with that to prove the possibilities but it's really something! In the next two weeks, I will work on:
For now, I don't need anything further from any of you, just wanted to let you know where things stand. |
|
Thanks @xylar this sounds like really great progress and some real promising capabilities! If you don't mind I'd like to log a summary of this under the NGD ocean 6 month report so you can be credited for this work. |
Correct documentation of 60layerPHC grid
This merge includes a large number of updates to the User's Guide following recent moves to classes and a new naming convention. Machine usage has also been updated to be consistent with recent testing with E3SM modules. The Developer's Guide has also been updated with the change from "configuration" to "test group".
Add a glossary.
|
Thanks @vanroekel! And thanks @mark-petersen and @matthewhoffman as well! I just added the documentation and will merge as soon as CI passes (if it does). |
This merge creates a
compasspython package that can be used to list, set up, run, validate, and clean up test cases and set up, run, and clean up test suites.This prototype contains
landiceandoceanMPAS cores. Thelandicecore has all test cases in thesia_integrationtest suite and a few other related test cases in the test groups that are in that test suite. Theoceancore has all of the test cases for thebaroclinic_channeltest group, all test cases from thenightlyregression suite, and theQU240,QUwISC240,EC30to60,ECwIsC30to60andSOwISC12to60meshes from "legacy" COMPASS (COMPASS as it currently exists).A significant number of python modules (python files) and packages (subdirectories) have been added within
compassthat break up the tasks related to listing, setting up, and cleaning up test cases that were previously inlist_testcases.py,setup_testcase.pyandclean_testcase.pyas well as functionality related to test suites that was inmanage_regression_suite.pybefore. New functionality has also been added to make setting up and running test cases easier, and to promote more code reuse than the current COMPASS framework permits. See the design document in #29 for more details.Each test case has an associated configuration file (
.cfg) that is constructed by combining config options from various sources: a core-specific config file, one associated with the machine (if running on a "supported" machine), one for the configuration, one for the test case, and a user-defined config file. Typically, these contain different types of options with different purposes. For example, the core's config file points to namelist and streams templates that are core specific. The machine config file contains information about the batch queuing system, the number of cores per node, etc. The test group and test case config files contain options that can be treated as the python equivalents of the namelist options used in the Fortran MPAS-Model code. The user-defined config file can override any config options from these other sources, and can provide paths to initial conditions, meshes and other data that the core or test case may require. An effort has been made to automate most of this so a user config file should only be needed on a non-supported machine. Users can also edit the local config file within a test case (and symlinked within each step) that collects all of the config options for that test case before running.closes #11
closes #9
closes #20