Add a list of test types to the users guide #2062

fischer-ncar · 2017-11-14T21:52:26Z

Add a list of test types to the users guide section 4.

Test suite:
Test baseline:
Test namelist changes:
Test status: [bit for bit, roundoff, climate changing]

Fixes [CIME Github issue #]

User interface changes?:

Update gh-pages html (Y/N)?: Y

Code review:

Test suite: Test baseline: Test namelist changes: Test status: [bit for bit, roundoff, climate changing] Fixes [CIME Github issue #] User interface changes?: Update gh-pages html (Y/N)?: Y Code review:

the users guide. Test suite: Test baseline: Test namelist changes: Test status: [bit for bit, roundoff, climate changing] Fixes [CIME Github issue #] User interface changes?: Update gh-pages html (Y/N)?: Y Code review:

billsacks · 2017-11-14T22:56:09Z

Thanks for doing this, @fischer-ncar ! It's not clear to me how these tests are organized. I'd suggest either organizing them alphabetically or by broad category (restart tests, tests of task/thread count, etc.)

fischer-ncar · 2017-11-15T15:11:10Z

@billsacks, I'll arrange them by broad categories (restart tests, task/thread counts, performance, misc.)

Test baseline: Test namelist changes: Test status: [bit for bit, roundoff, climate changing] Fixes [CIME Github issue #] User interface changes?: Update gh-pages html (Y/N)?: Y Code review:

fischer-ncar · 2017-11-15T18:24:08Z

I just pushed up a change that has the testtypes arranged with restarts first, then transitioning to thread/mpi tests. Followed by performance tests, multi-driver/multi-instance tests, misc. "smoke" tests, then data assimilation related tests.

billsacks · 2017-11-15T18:37:48Z

Thanks!

bertinia · 2017-11-15T19:05:43Z

doc/source/users_guide/testing.rst

@@ -4,9 +4,6 @@
 Testing with create_test
 **************************

-.. todo ::
-
-   add list and descriptions on functionality tests (ERS, ERI, ect)

 create_test is the tool we use to test cime-driven models. It can be used as a easy way to


should be "used as an easy"

bertinia · 2017-11-15T19:06:55Z

doc/source/users_guide/testing.rst

@@ -4,9 +4,6 @@
 Testing with create_test
 **************************

-.. todo ::
-
-   add list and descriptions on functionality tests (ERS, ERI, ect)

 create_test is the tool we use to test cime-driven models. It can be used as a easy way to
 run a single basic test or can be used to run an entire suite of tests. It has parallelism


should be "or an entire suite of tests".

bertinia

Just a couple of wording and formatting changes for clarity:

create_test should be in bold
"It has parallelism supplied." should be "create_test runs a test suite in parallel for improved performance."
"TESTMODS are optional and can be used to make some env XML changes prior to running a test."
(suffix) should be (file suffix)

bertinia · 2017-11-15T19:23:20Z

doc/source/users_guide/testing.rst

+
+======================= =====================================================================================
+
+
 Each test run by create test will be put through the following mandatory phases:


This should be: "Each test run by create_test includes the following mandatory steps:"

bertinia

@rljacob please review section 4.1 of the CIME user's guide "Using create_test (ACME)". Should this be in an appendix? And is ACME still the correct model name to be used in the documentation?
@fischer-ncar would you update section 4.3 and 4.4?

rljacob · 2017-11-15T19:38:30Z

Yes we decided in the Oct 18th meeting to move model-specific usage to an appendix. And it should say "E3SM" everywhere instead of "ACME".

fischer-ncar · 2017-11-15T19:44:22Z

It looks like section 4.4 Adding tests which is using the update_acme_tests.py should be moved to an appendix too.

rljacob · 2017-11-15T19:48:07Z

That mechanism needs to be generalized so that the list of tests lives in config/acme. That is on @jgfouca 's todo list.

Test suite: Test baseline: Test namelist changes: Test status: [bit for bit, roundoff, climate changing] Fixes [CIME Github issue #] User interface changes?: Update gh-pages html (Y/N)?: Y Code review:

fischer-ncar · 2017-11-15T21:51:10Z

@bertinia, I pushed the changes you requested, except for the 4.3 and 4.4 section updates.

…branch-for-to-acme-12-06-2017-pr * commit '530c337666de1af28698c74b9e58e8baf93a6bbf': fix issue in scripts_regression_tests Add ability to customize CaseStatus success message for a phase. Fix bug in compare namelists add test for xmlquery DOCN_MODE Fix memory leak of newtimers in merge_thread_data in splitting comp vars make sure comp isnt subset of name Update ChangeLog nthrds > 1 should set build_threaded flag Update ChangeLog Update ChangeLog fix pylint issue fix gptl threaded build, batch change for cheyenne correct path for includes used in build Check for unicode in expect() update ACME to E3SM and changes related to Issue #1519 and PR #2062 Do not use normalization when mapping glc2ocn Add comments about not needing to add fields to x2o_fluxes and x2i_fluxes Point to new glc2ocn mapping files Allow different glc2ocn mappings for liq and ice

#1963) Fixes: * DOCN_MODE fix for xmlchange * compare_namelist bug in rare code path * Fix memleak in gptl * Fix setting of build_threaded * Fix unicode problems with expect Feature list: * Add job-id info for CaseStatus entry for job submit * Allow different glc2ocn mappings for liq and ice [BFB] * jgfouca/branch-for-to-acme-12-06-2017-pr: (21 commits) Ignore buildlib python bytecode files This is no longer needed fix issue in scripts_regression_tests Add ability to customize CaseStatus success message for a phase. Fix bug in compare namelists add test for xmlquery DOCN_MODE Fix memory leak of newtimers in merge_thread_data in splitting comp vars make sure comp isnt subset of name Update ChangeLog nthrds > 1 should set build_threaded flag Update ChangeLog Update ChangeLog fix pylint issue fix gptl threaded build, batch change for cheyenne correct path for includes used in build Check for unicode in expect() update ACME to E3SM and changes related to Issue #1519 and PR #2062 Do not use normalization when mapping glc2ocn Add comments about not needing to add fields to x2o_fluxes and x2i_fluxes Point to new glc2ocn mapping files ...

…branch-for-to-acme-12-06-2017-pr * commit '530c337666de1af28698c74b9e58e8baf93a6bbf': fix issue in scripts_regression_tests Add ability to customize CaseStatus success message for a phase. Fix bug in compare namelists add test for xmlquery DOCN_MODE Fix memory leak of newtimers in merge_thread_data in splitting comp vars make sure comp isnt subset of name Update ChangeLog nthrds > 1 should set build_threaded flag Update ChangeLog Update ChangeLog fix pylint issue fix gptl threaded build, batch change for cheyenne correct path for includes used in build Check for unicode in expect() update ACME to E3SM and changes related to Issue #1519 and PR #2062 Do not use normalization when mapping glc2ocn Add comments about not needing to add fields to x2o_fluxes and x2i_fluxes Point to new glc2ocn mapping files Allow different glc2ocn mappings for liq and ice

#1963) Fixes: * DOCN_MODE fix for xmlchange * compare_namelist bug in rare code path * Fix memleak in gptl * Fix setting of build_threaded * Fix unicode problems with expect Feature list: * Add job-id info for CaseStatus entry for job submit * Allow different glc2ocn mappings for liq and ice [BFB] * jgfouca/branch-for-to-acme-12-06-2017-pr: (21 commits) Ignore buildlib python bytecode files This is no longer needed fix issue in scripts_regression_tests Add ability to customize CaseStatus success message for a phase. Fix bug in compare namelists add test for xmlquery DOCN_MODE Fix memory leak of newtimers in merge_thread_data in splitting comp vars make sure comp isnt subset of name Update ChangeLog nthrds > 1 should set build_threaded flag Update ChangeLog Update ChangeLog fix pylint issue fix gptl threaded build, batch change for cheyenne correct path for includes used in build Check for unicode in expect() update ACME to E3SM and changes related to Issue #1519 and PR #2062 Do not use normalization when mapping glc2ocn Add comments about not needing to add fields to x2o_fluxes and x2i_fluxes Point to new glc2ocn mapping files ...

…branch-for-to-acme-12-06-2017-pr * commit '530c337666de1af28698c74b9e58e8baf93a6bbf': fix issue in scripts_regression_tests Add ability to customize CaseStatus success message for a phase. Fix bug in compare namelists add test for xmlquery DOCN_MODE Fix memory leak of newtimers in merge_thread_data in splitting comp vars make sure comp isnt subset of name Update ChangeLog nthrds > 1 should set build_threaded flag Update ChangeLog Update ChangeLog fix pylint issue fix gptl threaded build, batch change for cheyenne correct path for includes used in build Check for unicode in expect() update ACME to E3SM and changes related to Issue #1519 and PR #2062 Do not use normalization when mapping glc2ocn Add comments about not needing to add fields to x2o_fluxes and x2i_fluxes Point to new glc2ocn mapping files Allow different glc2ocn mappings for liq and ice

#1963) Fixes: * DOCN_MODE fix for xmlchange * compare_namelist bug in rare code path * Fix memleak in gptl * Fix setting of build_threaded * Fix unicode problems with expect Feature list: * Add job-id info for CaseStatus entry for job submit * Allow different glc2ocn mappings for liq and ice [BFB] * jgfouca/branch-for-to-acme-12-06-2017-pr: (21 commits) Ignore buildlib python bytecode files This is no longer needed fix issue in scripts_regression_tests Add ability to customize CaseStatus success message for a phase. Fix bug in compare namelists add test for xmlquery DOCN_MODE Fix memory leak of newtimers in merge_thread_data in splitting comp vars make sure comp isnt subset of name Update ChangeLog nthrds > 1 should set build_threaded flag Update ChangeLog Update ChangeLog fix pylint issue fix gptl threaded build, batch change for cheyenne correct path for includes used in build Check for unicode in expect() update ACME to E3SM and changes related to Issue #1519 and PR #2062 Do not use normalization when mapping glc2ocn Add comments about not needing to add fields to x2o_fluxes and x2i_fluxes Point to new glc2ocn mapping files ...

fischer-ncar added 2 commits November 14, 2017 14:41

Add list of test types to users guide.

aae8c78

Test suite: Test baseline: Test namelist changes: Test status: [bit for bit, roundoff, climate changing] Fixes [CIME Github issue #] User interface changes?: Update gh-pages html (Y/N)?: Y Code review:

Add a list of the different test types to section 4 of

2f7b948

the users guide. Test suite: Test baseline: Test namelist changes: Test status: [bit for bit, roundoff, climate changing] Fixes [CIME Github issue #] User interface changes?: Update gh-pages html (Y/N)?: Y Code review:

fischer-ncar requested a review from bertinia November 14, 2017 21:52

fischer-ncar assigned bertinia Nov 14, 2017

Test suite:

42e7527

Test baseline: Test namelist changes: Test status: [bit for bit, roundoff, climate changing] Fixes [CIME Github issue #] User interface changes?: Update gh-pages html (Y/N)?: Y Code review:

bertinia reviewed Nov 15, 2017

View reviewed changes

bertinia requested changes Nov 15, 2017

View reviewed changes

bertinia reviewed Nov 15, 2017

View reviewed changes

bertinia requested changes Nov 15, 2017

View reviewed changes

rljacob added the in progress label Nov 15, 2017

Changes requested by bertinia

fb7dc2d

Test suite: Test baseline: Test namelist changes: Test status: [bit for bit, roundoff, climate changing] Fixes [CIME Github issue #] User interface changes?: Update gh-pages html (Y/N)?: Y Code review:

bertinia merged commit 92f74f8 into master Nov 15, 2017

bertinia removed the in progress label Nov 15, 2017

bertinia mentioned this pull request Nov 16, 2017

update CIME RST documentation #2078

Merged

fischer-ncar deleted the fischer/testtype_doc branch March 11, 2018 18:26

fischer-ncar mentioned this pull request Apr 23, 2018

Documentation for different Test Codes (SMS, ERS, ERI ...) #1685

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add a list of test types to the users guide #2062

Add a list of test types to the users guide #2062

fischer-ncar commented Nov 14, 2017

billsacks commented Nov 14, 2017

fischer-ncar commented Nov 15, 2017

fischer-ncar commented Nov 15, 2017

billsacks commented Nov 15, 2017

bertinia Nov 15, 2017

bertinia Nov 15, 2017

bertinia left a comment

bertinia Nov 15, 2017

bertinia left a comment

rljacob commented Nov 15, 2017

fischer-ncar commented Nov 15, 2017

rljacob commented Nov 15, 2017

fischer-ncar commented Nov 15, 2017


		======================= =====================================================================================


		Each test run by create test will be put through the following mandatory phases:

Add a list of test types to the users guide #2062

Add a list of test types to the users guide #2062

Conversation

fischer-ncar commented Nov 14, 2017

billsacks commented Nov 14, 2017

fischer-ncar commented Nov 15, 2017

fischer-ncar commented Nov 15, 2017

billsacks commented Nov 15, 2017

bertinia Nov 15, 2017

Choose a reason for hiding this comment

bertinia Nov 15, 2017

Choose a reason for hiding this comment

bertinia left a comment

Choose a reason for hiding this comment

bertinia Nov 15, 2017

Choose a reason for hiding this comment

bertinia left a comment

Choose a reason for hiding this comment

rljacob commented Nov 15, 2017

fischer-ncar commented Nov 15, 2017

rljacob commented Nov 15, 2017

fischer-ncar commented Nov 15, 2017