Merge remote-tracking branch 'esmci_remote_for_split/master' into jgf…

…ouca/branch-for-acme-split-10-17-2017-pr * esmci_remote_for_split/master: (132 commits) Data model documentation edits Update ChangeLog Get melvin pylint checks working again. remove debug print statement fix accidental commit correction replace component with compclass, fix pylint error remove need for special XML class attribute _component_value_list Add details on year_align Improve documentation of CPLHIST_YR_ALIGN variables, from Keith Lindsay Minor documentation fix from Mariana Fix a slightly incorrect error message in summarize_cprnc_diffs Fix py3 for erio test fix conversion issue in test fix test to match string type in convert_to_string, add unit tests PR #1949 This is a port to python3 while maintaining compatibility with python 2.7. The new module six.py provides portability between the two. This PR deserves careful testing and discussion. consistency changes for CPLHIST defaults and updated documentation fix whitespace Make some documentation consistent Remove some log statements ...
ESMCI · Oct 17, 2017 · 84e6bd6 · 84e6bd6
2 parents 4382289 + a915609
commit 84e6bd6
Show file tree

Hide file tree

Showing 297 changed files with 18,976 additions and 12,503 deletions.
diff --git a/ChangeLog b/ChangeLog
diff --git a/LICENSE.TXT b/LICENSE.TXT
@@ -4,12 +4,15 @@ All rights reserved.
 Copyright (c) 2017, Sandia Corporation. 
 Under the terms of Contract DE-AC04-94AL85000 with Sandia Corporation,
 the U.S. Government retains certain rights in this software.
+  and
+Copyright (c) 2017, UChicago Argonne, LLC, All Rights Reserved 
+under Contract No. DE-AC02-06CH11357 with the Department of Energy
 
 Developed by:
               University Corporation for Atmospheric Research - National Center for Atmospheric Research
               https://www2.cesm.ucar.edu/working-groups/sewg
                 and
-              DOE BER ACME project team members, including those at SNL and ANL
+              DOE BER E3SM project team members, including those at SNL and ANL
 
 Permission is hereby granted, free of charge, to any person obtaining
 a copy of this software and associated documentation files (the "Software"),

diff --git a/README.md b/README.md
@@ -1,48 +1,40 @@
 # cime
 Common Infrastructure for Modeling the Earth
 
-**cime** (pronounced: seem) represents the infrastructure code for the 
+CIME, pronounced “SEAM”, contains the support scripts (configure, build, run, test), data models, essential
+utility libraries, a “main” and other tools that are needed to build a single-executable coupled Earth System Model.
+CIME is available in a stand-alone package that can be compiled and tested without active prognostic components
+but is typically included in the source of a climate model. CIME does not contain: any active components,
+any intra-component coupling capability (such as atmosphere physics-dynamics coupling).
+
+*cime* (pronounced: seem) is currently used by the
 <a href="http://www2.cesm.ucar.edu">Community Earth System Model </a>
-     (CESM) and the <a href="https://climatemodeling.science.energy.gov/projects/accelerated-climate-modeling-energy">
-Accelerated Climate Model for Energy </a> (ACME).
-*cime* providess scripts for configuration, build, and testing of
-models, as well as code for data and stub models for climate components,
-and a driver code for bringing all the climate components together in a single executable.
+     (CESM) and the <a href="https://climatemodeling.science.energy.gov/projects/energy-exascale-earth-system-model">
+Energy Exascale Earth System Model</a> (E3SM).
 
 # Documentation
 
 See <a href="http://esmci.github.io/cime">esmci.github.io/cime</a>
 
 # Developers
 
-*cime* was initially extracted from CESM as a stand-alone capability in 2015
-by members of the CSEG group at NCAR, the software engineering team of
-the CESM model. The CSEG group had been developing this infrastrucure within
-CESM for a number of years using NSF and DOE funding. After version 4 was released,
-a joint development partnership was started with the software engineering group of
-ACME, a DOE-funded project, which had branched from CESM in 2014. 
-Starting with development of version 5, *cime* is cooperative effort with contributions
-and ownership from members of both teams. 
-
-The following table documents the developers who have contributed to *cime*,
-showing what versions of they contributed to, and with what source(s) of support.
-
-Name     |     Institution  |  Versions  | Funding Source (for versions)
----------|------------------|----------|----------------------
-Mariana Vertenstein | NCAR  |  1 - D   |   NSF, DOE
-Jim Edwards |         NCAR  |  1 - D   |   NSF (1-D), DOE(1-2)
-Jim Foucar  |         SNL   |  5 - D   |   DOE
-Rob Jacob |      ANL  |  5 - D   |   DOE
-Bill Sacks |    NCAR |  1 - D  | NSF, DOE
-Andreas Wilke |  ANL  | 5 - D | DOE
-Jason Sarich | ANL | 5 - D | DOE
-Michael Deakin | SNL | 5 - D | DOE
-Erich Foster   | SNL | 5 - D | DOE
-Alice Bertini | NCAR | 1 - D | NSF
-Chris Fischer | NCAR | 5 - D | NSF
-Steve Goldhaber | NCAR | 1 - D | NSF, DOE
-Mike Levy | NCAR | 1 - D | NSF, DOE
-Sean Santos |         NCAR  |  1 - 4   |   NSF
-
-_Key: D = Current development version (i.e. still active on project)_
+## Lead Developers
+Case Control System: Jim Edwards (NCAR), Jim Foucar (SNL)
+
+MCT-based Coupler/Driver:  Mariana Vertenstein (NCAR), Robert Jacob (ANL)
+
+Data Models:  Mariana Vertenstein (NCAR)
+
+## Also Developed by
+Alice Bertini (NCAR), Tony Craig (NCAR), Michael Deakin (SNL), Chris Fischer (NCAR), Steve Goldhaber (NCAR),
+Erich Foster (SNL), Mike Levy (NCAR), Bill Sacks (NCAR), Andrew Salinger (SNL), Sean Santos (NCAR), Jason Sarich (ANL),
+Andreas Wilke (ANL).
+
+# Acknowledgements
+
+CIME is jointly developed with support from the Earth System Modeling program of DOE's BER office and the CESM program
+of NSF's Division of Atmospheric and Geospace Sciences.
+
+# License
 
+CIME is free software made available under the BSD License. For details see the LICENSE file.
diff --git a/config/acme/allactive/config_pesall.xml b/config/acme/allactive/config_pesall.xml
@@ -4535,6 +4535,8 @@
     <mach name="anvil">
       <pes compset="any" pesize="S">
         <comment>ne30_ne30 grid on 40 nodes 36 ppn pure-MPI</comment>
+        <MAX_MPITASKS_PER_NODE>32</MAX_MPITASKS_PER_NODE>
+        <MAX_TASKS_PER_NODE>32</MAX_TASKS_PER_NODE>
         <ntasks>
           <ntasks_atm>1350</ntasks_atm>
           <ntasks_lnd>72</ntasks_lnd>
@@ -6741,7 +6743,7 @@
     <mach name="anvil">
       <pes compset=".*CAM5.+CLM45.+MPASCICE.+MPASO.+MOSART.+SGLC.+SWAV" pesize="S">
         <comment> -compset A_WCYCL* -res ne30_oEC* on 32 nodes pure-MPI </comment>
-        <PES_PER_NODE>32</PES_PER_NODE>
+        <MAX_MPITASKS_PER_NODE>32</MAX_MPITASKS_PER_NODE>
         <MAX_TASKS_PER_NODE>32</MAX_TASKS_PER_NODE>
         <ntasks>
           <ntasks_atm>675</ntasks_atm>
@@ -6776,7 +6778,7 @@
       </pes>
       <pes compset=".*CAM5.+CLM45.+MPASCICE.+MPASO.+MOSART.+SGLC.+SWAV" pesize="any">
         <comment> -compset A_WCYCL* -res ne30_oEC* on 59 nodes pure-MPI </comment>
-        <PES_PER_NODE>32</PES_PER_NODE>
+        <MAX_MPITASKS_PER_NODE>32</MAX_MPITASKS_PER_NODE>
         <MAX_TASKS_PER_NODE>32</MAX_TASKS_PER_NODE>
         <ntasks>
           <ntasks_atm>1350</ntasks_atm>
@@ -6811,7 +6813,7 @@
       </pes>
       <pes compset=".*CAM5.+CLM45.+MPASCICE.+MPASO.+MOSART.+SGLC.+SWAV" pesize="L">
         <comment> -compset A_WCYCL* -res ne30_oEC* on 115 nodes pure-MPI </comment>
-        <PES_PER_NODE>32</PES_PER_NODE>
+        <MAX_MPITASKS_PER_NODE>32</MAX_MPITASKS_PER_NODE>
         <MAX_TASKS_PER_NODE>32</MAX_TASKS_PER_NODE>
         <ntasks>
           <ntasks_atm>2700</ntasks_atm>

diff --git a/config/acme/machines/config_batch.xml b/config/acme/machines/config_batch.xml
@@ -16,6 +16,7 @@
   <batch_system type="template" >
     <batch_query args=""></batch_query>
     <batch_submit></batch_submit>
+    <batch_cancel></batch_cancel>
     <batch_redirect></batch_redirect>
     <batch_directive></batch_directive>
     <directives>
@@ -26,6 +27,7 @@
   <batch_system type="none" >
     <batch_query args=""></batch_query>
     <batch_submit></batch_submit>
+    <batch_cancel></batch_cancel>
     <batch_redirect></batch_redirect>
     <batch_directive></batch_directive>
     <directives>
@@ -36,6 +38,8 @@
    <batch_system type="cobalt" >
      <batch_query>qstat</batch_query>
      <batch_submit>qsub</batch_submit>
+     <batch_cancel>qdel</batch_cancel>
+     <batch_env>-v</batch_env>
      <batch_directive></batch_directive>
      <jobid_pattern>(\d+)</jobid_pattern>
      <depend_string> --dependencies</depend_string>
@@ -56,6 +60,8 @@
   <batch_system type="cobalt_theta" >
     <batch_query>qstat</batch_query>
     <batch_submit>qsub</batch_submit>
+    <batch_cancel>qdel</batch_cancel>
+    <batch_env>-v</batch_env>
     <batch_directive>#COBALT</batch_directive>
     <jobid_pattern>(\d+)</jobid_pattern>
     <depend_string> --dependencies</depend_string>
@@ -74,6 +80,7 @@
   <batch_system type="lsf" version="10.1">
     <batch_query args=" -w" >bjobs</batch_query>
     <batch_submit>bsub</batch_submit>
+    <batch_cancel>bkill</batch_cancel>
     <batch_redirect>&lt;</batch_redirect>
     <batch_directive>#BSUB</batch_directive>
     <jobid_pattern>&lt;(\d+)&gt;</jobid_pattern>
@@ -94,15 +101,17 @@
 	  <!-- The following option causes problems with lsf version on Summitdev.
 	  If desired, this should be in specific machine section. -->
       <!-- <directive default="poe"         > -a {{ poe }} </directive> -->
-      <directive default="acme.stdout" > -o {{ output_error_path }}.%J  </directive>
-      <directive default="acme.stderr" > -e {{ output_error_path }}.%J  </directive>
+      <directive default="acme.stdout" > -o {{ job_id }}.%J  </directive>
+      <directive default="acme.stderr" > -e {{ job_id }}.%J  </directive>
       <directive                       > -J {{ job_id }} </directive>
     </directives>
   </batch_system>
 
   <batch_system type="pbs" >
     <batch_query args="-f" >qstat</batch_query>
     <batch_submit>qsub </batch_submit>
+    <batch_cancel>qdel</batch_cancel>
+    <batch_env>-v</batch_env>
     <batch_directive>#PBS</batch_directive>
     <jobid_pattern>^(\S+)$</jobid_pattern>
     <depend_string> -W depend=afterok:jobid</depend_string>
@@ -118,7 +127,7 @@
     <directives>
       <directive> -N {{ job_id }}</directive>
       <directive default="n"> -r {{ rerunnable }} </directive>
-      <!-- <directive> -j oe {{ output_error_path }} </directive> -->
+      <!-- <directive> -j oe {{ job_id }} </directive> -->
       <directive> -j oe </directive>
       <directive default="ae"  > -m {{ mail_options }} </directive>
       <directive> -V </directive>
@@ -128,6 +137,7 @@
   <batch_system type="moab" >
     <batch_query>showq</batch_query>
     <batch_submit>msub </batch_submit>
+    <batch_cancel>canceljob</batch_cancel>
     <batch_directive>#MSUB</batch_directive>
     <jobid_pattern>(\d+)$</jobid_pattern>
     <depend_string> -W depend=afterok:jobid</depend_string>
@@ -151,6 +161,7 @@
   <!-- for lawrence livermore computing -->
    <batch_system type="lc_slurm">
      <batch_submit>sbatch</batch_submit>
+     <batch_cancel>scancel</batch_cancel>
      <batch_directive>#SBATCH</batch_directive>
      <jobid_pattern>(\d+)$</jobid_pattern>
      <depend_string> -l depend=jobid</depend_string>
@@ -171,14 +182,15 @@
       <directive> -A {{ project }} </directive>
      </directives>
      <queues>
-        <queue walltimemax="01:00:00" jobmin="1" jobmax="4320" default="true">pbatch</queue>
+        <queue walltimemax="01:00:00" nodemin="1" nodemax="270" default="true">pbatch</queue>
      </queues>
    </batch_system>
   <!-- for lawrence livermore computing -->
 
    <batch_system type="slurm" >
      <batch_query per_job_arg="-j">squeue</batch_query>
      <batch_submit>sbatch</batch_submit>
+     <batch_cancel>scancel</batch_cancel>
      <batch_directive>#SBATCH</batch_directive>
      <jobid_pattern>(\d+)$</jobid_pattern>
      <depend_string> --dependency=afterok:jobid</depend_string>
@@ -206,8 +218,8 @@
         <directive>-l nodes={{ num_nodes }}:ppn={{ tasks_per_node }}</directive>
       </directives>
       <queues>
-	<queue walltimemax="01:00:00" jobmin="1" jobmax="64" strict="true">shared</queue>
-	<queue walltimemax="03:00:00" jobmin="1" jobmax="4096" default="true">batch</queue>
+	<queue walltimemax="01:00:00" nodemin="1" nodemax="4" strict="true">shared</queue>
+	<queue walltimemax="03:00:00" nodemin="1" nodemax="256" default="true">batch</queue>
       </queues>
     </batch_system>
 
@@ -218,15 +230,15 @@
         <directive>-l nodes={{ num_nodes }}:ppn={{ tasks_per_node }}</directive>
       </directives>
       <queues>
-	<queue walltimemax="01:00:00" jobmin="1" jobmax="4320" default="true">acme</queue>
+	<queue walltimemax="01:00:00" nodemin="1" nodemax="120" default="true">acme</queue>
       </queues>
     </batch_system>
 
     <batch_system MACH="bebop" type="slurm" >
       <queues>
-	<queue walltimemax="00:30:00" jobmin="1" jobmax="2304" strict="true">debug</queue>
-        <queue walltimemax="01:00:00" jobmin="1" jobmax="21888" default="true">bdw</queue>
-        <queue walltimemax="01:00:00" jobmin="1" jobmax="18432">knl</queue>
+	<queue walltimemax="00:30:00" nodemin="1" nodemax="64" strict="true">debug</queue>
+        <queue walltimemax="01:00:00" nodemin="1" nodemax="608" default="true">bdw</queue>
+        <queue walltimemax="01:00:00" nodemin="1" nodemax="512">knl</queue>
       </queues>
     </batch_system>
 
@@ -237,14 +249,14 @@
       <directive>-l  nodes={{ num_nodes }}</directive>
     </directives>
     <queues>
-      <queue walltimemax="00:30:00" jobmin="1" jobmax="9999" default="true">batch</queue>
+      <queue walltimemax="00:30:00" nodemin="1" nodemax="312" default="true">batch</queue>
     </queues>
    </batch_system>
 
     <batch_system MACH="edison" type="slurm" >
       <queues>
-	<queue walltimemax="00:30:00" jobmin="1" jobmax="12288" strict="true">debug</queue>
-        <queue walltimemax="01:30:00" jobmin="1" jobmax="150000" default="true">regular</queue>
+	<queue walltimemax="00:30:00" nodemin="1" nodemax="512" strict="true">debug</queue>
+        <queue walltimemax="01:30:00" nodemin="1" nodemax="6250" default="true">regular</queue>
       </queues>
     </batch_system>
 
@@ -253,8 +265,8 @@
         <directive> --constraint=haswell</directive>
       </directives>
       <queues>
-	<queue walltimemax="00:30:00" jobmin="1" jobmax="4096" strict="true">debug</queue>
-        <queue walltimemax="01:00:00" jobmin="1" jobmax="10000" default="true">regular</queue>
+	<queue walltimemax="00:30:00" nodemin="1" nodemax="128" strict="true">debug</queue>
+        <queue walltimemax="01:00:00" nodemin="1" nodemax="312" default="true">regular</queue>
       </queues>
     </batch_system>
 
@@ -263,27 +275,27 @@
         <directive> --constraint=knl,quad,cache</directive>
       </directives>
       <queues>
-	<queue walltimemax="00:30:00" jobmin="1" jobmax="100000" strict="true">debug</queue>
-        <queue walltimemax="01:00:00" jobmin="1" jobmax="3000000" default="true">regular</queue>
+	<queue walltimemax="00:30:00" nodemin="1" nodemax="390" strict="true">debug</queue>
+        <queue walltimemax="01:00:00" nodemin="1" nodemax="11718" default="true">regular</queue>
       </queues>
     </batch_system>
 
     <batch_system MACH="mira" type="cobalt">
       <queues>
-        <queue walltimemax="03:00:00" jobmin="1" jobmax="786432" default="true">default</queue>
+        <queue walltimemax="03:00:00" nodemin="1" nodemax="12288" default="true">default</queue>
       </queues>
     </batch_system>
 
     <batch_system MACH="cetus" type="cobalt">
       <queues>
-        <queue walltimemax="01:00:00" jobmin="1" jobmax="65536" default="true">default</queue>
+        <queue walltimemax="01:00:00" nodemin="1" nodemax="1024" default="true">default</queue>
       </queues>
     </batch_system>
 
     <batch_system MACH="theta" type="cobalt_theta">
       <queues>
-        <queue walltimemin="00:30:00" walltimemax="02:00:00" jobmin="512" jobmax="231936" default="true">default</queue>
-        <queue walltimemax="01:00:00" jobmin="1" jobmax="1024" strict="true">debug-cache-quad</queue>
+        <queue walltimemin="00:30:00" walltimemax="02:00:00" nodemin="4" nodemax="1812" default="true">default</queue>
+        <queue walltimemax="01:00:00" nodemin="1" nodemax="8" strict="true">debug-cache-quad</queue>
       </queues>
     </batch_system>
 
@@ -293,7 +305,7 @@
 	<directive>[email protected]</directive>
       </directives>
       <queues>
-	<queue walltimemax="00:30:00" jobmin="1" jobmax="9999" default="true">small</queue>
+	<queue walltimemax="00:30:00" nodemin="1" nodemax="624" default="true">small</queue>
       </queues>
     </batch_system>
 
@@ -305,7 +317,7 @@
        <directive>--error=slurm.err</directive>
     </directives>
     <queues>
-      <queue walltimemax="00:59:00" jobmin="1" jobmax="9999" default="true">slurm</queue>
+      <queue walltimemax="00:59:00" nodemin="1" nodemax="416" default="true">slurm</queue>
     </queues>
    </batch_system>
 
@@ -316,10 +328,10 @@
        <directive>[email protected]</directive>
        <directive>--output=slurm.out</directive>
        <directive>--error=slurm.err</directive>
-    </directives>
-    <queues>
-      <queue walltimemax="00:59:00" jobmin="1" jobmax="9999" default="true">slurm</queue>
-    </queues>
+     </directives>
+     <queues>
+       <queue walltimemax="00:59:00" nodemin="1" nodemax="1249" default="true">slurm</queue>
+     </queues>
    </batch_system>
 
   <batch_system MACH="skybridge" type="slurm" >
@@ -423,8 +435,8 @@
        <directive>-env "all"</directive>
      </directives>
      <queues>
-       <queue walltimemax="02:00:00" jobmin="0" jobmax="299008" default="true">batch</queue>
-       <queue walltimemax="01:00:00" jobmin="0" jobmax="299008" strict="true">debug</queue>
+       <queue walltimemax="02:00:00" nodemin="0" nodemax="18688" default="true">batch</queue>
+       <queue walltimemax="01:00:00" nodemin="0" nodemax="18688" strict="true">debug</queue>
      </queues>
    </batch_system>
 
@@ -433,7 +445,7 @@
        <directive>-P {{ project }}</directive>
      </directives>
      <queues>
-       <queue walltimemax="01:00" jobmin="0" jobmax="8640" default="true">batch</queue>
+       <queue walltimemax="01:00" nodemin="0" nodemax="54" default="true">batch</queue>
 	   <!--
 	   jobmax = 54nodes*20cores*16th = 8640
 	   Nodes	Max Walltime
@@ -451,7 +463,7 @@
        <directive>--ntasks-per-node={{ tasks_per_node }}</directive>
      </directives>
     <queues>
-      <queue walltimemax="01:00:00" jobmin="0" jobmax="64" default="true">lr3</queue>
+      <queue walltimemax="01:00:00" nodemin="0" nodemax="6" default="true">lr3</queue>
     </queues>
    </batch_system>
 
@@ -463,7 +475,7 @@
        <directive>--ntasks-per-node={{ tasks_per_node }}</directive>
      </directives>
     <queues>
-      <queue walltimemax="01:00:00" jobmin="0" jobmax="64" default="true">lr3</queue>
+      <queue walltimemax="01:00:00" nodemin="0" nodemax="4" default="true">lr3</queue>
     </queues>
    </batch_system>