rfc21: add PRIORITY state

spec_21.rst

@@ -95,7 +95,13 @@ DEPEND
    The job is blocked waiting for dependencies to be satisfied. The job manager
    makes a request to the dependency service and receives a response once
    the job’s dependencies are satisfied, then logs the ``depend`` event.
-   The state transitions to SCHED.
+   The state transitions to PRIORITY.
+
+PRIORITY
+   The job is waiting for a priority to be assigned by the job manager
+   priority plugin.  Upon priority assignment, the job manager posts a
+   ``prioritize`` event.  The state transitions to SCHED.
+   Note: this event is internal and is not posted to the job eventlog [#f2]_.
 
 SCHED
    The job is blocked waiting for resources. The job manager sends an
@@ -129,7 +135,7 @@ In the interest of encouraging consistent language, we define the following
 "virtual states" as shorthand for the union of two or more actual job states:
 
 PENDING
-  The job is in DEPEND or SCHED states.
+  The job is in DEPEND, PRIORITY, or SCHED states.
 
 RUNNING
   The job is in RUN or CLEANUP states.
@@ -175,7 +181,7 @@ Job was submitted.
 The following keys are REQUIRED in the event context object:
 
 priority
-   (integer) Initial priority in the range of 0-31.
+   (integer) Initial administrative priority in the range of 0-31.
 
 userid
    (integer) Authenticated user ID of submitter.
@@ -190,15 +196,32 @@ Example:
    {"timestamp":1552593348.073045,"name":"submit","context":{"priority":16,"userid":5588,"flags":0}}
 
 
+Prioritize Event
+^^^^^^^^^^^^^^^^
+
+Job has been (re-)assigned a priority by the job shell priority plugin:
+
+The following keys are REQUIRED in the event context object:
+
+priority
+   (double) New priority in the range of 0-1.0.
+
+.. code:: json
+
+   {"timestamp":1552593547.411336,"name":"prioritize","context":{"priority":0.5555555555555}}
+
+Note: this event is internal and is not posted to the job eventlog [#f2]_.
+
+
 Priority Event
 ^^^^^^^^^^^^^^
 
-Job is to be re-prioritized.
+Job's administrative priority has changed.
 
 The following keys are REQUIRED in the event context object:
 
 priority
-   (integer) New priority in the range of 0-31.
+   (integer) New administrative priority in the range of 0-31.
 
 userid
    (integer) Authenticated user ID of requester.
@@ -283,7 +306,7 @@ The following keys are REQUIRED in the event context object:
 
 status
    (integer) The largest of the job shell wait status codes, as
-   defined by POSIX wait(2) [#f1]_
+   defined by POSIX wait(2) [#f1]_.
 
 Example:
 
@@ -379,22 +402,15 @@ Once a given state has been signaled (with a KVS snapshot reference), the
 following invariants hold with respect to the KVS job schema described in
 RFC 16:
 
-DEPEND
-   TBD
-
-SCHED
-   TBD
-
-RUN
-   TBD
-
 CLEANUP
    Either an exception has been logged to ``job.<jobid>.eventlog``,
    or a global status code from the application is available (TBD).
 
 INACTIVE
    ``job.<jobid>`` contains the final snapshot of the job schema.
 
+.. [#f2] The ``prioritize`` event is not posted to the job eventlog, since an updated priority can be easily recalculated and some priority plugins may frequently adjust pending jobs.  As a consequence, a job may regress from SCHED to PRIORITY when Flux restarts.
+
 .. [#f1] `wait, waitpid - wait for a child process to stop or terminate <http://pubs.opengroup.org/onlinepubs/009604499/functions/wait.html>`__; The Open Group Base Specifications Issue 6; IEEE Std 1003.1, 2004 Edition
 
 .. |states| image:: data/spec_21/states.svg