-
Notifications
You must be signed in to change notification settings - Fork 94
ISO 8601
How things might possibly look if we used ISO 8601:2000 formatted date/times, durations, and recurrence syntax in cylc.
ISO 8601 is not just about writing dates and times in a universal format - it's also about writing durations and recurring intervals of time. These are commonly used in cylc.
We have quite a few issues that hang on expressing when exactly tasks run - so if we're looking at ISO 8601, we might as well explore if it can help us with these issues.
Scroll to the bottom to see some examples.
See #119 for the ISO 8601 cylc issue
See #119 comment for a good ISO 8601 summary by @m214089.
See ISO 8601:2004 for the ISO 8601 2004 version.
For example:
[[dependencies]]
[[[6,12]]]The examples use an initial cycle time of 20130325 and a final cycle time of 20130405T12.
| Purpose | Current | Proposed | Expanded | Notes | In Context |
|---|---|---|---|---|---|
| Repeat every day starting at 06:00 after initial cycle time | 6 |
T06 |
R/T06/P1D |
implicit P1D
|
R/20130325T06/P1D |
| As above, also at 12:00 | 6, 12 |
T06, T12 |
R/T06/P1D, R/T12/P1D
|
as above |
R/20130325T06/P1D, R/20130325T12/P1D
|
| Repeat every month on the first of the month after the initial cycle time | Monthly(...) |
01T |
R/01T/P1M |
implicit P1M
|
R/20130401/P1M |
| Repeat every month on the 14th of the month after the initial cycle time | Monthly(...) |
14T |
R/14T/P1M |
as above | R/20130414/P1M |
| Repeat every year on April 2nd after the initial cycle time | Yearly(...) |
0402T |
R/0402T/P1Y |
implicit P1Y
|
R/20130402/P1Y |
| Repeat every 2 days, starting at 06:00 after initial cycle time | Daily(...) |
T06/P2D |
R/T06/P2D |
- | R/20130325T06/P2D |
| Repeat hourly, starting at 18:30 after initial cycle time | N/A | T1830/P1H |
R/T1830/P1H |
- | R/20130325T1830/P1H |
| Repeat hourly, starting at the initial cycle time | 0,1,2,3,4.... |
P1H |
R//P1H |
implicit initial cycle time | R/20130325/P1H |
| Repeat every 5 minutes, starting at the initial cycle time | N/A | PT5M |
R//PT5M |
as above | R/20130325/PT5M |
| Repeat weekly, starting 1 day after initial cycle time | N/A | +P1D/P1W |
R/+P1D/P1W |
+/-P?? is an offset |
R/20130326/P1W |
| Repeat every day, three times, starting at 12:00 after the initial cycle time | N/A | R3/T12/P1D |
R3/T12/P1D |
- | R3/2013032512/P1D |
| Repeat once, at the 1st of the month after the initial cycle time | N/A | R1/01T |
R1/01T/P?? |
no duration necessary | R1/20130401/P?? |
| Repeat once, at this (initial cycle time) date |
initial? |
R1 |
R1/??/P?? |
implied initial cycle time | R1/20130325/P?? |
[cylc]
UTC mode = True
[scheduling]
initial cycle time = 2013032500
final cycle time = 2013040412
[[special tasks]]
cold-start = a_cold
[[dependencies]]
[[[ 6 ]]]
graph = "a[T-24] | a_cold => a"
[[[ 0, 12 ]]]
graph = "b => c"
[[[ 0, 6, 12, 18 ]]]
graph = "d[T-6] | d_cold => d"[cylc]
UTC mode = True
[scheduling]
initial cycle time = 20130325
final cycle time = 20130404T12
[[dependencies]]
[[[ R1/T06 ]]] # 1
graph = """
a_cold => a
d_cold => d
"""
[[[ T06 ]]]
graph = "a[-1D] => a"
[[[ T06, T12 ]]]
graph = "b => c"
[[[ T06/P6H ]]] # 2
graph = "d[-6H] => d"-
- Repeat once, first occurrence of
T06H, impliesR1/20130325T06/P??(undefined period of repetition).
- Repeat once, first occurrence of
-
- Repeat every 6 hours, starting at 06:00 (
R/20130325T06/P6H).
- Repeat every 6 hours, starting at 06:00 (
[scheduling]
initial cycle time = 20130325
final cycle time = 20130404T12
[[dependencies]]
graph = "install_cold"
[[[ 0 ]]]
graph = "install_cold & foo[T-24] => foo"
[[[ 6 ]]]
graph = "install_cold & bar[T-24] => bar"[scheduling]
initial cycle time = 20130325
final cycle time = 20130404T12
[[dependencies]]
[[[ R1 ]]]
graph = "install_cold"
[[[ R1/T00 ]]]
graph = "install_cold[^] => foo"
[[[ R1/T04]]]
graph = "install_cold[^] => bar"
[[[ T00 ]]]
graph = "foo[-P1D] => foo"
[[[ T04 ]]]
graph = "bar[-P1D] => bar"We expect all new syntax to be distinguishable from the current syntax (e.g. T06 rather than 6). This means a special suite.rc flag isn't required (e.g. ISO mode = True).
However, as cycle time conventions will change for output files and directories, we expect there may be a requirement in the short term to keep the old convention by e.g. having cycle time format = YYYYMMDDhhmm or old cycle time mode = True...
This proposes writing suite times, durations, and recurrences in an abbreviated form of the ISO 8601 format (also allowing the formal syntax). We use the context for an abbreviated element to generate the formal syntax.
Cycle times (as written in the suite.rc, such as initial cycle time and final cycle time) are date/times that specify the context date/times for some non-absolute times and recurring time intervals in the suite.
They should be specified using the ISO 8601 Date/Time format in basic, compact form.
Dates and times, in the basic, compact format that we want to use, are represented like this:
- Normal year/month/day dates, no times, (standard and expanded representation):
YYYYMMDD e.g. 20131225
YYYY e.g. 2013
YY e.g. 20 (century) implying 2000.
+YYYYYYMMDD, where extra Y stands for an agreed number of extra years e.g. +0020131225
We should agree to use two extra year digits. The plus is required.
-YYYYYYMMDD, e.g. -0000550101 (56 BC)
+YYYYYY e.g. +002013
-YYYYYY e.g. -000055
+YYYY e.g. +0020 (century) implying 2000.
-YYYY e.g. -0000 (century) implying -0000.
-
Week number and day number may also be specified.
-
It is possible to specify omission of components using hyphens - see the standard for details.
-
Times:
hhmmss e.g. 061000 (ten past 6 in the morning).
-
Decimal fractions are allowed, with a comma (recommended) or full stop as the decimal separator.
-
Omitted trailing components assumed zero.
-
Dates and times:
YYYYMMDDThhmmss
YYYYMMDDThhmmssZ
YYYYMMDDThhmmss+hhmm
YYYYMMDDThhmmss-hhmm
(same with +YYYYYY and -YYYYYY)
If UTC mode = True, dates and times are implicitly Zulu time (Z). Otherwise, if Z is not specified in the initial cycle time, the local timezone will be used for it. The same applies independently for the final cycle time.
initial cycle time and final cycle time may use any of the above formatting. We would suggest that most suites use the 4-digit year specification and that the +/- 6 digit year form should be used only if required. final cycle time can also be specified relative to initial cycle time, as in the
dependencies section headings explained below.
Sub-minute information will not be considered (will be set to zero) and may be flagged as an error on suite validation.
Decimal fractions are allowed in the ISO standard, but given that we don't support sub-minute information, this is unnecessary and we won't support it in cylc.
These should use the same form as initial cycle time and final cycle time, but there is a debate about the precision of the format. There are (at least) 4 options:
- Complete precision, plus/minus 6 digit year down to minute level (
+YYYYYYMMDDThhmmor-YYYYYYMMDDThhmm) - Complete precision, 4 or plus/minus 6 digit year down to minute level (depending on usage).
- Dynamic precision, minimum necessary to uniquely represent all cycle times.
- User-configured minimum precision (warn if the user does not use enough precision) - e.g. climate suites may want to just use
YYYYMM.
The decision was made to go for year-dynamic output and task label formats down to the minute level, [EDIT:used to be 'without'] with timezone information.
Section names specify a recurrence or series, not a single date/time.
- This is true for both the proposed and current cylc format. In the current syntax:
[[[ 6 ]]]means "every day at 0600", not "0600". The Yearly() type of syntax is more explicit about this.
- We use either the full or an abbreviated form of the ISO 8601 format for recurring times
- Abbreviations are extrapolated given the context and some simple rules below.
- Power usage can make use of the full ISO 8601 recurrence syntax.
The ISO 8601 standard allows for four ways of specifying a recurring series of date/times. The way best suited for abbreviation in this context is the third:
Rn/START_TIME/PERIOD
which looks something like this:
Rn/YYYYMMDDThhmmss/PnYnMnDTnHnMnS
- R is a character that marks this as a recurring pattern. The first n is an optional number that limits the number of recurrences.
-
R1implies a single repetition -
R5implies 5 repetitions -
YYYYMMDDThhmmssis a start date/time expressed in standard ISO 8601 format . ISO 8601 allows truncation of the time to, for example,Thhif the parties agree on the context (our context start time is the initial cycle time - more later). -
01Timplies00:00:00on the first day of a month (exactly which month depends on the context) -
T06implies 6 a.m. -
20implies the start of the century beginning with20i.e.20000101T000000 -
PnYnMnDTnHnMnSis the ISO 8601 duration syntax. ThePcharacter marks this as a duration. This is followed by (optionally specified) sets of numbers (n) and units:nYfor number of years,nMfor number of months, and so on. If a set of number and unit is missing, it is assumed zero. For example, -
PT6His a 6 hour duration. -
P1Y6Mis a 1 year + 6 months duration -
PT6Mis a 6 minute duration -
P6Mis a 6 month duration - Using weeks is also allowed e.g.
P3Wfor every 3 weeks.
We should also support the 1st and 4th ISO 8601 recurrence syntax specifications:
Rn/START_TIME/END_TIME (number is not optional) - e.g. R6/20130401/20131001 implies repeat 6 times between the start and end times, which in this case is every month.
Rn/PERIOD/END_TIME - e.g. R2/P1D/20130415 implies repeat twice every day counting backwards from the end time, which means dates of 20130414 and 20130415.
Multiple, comma-separated recurrence specifications may be used within a single section (as with the current format).
A date/time followed by a forward slash (also called a solidus) and an ISO 8601 period (duration syntax) implies a leading R/ in ISO 8601 form. This means that T00/P2D implies R/T00/P2D.
A date/time without a period (e.g. T06) showing a particular set of date/time components implies a period that is equal to 1 of the next largest date/time component.
-
T06impliesR/T06/P1D(1 day period). -
01T(1st day in a month) would implyR/01T/P1M(1 month period). -
-85(the year 85 in a century) would implyR/-85/P100Y(a century period). -
20(a century in ISO 8601, implying 2000) or2000is invalid without a period, as there is no larger component.
Within the full syntax and the abbreviations, truncated start times (e.g. T06) imply the next such time immediately following (and including) the initial cycle time.
- For example:
[[[ T06 ]]]
graph = "a"implies:
[[[ R/T06/P1D ]]]
graph = "a"which, with an initial cycle time of 20130325T00, implies R/20130325T06/P1D. The same expression given an initial cycle time of 20130325T12 would imply R/20130326T06/P1D, the day afterwards. An initial cycle time of 20130325T06 would imply R/20130325T06/P1D, equal to the initial cycle time.
When using the full fourth ISO 8601 recurring format definition (rare, not explained here), truncated end times similarly imply the final cycle time.
A null string for the start time implies the initial cycle time:
[[[ R//P1D ]]]
graph = "a"If we have an initial cycle time of 20130325T12, this implies R/20130325T12/P1D - repeat every day starting at the initial cycle time. When using the 4th ISO recurrence definition (R/PERIOD/END_TIME), a null string for the end time implies the final cycle time - e.g. R4/P1D implies repeat every day counting backwards from the final cycle time.
Allowing a null string for the start time implies that R//P1D can be abbreviated as P1D - an expression beginning with 'P' will be parsed as a duration and imply a recurring series of times starting at the initial cycle time. We will allow the T designator to be absent when the context is clear (it is necessary for distinguishing months and minutes). This means that P6H implies R//PT6H (a null start time), which for an initial cycle time of 20130325T00 implies R/20130325T00/PT6H. This means that dependencies that used to be written as 0, 6, 12, 18 can be written as P6H, if the initial cycle time begins at one of 00:00, 06:00, 12:00, or 18:00.
The ISO 8601 format supports writing the number of repetitions to be specified - R1/T06/P1D implies that this is only run once at T06 immediately following (or at) the initial cycle time. This has applications for cold start tasks. Writing just R1 must imply R1//P??? (undefined duration, any number) which, for an initial cycle time of 20130325T00 implies R1/20130325T00/P???? which is a task that runs exactly at the initial cycle time and not again.
We may want to specify our own way of writing a date/time plus a duration (period) as a way of writing an offset date/time. This should be written as a date/time followed by a positive or negative duration (sign is compulsory). In absolute form with a 4-digit year, it would look like this:
YYYYMMDDThhmm+PnYnMnDnHnM
YYYYMMDDThhmm-PnYnMnDnHnM
The above rules for writing abbreviated dates and times would then apply, so we could have:
Thhmm+PnYnMnDnHnM
For example:
T06+P1D
in the context of:
R/T06+P1D
which implies:
R/T06+P1D/P1D
means "repeat every day at 0600 immediately following (or at) the initial cycle time plus 1 day". An initial cycle time of 20130401 would then imply repeating at:
20130402T06
20130403T06
20130404T06
...
and so on.
The use of the plus/minus sign in the duration implies that we can distinguish a duration from a null time string with an offset, so:
R/+P2D/P1D
means repeat every day, starting two days after the initial cycle time (our usual R/START_TIME/PERIOD recurrence syntax). For an initial cycle time of 20130401, this implies R/20130401+P2D/P1D which implies R/20130403/P1D. Similarly,
R/P2D/+P1D
is of the form R/PERIOD/END_TIME which uses the 4th ISO recurrence syntax definition and implies repeat every 2 days counting backwards from 1 day after the final cycle time. For a final cycle time of 20130515, this implies R/P2D/20130515+P1D which implies R/P2D/20130516.
Tasks can refer to tasks from other cycle times. They therefore need to specify which time this is, whether an absolute time or relative to their own cycle time.
(Copied from the Dependencies Sections text)
PnYnMnDTnHnMnS is the ISO 8601 duration syntax. The P character marks this as a duration. This is followed by (optionally specified) sets of numbers (n) and units: nY for number of years, nM for number of months, and so on. If a set of number and unit is missing, it is assumed zero. For example,
-
PT6His a 6 hour duration. -
P1Y6Mis a 1 year + 6 months duration -
PT6Mis a 6 minute duration -
P6Mis a 6 month duration
We allow absolute times to be used (but don't expect them to be used).
Null string is treated as a truncated absolute time, as a special case.
The context for a truncated absolute time is the initial cycle time. Truncated absolute times are evaluated as in the dependencies section.
-
d[T06]implies 6 a.m. immediately following the current cycle time. -
d[^](caret or hat) implies the initial cycle time (analogous to how^means the beginning of the string in a regular expression). -
d[^T06]implies 6 a.m. immediately following the initial cycle time. -
d[^+P1D]implies 1 day after the initial cycle time. -
d[$](dollar) implies the final cycle time. -
d[$-P1D]implies 1 day before the final cycle time.
Relative times are distinguished by the use of durations. The context for a relative time is the cycle time of the task itself, not the initial cycle time.
We can then specify the relative time as a positive or negative time interval (duration), to be applied to the cycle time of the task itself.
The ISO 8601 format for durations should be supported.
We should allow the T designator to be optional where the usage is unambiguous.
In this particular context, we should also support (and use as a standard) removing the leading P in the duration format.
This means that d[-PT6H] => d can be written as d[-6H] => d.
There are some other options in cylc that are lengths of time and should be in the ISO 8601 duration format for completeness:
[cylc]
[[job submission]]
delay between batches = integer( min=0, default=0 ) # seconds
[[event handler submission]]
delay between batches = integer( min=0, default=0 ) # seconds
[[poll and kill command submission]]
delay between batches = integer( min=0, default=0 ) # seconds
[[event hooks]]
timeout = float( default=None )
[[accelerated clock]]
# Note: not rate, as this is seconds per hour and dimensionless.
offset = integer( default=24 ) # Hours
[[reference test]]
live mode suite timeout = float( default=None )
dummy mode suite timeout = float( default=None )
simulation mode suite timeout = float( default=None )
[scheduling]
runahead limit = integer( min=0, default=None )
[runtime]
[[__many__]]
retry delays = force_list( default=list() )
submission polling intervals = force_list( default=list() )
execution polling intervals = force_list( default=list() )
[[[simulation mode]]]
run time range = list( default=list(1,16))
[[[job submission]]]
retry delays = force_list( default=list() )
[[[event hooks]]]
submission timeout = float( default=None )
execution timeout = float( default=None )
[visualization]
initial cycle time = integer( default=None )
final cycle time = integer( default=None )
[[runtime graph]]
cutoff = integer( default=24 )We may use some keywords to make some things more user-friendly, especially for the initial "R1" section.
The 2004 version of the ISO 8601 standard removes the official date/time truncated representation (using "-") which is very useful. This document was written with the 2000 version in mind - however, we could just repurpose the "-" as part of our special abbreviation syntax and call it 2004-based.
How does warm-starting a suite fit with initial and initial offset graph sections?
- The removal of dependence on previous cycles removes the need for cold-start "succeeded" tasks being inserted to get things going - a warm start will effectively keep the
initial cycle timeas is and start mid-way through.
How should altering the initial cycle time on the command line in cylc run affect the suite?
- We should distinguish "start time" from "initial cycle time" - see warm-start comment above.
How should altering the final cycle time affect the suite if there are final graph sections?
- It must necessarily alter the final graph sections - however, setting a "STOP" time should not affect the graph.
[cylc]
UTC mode = True
[scheduling]
initial cycle time = 20130325
final cycle time = 20130404T12
[[dependencies]]
# Simple dependency section examples:
[[[ R1 ]]] # = R1/20130325/P??
graph = "a_initial" # Repeat once at initial cycle time
[[[ R1/+P6H ]]] # = R1/20130325+P6H/P?? = R1/20130325T06/P??
graph = "a0 => b0" # Repeat once, 6 hours after the initial cycle time
[[[ T06, T12 ]]] # = R/20130325T06/P1D, R/20130325T12/P1D
graph = "a1 => b1" # Repeat every day, starting at 06:00.
# Repeat every day, starting at 12:00.
[[[ R1/T06 ]]] # = R1/20130325T06/P1D
graph = "a2 => b2" # Repeat once, at the next 06:00.
[[[ 01T ]]] # = R1/20130401/P1M
graph = "a3 => b3" # Repeat every month, starting at the first day of the month.
[[[ P3H ]]] # = R1/20130325/P3H
graph = "a4 => b4" # Repeat every 3 hours, starting at the initial cycle time.
[[[ T00/P7H ]]] # = R/20130325T00/P7H
graph = "a5 => b5" # Repeat every 7 hours, starting at the next 00:00 am after
# (or including) the initial cycle time.
# Advanced-use dependency section examples:
[[[ R5//P6H ]]] # = R5/20130325/P6H
graph = "a6 => b6" # Repeat five times, every 6 hours, starting at the initial cycle time
[[[ 01T/P3Y4DT3M ]]] # = R/20130401/P3Y4DT3M
graph = "a7 => b7" # Repeat every 3 years, 4 days, and 3 minutes,
# starting at the next 1st day of the month
[[[ W115/P1W, W-3T06/P2W ]]] # = R/2013W11/P1W, R/20130327T06/P2W
graph = "a8 => b8" # Repeat every week, starting on Friday in the next 11th ordinal
# week of a year.
# Repeat every fortnight, starting next Wednesday at 6 a.m.
[[[ R1/20130401 ]]] # = R1/20130401/P?? (=20130401).
graph = "a9 => b9" # Repeat once at the date/time 20130401
[[[ R3//PT4M ]]] # = R3/20130325/PT4M
graph = "a10 => b10" # Repeat three times at an interval of 4 minutes, starting at the
# initial cycle time. This means run at:
# 20130325T0000
# 20130325T0004
# 20130325T0008
[[[ R5/P2H ]]] # = R5/P2H/20130404T12 (final cycle time)
graph = "a11 => b11" # Using the fourth ISO recurrence definition, repeat five times counting
# backwards from the final cycle time every 2 hours.
# This means run at:
# 20130404T04
# 20130404T06
# 20130404T08
# 20130404T10
# 20130404T12
[[[ R3/P1D/T06 ]]] # = R3/P1D/20130405T06
graph = "a12 => b12" # Using the fourth ISO recurrence definition, repeat three times counting
# backwards 1 day from 06:00 immediately following (or including) the
# final cycle time.
# This means (post final cycle times do not run) run at:
# 20130403T06
# 20130404T06
# This avoids hard-coding the cycle time.
[[[ R1/P1W ]]] # = R1/P???/20130404T12
graph = "a13 => b13" # Using the fourth ISO recurrence definition, repeat once at the
# final cycle time (period is meaningless for R1 and could be anything).
[[[ R1/+P1D ]]] # = R1/20130325+P1D/P??? = R1/20130326/P???
graph = "a14 => b14" # Repeat once, 1 day after the initial cycle time.
[[[ R1//-P1D ]]] # = R1/P???/20130404T12-P1D = R1/P???/20130403T12
graph = "a15 => b15" # Repeat once, 1 day before the final cycle time.
[[[ R/+P6H/P1D ]]] # = R/20130325+P6H/P1D = R/20130325T06/P1D
graph = "a16 => b16" # Repeat every day, starting at the initial cycle time plus 6 hours.
# Simple dependency graph examples:
[[[ T00 ]]]
graph = "a[-PT6M] => b" # b depends on a from 6 minutes ago.
[[[ T01 ]]]
graph = "a[-PT6H] => b" # b depends on a from 6 hours ago.
[[[ T02 ]]]
graph = "a[-P1Y] => b" # b depends on a from a year ago.
[[[ T03 ]]]
graph = "a => a[T06]" # a triggers future a at the next T06 occurrence
# Advanced-use dependency graph examples:
[[[ T04 ]]]
graph = "a[20130326T06] => b" # b depends on a at the absolute cycle time of `20130326T06`
[[[ T05 ]]]
graph = "a[^] => b" # b depends on a at the initial cycle time (special case)
[[[ T06 ]]]
graph = "a[^+PT4H] => b" # b depends on a at 4 hours after the initial cycle time
[[[ T07 ]]]
graph = "a[^T07] => b" # b depends on a at `07:00` following the initial cycle timeA relative cycle dependency under the new syntax will look like:
[[[ T00 ]]]
graph = "a[-P1D] => a"
(which is the same as:
[[[ 0 ]]]
graph = "a[T-24] => a"
in the current syntax)
This syntax defines dependencies relative to the current cycle. However, we will support absolute cycle dependencies in the future, such as depending on cycles at or near the initial cycle time.
At the moment we have the following syntax for writing absolute cycle dependencies in graphs: (bottom of https://github.com/cylc/cylc/wiki/ISO-8601):
[[[ T04 ]]]
graph = "a[20130326T06] => b" # b depends on a at the absolute cycle time of `20130326T06`
[[[ T05 ]]]
graph = "a[] => b" # b depends on a at the initial cycle time (special case)
[[[ T06 ]]]
graph = "a[][+PT4H] => b" # b depends on a at 4 hours after the initial cycle time
[[[ T07 ]]]
graph = "a[][T07] => b" # b depends on a at `07:00` following the initial cycle time
(we would usually expect these kinds of absolute dependencies to be one-off R1 or other limited repetitions)
The first case is fine, but I'm kind of uncomfortable with the
rest - a blank [] seems more likely to mean the current cycle
rather than the initial cycle, given the relative syntax at the
top of the email.
I do quite like the idea of writing the T04 case as:
[[[ T04 ]]]
graph = "a.20130326T06 => b" # b depends on a at the absolute cycle time of `20130326T06`
This follows the task_name.cycle_point usage elsewhere in cylc.
This would imply:
[[[ T05 ]]]
graph = "a. => b" # b depends on a at the initial cycle time
[[[ T06 ]]]
graph = "a.+PT4H => b" # b depends on a at 4 hours after the initial cycle time
[[[ T07 ]]]
graph = "a.T07 => b" # b depends on a at `07:00` following the initial cycle time
I think this is a little clearer than the null square bracket usage, but it still isn't perfect.
Ben
Ben,
I like the suggestion, and generally prefer it to the square-bracket notation, but maybe foo. => bar still looks too much like "foo => bar"? What about a special symbol to represent the initial cycle, e.g. "t1"? In fact it could be worse than an initial cycle problem - I wonder if we might need the ability to have several future cycles trigger off a task at an earlier R1 that is not the initial cycle. How about requiring a cycle label in the section heading?:
[[[R1:t1]]] # initial cycle
graph = "start => foo"
Then in other sections, "start.t1 => bar".
Hilary
A symbol might be OK - but it needs to naturally combine with e.g. "T06", which makes it difficult.
An @ might be a better delimiter than ".", thinking about it.
I see your point about the earlier R1 (although we can do something like
foo[][-P2D]
foo.-P2D
foo.-P2D
foo@-P2D
to say 2 days before the initial cycle time). Your idea of 'tagging' a particular date/time with a user-defined symbol is a really nice power feature.
Ben
Keep everything inside a single set of square brackets
Use ''^'' prefix for "relative to initial cycle point" meaning
Use ''$'' prefix for "relative to final cycle point" meaning
Evaluate named tags if we get a good use case down the line.
foo[^T00] => bar
foo[^+PT6H] => bar
foo[$-P1D] => bar
Warm-start currently resets this, so that R1 tasks do run - it might be
better if we altered warm start to use the suite.rc initial cycle time
(arguably forcing the user to provide it as another argument or option if it
is missing).
Don't let warm-start alter initial cycle point. Handle "pre-initial" cycling
as "pre-warm-start-cycle-point" cycling. Force the user to specify the
initial cycle point as an argument or option to cylc run --warm, if it is
not specified in the suite.rc.
What should happen if you change the initial cycle time and/or the final cycle time as part of a reload or restart?
Allow final cycle time to be changed mid-run. Do not allow initial cycle time to be changed mid-run (restart would fail, reload would fail).
We have used cycle point instead of cycle time in the cylc documentation,
partly due to the support for arbitrary cycling (e.g. integer).
The idea is to change this suite.rc option, as well as final cycle time and
the corresponding environment variables and $CYLC_TASK_CYCLE_TIME to be
_point, _POINT instead of _time, _TIME.
Change the syntax for all of these - but keep old environment variables for backwards compatibility, even in new-style suites. Old options will not be supported in new-style suites.
Rose will follow cylc, and support a $ROSE_TASK_CYCLE_POINT alongside
a deprecated $ROSE_TASK_CYCLE_TIME.
This has been superseded by a runahead factor, which is a multiple of the shortest occurring cycle interval in the suite (this is how the automatic runahead limit was calculated before).
However, it might be better to keep the old absolute how-many-hours variant of this for some suites.
runahead limit agreed to be generally more useful for normal users/suites. Longer-term changes may be possible (dynamic-per-task related?). Revert runahead limit removal, retire runahead factor.
Are we keeping this functionality? It currently does not work in the ISO branch.
Leave non-functional and remove the old code and documentation for now - but have a think about what to do. Maybe integer cycling could be used for this kind of use case.
What should happen when a user specifies tasks to run at the final cycle time, but does not provide a final cycle time? Should it fail validation? Should it give a warning?
(Ben thinks just a warning)
Just fail if the user does something like this. If they don't know a final cycle time in this case, they can always set a distant or putative final cycle time and then change it later with a reload.
This requires a cycle point format change mid-suite - Ben is seeing if this is easy to do. If not, is it worth it?
Don't allow if this happens during a run - stop the suite. However, we should check to make sure this is properly covered during suite validation.
other stuff:
- support ISO 8601 interval syntax for the hours/minutes/seconds inputs in the suite.rc file (solution: yes)
- document integer cycling? (solution: yes, after a look at the notation)