cylc-rose: cylc list compatibility

[wxtim]

These changes address #4288
These changes are twinned with cylc/cylc-rose#65

Make the following scripts work with cylc-rose:

Internal Todo List

• Cylc list
• Cylc graph
• Cylc config
• Cylc validate

Standard checklist

• I have read CONTRIBUTING.md and added my name as a Code Contributor.
• Contains logically grouped changes (else tidy your branch by rebase).
• Does not contain off-topic changes (use other PRs for other changes).
• Applied any dependency changes to both setup.py and
  conda-environment.yml.
• Does not need tests (tests are in cylc-rose).
• Appropriate change log entry included: update change log wxtim/cylc#30 (Can be merged before merging this PR, avoids conflict)
• Documented changes in cylc list docstring: Auto documents using the Cylc Rose args.

How to review:

Create workflows with a rose-suite.conf in

• ~/cylc-src/
• ~/cylc-run/ (using cylc install)
• some other folder somewhere else entirely

You might use jinja2 or empy in the workflow to carry out tests like these.

Try each of the four cylc CLI commands with and without the extra rose options provided by this function/

Ideally these commands and cylc install should behave in a consistent way.

[kinow]

File containing a list of tests which need to be run locally with cylc/cylc-rose#65 installed: These will fail on GH Actions even if this PR is OK.

fail_online.txt

I think you fixed this right? (In GH UI, we can mark a comment like this as "Outdated" and hide it using the three dots at the top right corner of the comment area)

[kinow]

Code and tests look good! Installing everything (flow + rose from both new branches) to test it now.

[kinow]

@wxtim I used @oliver-sanders ' example from the linked issue.

1. Created ~/cylc-src/rose1/flow.cylc with the following content:

[scheduling]
    initial cycle point = {{ ICP }}
    [[graph]]
        R1 = foo

Then created a ~/cylc-src/rose1/rose-suite.conf (not using myopt as in his example for now, will after this simpler example works).

[template variables]

ICP = "20000101T0000Z"

3. Installed the workflow

(venv) kinow@ranma:~/cylc-src/rose1$ cylc install
INSTALLED rose1/run1 from /home/kinow/cylc-src/rose1

4. Verified the files were installed/copied correctly

(venv) kinow@ranma:~/cylc-src/rose1$ ls ~/cylc-run/rose1/run1/
flow.cylc  log  opt  rose-suite.conf

5. Tried cylc validate rose1/run1

(venv) kinow@ranma:~/cylc-src/rose1$ cylc validate rose1/run1
IllegalValueError: (type=cycle point) [scheduling]initial cycle point = {{ ICP }}

Any idea what step I missed? I am trying @oliver-sanders ' example first as that was in the ticket, and looks like a simple way to test this PR. Will try more Jinja2/empy once I get this one working :)

Thanks!!!
Bruno

[wxtim]

@kinow

WF not working / Logging bug?

Any idea what step I missed? I am trying @oliver-sanders ' example first as that was in the ticket, and looks like a simple way to test this PR. Will try more Jinja2/empy once I get this one working :)

You might have not explicitly stated because it seems obvious, but I don't thing you have put a #!jinja2 line in your flow.cylc: This is fine if you are using the Rose 1 legacy rose-suite.conf:[jinja2:suite.rc] section heading, but with [template variables] there is no way way Cylc-Rose can deduce the templating language.

Created an issue: cylc/cylc-rose#67. I believe that this is a logging bug.

Outdated comments

I think you fixed this right? (In GH UI, we can mark a comment like this as "Outdated" and hide it using the three dots at the top right corner of the comment area)

So done

[kinow]

Ah! You are right @wxtim ! Added the shebang, and also the runtime section, now it's working fine.

#!jinja2

[scheduling]
    initial cycle point = {{ ICP }}
    [[graph]]
        R1 = foo

[runtime]
[[foo]]

(venv) kinow@ranma:~/cylc-src/rose1$ cylc install 
INSTALLED rose1/run1 from /home/kinow/cylc-src/rose1
(venv) kinow@ranma:~/cylc-src/rose1$ cylc validate rose1/run1
Valid for cylc-8.0b2.dev

Will finish the review tomorrow morning. Thanks again!! 👍

[kinow]

Tested the other commands, list, config, and graph. Everything worked fine. Changed the workflow to be:

#!empy

[scheduling]
    initial cycle point = @ICP
    [[graph]]
        R1 = foo

[runtime]
[[foo]]

And confirmed I had empty installed. Everything worked fine too :-) Great job @wxtim !

p.s.: we probably have a ticket somewhere to rename rose-suite.conf to rose-workflow.conf I think?

[wxtim]

p.s.: we probably have a ticket somewhere to rename rose-suite.conf to rose-workflow.conf I think?

No. A rose suite is a config for a Cylc workflow. No immediate plans to change the term.

[datamel]

Thanks @wxtim
Looks great. I have read the code, checked out the branch (with its sibling), ran the tests and the manual one as detailed above. I have also locally run the functional tests which were affected. No issues found for me.

[oliver-sanders]

@wxtim, just noticed this PR was marked as "partially addresses #4288", could you add a note to #4288 covering what's left for future work.