cwltool 2023

[mr-c]

• CWL: upgrade to last cwltool version with Python3.6 support
• tox fixes

[tiborsimko]

Hi @mr-c Thanks for the pull request! Note that in order to upgrade cwltool to the latest version, we need not only to update the pinned version here, but also to adapt the reana-workflow-engine-cwl how the cwltool is being invoked by REANA. This can be quite a lot of work in case there were internal API changes to cwltool touching how REANA calls it.

To cite an example, here is what was necessary to do during the last upgrade: https://github.com/reanahub/reana-workflow-engine-cwl/pull/192/files as well as some related PRs to be able to run the conformance tests, such as this one https://github.com/reanahub/reana-client/pull/538/files

Do you see many possibly breaking changes in cwltool between 3.1.20210628163208 and 3.1.20230719185429 versions? If you think it is a good time to make a cwltool upgrade, then we could make an issue about it under https://github.com/reanahub/reana-workflow-engine-cwl/issues to centralise all the necessary work for the upgrade to happen.

[mr-c]

@tiborsimko Yes, it is likely there are significant changes; what's the easiest way to test? ./run-tests.sh --check-pytest only ran 1 trivial test for me

[tiborsimko]

@tiborsimko Yes, it is likely there are significant changes; what's the easiest way to test?

One would have to install REANA locally in development mode, compile new reana-workflow-engine-cwl image using the new cwltool version coming from reana-commons, deploy REANA cluster, and test there. We have some documentation on how to do this (see e.g. here) but it's a non-trivial process if API underwent some changes. Please let me know if you'd like to try it out yourself? Otherwise we can try to have a look at around Q1 2024 or so?