Implementation of store RDF export of the workflow in CWL Prov RO-Bundle #1709
Conversation
…dle. Added file size to prov RO-Bundle Creates a workflow.ttl file in the default folder while running. Then closes the file and after the provenance folder is created it is copied to the provenance/workflow/ folder. Update cwltool/main.py Co-authored-by: Michael R. Crusoe <[email protected]> temp directory creation and removal added
Codecov Report
@@ Coverage Diff @@
## main #1709 +/- ##
===========================================
- Coverage 83.91% 71.96% -11.95%
===========================================
Files 46 46
Lines 8150 8269 +119
Branches 2169 2167 -2
===========================================
- Hits 6839 5951 -888
- Misses 842 1832 +990
- Partials 469 486 +17
|
Fix needed for the type for the activity function in adding the workflow step
looking into accessing the arguments parser from the checksum_only option so we can add a --no-data like option to cwltool
…hould be used and if the SHA1 string can be a URI (faster indexing in triple stores)
There was a problem hiding this comment.
GH Actions is showing some linter errors. I think running make format should fix the code locally, then you can review if it modified only the files you modified, and push it again (sometimes it modifies more than what I changed, and then CI keeps failing; just ignore if make format changes more than what you expected).
Thanks many jobs now fail as well due to: As I am removing the provenance /data files and will turn it into an option later. This way we can use input files again that are ±100gb in size and prevent them from being copied to the data folder |
…e arguments from the provenance environment
…e can run it when there is no data in the data provenance folder.
|
This pull request introduces 6 alerts when merging 21ecba9 into f3a35f3 - view on LGTM.com new alerts:
|
|
This pull request introduces 6 alerts when merging 879d5ce into f3a35f3 - view on LGTM.com new alerts:
|
|
This pull request introduces 3 alerts when merging 211348a into f3a35f3 - view on LGTM.com new alerts:
|
|
This pull request introduces 3 alerts when merging bd61e43 into 0e2ced5 - view on LGTM.com new alerts:
|
|
This pull request introduces 2 alerts when merging ad90be6 into 0e2ced5 - view on LGTM.com new alerts:
|
…a testing. The file is still created but remains empty at the moment
|
This pull request introduces 1 alert when merging d3048af into 87f3b01 - view on LGTM.com new alerts:
|
| @@ -48,7 +48,7 @@ def cwltool(tmp_path: Path, *args: Any) -> Path: | |||
| def cwltool_no_data(tmp_path: Path, *args: Any) -> Path: | |||
| prov_folder = tmp_path / "provenance" | |||
| prov_folder.mkdir() | |||
| new_args = ["--no-data", "--provenance", str(prov_folder)] | |||
| new_args = ["--enable-ext", "--no-data", "--provenance", str(prov_folder)] | |||
There was a problem hiding this comment.
--enable-ext shouldn't be required when using loadListing with CWL v1.1+
|
This pull request introduces 1 alert when merging ac532d4 into 1d23218 - view on LGTM.com new alerts:
|
…t present in the field object.
|
This pull request introduces 1 alert when merging fb5a65a into 1d23218 - view on LGTM.com new alerts:
|
|
This pull request introduces 1 alert when merging d01a0df into 6f0e1d9 - view on LGTM.com new alerts:
|
|
This pull request introduces 1 alert when merging 315e78f into 6f0e1d9 - view on LGTM.com new alerts:
|
# Conflicts: # .gitignore # cwltool/cwlprov/provenance_profile.py # cwltool/cwlprov/ro.py # cwltool/executors.py # cwltool/main.py
Keeping as draft while we test proposed enhancements to CWLProv
Added file size to prov RO-Bundle
Creates a workflow.ttl file in the default folder while running.
Then closes the file and after the provenance folder is created it is copied to the provenance/workflow/ folder.
temp directory creation and removal added