Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Test] Chaining sub-workflows #171

Open
Tracked by #141
fmigneault opened this issue Jun 9, 2020 · 0 comments
Open
Tracked by #141

[Test] Chaining sub-workflows #171

fmigneault opened this issue Jun 9, 2020 · 0 comments
Assignees
Labels
ci/tests Tests of the package and features triage/feature New requested feature.

Comments

@fmigneault
Copy link
Collaborator

Technically it would be possible to chain sub-workflows as step of a bigger workflow.
This has to be tested though in the context of weaver.

There is a good chance that data-source resolution (where to dispatch the step process) fails since weaver looks for a remote ADES to run the process. Since in that case the step would be a workflow, it needs to be executed by an EMS.
There is also a change that weaver auto-resolves the default data-source to 'self' and it could then transparently work, but it is not a guarantee.

We should probably consider some package type check that enforce resolution of a sub-workflow to 'self' to avoid unexpected problems.

It is also important to resolve to 'self' instead of "any remote EMS" because the sub-workflow step will need to find all sub-sub-processes to call them. Dispatching a sub-workflow step to another EMS would require to recursively dispatch every underlying process application, which rapidly becomes unmanageable. Also, there is technically no advantage to deploy to a remote EMS since its job should be only to monitor remote jobs, which should be low in terms of resource requirements.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci/tests Tests of the package and features triage/feature New requested feature.
Projects
None yet
Development

No branches or pull requests

1 participant