Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Get CWL workflows from GitHub, GitLab or Bitbucket #46

Closed
uniqueg opened this issue Oct 2, 2018 · 2 comments
Closed

Get CWL workflows from GitHub, GitLab or Bitbucket #46

uniqueg opened this issue Oct 2, 2018 · 2 comments
Assignees

Comments

@uniqueg
Copy link
Member

uniqueg commented Oct 2, 2018

Is your feature request related to a problem? Please describe.
Currently, the app only supports running workflows that are available on the local file system, which severely limits usefulness of the service in the wild. While the multipart/form-data field workflow_attachment in the workflow run requests allows for sending workflow files via the request itself, there are some problems with implementing that approach (see #6), so an additional solution would be to allow users/clients to point to workflows inside (public) Git repos.

Describe the solution you'd like
A valid URL pointing to the main CWL workflow file is passed to the workflow_url field in the run request (POST \runs). The URL is processed to derive the Git repo container name (ending with .git), as well as the relative location of the file inside the cloned repo. The local path is then constructed from the download directory, the repo name and the relative path portion, and is then passed to cwl-tes in the same way that locally available files are currently passed. There will be three ways of specifying an accompanying main workflow parameters file, that will be checked in the following order:

  1. The file is constructed from the workflow_params field (current approach)
  2. The relative workflow parameters file location inside the Git repo is appended to the workflow_url field, delimited by either ,, ; or |
  3. If no workflow_params are found and the workflow_url field is not amended, it will be attempted to find a .yaml file with the same dirname and basename as the main workflow file, or otherwise any single .yaml file in the same directory.

For the time being at least, cloning of the repository will not be executed in the background. A BadRequest/400 error will be returned if anything goes wrong, such as:

  • the repo cannot be cloned (e.g., wrong URL, access restricted)
  • the workflow file cannot be found
  • the parameter file cannot be found

Describe alternatives you've considered
Implementation of workflow_attachment (#6) by this service and corresponding clients may render this approach obsolete. It should therefore rather looked at as a convenient way of passing workflows by users via the Swagger UI (which does not support multi file upload for Swagger/OpenAPI v2 specs).

Additional context
N/A

@uniqueg uniqueg added this to the Working prototype with CWL-TES milestone Oct 2, 2018
@uniqueg uniqueg self-assigned this Oct 2, 2018
@uniqueg
Copy link
Member Author

uniqueg commented Oct 9, 2018

Closed by c072880

@uniqueg
Copy link
Member Author

uniqueg commented Oct 9, 2018

Hot fix in 0dd70d7

@uniqueg uniqueg closed this as completed Oct 9, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant