Skip to content
This repository has been archived by the owner on Mar 10, 2023. It is now read-only.

Cloud Portal Application for bioexcel cwl execution

Notifications You must be signed in to change notification settings

EMBL-EBI-TSI/cpa-bioexcel-cwl

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

57 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Note The EBI Cloud Portal has been retired and its code base is no longer updated. This includes all tooling for the VM-related features of the BioExcel Portal.

If you would like to contact the Cloud Portal authors, please leave us feedback via the EBI contact form mentioning the EBI Cloud Portal in the Subject.

As of March 2023 the BioExcel project itself is ongoing and you can find more information on its official website.

ECP (EBI Cloud Portal) Application - bioexcel-cwl

ECP BioExcel application to create a VM with tools such as CWL, toil, docker and nfs client installed. If deploying in OpenStack it will use a bespoke Ubuntu 18.04 image as a base OS, with Ansible installing the CWL, toil and docker tools.


Requirements

A base Ubuntu 18.04 image with some Python packages pre-installed. On the EBI's Embassy4 OpenStack cloud the image is currently called ubuntu-18_04-python, although the presence of this image will need to be verified by someone with access to Embassy4 before deploying this application.

Instructions

To use this application in the ECP

  • Log into the ECP (https://cloud-portal.ebi.ac.uk/)
  • Define some portal "Cloud Credentials" and "Deployment Parameters" as appropriate (see below for examples)
  • Define a portal "Configuration" (derived from "Cloud Credentials" and "Deployment Parameters")
  • Create a new project (by using this GitHub URL)
  • Select the project and use the "Configuration"
  • Click "DEPLOY"

Testing

To test a deployed VM

These instructions assume that a workflow_url value of https://github.com/douglowe/biobb_hpc_cwl_md_list was specified, in which case visit that GitLab repo for up-to-date information, but it works with commit #97122f2 at least!

  • ssh into the VM
  • (cwl_bioexcel) ubuntu@tsi1632989604005-1:~/cwl_bioexcel$ cd ../workflow/
  • (cwl_bioexcel) ubuntu@tsi1632989604005-1:~/workflow$ cwl-runner md_launch.cwl md_list_input_descriptions.yml

That should kick off a series of workflow steps.

"Cloud Credentials"

These are new for Embassy4 submissions and replace the previous username/password mechanism.

Credential name Description (derived from RedHat docs)
OS_APPLICATION_CREDENTIAL_ID With v3applicationcredential: application credential ID
OS_APPLICATION_CREDENTIAL_SECRET With v3applicationcredential: application credential auth secret

Default values

"Deployment Parameters"

The current default values can be determined from the content of variables.tf (or in some cases the deploy.sh script). If you want to use your own values, specify them by matching against the parameter name in the "Deployment Parameters".

Parameter name Default parameter value Overwrite
default?
Description
user_name default.user_name Optional Your email
tenent_name default.tenant_name Yes OpenStack tenant name
auth_url https://uk1.embassy.ebi.ac.uk:5000 Optional OpenStack API Identity Service URL
region RegionOne TBD OpenStack Region
name cwl vm environment TBD Abitrary deployment name
instances 1 TBD Number of instances to install
deployment_path . TBD Local path to save terraform deployment output
disk_image_name ubuntu-18_04-python Optional OpenStack Compute Image name
machine_type 2c4m80d Optional OpenStack Machine type ("flavor")
floating_ip_pool public Optional OpenStack Network name for Floating IPs
network_name undefined Yes OpenStack Network name
remote_user ubuntu TBD Remove VM user
local_folder nfs-imports TBD Local folder
ssh_key undefined TBD Public key to inject into the VM
public_key_path ~/.ssh/id_rsa.pub TBD Path of public SSH key to be injected in the VM
private_key_path ~/.ssh/id_rsa TBD Path of private SSH key to connect to the VM
user_private_key_path ~/.ssh/id_rsa TBD (see variables.tf)
nfs_install yes or no TBD
workflow_url undefined Optional e.g. https://github.com/douglowe/biobb_hpc_cwl_md_list

Inputs

Parameter name Parameter value
nfs_server_host IP
remote_folder /var/nfs

Open Ports

The virtual machine provided will have this ports open.

ICMP: all
TCP : 22

Ansible roles

Ansible Galaxy: indigo-dc.nfs https://galaxy.ansible.com/indigo-dc/nfs/

About

Cloud Portal Application for bioexcel cwl execution

Resources

Stars

Watchers

Forks

Packages

No packages published

Contributors 4

  •  
  •  
  •  
  •