GW Cloud - Bilby module

GW Cloud Bilby module for running bilby jobs from the web.

Installing locally

Virtualenv

• In the src/ directory, set up a python virutalenv by running

virtualenv -p 3.12 venv

• Use the virtualenv with

. venv/bin/active

• Install the requirements with

pip install -r requirements.txt

• Migrate the database with

# In the venv
python development-manage.py migrate

Mysql

If you want to use actual data, you will need to install and configure mysql as well. This is due to the production server using mysql and therefore difficulties in importing a mysql data dump into sqlite

Note Some tests fail when running the test suite with a mysql backend. Investigations as to why continue.

• Install mysql-client and mysql-server for your distro

sudo apt-get install mysql-client mysql-server

• Install the mysqlclient python package

# Inside the venv
pip install mysqlclient
# note - you may need to install additional packages for this to work

• Create schema in mysql : gwcloud_bilby
• Create mysql user and grant it all privileges on that schema
• Generate a data dump from production (this is left as an exercise to the reader i.e., get someone with access to do it for you)
• Import the data dump with

mysql -u <username> -p gwcloud_bilby < <production_dump.sql>

• Add a local settings file at src/gwcloud_bilby/local.py and add a DATABASES= section defining the mysql connection (see environemnt.py in that directory for an example)
• Migrate the database

# Inside the venv
python development-manage.py migrate

elasticsearch

To run locally you will need access to an elasticsearch instance, which can be set up by following

docker network create elastic
docker pull docker.elastic.co/elasticsearch/elasticsearch:8.8.1
docker run --name elasticsearch --net elastic -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" -e "xpack.security.enabled=false" -t docker.elastic.co/elasticsearch/elasticsearch:8.8.1

This will work with the out-of-the-box development settings.

Jobcontroller

In order to fetch results or file lists you will need to have access to the production job controller. To do this, add a JOB_CONTROLLER_JWT_SECRET= field to the local.py settings file. This JWT_SECRET can be generated by Lewis

Frontend

Inside the src/react directory:

nvm install $(cat .nvmrc)
nvm use $(cat .nvmrc)
npm install

Running locally

You will need to run in two seperate terminals - one for the django host and one for the react dev-server.

NOTE You will also need to be running the react-host and gwcloud_auth locally.

django

# In src/ directory
. venv/bin/activate
python development-manage.py runserver 8001

react

# in src/react directory
npm use
npm run relay
npm run start

Other commands

• To synchronise the django databases of jobs with the es instance

# Inside the venv
python development-manage.py es_ingest

• To regenerate the graphql schema

# Inside the venv
python development-manage.py graphql_schema

• To recompile the graphql schema on the frontend

# Inside the frontend directory
nvm use
npm run relay