GW Cloud Bilby module for running bilby jobs from the web.
- In the
src/
directory, set up a python virutalenv by running
virtualenv -p 3.12 venv
- Use the virtualenv with
. venv/bin/active
- Install the requirements with
pip install -r requirements.txt
- Migrate the database with
# In the venv
python development-manage.py migrate
If you want to use actual data, you will need to install and configure mysql as well. This is due to the production server using mysql and therefore difficulties in importing a mysql data dump into sqlite
Note Some tests fail when running the test suite with a mysql backend. Investigations as to why continue.
- Install mysql-client and mysql-server for your distro
sudo apt-get install mysql-client mysql-server
- Install the mysqlclient python package
# Inside the venv
pip install mysqlclient
# note - you may need to install additional packages for this to work
- Create schema in mysql :
gwcloud_bilby
- Create mysql user and grant it all privileges on that schema
- Generate a data dump from production (this is left as an exercise to the reader i.e., get someone with access to do it for you)
- Import the data dump with
mysql -u <username> -p gwcloud_bilby < <production_dump.sql>
- Add a local settings file at
src/gwcloud_bilby/local.py
and add aDATABASES=
section defining the mysql connection (seeenvironemnt.py
in that directory for an example) - Migrate the database
# Inside the venv
python development-manage.py migrate
To run locally you will need access to an elasticsearch instance, which can be set up by following
docker network create elastic
docker pull docker.elastic.co/elasticsearch/elasticsearch:8.8.1
docker run --name elasticsearch --net elastic -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" -e "xpack.security.enabled=false" -t docker.elastic.co/elasticsearch/elasticsearch:8.8.1
This will work with the out-of-the-box development settings.
In order to fetch results or file lists you will need to have access to the production job controller. To do this, add a JOB_CONTROLLER_JWT_SECRET=
field to the local.py
settings file. This JWT_SECRET
can be generated by Lewis
Inside the src/react
directory:
nvm install $(cat .nvmrc)
nvm use $(cat .nvmrc)
npm install
You will need to run in two seperate terminals - one for the django host and one for the react dev-server.
NOTE You will also need to be running the react-host and gwcloud_auth locally.
# In src/ directory
. venv/bin/activate
python development-manage.py runserver 8001
# in src/react directory
npm use
npm run relay
npm run start
- To synchronise the django databases of jobs with the es instance
# Inside the venv
python development-manage.py es_ingest
- To regenerate the graphql schema
# Inside the venv
python development-manage.py graphql_schema
- To recompile the graphql schema on the frontend
# Inside the frontend directory
nvm use
npm run relay