The application consists of the following components:
- Database: MariaDB
- Server: A REST-API backend using the SQLAlchemy Database Toolkit for Python and Flask
- Client: A Web-GUI using Vue.js
At launch time, the app uses tables defined in SetupService
to populate the database. These tables must exist and be located under IMPORT_PATH
. By default, this path is located under server/import, and can be specified in the environment file. Once setup is complete, assembly and annotation files must be added. See Flask CLI - Setup for details.
For project and data management, consult the Documentation tab on the Web-GUI, or Flask CLI - Project and data management.
The recommended way to run Sci-ModoM in production is to use Podman to create, manage, and deploy the application and the database containers, see Container setup for details.
Set up a MariaDB database. One way to do this is to run a MariaDB container image, see the General and Development setup sections in Container setup.
Create a Python3 virtual environment and activate it:
python3 -m venv ~/.virtualenvs/scimodom
source ~/.virtualenvs/scimodom/bin/activate
Get the source code and install:
git clone https://github.com/dieterich-lab/scimodom.git
cd scimodom/server
pip install --upgrade pip setuptools
pip --verbose install -e '.[dev,tests,docs]' 2>&1 | tee install.log
Note: The package depends on mysqlclient. You may have to install MySQL development headers and libraries before. You also need a working installation of Bedtools, e.g. pre-compiled binaries.
Set up your environment configuration in server/.env:
DATABASE_URI=mysql+mysqldb://scimodom:[email protected]:3307/scimodom
SECRET_KEY=SECRET_KEY
SESSION_COOKIE_SAMESITE=None
SESSION_COOKIE_SECURE=True
SMTP_SERVER=mail-server.my-site.org
[email protected]
[email protected]
HTTP_PUBLIC_URL=http://localhost:5173/
UPLOAD_PATH=/path/to/upload
DATA_PATH=/path/to/data
BEDTOOLS_TMP_PATH=/path/to/tmp
where PASSWORD
is the password for the scimodom user for the MariaDB database in docker/secrets/mariadb-scimodom, 3307
is the MARIADB_DEV_PORT
(we recommend to use a non-standard port e.g. 3307 to avoid clashes with a local MariaDB/MySQL installation), and SECRET_KEY
is the key found in docker/secrets/flask-secret, see Container setup for details. You need to adjust the paths, and make sure they are valid and exist.
Important: If the host name localhost is used in the DATABASE_URI
the database driver will assume that the database is contacted using a named
socket. That will not work if a container is used!
You can also create a server/.flaskenv file with Flask-only variables:
FLASK_APP=src/scimodom/app
FLASK_DEBUG=True
Start the database container under the docker directory, see Container setup (Development setup). Under the server directory, initialize the database schema:
alembic upgrade head
and start the API backend:
flask run
Most Python IDEs can run this process in the integrated debugger. You are now ready to add assemblies, annotations, projects, and data (General setup).
To register in development mode, use the Sign up button. This requires a functional email server. You first need build the frontend (see Client setup below). Once you receive a link via email, click on this link, but change the frontend server address to that of the Flask development server URL, e.g. change http://localhost:5173/ to http://localhost:5000. This is only necessary if you run the database using a container and connect it with the local Flask application.
Note that email functionality may be limited, as your mail server must be willing to relay emails for your SMTP_FROM_ADDRESS
, e.g. Google or Gmail addresses will most likely not work. This may be problematic if you wish to register, as registration is done via a link sent by email. One way to avoid this problem is to patch the database. Open a python console under your environment and do the following
from werkzeug.security import generate_password_hash
generate_password_hash("mypassword", method="pbkdf2")
# this will return e.g. 'pbkdf2:sha256:600000$vpYjirPAT8xBuPHo$1001474730f96085cdafbf0f159d12e20ec36342b4faddbf226d637c695ee642'
Then go to the database, see e.g. Container setup - Manual database connection and do the following:
INSERT INTO user (email, state, password_hash) VALUES ('test@uni-heidelberg', 'active', 'pbkdf2:sha256:600000$vpYjirPAT8xBuPHo$1001474730f96085cdafbf0f159d12e20ec36342b4faddbf226d637c695ee642');
A new user is now registered, and you can login using whatever email address you used e.g. "test@uni-heidelberg" with the chosen password e.g. "mypassword".
The first time, you need to install the local packages that the project needs, go to the client directory and execute:
npm install
This creates a node_modules folder. You are now ready to bring up the frontend
npm run dev
The application is now available at http://localhost:5173/, and any change you make e.g. to the HTML code will be reflected in the page you see in your browser.
To test the bundled frontend, run:
npm run build
This will populate the folder dist with the bundled static HTML/JavaScript code as it should be deployed in production. The server can now also serve this code. The complete application is now also available under the Flask development server URL e.g. at http://127.0.0.1:5000.
Under the server directory:
# the first time, you might have to
pre-commit install
# the first time pre-commit runs on a file it will automatically download, install, and run the hook
# runs on all file at commit or run manually
pre-commit run --all-files
# to run individual hooks use pre-commit run <hook_id>
# run static type checker
mypy -p scimodom
To execute the tests, run under the server directory:
pytest tests
The database schema is tracked using Alembic. Changes to the database must be coded at two locations:
- An Alembic migration script under server/migrations/versions
- The model must be updated in server/src/scimodom/database/models.py
Any change to the schema is generally tracked with:
alembic revision [--autogenerate] -m "message"
alembic upgrade head
GNU Affero General Public Licence (AGPL) version 3.