Clone the source code:
git clone https://github.com/nexB/vulnerablecode.git && cd vulnerablecode
-
Python 3.6+
-
PostgreSQL 9+ or Docker
-
Compiler toolchain and development files for Python and PostgreSQL
On Debian-based distros, these can be installed with sudo apt install python3-venv python3-dev postgresql libpq-dev build-essential. Leave out postgresql if you want to run it in Docker.
Either run PostgreSQL in Docker:
docker run --name pg-vulnerablecode -e POSTGRES_USER=vulnerablecode -e POSTGRES_PASSWORD=vulnerablecode -e POSTGRES_DB=vulnerablecode -p 5432:5432 postgres
Or without:
-
Create a user named
vulnerablecode. Usevulnerablecodeas password when prompted:sudo -u postgres createuser --no-createrole --no-superuser --login --inherit --createdb --pwprompt vulnerablecode -
Create a databased named
vulnerablecode:createdb --encoding=utf-8 --owner=vulnerablecode --user=vulnerablecode --password --host=localhost --port=5432 vulnerablecode
Activate a virtualenv, install dependencies, and run the database migrations:
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
DJANGO_DEV=1 python manage.py migrate
The environment variable DJANGO_DEV is used to load settings suitable for development, defined in vulnerablecode/dev.py. If you don't want to type
it every time use export DJANGO_DEV=1 instead.
When not running in development mode, an environment variable named SECRET_KEY needs to be set. The recommended way to generate this key is to use
the code Django includes for this purpose: SECRET_KEY=$(python -c "from django.core.management import utils; print(utils.get_random_secret_key())").
pycodestyle --exclude=migrations,settings.py,venv --max-line-length=100 .
python -m pytest -v vulnerabilities/tests/test_scrapers.py vulnerabilities/tests/test_api_data.py
For Django based tests
DJANGO_DEV=1 python manage.py test vulnerabilities/tests
DJANGO_DEV=1 python manage.py import --all
If you want to run the import periodically, you can use a systemd timer:
$ cat ~/.config/systemd/user/vulnerablecode.service
[Unit]
Description=Update vulnerability database
[Service]
Type=oneshot
Environment="DJANGO_DEV=1"
ExecStart=/path/to/venv/bin/python /path/to/vulnerablecode/manage.py import --all
$ cat ~/.config/systemd/user/vulnerablecode.timer
[Unit]
Description=Periodically update vulnerability database
[Timer]
OnCalendar=daily
[Install]
WantedBy=multi-user.target
Start it with
systemctl --user daemon-reload && systemctl --user start vulnerablecode.timer
Start the webserver
DJANGO_DEV=1 python manage.py runserver
In your browser access:
http://127.0.0.1:8000/api/
http://127.0.0.1:8000/api/packages/?name=<package_name>
See https://devcenter.heroku.com/articles/django-app-configuration#creating-a-new-django-project https://devcenter.heroku.com/articles/deploying-python#how-to-keep-build-artifacts-out-of-git
-
Create an Heroku account
-
Download and install the Heroku CLI https://devcenter.heroku.com/articles/heroku-cli#download-and-install
-
Run a local webserver:
heroku local web -
Login:
heroku login -
Create Heroku app:
heroku create -
Generate a secret key and pass it as an environment variable:
heroku config:set SECRET_KEY=$(python -c "from django.core.management import utils; print(utils.get_random_secret_key())") -
Deploy:
git push heroku <branch>:master -
Migrate the database:
heroku run python manage.py migrate -
Load the data referring to chapter "Data import" above.
-
To check the logs:
heroku logs --tail
Note: Running jobs with Heroku Scheduler might incur costs. If you haven't already, you need to add a credit card in your account (https://dashboard.heroku.com/account/billing).
-
Install the Scheduler add-on:
heroku addons:create scheduler:standard -
Open the Scheduler dashboard:
heroku addons:open scheduler -
Click on "Create job" and enter
python manage.py import --allunder "Run Command"