Luigi Covid Pipeline

Managing a ML pipeline with Luigi.

Case study: reports of the Italian COVID-19 situation.

We use the data of the Italian Civil Protection, available at this repository.

Installation

Clone the respository:

git clone https://github.com/elianap/luigi-covid-pipeline.git

You can firstly create an environment, for example using conda:

conda create --name luigi3 python=3

Then install the required package (in requirements.txt) via pip.

To install luigi:

pip install luigi

Run Luigi

Run luigi scheduler

PYTHONPATH='.' luigid

Run Luigi task

luigi --module <module_name> <task_name> --<parameter1_name> <par_value> 

Example

Generate a daily report:

luigi --module covid_pipeline AggregateInReport