This repository contains a collection of containerized (dockerized) time series anomaly detection methods that can easily be evaluated using TimeEval. Some of the algorithm's source code is access restricted and we just provide the TimeEval stubs and manifests. We are happy to share our TimeEval adaptations of excluded algorithms upon request, if the original authors approve this.
Each folder contains the implementation of an algorithm that is built into a runnable Docker container using GitHub Actions.
We host the algorithm Docker images on GitHub.
Thus, the namespace prefix (repository) for the Docker images is ghcr.io/timeeval/.
Please always use a version-tagged Docker image for your algorithms to ensure reproducibility!
You can pull the TimeEval algorithm images directly from the GitHub registry. This registry does not require authentication.
docker pull ghcr.io/timeeval/<algorithm_name>:0.3.0Each algorithm in this repository is bundled in a self-contained Docker image so that it can be executed with a single command and no additional dependencies must be installed. This allows you to test the algorithm without installing its dependencies on your machine. The only requirement is a (x86-)Docker runtime.
In the following, we assume that you want to build the Docker image for the lof-algorithm.
⚠️ Use correct version tags!Please tag the base and intermediate images with the correct version. You can find the required version for each algorithm image in its
Dockerfile. E.g. forlofthe version forpyodmust be0.3.0(as of 2023-12-16):FROM ghcr.io/timeeval/pyod:0.3.0
You need the following tools installed on your development machine:
- git
- docker
- access to this repository
- (optionally) Docker BuildKit
Please make yourself familiar with the concepts of TimeEval, and read the TimeEval documentation and this document carefully!
You'll need the required base Docker images to build your algorithm's image. You can either pull the base image from the registry or build it yourself. In this guide, we assume that you want to build all Docker images locally.
-
Clone this repository and change to its root folder
git clone https://github.com/TimeEval/TimeEval-algorithms.git cd TimeEval-algorithms -
Change to the
0-base-imagesfolder:cd 0-base-images -
Build your desired base image:
docker build -t ghcr.io/timeeval/python3-base:0.3.0 python3-base
Because the algorithm lof depends on an intermediate image, we, first, need to build the required intermediate image pyod.
Please see the table in this repository's README for the dependencies.
You can build the intermediate image pyod using these commands:
cd ../1-intermediate-images
docker build -t ghcr.io/timeeval/pyod:0.2.5 pyodOnce you have built all dependent images, you can build your algorithm image from the base image(s):
-
Change to the repository's root directory:
cd .. -
Build the Docker image for your algorithm (
lofin this case):docker build -t ghcr.io/timeeval/lof:0.3.0 ./lof
Testing an algorithm locally can be done in two different ways:
- Test the algorithm's code directly (using the tools provided by the programming language)
- Test the algorithm within its docker container
The first option is specific to the programming language, so we won't cover it here.
Each algorithm in this repository will be bundled in a self-contained Docker image so that it can be executed with a single command and no additional dependencies must be installed. This allows you to test the algorithm without installing its dependencies on your machine. The only requirement is a (x86-)Docker runtime. Follow the below steps to test your algorithm using Docker (examples assume that you want to build the image for the LOF algorithm):
-
Pull or build the algorithm image We refer the reader to the previous section for detailed instructions.
-
Train your algorithm (optional) If your algorithm is supervised or semi-supervised, execute the following command to perform the training step (not necessary for LOF):
mkdir -p results docker run --rm \ -v $(pwd)/data:/data:ro \ -v $(pwd)/results:/results:rw \ # -e LOCAL_UID=<current user id> \ # -e LOCAL_GID=<current groupid> \ ghcr.io/timeeval/<your_algorithm>:latest execute-algorithm '{ "executionType": "train", "dataInput": "/data/dataset.csv", "dataOutput": "/results/anomaly_scores.ts", "modelInput": "/results/model.pkl", "modelOutput": "/results/model.pkl", "customParameters": {} }'Be warned that the result and model files will be written to the
results-directory as the root-user if you do not pass the optional environment variablesLOCAL_UIDandLOCAL_GIDto the container. -
Execute your algorithm Run the following command to perform the execution step of your algorithm:
mkdir -p results TIMEEVAL_ALGORITHM=lof docker run --rm \ -v $(pwd)/data:/data:ro \ -v $(pwd)/results:/results:rw \ # -e LOCAL_UID=<current user id> \ # -e LOCAL_GID=<current groupid> \ ghcr.io/timeeval/${TIMEEVAL_ALGORITHM}:latest execute-algorithm '{ "executionType": "execute", "dataInput": "/data/dataset.csv", "dataOutput": "/results/anomaly_scores.ts", "modelInput": "/results/model.pkl", "modelOutput": "/results/model.pkl", "customParameters": {} }'Be warned that the result and model files will be written to the
results-directory as the root-user if you do not pass the optional environment variablesLOCAL_UIDandLOCAL_GIDto the container.