Add documentation for glvd (#43)

README.md

@@ -1 +1,37 @@
 # Garden Linux Vulnerability Database
+
+This repository contains the Security Tracker of Garden Linux. The Security Tracker is called `glvd` and it is an application written in Python that is operated within a Debian testing container. By offering a container image, the Security Tracker can simply be operated on any machine via tools like `docker` or `podman` but it could also be used for container orchestration tools like Kubernetes in order to run it at scale.
+
+More information about the infrastructure on which `glvd` will be operated, can be found here:
+* [gardenlinux/glvd-infrastructure](https://github.com/gardenlinux/glvd-infrastructure)
+
+This repository on the other hand contains the actual source code of the Security Tracker.
+
+## Repository Structure
+Thereby, this repostory contains the following directories:
+
+- `docs/`: This directory contains documentation regarding `glvd`.
+- `src/`: This directory contains the source files of `glvd`.
+  - `glvd/`: The main directory of the Security Tracker.
+    - `cli/`: Command Line Interface for running operational tasks on `glvd`.
+    - `data/`: The backend implementation for dealing with the Security Tracker data like CPEs, CVEs and Debian Sources.
+    - `database/`: Contains the sqlalchemy classes for representing each table used by `glvd`.
+    - `web/`: The actual web application and its endpoint that can be called to receive vulnerabilities from the Security Tracker. This code represents the API.
+- `tests/`: This directory contains all tests (e.g. unit tests) used by pytest regarding `glvd`.
+
+Other important files are:
+- [Containerfile](./Containerfile): This file specifies the corresponding container of `glvd`.
+- [openapi-v1.yaml](./openapi-v1.yaml): This configuration defines the API endpoints of `glvd`.
+- [pyproject.toml](./pyproject.toml): The configuration file for defining the Python project / application.
+- [setup.cfg](./setup.cfg): Configuration file for defining the metadata of the Python project typically used by setuptools.
+
+## Documentation
+
+### Client
+The client documentation can be found here: [docs/01_client.md](./docs/01_client.md)
+
+### Server
+The server documentation can be found here: [docs/02_server.md](./docs/02_server.md)
+
+### Ingestion
+The ingestion documentation can be found here: [docs/03_ingestion.md](./docs/03_ingestion.md)

docs/01_client.md

@@ -0,0 +1,53 @@
+# Client
+
+The Garden Linux Security Tracker `glvd` can be access either via a simple HTTP client or it can be access via Garden Linux own Security Tracker Client.
+
+## HTTP Client
+
+The Garden Linux Security Tracker provides API endpoints that can be access with every HTTP client. A user can access it via `curl` for example, which would look like this:
+```bash
+$ curl -s http://glvd.gardenlinux.io/<API ENDPOINT>
+$ curl -s http://glvd.gardenlinux.io/v1/cves/CVE-2019-13057
+```
+The HTTP server will return a JSON output which could be parsed by tools like `jq` for example.
+Which API endpoint exists can be read in the corresponding API specification. This can be found here: [openapi-v1.yaml](../openapi-v1.yaml)
+
+## GLVD Client
+This repository offers a lightwight `glvd` client written in Python with which a user can send request to the `glvd` API endpoints. In order to run this client, some prerequisites must be fulfilled on the client's system before executing the corresponding client.
+
+### Prerequisites
+
+In order to run the `glvd` client, one must ensure that the following packages are installed
+```bash
+$ apt update
+$ apt install -y git python3-pip python3-apt
+```
+
+After that, the actual client can be installed
+```bash
+$ pip install git+https://github.com/gardenlinux/glvd --system-break-packages
+```
+
+### Execution
+
+The Garden Linux Security Tracker client offers two modes:
+
+##### CVE Mode:
+```bash
+$ glvd cve <CVE>
+$ glvd cve CVE-2019-13057 --server http://glvd.gardenlinux.io
+```
+This mode allows you to get information about a given CVE as long as Garden Linux is affected by this CVE. If Garden Linux is not affected by this, the client will return the following message: 
+* `CVE-2019-13058 not found`
+
+The CVE information is returned in JSON format so the output can be parsed with tools like `jq` for example.
+
+##### CVE APT Mode:
+```bash
+$ glvd cve-apt --server http://glvd.gardenlinux.io
+```
+This mode returns all CVEs to all packages installed on the local system. The system itself must fullfill the following criteria:
+* Debian: Buster, Bullseye, Bookworm or Trixie
+* Garden Linux: >= 1337.0
+
+Thereby, the client returns a list of CVEs in JSON format. This output can then be parsed by tools like `jq` for example.

docs/02_server.md

@@ -0,0 +1,54 @@
+# Server
+
+The Garden Linux Security Tracker `glvd` uses [Quart](https://quart.palletsprojects.com/en/latest/) which is a Fast Python web microframework.
+
+## Purposes
+
+This framework can be used for providing simple webpages as well as for hosting APIs. Garden Linux's Security Tracker is using this framework to provide both: 
+
+#### Web App
+As of now, there is no Web App implemented in `glvd`. In future, there could be a web page which allows to request vulnerability information in an easy and fast way similar to Debian's Security Tracker.
+
+#### API
+
+The general API is described in the following specificiation:
+* [openapi-v1.yaml](../openapi-v1.yaml)
+
+Thereby, it offers the following API endpoints:
+
+---
+##### Endpoint: `v1/cves/findByCpe`:
+Finds CVE by CPE
+
+Supported HTTP methods:
+* GET
+
+| Parameter | Description | 
+|-----------|-------------|
+| `cpeName` |  CPE name to search for, only Debian/Garden Linux related CPE can be used |
+| `cvssV3SeverityMin` | The min severity that a CVE must have. |
+| `debVersionEnd` | The maximum version | 
+
+---
+##### Endpoint `v1/cves/findBySources`:
+Finds CVE by source packages
+
+Supported HTTP methods:
+* POST
+
+| Parameter | Description | 
+|-----------|-------------|
+| `cvssV3SeverityMin` | The min severity that a CVE must have. |
+
+The POST body must contain a list of source packages for which one searches the corresponding CVEs. A list item could look like this: `debian_bookworm_glibc_2.36-9+deb12u3`
+
+---
+##### Endpoint `v1/cves/findBySources`:
+Find CVE by ID
+
+Supported HTTP Methods:
+* GET
+
+| Parameter | Description | 
+|-----------|-------------|
+| `cveId` | The ID of an CVE (e.g. `CVE-1999-0250`) |

docs/03_ingestion.md

@@ -0,0 +1,94 @@
+# Ingestion
+
+The Security Tracker `glvd` uses a database to access all the data it needs to provide information about CVEs that are relevant in Garden Linux. Thereby, the data is gathered from multiple sources and stored in a PostgreSQL database. From there, the Security Tracker uses this data and offers it in a JSON format via its API endpoints.
+
+## Introduction
+
+The data is gathered on daily bases and pushed into the corresponding PostgreSQL database. This process step is called data ingestion and for this the following Git repository has been created:
+* [gardenlinux/glvd-data-ingestion](https://github.com/gardenlinux/glvd-data-ingestion)
+
+This repository uses Github Actions to gather information from NVD, Debian and Garden Linux to push them into the database. In addition, it also combines information, so that most of the processing is already done in the database and the application logic of the Security Tracker can be kept small.
+
+## Database
+
+The PostgreSQL database is operated as a container on an EC2 instance running on AWS. This infrastructure is provided by the following Git repository:
+* [gardenlinux/glvd-infrastructure](https://github.com/gardenlinux/glvd-infrastructure)
+
+In order to run a PostgreSQL container that fits the `glvd`'s needs, a dedicated Git repository has been created which takes the generally available PostgreSQL container image and installs required extensions and configuration into the container needed by the Security Tracker. A corresponding Github Action builds the container and publishes it. You can find the repository and the corresponding container image here:
+* [gardenlinux/glvd-postgres](https://github.com/gardenlinux/glvd-postgres)
+
+Next to this, there are a handfull of tables that are consumed by `glvd`. The following section shows, what database tables exists, how they are defined and which purpose those have:
+
+---
+##### Table: `nvd_cve`
+This table contains the CVEs from NVD so that `glvd` can immediately access all available CVEs without querying the NVD API endpoint.
+
+| Attribut | Description |
+|----------|-------------|
+|`cve_id`    | The ID of the CVE. (e.g `CVE-1999-0095`) |
+|`last_mod`    | The last modification date of this data (e.g `2019-06-11 20:29:00.263+00`). |
+|`data`   | The actual CVE data from NVD. This is stored as a JSON.|
+
+---
+##### Table: `all_cve`
+Compared to `nvd_cve`, this table contains all known CVEs of the `glvd`. This table is filled by the `combine-all` ingestion command which takes not only NVD CVEs but also CVEs that are only known by for example Debian. In other words, the `nvd_cve` is a subset of the `all_cve` table.
+
+| Attribut | Description |
+|----------|-------------|
+|`cve_id`    | The ID of the CVE. (e.g `CVE-1999-0095`) |
+|`last_mod`    | The last modification date of this data (e.g `2019-06-11 20:29:00.263+00`). |
+|`data`   | The actual CVE data from NVD. This is stored as a JSON.|
+
+---
+##### Table: `dist_cpe`
+This table contains all available CPEs in `glvd` split between multiple products (e.g `debian_linux` or `gardenlinux`). Thereby, those CPEs represent each available distribution in `glvd`. Typically, these are all Debian distributions and all Garden Linux ones.
+
+| Attribut | Description |
+|----------|-------------|
+|`id`    | ID of the entry |
+|`cpe_vendor`    | The vendor of the CPE (e.g `debian`) |
+|`cpe_product`    | The product of the CPE (e.g `debian_linux`) |
+|`cpe_version`    | The version of the CPE (e.g `11`) |
+|`deb_codename`    | The codename of an CPE product (e.g `bullseye`) |
+
+---
+##### Table: `debsrc`
+Contains all source packages of each supported distribution of `glvd`. What distributions are supported is defined via the `dist_cpe` table. Each distribution is there represented by a CPE.
+| Attribut | Description |
+|----------|-------------|
+|`dist_id`    | The ID of the corresponding distribution. This references the `id` attribute of `dist_cpe`. |
+|`last_mod`    | The last modification date of this data (e.g `2019-06-11 20:29:00.263+00`) |
+|`deb_source`    | The name of the source package (e.g `0ad`) |
+|`deb_version`   | The version of the source package (e.g `0.0.23.1-2`) |
+
+---
+##### Table: `debsec_cve`
+This table contains the entries of the CVE list (`debsec``) of each distribution defined in the `dist_cpe` table. Each CPE product provides its own CVE list which explains what CVE exists in context of the distribution and what the status of this corresponding CVE is. In some cases, CVE might be already fixed by the distribution. In such cases, the list would contain an entry explaining that the distribution is not affected by the corresponding CVE.
+
+For Debian for example, this table contains the CVE entries from this list here:
+* https://salsa.debian.org/security-tracker-team/security-tracker/-/blob/master/data/CVE/list
+
+| Attribut | Description |
+|----------|-------------|
+|`dist_id`    | The ID of the corresponding distribution. This references the `id` attribute of `dist_cpe`. |
+|`cve_id`   | The ID of the CVE itself. This references the `cve_id` attribute of `all_cve`. |
+|`deb_source`    | References |
+|`deb_version_fixed`    | Defines if the source package's version is fixed or not |
+|`debsec_tag`    | A specific tag of the CVE list entry |
+|`debsec_note`    | A specific note for the CVE list entry |
+
+---
+##### Table: `deb_cve`
+Whereas the `devsec_cve` table is the plain representation of the given CVE list of a distribution, this `deb_cve` table basically contains the same information but is enriched by additional information that are put together, so that processing is easier for `glvd`.
+
+| Attribut | Description |
+|----------|-------------|
+|`dist_id`    | The ID of the corresponding distribution. This references the `id` attribute of `dist_cpe`. |
+|`cve_id`    | The ID of the CVE itself. This references the `cve_id` attribute of `all_cve`. |
+|`last_mod`    | The last modification date of this data (e.g `2019-06-11 20:29:00.263+00`) |
+|`cvss_severity`    | The CVSS Severity of the given CVE in comparision to the package and the distribution. |
+|`deb_source`    | The name of the source package (e.g `389-ds-base`) |
+|`deb_version`    | The latest version of the source package (e.g `2.3.1+dfsg1-1`) |
+|`deb_version_fixed`    | The source pacakge version in which a given CVE has been fixed (e.g `1.2.11.15-1`)|
+|`debsec_vulnerable`    | Defines if the CVE list entry (debsec) is affected by the CVE or not (e.g `f`) |
+|`data_cpe_match`    | The CPE match string for the given CVE in relation to the source package (e.g `{"criteria": "cpe:2.3:o:debian:debian_linux:12:*:*:*:*:*:*:deb_source\\=389-ds-base", "deb": {"versionLatest": "2.3.1+dfsg1-1", "versionEndExcluding": "1.2.11.15-1"}, "vulnerable": false}`)|