From 40115e3706b0c901066c279d671372250bc1b1bc Mon Sep 17 00:00:00 2001 From: Steven Smith <77019920+stevsmit@users.noreply.github.com> Date: Fri, 10 Nov 2023 14:03:08 -0500 Subject: [PATCH] 6180 clair updates (#802) * Reworks clair documentation * 2 --------- Co-authored-by: Steven Smith --- clair/master.adoc | 3 +- modules/clair-concepts.adoc | 19 +- modules/clair-updater-urls.adoc | 35 +- modules/clair-updaters.adoc | 84 +-- .../clair-vulnerability-scanner-hosts.adoc | 17 +- modules/config-fields-clair-auth.adoc | 17 +- modules/config-fields-clair-indexer.adoc | 21 +- modules/config-fields-clair-matcher.adoc | 28 +- modules/config-fields-clair-matchers.adoc | 33 +- modules/config-fields-clair-metrics.adoc | 18 +- modules/config-fields-clair-notifiers.adoc | 124 +++- modules/config-fields-clair-trace.adoc | 22 +- modules/config-fields-clair-updaters.adoc | 32 +- modules/config-fields-general-clair.adoc | 17 +- modules/config-fields-overview.adoc | 2 +- modules/configuring-clair-updaters.adoc | 594 ++++++++++++++++++ 16 files changed, 912 insertions(+), 154 deletions(-) create mode 100644 modules/configuring-clair-updaters.adoc diff --git a/clair/master.adoc b/clair/master.adoc index 3229bbe05..81d9ba34d 100644 --- a/clair/master.adoc +++ b/clair/master.adoc @@ -20,7 +20,8 @@ include::modules/clair-concepts.adoc[leveloffset=+1] include::modules/clair-authentication.adoc[leveloffset=+2] //include::modules/testing-clair.adoc[leveloffset=+1] include::modules/clair-updaters.adoc[leveloffset=+2] -include::modules/clair-updater-urls.adoc[leveloffset=+3] +include::modules/clair-updater-urls.adoc[leveloffset=+2] +include::modules/configuring-clair-updaters.adoc[leveloffset=+2] include::modules/clair-cve.adoc[leveloffset=+2] include::modules/fips-overview.adoc[leveloffset=+2] diff --git a/modules/clair-concepts.adoc b/modules/clair-concepts.adoc index 761779a10..13b82f5e2 100644 --- a/modules/clair-concepts.adoc +++ b/modules/clair-concepts.adoc @@ -16,12 +16,11 @@ A Clair analysis is broken down into three distinct parts: indexing, matching, a [id="clair-indexing-concept"] === Indexing -Clair's indexer service is responsible for indexing a manifest. In Clair, manifests are representations of a container image. The indexer service is the component that Clair uses to understand the contents of layers. Clair leverages the fact that Open Container Initiative (OCI) manifests and layers are content addressed to reduce duplicate work. +Clair's indexer service plays a crucial role in understanding the makeup of a container image. In Clair, container image representations called "manifests." Manifests are used to comprehend the contents of the image's layers. To streamline this process, Clair takes advantage of the fact that Open Container Initiative (OCI) manifests and layers are designed for content addressing, reducing repetitive tasks. -Indexing involves taking a manifest representing a container image and computing its constituent parts. The indexer tries to discover what packages exist in the image, what distribution the image is derived from, and what package repositories are used within the image. When this information is computed, it is persisted into an `IndexReport`. - -The `IndexReport` is stored in Clair's database. It can be fed to a `matcher` node to compute the vulnerability report. +During indexing, a manifest that represents a container image is taken and broken down into its essential components. The indexer's job is to uncover the image's contained packages, its origin distribution, and the package repositories it relies on. This valuable information is then recorded and stored within Clair's database. The insights gathered during indexing serve as the basis for generating a comprehensive vulnerability report. This report can be seamlessly transferred to a matcher node for further analysis and action, helping users make informed decisions about their container images' security. +//// [id="content-addressability"] ==== Content addressability @@ -32,15 +31,16 @@ For example, consider how many images in a registry might use `ubuntu:artful` as In some cases, Clair should re-index a manifest. For example, when an internal component such as a package scanner is updated, Clair performs the analysis with the new package scanner. Clair has enough information to determine that a component has changed and that the `IndexReport` might be different the second time, and as a result it re-indexes the manifest. A client can track Clair's `index_state` endpoint to understand when an internal component has changed, and can subsequently issue re-indexes. See the Clair API guide to learn how to view Clair's API specification. +//// [id="clair-matching-concept"] === Matching -With Clair, a matcher node is responsible for matching vulnerabilities to a provided `IndexReport`. +With Clair, a matcher node is responsible for matching vulnerabilities to a provided index report. -Matchers are responsible for keeping the database of vulnerabilities up to date. Matchers will typically run a set of updaters, which periodically probe their data sources for new content. New vulnerabilities are stored in the database when they are discovered. +Matchers are responsible for keeping the database of vulnerabilities up to date. Matchers run a set of updaters, which periodically probe their data sources for new content. New vulnerabilities are stored in the database when they are discovered. -The matcher API is designed to be used often. It is designed to always provide the most recent `VulnerabilityReport` when queried. The `VulnerabilityReport` summarizes both a manifest's content and any vulnerabilities affecting the content. +The matcher API is designed to be used often. It is designed to always provide the most recent vulnerability report when queried. The vulnerability report summarizes both a manifest's content and any vulnerabilities affecting the content. // See. . . to learn more about how to view the Clair API specification and to work with the matcher API. @@ -63,7 +63,7 @@ Clair uses a notifier service that keeps track of new security database updates When the notifier becomes aware of new vulnerabilities affecting a previously indexed manifest, it uses the configured methods in your `config.yaml` file to issue notifications about the new changes. Returned notifications express the most severe vulnerability discovered because of the change. This avoids creating excessive notifications for the same security database update. When a user receives a notification, it issues a new request against the matcher to receive an up to date vulnerability report. - +//// The notification schema is the JSON marshalled form of the following types: [source,json] @@ -92,6 +92,7 @@ type VulnSummary struct { Links string `json:"links"` } ---- +//// You can subscribe to notifications through the following mechanics: @@ -101,6 +102,7 @@ You can subscribe to notifications through the following mechanics: Configuring the notifier is done through the Clair YAML configuration file. +//// [id=webhook-delivery] ==== Webhook delivery @@ -153,3 +155,4 @@ The notifier runs in `NOTIFIER_TEST_MODE` until the environment variable is clea To delete the notification, you can use the `DELETE` API call. Deleting a notification ID manually cleans up resources in the notifier. If you do not use the `DELETE` API call, the notifier waits a predetermined length of time before clearing delivered notifications from its database. // For more information on the `DELETE` API call, see. . . +//// \ No newline at end of file diff --git a/modules/clair-updater-urls.adoc b/modules/clair-updater-urls.adoc index 1cb1f81c2..551b61903 100644 --- a/modules/clair-updater-urls.adoc +++ b/modules/clair-updater-urls.adoc @@ -4,34 +4,35 @@ :_content-type: CONCEPT [id="clair-updater-urls"] -= Clair updater URLs += Information about Clair updaters -The default updaters are tracked in `updater/defaults/defaults.go`. +The following table provides details about each Clair updater, including the configuration parameter, a brief description, relevant URLs, and the associated components that they interact with. This list is not exhaustive, and some servers might issue redirects, while certain request URLs are dynamically constructed to ensure accurate vulnerability data retrieval. -The following table lists the HTTP hosts and paths that Clair attempts to talk to in a default configuration. This list is non-exhaustive. Some servers issue redirects and some request URLs are constructed dynamically. +For Clair, each updater is responsible for fetching and parsing vulnerability data related to a specific package type or distribution. For example, the Debian updater focuses on Debian-based Linux distributions, while the AWS updater focuses on vulnerabilities specific to Amazon Web Services' Linux distributions. Understanding the package type is important for vulnerability management because different package types might have unique security concerns and require specific updates and patches. [NOTE] ==== -If you are using a proxy server in your environment with Clair's updater URLs, you must identify which URL needs to be added to the proxy allowlist to ensure that Clair can access them unimpeded. For more information, see "Information about using Clair in a proxy environment". +If you are using a proxy server in your environment with Clair's updater URLs, you must identify which URL needs to be added to the proxy allowlist to ensure that Clair can access them unimpeded. Use the following table to add updater URLs to your proxy allowlist. ==== -[cols="1a,2a,2a",options="header"] +.Clair updater information +[cols="1a,3a,3a,2a",options="header"] |=== -|Updater | URLs | Component -|`alpine` | * `\https://secdb.alpinelinux.org/` | Alpine Linux SecDB database -|`aws` | * `\http://repo.us-west-2.amazonaws.com/2018.03/updates/x86_64/mirror.list` +|Updater |Description | URLs | Component +|`alpine` |The Alpine updater is responsible for fetching and parsing vulnerability data related to packages in Alpine Linux distributions. | * `\https://secdb.alpinelinux.org/` | Alpine Linux SecDB database +|`aws` | The AWS updater is focused on AWS Linux-based packages, ensuring that vulnerability information specific to Amazon Web Services' custom Linux distributions is kept up-to-date. | * `\http://repo.us-west-2.amazonaws.com/2018.03/updates/x86_64/mirror.list` * `\https://cdn.amazonlinux.com/2/core/latest/x86_64/mirror.list` * `\https://cdn.amazonlinux.com/al2023/core/mirrors/latest/x86_64/mirror.list` | Amazon Web Services (AWS) UpdateInfo -|`debian` | * `\https://deb.debian.org/` +|`debian` |The Debian updater is essential for tracking vulnerabilities in packages associated with Debian-based Linux distributions. | * `\https://deb.debian.org/` * `\https://security-tracker.debian.org/tracker/data/json` | Debian Security Tracker -|`clair.cvss` | * `\https://nvd.nist.gov/feeds/json/cve/1.1/` | National Vulnerability Database (NVD) feed for Common Vulnerabilities and Exposures (CVE) data in JSON format -|`oracle` | * `\https://linux.oracle.com/security/oval/com.oracle.elsa-*.xml.bz2` | Oracle Oval database -|`photon` | * `\https://packages.vmware.com/photon/photon_oval_definitions/` | VMware Photon OS oval definitions -|`rhel` | * `\https://access.redhat.com/security/cve/` +|`clair.cvss`| The Clair Common Vulnerability Scoring System (CVSS) updater focuses on maintaining data about vulnerabilities and their associated CVSS scores. This is not tied to a specific package type but rather to the severity and risk assessment of vulnerabilities in general. | * `\https://nvd.nist.gov/feeds/json/cve/1.1/` | National Vulnerability Database (NVD) feed for Common Vulnerabilities and Exposures (CVE) data in JSON format +|`oracle` |The Oracle updater is dedicated to Oracle Linux packages, maintaining data on vulnerabilities that affect Oracle Linux systems. | * `\https://linux.oracle.com/security/oval/com.oracle.elsa-*.xml.bz2` | Oracle Oval database +|`photon`| The Photon updater deals with packages in VMware Photon OS. | * `\https://packages.vmware.com/photon/photon_oval_definitions/` | VMware Photon OS oval definitions +|`rhel` |The Red Hat Enterprise Linux (RHEL) updater is responsible for maintaining vulnerability data for packages in Red Hat's Enterprise Linux distribution. | * `\https://access.redhat.com/security/cve/` * `\https://access.redhat.com/security/data/oval/v2/PULP_MANIFEST` | Red Hat Enterprise Linux (RHEL) Oval database -|`rhcc` | * `\https://access.redhat.com/security/data/metrics/cvemap.xml` | Resource Handler Configuration Controller (RHCC) database -|`suse` | * `\https://support.novell.com/security/oval/` | SUSE Oval database -|`ubuntu` | * `\https://security-metadata.canonical.com/oval/com.ubuntu.*.cve.oval.xml` +|`rhcc` | The Red Hat Container Catalog (RHCC) updater is connected to Red Hat's container images. This updater ensures that vulnerability information related to Red Hat's containerized software is kept current. | * `\https://access.redhat.com/security/data/metrics/cvemap.xml` | Resource Handler Configuration Controller (RHCC) database +|`suse`| The SUSE updater manages vulnerability information for packages in the SUSE Linux distribution family, including openSUSE, SUSE Enterprise Linux, and others. | * `\https://support.novell.com/security/oval/` | SUSE Oval database +|`ubuntu` | The Ubuntu updater is dedicated to tracking vulnerabilities in packages associated with Ubuntu-based Linux distributions. Ubuntu is a popular distribution in the Linux ecosystem. | * `\https://security-metadata.canonical.com/oval/com.ubuntu.*.cve.oval.xml` * `\https://api.launchpad.net/1.0/` | Ubuntu Oval Database -|`osv` | * `\https://osv-vulnerabilities.storage.googleapis.com/` | Open Source Vulnerabilities database +|`osv` | The Open Source Vulnerability (OSV) updater specializes in tracking vulnerabilities within open source software components. OSV is a critical resource that provides detailed information about security issues found in various open source projects. | * `\https://osv-vulnerabilities.storage.googleapis.com/` | Open Source Vulnerabilities database |=== \ No newline at end of file diff --git a/modules/clair-updaters.adoc b/modules/clair-updaters.adoc index c19e78eaa..73db138e5 100644 --- a/modules/clair-updaters.adoc +++ b/modules/clair-updaters.adoc @@ -8,86 +8,4 @@ Clair uses `Go` packages called _updaters_ that contain the logic of fetching and parsing different vulnerability databases. -Updaters are usually paired with a matcher to interpret if, and how, any vulnerability is related to a package. Administrators might want to update the vulnerability database less frequently, or not import vulnerabilities from databases that they know will not be used. - -[id="configuring-updaters"] -== Configuring updaters - -Updaters can be configured by the `updaters` key at the top of the configuration. If updaters are being run automatically within the matcher process, which is the default setting, the period for running updaters is configured under the matcher's configuration field. - -[id="updater-sets"] -=== Updater sets - -The following sets can be configured with Clair updaters: - -* `alpine` -* `aws` -* `debian` -* `oracle` -* `photon` -* `rhcc` -* `rhel` -* `suse` -* `ubuntu` -* `osv` -* `clair.cvss` - -[id="selecting-updater-sets"] -=== Selecting updater sets - -Specific sets of updaters can be selected by the `sets` list. For example: - -[source,yaml] ----- -updaters: - sets: - - rhel ----- - -If the `sets` field is not populated, it defaults to using all sets. - -[id="filtering-updater-sets"] -=== Filtering updater sets - -To reject an updater from running without disabling an entire set, the `filter` option can be used. - -In the following example, the string is interpreted as a Go `regexp` package. This rejects any updater with a name that does not match. - -[NOTE] -==== -This means that an empty string matches any string. It does not mean that it matches no strings. -==== - -[source,yaml] ----- -updaters: - filter: '^$' ----- - -[id="configuring-specific-updaters"] -=== Configuring specific updaters - -Configuration for specific updaters can be passed by putting a key underneath the `config` parameter of the `updaters` object. The name of an updater might be constructed dynamically, and users should examine logs to ensure updater names are accurate. The specific object that an updater expects should be covered in the updater's documentation. - -In the following example, the `rhel` updater fetches a manifest from a different location: - -[source,yaml] ----- -updaters: - config: - rhel: - url: https://example.com/mirror/oval/PULP_MANIFEST ----- - -[id="disabling-clair-updater-component-managed-db"] -=== Disabling the Clair Updater component - -In some scenarios, users might want to disable the Clair updater component. Disabling updaters is required when running {productname} in a disconnected environment. - -In the following example, Clair updaters are disabled: - -[source,yaml] ----- -matcher: - disable_updaters: true ----- \ No newline at end of file +Updaters are usually paired with a matcher to interpret if, and how, any vulnerability is related to a package. Administrators might want to update the vulnerability database less frequently, or not import vulnerabilities from databases that they know will not be used. \ No newline at end of file diff --git a/modules/clair-vulnerability-scanner-hosts.adoc b/modules/clair-vulnerability-scanner-hosts.adoc index 28d42cfdb..9aef6f842 100644 --- a/modules/clair-vulnerability-scanner-hosts.adoc +++ b/modules/clair-vulnerability-scanner-hosts.adoc @@ -17,8 +17,19 @@ Clair uses the following vulnerability databases to report for issues in your im * VMWare Photon OS database * Amazon Web Services (AWS) UpdateInfo * link:https://osv.dev/[Open Source Vulnerability (OSV) Database] -+ -Clair reports vulnerability and security information for `golang`, `java`, and `ruby` ecosystems through OSV. For information about how Clair does security mapping with the different databases, see -link:https://quay.github.io/claircore/concepts/severity_mapping.html[ClairCore Severity Mapping]. \ No newline at end of file +link:https://quay.github.io/claircore/concepts/severity_mapping.html[Claircore Severity Mapping]. + +[id="information-about-clair-osv"] +== Information about Open Source Vulnerability (OSV) database for Clair + +Open Source Vulnerability (OSV) is a vulnerability database and monitoring service that focuses on tracking and managing security vulnerabilities in open source software. + +OSV provides a comprehensive and up-to-date database of known security vulnerabilities in open source projects. It covers a wide range of open source software, including libraries, frameworks, and other components that are used in software development. For a full list of included ecosystems, see link:https://ossf.github.io/osv-schema/#affectedpackage-field[defined ecosystems]. + +Clair also reports vulnerability and security information for `golang`, `java`, and `ruby` ecosystems through the Open Source Vulnerability (OSV) database. + +By leveraging OSV, developers and organizations can proactively monitor and address security vulnerabilities in open source components that they use, which helps to reduce the risk of security breaches and data compromises in projects. + +For more information about OSV, see link:https://osv.dev/[the OSV website]. \ No newline at end of file diff --git a/modules/config-fields-clair-auth.adoc b/modules/config-fields-clair-auth.adoc index 131092576..c68b309b7 100644 --- a/modules/config-fields-clair-auth.adoc +++ b/modules/config-fields-clair-auth.adoc @@ -5,7 +5,6 @@ The following authorization configuration fields are available for Clair. [cols="3a,1a,2a",options="header"] - |=== | Field | Type | Description | **auth** | Object | Defines Clair's external and intra-service JWT based authentication. If multiple `auth` mechanisms are defined, Clair picks one. Currently, multiple mechanisms are unsupported. @@ -16,3 +15,19 @@ The following authorization configuration fields are available for Clair. | **.psk.iss** | String | A list of JWT issuers to verify. An empty list accepts any issuer in a JWT claim. |=== + +[discrete] +== Example authorization configuration + +The following `authorization` snippet is for a minimal configuration. + +.Example authorization configuration +[source,yaml] +---- +# ... +auth: + psk: + key: MTU5YzA4Y2ZkNzJoMQ== <1> + iss: ["quay"] +# ... +---- \ No newline at end of file diff --git a/modules/config-fields-clair-indexer.adoc b/modules/config-fields-clair-indexer.adoc index 08f2f05b1..81f4467ea 100644 --- a/modules/config-fields-clair-indexer.adoc +++ b/modules/config-fields-clair-indexer.adoc @@ -2,7 +2,7 @@ [id="config-fields-clair-indexer"] = Clair indexer configuration fields -The following indexer configuration fields are available for Clair. +The following table describes the configuration fields for Clair's `indexer` component. [cols="3a,1a,2a",options="header"] |=== @@ -32,4 +32,21 @@ Scanner allows for passing configuration options to layer scanners. The scanner | **.scanner.package** | String | A map with the name of a particular scanner and arbitrary YAML as a value. | **.scanner.repo** | String | A map with the name of a particular scanner and arbitrary YAML as a value. -|=== \ No newline at end of file +|=== + +[discrete] +== Example indexer configuration + +The following example shows a hypothetical indexer configuration for Clair. + +.Example indexer configuration +[source,yaml] +---- +# ... +indexer: + connstring: host=quay-server.example.com port=5433 dbname=clair user=clairuser password=clairpass sslmode=disable + scanlock_retry: 10 + layer_scan_concurrency: 5 + migrations: true +# ... +---- diff --git a/modules/config-fields-clair-matcher.adoc b/modules/config-fields-clair-matcher.adoc index fd8668d27..8415bd7aa 100644 --- a/modules/config-fields-clair-matcher.adoc +++ b/modules/config-fields-clair-matcher.adoc @@ -2,7 +2,7 @@ [id="config-fields-clair-matcher"] = Clair matcher configuration fields -The following matcher configuration fields are available for Clair. +The following table describes the configuration fields for Clair's `matcher` component. [NOTE] ==== @@ -24,7 +24,7 @@ Clair allows for a custom connection pool size. This number directly sets how ma This parameter will be ignored in a future version. Users should configure this through the connection string. -| **.indexer_addr** | String | A matcher contacts an indexer to create a `VulnerabilityReport`. The location of this indexer is required. +| **.indexer_addr** | String | A matcher contacts an indexer to create a vulnerability report. The location of this indexer is required. Defaults to `30m`. @@ -36,9 +36,31 @@ Defaults to `30m`. | **.disable_updaters** | Boolean | Whether to run background updates or not. +Default: `False` + | **.update_retention** | Integer | Sets the number of update operations to retain between garbage collection cycles. This should be set to a safe MAX value based on database size constraints. Defaults to `10m`. If a value of less than `0` is provided, garbage collection is disabled. `2` is the minimum value to ensure updates can be compared to notifications. -|=== \ No newline at end of file +|=== + +[discrete] +== Example matcher configuration + +.Example matcher configuration +[source,yaml] +---- +# ... +matcher: + connstring: >- + host= port=5432 dbname= user= password=D + sslmode=verify-ca sslcert=/etc/clair/ssl/cert.pem sslkey=/etc/clair/ssl/key.pem + sslrootcert=/etc/clair/ssl/ca.pem + indexer_addr: http://clair-v4/ + disable_updaters: false + migrations: true + period: 6h + update_retention: 2 +# ... +---- \ No newline at end of file diff --git a/modules/config-fields-clair-matchers.adoc b/modules/config-fields-clair-matchers.adoc index 9f71683fd..4c3e95396 100644 --- a/modules/config-fields-clair-matchers.adoc +++ b/modules/config-fields-clair-matchers.adoc @@ -2,31 +2,42 @@ [id="config-fields-clair-matchers"] = Clair matchers configuration fields -The following matchers configuration fields are available for Clair. +The following table describes the configuration fields for Clair's `matchers` component. [NOTE] ==== Differs from `matcher` configuration fields. ==== +.Matchers configuration fields [cols="3a,1a,2a",options="header"] |=== | Field | Type | Description -| **matchers** | Array of strings | Provides configuration for the in-tree `matchers` and `remotematchers`. +| **matchers** | Array of strings | Provides configuration for the in-tree `matchers`. -| **.names** | String | A list of string values informing the matcher factory about enabled matchers. If value is set to `null`, the default list of matchers run: -*alpine*, *aws*, *debian*, *oracle*, *photon*, *clair.cvss*, *rhel*, *rhcc*, *suse*, *ubuntu*, *osv* +| **.names** | String | A list of string values informing the matcher factory about enabled matchers. If value is set to `null`, the default list of matchers run. The following strings are accepted: +*alpine-matcher*, *aws-matcher*, *debian-matcher*, *gobin*, *java-maven*, *oracle*, *photon*, *python*, *rhel*, *rhel-container-matcher*, *ruby*, *suse*, *ubuntu-matcher* | **.config** | String | Provides configuration to a specific matcher. A map keyed by the name of the matcher containing a sub-object which will be provided to the matchers factory constructor. For example: +|=== + +[discrete] +== Example matchers configuration + +The following example shows a hypothetical Clair deployment that only requires only the `alpine`, `aws`, `debian`, `oracle` matchers. + +.Example matchers configuration [source,yaml] ---- -config: - python: - ignore_vulns: - - CVE-XYZ - - CVE-ABC ----- -|=== \ No newline at end of file +# ... +matchers: + names: + - "alpine-matcher" + - "aws" + - "debian" + - "oracle" +# ... +---- \ No newline at end of file diff --git a/modules/config-fields-clair-metrics.adoc b/modules/config-fields-clair-metrics.adoc index 983fd4cef..0acb7c719 100644 --- a/modules/config-fields-clair-metrics.adoc +++ b/modules/config-fields-clair-metrics.adoc @@ -15,4 +15,20 @@ The following metrics configuration fields are available for Clair. | **.prometheus** | String | Configuration for a Prometheus metrics exporter. | **.prometheus.endpoint** | String | Defines the path where metrics are served. -|=== \ No newline at end of file +|=== + +[discrete] +== Example metrics configuration + +The following example shows a hypothetical metrics configuration for Clair. + +.Example metrics configuration +[source,yaml] +---- +# ... +metrics: + name: "prometheus" + prometheus: + endpoint: "/metricsz" +# ... +---- \ No newline at end of file diff --git a/modules/config-fields-clair-notifiers.adoc b/modules/config-fields-clair-notifiers.adoc index eccfd2e6c..87afce75e 100644 --- a/modules/config-fields-clair-notifiers.adoc +++ b/modules/config-fields-clair-notifiers.adoc @@ -2,10 +2,9 @@ [id="config-fields-clair-notifiers"] = Clair notifier configuration fields -The following notifier configuration fields are available for Clair. +The general notifier configuration fields for Clair are listed below. [cols="3a,1a,2a",options="header"] - |=== | Field | Type | Description | **notifier** | Object | Provides Clair notifier node configuration. @@ -23,6 +22,43 @@ The following notifier configuration fields are available for Clair. | **.delivery_interval** | String | The frequency at which the notifier attempts delivery of created, or previously failed, notifications. | **.disable_summary** | Boolean | Controls whether notifications should be summarized to one per manifest. +|=== + +[discrete] +== Example notifier configuration + +The following `notifier` snippet is for a minimal configuration. + +.Example notifier configuration +[source,yaml] +---- +# ... +notifier: + connstring: >- + host=DB_HOST port=5432 dbname=notifier user=DB_USER password=DB_PASS + sslmode=verify-ca sslcert=/etc/clair/ssl/cert.pem sslkey=/etc/clair/ssl/key.pem + sslrootcert=/etc/clair/ssl/ca.pem + indexer_addr: http://clair-v4/ + matcher_addr: http://clair-v4/ + delivery_interval: 5s + migrations: true + poll_interval: 15s + webhook: + target: "http://webhook/" + callback: "http://clair-notifier/notifier/api/v1/notifications" + headers: "" + amqp: null + stomp: null +# ... +---- + +[id="clair-webhook-config-fields"] +== Clair webhook configuration fields + +The following webhook fields are available for the Clair notifier environment. + +.Clair webhook fields +|=== | **.webhook** | Object | Configures the notifier for webhook delivery. @@ -33,7 +69,29 @@ The following notifier configuration fields are available for Clair. This will typically be where the Clair notifier is hosted. | **.webhook.headers** | String | A map associating a header name to a list of values. +|=== + +[discrete] +== Example webhook configuration + +.Example webhook configuration +[source,yaml] +---- +# ... +notifier: +# ... + webhook: + target: "http://webhook/" + callback: "http://clair-notifier/notifier/api/v1/notifications" +# ... +---- + +[id="clair-amqp-config-fields"] +== Clair amqp configuration fields +The following Advanced Message Queuing Protocol (AMQP) fields are available for the Clair notifier environment. + +|=== | **.amqp** | Object | Configures the notifier for AMQP delivery. [NOTE] @@ -73,7 +131,42 @@ Clair also allows `SSL_CERT_DIR`, as documented for the Go `crypto/x509` package ==== | **.amqp.tls.key** | String | The filesystem path where a TLS/SSL private key can be read. +|=== + +[discrete] +== Example AMQP configuration + +The following example shows a hypothetical AMQP configuration for Clair. + +.Example AMQP configuration +[source,yaml] +---- +# ... +notifier: +# ... + amqp: + exchange: + name: "" + type: "direct" + durable: true + auto_delete: false + uris: ["amqp://user:pass@host:10000/vhost"] + direct: false + routing_key: "notifications" + callback: "http://clair-notifier/notifier/api/v1/notifications" + tls: + root_ca: "optional/path/to/rootca" + cert: "madatory/path/to/cert" + key: "madatory/path/to/key" +# ... +---- + +[id="clair-stomp-config-fields"] +== Clair STOMP configuration fields + +The following Simple Text Oriented Message Protocol (STOMP) fields are available for the Clair notifier environment. +|=== | **.stomp** | Object | Configures the notifier for STOMP delivery. | **.stomp.direct** | Boolean | If `true`, the notifier delivers individual notifications (not a callback) to the configured STOMP broker. @@ -104,4 +197,29 @@ Clair also respects `SSL_CERT_DIR`, as documented for the Go `crypto/x509` packa | **.stomp.user.login** | String | The STOMP login to connect with. | **.stomp.user.passcode** | String | The STOMP passcode to connect with. -|=== \ No newline at end of file +|=== + +[discrete] +== Example STOMP configuration + +The following example shows a hypothetical STOMP configuration for Clair. + +.Example STOMP configuration +[source,yaml] +---- +# ... +notifier: +# ... + stomp: + desitnation: "notifications" + direct: false + callback: "http://clair-notifier/notifier/api/v1/notifications" + login: + login: "username" + passcode: "passcode" + tls: + root_ca: "optional/path/to/rootca" + cert: "madatory/path/to/cert" + key: "madatory/path/to/key" +# ... +---- \ No newline at end of file diff --git a/modules/config-fields-clair-trace.adoc b/modules/config-fields-clair-trace.adoc index 499eb7e74..9f4f17d17 100644 --- a/modules/config-fields-clair-trace.adoc +++ b/modules/config-fields-clair-trace.adoc @@ -5,7 +5,6 @@ The following trace configuration fields are available for Clair. [cols="3a,1a,2a",options="header"] - |=== | Field | Type | Description | **trace** | Object | Defines distributed tracing configuration based on OpenTelemetry. @@ -33,4 +32,23 @@ The following trace configuration fields are available for Clair. | **.jaeger.tags** | String | Key-value pairs to provide additional metadata. | **.jaeger.buffer_max** | Integer | The maximum number of spans that can be buffered in memory before they are sent to the Jaeger backend for storage and analysis. -|=== \ No newline at end of file +|=== + +[discrete] +== Example trace configuration + +The following example shows a hypothetical trace configuration for Clair. + +.Example trace configuration +[source,yaml] +---- +# ... +trace: + name: "jaeger" + probability: 1 + jaeger: + agent: + endpoint: "localhost:6831" + service_name: "clair" +# ... +---- \ No newline at end of file diff --git a/modules/config-fields-clair-updaters.adoc b/modules/config-fields-clair-updaters.adoc index deba078fe..82d40099e 100644 --- a/modules/config-fields-clair-updaters.adoc +++ b/modules/config-fields-clair-updaters.adoc @@ -2,8 +2,9 @@ [id="config-fields-clair-updaters"] = Clair updaters configuration fields -The following updaters configuration fields are available for Clair. +The following table describes the configuration fields for Clair's `updaters` component. +.Updaters configuration fields [cols="3a,1a,2a",options="header"] |=== | Field | Type | Description @@ -11,32 +12,29 @@ The following updaters configuration fields are available for Clair. | **.sets** | String | A list of values informing the update manager which updaters to run. -If value is set to `null`, the default set of updaters runs the following: *alpine*, *aws*, *debian*, *oracle*, *photon*, *pyupio*, *rhel*, *suse*, *ubuntu* +If value is set to `null`, the default set of updaters runs the following: *alpine*, *aws*, *clair.cvss*, *debian*, *oracle*, *photon*, *osv*, *rhel*, *rhcc* *suse*, *ubuntu* If left blank, zero updaters run. | **.config** | String | Provides configuration to specific updater sets. -A map keyed by the name of the updater set containing a sub-object which will be provided to the updater set's constructor. For example: - -[source,yaml] ----- -config: - ubuntu: - security_tracker_url: http://security.url - ignore_distributions: - - cosmic ----- +A map keyed by the name of the updater set containing a sub-object which will be provided to the updater set's constructor. For a list of the sub-objects for each updater, see "Advanced updater configuration". +|=== -| **..ignore_unpatched** | Boolean | Whether to enable reporting of unpatched vulnerabilities. +[discrete] +== Example updaters configuration -**Default:** `false` +In the following configuration, only the `rhel` set is configured. The `ignore_unpatched` variable, which is specific to the `rhel` updater, is also defined. -[source,terminal] +.Example updaters configuration +[source,yaml] ---- +# ... updaters: + sets: + - rhel config: rhel: ignore_unpatched: false ----- -|=== \ No newline at end of file +# ... +---- \ No newline at end of file diff --git a/modules/config-fields-general-clair.adoc b/modules/config-fields-general-clair.adoc index 24e2ccafd..2bda71416 100644 --- a/modules/config-fields-general-clair.adoc +++ b/modules/config-fields-general-clair.adoc @@ -2,7 +2,7 @@ [id="config-fields-required-clair"] = Clair general fields -The following section describes the general configuration fields available for a Clair deployment: +The following table describes the general configuration fields available for a Clair deployment. [cols="3a,1a,2a",options="header"] |=== @@ -19,3 +19,18 @@ Default: `:6060` | **.cert** | String | The TLS certificate to be used. Must be a full-chain certificate. |=== + +[discrete] +== Example configuration for general Clair fields + +The following example shows a Clair configuration. + +.Example configuration for general Clair fields +[source,yaml] +---- +# ... +http_listen_addr: 0.0.0.0:6060 +introspection_addr: 0.0.0.0:8089 +log_level: info +# ... +---- \ No newline at end of file diff --git a/modules/config-fields-overview.adoc b/modules/config-fields-overview.adoc index 0cfa4cc15..59578a143 100644 --- a/modules/config-fields-overview.adoc +++ b/modules/config-fields-overview.adoc @@ -69,7 +69,7 @@ tls: {} indexer: connstring: "" scanlock_retry: 0 - layer_scan_concurrency: 0 + layer_scan_concurrency: 5 migrations: false scanner: {} airgap: false diff --git a/modules/configuring-clair-updaters.adoc b/modules/configuring-clair-updaters.adoc new file mode 100644 index 000000000..95d4cd30c --- /dev/null +++ b/modules/configuring-clair-updaters.adoc @@ -0,0 +1,594 @@ + +[id="configuring-updaters"] += Configuring updaters + +Updaters can be configured by the `updaters.sets` key in your `clair-config.yaml` file. + +[IMPORTANT] +==== +* If the `sets` field is not populated, it defaults to using all sets. In using all sets, Clair tries to reach the URL or URLs of each updater. If you are using a proxy environment, you must add these URLs to your proxy allowlist. +* If updaters are being run automatically within the matcher process, which is the default setting, the period for running updaters is configured under the matcher's configuration field. +==== + +[id="selecting-updater-sets"] +== Selecting specific updater sets + +Use the following references to select one, or multiple, updaters for your {productname} deployment. + +[discrete] +[id="configuring-clair-multiple-updaters"] +=== Configuring Clair for multiple updaters + +.Multiple specific updaters +[source,yaml] +---- +#... +updaters: + sets: + - alpine + - aws + - osv +#... +---- + +[discrete] +[id="configuring-clair-alpine"] +=== Configuring Clair for Alpine + +.Alpine config.yaml example +[source,yaml] +---- +#... +updaters: + sets: + - alpine +#... +---- + +[discrete] +[id="configuring-clair-aws"] +=== Configuring Clair for AWS + +.AWS config.yaml example +[source,yaml] +---- +#... +updaters: + sets: + - aws +#... +---- + +[discrete] +[id="configuring-clair-debian"] +=== Configuring Clair for Debian + +.Debian config.yaml example +[source,yaml] +---- +#... +updaters: + sets: + - debian +#... +---- + +[discrete] +[id="configuring-clair-clair-cvss"] +=== Configuring Clair for Clair CVSS + +.Clair CVSS config.yaml example +[source,yaml] +---- +#... +updaters: + sets: + - clair.cvss +#... +---- + +[discrete] +[id="configuring-clair-oracle"] +=== Configuring Clair for Oracle + +.Oracle config.yaml example +[source,yaml] +---- +#... +updaters: + sets: + - oracle +#... +---- + +[discrete] +[id="configuring-clair-photon"] +=== Configuring Clair for Photon +.Photon config.yaml example +[source,yaml] +---- +#... +updaters: + sets: + - photon +#... +---- + +[discrete] +[id="configuring-clair-suse"] +=== Configuring Clair for SUSE + +.SUSE config.yaml example +[source,yaml] +---- +#... +updaters: + sets: + - suse +#... +---- + +[discrete] +[id="configuring-clair-ubuntu"] +=== Configuring Clair for Ubuntu + +.Ubuntu config.yaml example +[source,yaml] +---- +#... +updaters: + sets: + - ubuntu +#... +---- + +[discrete] +[id="configuring-clair-osv"] +=== Configuring Clair for OSV + +.OSV config.yaml example +[source,yaml] +---- +#... +updaters: + sets: + - osv +#... +---- + +[id="full-rhel-coverage"] +== Selecting updater sets for full {rhel} coverage + +For full coverage of vulnerabilities in {rhel}, you must use the following updater sets: + +* `rhel`. This updater ensures that you have the latest information on the vulnerabilities that affect {rhel-short}. +* `rhcc`. This updater keeps track of vulnerabilities related to Red hat's container images. +* `clair.cvss`. This updater offers a comprehensive view of the severity and risk assessment of vulnerabilities by providing Common Vulnerabilities and Exposures (CVE) scores. +* `osv`. This updater focuses on tracking vulnerabilities in open-source software components. This updater is recommended due to how common the use of Java and Go are in {rhel-short} products. + +.{rhel-short} updaters example +[source,yaml] +---- +#... +updaters: + sets: + - rhel + - rhcc + - clair.cvss + - osv +#... +---- + +[id="configuring-specific-updaters"] +== Advanced updater configuration + +In some cases, users might want to configure updaters for specific behavior, for example, if you want to allowlist specific ecosystems for the Open Source Vulnerabilities (OSV) updaters. + +Advanced updater configuration might be useful for proxy deployments or air gapped deployments. Configuration for specific updaters in these scenarios can be passed by putting a key underneath the `config` environment variable of the `updaters` object. Users should examine their Clair logs to double-check names. + +The following YAML snippets detail the various settings available to some Clair updater + +[IMPORTANT] +==== +For more users, advanced updater configuration is unnecessary. +==== + +[discrete] +=== Configuring the alpine updater + +[source,yaml] +---- +#... +updaters: + sets: + - apline + config: + alpine: + url: https://secdb.alpinelinux.org/ +#... +---- + +[discrete] +=== Configuring the debian updater + +[source,yaml] +---- +#... +updaters: + sets: + - debian + config: + debian: + mirror_url: https://deb.debian.org/ + json_url: https://security-tracker.debian.org/tracker/data/json +#... +---- + +[discrete] +=== Configuring the clair.cvss updater + +[source,yaml] +---- +#... +updaters: + config: + clair.cvss: + url: https://nvd.nist.gov/feeds/json/cve/1.1/ +#... +---- + +[discrete] +=== Configuring the oracle updater + +[source,yaml] +---- +#... +updaters: + sets: + - oracle + config: + oracle-2023-updater: + url: + - https://linux.oracle.com/security/oval/com.oracle.elsa-2023.xml.bz2 + oracle-2022-updater: + url: + - https://linux.oracle.com/security/oval/com.oracle.elsa-2022.xml.bz2 +#... +---- + +[discrete] +=== Configuring the photon updater + +[source,yaml] +---- +#... +updaters: + sets: + - photon + config: + photon: + url: https://packages.vmware.com/photon/photon_oval_definitions/ +#... +---- + +[discrete] +=== Configuring the rhel updater + +[source,yaml] +---- +#... +updaters: + sets: + - rhel + config: + rhel: + url: https://access.redhat.com/security/data/oval/v2/PULP_MANIFEST + ignore_unpatched: true <1> +#... +---- +<1> Boolean. Whether to include information about vulnerabilities that do not have corresponding patches or updates available. + +[discrete] +=== Configuring the rhcc updater + +[source,yaml] +---- +#... +updaters: + sets: + - rhcc + config: + rhcc: + url: https://access.redhat.com/security/data/metrics/cvemap.xml +#... +---- + +[discrete] +=== Configuring the suse updater + +[source,yaml] +---- +#... +updaters: + sets: + - suse + config: + suse: + url: https://support.novell.com/security/oval/ +#... +---- + +[discrete] +=== Configuring the ubuntu updater + +[source,yaml] +---- +#... +updaters: + config: + ubuntu: + url: https://api.launchpad.net/1.0/ + name: ubuntu + force: <1> + - name: focal <2> + version: 20.04 <3> +#... +---- +<1> Used to force the inclusion of specific distribution and version details in the resulting UpdaterSet, regardless of their status in the API response. Useful when you want to ensure that particular distributions and versions are consistently included in your updater configuration. +<2> Specifies the distribution name that you want to force to be included in the UpdaterSet. +<3> Specifies the version of the distribution you want to force into the UpdaterSet. + +[discrete] +=== Configuring the osv updater + +[source,yaml] +---- +#... +updaters: + sets: + - osv + config: + osv: + url: https://osv-vulnerabilities.storage.googleapis.com/ + allowlist: <1> + - npm + - pypi +#... +---- +<1> The list of ecosystems to allow. When left unset, all ecosystems are allowed. Must be lowercase. For a list of supported ecosystems, see the documentation for link:https://ossf.github.io/osv-schema/#affectedpackage-field[defined ecosystems]. + +[id="disabling-clair-updater-component-managed-db"] +== Disabling the Clair Updater component + +In some scenarios, users might want to disable the Clair updater component. Disabling updaters is required when running {productname} in a disconnected environment. + +In the following example, Clair updaters are disabled: + +[source,yaml] +---- +#... +matcher: + disable_updaters: true +#... +---- + + +//// + + +The following sections outline how one might configure specific updaters in Clair when it is being used in a proxy environment. + +[IMPORTANT] +==== +These are examples, and depending on how your proxy server is configured to route requests might impact how your `clair-config.yaml` file structure is determined. +==== + +[discrete] +=== Configuring the alpine updater for proxy environments + +[source,yaml] +---- +#... +updaters: + sets: + - apline + config: + alpine: + url: https:///secdb/alpine/ <1> +---- +<1> Based on the `alpine` updater URL `\https://secdb.alpinelinux.org/`. + +[discrete] +=== Configuring the aws updater for proxy environments + +[source,yaml] +---- +#... +updaters: + sets: + - aws + config: + aws: + url: https:///updates/x86_64/mirror.list <1> + url: https:///core/latest/x86_64/mirror.list <2> + url: https:///al2023/core/mirrors/latest/x86_64/mirror.list <3> +---- +<1> Based on the `aws` updater URL `\http://repo.us-west-2.amazonaws.com/2018.03/updates/x86_64/mirror.list`. +<2> Based on the `aws` updater URL `\https://cdn.amazonlinux.com/2/core/latest/x86_64/mirror.list`. +<3> Based on the `aws` updater URL `\https://cdn.amazonlinux.com/al2023/core/mirrors/latest/x86_64/mirror.list`. + +[discrete] +=== Configuring the debian updater for proxy environments + +[source,yaml] +---- +#... +updaters: + sets: + - debian + config: + debian: + mirror_url: https:///debian-archive/ <1> + json_url: https:///debian-json/ <2> +---- +<1> Based on the `debian` updater URL `\https://deb.debian.org/`. +<2> Based on the `debian` updater URL `\https://security-tracker.debian.org/tracker/data/json`. + +[discrete] +=== Configuring the clair.cvss updater for proxy environments + +[source,yaml] +---- +#... +updaters: + config: + clair.cvss: + url: https:///feeds/json/cve/1.1/ <1> +---- +<1> Based on the `clair.cvss` updater URL `\https://nvd.nist.gov/feeds/json/cve/1.1/`. + +[discrete] +=== Configuring the oracle updater for proxy environments + +[source,yaml] +---- +#... +updaters: + sets: + - oracle + config: + oracle: + url: https:///security/oval/com.oracle.elsa-*.xml.bz2 <1> +#... +---- +<1> Based on the `oracle` updater URL `\https://linux.oracle.com/security/oval/com.oracle.elsa-*.xml.bz2`. + +[discrete] +=== Configuring the photon updater for proxy environments + +[source,yaml] +---- +#... +updaters: + sets: + - photon + config: + photon: + url: https:///photon/photon_oval_definitions/ <1> +#... +---- +<1> Based on the `photon` updater URL `\https://packages.vmware.com/photon/photon_oval_definitions/`. + + +[discrete] +=== Configuring the rhel updater for proxy environments + +[source,yaml] +---- +#... +updaters: + sets: + - rhel + config: + rhel: + url: https:///mirror/oval/PULP_MANIFEST <1> + url: https:///security/cve/ <2> + ignore_unpatched: true <3> +#... +---- +<1> Based on the `rhel` updater URL `\https://access.redhat.com/security/data/oval/v2/PULP_MANIFEST`. +<2> Based on the `rhel` updater URL `\https://access.redhat.com/security/cve/`. +<3> Boolean. Whether to include information about vulnerabilities that do not have corresponding patches or updates available. + +[discrete] +=== Configuring the rhcc updater for proxy environments + +[source,yaml] +---- +#... +updaters: + sets: + - rhcc + config: + rhcc: + url: https:///security/data/metrics/cvemap.xml <1> +#... +---- +<1> Based on the `rhcc` updater URL `\https://access.redhat.com/security/data/metrics/cvemap.xml`. + +[discrete] +=== Configuring the suse updater for proxy environments + +[source,yaml] +---- +#... +updaters: + sets: + - suse + config: + suse: + url: https:///security/oval/ <1> +#... +---- +<1> Based on the `suse` updater URL `\https://support.novell.com/security/oval/`. + +[discrete] +=== Configuring the ubuntu updater for proxy environments + +[source,yaml] +---- +#... +updaters: + sets: + - ubuntu + config: + ubuntu: + - url: https:///ubuntu-cve-oval/ <1> + - url: https:///ubuntu-launchpad-api/ <2> + name: ubuntu + force: <3> + - name: focal <4> + version: 20.04 <5> +#... +---- +<1> Based on the `ubuntu` updater URL `\https://security-metadata.canonical.com/oval/com.ubuntu.*.cve.oval.xml`. +<2> Based on the `ubuntu` updater URL `\https://api.launchpad.net/1.0/`. +<3> Used to force the inclusion of specific distribution and version details in the resulting UpdaterSet, regardless of their status in the API response. Useful when you want to ensure that particular distributions and versions are consistently included in your updater configuration. +<4> Specifies the distribution name that you want to force to be included in the UpdaterSet. In this case, it's set to `focal` to specify the Ubuntu distribution with the name `focal`. +<5> Specifies the version of the distribution you want to force into the UpdaterSet. Here, it's set to `20.04` to indicate that the specific version of the `focal` distribution to be included is `20.04`. + +[discrete] +=== Configuring the osv updater for proxy environments + +[source,yaml] +---- +#... +updaters: + sets: + - osv + config: + osv: + url: https:///osv-vulnerabilities/ <1> + allowlist: <2> + - npm + - PyPI +#... +---- +<1> Based on the `osv` updater URL `\https://osv-vulnerabilities.storage.googleapis.com/`. +<2> The list of ecosystems to allow. When left unset, all ecosystems are allowed. For a list of supported ecosystems, see the documentation for link:https://ossf.github.io/osv-schema/#affectedpackage-field[defined ecosystems]. + +[id="disabling-clair-updater-component-managed-db"] +== Disabling the Clair Updater component + +In some scenarios, users might want to disable the Clair updater component. Disabling updaters is required when running {productname} in a disconnected environment. + +In the following example, Clair updaters are disabled: + +[source,yaml] +---- +#... +matcher: + disable_updaters: true +#... +---- +//// \ No newline at end of file