6180 clair updates (#802)

* Reworks clair documentation

* 2

---------

Co-authored-by: Steven Smith <[email protected]>

clair/master.adoc

@@ -20,7 +20,8 @@ include::modules/clair-concepts.adoc[leveloffset=+1]
 include::modules/clair-authentication.adoc[leveloffset=+2]
 //include::modules/testing-clair.adoc[leveloffset=+1]
 include::modules/clair-updaters.adoc[leveloffset=+2]
-include::modules/clair-updater-urls.adoc[leveloffset=+3]
+include::modules/clair-updater-urls.adoc[leveloffset=+2]
+include::modules/configuring-clair-updaters.adoc[leveloffset=+2]
 include::modules/clair-cve.adoc[leveloffset=+2]
 include::modules/fips-overview.adoc[leveloffset=+2]
 

modules/clair-concepts.adoc

@@ -16,12 +16,11 @@ A Clair analysis is broken down into three distinct parts: indexing, matching, a
 [id="clair-indexing-concept"]
 === Indexing
 
-Clair's indexer service is responsible for indexing a manifest. In Clair, manifests are representations of a container image. The indexer service is the component that Clair uses to understand the contents of layers. Clair leverages the fact that Open Container Initiative (OCI) manifests and layers are content addressed to reduce duplicate work.
+Clair's indexer service plays a crucial role in understanding the makeup of a container image. In Clair, container image representations called "manifests." Manifests are used to comprehend the contents of the image's layers. To streamline this process, Clair takes advantage of the fact that Open Container Initiative (OCI) manifests and layers are designed for content addressing, reducing repetitive tasks.
 
-Indexing involves taking a manifest representing a container image and computing its constituent parts. The indexer tries to discover what packages exist in the image, what distribution the image is derived from, and what package repositories are used within the image. When this information is computed, it is persisted into an `IndexReport`.
-
-The `IndexReport` is stored in Clair's database. It can be fed to a `matcher` node to compute the vulnerability report.
+During indexing, a manifest that represents a container image is taken and broken down into its essential components. The indexer's job is to uncover the image's contained packages, its origin distribution, and the package repositories it relies on. This valuable information is then recorded and stored within Clair's database. The insights gathered during indexing  serve as the basis for generating a comprehensive vulnerability report. This report can be seamlessly transferred to a matcher node for further analysis and action, helping users make informed decisions about their container images' security.
 
+////
 [id="content-addressability"]
 ==== Content addressability
 
@@ -32,15 +31,16 @@ For example, consider how many images in a registry might use `ubuntu:artful` as
 In some cases, Clair should re-index a manifest. For example, when an internal component such as a package scanner is updated, Clair performs the analysis with the new package scanner. Clair has enough information to determine that a component has changed and that the `IndexReport` might be different the second time, and as a result it re-indexes the manifest.
 
 A client can track Clair's `index_state` endpoint to understand when an internal component has changed, and can subsequently issue re-indexes. See the Clair API guide to learn how to view Clair's API specification.
+////
 
 [id="clair-matching-concept"]
 === Matching
 
-With Clair, a matcher node is responsible for matching vulnerabilities to a provided `IndexReport`.
+With Clair, a matcher node is responsible for matching vulnerabilities to a provided index report. 
 
-Matchers are responsible for keeping the database of vulnerabilities up to date. Matchers will typically run a set of updaters, which periodically probe their data sources for new content. New vulnerabilities are stored in the database when they are discovered.
+Matchers are responsible for keeping the database of vulnerabilities up to date. Matchers run a set of updaters, which periodically probe their data sources for new content. New vulnerabilities are stored in the database when they are discovered.
 
-The matcher API is designed to be used often. It is designed to always provide the most recent `VulnerabilityReport` when queried. The `VulnerabilityReport` summarizes both a manifest's content and any vulnerabilities affecting the content.
+The matcher API is designed to be used often. It is designed to always provide the most recent vulnerability report when queried. The vulnerability report summarizes both a manifest's content and any vulnerabilities affecting the content.
 
 // See. . . to learn more about how to view the Clair API specification and to work with the matcher API.
 
@@ -63,7 +63,7 @@ Clair uses a notifier service that keeps track of new security database updates
 When the notifier becomes aware of new vulnerabilities affecting a previously indexed manifest, it uses the configured methods in your `config.yaml` file to issue notifications about the new changes. Returned notifications express the most severe vulnerability discovered because of the change. This avoids creating excessive notifications for the same security database update.
 
 When a user receives a notification, it issues a new request against the matcher to receive an up to date vulnerability report.
-
+////
 The notification schema is the JSON marshalled form of the following types:
 
 [source,json]
@@ -92,6 +92,7 @@ type VulnSummary struct {
     Links          string                  `json:"links"`
 }
 ----
+////
 
 You can subscribe to notifications through the following mechanics:
 
@@ -101,6 +102,7 @@ You can subscribe to notifications through the following mechanics:
 
 Configuring the notifier is done through the Clair YAML configuration file.
 
+////
 [id=webhook-delivery]
 ==== Webhook delivery
 
@@ -153,3 +155,4 @@ The notifier runs in `NOTIFIER_TEST_MODE` until the environment variable is clea
 To delete the notification, you can use the `DELETE` API call. Deleting a notification ID manually cleans up resources in the notifier. If you do not use the `DELETE` API call, the notifier waits a predetermined length of time before clearing delivered notifications from its database.
 
 // For more information on the `DELETE` API call, see. . .
+////

modules/clair-updater-urls.adoc

@@ -4,34 +4,35 @@
 
 :_content-type: CONCEPT
 [id="clair-updater-urls"]
-= Clair updater URLs
+= Information about Clair updaters
 
-The default updaters are tracked in `updater/defaults/defaults.go`.
+The following table provides details about each Clair updater, including the configuration parameter, a brief description, relevant URLs, and the associated components that they interact with. This list is not exhaustive, and some servers might issue redirects, while certain request URLs are dynamically constructed to ensure accurate vulnerability data retrieval.
 
-The following table lists the HTTP hosts and paths that Clair attempts to talk to in a default configuration. This list is non-exhaustive. Some servers issue redirects and some request URLs are constructed dynamically. 
+For Clair, each updater is responsible for fetching and parsing vulnerability data related to a specific package type or distribution. For example, the Debian updater focuses on Debian-based Linux distributions, while the AWS updater focuses on vulnerabilities specific to Amazon Web Services' Linux distributions. Understanding the package type is important for vulnerability management because different package types might have unique security concerns and require specific updates and patches.
 
 [NOTE]
 ====
-If you are using a proxy server in your environment with Clair's updater URLs, you must identify which URL needs to be added to the proxy allowlist to ensure that Clair can access them unimpeded. For more information, see "Information about using Clair in a proxy environment".
+If you are using a proxy server in your environment with Clair's updater URLs, you must identify which URL needs to be added to the proxy allowlist to ensure that Clair can access them unimpeded. Use the following table to add updater URLs to your proxy allowlist.
 ====
 
-[cols="1a,2a,2a",options="header"]
+.Clair updater information
+[cols="1a,3a,3a,2a",options="header"]
 |===
-|Updater | URLs | Component 
-|`alpine` | * `\https://secdb.alpinelinux.org/` | Alpine Linux SecDB database
-|`aws` | * `\http://repo.us-west-2.amazonaws.com/2018.03/updates/x86_64/mirror.list`
+|Updater |Description  | URLs | Component 
+|`alpine` |The Alpine updater is responsible for fetching and parsing vulnerability data related to packages in Alpine Linux distributions. | * `\https://secdb.alpinelinux.org/` | Alpine Linux SecDB database
+|`aws` | The AWS updater is focused on AWS Linux-based packages, ensuring that vulnerability information specific to Amazon Web Services' custom Linux distributions is kept up-to-date. | * `\http://repo.us-west-2.amazonaws.com/2018.03/updates/x86_64/mirror.list`
 * `\https://cdn.amazonlinux.com/2/core/latest/x86_64/mirror.list` 
 * `\https://cdn.amazonlinux.com/al2023/core/mirrors/latest/x86_64/mirror.list` | Amazon Web Services (AWS) UpdateInfo
-|`debian` | * `\https://deb.debian.org/`
+|`debian` |The Debian updater is essential for tracking vulnerabilities in packages associated with Debian-based Linux distributions.  | * `\https://deb.debian.org/`
 * `\https://security-tracker.debian.org/tracker/data/json` | Debian Security Tracker
-|`clair.cvss` | * `\https://nvd.nist.gov/feeds/json/cve/1.1/` | National Vulnerability Database (NVD) feed for Common Vulnerabilities and Exposures (CVE) data in JSON format
-|`oracle` | * `\https://linux.oracle.com/security/oval/com.oracle.elsa-*.xml.bz2` |  Oracle Oval database 
-|`photon` | * `\https://packages.vmware.com/photon/photon_oval_definitions/` | VMware Photon OS oval definitions 
-|`rhel` | * `\https://access.redhat.com/security/cve/`
+|`clair.cvss`| The Clair Common Vulnerability Scoring System (CVSS) updater focuses on maintaining data about vulnerabilities and their associated CVSS scores. This is not tied to a specific package type but rather to the severity and risk assessment of vulnerabilities in general. | * `\https://nvd.nist.gov/feeds/json/cve/1.1/` | National Vulnerability Database (NVD) feed for Common Vulnerabilities and Exposures (CVE) data in JSON format
+|`oracle` |The Oracle updater is dedicated to Oracle Linux packages, maintaining data on vulnerabilities that affect Oracle Linux systems. | * `\https://linux.oracle.com/security/oval/com.oracle.elsa-*.xml.bz2` |  Oracle Oval database 
+|`photon`|  The Photon updater deals with packages in VMware Photon OS. | * `\https://packages.vmware.com/photon/photon_oval_definitions/` | VMware Photon OS oval definitions 
+|`rhel` |The Red Hat Enterprise Linux (RHEL) updater is responsible for maintaining vulnerability data for packages in Red Hat's Enterprise Linux distribution. | * `\https://access.redhat.com/security/cve/`
 * `\https://access.redhat.com/security/data/oval/v2/PULP_MANIFEST` | Red Hat Enterprise Linux (RHEL) Oval database 
-|`rhcc` | * `\https://access.redhat.com/security/data/metrics/cvemap.xml` | Resource Handler Configuration Controller (RHCC) database 
-|`suse` | * `\https://support.novell.com/security/oval/` | SUSE Oval database 
-|`ubuntu` | * `\https://security-metadata.canonical.com/oval/com.ubuntu.*.cve.oval.xml`
+|`rhcc` | The Red Hat Container Catalog (RHCC) updater is connected to Red Hat's container images. This updater ensures that vulnerability information related to Red Hat's containerized software is kept current. | * `\https://access.redhat.com/security/data/metrics/cvemap.xml` | Resource Handler Configuration Controller (RHCC) database 
+|`suse`| The SUSE updater manages vulnerability information for packages in the SUSE Linux distribution family, including openSUSE, SUSE Enterprise Linux, and others. | * `\https://support.novell.com/security/oval/` | SUSE Oval database 
+|`ubuntu` | The Ubuntu updater is dedicated to tracking vulnerabilities in packages associated with Ubuntu-based Linux distributions. Ubuntu is a popular distribution in the Linux ecosystem. | * `\https://security-metadata.canonical.com/oval/com.ubuntu.*.cve.oval.xml`
 * `\https://api.launchpad.net/1.0/` | Ubuntu Oval Database
-|`osv` | * `\https://osv-vulnerabilities.storage.googleapis.com/` | Open Source Vulnerabilities database
+|`osv` | The Open Source Vulnerability (OSV) updater specializes in tracking vulnerabilities within open source software components. OSV is a critical resource that provides detailed information about security issues found in various open source projects. | * `\https://osv-vulnerabilities.storage.googleapis.com/` | Open Source Vulnerabilities database
 |===

modules/clair-updaters.adoc

@@ -8,86 +8,4 @@
 
 Clair uses `Go` packages called _updaters_ that contain the logic of fetching and parsing different vulnerability databases.
 
-Updaters are usually paired with a matcher to interpret if, and how, any vulnerability is related to a package. Administrators might want to update the vulnerability database less frequently, or not import vulnerabilities from databases that they know will not be used.
-
-[id="configuring-updaters"]
-== Configuring updaters
-
-Updaters can be configured by the `updaters` key at the top of the configuration. If updaters are being run automatically within the matcher process, which is the default setting, the period for running updaters is configured under the matcher's configuration field.
-
-[id="updater-sets"]
-=== Updater sets
-
-The following sets can be configured with Clair updaters:
-
-* `alpine`
-* `aws`
-* `debian`
-* `oracle`
-* `photon`
-* `rhcc`
-* `rhel`
-* `suse`
-* `ubuntu`
-* `osv`
-* `clair.cvss`
-
-[id="selecting-updater-sets"]
-=== Selecting updater sets
-
-Specific sets of updaters can be selected by the `sets` list. For example:
-
-[source,yaml]
-----
-updaters:
-  sets:
-    - rhel
-----
-
-If the `sets` field is not populated, it defaults to using all sets.
-
-[id="filtering-updater-sets"]
-=== Filtering updater sets
-
-To reject an updater from running without disabling an entire set, the `filter` option can be used.
-
-In the following example, the string is interpreted as a Go `regexp` package. This rejects any updater with a name that does not match.
-
-[NOTE]
-====
-This means that an empty string matches any string. It does not mean that it matches no strings.
-====
-
-[source,yaml]
-----
-updaters:
-  filter: '^$'
-----
-
-[id="configuring-specific-updaters"]
-=== Configuring specific updaters
-
-Configuration for specific updaters can be passed by putting a key underneath the `config` parameter of the `updaters` object. The name of an updater might be constructed dynamically, and users should examine logs to ensure updater names are accurate. The specific object that an updater expects should be covered in the updater's documentation.
-
-In the following example, the `rhel` updater fetches a manifest from a different location:
-
-[source,yaml]
-----
-updaters:
-  config:
-    rhel:
-      url: https://example.com/mirror/oval/PULP_MANIFEST
-----
-
-[id="disabling-clair-updater-component-managed-db"]
-=== Disabling the Clair Updater component
-
-In some scenarios, users might want to disable the Clair updater component. Disabling updaters is required when running {productname} in a disconnected environment.
-
-In the following example, Clair updaters are disabled:
-
-[source,yaml]
-----
-matcher:
-  disable_updaters: true
-----
+Updaters are usually paired with a matcher to interpret if, and how, any vulnerability is related to a package. Administrators might want to update the vulnerability database less frequently, or not import vulnerabilities from databases that they know will not be used.

modules/clair-vulnerability-scanner-hosts.adoc

@@ -17,8 +17,19 @@ Clair uses the following vulnerability databases to report for issues in your im
 * VMWare Photon OS database
 * Amazon Web Services (AWS) UpdateInfo
 * link:https://osv.dev/[Open Source Vulnerability (OSV) Database]
-+
-Clair reports vulnerability and security information for `golang`, `java`, and `ruby` ecosystems through OSV.
 
 For information about how Clair does security mapping with the different databases, see
-link:https://quay.github.io/claircore/concepts/severity_mapping.html[ClairCore Severity Mapping].
+link:https://quay.github.io/claircore/concepts/severity_mapping.html[Claircore Severity Mapping].
+
+[id="information-about-clair-osv"]
+== Information about Open Source Vulnerability (OSV) database for Clair
+
+Open Source Vulnerability (OSV) is a vulnerability database and monitoring service that focuses on tracking and managing security vulnerabilities in open source software. 
+
+OSV provides a comprehensive and up-to-date database of known security vulnerabilities in open source projects. It covers a wide range of open source software, including libraries, frameworks, and other components that are used in software development. For a full list of included ecosystems, see link:https://ossf.github.io/osv-schema/#affectedpackage-field[defined ecosystems]. 
+
+Clair also reports vulnerability and security information for `golang`, `java`, and `ruby` ecosystems through the Open Source Vulnerability (OSV) database.
+
+By leveraging OSV, developers and organizations can proactively monitor and address security vulnerabilities in open source components that they use, which helps to reduce the risk of security breaches and data compromises in projects. 
+
+For more information about OSV, see link:https://osv.dev/[the OSV website].

modules/config-fields-clair-auth.adoc

@@ -5,7 +5,6 @@
 The following authorization configuration fields are available for Clair.
 
 [cols="3a,1a,2a",options="header"]
-
 |===
 | Field | Type | Description
 | **auth** | Object | Defines Clair's external and intra-service JWT based authentication. If multiple `auth` mechanisms are defined, Clair picks one. Currently, multiple mechanisms are unsupported.
@@ -16,3 +15,19 @@ The following authorization configuration fields are available for Clair.
 
 | **.psk.iss** | String | A list of JWT issuers to verify. An empty list accepts any issuer in a JWT claim.
 |===
+
+[discrete]
+== Example authorization configuration
+
+The following `authorization` snippet is for a minimal configuration. 
+
+.Example authorization configuration
+[source,yaml]
+----
+# ...
+auth:
+  psk:
+    key: MTU5YzA4Y2ZkNzJoMQ== <1>
+    iss: ["quay"]
+# ...
+----

modules/config-fields-clair-indexer.adoc

@@ -2,7 +2,7 @@
 [id="config-fields-clair-indexer"]
 = Clair indexer configuration fields
 
-The following indexer configuration fields are available for Clair.
+The following table describes the configuration fields for Clair's `indexer` component.
 
 [cols="3a,1a,2a",options="header"]
 |===
@@ -32,4 +32,21 @@ Scanner allows for passing configuration options to layer scanners. The scanner
 | **.scanner.package**  | String | A map with the name of a particular scanner and arbitrary YAML as a value.
 
 | **.scanner.repo** | String | A map with the name of a particular scanner and arbitrary YAML as a value.
-|===
+|===
+
+[discrete]
+== Example indexer configuration
+
+The following example shows a hypothetical indexer configuration for Clair. 
+
+.Example indexer configuration
+[source,yaml]
+----
+# ...
+indexer:
+  connstring: host=quay-server.example.com port=5433 dbname=clair user=clairuser password=clairpass sslmode=disable 
+  scanlock_retry: 10
+  layer_scan_concurrency: 5
+  migrations: true
+# ...
+----