Skip to content

Commit

Permalink
Rework checksum docs
Browse files Browse the repository at this point in the history
  • Loading branch information
quba42 committed Aug 30, 2023
1 parent 59775f3 commit 7086d62
Show file tree
Hide file tree
Showing 2 changed files with 26 additions and 41 deletions.
4 changes: 2 additions & 2 deletions docs/external_references.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,13 @@
.. _pulpcore documentation:
https://docs.pulpproject.org/pulpcore/
.. _pulpcore configuration documentation:
https://docs.pulpproject.org/pulpcore/installation/configuration.html
https://docs.pulpproject.org/pulpcore/configuration/index.html
.. _pulpcore installation options:
https://docs.pulpproject.org/pulpcore/installation/index.html
.. _pulpcore plugin API deprecation policy:
https://docs.pulpproject.org/pulpcore/plugins/plugin-writer/concepts/index.html#plugin-api-stability-and-deprecation-policy
.. _pulpcore settings documentation:
https://docs.pulpproject.org/pulpcore/settings.html#pulp-settings
https://docs.pulpproject.org/pulpcore/configuration/settings.html#pulp-settings
.. _pulpcore import-export docs:
https://docs.pulpproject.org/pulpcore/workflows/import-export.html
.. _pulpcore metadata signing docs:
Expand Down
63 changes: 24 additions & 39 deletions docs/workflows/checksums.rst
Original file line number Diff line number Diff line change
@@ -1,51 +1,38 @@
.. _checksums:

Configure Checksum Support
Configuring Checksums
================================================================================

.. include:: ../external_references.rst

``pulp_deb`` supports the following checksum algorithms: MD5, SHA-1, SHA-256, and SHA-512.
However, only algorithms that are present in the ``ALLOWED_CONTENT_CHECKSUMS`` setting will actually be used.
What checksum algorithems are available on any given Pulp instance, is controlled by the pulpcore ``ALLOWED_CONTENT_CHECKSUMS`` setting.
If enabled, ``pulp_deb`` will make use of MD5, SHA-1, SHA-256, and SHA-512.
SHA-256 is required and cannot be disabled.

.. warning::
Starting with pulpcore version 3.11, pulpcore will remove ``md5`` and ``sha1`` from the list of ``ALLOWED_CONTENT_CHECKSUMS`` by default.
At that point, those checksums will no longer be available for use by the ``pulp_deb`` plugin, unless users explicitly opt back in.
Without opt in, the ``pulp_deb`` plugin will be unable to include the relevant metadata fields (``MD5Sum``, and ``SHA1``) in any metadata files (such as release files and package indecies), published by the plugin's APT publisher.
Since it is almost universal practice within the Debian ecosystem to publish MD5 in particular, there is no telling what might break without it.
As a result, it is recommended most ``pulp_deb`` users opt in to at least MD5.
If you are unable to enable MD5 for compliance reasons, or if you are feeling adventurous, you are welcome to put the claim that MD5 is merely "highly recommended" to the test.

See the `release file format`_ description in the Debian Wiki for more information.


Background: What are the checksums used for?
--------------------------------------------------------------------------------
.. important::
For compliance reasons, MD5 and SHA-1 have been disabled by default since pulpcore 3.11.

APT repository metadata files (release files and package indecies) will typically include several checksums for all files that they reference.
During synchronization, ``pulp_deb`` will check the downloaded files against the checksums provided (so long as the checksum type is one of the supported algorithms and is present in the ``ALLOWED_CONTENT_CHECKSUMS`` setting).
Together with a valid (and trusted) release file signature this will guarantee the integrity of the synchronized repository.

During publication, the APT publisher will include any supported (and permitted) checksumtypes in any metadata files it publishes.
.. note::
It is almost universal practice within the Debian ecosystem to make use of MD5 in particular.
If your Pulp instance is configured to disallow the MD5 checksum algorithm, ``pulp_deb`` will log a warning with every sync as well as every use of the APT publisher.
You can disable these warnings, by changing the ``FORBIDDEN_CHECKSUM_WARNINGS`` setting to ``False``.

.. warning::
While the APT publisher will respect the ``ALLOWED_CONTENT_CHECKSUMS`` setting, the verbatim publisher will publish the upstream metadata files *verbatim* (the exact same metadata file that was synchronized).
As a result, the verbatim publisher is totally independent of (and does not respect) the ``ALLOWED_CONTENT_CHECKSUMS`` setting.
While the APT publisher respects the ``ALLOWED_CONTENT_CHECKSUMS`` setting, the verbatim publisher will mirror an exact copy of the synced upstream metadata.
By its very nature, the verbatim publisher does not respect the ``ALLOWED_CONTENT_CHECKSUMS`` setting.
If your policy prohibits the presence of certain checksum types, either do not use the verbatim publisher, or only synchronize repositories that are already compliant with your policy.


Opting in to MD5 and/or SHA1
Enabling MD5 and/or SHA1
--------------------------------------------------------------------------------

.. note::
Up to pulpcore version 3.11 all supported checksums were enabled by default.
In addition all ``pulp_deb`` versions compatible with pulpcore up to 3.11 simply break if the user actively disables any of the checksum types supported by the plugin.

To opt in to MD5 and SHA1 checksum handling users simply need to manually configure the ``ALLOWED_CONTENT_CHECKSUMS`` setting in the Pulp configuration file (normally found at ``/etc/pulp/settings.py``).
In order to opt in to MD5 and/or SHA1, the values ``md5`` and/or ``sha1`` must be added to the list configured for the ``ALLOWED_CONTENT_CHECKSUMS`` setting.
In addition a ``pulpcore-manager`` command must be run to generate any missing checksums within the DB for the settings change to take effect.

See the `pulpcore configuration documentation`_ for more information on configuration files and look for the section on ``ALLOWED_CONTENT_CHECKSUMS`` in the `pulpcore settings documentation`_ for more information on this setting.
See the `pulpcore configuration documentation`_ for more information on how to apply settings.
Look for the section on ``ALLOWED_CONTENT_CHECKSUMS`` in the `pulpcore settings documentation`_ in particular.

Once you have updated your configuration file you will likely need to halt your Pulp instance, and then run the following command to ensure your DB is made consistent with your ``ALLOWED_CONTENT_CHECKSUMS`` setting:
Once you have updated your configuration file you will need to halt your Pulp instance, and then run the following command to ensure your DB is made consistent with your ``ALLOWED_CONTENT_CHECKSUMS`` setting:

.. code-block:: none
Expand All @@ -59,13 +46,11 @@ Once you have updated your configuration file you will likely need to halt your
Security Implications
--------------------------------------------------------------------------------

APT repository metadata files (release files and package indecies) will typically include several checksums for all referenced files.
During synchronization, ``pulp_deb`` will check the downloaded files against the provided checksums.
Together with a valid (and trusted) release file signature this will guarantee the integrity of the synchronized repository.
During publication, the APT publisher will include any supported (and permitted) checksumtypes in any metadata files it publishes.

While the MD5 and SHA-1 algorithms are no longer considered secure, ``pulp_deb`` *requires* the presence of SHA-256 checksums to function, and is therefore *never* dependent on unsecure algorithms for integrity checking.
MD5 and SHA-1 are checked and used *in addition* to the other algorithms, purely for backwards compatibility.
MD5 and SHA-1 are checked and used *in addition* to the other algorithms.
This is consistent with widespread usage within the larger Debian ecosystem.


Disabling forbidden checksum warnings
--------------------------------------------------------------------------------

If your Pulp instance is configured to disallow the MD5 checksum algorithm, ``pulp_deb`` will emit a warning message with every sync as well as every use of the APT publisher.
You can disable these warnings, by setting ``FORBIDDEN_CHECKSUM_WARNINGS = False`` in your Pulp configuration file.

0 comments on commit 7086d62

Please sign in to comment.