Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CRAYSAT-1896: Add support for CFS v2 or v3 to sat bootprep #288

Merged
merged 10 commits into from
Dec 2, 2024

Commits on Nov 25, 2024

  1. CRAYSAT-1896: Add support for CFS v2 or v3 to sat bootprep

    Add support for using either the CFS v2 or v3 API in `sat bootprep`
    based on the value of the command-line option `--cfs-version` or the
    config-file option `cfs.api_version`. Before this change, `sat bootprep`
    was hard-coded to always use CFS v2.
    
    Update the `InputConfiguration`, `InputConfigurationLayer`, and
    `AdditionalInventory` classes to keep a reference to the instance of the
    version-specific subclass of `CFSClientBase`, so that they can delegate
    operations to the appropriate version of the CFS API.
    
    This removes a lot of code that required knowledge of the CFS API and
    instead uses classes and methods defined in the `csm_api_client`
    library, which has already been updated to support both CFS v2 and v3.
    This should reduce code duplication across `sat` and `csm_api_client`.
    
    Some unit tests are no longer needed here because that functionality is
    tested in the `csm_api_client` instead. Other unit tests are updated to
    include the `cfs_client` instance that needs to be passed into the input
    layer and configuration objects.
    
    This also makes the `playbook` property required in the schema because
    the CFS v3 API requires that the playbook be specified. This is
    technically a backwards-incompatible schema change because it can
    invalidate certain bootprep input files if they define layers without
    playbooks. The default bootprep files contain a playbook for every
    layer and do not depend on the CFS default playbook value, so this
    should be a pretty safe and non-disruptive change.
    
    Test Description:
    Unit tests only so far.
    
    Still needs to be tested on a real system.
    haasken-hpe committed Nov 25, 2024
    Configuration menu
    Copy the full SHA
    8c87d83 View commit details
    Browse the repository at this point in the history
  2. CRAYSAT-1896: Add support for paging of components

    The CFS v3 API returns paged results for the `/components` API resource,
    which needs to be handled by the client. Upgrade the version of the
    csm_api_client library and use the new `get_configurations` method to
    handle the paging.
    
    Test Description:
    Not yet tested.
    haasken-hpe committed Nov 25, 2024
    Configuration menu
    Copy the full SHA
    4536425 View commit details
    Browse the repository at this point in the history
  3. CRAYSAT-1896: Fix handling of "latest" product-based layers

    If the value "latest" is specified for the version of a product in a
    product-based layer in a CFS configuration in a bootprep input file,
    pass in `None` to `from_product_catalog` to properly get the product
    catalog data associated with the latest version of the product.
    
    Test Description:
    Not tested yet.
    haasken-hpe committed Nov 25, 2024
    Configuration menu
    Copy the full SHA
    f78664e View commit details
    Browse the repository at this point in the history

Commits on Nov 26, 2024

  1. CRAYSAT-1896: Add --cfs-version option to sat bootprep run

    Add the `--cfs-version` command-line option to `sat bootprep run`. This
    option is used to select the version of the CFS API to use when creating
    CFS configurations or checking for the existence of CFS configurations
    when creating other items in the bootprep input file.
    
    Add this option to the man page as well.
    
    Test Description:
    Built man page and examined in `man`.
    
    Other testing yet to be performed on a system.
    haasken-hpe committed Nov 26, 2024
    Configuration menu
    Copy the full SHA
    a0a0dc8 View commit details
    Browse the repository at this point in the history
  2. CRAYSAT-1896: Add debug logs around loading ProductCatalog

    A delay has been observed in `sat bootprep`, and it may be due to the
    loading of product catalog data taking longer with the split product
    catalog config maps. Add a debug-level log message before and after
    loading the product catalog, so we can see what's happening.
    haasken-hpe committed Nov 26, 2024
    Configuration menu
    Copy the full SHA
    34d62f7 View commit details
    Browse the repository at this point in the history
  3. CRAYSAT-1896: Validate playbook in validation method instead of schema

    Instead of requiring that the playbook be present in the bootprep input
    file through the schema, validate it specifically when `sat` is
    configured to use the CFS v3 API, since the CFS v2 API does not require
    that CFS configuration layers have a playbook. This will allow people to
    continue using old bootprep input files that have CFS configuration
    layers that do not specify a playbook as long as they use CFS v2. This
    is slightly less disruptive than making a schema change.
    
    Test Description:
    Unit tests only so far.
    haasken-hpe committed Nov 26, 2024
    Configuration menu
    Copy the full SHA
    e5bc550 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    cdc4c8c View commit details
    Browse the repository at this point in the history
  5. CRAYSAT-1896: Fix validation of the InputConfiguration

    The `InputConfigurationLayer` instances stored in the `layers` attribute
    of the `InputConfiguration` class were not being validated when the
    `validate` method of the `InputConfiguration` was called because this
    class did not have any validation methods. Add a validation method that
    validates the layers, and make the `InputConfigurationLayer` class
    inherit from `Validatable`, so that it has a `validate` method itself.
    
    This has not been added for `AdditionalInventoryLayer` yet because there
    are currently no validation methods on that class.
    
    Note that an `index` is added to the constructor of the
    `InputConfigurationLayer` class to allow for the `__str__` to
    unambiguously refer to the layer within the CFS configuration.
    
    Test Description:
    Unit tests have been added to test the `validate` method on the
    `InputConfiguration` class.
    haasken-hpe committed Nov 26, 2024
    Configuration menu
    Copy the full SHA
    1d056dc View commit details
    Browse the repository at this point in the history
  6. CRAYSAT-1896: Add Jinja2 template support to more bootprep fields

    Add support for Jinja2 template rendering to several more properties in
    the `sat bootprep` input file. The following properties now support
    Jinja2 templating:
    
    - configurations[].layers[].git.commit
    - configurations[].layers[].git.url
    - configurations[].layers[].product.commit
    - All properties under images[].base.ims
    - session_templates[].image.ims.id
    
    Allowing Jinja2 template rendering for these fields is helpful in
    writing automated test cases for `sat bootprep` that can run on any
    system. These values will vary from system to system, so it's helpful to
    pass this information in to the bootprep input file as a variable.
    
    Test Description:
    Added unit tests pass. Will also be tested on a system with bootprep
    input files that use Jinja templates in these properties.
    haasken-hpe committed Nov 26, 2024
    Configuration menu
    Copy the full SHA
    8a1f31b View commit details
    Browse the repository at this point in the history
  7. CRAYSAT-1896: Add Jinja template support to additional_inventory

    Add support for Jinja template rendering to the `name`, `url`, `branch`,
    and `commit` properties of the `additional_inventory` of a CFS
    configuration specified in a bootprep input file.
    
    This is useful for writing automated tests that obtain this information
    from a system and pass it in through variables with `--vars-file`.
    
    Test Description:
    The added unit tests pass, and it will be tested on rocket as well.
    haasken-hpe committed Nov 26, 2024
    Configuration menu
    Copy the full SHA
    c751745 View commit details
    Browse the repository at this point in the history