complete implementation of registries.json

[k-rister]

This is a meta issue that is going to be used to track the various pieces of work that are going to be needed to implement the registries.json spec. Due to the multi-project nature of Crucible this will involve several steps that will generally be implemented as unique PRs that will be submitted in a specific order so that a seamless transition can be made.

[k-rister]

This list of steps is a work in progress...

• 1. The registries.json needs to be created. Initially, this will be done by migrating values from /etc/sysconfig/crucible to $CRUCIBLE_HOME/config/registries.json. By starting with a migration of the values (rather than modifying the installer) we can ensure that the existing installations should be able to seamlessly transition when they do an upgrade (it was learned/discovered on previous work of a similar nature that starting with modifying the installation process prevents us from getting proper coverage of the upgrade process in CI). Once the registries.json is created the values it holds will be loaded into the existing environment variables to continue operating as before.
• 2. rickshaw-run needs to have a CLI handling to accept a parameter that points to the new registries.json.
• 3. Crucible's call of rickshaw-run will need to be modified to use the new CLI parameter created in the previous step.
• 4. Now that rickshaw-run knows where registries.json is it can be modified to load values from this file directly rather than relying on the environment variables that Crucible currently creates. Additionally, there are values (related to Quay API calls) that are currently stored in rickshaw-settings.json that need to be migrated to registries.json. Once they are migrated then they can also be loaded from the new location rather than the old one.
• 5. The environment variables that rickshaw-run used to rely on can be removed from Crucible.
• 6. Support for the environment variables previously used by rickshaw-run can be removed from rickshaw-run itself.
• 7. The Crucible installer can now be updated to natively create and populate registries.json with the values specified at installation time. Since the work will have been done to provide seamless transition for existing installs we can now remove the need (but not the functionality) for that transition to occur on new installations. The functionality for transitioning will be removed in the future at some point when it is deemed to no longer be needed
• 8. Once the Crucible installer is changed there will be need to be corresponding updates to the way that crucible-ci performs installations. Specifically with the expiration refresh handling.
• 9. add support for tokenized access on engine image pull rickshaw#530
• 10. add support for tokenized access on userenv image pull workshop#82
• 11. Add support for "userenv" repositories that rickshaw-run will search (in addition to searching it's own userenv library)
• 12. Update crucible itself to enable the "userenv" repositories and properly use rickshaw-run's new argument to tell it about them.
• 13. Add some config management interfaces to crucible to make configuring and using the new functionality easier.
• 14. Add support for tokenized authentication during engine pulls to k8s endpoint (add support for tokenized access on engine image pull rickshaw#530)

[k-rister]

Step 1 is being handled by #416

[k-rister]

Step 1 has been completed with #416 being merged.

[k-rister]

Step 2 is being handled by perftool-incubator/rickshaw#559

[k-rister]

Step 2 has been completed with perftool-incubator/rickshaw#559 being merged.

[k-rister]

Step 3 is being handled by #418

[k-rister]

Step 3 has been completed with #418 being merged.

[k-rister]

Step 4 is being handled by perftool-incubator/rickshaw#560

[k-rister]

Just a note, perftool-incubator/rickshaw#560 is only handling part of Step 4 (the quay migration part).

[k-rister]

perftool-incubator/rickshaw#560 has been merged.

[k-rister]

The 2nd part of Step 4 is being handled by perftool-incubator/rickshaw#562

[k-rister]

perftool-incubator/rickshaw#562 has been merged.

Step 4 is complete and so is step 6.

[k-rister]

Step 5 is being addressed by #419

[k-rister]

Step 5 is completed by the merge of #419

[k-rister]

Working on Step 7 in #421

[k-rister]

Encountered a blocking bug in rickshaw-run while working on Step 7. Fix is in perftool-incubator/rickshaw#565

[k-rister]

perftool-incubator/rickshaw#565 has been merged

[k-rister]

Waiting on a review of #421

[k-rister]

Step 7 is completed by the merge of #421

[k-rister]

Step 8 is being addressed by perftool-incubator/crucible-ci#150

[k-rister]

Step 8 is completed by the merge of perftool-incubator/crucible-ci#150

[k-rister]

Steps 9 & 10 are being addressed in tandem because they go hand in hand -- it does not make sense to support userenvs that require authenticated access and then push the resulting engine to a repository that does not require authenticated access.

The first step is to add support to workshop to handle authenticated userenv image pulls by locating tokens via the registries.json. This is being addressed by perftool-incubator/workshop#87

[k-rister]

Support in workshop has been "completed" with the merge of perftool-incubator/workshop#87

[k-rister]

The second part of steps 9 & 10 is being handled by this PR: perftool-incubator/rickshaw#568

[k-rister]

perftool-incubator/rickshaw#568 has been merged and it resolves steps 9, 10, & 11.

[k-rister]

#426 has been created to handle step 12

[k-rister]

#426 has been merged to complete step 12.

[k-rister]

Step 13 is being handled by #428

[k-rister]

#428 has been merged to complete step 13.