Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Create helm-chart-collator Image for Treasuremap #162

Closed
lb4368 opened this issue May 26, 2021 · 7 comments
Closed

Create helm-chart-collator Image for Treasuremap #162

lb4368 opened this issue May 26, 2021 · 7 comments
Assignees
@seaneagan
Labels
2-Manifests Relates to manifest/document set related issues design needed Requires design approach/discussion enhancement New feature or request priority/medium Default priority for items size m 2-5 days [moderate complexity, generic code, or enhancement to existing feature]]
Milestone
v2.1

Comments

@lb4368
Copy link

lb4368 commented May 26, 2021 •
edited
Loading

Problem description
Treasuremap manifests currently use the default quay.io/airshipit/helm-chart-collator:latest image. This requires that charts needed by treasuremap (e.g. Dex) be managed in the charts.yaml file in the images repository where the helm-chart-collator is defined and built. It would be better if treasuremap managed its own charts.yaml and created a helm-chart-collator image specific for the needs of treasuremap. This would make it easier to place all charts needed by treasuremap into a single helm-chart-collator repository image to avoid having to access multiple helm repositories during deployment. This may also make it easier for downstream sites to build their own collators using the treasuremap charts.yaml as a base.

Proposed change

  1. Update treasuremap to create associated helm-chart-collator images compatible with each treasuremap release.
  2. The treasuremap helm-chart-collator should include helm charts used for all treasuremap manifest functions such that the collator is the only HelmRepository used by treasuremap.
  3. Update treasuremap functions to use the helm-chart-collator accordingly.

Potential impacts

  • Will need to care for the version management of the helm-chart-collator images created for treasuremap
@lb4368 lb4368 added enhancement New feature or request triage labels May 26, 2021
@seaneagan
Copy link
Contributor

Items 2. and 3. are definitely needed so we can deploy without reaching out to external helm repositories. For item 1. during the CI process (master branch) it may prove difficult to both update the helm-chart-collator input (charts.yaml) and use the output (image) in a single commit, since it will need to be stored somewhere (likely quay.io) before it can be referenced/used, unless we can create a local docker repository to store it in or something. So I am not sure how we can get around needing two commits:

  1. Update charts.yaml (CI pushes to quay.io on merge)
  2. Update treasuremap to use image from 1)

Thus, putting the charts.yaml in treasuremap directly will have the down side of needing to wait for the (expensive) treasuremap gates for both commits, and on the first commit there is really nothing to validate, unless we can do some fancy zuul job trigger file exclusion. It would be nice to keep the charts versions in treasuremap where they will be used, but maybe keeping it in airship/images for now would be simplest. In any case I think we should probably change the name to indicate that it is for use by treasuremap e.g. quay.io/airshipit/treasuremap-helm-charts.

@jezogwza jezogwza added design needed Requires design approach/discussion and removed triage labels Jun 2, 2021
@jezogwza jezogwza added 2-Manifests Relates to manifest/document set related issues priority/medium Default priority for items labels Jun 2, 2021
@jezogwza jezogwza modified the milestones: v2.1, Future Jun 2, 2021
@seaneagan
Copy link
Contributor

Also the helm-chart-collator and helm-chart-repository functions and catalogue entries (hcc image and helm repo url) are currently in airshipctl, and we are currently just referencing the latest image. We should be pinning to an image sha and this should be done in treasuremap, since this is where charts are actually consumed. If we leave the functions in airshipctl, we should just leave the default hcc image ref empty. We could also move the functions to treasuremap if preferred.

@seaneagan seaneagan self-assigned this Jun 8, 2021
@seaneagan seaneagan added the size m 2-5 days [moderate complexity, generic code, or enhancement to existing feature]] label Jun 9, 2021
airshipbot pushed a commit that referenced this issue Jun 9, 2021
@seaneagan
Pin hcc image
7a1e682 
To avoid breaking changes.

Relates-To: #162
Signed-off-by: Sean Eagan <[email protected]>
Change-Id: I055efa6f460809c364df6abf33f200f4c245697e
@lb4368 lb4368 modified the milestones: Future, v2.1 Jun 9, 2021
@seaneagan
Copy link
Contributor

https://review.opendev.org/c/airship/images/+/794838

airshipbot pushed a commit to airshipit/images that referenced this issue Jun 14, 2021
@seaneagan
Add charts consumed by treasuremap to hcc
e5aedc4 
This adds all charts currently used by treasuremap to hcc.
This will allow us to rely solely on hcc in treasuremap,
and not have to reach out to external helm repositories.

Relates to: airshipit/treasuremap#162

Signed-off-by: Sean Eagan <[email protected]>
Change-Id: I77a7dacdd146c37803a6e044a482819ecd914605
@seaneagan
Copy link
Contributor

I created airshipit/airshipctl#582 as well to make it easy to use other helm chart repositories during development time as this will still be useful.

@seaneagan
Copy link
Contributor

https://review.opendev.org/c/airship/treasuremap/+/796933

@seaneagan
Copy link
Contributor

https://review.opendev.org/c/airship/treasuremap/+/797138

airshipbot pushed a commit that referenced this issue Jul 9, 2021
@seaneagan @aostapenko
Deploy helm-chart-collator to sub-clusters
bf745d0 
- Deploy the helm-chart-collator service and HelmRepository
  to sub-clusters so that HelmReleases there can use it
- Integrate flux helm composite replacements [0].
- Move lma services to sub-directory of the workload to be
  able to set their namespace independent of the rest of the
  workload, currently helm-chart-collator.

[0]: https://review.opendev.org/c/airship/airshipctl/+/790791

Relates-To: #162
Signed-off-by: Sean Eagan <[email protected]>
Change-Id: Ib9f3add70a84ab75bfb0b1b6b7083048e469a24f
airshipbot pushed a commit that referenced this issue Jul 13, 2021
@seaneagan @aostapenko
Use helm-chart-collator for all charts
a0c1fc7 
- Update the helm-chart-collator image to one which:
  - uses versions unique to the git source [0]
  - includes all helm charts used by treasuremap [1]
- Rewire all the HelmReleases to point at the hcc HelmRepository
- Removes all the other HelmRepositories and associated config

[0]: https://review.opendev.org/c/airship/images/+/794657
[1]: https://review.opendev.org/c/airship/images/+/794838

Relates-To: #162
Relates-To: #164
Signed-off-by: Sean Eagan <[email protected]>
Change-Id: Ia96820b627d76feee7909471dd98a27de8594bf1
@lb4368
Copy link
Author

lb4368 commented Aug 25, 2021

Closed per patchset merge

@lb4368 lb4368 closed this as completed Aug 25, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@seaneagan seaneagan

Labels
2-Manifests Relates to manifest/document set related issues design needed Requires design approach/discussion enhancement New feature or request priority/medium Default priority for items size m 2-5 days [moderate complexity, generic code, or enhancement to existing feature]]
Projects
None yet
Milestone
v2.1
Development

No branches or pull requests
3 participants
@seaneagan @lb4368 @jezogwza