[stable+incubator/ASF charts] Migration of ASF projects to apache/charts git

[lwj5]

Is your feature request related to a problem? Please describe.
As part of the deprecation and migration https://github.com/helm/charts/issues/21103, there are a couple of charts that are related to the ASF project.

Describe the solution you'd like
I'm trying to start a discussion about moving these charts to the apache github repo. https://lists.apache.org/thread.html/rcb608739206d788785081073a0deb417ffa9981634975fc5525dc769%40%3Cdev.community.apache.org%3E

The charts from helm/charts in question are those that have yet to be adopted:

Chart	Owner/Maintainer	Possible migration to apache/helm-charts	Comments
stable/airflow	@gsemet @maver1ck @thesuperzapper	[ ]	Will be maintained externally
incubator/cassandra	@KongZ @maver1ck @maorfr	[x]
incubator/druid	@maver1ck @AWaterColorPen	[x]
stable/hadoop	@danisla	[x]
stable/ignite	@aasiutin @GreyTeardrop	[x]
incubator/kafka	@benjigoldberg	[x]
incubator/solr	@ian-thebridge-lucidworks	[]	PreferredAI/charts
stable/spark	@lachie83 @daixiang0	[x]
stable/zeppelin	@danisla	[x]
apache/airflow/tree/master/chart		[x]	Current ASF chart

Additional context
Let me know if I missed any

What do you guys think?

[gsemet]

Hi. For Airflow, we have started a discussion with ASF team so that they can take ownership or at least host the stable/airflow chart until their own one (based on a different chart that may be incompatible) is stable enough and feature-aligned.

We would like to see the current stable/airflow chart alive after November somewhere, at ASF it would be perfect. The other option would be to host it out of ASF, on a community github group dedicated only for maintaining this chart.

[potiuk]

Just to clarify the state of the Helm Chart from the Apache Airflow community as seems that we are going a bick back-forth discussing it in various discussions.

The current state of the discussion mentioned by @gsemet (apache/airflow#10523) is that the Airflow "helm/stable" chart (which was never maintained nor endorsed by the Airflow Community) is supposed be taken over by Bitnami. This proposal was originated by @scottrigby from the Helm team - and he helped to contact the maintainers of the chart (@gsemet and @thesuperzapper) with @carrodher from Bitnami - who agreed to take it over (especially that the Bitnami chart is pretty much the older version of the helm chart in the "helm/stable" and effort to synchronize it by the community will be limited).

In the Apache Airflow community, quite a long time ago we decided that we accepted donation of the Astronomer's Helm Chart and we are developing our own version of the helm chart. It is not yet "ready" to be released - but we are super happy to work out together with other ASF projects (already took part in the discussion) a common policy/standard for releasing charts and when it is ready - we will likely synchronize the development of the "community" chart and release it then following the agreed ASF policies.

So - just to clarify the issue description - the chart to be adopted as the official ASF chart is https://github.com/apache/airflow/tree/master/chart rather than not https://github.com/helm/charts/tree/master/stable/airflow

This might be worth correcting that in the issue description @lwj5 ^^

[gsemet]

Ah, if the Bitnami can be synced up (rebased?) with stable/airflow, that would be perfect ! I did not understood that they used to share a bit of history, my bad.

[scottrigby]

@gsemet in that case, would you mind amending your comment on bitnami/charts#3580 so that the potential for collaboration can remain open there? The Bitnami folks may not be able to easily track this here.

[scottrigby]

Also cross-posting this comment about incubator/solr: #22863 (comment). Thought if Apache would want to take this, that would be ideal.

[scottrigby]

@potiuk I've also sent a follow-up mail to the apache mailing list thread 😄 ✉️

[potiuk]

Yep. I saw it. Cool :).

I think the Yaml index + .tarball seems like good approach for hosting the releases. Very close to what SVN for https://downloads.apache.org/ is used for so far (ASF uses it to hold archived sources + binary packages if applicable). In this case it might even mean eventually that we do not need ASF "support" - we might simply need some tooling to generate the Yaml files when uploading the tarball archives.

You mentioned that you have some tools for that ? Can you please point me to some links to that? We will need something eventually for Airflow, but I might also want to help at the ASF level and would love to explore a bit more on how much work it might need to develop the right tooling.

Per your question there - I am afraid ASF won't be able to host non-Apache owned software. ASF has very strong policies about endorsing code/tools/organisations that are not ASF-community-managed and hosting anything @ "apache.org" websites https://www.apache.org/foundation/marks/pmcs#websites . For example we had to agree on wording and disclaimer to be able to put the "Ecosystem" page at "airflow.apache.org" with this disclaimer:

Those resources and services are not maintained, nor endorsed by the Apache Airflow Community and Apache Airflow project (maintained by the Committers and the Airflow PMC). Use them at your sole discretion. The community does not verify the licences nor validity of those tools, so it’s your responsibility to verify them.

The licence is not enough, I am afraid, I believe "Licensor" is what is important there, but I will let others chime-in, I've already said what I wanted to in the thread and (As the ASF is all about - we should give voice to everyone in the community :).

[scottrigby]

Nice 😄

Helm repo CI/CD tools:

• for CI the underlying tool is helm/chart-testing (to lint and test charts in a repo). To automate this on GitHub, we have created @helm/chart-testing-action GitHub Action
• for CD the underlying tool is helm/chart-releaser. With @helm/chart-releaser GitHub Action
• here's a working demo repo with details steps to set up the GH Actions: https://github.com/helm/charts-repo-actions-demo
• alternatively, if you prefer not to use the actions, but just the underlying tools, this is also entirely possible. A number of helm repos do this in various ways using different CI systems

Hosting:

I am afraid ASF won't be able to host non-Apache owned software

This makes sense. If I understand correctly, if Apache adopts the charts that install Apache apps, then those charts will be "owned" by Apache, but not the other charts so only these owned chart packages can be hosted. Is that correct?

[lwj5]

So - just to clarify the issue description - the chart to be adopted as the official ASF chart is https://github.com/apache/airflow/tree/master/chart rather than not https://github.com/helm/charts/tree/master/stable/airflow

This might be worth correcting that in the issue description @lwj5 ^^

Amended :)

Per your question there - I am afraid ASF won't be able to host non-Apache owned software. ASF has very strong policies about endorsing code/tools/organisations that are not ASF-community-managed and hosting anything @ "apache.org" websites https://www.apache.org/foundation/marks/pmcs#websites . For example we had to agree on wording and disclaimer to be able to put the "Ecosystem" page at "airflow.apache.org" with this disclaimer:

I think that the good thing about helm charts is that they can be hosted almost anywhere, like what you said, all that is required is an index yaml and the tarballs. This means that even if ASF determines that the charts are not owned by them, it is still possible to host it directly from the ASF GitHub charts repo. This can be achieved with helm/chart-releaser mentioned by @scottrigby or other methods like this https://medium.com/@mattiaperi/create-a-public-helm-chart-repository-with-github-pages-49b180dbb417 and with CI. There are some benefits to this method, including being able to leverage on GitHub's infra + uptime, and not needing to deal with mirroring delay.

I wouldn't worry too much about this as I think regardless of what ASF's take on this matter is, there is a fallback on hosting these charts in the repo. Now I just need to figure out how to create a space in the apache wiki, so that we can collate the info.

[scottrigby]

Just one more quick note on this - wherever these chart repos end up being hosted, would someone please make sure to list them in the CNCF https://artifacthub.io/ for end user discoverability? PRs can then be created in this repo to properly deprecate the newly hosted charts (per https://github.com/helm/charts/blob/master/PROCESSES.md#deprecating-a-chart).

I have set up an Org in Artifact Hub for the Apache project. Whichever Apache org members who want to help set up helm repos please register a user there, and let me know the usernames so I can add you as owners of that org and you can manage it from there. A metadata file can be added to your repo to be "verified", which will help trust. There are other trust mechanisms being put into place there as well if you'd like to keep in touch about it.

[potiuk]

This means that even if ASF determines that the charts are not owned by them, it is still possible to host it directly from the ASF GitHub charts repo.

That ain't so easy. Apache Branding Rules are rather strict about it, that anything from *.apache.org directing users to a 3rd-party must have a clear disclaimer so that you know the 3rd party you are referring to is not controlled by ASF.
This is quite clear for the websites, but it might not be entirely clear whether it's ok for anything under "downloads.apache.org" - anything there must be signed and checksummed, so I believe pointing to an external link that ASF project has no control over will be a non-go (but this is very important point in the discussion that I will definitely make).

Just one more quick note on this - wherever these chart repos end up being hosted, would someone please make sure to list them in the CNCF https://artifacthub.io/ for end user discoverability? PRs can then be created in this repo to properly deprecate the newly hosted charts (per https://github.com/helm/charts/blob/master/PROCESSES.md#deprecating-a-chart).

@scottrigby - I am not even an ASF member - I am just a PMC of one of the 300+ projects, so I have no more powers to make the things happen than to take part in the discussion (and contribute my time). I will try to make that point - but I think it's best if you keep on being involved in the discussion and make sure those points are addreessed - if you have to make sure some of those points happen. In the ASF the devlist is where the things happen - not Github Issues, I am afraid: https://lists.apache.org/thread.html/rcb608739206d788785081073a0deb417ffa9981634975fc5525dc769%40%3Cdev.community.apache.org%3E

This is really one of the points that must be clear in the policy that we come up with. You have to remember that ASF is a foundation with > 300 active projects. And those project are independently managed by their PMC (Project Management Committees). There is not a single person that can tell them what to do, and there is no-one in ASF that can take on the tasks on pinging those 300 projects and following up with them. The best ASF can do is to work-out and publish a Policy about it, make this part of the "release policy" that legally binds the PMC members (they are idemnified by ASF from any harm as long as they follow the release policy). And then hope it will be followed. No-one in ASF can "make sure" this happens, really.

I have set up an Org in Artifact Hub for the Apache project. Whichever Apache org members who want to help set up helm repos please register a user there, and let me know the usernames so I can add you as owners of that org and you can manage it from there. A metadata file can be added to your repo to be "verified", which will help trust. There are other trust mechanisms being put into place there as well if you'd like to keep in touch about it.

This cannot be done per "Apache" I am afraid - unless this is something Apache Infrastructure team picks up. They are the only ones that can do stuff for "all projects" - but they are very reluctant - understandably so - to take on new tasks. Ideally they want to make things happen so that each project is independent in managing whatever is needed for the project. So if there is a need to setup a trust or register something in https://artifacthub.io/, there must be a way for PMCS or committers of those projects to do it independently. The policy will have to describe how to do it - and PMC members have to have power to that on their own I am afraid (and those PMCs change, and new projects come and go). So I think we will have to come up with some self-management way - if this is necessary - something to be discussed in the thread, so I encourage you to chime-in about it in the devlist. I will chime in there, but I think it should be raised as a clear expectation on the devlist.

[scottrigby]

Adding stable/zeppelin to the list of charts that install Apache projects.

[potiuk]

Just for your information - I created a proposal for Apache Software Foundation to introduce changes to the "ASF release policies", to make it clear and straightforward to release "convenience packages" in the form of "software packaging" (such as Helm Charts and Container Images) rather than "compiled packages" as recognised so far by the ASF policies.

The proposal is here: https://cwiki.apache.org/confluence/display/COMDEV/Updates+of+policies+for+the+convenience+packages

The discussion in the ComDev ASF mailing list is here: https://lists.apache.org/thread.html/r49c3ef0a8423664c564c0c2719056662021f03b5678ef5b249892c10%40%3Cdev.community.apache.org%3E

We are going to discuss it and propose to the ASF board to vote on the changes.

I look forward to all comments an I hope it can pave the way for the ASF to provide a coherent approach for releasing Container Images, Helm Charts for all ASF projects.

[thesuperzapper]

Also, just as an update for the stable/airflow chart (not to be confused with the yet-unreleased apache/airflow chart).

It will be migrating to a simple GitHub pages hosted repository, to live out the remainder of its life until the apache/airflow chart is ready for wide adoption. We were looking into hosting it on bitnami/charts but don't believe the effort and risks of merging with bitnami/airflow are worth it for the existing users of stable/airflow, especially given it will be deprecated in favour of apache/airflow at some later date.

@gsemet and I will be maintaining stable/airflow in its new home until that time.

[gsemet]

I like the idea that all community supported charts for ASF software would be put at the same place, and will continue to live like they are today. Like @thesuperzapper said, we started the migration for Airflow alone.

[scottrigby]

@lwj5 Would you please also add incubator/druid to the list, just so we can track these (I could add it, but don't want to get in a situation where we're both editing the issue description at the same time).

However, after a conversation with @potiuk it does not seem likely that a single ASF charts repo will happen. In order for charts repos to be acceptable for ASF distribution, his policy proposal (see #23685 (comment)) would first have to be accepted, and then ASF projects will have a policy to be able to distribute helm charts. If past proposals are any indication, this process may take months – and is not something we can count on happening before the stable and incubator repos end support on 13 November. So we will want to think about where these live in the meantime.

About ASF chart source code and helm repo location – If I understood correctly @potiuk, a much more likely outcome would be each project hosting the relevant helm chart in their own repo (whether in the same git repo as their project source code, or in a separate namespaced git repo like PROJECT_NAME-helm-charts, as each project maintainers group is able to create git repos like this). Does this sound correct?

[gsemet]

What about creating a community github namespace to hold our various charts? Like "asf-community-charts/airflow", "asf-community-charts/druid", ... ?

[potiuk]

What about creating a community github namespace to hold our various charts? Like "asf-community-charts/airflow", "asf-community-charts/druid", ... ?

After talking to @scottrigby I think that would be kind-of defeating the purpose as the whole point of the change in removing the stable/helm chart is to get rid of the centrailzed model. Replacing it with the asf repo means that someone (who?) should be managing all that (access, owners, PRs, approving them, etc.). This was the whole point why Helm team decided not to manage it any more. This is also a prefferred ASF model as I understand it - everthing that is not absolutely necessary is managed by the projects, not on the ASF level. So there is little benefit of having the repo as most likely each project will end up with their own way of managin the charts and the most you can get out of it is a common index of those (but even this is not very likely IMHO). There are very little benefits of having it as there are plenty of indexes available.

And on top of it you might have branding issue with ASF (https://www.apache.org/foundation/marks/). The ASF Brand is the most valuable asset of it and the rules to protect them are very strict. You cannot legally publish "ASF named" software that is not ASF managed and publishes the software related to ASF. You can publish it as you like and point to particular projects, but using ASF braiding is strictly controlled. Even when you organize 3rd party events for your projects you cannot use Apache words for it (hence airflowsummit.org rather than apacheairflowsummit.org), It needs to be managed by community that is bound by the ASF rules (voting etc.) in order to use Apache or ASF when you are providing software. I am not a lawyer, but I believe you'd be breaking law if you do.

Of course it might be that there are plenty of violations out there of people who are not aware of, but if ASF finds out and the brand VP will decide it is important enough, they might ask those to bring it down.

BTW. You might be interested that with the upcoming 2.0 release we decided that the Helm Chart release will be done together with 2.0 (or maybe even 2.1 depending of the state of it). and we are not planning any 1.10.* release for it as we will be improving it along the way.

So feel free to continue it in a separate repository as you planned if you do not want to work with the Bitnami to sync it up.

[scottrigby]

@lwj5 would you please add stable/ignite to the issue body? 🙂

[scottrigby]

@potiuk I agree, after talking I now have a better understanding of ASF's project policies - thanks for that. Based on that, and on your ASF convenience packages policy revision draft in progress, I understand official ASF chart hosting will most likely be decided by each relevant Apache project team.

But I wouldn't say hosting logical groupings of charts together in a helm repo necessarily defeats the purpose of decentralizing the helm/charts repo. My opinion is grouping can sometimes be very useful, if the same people share common concerns and policies. For future ref, I listed some pros and cons in this comment about a single charts repo for all the grafana community charts grafana/loki#2593 (comment)

While I understand grouping the charts will not work for an official ASF charts repo, it is the fastest and easiest to maintain route for a temporary, unofficial charts repo. Because while an official location is preferable, it seems the stable charts that install Apache software will not all be able to be hosted even individually by the upstream ASF projects before the helm/charts repo is deprecated and packages garbage collected mid November. Therefore if the maintainers of these charts want to continue making them available, they'll need to put them somewhere. We have repeatable steps to do this efficiently (with git history, previous package version history, charts CI/CD, and other helpful processes), which I'd be happy to help set up or advise on. Then when/if each ASF project team decides to adopt the chart(s) related to their project, we can discuss transferring those charts individually, discuss how collaboration with those charts maintainers might still happen (for example, the chart maintainers would likely have to modify their processes to follow ASF project procedures and policies, etc). We can cross that bridge after your policy revision is approved, then at the pace and discretion of each related ASF project team.

For now - since we're under a deadline - we should arrive at a decision about where these charts will live in the interim (not only the stable/airflow chart but all the ASF charts people wish to continue maintaining), and identify any blockers. But can we clarify the legal note above? Wouldn't something like "asf-community-charts" (as @gsemet suggested in #23685 (comment)) or "apache-community-charts" be considered permitted nominative fair use per https://www.apache.org/foundation/marks/#products? If not, how is that different from the ASF project charts already in the stable and incubator repos? I'd like to clarify ASAP if they really need to be split into a dozen or so fully functional helm repos, or if there's nothing preventing a temporary home for them all together, as long as the messaging is correct.

[potiuk]

@scottrigby - I am not the one who can make decision on the naming/ownership :).. But I think if you have a proposal for an interim solution, you can simply write your proposal in dev@com list. I think the most important part is who and how manages such repo - I am not sure if there is an obvious candidate for that.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Any further update will cause the issue/pull request to no longer be considered stale. Thank you for your contributions.

[bmaehr]

Any new home for SOLR yet?

[lwj5]

@bmaehr nope, do you want to create one for it?

I've some commits to push as well

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Any further update will cause the issue/pull request to no longer be considered stale. Thank you for your contributions.

[lwj5]

Looks like Bloomberg has donated a solr-operator to ASF, so I've picked up this Solr. Here's the new repo, https://github.com/PreferredAI/charts. Will set up helm hub link + add update instructions tmr

[AWaterColorPen]

Is it possible to migrate some charts to personal repo to keep it alive for the present?

[lwj5]

@AWaterColorPen please do.

From what I see through @potiuk discussion, it looks extremely unlikely that there will be a single repo for these charts. It's also difficult for individual ASF projects to adopt the chart due to liability, and that they are also using docker community images.

Let me know the new repo, I'll update the link here if you do adopt them.

Update:
solr repo is up, you can upgrade mostly in-place. minor upgrade issue PreferredAI/helm-charts#5, can be manually done

[Snehil03]

Hello,
I did not saw incubator/fluentd-cloudwatch helm chart .
I hope it's right place to search for that chart.

Thanks !

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Any further update will cause the issue/pull request to no longer be considered stale. Thank you for your contributions.

[stale]

This issue is being automatically closed due to inactivity.