Split prometheus-boshrelease to ease maintenance

[psycofdj]

Hello folks,

I wish to open a discussion about the maintenance of this repository, the problems we are facing and possible solutions. Please tell me how you feel about this.

cc @ArthurHlt @benjaminguttmann-avtq @bkez322

context

In the current state, this bosh-release holds a tremendous number functionalities that applies to a vast diversity of components. Some numbers:

• number of jobs: 203
• number of packages: 35
• number of blobs: 46

In my opinion, this amount of products brings a number of problems:

• as a maintainer, I have no experience the majority of these jobs and I can't neither provide support, nor give an enlightened opinion on PRs targeting them. Most of the time, I don't even have on hand the technical stack that would allow me to test anything.
• as a user, I can't easily tell which are the tools that have an active maintenance with a wide community of users and which are here with no activity since 5 years
• it is also hard to tell apart the lack of support of an exporter and the lack of bump of the exporter into the bosh release

Beyond the number of products, I also have the feeling that the repository lacks automation. Automation that could ease the creation of blobs bumping PRs, generating release candidates, help to identify stalled issues... etc.

The current situation reminds me the cloudfoundry cf-release "god release" of 2017.

proposal

In the same way pivotal splitted cf-release into many releases and cf-deployment, a solution would be to split this giant bosh-release in many smaller ones and create a prometheus-deployment repository that will keep track of:

• the main deployment manifest
• the versions of "child" bosh-releases
• the ops-file to activate and configure each functionality
• the release/changelog aggregation

With products splitted into small releases and repositories, it would certainly be easier to find maintainers for each one. Or, at least, identify the feature as unmaintained.

Each repository would have more latitude to implement automation and issue policy.

In the case of a regression is identified in a given prometheus-deployment release, it would be possible for users to do a partial rollback by pointing to a previous version of a specific sub-release.

Implementing automatic PR creation to bump sub-components would also be simpler.

The whole thing would be fairly transparent to users since it would only modify the bosh-release names of each job in the main manifest.

details

This idea is not to create as many release than we have packages and job. The idea is to define what functions should be held by each release.

For instance: nginx, alertmanager, prometheus, grafana and associated dashboards and alerts should probable remain in a prometheus-core release.

In a similar way cf_exporter, firehose_exporter with cloudfoundry dashboard and alerts in another one like prometheus-cf.

etc, etc

contribution

If this poposal gets the attention of the community, my team and I are volunteers to do all the plumbing for this move:

• create all the repositories
• imports the blobs in their current version
• create each release with associated blobs, packages and job
• cut a first version of prometheus-deployment with all the manifests and ops-files

[benjaminguttmann-avtq]

Hi @psycofdj,

thanks for opening that discussion. Let me share my point of view here.

I think the lack of automation is a real issue in the release. Auto prepare blob bumps for example would be a real benefit.

Beside splitting up the whole release, I think in case of the CF one, the situation was different, the community had power over all the repos and had the work power to add dedicated maintainers for each of the repos.
In our case (Prometheus) we only have the power to maintain a couple of the exporters.
If we now split it up, we would need to create bosh releases for all the existing exporters, and maintain those which would not solve the issue as the actual exporter might still not get maintained anymore.

I also think it might not be necessary that every person can answer every question that is placed here, but with a team of maintainers I think there will be at least one maintainer that can give a proper answer or has the possibility to test things.