Package CHANGELOGs

[ycombinator]

Forked off from discussion about package CHANGELOGs in #14.

We need to decide how best to capture CHANGELOGs for packages. Some ideas that have been proposed so far:

• Writing a file at the root of the package's folder,
• Generating them from commit messages,
• Generating them from PR descriptions

[UPDATE] After discussion below and consensus on where and how package CHANGELOGs should be stored, I'm adding a few checklist items below for next steps to implement the agreed-upon changes:

• Create PR to package spec to make CHANGELOG files required: Introducing package CHANGELOGs #131
• Once package spec PR is approved but not yet merged:
  • Update integrations repo with skeleton CHANGELOG files for each integration package: Adding package changelogs integrations#675
  • Inform non-integrations repo package owners about adding CHANGELOG files: email sent
• Once existing packages have been updated with skeleton CHANGELOG files:
  • Merge package spec PR: Introducing package CHANGELOGs #131
  • Create/review/merge PR to update elastic-package with package spec changes: Updating package spec dependency elastic-package#252
  • Create/review/merge PR to update integrations repo with elastic-package changes: Adding package changelogs integrations#675

[ycombinator]

Previous discussion on package CHANGELOGs, in chronological order, copied over from #14 for ready reference:

---

@mtojek said:

I'm allergic to changelogs, probably @ruflin would like to elaborate on them :) I suppose we will have to start using them, but I'm not sure if people get mad, when you ask them to fill them up.

---

@ruflin said:

@mtojek For changelog: We need a way to tell users of the packages on what change between versions. I'm not tied at all to changelogs but we need a solution.

---

@ycombinator said:

My 2 cents: I like the proposed CHANGELOG yml format. It captures what we need, but in a structured fashion. So one can imagine generating human-readable changelog documentation from it, etc.

I'm allergic to changelogs,

@mtojek could you elaborate on why this is? maybe there's a drawback that I'm not seeing currently.

---

@urso said:

And I'm getting tired of yml :)

What issue does using YAML solve here?

---

@ycombinator said:

I'm not married to the YML but I like that it provides some structure while also being pretty much human-readable. I suppose we would get enough structure from using asciidoc too, as we did in Beats, or some other format. So I'm good with either. But I do think it's useful to have package CHANGELOGs so we can communicate to users what changed in a package version.

---

@mtojek said:

@mtojek could you elaborate on why this is? maybe there's a drawback that I'm not seeing currently.

The same one as for Beats and Package-Registry - it's really easy to forget about it and make it inconsistent. I'm thinking about generating it out of commit messages. At least we'll have an order in Git log :)

---

@urso said:

it's really easy to forget about it and make it inconsistent. I'm thinking about generating it out of commit messages. At least we'll have an order in Git log :)

For Beats we want to move to adding Changelog information to the PR description. Changelog are a pain and often need editing.

I'm not married to the YML but I like that it provides some structure while also being pretty much human-readable

I wonder how much value the format provides. Using YAML means we have to convert/process the file eventually. What is the structure actually required for (maybe I'm missing something)?

The advantage of YAML is that it is easier to process for automation purposes.

With structure you still have to be careful. One reason we added CHANGELOG.next.asciidoc to Beats is because developers are often wrong about the actual version a change is include. If releases get postponed the changelog can actually be wrong again (yep, maintaining Beats changelog was a not so fun task). I guess it is not so much a problem for integrations, but putting the release version into your YAML structure per entry is potentially asking for some heavy editing later on (also depends on FF and release procedures and policies).

All in all it would be nice if we eventually settle for a common way on how to handle changelogs for related projects, in order to make live easier for developers that have to touch all these related projects.

[ycombinator]

I wonder how much value the format provides. Using YAML means we have to convert/process the file eventually. What is the structure actually required for (maybe I'm missing something)?

The advantage of YAML is that it is easier to process for automation purposes.

Yeah, fair points. I was thinking having some light structure from the start is good because adding structure to unstructured data is always harder later on. That said, I do not have a concrete use case in mind for structured CHANGELOGs today either! So I'm okay not using YAML for CHANGELOGs.

I'm thinking about generating it out of commit messages. At least we'll have an order in Git log :)

For Beats we want to move to adding Changelog information to the PR description. Changelog are a pain and often need editing.

Given the tooling that's being developed around packages, I think generating CHANGELOGs from PRs as part of the step that publishes a new version of a package to the package registry/storage might be the way to go. I think we will still need to store the generated CHANGELOGs somewhere though? Maybe it's worth discussing a bit about where users might be expected to view package CHANGELOGs?

[ruflin]

There is a fundamental difference here form the changelog for package a package and the overall Beats changelog. In the future, only a subset of the packages will come from the integrations repository. This means we also need to provide a manual way for the changelog.

One very annoying thing around changelogs in Beats was that we had constant conflicts. This is because ALL changes went into a single changelog. But each package has its own and I expect 1, max 2 engineers work on the same package and unlikely at the same time. Because of this, conflicts will be hopefully rare.

Last, I went for the structure format to allow the part that visualises the changelog to decide on how it visualises it, perhaps only picks the last entries or the entries since the last version installed.

[ruflin]

One more thing that just came to my mind: It is possible that in the integrations repo the changelog is created based on PR's, not objections there. But what we discuss here is the final format of the changelog when it is shipped as part of the package.

[ycombinator]

Recently there has again been desire for packages to have package CHANGELOGs. The most recent motivation comes from folks QA'ing packages so they can easily track changes across package versions they are testing. I'd like to resurrect this discussion now with a hope that we can arrive at some consensus at least for an initial implementation of package CHANGELOGs, just to get things moving.

I'll start by summarizing the discussion in the comments above so far.

• We want packages to have CHANGELOGs 🙂 .
• We want them to be contained in the package itself, i.e. they could be written to directly by a human or by automation (e.g. by parsing PRs from packages that live in the integrations repo).
• (This one not something we have consensus on so far) The CHANGELOG that's contained in the package should be structured. One use case this enables is for the Fleet UI to render the CHANGELOG on the Integration Detail page.
• (This one is definitely just my opinion, definitely no consensus here 🙂). The CHANGELOG format should be YAML. It provides the needed structure (see previous bullet) while also keeping things relatively human-readable.

---

I think we need to separate the ideas of:

1. where and how package CHANGELOGs are stored, from
2. how CHANGELOG entries might be generated.

I think this separation will allow us to make progress on this issue.

For 1. — in the interest of moving this issue forward — I propose that packages contain a required CHANGELOG.yml file at the package root. The existence and structure of this file can be enforced by the package spec. If the YML format becomes burdensome, we can revisit it in the future and change it to something we prefer more. Or if someone has an alternative structured format to suggest now, that's fine too, let's discuss it. But I do think having a structured format is important for the Fleet UI use case mentioned earlier.

Keep in mind that we can always decide to go from structured to unstructured (e.g. YAML to MD) later or change the format of the structure (e.g. YML to JSON) later but going from unstructured to structured later will probably be cumbersome and labor-intensive.

For 2., as a follow up step, we can build tooling to generate an entry in the CHANGELOG.yml file based on commits or PR descriptions. But this step should not be seen as a requirement or blocker, more of an added convenience. Until there is tooling or even after there is tooling, humans should be able to manually create entries in CHANGELOG.yml files. This ensures that packages outside of the integrations repo or ones that don't/can't take advantage of any tooling can still have CHANGELOGs that are compatible with CHANGELOGs generated by tooling. So could we defer this discussion until we have consensus on 1. first?

@urso @mtojek @ruflin WDYT?

[ph]

That sound like a really good plan, @ycombinator

[ycombinator]

Thanks @ph @urso @mtojek and @kaiyan-sheng for consensus on the format and location of package CHANGELOGs (point 1 from my last comment). I've updated this issue's description to safely rollout the package changelog.yml changes through various repos.

We can continue the discussion on point 2 now:

how CHANGELOG entries might be generated.

@mtojek I think you had some ideas about how to go about this. Do you mind reiterating / elaborating on them a bit now so we can discuss them and arrive at some consensus?

[ycombinator]

All items in checklist (in issue description) have now been completed. Closing issue.