Changelogs

[pixelzoom]

At 5/13/21 status meeting, a potential need for a changelog was identified. This would enabled PhET to easily determine what was "new" in a version, and communicate that info to users and clients. Further discussion will happen in a subgroup, composed of KP, JB, CM, AR, SR, CM, ON.

[oliver-phet]

I just created an initial event for the next hour we all have availability: 5/24 11:30-12:30PM MST.

[pixelzoom]

Thoughts, questions, etc.

• changelog filename, location, and format
• Is the changelog stored in GitHub delivered directly to users/clients, or does it get processed/filtered/curated?
• What types of changes are noted in the changelog, what types are not?
• Standards for changelog entries (tone, grammar, punctuation, uppercase/lowercase, terminology,...)
• Should GitHub issue numbers appear in the file? It would be internally useful. But do you want to make it more likely for users/clients to visit GitHub issues? If the changelog is filtered before delivery, GitHub issue numbers could be stripped.
• Maintenace process: Automated changelog entries for maintenance releases? Entries for ALL production releases in master changelog? Update changelog in the release branch?
• How to communicate changelog to users/clients? Should the sim's webpage link to the changelog? Should it be linked in email annoucements?
• Are separate changelogs needs for brand=phet vs brand=phet-io?
• Is a PhET-iO changelog needed, or is that covered by the Migration Guide and Migration Engine?

[oliver-phet]

Could we leverage pull requests to capture issues/changes that are included in a "new" version of a sim?

[pixelzoom]

Another thought, an alternative to manually making changelog entries "as we go"... Write a script to gather the commit messages between production version deploys, then manually curate those commit messages into a changelog. (This script seems like it would be relatively easy for the sim repo, but more work for dependency repos.) Use this script and "manual curation" to update the changelog just before publishing each production version.

[kathy-phet]

JB, CM, ON, AR, AM, KP:
Add a (changelog.md) file in each repo that is a history of the changelog for that sim for phet brand
Add a (changelogphetio.md)? How does it interface with migration guide, api changes, studio changes, etc?
(optionally) Create an unpublishedChanges.md file if useful or Track the changes in the github issue that are "Update Changelog"
Add a link to the sim page: https://github.com/phetsims/molecule-shapes/blob/master/CHANGELOG.md
Add a link into the About dialog
Add a link into the Update dialog
Change log captured in release branches? especially for release branches? How does this intersect with maintenance release process? How will this be automated?

Change log needs to have:

• Version Number
• Release Date
• List of changes (edited before each public release)

Process of creating change log:

• With each commit, add a comment. If there is a changelog specific entry, add a marker for that e.g.
• Changelog: Modified range on Mass sliders from 1-1000 to 10-1000
• At publication time, a tool would go through and grab commit comments since last publication and create a text file that would then be hand edited, and added to the .md file.
• For common code changes, the tool would also pull commit comments for the common code to. List them by repo as it goes across the common code repos. Grouped are going to be easier to edit.

Changelog for PhET-iO vs PhET brand. focus right now on the PhET brand only.

[pixelzoom]

Next steps:

• discuss at developer meeting
• investigate script for gathering commits since the more recent production version
• evaluate the effort required to turn a list of commits into curated changelog entries

[jessegreenberg]

This was discussed during 5/27/21 developer meeting:

The plan and overview was presented to the developer team. No one had concerns. We will need to allocate some time for a developer (or subteam) to work on this, we will bring back to management to discuss.

@oliver-phet thought about pull requests and wondered if those could be used to naturally track changes made to sims and mostly common code. In addition to more change documentation, this might provide more stability. It is outside of our process, and we agreed we could learn more about this from people that use pull requests regularly.

[pixelzoom]

Additional notes from 5/27/21 developer meeting:

• @oliver-phet raised the topic of pull requests again, see Using pull requests in PhET development #1086.

• There was agreement that a good first step will be to create the script identified in Changelogs #1084 (comment), and evaluate the output. But no one felt like they currently have the time to volunteer to do that work.

• It's unclear what the priority or timeline for this issue is. The "epic" label was added to this issue.

Assigning to @kathy-phet for prioritization, scheduling, and assignment.

[samreid]

I was unable to attend the discussion at the 5/27/21 developer meeting. The idea of including a specific marker in certain commit messages then aggregating them with a script has the following disadvantages:

• It makes it difficult to add a forgotten changelog message. For example, if a developer neglected to add "Changelog: " to a commit message before pushing, it is difficult to go back and add it. It would be awkward to add another unrelated commit later that doesn't do anything, just to add the forgotten commit message.
• It makes it difficult to revise a bad changelog message. For example, if a developer inadvertently pushed "Changelog: changed gravty to 9.8" in a commit, it is difficult to make a correction.
• It is sometimes difficult to know when the "final" commit for an issue is done. For example, sometimes we add a commit, thinking we are done, but then we have to add a few more essential commits to actually finish it up. A false positive "Changelog: " entry could be picked up by a build between those points.
• Likewise, if there are 10 commits that address an issue, if the final commit is the one we decide to be in the changelog (since it has the right SHA), that prevents us from putting a technical-oriented commit message in that commit, since it has to be the user-facing commit message.

Git commit messages are not designed to be rewritten, and I don't think we should work toward a solution that makes it difficult to edit change logs or requires a nonstandard approach of editing change logs.

The solution to these problems is to maintain a changes.txt or changes.md file in each repository, which is manually added to at the time of a user-facing change (often as part of a commit that implements the described solution). If a user forgets to update the changelog.txt file, they can add an entry later. Corrections or improvements can be made at any time.

This is straightforward for simulation repositories, since they can have version delimiters, which can be automatically inserted by the build process:

# 2021-05-30 - Published 1.1
Increased the default gravity from 9.0 to 9.8
Added a checkbox to disable friction

# 2021-02-28 - Published 1.0
Initial Publication

For common code repos, we may need to annotate each change with a date, so that we find all common code changes that impact a project. For example:

# 2021-05-28
Increased the slider track width
Added sound effects to ComboBox

# 2021-05-24
Made checkboxes smaller by default

# 2021-05-21
Radio buttons are now blue

This way, when we publish the next version of a sim, we can aggregate the relevant common code changes from dependencies and output them together (perhaps integrated into the sim's changes.txt or perhaps in a new combined report build artifact).

In my opinion, this approach or something like it will serve our project well. Some known problems or questions regarding this proposal:

• It would be nice to have the same syntax for sim and common repo change entries, but common code repos require timestamps or SHA-stamps and sim repos do not.
• We will need fine-grained timestamps to common code, so that sims built on that day know whether or not they include the change. A fine-grained timestamp could do it, but we will have to manage different time zones which will make it difficult for people outside the standard time-zone to read and write. Perhaps better to paste a SHA, but then we cannot append to the repo's changes.txt at the same time as the commit since we don't know the SHA yet.
• Do we want an opportunity to exclude common code changes that don't apply? For instance, if we are publishing a sim with no sliders, but a sun change to slider occurred in the date range, should we have a way to exclude that from the aggregated change log?

Regarding pull requests: managing changelogs through pull requests is not a good fit for our project for a few reasons: pull requests must be done in a branch which creates more overhead. Next, it seems odd for Developer X to create a pull request, then approve their own request. It adds a lot of overhead to something that should be simple, like adding a line of text to a text file.

[zepumph]

This came up again in phetsims/density#202. We are worried about not having a solution for this before we start to scale. The current business revolves around the Standard PhET-iO Wrapper, but it seems like we should have something for custom wrappers.

We will continue to want to push people to new versions, and release notes may help communicate the new value and features a new version has in addition to providing an upgrade guide.