Plan to release etcd v3.4.34

[ivanvc]

What would you like to be added?

The etcd patch release criteria has been met for our release-3.4 stable release branch so we should release v3.4.34.

The list of commits included since the previous release is: v3.4.33...release-3.4:

• [3.4] Install shellcheck if it is not present #18248
• [3.4]: Bump Go version to 1.21.12: GO-2024-2963 fix. #18272
• [3.4] installed revive on CI and added doc.go to fix revive linter errors #18215
• [3.4] client/v3/watch.go: use fmt go pkg for gRPC metadata map printing #18311
• [3.4] Remove gsutil acl command for bucket permissions #18345
• [3.4] build(deps): bump distroless/static-debian11 from 6d31326 to 1dbe426 #18389
• [3.4] Upgrade gofail to v0.2.0 #18396
• [3.4] go version bump from 1.21.12 to 1.21.13 #18422
• [3.4] Skip leadership check if the etcd instance is active processing heartbeats #18440
• [3.4] Complete backport of OWNERS file #18448
• [3.4] test: extend leaseRevoke and leaseRenew test to support 5 members cluster #18452
• [3.4] Bump go toolchain to 1.22.6 #18445
• [3.4] mvcc/*_test.go: should not use duplicate revision.Main for one key #18456
• [3.4] tests/e2e: add e2e test to reproduce issue 18089 #18455
• [3.4] *: keep tombstone if revision == compactAtRev  #18475
• [3.4] Bump bbolt to 1.3.11 #18488
• [3.4] update Go to 1.22.7 #18549

Work in progress CHANGELOG is: https://github.com/etcd-io/etcd/blob/main/CHANGELOG/CHANGELOG-3.4.md#v3434-tbd

List of pull requests we still need to backport from main to release-3.4 before the patch release is issued:

• Update Go main branch to 1.23.1 release branches to 1.22.7 #18548

Release team

GitHub handle	Role
@ivanvc	Release lead
@ahrtr	Release advisor
@henrybear327	Release shadow

Why is this needed?

Regular patch releases are vital to ensure our users have bug-free and secure software.

[serathius]

@ivanvc any blockers for release that I can help with?

[ahrtr]

@ivanvc please feel free to drive/lead the release process, thx cc @jmhbnz

[henrybear327]

@ivanvc can I shadow this release :) Thank you!

[ivanvc]

@serathius, @ahrtr, no blockers. I was a bit busy the last weeks so that I couldn't follow up on the releases.

That being said, I'm happy to lead this release. Since we did the last release, we need to get a list of pull requests/issues that require a backport. @henrybear327, can you help me do a pass over them? I'll also do one. I'm checking at issues from the next list: https://github.com/etcd-io/etcd/issues?page=1&q=is%3Aissue+label%3Atype%2Fbug+created%3A%3E%3D2024-06-14, and pull requests from: https://github.com/etcd-io/etcd/issues?q=is%3Apullrequest+label%3Atype%2Fbug+created%3A%3E%3D2024-06-14

@ahrtr, @jmhbnz, @serathius, @spzala, @wenjiaswe, with me as the release lead, would any of you want to be emeritus lead, advisor, or shadow? Given that it will be my first release, I'd appreciate guidance from someone.

[ivanvc]

I see that we have this issue with an open PR marked as backport to v3.4 and v3.5: #18267. Do we want to merge its PR (#18268)? It looks like it never got a review.

[ahrtr]

Do we want to merge its PR (#18268)?

NO, the PR changes the most important part of etcdserver, but without proper test cases. Since it's low priority/impact, we can revisit it in future when we have bandwidth.

[ahrtr]

@ahrtr, @jmhbnz, @serathius, @spzala, @wenjiaswe, with me as the release lead, would any of you want to be emeritus lead, advisor, or shadow? Given that it will be my first release, I'd appreciate guidance from someone.

I am happy to join the release meeting for 3.4.34 this time. Thanks

[henrybear327]

@serathius, @ahrtr, no blockers. I was a bit busy the last weeks so that I couldn't follow up on the releases.

That being said, I'm happy to lead this release. Since we did the last release, we need to get a list of pull requests/issues that require a backport. @henrybear327, can you help me do a pass over them? I'll also do one. I'm checking at issues from the next list: https://github.com/etcd-io/etcd/issues?page=1&q=is%3Aissue+label%3Atype%2Fbug+created%3A%3E%3D2024-06-14, and pull requests from: https://github.com/etcd-io/etcd/issues?q=is%3Apullrequest+label%3Atype%2Fbug+created%3A%3E%3D2024-06-14

@ahrtr, @jmhbnz, @serathius, @spzala, @wenjiaswe, with me as the release lead, would any of you want to be emeritus lead, advisor, or shadow? Given that it will be my first release, I'd appreciate guidance from someone.

Sorry for the late reply @ivanvc. I don't see any outstanding / missed issues or PRs so far! Thanks

[ivanvc]

We'll be doing the release after updating to the Go 1.22.7 release #18485 (comment).

[ivanvc]

Hi team (@ahrtr, @ahrtr), With v3.5.16 being released tomorrow, I wanted to check your availability to do the v3.4.34 release. Would this Wednesday at 11 a.m. PT/7 p.m. BST/8 p.m. CEST or Thursday at 10 a.m. PT/6 p.m. BST/7 p.m. CEST (before the etcd community/triage meeting) work for you?

[ahrtr]

Wednesday at 11 a.m. PT/7 p.m. BST

Works for me.

[ivanvc]

We just finished the release. We noted a couple of improvements to the documentation:

1. Update the release note for 3.4 specific instructions before step 6

   1. Manually create the release tag

      gpg --list-keys --with-colons "<[email protected]>" | awk -F: '/^pub:/ { print $5 }'
      git tag --local-user "<GPG key from previous step>" --sign "v${VERSION}" --message "v${VERSION}"
      

   2. Manually push the tag to GitHub git push origin ${VERSION}

2. In step 7, add a bullet to check the historic binary size for each arch. If there's a big difference, check if the binaries work

3. Before step 9, add a note on removing quay.io login from ~/.docker/config.json

[ahrtr]

We just finished the release. We noted a couple of improvements to the documentation:

1. Update the release note for 3.4 specific instructions before step 6

   1. Manually create the release tag

      gpg --list-keys --with-colons "<[email protected]>" | awk -F: '/^pub:/ { print $5 }'
      git tag --local-user "<GPG key from previous step>" --sign "v${VERSION}" --message "v${VERSION}"
      

   2. Manually push the tag to GitHub git push origin ${VERSION}

2. In step 7, add a bullet to check the historic binary size for each arch. If there's a big difference, check if the binaries work

3. Before step 9, add a note on removing quay.io login from ~/.docker/config.json

Also as James pointed out in the meeting, please feel free to review the release script and try to avoid the manual steps if possible.

The only reason why I manually executed the steps when I was releasing 3.4.x is that I was trying to reduce the risk as much as possible, because there was no one I could ask help for when I was releasing 3.4.x.

[henrybear327]

We just finished the release. We noted a couple of improvements to the documentation:

1. Update the release note for 3.4 specific instructions before step 6

   1. Manually create the release tag

      gpg --list-keys --with-colons "<[email protected]>" | awk -F: '/^pub:/ { print $5 }'
      git tag --local-user "<GPG key from previous step>" --sign "v${VERSION}" --message "v${VERSION}"
      

   2. Manually push the tag to GitHub git push origin ${VERSION}

2. In step 7, add a bullet to check the historic binary size for each arch. If there's a big difference, check if the binaries work

3. Before step 9, add a note on removing quay.io login from ~/.docker/config.json

Also check for newlines when composing the announcement message!

[ivanvc]

Hi team, I was looking into implementing the tagging into 3.4's release script, but I found that the version bumping and tagging are already implemented. Refer to the following snippets, which are from the release-3.4 branch:

etcd/scripts/release.sh

Lines 93 to 107 in a1375b7

	# If the release tag does not already exist remotely, create it.
	log_callout "Create tag if not present"
	if [ "${remote_tag_exists}" -eq 0 ]; then
	# Bump version/version.go to release version.
	local source_version
	source_version=$(grep -E "\s+Version\s*=" version/version.go | sed -e "s/.*\"\(.*\)\".*/\1/g")
	if [[ "${source_version}" != "${VERSION}" ]]; then
	source_minor_version=$(echo "${source_version}" | cut -d. -f 1-2)
	if [[ "${source_minor_version}" != "${MINOR_VERSION}" ]]; then
	log_error "Wrong etcd minor version in version/version.go. Expected ${MINOR_VERSION} but got ${source_minor_version}. Aborting."
	exit 1
	fi
	echo "Updating version from ${source_version} to ${VERSION} in version/version.go"
	sed -i "s/${source_version}/${VERSION}/g" version/version.go
	fi

etcd/scripts/release.sh

Lines 133 to 145 in a1375b7

	# Tag release.
	if [ "$(git tag --list | grep -c "${RELEASE_VERSION}")" -gt 0 ]; then
	log_callout "Skipping tag step. git tag ${RELEASE_VERSION} already exists."
	else
	log_callout "Tagging release..."
	gitemail=$(git config --get user.email)
	KEYID=$(gpg --list-keys --with-colons "${gitemail}" | awk -F: '/^pub:/ { print $5 }')
	if [[ -z "${KEYID}" ]]; then
	log_error "Failed to load gpg key. Is gpg set up correctly for etcd releases?"
	exit 1
	fi
	git tag --local-user "${KEYID}" --sign "${RELEASE_VERSION}" --message "${RELEASE_VERSION}"
	fi

So, I don't think we need to make any changes to this.

[ahrtr]

So, I don't think we need to make any changes to this.

Right, but the script has never been verified/tested :)

[ivanvc]

Right, but the script has never been verified/tested :)

Do you think we should try at next's 3.4 release? Or, what would be a good way to test this?

[jmhbnz]

Do you think we should try at next's 3.4 release? Or, what would be a good way to test this?

I have a plan to safely verify this at the next 3.4 release. There are additional flags we can use in the script to test changes locally.

[ivanvc]

Ah, @jmhbnz, I didn't see your reply, so I was finding a way to test this. I managed to do it using my fork (https://github.com/ivanvc/etcd/tree/release-3.4) and passed the REPOSITORY variable set to my fork. I verified that it partially works:

1. It did update version/version.go, and pushed it to the remote (ivanvc@853b46f).
2. It did create the tag but didn't push it. After reviewing the script, I verified that it does not push the tag.

Is it okay if I implement pushing the tag, and we test it during the next v3.4 after I verify it works on my fork?

[jmhbnz]

Is it okay if I implement pushing the tag, and we test it during the next v3.4 after I verify it works on my fork?

Absolutely! - I would like to create some consistency between both stable release branches so that both work the same way with the same level of automation.

[ivanvc]

With all tasks completed, I'll go ahead and close this issue. Thanks, team, for helping with another successful v3.4 release 🎉

[jcferretti]

Do we want to merge its PR (#18268)?

NO, the PR changes the most important part of etcdserver, but without proper test cases. Since it's low priority/impact, we can revisit it in future when we have bandwidth.

There have been no responses on that PR for 2 months.
If you had mentioned anything about the existing tests and how it is that they are lacking or what you would actually want to see, I could have tried to do something about it.