Skip to content

Commit

Permalink
Squashed 'release-tools/' changes from 670bb0e..b54c1ba
Browse files Browse the repository at this point in the history
b54c1ba Merge pull request kubernetes-csi#246 from xing-yang/go_1.21
5436c81 Change go version to 1.21.5
267b40e Merge pull request kubernetes-csi#244 from carlory/sig-storage
b42e5a2 nominate self (carlory) as kubernetes-csi reviewer
a17f536 Merge pull request kubernetes-csi#210 from sunnylovestiramisu/sidecar
011033d Use set -x instead of die
5deaf66 Add wrapper script for sidecar release
f8c8cc4 Merge pull request kubernetes-csi#237 from msau42/prow
b36b5bf Merge pull request kubernetes-csi#240 from dannawang0221/upgrade-go-version
adfddcc Merge pull request kubernetes-csi#243 from pohly/git-subtree-pull-fix
c465088 pull-test.sh: avoid "git subtree pull" error
7b175a1 Update csi-test version to v5.2.0
987c90c Update go version to 1.21 to match k/k
2c625d4 Add script to generate patch release notes
f9d5b9c Merge pull request kubernetes-csi#236 from mowangdk/feature/bump_csi-driver-host-path_version
b01fd53 Bump csi-driver-host-path version up to v1.12.0
984feec Merge pull request kubernetes-csi#234 from siddhikhapare/csi-tools
1f7e605 fixed broken links of testgrid dashboard
de2fba8 Merge pull request kubernetes-csi#233 from andyzhangx/andyzhangx-patch-1
cee895e remove windows 20H2 build since it's EOL long time ago

git-subtree-dir: release-tools
git-subtree-split: b54c1ba
  • Loading branch information
andyzhangx committed Dec 16, 2023
1 parent ffbfd04 commit 159c9a0
Show file tree
Hide file tree
Showing 6 changed files with 283 additions and 52 deletions.
1 change: 1 addition & 0 deletions KUBERNETES_CSI_OWNERS_ALIASES
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ aliases:
# when they are temporarily unable to review PRs.
kubernetes-csi-reviewers:
- andyzhangx
- carlory
- chrishenzie
- ggriffiths
- gnufied
Expand Down
90 changes: 42 additions & 48 deletions SIDECAR_RELEASE_PROCESS.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ The release manager must:
Whenever a new Kubernetes minor version is released, our kubernetes-csi CI jobs
must be updated.

[Our CI jobs](https://k8s-testgrid.appspot.com/sig-storage-csi-ci) have the
[Our CI jobs](https://testgrid.k8s.io/sig-storage-csi-ci) have the
naming convention `<hostpath-deployment-version>-on-<kubernetes-version>`.

1. Jobs should be actively monitored to find and fix failures in sidecars and
Expand Down Expand Up @@ -46,62 +46,56 @@ naming convention `<hostpath-deployment-version>-on-<kubernetes-version>`.
## Release Process
1. Identify all issues and ongoing PRs that should go into the release, and
drive them to resolution.
1. Download the latest version of the
[K8s release notes generator](https://github.com/kubernetes/release/tree/HEAD/cmd/release-notes)
1. Create a
[Github personal access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token)
with `repo:public_repo` access
1. Generate release notes for the release. Replace arguments with the relevant
information.
* Clean up old cached information (also needed if you are generating release
notes for multiple repos)
```bash
rm -rf /tmp/k8s-repo
```
* For new minor releases on master:
```bash
GITHUB_TOKEN=<token> release-notes \
--discover=mergebase-to-latest \
--org=kubernetes-csi \
--repo=external-provisioner \
--required-author="" \
--markdown-links \
--output out.md
```
* For new patch releases on a release branch:
```bash
GITHUB_TOKEN=<token> release-notes \
--discover=patch-to-latest \
--branch=release-1.1 \
--org=kubernetes-csi \
--repo=external-provisioner \
--required-author="" \
--markdown-links \
--output out.md
```
1. Compare the generated output to the new commits for the release to check if
any notable change missed a release note.
1. Reword release notes as needed. Make sure to check notes for breaking
changes and deprecations.
1. If release is a new major/minor version, create a new `CHANGELOG-<major>.<minor>.md`
file. Otherwise, add the release notes to the top of the existing CHANGELOG
file for that minor version.
1. Submit a PR for the CHANGELOG changes.
1. Submit a PR for README changes, in particular, Compatibility, Feature status,
and any other sections that may need updating.
1. Update dependencies for sidecars via
[go-modules-update.sh](https://github.com/kubernetes-csi/csi-driver-host-path/blob/HEAD/release-tools/go-modules-update.sh),
and get PRs approved and merged.
1. Check that all [canary CI
jobs](https://k8s-testgrid.appspot.com/sig-storage-csi-ci) are passing,
jobs](https://testgrid.k8s.io/sig-storage-csi-ci) are passing,
and that test coverage is adequate for the changes that are going into the release.
1. Check that the post-\<sidecar\>-push-images builds are succeeding.
[Example](https://k8s-testgrid.appspot.com/sig-storage-image-build#post-external-snapshotter-push-images)
[Example](https://testgrid.k8s.io/sig-storage-image-build#post-external-snapshotter-push-images)
1. Generate release notes.
1. Download the latest version of the [K8s release notes generator](https://github.com/kubernetes/release/tree/HEAD/cmd/release-notes)
1. Create a
[Github personal access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token)
with `repo:public_repo` access
1. For patch release, use the script generate_patch_release_notes.sh. Read the instructions at the top of the
script. The script also creates PRs for each branch.
1. For new minor releases, follow these steps and replace arguments with the relevant
information.
* Clean up old cached information (also needed if you are generating release
notes for multiple repos)
```bash
rm -rf /tmp/k8s-repo
```
* For new minor releases on master:
```bash
GITHUB_TOKEN=<token> release-notes \
--discover=mergebase-to-latest \
--org=kubernetes-csi \
--repo=external-provisioner \
--required-author="" \
--markdown-links \
--output out.md
```
1. Compare the generated output to the new commits for the release to check if
any notable change missed a release note.
1. Reword release notes as needed, ideally in the original PRs so that the
release notes can be regnerated. Make sure to check notes for breaking
changes and deprecations.
1. If release is a new major/minor version, create a new `CHANGELOG-<major>.<minor>.md`
file.
1. Submit a PR for the CHANGELOG changes.
1. Submit a PR for README changes, in particular, Compatibility, Feature status,
and any other sections that may need updating.
1. Make sure that no new PRs have merged in the meantime, and no PRs are in
flight and soon to be merged.
1. Create a new release following a previous release as a template. Be sure to select the correct
branch. This requires Github release permissions as required by the prerequisites.
[external-provisioner example](https://github.com/kubernetes-csi/external-provisioner/releases/new)
1. If release was a new major/minor version, create a new `release-<minor>`
branch at that commit.
1. Check [image build status](https://k8s-testgrid.appspot.com/sig-storage-image-build).
1. Check [image build status](https://testgrid.k8s.io/sig-storage-image-build).
1. Promote images from k8s-staging-sig-storage to registry.k8s.io/sig-storage. From
the [k8s image
repo](https://github.com/kubernetes/k8s.io/tree/HEAD/registry.k8s.io/images/k8s-staging-sig-storage),
Expand All @@ -120,7 +114,7 @@ naming convention `<hostpath-deployment-version>-on-<kubernetes-version>`.

The following jobs are triggered after tagging to produce the corresponding
image(s):
https://k8s-testgrid.appspot.com/sig-storage-image-build
https://testgrid.k8s.io/sig-storage-image-build

Clicking on a failed build job opens that job in https://prow.k8s.io. Next to
the job title is a rerun icon (circle with arrow). Clicking it opens a popup
Expand Down
106 changes: 106 additions & 0 deletions generate_patch_release_notes.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
#!/bin/bash

# Copyright 2023 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.


# Usage: generate_patch_release_notes.sh
#
# Generates and creates PRs for kubernetes-csi patch releases.
#
# Required environment variables
# CSI_RELEASE_TOKEN: Github token needed for generating release notes
# GITHUB_USER: Github username to create PRs with
#
# Instructions:
# 1. Login with "gh auth login"
# 2. Copy this script to the kubernetes-csi directory (one directory above the
# repos)
# 3. Update the repos and versions in the $releases array
# 4. Set environment variables
# 5. Run script from the kubernetes-csi directory
#
# Caveats:
# - This script doesn't handle regenerating and updating existing PRs yet.
# It might work if you comment out the PR creation line

set -e
set -x

releases=(
# "external-attacher 4.4.1"
# "external-provisioner 3.6.1"
# "external-snapshotter 6.2.3"
)

function gen_patch_relnotes() {
rm out.md || true
rm -rf /tmp/k8s-repo || true
GITHUB_TOKEN="$CSI_RELEASE_TOKEN" \
release-notes --discover=patch-to-latest --branch="$2" \
--org=kubernetes-csi --repo="$1" \
--required-author="" --markdown-links --output out.md
}

for rel in "${releases[@]}"; do
read -r repo version <<< "$rel"

# Parse minor version
minorPattern="(^[[:digit:]]+\.[[:digit:]]+)\."
[[ "$version" =~ $minorPattern ]]
minor="${BASH_REMATCH[1]}"

echo "$repo" "$version" "$minor"

pushd "$repo/CHANGELOG"

git fetch upstream

# Create branch
branch="changelog-release-$minor"
git checkout master
git branch -D "$branch" || true
git checkout --track "upstream/release-$minor" -b "$branch"

# Generate release notes
gen_patch_relnotes "$repo" "release-$minor"
cat > tmp.md <<EOF
# Release notes for v$version
[Documentation](https://kubernetes-csi.github.io)
EOF

cat out.md >> tmp.md
echo >> tmp.md

file="CHANGELOG-$minor.md"
cat "$file" >> tmp.md
mv tmp.md "$file"

git add -u
git commit -m "Add changelog for $version"
git push -f origin "$branch"

# Create PR
prbody=$(cat <<EOF
\`\`\`release-note
NONE
\`\`\`
EOF
)
gh pr create --title="Changelog for v$version" --body "$prbody" --head "$GITHUB_USER:$branch" --base "release-$minor" --repo="kubernetes-csi/$repo"

popd
done
129 changes: 129 additions & 0 deletions go-modules-update.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,129 @@
#!/bin/sh

# Copyright 2023 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.


# Usage: go-modules-update.sh
#
# Batch update dependencies for sidecars.
#
# Required environment variables
# CSI_RELEASE_TOKEN: Github token needed for generating release notes
# GITHUB_USER: Github username to create PRs with
#
# Instructions:
# 1. Login with "gh auth login"
# 2. Copy this script to the kubernetes-csi directory (one directory above the
# repos)
# 3. Update the repos and master branch so locally it has the latest upstream
# change
# 4. Set environment variables
# 5. Run script from the kubernetes-csi directory
#
# Caveats:
# - This script doesn't handle interface incompatibility of updates.
# You need to resolve interface incompatibility case by case. The
# most frequent case is to update the interface(new parameters,
# name change of the method, etc.)in the sidecar repo and make sure
# the build and test pass.


set -e
set -x

MAX_RETRY=10

# Get the options
while getopts ":u:v:" option; do
case $option in
u) # Set username
username=$OPTARG;;
v) # Set version
v=$OPTARG;;
\?) # Invalid option
echo "Error: Invalid option: $OPTARG"
exit;;
esac
done

# Only need to do this once
gh auth login

while read -r repo branches; do
if [ "$repo" != "#" ]; then
(
cd "$repo"
git fetch origin
for i in $branches; do
if [ "$(git rev-parse --verify "module-update-$i" 2>/dev/null)" ]; then
git checkout master && git branch -d "module-update-$i"
fi
git checkout -B "module-update-$i" "origin/$i"
rm -rf .git/MERGE*
if ! git subtree pull --squash --prefix=release-tools https://github.com/kubernetes-csi/csi-release-tools.git master; then
# Sometimes "--squash" leads to merge conflicts. Because we know that "release-tools"
# is an unmodified copy of csi-release-tools, we can automatically resolve that
# by replacing it completely.
if [ -e .git/MERGE_MSG ] && [ -e .git/FETCH_HEAD ] && grep -q "^# Conflict" .git/MERGE_MSG; then
rm -rf release-tools
mkdir release-tools
git archive FETCH_HEAD | tar -C release-tools -xf -
git add release-tools
git commit --file=.git/MERGE_MSG
else
exit 1
fi
fi
RETRY=0
while ! ./release-tools/go-get-kubernetes.sh -p "$v" && RETRY < $MAX_RETRY
do
RETRY=$((RETRY+1))
go mod tidy && go mod vendor && go mod tidy
done
go mod tidy && go mod vendor && go mod tidy
git add --all
git commit -m "Update dependency go modules for k8s v$v"
git remote set-url origin "https://github.com/$username/$repo.git"
make test
git push origin "module-update-$i" --force
# Create PR
prbody=$(cat <<EOF
Ran kubernetes-csi/csi-release-tools go-get-kubernetes.sh -p ${v}.
\`\`\`release-note
Update kubernetes dependencies to v${v}
\`\`\`
EOF
)
gh pr create --title="Update dependency go modules for k8s v$v" --body "$prbody" --head "$username:module-update-master" --base "master" --repo="kubernetes-csi/$repo"
done
)
fi
done <<EOF
csi-driver-host-path master
csi-driver-iscsi master
csi-driver-nfs master
csi-lib-utils master
csi-proxy master
csi-test master
external-attacher master
external-health-monitor master
external-provisioner master
external-resizer master
external-snapshotter master
livenessprobe master
node-driver-registrar master
EOF
Loading

0 comments on commit 159c9a0

Please sign in to comment.