v1.18.0-alpha.3
Pre-releaseThis version contains a critical update to etcd-manager: 1 year after creation (or first adopting etcd-manager), clusters will stop responding due to expiration of a TLS certificate. Upgrading kops to 1.18.0-alpha.3 (or the latest versions of the 1.15, 1.16, 1.17 or 1.18 series) and running kops update
followed by a kops rolling-update
will fix the issue. Please see the advisory for the full details.
kops 1.18.0 alpha.3 is the next alpha in the 1.18 series for kops.
Please see the release notes for the full list of changes.
Significant changes
-
The default Docker version has been changed to 19.03.8.
-
Support for Amazon Linux 2 has been improved and will work with the default Docker version.
-
containerd has been added and can be selected as an alternate container runtime for Kubernetes. Enable by using the
--container-runtime containerd
flag when creating a cluster or by settingspec.containerRuntime: containerd
. -
Rolling updates now support surging and parallelism within an instance group. For details see the documentation.
-
Cilium CNI can now use AWS networking natively through the AWS ENI IPAM mode. Kops can also run a Kubernetes cluster entirely without kube-proxy using Cilium's BPF NodePort implementation.
-
Cilium CNI can now use a dedicated etcd cluster managed by etcd-manager for synchronizing agent state instead of CRDs.
-
The Terraform target now supports Terraform 0.12 syntax (HCL2) by default. See the Required Actions item below.
-
New clusters in GCE are configured to run the metadata-proxy by default. The proxy runs as a DaemonSet and lands on nodes with the nodeLabel
cloud.google.com/metadata-proxy-ready: "true"
. If you want to enable metadata-proxy on an existing cluster/instance group, add that nodeLabel to your instancegroup specs (kops edit ig ...
) and runkops update cluster
. When the changes are applied, the proxy will roll out to those targeted nodes. -
GCE has a new flag:
--gce-service-account
. This takes the email of an existing GCP service account and launches the instances with it. This setting applies to the whole cluster (ie: it is not currently designed to support Instance Groups with different service accounts). If you do not specify a service account during cluster creation, the default compute service account will be used which matches the prior behavior. -
Google API client libraries updated from v0.beta to v1.
Breaking changes
-
Support for Docker versions 1.11, 1.12 and 1.13 has been removed because of the dockerproject.org shut down. Those affected must upgrade to a newer Docker version.
-
Terraform users on AWS may need to rename some resources in their state file in order to prepare for Terraform 0.12 support. See Required Actions below.
-
The Docker
health-check
service is now disabled by default. It shouldn't be needed anymore, but it can still be enabled by settingspec.docker.healthCheck: true
. It is recommended to also check node-problem-detector and draino as replacements. See Required Actions below. -
Lyft CNI plugin default subnet tags changed from from
Type: pod
toKubernetesCluster: myclustername.mydns.io
. Subnets intended for use by the plugin will need to be tagged with this new tag and additional tag filters may need to be added to the cluster spec in order to achieve the desired set of subnets. -
Support for basic authentication has been disabled by default and will be removed in Kubernetes 1.19.
-
Support for Kubernetes versions prior to 1.9 has been removed.
-
Kubernetes 1.9 users will need to enable the PodPriority feature gate. See Required Actions below.
-
A controller is now used to apply labels to nodes. If you are not using AWS, GCE or OpenStack your (non-master) nodes may not have labels applied correctly.
-
The
kops/v1alpha1
API has been removed. Users ofkops replace
will need to supply v1alpha2 resources. -
Please see the notes in the 1.15 release about the apiGroup changing from kops to kops.k8s.io
Required Actions
-
Terraform users on AWS may need to rename resources in their terraform state file in order to prepare for future Terraform 0.12 support.
Terraform 0.12 no longer supports resource names starting with digits. In Kops, both the default route and additional VPC CIDR associations are affected. See #7957 for more information.- The default route was named
aws_route.0-0-0-0--0
and will now be namedaws_route.route-0-0-0-0--0
. - Additional CIDR blocks associated with a VPC were similarly named the hyphenated CIDR block with two hyphens for the
/
, for exampleaws_vpc_ipv4_cidr_block_association.10-1-0-0--16
. These will now be prefixed withcidr-
, for exampleaws_vpc_ipv4_cidr_block_association.cidr-10-1-0-0--16
.
To prevent downtime, follow these steps with the new version of Kops:
kops update cluster --target terraform ... terraform plan # Observe any aws_route or aws_vpc_ipv4_cidr_block_association resources being destroyed and recreated # Run these commands as necessary. The exact names may differ; use what is outputted by terraform plan terraform state mv aws_route.0-0-0-0--0 aws_route.route-0-0-0-0--0 terraform state mv aws_vpc_ipv4_cidr_block_association.10-1-0-0--16 aws_vpc_ipv4_cidr_block_association.cidr-10-1-0-0--16 terraform plan # Ensure these resources are no longer being destroyed and recreated terraform apply
- The default route was named
-
Kubernetes 1.9 users will need to enable the PodPriority feature gate. This is required for newer versions of Kops.
To enable the Pod priority feature, follow these steps:
kops edit cluster # Add the following section spec: kubelet: featureGates: PodPriority: "true"
-
If a custom Kops build was used on a cluster, a kops-controller Deployment may have been created that should get deleted.
Runkubectl -n kube-system delete deployment kops-controller
after upgrading to Kops 1.16.0-beta.1 or later.
Deprecations
- Support for Kubernetes versions 1.9 and 1.10 are deprecated and will be removed in kops 1.19.
Changes from v1.18.0-alpha.2 to v1.18.0-alpha.3
- Release notes for 1.18.0-alpha.2 @justinsb #8480
- Update cilium to 1.6.6 @olemarkus #8484
- Release notes for 1.17.0-alpha.3 @justinsb #8485
- Make it possible to enable Prometheus metrics for Cilium @olemarkus #8433
- GCS: Don't try to set ACLs if bucket-policy only is set @justinsb #8493
- Revert "Merge pull request #8452 from maruina/coredns-1.6.7" @gjtempleton #8503
- GCS: Fix bug where around retry on GCS @justinsb #8495
- Fix Netlify deps for tests @hakman #8506
- GCS: Log ACLs if we're writing them @justinsb #8496
- Move cloudbuild.yaml to repo root @rifelpet #8505
- Add support for containerd v1.3.3 @hakman #8504
- Use BindMount task for /home/kubernetes/bin @justinsb #8461
- Release notes for 1.15.2 @justinsb #8513
- Image Pushing - Remove .git dependency from get_workspace_status.sh @rifelpet #8514
- [vendoring] Update google libs to latest @geojaz #8516
- cloudbuild: Try uploading our binaries to the staging bucket also @justinsb #8518
- Return more errors at once during API validation @johngmyers #8449
- Release notes for 1.16.0-beta.2 @justinsb #8519
- Bump alphas to stable @olemarkus #8489
- Use larger machine type for cloudbuild jobs @rifelpet #8520
- Mark dns-controller and kops-controller as non-root @johngmyers #8497
- Alicloud: support modification of LaunchConfiguration @bittopaz #8246
- Fix Machine types vcpus @mikesplain #8522
- Update AWS IAM Authenticator to 0.5.0 @rifelpet #8423
- Release notes for 1.17.0-alpha.4 @justinsb #8524
- Add unit test for pkg/jsonutils/streamwriter.go @tiendc #8475
- channels: bump k8s versions after feb. patch releases @idealhack #8532
- Add events RBAC permissions to kops-controller @rifelpet #8535
- Update amazon-vpc-cni-k8s to v1.6.0 @hakman #8538
- Cilium nodeport @olemarkus #8220
- Update IAM permissions for amazon-vpc-cni-k8s 1.6.0 @rifelpet #8548
- Only create git tags with the v prefix @rifelpet #8547
- Switch AWS IAM Authenticator to use non-scratch image @rifelpet #8555
- Add Kubernetes deprecation/removal release notes @johngmyers #8443
- go-bindata: don't store file metadata @justinsb #8565
- Replace unmaintained go-bindata @justinsb #8561
- gzip: use --no-name to avoid saving timestamp @justinsb #8566
- Update makefile to reduce dependency on GOPATH @rifelpet #8570
- Fix field names in api validation @johngmyers #8569
- Update golang to 1.13.8 @justinsb #8574
- Hack directory cleanup @rifelpet #8572
- Cilium eni @olemarkus #8316
- EnsureFileOwner: cleanup logic @justinsb #8576
- Use UTC when calculating copyright year @johngmyers #8575
- makefile: use more hash helpers when building images @justinsb #8568
- Add External Policies (AWS managed policy attachments) @mattouille #7837
- Fix inappropriate use of field.Invalid() @johngmyers #8578
- Use cluster name as default subnet tag for Lyft CNI @hakman #8571
- Staging build: don't use _GIT_TAG @justinsb #8585
- prow-postsubmit: Upload to a consistently named directory @justinsb #8586
- Spotinst: Add OWNERS file to Spotinst specific packages @liranp #8533
- Bump Cilium to 1.7 for k8s 1.12+ @olemarkus #8589
- add s3 region @zetaab #8592
- Add unit test for pkg/nodelabels/builder.go @tiendc #8539
- Fix typo in the cilium default version @olemarkus #8591
- Add unit test for pkg/apis/kops/util/labels.go @tiendc #8595
- Fail cluster validation if a master missing kube-controller-manager @johngmyers #8600
- Update tags support for LaunchTemplates @hakman #8466
- Fix a bug in UT apis/kops/util/labels_test.go @tiendc #8604
- [Digital Ocean] Update documentation for multi-master HA setup @srikiz #8601
- Add release notes and docs for containerd @hakman #8603
- Update coredns to 1.6.7 @maruina #8602
- Use supported kubernetes versions in tests @johngmyers #8610
- kubelet: map rotate-certificates options @justinsb #8581
- Resolve TODO in /pkg/model/component/context.go @sshukun #8579
- Fix typo in comment for struct LyftVPCNetworkingSpec @fenggw-fnst #8590
- Cilium fix bpffs check @olemarkus #8599
- Spotinst: Add support for blacklisting or whitelisting instance types in Ocean @liranp #8549
- Don't try to configure a bastion DNS name in gossip mode @justinsb #8573
- use v0.29 in prometheus operator @coryschwartz #8563
- Update Calico CNI Networking Documentation Links @jpugliesi #8473
- Spotinst: Support for Tags (Cloud Labels) in Ocean Launch Spec @liranp #8616
- Refuse to read channels with addons with unparseable versions @johngmyers #8408
- Adding Unit Test for pkg/dns/gossip.go #8631
- Validate at most one networking option chosen @johngmyers #8617
- Add support for custom env vars in amazon-vpc-cni @rifelpet #8276
- Promote alphas to stable @olemarkus #8632
- Release notes for 1.16.0 @justinsb #8637
- Update Calico and Canal to v3.12.0 @hakman #8618
- Release notes for 1.17.0 beta.1 @justinsb #8641
- Update release process @justinsb #8453
- Fix node-authorizer/pkg/server staticcheck failure @hakman #8649
- Bump Calico and Canal version tags for older k8s @hakman #8646
- Add the aws iam authenticator known issue to the 1.17 release notes @rifelpet #8651
- Install packages updates during bootstrap @hakman #8635
- Treat nil of LaunchTemplateSpecification.Version as $Default [0] @qqshfox #8567
- Cleanup release documentation @rifelpet #8654
- Fix condition for innstall packages updates during bootstrap @hakman #8657
- Delete launch templates when using EnableLaunchTemplates @hakman #8639
- Update release notes for the default instance type change @hakman #8658
- Remove not-yet-released blurb now that 1.16.0 has been released @johngmyers #8659
- Add "owned" tag for volumes and instances created with launch templates @hakman #8660
- support apiserver --kubelet-certificate-authority flag @tanjunchen #8661
- Fix punctuation @johngmyers #8663
- Fix SecurityGroup check when there are no Network Interfaces associated with a LaunchTemplate @KashifSaadat #8666
- Revert AWS IAM Authenticator upgrade to 0.5.0 on master @rifelpet #8670
- Update compatibility matrix @johngmyers #8672
- Option to surge during rolling update @johngmyers #8313
- retry nvidia-device-plugin.sh when failed @qqshfox #7989
- Changed "you cache" to "your cache" @medined #8669
- [Digital Ocean] Fix minor document issue @srikiz #8676
- Default maxSurge to 1 on AWS @johngmyers #8674
- Update roadmap @johngmyers #8675
- amazon-vpc-routed-eni env variable config @mikesplain #8682
- [Docs] Add service account token volumes @jmymy #8680
- Update rolling update documentation @johngmyers #8673
- Add more API validation for Cilium @johngmyers #8667
- Update Weave Net to version 2.6.1 @hakman #8691
- Update etcd-manager to 3.0.20200307 @justinsb #8692
- [Docs] add link for Service Account Token Volumes @jmymy #8688
- Add UT for util/pkg/vfs/fs.go @tiendc #8686
- remove // TODO path.Join here? @tanjunchen #8701
- Refactor/simplify rolling update @johngmyers #8709
- Add UT for util/pkg/proxy/proxy.go @tiendc #8640
- Adding Unit Test for util/pkg/exec/exec.go #8712
- remove // TODO move this into networking.go :( @tanjunchen #8711
- Bump Cilium to 1.7.1 for k8s 1.12+ @errordeveloper #8713
- Use BPF_FS_MAGIC from go sys lib instead of hardcode @odinuge #8650
- Fix uploading of file assets @johngmyers #8694
- Add labels to OWNERS files @rifelpet #8717
- Support the kops.k8s.io/needs-update annotation on nodes @johngmyers #8272
- When nodeport is enabled, use strict kube-proxy-replacement to ensure… @olemarkus #8655
- Upgrade AWS SDK @rifelpet #8721
- fix OWNERS labels format @rifelpet #8725
- Adding docs for calico MTU @vgunapati,@rifelpet #8724
- Add support for Ubuntu 20.04 (Focal) @hakman #8727
- Use PriorityClassName instead of namespace in cluster validation @johngmyers #8446
- Fix kuberouter for k8s 1.16+ @UnderMyBed #8697
- Update cilium docs with note about eni and bpf nodeport @olemarkus #8551
- Update alpha channel with the march updates @olemarkus #8733
- Document all the fields in the Cilium spec @olemarkus #8559
- Update kube-router to v0.4.0 @hakman #8735
- Add UT for util/pkg/vfs/memfs.go @tiendc #8730
- Remove unused integration test files, make complex cluster spec consistent @rifelpet #8739
- Set kube-proxy-replacement to partial @olemarkus #8741
- Create New Default StorageClass: kops-ssd-1-17 @joshbranham #8582
- Fix template for kube-router v0.4.0 @hakman #8742
- Node metadata-concealment in GCE @geojaz #8634
- Add cloud ntp addresses @simonmacklin #8312
- Update support for docker tgz package @hakman #8525
- Add option to use etcd managed by cilium-etcd-operator as kvstore @olemarkus #8629
- Tweak the featureflag.GoogleCloudBucketACL.Enabled @geojaz #8747
- Move "docker-healthcheck" to DockerBuilder @hakman #8221
- Update Docker to 19.03.8 for 1.18 @hakman #8748
- Fix template for kube-router v0.4.0 #2 @hakman #8751
- etcd admin doc @sdarwin #8729
- Allow users to overwrite etcd settings. @rdrgmnzs #8402
- Update apiVersion to resolve 'no matches for kind "Deployment" in ver… @medined #8732
- Fix command descriptions to match new cluster validation behavior @johngmyers #8755
- Deprecate the route53-mapper addon @rifelpet #8753
- Use containerd 1.2.13 with Docker 19.03.8 @hakman #8756
- Expose etcd metrics. @michalschott #8162
- Add LaunchTemplate integration tests @hakman #8758
- Add root volume tags for CF and TF targets @hakman #8693
- Gce tests lifecycle @geojaz #8760
- Update lyft CNI to 0.6.0 @maruina #8757
- Update Weave Net to version 2.6.2 @hakman #8764
- Fix NTP failures after #8312 @hakman #8763
- add support for OpenStack Octavia AllowedCIDRs @zetaab #8762
- Spotinst: Upgrade the Spotinst controller to version 1.0.57 @liranp #8770
- Add missing priorityClassName to addons @johngmyers #8752
- Allow configuration of enable-remote-node-identity @olemarkus #8731
- Wait for validation to succeed N consecutive times @hakman #8515
- Set default log level to "info" for containerd @hakman #8782
- Upgrade to bazel 2.2.0 @rifelpet #8790
- Add cilium option tofqdns-dns-reject-response-code @fentas #8779
- Update bazel rules @rifelpet #8749
- Make cilium operator health check go against localhost IP @olemarkus #8789
- Canal v3.13.1 @KashifSaadat #8795
- Fix weaveworks/mesh vendoring @rifelpet #8797
- Update default users for kubeconfig with supported distros @hakman #8798
- Update default user for CoreOS, Flatcar for kubecfg @KashifSaadat #8800
- Add UT for upup/pkg/fi/files.go @tiendc #8776
- Deprecate support for Kubernetes 1.10 @johngmyers #8799
- Fix terraform json field tags @rifelpet #8813
- Remove v1alpha1 API @johngmyers #8431
- Disable anonymous auth in integration test clusters @rifelpet #8816
- Refactor integration tests @rifelpet #8737
- Fix dns-controller flapping on spot instances @johngmyers #8821
- Adding support for the Middle East (Bahrain) region (me-south-1) @liranp #8820
- Don't set terraform user_data if it is empty @rifelpet #8744
- Add terraform testing @rifelpet #8734
- Add support for Spot block in launch template @thejasbabu #8802
- Cleanup InstanceGroup MixedInstancePolicy docs @gjtempleton #8832
- Enable stamping on bazel image builds @rifelpet #8835
- Remove unused ECU and Burstable fields from AWSMachineTypeInfo @rifelpet #8844
- Use CNI 0.8.5 for Kubernetes 1.18+ @hakman #8848
- Update GCB jobs to use bazel 2.2.0 @rifelpet #8849
- Adds a gce-service-account flag so you BYO service-account @geojaz #8761
- Balk at using Kubernetes versions too new to be supported @johngmyers #8700
- Remove basic authentication support for k8s 1.19+ @hakman #8783
- Make kubelet topology-manager-policy configurable @olemarkus #8833
- Switch to the k/k implementation of drain.Helper @johngmyers #8852
- Update build instructions for dns controller @justinsb #8853
- Switch to using ec2.DescribeInstanceTypes for building the MachineTypes list @rifelpet #8847
- Remove support for Docker 1.11, 1.12 and 1.13 @hakman #8855
- Cleanup after #8855 @hakman #8846
- fixes auth problem in case of openstack kops-controller @zetaab #8862
- Update metrics-server deployment apiVersion to apps/v1 @elisiano #8842
- Update Calico and Canal to v3.13.2 @hakman #8865
- GCE: Delete cluster will also delete the DNS entries created by kubernetes @mccare,@justinsb #8250
- Add Terraform 0.12 support @rifelpet #8825
- Don't compress bindata & allow Kops to be imported as a package. @rdrgmnzs,@justinsb #8584
- Validate cluster N times in rolling-update @zetaab #8868
- Update go.mod for k8s 1.17 @justinsb #8873
- pkg: add some unit tests @q384566678 #8872
- add verify-bindata.sh & separate bindata check and update make commands. @rdrgmnzs #8876
- Update controller-tools and generate v1 CRDs @rifelpet #8875
- Don't delete bindata on "make clean" @rifelpet #8877
- Fix Terraform 0.12 support for GCE @rifelpet #8878
- Makefile: simplify running go-bindata @justinsb #8880
- update gophercloud dependencies @zetaab #8884
- Update to golang 1.13.9 to match 1.18.1 @justinsb #8882
- Use kubescheduler.config.k8s.io/v1alpha2 for Kubernetes 1.18+ @hakman #8883
- Fix addon/ing/nginx v1.6.0 deployment apiVersion and selector @mousavian #8857
- Update go to 1.13.9 (continued) @justinsb #8886
- Replace deprecated x/net/context with context @justinsb #8885
- Add go 1.14 to travis testing @justinsb #8893
- Update 1.18 release notes now that Terraform 0.12 is supported @rifelpet #8892
- Add Docker version and new distros to release notes @hakman #8894
- NodeLocal DNSCache @mazzy89,@olemarkus #8780
- Remove duplicated docs @olemarkus #8895
- Adds docsite hidden note on index.md @mikesplain #8900
- Clean up the kubectl docs @olemarkus #8901
- Update to kubernetes 1.18.1 dependencies @justinsb #8879
- Add CloudLabels tags to additional AWS resources @rifelpet #8903
- Add unit test for func IsGTE @Hellcatlk #8905
- Clean up the spec docs @olemarkus #8902
- Etcd docs cleanup @olemarkus #8907
- Exclude GO 1.14 for OSX @hakman #8918
- Update security_groups.md @zhming0 #8912
- Use etcd-manager for the cilium etcd cluster @olemarkus #8750
- Bastion additionnal s gs @nvanheuverzwijn #8921
- Change NodeLocalDNS Enabled to *bool @justinsb #8930
- Rolling-update validation harmonization @justinsb #8931
- Upgrade CUDA from 9.1 to 10.0 @fifar #8482
- Import package aliases modified @tomesm #8933
- Remove irrelevant TODO comment from userdata @rifelpet #8936
- Update the testing page @olemarkus #8934
- Spotinst: Support for Headroom, Taints and Subnets in Ocean Launch Spec and User Data fixes @liranp #8294
- Partially implement MountSensitive @justinsb #8939
- Use latest patch version for GO in tests @hakman #8940
- Add support for containerd v1.3.4 @hakman #8941
- Add unit test for func rewriteAPIGroup @hs0210 #8947
- Load the correct certificate before deleting @olemarkus #8945
- Add unit test for func JoinSuffixes @hs0210 #8955
- Update rotating secrets docs @olemarkus #8948
- Fix net.bridge setting for Flannel on CentOS 7 @bittopaz #8381
- Turn on flannel sysctl fixes on rhel7 as well @justinsb #8958
- Add unit test case for pkg/systemd/escaping_test.go @Hellcatlk #8906
- Add release notes for 1.16.1 @justinsb #8966
- Modify the wrong name for unit test case @Hellcatlk #8970
- Bumps the gcp client libraries to v0.22.0. @geojaz #8969
- Migrates GCE sdk from v0.beta to v1 @geojaz #8972
- Update Calico and Canal to latest patch versions @hakman #8961
- Update alpha and stable channels with April updates @MoShitrit #8974
- Make addons directory link absolute to avoid 404 @dj80hd #8986
- Make CPU Requests for Calico configurable instead of hard-coded @MoShitrit #8987
- Ignore already-deleted nodes during rolling update @johngmyers #8994
- Add initial github actions workflow @irons #8990
- feat(openstack): propagate cloud labels to machines @mitch000001 #8999
- Profiling and protect kernel defaults flags @michalschott #8967
- Allow single to multi-master cluster @olemarkus #8980
- Bump cilium to 1.7.2 @olemarkus #8982
- GCE: needs to use v1 network resources, not beta @geojaz #9004
- Default to Debian 9 (Stretch) for Kubernetes 1.8, 1.9 and 1.10 @hakman #8977
- Fix missing changes in Weave manifest @hakman #8965
- Spotinst: Support for Grace Period in Ocean Cluster @liranp #9018
- Upgrade amazon vpc cni to 1.6.1 @rifelpet #9020
- Add af-south-1 and eu-south-1 regions (AWS) @MoShitrit #9021
- Update to etcd-manager 3.0.20200429 @justinsb #9016
- Use KOPS_RSA_PRIVATE_KEY_SIZE for more keys @johngmyers #8988
- Don't force bazel to use python2 @olemarkus #8998
- GCE DNS Cleanup: Print record name @justinsb #8869
- Bump cilium to 1.7.3 @olemarkus #9026
- Update docs on supported distros and images @hakman #9029
- Rename tests to make it easier to search for failing tests @rifelpet #9033
- Use raw tarfile for AmazonLinux2 + Containerd 1.2.10 @justinsb #9032
- Correct expression typos on docs/aws.md @gpontesss,@rifelpet #7733
- Update the cross-account example with working policy @olemarkus #9019
- Allow changing AZ of masters @olemarkus #9017
- Update k8s.gcr.io/pause image to version 3.2 @hakman #9039
- Update docs on supported distros and release notes @hakman #9034
- Updating stable channel to use latest k8s versions @MoShitrit #9038
- Support S3 Virtual Hosted Style @johanneswuerbach #8787
- Feature spotinst controller v1.0.58 @acaire #9022
- Ubuntu focal: Use generic docker & container packages @justinsb #9044
- Move GCEServiceAccount into CloudConfig @johngmyers #9055
- Don't restrict GitHub Actions to master @johngmyers #9054
- Update service_account_token_volumes.md @rvirdiz #9050
- Don't install static utils for Flatcar @hakman #8978
- Validate more static pods on masters @johngmyers #9049
- util/pkg/vfs/fs.go: Add Unit Test for WriteTo @littleroad #8997
- Add more ca-central-1 AZs, add ap-northeast-3 AZs @johngmyers #9064
- Update docs on supported distros and release notes @hakman #9059
- Release notes for 1.15.3 @justinsb #9070
- Release notes for 1.16.2 @justinsb #9071