Skip to content

Commit

Permalink
chore: merge dev into master (#5557)
Browse files Browse the repository at this point in the history
Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: Artun Duman <[email protected]>
Co-authored-by: Artun Duman <[email protected]>
Co-authored-by: Cameron Meissner <[email protected]>
Co-authored-by: Sylvain Boily <[email protected]>
Co-authored-by: lilypan26 <[email protected]>
Co-authored-by: Lily Pan <[email protected]>
Co-authored-by: r2k1 <[email protected]>
Co-authored-by: beilei <[email protected]>
Co-authored-by: Ganeshkumar Ashokavardhanan <[email protected]>
Co-authored-by: aks-node <[email protected]>
Co-authored-by: Jason Jung <[email protected]>
Co-authored-by: wenhug <[email protected]>
Co-authored-by: Wen Huang <[email protected]>
Co-authored-by: Tim Wright <[email protected]>
Co-authored-by: Chou Hu <[email protected]>
Co-authored-by: Devinwong <[email protected]>
Co-authored-by: Junjie Zhang <[email protected]>
Co-authored-by: Alison <[email protected]>
Co-authored-by: Zachary <[email protected]>
Co-authored-by: Zachary Bailey <[email protected]>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Xinhe Li <[email protected]>
Co-authored-by: Henry Li <[email protected]>
Co-authored-by: Henry Li <[email protected]>
Co-authored-by: Mikołaj Umański <[email protected]>
Co-authored-by: Mikolaj Umanski <[email protected]>
Co-authored-by: aks-node-assistant <[email protected]>
Co-authored-by: anujmaheshwari1 <[email protected]>
Co-authored-by: anujmaheshwari1 <[email protected]>
Co-authored-by: aks-node-sig-release-assistant[bot] <190555641+aks-node-sig-release-assistant[bot]@users.noreply.github.com>
Co-authored-by: Behzad Mirkhanzadeh <[email protected]>
Co-authored-by: Ben Brady <[email protected]>
Co-authored-by: Anlan Du <[email protected]>
Co-authored-by: Andy Zhang <[email protected]>
Co-authored-by: John Payne <[email protected]>
Co-authored-by: Dallas Delaney <[email protected]>
Co-authored-by: Santhosh  Prabhu <[email protected]>
Co-authored-by: Zachary Bailey <[email protected]>
Co-authored-by: Nan Liu <[email protected]>
  • Loading branch information
1 parent 632c07d commit 79d2559
Show file tree
Hide file tree
Showing 796 changed files with 21,147 additions and 15,779 deletions.
5 changes: 3 additions & 2 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,10 @@
{
"name": "Go",
// Or use a Dockerfile or Docker Compose file. More info: https://containers.dev/guide/dockerfile
"image": "mcr.microsoft.com/devcontainers/go:1-1.20-bullseye",
"image": "mcr.microsoft.com/devcontainers/go:1-1.22-bullseye",
"features": {
"ghcr.io/devcontainers-contrib/features/protoc:1": {}
"ghcr.io/devcontainers-contrib/features/protoc:1": {},
"ghcr.io/devcontainers/features/azure-cli:1": {}
},

// Features to add to the dev container. More info: https://containers.dev/features.
Expand Down
50 changes: 46 additions & 4 deletions .github/README-RENOVATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
- [Okay, I just have 5 minutes. Please just tell me how to onboard a new package/container now to Renovate.json for auto-update.](#okay-i-just-have-5-minutes-please-just-tell-me-how-to-onboard-a-new-packagecontainer-now-to-renovatejson-for-auto-update)
- [What is the responsibility of a PR assignee?](#what-is-the-responsibility-of-a-pr-assignee)
- [What components are onboarded to Renovate for auto-update and what are not yet?](#what-components-are-onboarded-to-renovate-for-auto-update-and-what-are-not-yet)

- [Details on supporting the MAR OCI artifacts.](#details-on-supporting-the-mar-oci-artifacts)
# TL;DR
This readme is mainly describing how the renovate.json is constructed and the reasoning behind. If you are adding a new component to be cached in VHD, please refer to this [Readme-components](../parts/linux/cloud-init/artifacts/README-COMPONENTS.md) for tutorial. If you are onboarding a newly added component to Renovate automatic updates, you can jump to the [Hands-on guide and FAQ](#hands-on-guide-and-faq).

Expand Down Expand Up @@ -343,9 +343,51 @@ If your GitHub ID is placed in the `assignees` array, you are responsible for th
## What components are onboarded to Renovate for auto-update and what are not yet?
In general, if a component has the `"renovateTag": "<DO_NOT_UPDATE>"`, it means it's not monitored by Renovate and won't be updated automatically.

As of 9/18/2024,
As of 11/12/2024,
- All the container images are onboarded to Renovate for auto-update.
- PMC hosted packages, namely `runc` and `containerd`, are onboarded for auto-update.
- Acs-mirror hosted packages/binaries, namely `cni-plugins`, `azure-cni`, `cri-tools`, `kubernetes-binaries` and `azure-acr-credential-provider`, are NOT onboarded for auto-update yet. There are plans to move the acs-mirror hosted packages to MCR OCI which will be downloaded by Oras. We will wait for this transition to be completed to understand the details how to manage them.
- OCI artifacts hosted on MAR(aka MCR) such as `kubernetes-binaries`, `azure-acr-credential-provider` and `containerd-wasm-shims` are onboarded for auto-update.
- Acs-mirror hosted packages/binaries, namely `cni-plugins`, `azure-cni`, `cri-tools`, etc., are NOT onboarded for auto-update yet. There are plans to move the acs-mirror hosted packages to MCR OCI which will be downloaded by Oras. We will wait for this transition to be completed to understand the details how to manage them.

For the most up-to-date information, please refer to the actual configuration file `components.json`.

## Details on supporting the MAR OCI artifacts.
MAR OCI artifact is a bit special. The artifact is hosted/stored in a container registry (e.g. MCR, now rebranded to MAR), while it's not necessarily a container image. Instead it could be any format such as Helm charts, Software Bill of Materials (SBOM), a package or a tar/tgz file.
The `renovate.json` file is configured to support OCI artifact now. There is a packageRule like below to support auto updating OCI artifact, which is,
```
{
"matchDatasources": ["docker"],
"matchPackageNames": ["oss/binaries/kubernetes/kubernetes-node", "oss/binaries/kubernetes/azure-acr-credential-provider", "oss/binaries/deislabs/containerd-wasm-shims"],
"extractVersion": "^(?P<version>.*?)-[^-]*-[^-]*$"
},
```
Explanations as below.
1. The `datasource` should be `docker`.
2. The `packageName` should be one of those in the list.
3. In `extractVersion`, we use a regex to extract only part of the tag as the version to be stored in `latestVersion` in `components.json`.

Take `kubernetes-binaries` as an example. If you view all the tags from this list https://mcr.microsoft.com/v2/oss/binaries/kubernetes/kubernetes-node/tags/list?n=10000, you will notice that the format of the tags is quite varied, like, `v1.27.100-akslts-linux-amd64` , `v1.30.0-linux-amd64`, `v1.31.1-linux-arm64`. This regex is to capture only the values before the second-to-last dash (-). For example, if the tag is `v1.27.100-akslts-linux-amd64`, we capture `v1.27.100-akslts` as the version to be stored in `latestVersion` in `components.json`. If the tag is `v1.30.0-linux-amd64`, we capture `v1.30.0`. We do not capture the CPU architecture (amd64|arm64) to keep it generic, avoiding the need to define the same thing for both `amd64` and `arm64`.

For the most up-to-date information, please refer to the actual configuration file `components.json`.
3 packages in `components.json` are onboarded now: `oss/binaries/kubernetes/kubernetes-node`, `oss/binaries/kubernetes/azure-acr-credential-provider` and `oss/binaries/deislabs/containerd-wasm-shims`. You will see a new tag `OCI_registry` in `renovateTag`.

Continue using `kubernetes-binaries` as an example. Here is a block of version information defined as follows.
```
{
"k8sVersion": "1.31",
"renovateTag": "OCI_registry=https://mcr.microsoft.com, name=oss/binaries/kubernetes/kubernetes-node",
"latestVersion": "v1.31.2",
"previousLatestVersion": "v1.31.1"
}
```
where
1. `k8sVersion` is optional and specifies that it is tied to Kubernetes v1.31.
1. `renovateTag` defines the OCI registry and artifact name that Renovate should look up from its datasource.
1. `latestVersion` and `previousLatestVersion` define the versions to be cached as usual.

And next you will see
```
"downloadURL": "mcr.microsoft.com/oss/binaries/kubernetes/kubernetes-node:${version}-linux-${CPU_ARCH}"
```
where
- `${version}` will be resolved at runtime with the `latestVersion` and `previousLatestVersion` defined above.
- `${CPU_ARCH}` will be resolved at runtime depending on the CPU architecture of the Node (VM) under provisioning.
9 changes: 0 additions & 9 deletions .github/README.md

This file was deleted.

26 changes: 24 additions & 2 deletions .github/renovate.json
Original file line number Diff line number Diff line change
Expand Up @@ -73,8 +73,8 @@
"assignees": ["devinwong", "anujmaheshwari1", "cameronmeissner", "AlisonB319", "lilypan26", "djsly", "jason1028kr", "UtheMan", "zachary-bailey", "ganeshkumarashok"]
},
{
"matchPackageNames": ["azure-cni", "azure-cns", "containernetworking/azure-cni", "containernetworking/azure-cns"],
"assignees": ["rbtr", "behzad-mir", "QxBytes"]
"matchPackageNames": ["azure-cni", "azure-cns", "containernetworking/azure-cni", "containernetworking/azure-cns", "containernetworking/cilium/cilium"],
"assignees": ["rbtr", "behzad-mir", "QxBytes", "jpayne3506"]
},
{
"matchPackageNames": ["aks/aks-node-ca-watcher"],
Expand All @@ -84,10 +84,19 @@
"matchPackageNames": ["oss/kubernetes/coredns", "oss/v2/kubernetes/coredns"],
"assignees": ["SriHarsha001"]
},
{
"matchPackageNames": ["oss/binaries/kubernetes/azure-acr-credential-provider"],
"assignees": ["mainred"]
},
{
"matchPackageNames": ["moby-runc", "moby-containerd"],
"extractVersion": "^v?(?<version>.+)$"
},
{
"matchDatasources": ["docker"],
"matchPackageNames": ["oss/binaries/kubernetes/kubernetes-node", "oss/binaries/kubernetes/azure-acr-credential-provider", "oss/binaries/deislabs/containerd-wasm-shims"],
"extractVersion": "^(?P<version>.*?)-[^-]*-[^-]*$"
},
{
"matchPackageNames": ["aks/aks-gpu-cuda", "aks/aks-gpu-grid"],
"versioning": "regex:^(?<major>\\d+)\\.(?<minor>\\d+)\\.(?<patch>\\d+)-(?<prerelease>\\d{14})$",
Expand All @@ -110,6 +119,19 @@
"datasourceTemplate": "docker",
"autoReplaceStringTemplate": "\"renovateTag\": \"registry={{{registryUrl}}}, name={{{packageName}}}\",\n \"latestVersion\": \"{{{newValue}}}\"{{#if depType}},\n \"previousLatestVersion\": \"{{{currentValue}}}\"{{/if}}"
},
{
"customType": "regex",
"description": "auto update OCI artifacts in components.json",
"fileMatch": [
"parts/linux/cloud-init/artifacts/components.json"
],
"matchStringsStrategy": "any",
"matchStrings": [
"\"renovateTag\":\\s*\"OCI_registry=(?<registryUrl>[^,]+), name=(?<packageName>[^\"]+)\",\\s*\"latestVersion\":\\s*\"(?<currentValue>[^\"]+)\"(?:[^}]*\"previousLatestVersion\":\\s*\"(?<depType>[^\"]+)\")?"
],
"datasourceTemplate": "docker",
"autoReplaceStringTemplate": "\"renovateTag\": \"OCI_registry={{{registryUrl}}}, name={{{packageName}}}\",\n \"latestVersion\": \"{{{newValue}}}\"{{#if depType}},\n \"previousLatestVersion\": \"{{{currentValue}}}\"{{/if}}"
},
{
"customType": "regex",
"description": "auto update packages for OS ubuntu 18.04 in components.json",
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/auto-update.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ on:
jobs:
Auto:
name: Auto-update
runs-on: ubuntu-latest
runs-on: ubuntu-24.04
steps:
- uses: tibdex/auto-update@v2
with:
Expand Down
24 changes: 24 additions & 0 deletions .github/workflows/buf.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
name: Buf CI
on:
push:
paths:
- "aks-node-controller/proto/**"
- "aks-node-controller/buf.yaml"
- ".github/workflows/buf.yaml"
pull_request:
types: [opened, synchronize, reopened, labeled, unlabeled]
paths:
- "aks-node-controller/proto/**"
- "aks-node-controller/buf.yaml"
- ".github/workflows/buf.yaml"
permissions:
contents: read
pull-requests: write
jobs:
buf:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- uses: bufbuild/buf-action@v1
with:
input: aks-node-controller
2 changes: 1 addition & 1 deletion .github/workflows/cflite_batch.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ permissions: read-all

jobs:
BatchFuzzing:
runs-on: ubuntu-latest
runs-on: ubuntu-24.04
strategy:
fail-fast: false
matrix:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/cflite_build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ on:
permissions: read-all
jobs:
Build:
runs-on: ubuntu-latest
runs-on: ubuntu-24.04
concurrency:
group: ${{ github.workflow }}-${{ matrix.sanitizer }}-${{ github.ref }}
cancel-in-progress: true
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/cflite_prune.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ permissions: read-all

jobs:
Pruning:
runs-on: ubuntu-latest
runs-on: ubuntu-24.04
steps:
- name: Build Fuzzers
id: build
Expand All @@ -34,7 +34,7 @@ jobs:
storage-repo-branch: main # Optional. Defaults to "main"
storage-repo-branch-coverage: gh-pages # Optional. Defaults to "gh-pages".
Coverage:
runs-on: ubuntu-latest
runs-on: ubuntu-24.04
steps:
- name: Build Fuzzers
id: build
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/check-coverage.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ permissions:

jobs:
unit_tests:
runs-on: ubuntu-latest
runs-on: ubuntu-24.04
steps:
- name: Install Go
if: success()
Expand Down Expand Up @@ -36,7 +36,7 @@ jobs:
finish:
needs: [unit_tests]
if: ${{ success() }}
runs-on: ubuntu-latest
runs-on: ubuntu-24.04
steps:
- name: Coveralls Finished
uses: coverallsapp/github-action@v2
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/codeql-analysis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ on:
jobs:
analyze:
name: Analyze
runs-on: ubuntu-latest
runs-on: ubuntu-24.04
permissions:
actions: read
contents: read
Expand All @@ -48,7 +48,7 @@ jobs:

# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v2
uses: github/codeql-action/init@v3
with:
languages: ${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a config file.
Expand All @@ -62,7 +62,7 @@ jobs:
# Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
# If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@v2
uses: github/codeql-action/autobuild@v3

# ℹ️ Command-line programs to run using the OS shell.
# 📚 See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun
Expand All @@ -75,4 +75,4 @@ jobs:
# ./location_of_script_within_repo/buildscript.sh

- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v2
uses: github/codeql-action/analyze@v3
2 changes: 1 addition & 1 deletion .github/workflows/generate-kubelet-flags.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ on:

jobs:
generate-kubelet-flags:
runs-on: ubuntu-latest
runs-on: ubuntu-24.04
steps:
- name: Set up containerd
uses: crazy-max/ghaction-setup-containerd@v2
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/go-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ on: pull_request

jobs:
go-test:
runs-on: ubuntu-latest
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v3
Expand Down
45 changes: 0 additions & 45 deletions .github/workflows/golangci-lint-pr.yml

This file was deleted.

6 changes: 5 additions & 1 deletion .github/workflows/golangci-lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@ on:
branches:
- master
- main
pull_request:
branches:
- master
- main

permissions:
contents: read
Expand All @@ -13,7 +17,7 @@ permissions:
jobs:
golangci:
name: lint
runs-on: ubuntu-latest
runs-on: ubuntu-24.04
steps:
- uses: actions/setup-go@v3
with:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/shellcheck.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ on: pull_request

jobs:
shellcheck:
runs-on: ubuntu-latest
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v3
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/shellspec.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ on: pull_request

jobs:
shellspec:
runs-on: ubuntu-latest
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v3
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/validate-components.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ on: pull_request

jobs:
cue:
runs-on: ubuntu-latest
runs-on: ubuntu-24.04
environment: test
steps:
- uses: actions/checkout@v4
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/validate-image-version.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ on: pull_request

jobs:
validate-image-version:
runs-on: ubuntu-latest
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- run: |
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/validate-pull-request-source.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ on: pull_request

jobs:
validate-pull-request-source:
runs-on: ubuntu-latest
runs-on: ubuntu-24.04
steps:
- name: Validate if PR is not from a forked repo
run: |
Expand Down
Loading

0 comments on commit 79d2559

Please sign in to comment.