Azure · cameronmeissner · Oct 30, 2024 · Oct 30, 2024 · Oct 31, 2024 · Oct 31, 2024
@@ -3,9 +3,10 @@
 {
 	"name": "Go",
 	// Or use a Dockerfile or Docker Compose file. More info: https://containers.dev/guide/dockerfile
-	"image": "mcr.microsoft.com/devcontainers/go:1-1.20-bullseye",
+	"image": "mcr.microsoft.com/devcontainers/go:1-1.22-bullseye",
 	"features": {
-		"ghcr.io/devcontainers-contrib/features/protoc:1": {}
+		"ghcr.io/devcontainers-contrib/features/protoc:1": {},
+		"ghcr.io/devcontainers/features/azure-cli:1": {}
 	},
 
 	// Features to add to the dev container. More info: https://containers.dev/features.

@@ -20,7 +20,7 @@
   - [Okay, I just have 5 minutes. Please just tell me how to onboard a new package/container now to Renovate.json for auto-update.](#okay-i-just-have-5-minutes-please-just-tell-me-how-to-onboard-a-new-packagecontainer-now-to-renovatejson-for-auto-update)
   - [What is the responsibility of a PR assignee?](#what-is-the-responsibility-of-a-pr-assignee)
   - [What components are onboarded to Renovate for auto-update and what are not yet?](#what-components-are-onboarded-to-renovate-for-auto-update-and-what-are-not-yet)
-
+  - [Details on supporting the MAR OCI artifacts.](#details-on-supporting-the-mar-oci-artifacts)
 # TL;DR
 This readme is mainly describing how the renovate.json is constructed and the reasoning behind. If you are adding a new component to be cached in VHD, please refer to this [Readme-components](../parts/linux/cloud-init/artifacts/README-COMPONENTS.md) for tutorial. If you are onboarding a newly added component to Renovate automatic updates, you can jump to the [Hands-on guide and FAQ](#hands-on-guide-and-faq).
 
@@ -343,9 +343,51 @@ If your GitHub ID is placed in the `assignees` array, you are responsible for th
 ## What components are onboarded to Renovate for auto-update and what are not yet?
 In general, if a component has the `"renovateTag": "<DO_NOT_UPDATE>"`, it means it's not monitored by Renovate and won't be updated automatically.
 
-As of 9/18/2024,
+As of 11/12/2024,
 - All the container images are onboarded to Renovate for auto-update.
 - PMC hosted packages, namely `runc` and `containerd`, are onboarded for auto-update.
-- Acs-mirror hosted packages/binaries, namely `cni-plugins`, `azure-cni`, `cri-tools`, `kubernetes-binaries` and `azure-acr-credential-provider`, are NOT onboarded for auto-update yet. There are plans to move the acs-mirror hosted packages to MCR OCI which will be downloaded by Oras. We will wait for this transition to be completed to understand the details how to manage them.
+- OCI artifacts hosted on MAR(aka MCR) such as `kubernetes-binaries`, `azure-acr-credential-provider` and `containerd-wasm-shims` are onboarded for auto-update.
+- Acs-mirror hosted packages/binaries, namely `cni-plugins`, `azure-cni`, `cri-tools`, etc., are NOT onboarded for auto-update yet. There are plans to move the acs-mirror hosted packages to MCR OCI which will be downloaded by Oras. We will wait for this transition to be completed to understand the details how to manage them.
+
+For the most up-to-date information, please refer to the actual configuration file `components.json`.
+
+## Details on supporting the MAR OCI artifacts.
+MAR OCI artifact is a bit special. The artifact is hosted/stored in a container registry (e.g. MCR, now rebranded to MAR), while it's not necessarily a container image. Instead it could be any format such as Helm charts, Software Bill of Materials (SBOM), a package or a tar/tgz file.
+The `renovate.json` file is configured to support OCI artifact now. There is a packageRule like below to support auto updating OCI artifact, which is,
+```
+    {
+      "matchDatasources": ["docker"],
+      "matchPackageNames": ["oss/binaries/kubernetes/kubernetes-node", "oss/binaries/kubernetes/azure-acr-credential-provider", "oss/binaries/deislabs/containerd-wasm-shims"],
+      "extractVersion": "^(?P<version>.*?)-[^-]*-[^-]*$"
+    },
+```
+Explanations as below.
+1. The `datasource` should be `docker`.
+2. The `packageName` should be one of those in the list.
+3. In `extractVersion`, we use a regex to extract only part of the tag as the version to be stored in `latestVersion` in `components.json`.
+
+Take `kubernetes-binaries` as an example. If you view all the tags from this list https://mcr.microsoft.com/v2/oss/binaries/kubernetes/kubernetes-node/tags/list?n=10000, you will notice that the format of the tags is quite varied, like, `v1.27.100-akslts-linux-amd64` , `v1.30.0-linux-amd64`, `v1.31.1-linux-arm64`. This regex is to capture only the values before the second-to-last dash (-). For example, if the tag is `v1.27.100-akslts-linux-amd64`, we capture `v1.27.100-akslts` as the version to be stored in `latestVersion` in `components.json`. If the tag is `v1.30.0-linux-amd64`, we capture `v1.30.0`. We do not capture the CPU architecture (amd64|arm64) to keep it generic, avoiding the need to define the same thing for both `amd64` and `arm64`. 
 
-For the most up-to-date information, please refer to the actual configuration file `components.json`.
+3 packages in `components.json` are onboarded now: `oss/binaries/kubernetes/kubernetes-node`, `oss/binaries/kubernetes/azure-acr-credential-provider` and `oss/binaries/deislabs/containerd-wasm-shims`. You will see a new tag `OCI_registry` in `renovateTag`. 
+
+Continue using `kubernetes-binaries` as an example. Here is a block of version information defined as follows.
+```
+{
+  "k8sVersion": "1.31",
+  "renovateTag": "OCI_registry=https://mcr.microsoft.com, name=oss/binaries/kubernetes/kubernetes-node",
+  "latestVersion": "v1.31.2",
+  "previousLatestVersion": "v1.31.1"
+}
+```
+where
+1. `k8sVersion` is optional and specifies that it is tied to Kubernetes  v1.31.
+1. `renovateTag` defines the OCI registry and artifact name that Renovate should look up from its datasource.
+1. `latestVersion` and `previousLatestVersion` define the versions to be cached as usual.
+
+And next you will see
+```
+"downloadURL": "mcr.microsoft.com/oss/binaries/kubernetes/kubernetes-node:${version}-linux-${CPU_ARCH}"
+```
+where 
+- `${version}` will be resolved at runtime with the `latestVersion` and `previousLatestVersion` defined above.
+- `${CPU_ARCH}` will be resolved at runtime depending on the CPU architecture of the Node (VM) under provisioning.
@@ -73,8 +73,8 @@
       "assignees": ["devinwong", "anujmaheshwari1", "cameronmeissner", "AlisonB319", "lilypan26", "djsly", "jason1028kr", "UtheMan", "zachary-bailey", "ganeshkumarashok"]
     },
     {
-      "matchPackageNames": ["azure-cni", "azure-cns", "containernetworking/azure-cni", "containernetworking/azure-cns"],
-      "assignees": ["rbtr", "behzad-mir", "QxBytes"]
+      "matchPackageNames": ["azure-cni", "azure-cns", "containernetworking/azure-cni", "containernetworking/azure-cns", "containernetworking/cilium/cilium"],
+      "assignees": ["rbtr", "behzad-mir", "QxBytes", "jpayne3506"]
     },
     {
       "matchPackageNames": ["aks/aks-node-ca-watcher"],
@@ -84,9 +84,25 @@
       "matchPackageNames": ["oss/kubernetes/coredns", "oss/v2/kubernetes/coredns"],
       "assignees": ["SriHarsha001"]
     },
+    {
+      "matchPackageNames": ["oss/binaries/kubernetes/azure-acr-credential-provider"],
+      "assignees": ["mainred"]
+    },
     {
       "matchPackageNames": ["moby-runc", "moby-containerd"],
       "extractVersion": "^v?(?<version>.+)$"
+    },
+    {
+      "matchDatasources": ["docker"],
+      "matchPackageNames": ["oss/binaries/kubernetes/kubernetes-node", "oss/binaries/kubernetes/azure-acr-credential-provider", "oss/binaries/deislabs/containerd-wasm-shims"],
+      "extractVersion": "^(?P<version>.*?)-[^-]*-[^-]*$"
+    },
+    {      
+      "matchPackageNames": ["aks/aks-gpu-cuda", "aks/aks-gpu-grid"],      
+      "versioning": "regex:^(?<major>\\d+)\\.(?<minor>\\d+)\\.(?<patch>\\d+)-(?<prerelease>\\d{14})$",
+      "automerge": false,      
+      "enabled": true,
+      "ignoreUnstable": false
     }
   ],
   "customManagers": [
@@ -103,6 +119,19 @@
       "datasourceTemplate": "docker",
       "autoReplaceStringTemplate": "\"renovateTag\": \"registry={{{registryUrl}}}, name={{{packageName}}}\",\n          \"latestVersion\": \"{{{newValue}}}\"{{#if depType}},\n          \"previousLatestVersion\": \"{{{currentValue}}}\"{{/if}}"
     },
+    {
+      "customType": "regex",
+      "description": "auto update OCI artifacts in components.json",
+      "fileMatch": [
+        "parts/linux/cloud-init/artifacts/components.json"
+      ],
+      "matchStringsStrategy": "any",
+      "matchStrings": [
+        "\"renovateTag\":\\s*\"OCI_registry=(?<registryUrl>[^,]+), name=(?<packageName>[^\"]+)\",\\s*\"latestVersion\":\\s*\"(?<currentValue>[^\"]+)\"(?:[^}]*\"previousLatestVersion\":\\s*\"(?<depType>[^\"]+)\")?"
+      ],
+      "datasourceTemplate": "docker",
+      "autoReplaceStringTemplate": "\"renovateTag\": \"OCI_registry={{{registryUrl}}}, name={{{packageName}}}\",\n                \"latestVersion\": \"{{{newValue}}}\"{{#if depType}},\n                \"previousLatestVersion\": \"{{{currentValue}}}\"{{/if}}"
+    },
     {
       "customType": "regex",
       "description": "auto update packages for OS ubuntu 18.04 in components.json",

@@ -9,7 +9,7 @@ on:
 jobs:
   Auto:
     name: Auto-update
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-24.04
     steps:
       - uses: tibdex/auto-update@v2
         with:

@@ -0,0 +1,24 @@
+name: Buf CI
+on:
+  push:
+    paths:
+      - "aks-node-controller/proto/**"
+      - "aks-node-controller/buf.yaml"
+      - ".github/workflows/buf.yaml"
+  pull_request:
+    types: [opened, synchronize, reopened, labeled, unlabeled]
+    paths:
+      - "aks-node-controller/proto/**"
+      - "aks-node-controller/buf.yaml"
+      - ".github/workflows/buf.yaml"
+permissions:
+  contents: read
+  pull-requests: write
+jobs:
+  buf:
+    runs-on: ubuntu-24.04
+    steps:
+      - uses: actions/checkout@v4
+      - uses: bufbuild/buf-action@v1
+        with:
+          input: aks-node-controller
@@ -12,7 +12,7 @@ permissions: read-all
 
 jobs:
   BatchFuzzing:
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-24.04
     strategy:
       fail-fast: false
       matrix:

@@ -7,7 +7,7 @@ on:
 permissions: read-all
 jobs:
   Build:
-   runs-on: ubuntu-latest
+   runs-on: ubuntu-24.04
    concurrency:
      group: ${{ github.workflow }}-${{ matrix.sanitizer }}-${{ github.ref }}
      cancel-in-progress: true

@@ -11,7 +11,7 @@ permissions: read-all
 
 jobs:
   Pruning:
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-24.04
     steps:
     - name: Build Fuzzers
       id: build
@@ -34,7 +34,7 @@ jobs:
         storage-repo-branch: main   # Optional. Defaults to "main"
         storage-repo-branch-coverage: gh-pages  # Optional. Defaults to "gh-pages".
   Coverage:
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-24.04
     steps:
     - name: Build Fuzzers
       id: build

@@ -6,16 +6,9 @@ permissions:
   id-token: write
   contents: read
 
-env:
-  SUBSCRIPTION_ID: "8ecadfc9-d1a3-4ea4-b844-0d9f87e4d7c8"
-  RESOURCE_GROUP_NAME: "agentbaker-e2e-tests"
-  LOCATION: "eastus"
-  CLUSTER_NAME: "agentbaker-e2e-test-cluster"
-  AZURE_TENANT_ID: "72f988bf-86f1-41af-91ab-2d7cd011db47"
-
 jobs:
   unit_tests:
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-24.04
     steps:
       - name: Install Go
         if: success()
@@ -43,7 +36,7 @@ jobs:
   finish:
     needs: [unit_tests]
     if: ${{ success() }}
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-24.04
     steps:
       - name: Coveralls Finished
         uses: coverallsapp/github-action@v2

@@ -29,7 +29,7 @@ on:
 jobs:
   analyze:
     name: Analyze
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-24.04
     permissions:
       actions: read
       contents: read
@@ -48,7 +48,7 @@ jobs:
 
     # Initializes the CodeQL tools for scanning.
     - name: Initialize CodeQL
-      uses: github/codeql-action/init@v2
+      uses: github/codeql-action/init@v3
       with:
         languages: ${{ matrix.language }}
         # If you wish to specify custom queries, you can do so here or in a config file.
@@ -62,7 +62,7 @@ jobs:
     # Autobuild attempts to build any compiled languages  (C/C++, C#, or Java).
     # If this step fails, then you should remove it and run the build manually (see below)
     - name: Autobuild
-      uses: github/codeql-action/autobuild@v2
+      uses: github/codeql-action/autobuild@v3
 
     # ℹ️ Command-line programs to run using the OS shell.
     # 📚 See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun
@@ -75,4 +75,4 @@ jobs:
     #   ./location_of_script_within_repo/buildscript.sh
 
     - name: Perform CodeQL Analysis
-      uses: github/codeql-action/analyze@v2
+      uses: github/codeql-action/analyze@v3
@@ -4,7 +4,7 @@ on:
 
 jobs:
   generate-kubelet-flags:
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-24.04
     steps:
     - name: Set up containerd
       uses: crazy-max/ghaction-setup-containerd@v2

@@ -3,7 +3,7 @@ on: pull_request
 
 jobs:
   go-test:
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-24.04
     steps:
     - uses: actions/checkout@v4
     - uses: actions/setup-go@v3

@@ -4,6 +4,10 @@ on:
     branches:
       - master
       - main
+  pull_request:
+    branches:
+      - master
+      - main
 
 permissions:
   contents: read
@@ -13,7 +17,7 @@ permissions:
 jobs:
   golangci:
     name: lint
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-24.04
     steps:
       - uses: actions/setup-go@v3
         with:

@@ -3,7 +3,7 @@ on: pull_request
 
 jobs:
   shellcheck:
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-24.04
     steps:
     - uses: actions/checkout@v4
     - uses: actions/setup-go@v3

@@ -3,7 +3,7 @@ on: pull_request
 
 jobs:
   shellspec:
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-24.04
     steps:
     - uses: actions/checkout@v4
     - uses: actions/setup-go@v3

@@ -3,7 +3,7 @@ on: pull_request
 
 jobs: 
   cue:
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-24.04
     environment: test
     steps:
       - uses: actions/checkout@v4

@@ -3,7 +3,7 @@ on: pull_request
 
 jobs:
   validate-image-version:
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-24.04
     steps:
     - uses: actions/checkout@v4
     - run: |