This repository has been archived by the owner on Oct 24, 2023. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 521
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
ci: refactor Windows VHD build pipeline updates (#4105)
* chore: updating windows provisioing scripts to v0.0.10 Signed-off-by: marosset <[email protected]> * ci: refactor Windows VHD build pipeline to use templates and build multiple VHDs in parallel * fixing stage names in vhd-builder-windows.yaml * fixing --vhd-uri arg in final pub command * incresing timeout for windows vhd build task since contianerD images take longer to build * fixup * adding contianer-runtime to packer sa name * adding 13783 to storage account name * removing extra whitespace in configure-windows-vhd.ps1 * removing scripts replaced by new-sku-and-add-image-version.sh * specify container runtime when running e2e tests against newly built VHD * use v1.20 for e2e test clusters * updating windows containerd package used by CI * commenting out containerd VHD build for PR jobs * re-enabling contianerd VHD for CI builds * PR feedback
- Loading branch information
Showing
10 changed files
with
297 additions
and
259 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| trigger: none | ||
| pr: none | ||
|
|
||
| stages: | ||
|
|
||
| - stage: build_2019_containerd | ||
| dependsOn: [] | ||
| jobs: | ||
| - template: vhd-builder-windows-template.yaml | ||
| parameters: | ||
| containerRuntime: containerd | ||
| windowsServerVersion: 2019 | ||
|
|
||
| - stage: build_2019_docker | ||
| dependsOn: [] | ||
| jobs: | ||
| - template: vhd-builder-windows-template.yaml | ||
| parameters: | ||
| containerRuntime: docker | ||
| windowsServerVersion: 2019 | ||
|
|
||
| - stage: publish_2019_containerd | ||
| dependsOn: build_2019_containerd | ||
| jobs: | ||
| - template: vhd-publish-template.yaml | ||
| parameters: | ||
| containerRuntime: containerd | ||
| skuPrefix: '2019-datacenter-core-ctrd' | ||
| windowsServerVersion: 2019 | ||
|
|
||
| - stage: publish_2019_docker | ||
| dependsOn: build_2019_docker | ||
| jobs: | ||
| - template: vhd-publish-template.yaml | ||
| parameters: | ||
| containerRuntime: docker | ||
| skuPrefix: '2019-datacenter-core-smalldisk' | ||
| windowsServerVersion: 2019 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,123 @@ | ||
| parameters: | ||
| containerRuntime: '' | ||
| windowsServerVersion: '' | ||
|
|
||
| jobs: | ||
| - job: build_vhd_windows_${{ parameters.windowsServerVersion }}_${{ parameters.containerRuntime }} | ||
| timeoutInMinutes: 210 | ||
| strategy: | ||
| maxParallel: 0 | ||
| pool: | ||
| name: $(BUILD_POOL) | ||
| steps: | ||
| - script: | | ||
| docker run --rm \ | ||
| -v ${PWD}:/go/src/github.com/Azure/aks-engine \ | ||
| -w /go/src/github.com/Azure/aks-engine \ | ||
| -e CLIENT_ID=${CLIENT_ID} \ | ||
| -e CLIENT_SECRET="$(CLIENT_SECRET)" \ | ||
| -e TENANT_ID=${TENANT_ID} \ | ||
| -e AZURE_RESOURCE_GROUP_NAME=${AZURE_RESOURCE_GROUP_NAME} \ | ||
| -e AZURE_LOCATION=${AZURE_LOCATION} \ | ||
| -e AZURE_VM_SIZE=${PACKER_VM_SIZE} \ | ||
| -e GIT_BRANCH=$(Build.SourceBranchName) \ | ||
| -e GIT_REPO=$(Build.Repository.Uri) \ | ||
| -e GIT_VERSION=$(Build.SourceVersion) \ | ||
| -e BUILD_ID=$(Build.BuildId) \ | ||
| -e BUILD_NUMBER=$(Build.BuildNumber) \ | ||
| -e CONTAINER_RUNTIME=${{ parameters.containerRuntime }} \ | ||
| -e WINDOWS_SERVER_VERSION=${{ parameters.windowsServerVersion }} \ | ||
| ${DEIS_GO_DEV_IMAGE} make run-packer-windows | ||
| displayName: Building windows VHD | ||
| - script: | | ||
| docker run --rm \ | ||
| -v ${PWD}:/go/src/github.com/Azure/aks-engine \ | ||
| -w /go/src/github.com/Azure/aks-engine \ | ||
| ${DEIS_GO_DEV_IMAGE} make all | ||
| displayName: build aks-engine-test - make all | ||
| condition: and(succeeded(), eq(variables.COPY_VHD, 'False')) | ||
| - script: | | ||
| OS_DISK_URI="$(cat packer-output | grep "OSDiskUri:" | cut -d " " -f 2)" && \ | ||
| IMAGE_NAME="$(echo $OS_DISK_URI | cut -d "/" -f 3 | cut -d "." -f 1)" && \ | ||
| docker run --rm \ | ||
| -v ${PWD}:/go/src/github.com/Azure/aks-engine \ | ||
| -w /go/src/github.com/Azure/aks-engine \ | ||
| -e CLEANUP_ON_EXIT="true" \ | ||
| -e CLIENT_ID=${CLIENT_ID} \ | ||
| -e CLIENT_SECRET="$(CLIENT_SECRET)" \ | ||
| -e CLUSTER_DEFINITION="examples/e2e-tests/kubernetes/windows/hybrid/definition.json" \ | ||
| -e LOCATION=${AZURE_LOCATION} \ | ||
| -e ORCHESTRATOR_RELEASE="1.20" \ | ||
| -e SUBSCRIPTION_ID=${SUBSCRIPTION_ID} \ | ||
| -e TENANT_ID=${TENANT_ID} \ | ||
| -e WINDOWS_NODE_VHD_URL=${OS_DISK_URI} \ | ||
| -e USE_MANAGED_IDENTITY="false" \ | ||
| -e CONTAINER_RUNTIME=${{ parameters.containerRuntime }} \ | ||
| ${DEIS_GO_DEV_IMAGE} make test-kubernetes | ||
| displayName: run e2e tests | ||
| condition: and(succeeded(), eq(variables.COPY_VHD, 'False')) | ||
| - task: PublishPipelineArtifact@1 | ||
| inputs: | ||
| artifact: '${{ parameters.windowsServerVersion }}-${{ parameters.containerRuntime }}-vhd-release-notes' | ||
| path: 'release-notes.txt' | ||
|
|
||
| - script: | | ||
| OS_DISK_SAS="$(cat packer-output | grep "OSDiskUriReadOnlySas:" | cut -d " " -f 2)" && \ | ||
| VHD_NAME="$(echo $OS_DISK_SAS | cut -d "/" -f 8 | cut -d "?" -f 1)" && \ | ||
| docker run --rm \ | ||
| -v ${PWD}:/go/src/github.com/Azure/aks-engine \ | ||
| -w /go/src/github.com/Azure/aks-engine \ | ||
| -e CLIENT_ID=${CLIENT_ID} \ | ||
| -e CLIENT_SECRET="$(CLIENT_SECRET)" \ | ||
| -e TENANT_ID=${TENANT_ID} \ | ||
| -e CLASSIC_BLOB=${STORAGE_ACCT_BLOB_URL} \ | ||
| -e CLASSIC_SAS_TOKEN="$(STORAGE_ACCT_SAS_TOKEN)" \ | ||
| -e OS_DISK_SAS=${OS_DISK_SAS} \ | ||
| -e VHD_NAME=${VHD_NAME} \ | ||
| ${DEIS_GO_DEV_IMAGE} make az-copy | ||
| displayName: Copying resource to Classic Storage Account | ||
| condition: and(succeeded(), eq(variables.COPY_VHD, 'True')) | ||
| - script: | | ||
| OS_DISK_SAS="$(cat packer-output | grep "OSDiskUriReadOnlySas:" | cut -d " " -f 2)" && \ | ||
| VHD_NAME="$(echo $OS_DISK_SAS | cut -d "/" -f 8 | cut -d "?" -f 1)" && \ | ||
| docker run --rm \ | ||
| -v ${PWD}:/go/src/github.com/Azure/aks-engine \ | ||
| -w /go/src/github.com/Azure/aks-engine \ | ||
| -e CLIENT_ID=${CLIENT_ID} \ | ||
| -e CLIENT_SECRET="$(CLIENT_SECRET)" \ | ||
| -e TENANT_ID=${TENANT_ID} \ | ||
| -e CLASSIC_SA_CONNECTION_STRING="$(CLASSIC_SA_CONNECTION_STRING)" \ | ||
| -e STORAGE_ACCT_BLOB_URL=${STORAGE_ACCT_BLOB_URL} \ | ||
| -e VHD_NAME=${VHD_NAME} \ | ||
| ${DEIS_GO_DEV_IMAGE} make windows-vhd-publishing-info | ||
| displayName: Generating publishing info for Windows VHD | ||
| condition: and(succeeded(), eq(variables.COPY_VHD, 'True')) | ||
| - task: PublishPipelineArtifact@1 | ||
| inputs: | ||
| artifact: 'publishing_info_${{ parameters.windowsServerVersion }}_${{ parameters.containerRuntime }}' | ||
| path: 'windows-vhd-publishing-info.json' | ||
| condition: and(succeeded(), eq(variables.COPY_VHD, 'True')) | ||
|
|
||
| - script: | | ||
| SA_NAME="$(cat packer-output | grep "storage name:" | cut -d " " -f 3)" && \ | ||
| docker run --rm \ | ||
| -v ${PWD}:/go/src/github.com/Azure/aks-engine \ | ||
| -w /go/src/github.com/Azure/aks-engine \ | ||
| -e CLIENT_ID=${CLIENT_ID} \ | ||
| -e CLIENT_SECRET="$(CLIENT_SECRET)" \ | ||
| -e TENANT_ID=${TENANT_ID} \ | ||
| -e SA_NAME=${SA_NAME} \ | ||
| -e AZURE_RESOURCE_GROUP_NAME=${AZURE_RESOURCE_GROUP_NAME} \ | ||
| ${DEIS_GO_DEV_IMAGE} make delete-sa | ||
| displayName: cleanup - delete temp storage account | ||
| condition: always() | ||
| - script: | | ||
| sudo chown -R $USER:$USER . | ||
| displayName: cleanup - chown all files in work directory | ||
| condition: always() |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,44 @@ | ||
| parameters: | ||
| containerRuntime: '' | ||
| skuPrefix: '' | ||
| windowsServerVersion: '' | ||
|
|
||
| jobs: | ||
| - job: add_image_to_sku_${{ parameters.windowsServerVersion }}_${{ parameters.containerRuntime }} | ||
| pool: | ||
| name: $(BUILD_POOL) | ||
| steps: | ||
| - task: DownloadPipelineArtifact@2 | ||
| inputs: | ||
| artifact: 'publishing_info_${{ parameters.windowsServerVersion }}_${{ parameters.containerRuntime }}' | ||
| path: 'artifacts/vhd' | ||
| - script: | | ||
| tree artifacts | ||
| displayName: tree | ||
| - script: | | ||
| docker run --rm \ | ||
| -v ${PWD}:/go/src/github.com/Azure/aks-engine \ | ||
| -w /go/src/github.com/Azure/aks-engine \ | ||
| ${DEIS_GO_DEV_IMAGE} make bootstrap | ||
| displayName: Update tools | ||
| - script: | | ||
| docker run --rm \ | ||
| -v ${PWD}:/go/src/github.com/Azure/aks-engine \ | ||
| -w /go/src/github.com/Azure/aks-engine \ | ||
| -e AZURE_TENANT_ID=${PUB_SP_TENANT_ID} \ | ||
| -e AZURE_CLIENT_ID=${PUB_SP_CLIENT_ID} \ | ||
| -e AZURE_CLIENT_SECRET="$(PUB_SP_CLIENT_SECRET)" \ | ||
| -e SKU_PREFIX=${{ parameters.skuPrefix }} \ | ||
| -e SKU_TEMPLATE_FILE=${SKU_TEMPLATE_FILE} \ | ||
| -e PUBLISHER=${PUBLISHER} \ | ||
| -e OFFER=${OFFER} \ | ||
| -e CONTAINER_RUNTIME=${{ parameters.containerRuntime }} \ | ||
| ${DEIS_GO_DEV_IMAGE} ./vhd/publishing/new-sku-and-add-image-version.sh | ||
| displayName: Publish vhd | ||
| - script: | | ||
| docker run --rm \ | ||
| -v ${PWD}:/go/src/github.com/Azure/aks-engine \ | ||
| -w /go/src/github.com/Azure/aks-engine \ | ||
| ${DEIS_GO_DEV_IMAGE} make tools-clean | ||
| displayName: cleanup - clean hack/tools/bin | ||
| condition: always() |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.