Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ci] Try to use GCP Bazel cache for more jobs #20836

Merged
merged 4 commits into from
Jan 17, 2024
Merged

[ci] Try to use GCP Bazel cache for more jobs #20836

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
2aa0bb3
[ci] Use ci/bazelisk.sh script in CI workflow
jwnrt Jan 15, 2024
6c251f3
[ci] Add Bazel cache GCP key to more jobs
jwnrt Jan 15, 2024
6bc6e82
[ci] Fix ci/bazelisk.sh printing to stdout
jwnrt Jan 15, 2024
b92c555
[bazel] Allow ./bazelisk.sh to use --bazelrc flags
jwnrt Jan 17, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 13 additions & 47 deletions azure-pipelines.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ jobs:
- publish: $(Pipeline.Workspace)/opentitan-repo.tar.gz
artifact: opentitan-repo
displayName: Upload repository

- job: lint
displayName: Quality (quick lint)
# Run code quality checks (quick lint)
Expand Down Expand Up @@ -179,23 +180,13 @@ jobs:
dependsOn: lint
condition: and(succeeded(), eq(dependencies.lint.outputs['DetermineBuildType.onlyDocChanges'], '0'), eq(dependencies.lint.outputs['DetermineBuildType.onlyCdcChanges'], '0'))
pool: ci-public
variables:
- name: bazelCacheGcpKeyPath
value: ''
steps:
- template: ci/checkout-template.yml
- template: ci/install-package-dependencies.yml
- task: DownloadSecureFile@1
condition: eq(variables['Build.SourceBranchName'], 'master')
name: bazelCacheGcpKey
inputs:
secureFile: "bazel_cache_gcp_key.json"
# Set the remote cache GCP key path
- bash: echo "##vso[task.setvariable variable=bazelCacheGcpKeyPath]$(bazelCacheGcpKey.secureFilePath)"
condition: eq(variables['Build.SourceBranchName'], 'master')
displayName: GCP key path
- template: ci/load-bazel-cache-write-creds.yml
- bash: |
set -x -e

# Check the entire build graph for conflicts in loading or analysis
# phases. For context, see issue #18726.
# First, test with an empty bitstream cache entry.
Expand All @@ -214,7 +205,6 @@ jobs:
# shallow exclusion; tests deeper under //hw will still be found.
# * It excludes targets that depend on bitstream_splice rules, since the
# environment does not have access to Vivado.
export GCP_BAZEL_CACHE_KEY=$(bazelCacheGcpKeyPath)
TARGET_PATTERN_FILE=target_pattern.txt
echo //... > "${TARGET_PATTERN_FILE}"
echo -//quality/... >> "${TARGET_PATTERN_FILE}"
Expand Down Expand Up @@ -260,21 +250,10 @@ jobs:
timeoutInMinutes: 120
dependsOn: sw_build
pool: ci-public
variables:
- name: bazelCacheGcpKeyPath
value: ''
steps:
- template: ci/checkout-template.yml
- template: ci/install-package-dependencies.yml
- task: DownloadSecureFile@1
condition: eq(variables['Build.SourceBranchName'], 'master')
name: bazelCacheGcpKey
inputs:
secureFile: "bazel_cache_gcp_key.json"
# Set the remote cache GCP key path
- bash: echo "##vso[task.setvariable variable=bazelCacheGcpKeyPath]$(bazelCacheGcpKey.secureFilePath)"
condition: eq(variables['Build.SourceBranchName'], 'master')
displayName: GCP key path
- template: ci/load-bazel-cache-write-creds.yml
- download: current
artifact: target_pattern_file
- bash: |
Expand Down Expand Up @@ -317,24 +296,12 @@ jobs:
pool: ci-public
timeoutInMinutes: 240
dependsOn: lint
variables:
- name: bazelCacheGcpKeyPath
value: ''
steps:
- template: ci/checkout-template.yml
- template: ci/install-package-dependencies.yml
- task: DownloadSecureFile@1
condition: eq(variables['Build.SourceBranchName'], 'master')
name: bazelCacheGcpKey
inputs:
secureFile: "bazel_cache_gcp_key.json"
- bash: echo "##vso[task.setvariable variable=bazelCacheGcpKeyPath]$(bazelCacheGcpKey.secureFilePath)"
condition: eq(variables['Build.SourceBranchName'], 'master')
displayName: GCP key path
# Set the remote cache GCP key path
- template: ci/load-bazel-cache-write-creds.yml
- bash: |
set -x -e
export GCP_BAZEL_CACHE_KEY=$(bazelCacheGcpKeyPath)
ci/scripts/run-verilator-tests.sh
displayName: Build & execute tests
- template: ci/publish-bazel-test-results.yml
Expand Down Expand Up @@ -374,6 +341,7 @@ jobs:
parameters:
downloadPartialBuildBinFrom:
- chip_englishbreakfast_verilator
- template: ci/load-bazel-cache-write-creds.yml
- bash: |
. util/build_consts.sh
ci/scripts/run-english-breakfast-verilator-tests.sh
Expand Down Expand Up @@ -435,15 +403,7 @@ jobs:
steps:
- template: ci/checkout-template.yml
- template: ci/install-package-dependencies.yml
- task: DownloadSecureFile@1
condition: eq(variables['Build.SourceBranchName'], 'master')
name: bazelCacheGcpKey
inputs:
secureFile: "bazel_cache_gcp_key.json"
- bash: echo "##vso[task.setvariable variable=bazelCacheGcpKeyPath]$(bazelCacheGcpKey.secureFilePath)"
condition: eq(variables['Build.SourceBranchName'], 'master')
displayName: GCP key path
# Set the remote cache GCP key path
- template: ci/load-bazel-cache-write-creds.yml
- bash: |
ci/bazelisk.sh test --test_tag_filters=-nightly //sw/otbn/crypto/...
displayName: Execute tests
Expand Down Expand Up @@ -560,6 +520,7 @@ jobs:
downloadPartialBuildBinFrom:
- chip_earlgrey_cw310
- sw_build
- template: ci/load-bazel-cache-write-creds.yml
- bash: |
set -e
. util/build_consts.sh
Expand All @@ -586,6 +547,7 @@ jobs:
downloadPartialBuildBinFrom:
- chip_earlgrey_cw310
- sw_build
- template: ci/load-bazel-cache-write-creds.yml
- bash: |
set -e
. util/build_consts.sh
Expand Down Expand Up @@ -613,6 +575,7 @@ jobs:
- chip_earlgrey_cw310
- chip_earlgrey_cw310_hyperdebug
- sw_build
- template: ci/load-bazel-cache-write-creds.yml
- bash: |
set -e
. util/build_consts.sh
Expand All @@ -639,6 +602,7 @@ jobs:
downloadPartialBuildBinFrom:
- chip_earlgrey_cw310_hyperdebug
- sw_build
- template: ci/load-bazel-cache-write-creds.yml
# We run the update command twice to workaround an issue with udev on the container.
# Where rusb cannot dynamically update its device list in CI (udev is not completely
# functional). If the device is in normal mode, the first thing that opentitantool
Expand Down Expand Up @@ -678,6 +642,7 @@ jobs:
downloadPartialBuildBinFrom:
- chip_earlgrey_cw340
- sw_build
- template: ci/load-bazel-cache-write-creds.yml
- bash: |
set -e
. util/build_consts.sh
Expand All @@ -704,6 +669,7 @@ jobs:
downloadPartialBuildBinFrom:
- chip_earlgrey_cw310
- sw_build
- template: ci/load-bazel-cache-write-creds.yml
- bash: |
set -e
. util/build_consts.sh
Expand Down
5 changes: 3 additions & 2 deletions ci/bazelisk.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,13 @@ echo "Running bazelisk in $(pwd)."

# An additional bazelrc must be synthesized to specify precisely how to use the
# GCP bazel cache.
GCP_CREDS_FILE="$GCP_BAZEL_CACHE_KEY_SECUREFILEPATH"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am assuming that the _SECUREFILEPATH is added by DownloadSecureFile to the name? It's not really clear in the documentation https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/download-secure-file-v1?view=azure-pipelines

Copy link
Contributor Author

@jwnrt jwnrt Jan 17, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DownloadSecureFile will store the path to the file in a variable called $(GCP_BAZEL_CACHE_KEY.secureFilePath), and Azure also has a rule that you can access $(AZURE_VARIABLES.WITH.SCOPES) variables through environment variables by replacing .s with _s

GCP_BAZELRC="$(mktemp /tmp/XXXXXX.bazelrc)"
trap 'rm ${GCP_BAZELRC}' EXIT

if [[ -n "${GCP_BAZEL_CACHE_KEY}" && -f "${GCP_BAZEL_CACHE_KEY}" ]]; then
if [[ -n "$GCP_CREDS_FILE" && -f "$GCP_CREDS_FILE" ]]; then
echo "Applying GCP cache key; will upload to the cache."
echo "build --google_credentials=${GCP_BAZEL_CACHE_KEY}" >> "${GCP_BAZELRC}"
echo "build --google_credentials=${GCP_CREDS_FILE}" >> "${GCP_BAZELRC}"
else
echo "No key/invalid path to key. Download from cache only."
echo "build --remote_upload_local_results=false" >> "${GCP_BAZELRC}"
Expand Down
17 changes: 17 additions & 0 deletions ci/load-bazel-cache-write-creds.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Copyright lowRISC contributors.
# Licensed under the Apache License, Version 2.0, see LICENSE for details.
# SPDX-License-Identifier: Apache-2.0

# Azure Pipelines template for downloading a "secure file" containing the write
# credentials for a GCP bucket where we store a Bazel cache.
#
# The path to the downloaded file is automatically stored in an environment
# variable `GCP_BAZEL_CACHE_KEY_SECUREFILEPATH`. This file is loaded by the
# `ci/bazelisk.sh` script.

steps:
- task: DownloadSecureFile@1
condition: eq(variables['Build.SourceBranchName'], 'master')
name: GCP_BAZEL_CACHE_KEY
inputs:
secureFile: "bazel_cache_gcp_key.json"