Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(core): azure ci workflow #23453

Merged
merged 2 commits into from
May 16, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 27 additions & 20 deletions docs/shared/monorepo-ci-azure.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
Below is an example of an Azure Pipelines setup building and testing only what is affected.

```yaml {% fileName="azure-pipelines.yml" %}
name: CI

trigger:
- main
pr:
Expand All @@ -11,7 +13,7 @@ pr:
variables:
CI: 'true'
${{ if eq(variables['Build.Reason'], 'PullRequest') }}:
NX_BRANCH: $(System.PullRequest.PullRequestId) # You can use $(System.PullRequest.PullRequestNumber if your pipeline is triggered by a PR from GitHub ONLY)
NX_BRANCH: $(System.PullRequest.PullRequestNumber)
TARGET_BRANCH: $[replace(variables['System.PullRequest.TargetBranch'],'refs/heads/','origin/')]
BASE_SHA: $(git merge-base $(TARGET_BRANCH) HEAD)
${{ if ne(variables['Build.Reason'], 'PullRequest') }}:
Expand All @@ -26,16 +28,11 @@ jobs:
steps:
- checkout: self
fetchDepth: 0

# Set Azure Devops CLI default settings
- bash: az devops configure --defaults organization=$(System.TeamFoundationCollectionUri) project=$(System.TeamProject)
displayName: 'Set default Azure DevOps organization and project'
# Get last successfull commit from Azure Devops CLI
- displayName: 'Get last successful commit SHA'
condition: ne(variables['Build.Reason'], 'PullRequest')
env:
AZURE_DEVOPS_EXT_PAT: $(System.AccessToken)
bash: |
- bash: |
LAST_SHA=$(az pipelines build list --branch $(Build.SourceBranchName) --definition-ids $(System.DefinitionId) --result succeeded --top 1 --query "[0].triggerInfo.\"ci.sourceSha\"")
if [ -z "$LAST_SHA" ]
then
Expand All @@ -44,18 +41,23 @@ jobs:
echo "Last successful commit SHA: $LAST_SHA"
echo "##vso[task.setvariable variable=BASE_SHA]$LAST_SHA"
fi
displayName: 'Get last successful commit SHA'
condition: ne(variables['Build.Reason'], 'PullRequest')
env:
AZURE_DEVOPS_EXT_PAT: $(System.AccessToken)

# This line enables distribution
# Connect your workspace on nx.app and uncomment this to enable task distribution.
# The "--stop-agents-after" is optional, but allows idle agents to shut down once the "e2e-ci" targets have been requested
# - script: npx nx-cloud start-ci-run --distribute-on="5 linux-medium-js" --stop-agents-after="e2e-ci"
- script: npm ci
# - script: yarn nx-cloud start-ci-run --distribute-on="5 linux-medium-js" --stop-agents-after="e2e-ci"

# Required for nx affected if we're on a branch
- script: yarn install --frozen-lockfile
- script: git branch --track main origin/main
condition: eq(variables['Build.Reason'], 'PullRequest')

- script: npx nx-cloud record -- nx format:check --base=$(BASE_SHA)
- script: npx nx affected --base=$(BASE_SHA) -t lint test build e2e-ci
# Prepend any command with "nx-cloud record --" to record its logs to Nx Cloud
# - script: yarn nx-cloud record -- echo Hello World
- script: yarn nx affected --base=$(BASE_SHA) --head=$(HEAD_SHA) lint test build
- script: yarn nx affected --base=$(BASE_SHA) --head=$(HEAD_SHA) --parallel 1 e2e-ci
```

## Get the Commit of the Last Successful Build
Expand All @@ -66,30 +68,35 @@ use [Azure Devops CLI](https://learn.microsoft.com/en-us/cli/azure/pipelines?vie
First, we configure Devops CLI

```yaml
# Set Azure Devops default settings
# Set Azure Devops CLI default settings
- bash: az devops configure --defaults organization=$(System.TeamFoundationCollectionUri) project=$(System.TeamProject)
displayName: 'Configure Azure DevOps organization and project'
displayName: 'Set default Azure DevOps organization and project'
```

Then we can query the pipelines API (providing the auth token)

```yaml
# Get last successfully commit infos from Azure Devops
# Get last successfull commit from Azure Devops CLI
- bash: |
LAST_SHA=$(az pipelines build list --branch $(Build.SourceBranchName) --definition-ids $(System.DefinitionId) --result succeeded --top 1 --query "[0].triggerInfo.\"ci.sourceSha\"")
echo "Last successful commit SHA: $LAST_SHA"
echo "##vso[task.setvariable variable=BASE_SHA]$LAST_SHA"
if [ -z "$LAST_SHA" ]
then
echo "Last successful commit not found. Using fallback 'HEAD~1': $BASE_SHA"
else
echo "Last successful commit SHA: $LAST_SHA"
echo "##vso[task.setvariable variable=BASE_SHA]$LAST_SHA"
fi
displayName: 'Get last successful commit SHA'
condition: ne(variables['Build.Reason'], 'PullRequest')
env:
AZURE_DEVOPS_EXT_PAT: $(System.AccessToken)
```

We can target a specific build; in this example, we specified:

- The branch (--branch)
- The pipeline ID (--definition-ids)
- The result type (--result)
- The number of the result (-top)
- The number of the result (--top)

The command returns an entire JSON object with all the information. But we can narrow it down to the desired result with the `--query` param that uses [JMESPath](https://jmespath.org/)
format ([more details](https://learn.microsoft.com/en-us/cli/azure/query-azure-cli?tabs=concepts%2Cbash))
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,11 +30,7 @@ jobs:
- bash: az devops configure --defaults organization=$(System.TeamFoundationCollectionUri) project=$(System.TeamProject)
displayName: 'Set default Azure DevOps organization and project'
# Get last successfull commit from Azure Devops CLI
- displayName: 'Get last successful commit SHA'
condition: ne(variables['Build.Reason'], 'PullRequest')
env:
AZURE_DEVOPS_EXT_PAT: $(System.AccessToken)
bash: |
- bash: |
LAST_SHA=$(az pipelines build list --branch $(Build.SourceBranchName) --definition-ids $(System.DefinitionId) --result succeeded --top 1 --query "[0].triggerInfo.\\"ci.sourceSha\\"")
if [ -z "$LAST_SHA" ]
then
Expand All @@ -43,6 +39,10 @@ jobs:
echo "Last successful commit SHA: $LAST_SHA"
echo "##vso[task.setvariable variable=BASE_SHA]$LAST_SHA"
fi
displayName: 'Get last successful commit SHA'
condition: ne(variables['Build.Reason'], 'PullRequest')
env:
AZURE_DEVOPS_EXT_PAT: $(System.AccessToken)

# Connect your workspace on nx.app and uncomment this to enable task distribution.
# The "--stop-agents-after" is optional, but allows idle agents to shut down once the "e2e-ci" targets have been requested
Expand Down Expand Up @@ -246,11 +246,7 @@ jobs:
- bash: az devops configure --defaults organization=$(System.TeamFoundationCollectionUri) project=$(System.TeamProject)
displayName: 'Set default Azure DevOps organization and project'
# Get last successfull commit from Azure Devops CLI
- displayName: 'Get last successful commit SHA'
condition: ne(variables['Build.Reason'], 'PullRequest')
env:
AZURE_DEVOPS_EXT_PAT: $(System.AccessToken)
bash: |
- bash: |
LAST_SHA=$(az pipelines build list --branch $(Build.SourceBranchName) --definition-ids $(System.DefinitionId) --result succeeded --top 1 --query "[0].triggerInfo.\\"ci.sourceSha\\"")
if [ -z "$LAST_SHA" ]
then
Expand All @@ -259,6 +255,10 @@ jobs:
echo "Last successful commit SHA: $LAST_SHA"
echo "##vso[task.setvariable variable=BASE_SHA]$LAST_SHA"
fi
displayName: 'Get last successful commit SHA'
condition: ne(variables['Build.Reason'], 'PullRequest')
env:
AZURE_DEVOPS_EXT_PAT: $(System.AccessToken)

# Connect your workspace on nx.app and uncomment this to enable task distribution.
# The "--stop-agents-after" is optional, but allows idle agents to shut down once the "build" targets have been requested
Expand Down Expand Up @@ -483,11 +483,7 @@ jobs:
- bash: az devops configure --defaults organization=$(System.TeamFoundationCollectionUri) project=$(System.TeamProject)
displayName: 'Set default Azure DevOps organization and project'
# Get last successfull commit from Azure Devops CLI
- displayName: 'Get last successful commit SHA'
condition: ne(variables['Build.Reason'], 'PullRequest')
env:
AZURE_DEVOPS_EXT_PAT: $(System.AccessToken)
bash: |
- bash: |
LAST_SHA=$(az pipelines build list --branch $(Build.SourceBranchName) --definition-ids $(System.DefinitionId) --result succeeded --top 1 --query "[0].triggerInfo.\\"ci.sourceSha\\"")
if [ -z "$LAST_SHA" ]
then
Expand All @@ -496,6 +492,10 @@ jobs:
echo "Last successful commit SHA: $LAST_SHA"
echo "##vso[task.setvariable variable=BASE_SHA]$LAST_SHA"
fi
displayName: 'Get last successful commit SHA'
condition: ne(variables['Build.Reason'], 'PullRequest')
env:
AZURE_DEVOPS_EXT_PAT: $(System.AccessToken)

- script: npm install --prefix=$HOME/.local -g pnpm@8
displayName: Install PNPM
Expand Down Expand Up @@ -741,11 +741,7 @@ jobs:
- bash: az devops configure --defaults organization=$(System.TeamFoundationCollectionUri) project=$(System.TeamProject)
displayName: 'Set default Azure DevOps organization and project'
# Get last successfull commit from Azure Devops CLI
- displayName: 'Get last successful commit SHA'
condition: ne(variables['Build.Reason'], 'PullRequest')
env:
AZURE_DEVOPS_EXT_PAT: $(System.AccessToken)
bash: |
- bash: |
LAST_SHA=$(az pipelines build list --branch $(Build.SourceBranchName) --definition-ids $(System.DefinitionId) --result succeeded --top 1 --query "[0].triggerInfo.\\"ci.sourceSha\\"")
if [ -z "$LAST_SHA" ]
then
Expand All @@ -754,6 +750,10 @@ jobs:
echo "Last successful commit SHA: $LAST_SHA"
echo "##vso[task.setvariable variable=BASE_SHA]$LAST_SHA"
fi
displayName: 'Get last successful commit SHA'
condition: ne(variables['Build.Reason'], 'PullRequest')
env:
AZURE_DEVOPS_EXT_PAT: $(System.AccessToken)

# Connect your workspace on nx.app and uncomment this to enable task distribution.
# The "--stop-agents-after" is optional, but allows idle agents to shut down once the "build" targets have been requested
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,11 +27,7 @@ jobs:
- bash: az devops configure --defaults organization=$(System.TeamFoundationCollectionUri) project=$(System.TeamProject)
displayName: 'Set default Azure DevOps organization and project'
# Get last successfull commit from Azure Devops CLI
- displayName: 'Get last successful commit SHA'
condition: ne(variables['Build.Reason'], 'PullRequest')
env:
AZURE_DEVOPS_EXT_PAT: $(System.AccessToken)
bash: |
- bash: |
LAST_SHA=$(az pipelines build list --branch $(Build.SourceBranchName) --definition-ids $(System.DefinitionId) --result succeeded --top 1 --query "[0].triggerInfo.\"ci.sourceSha\"")
if [ -z "$LAST_SHA" ]
then
Expand All @@ -40,6 +36,10 @@ jobs:
echo "Last successful commit SHA: $LAST_SHA"
echo "##vso[task.setvariable variable=BASE_SHA]$LAST_SHA"
fi
displayName: 'Get last successful commit SHA'
condition: ne(variables['Build.Reason'], 'PullRequest')
env:
AZURE_DEVOPS_EXT_PAT: $(System.AccessToken)

<% if(packageManager == 'pnpm'){ %>
- script: npm install --prefix=$HOME/.local -g pnpm@8
Expand Down