Skip to content

Commit

Permalink
docs(core): update CI tutorials with better distribution screenshots (#…
Browse files Browse the repository at this point in the history
…27049)

The current screenshot doesn't show any tasks. This PR replaces it with
one that actually shoes tasks in progress, as well what the GitHub PR
looks like. Also updates the workflow config file to align with what we
generate for 19.5.1.

Previews:
-
https://nx-dev-git-docs-ci-tutorials-update-part-2-nrwl.vercel.app/ci/intro/tutorials/circle
-
https://nx-dev-git-docs-ci-tutorials-update-part-2-nrwl.vercel.app/ci/intro/tutorials/github-actions

(cherry picked from commit 31f9d9e)
  • Loading branch information
jaysoo authored and FrozenPandaz committed Jul 24, 2024
1 parent a315018 commit f6a6ba7
Show file tree
Hide file tree
Showing 5 changed files with 96 additions and 18 deletions.
62 changes: 52 additions & 10 deletions docs/nx-cloud/tutorial/circle.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,9 +80,9 @@ pnpm nx generate ci-workflow --ci=circleci

This generator will overwrite Circle CI's default `.circleci/config.yml` file to create a CI pipeline that will run the `lint`, `test`, `build` and `e2e` tasks for projects that are affected by any given PR.

The key lines in the CI pipeline are highlighted in this excerpt:
The key lines in the CI pipeline are:

```yml {% fileName=".circleci/config.yml" highlightLines=["25-27"] %}
```yml {% fileName=".circleci/config.yml" highlightLines=["27-29"] %}
version: 2.1

orbs:
Expand All @@ -99,8 +99,10 @@ jobs:
name: Install PNPM
command: npm install --prefix=$HOME/.local -g pnpm@8

# Connect your workspace on nx.app and uncomment this to enable task distribution.
# The "--stop-agents-after" is optional, but allows idle agents to shut down once the "e2e-ci" targets have been requested
# This enables task distribution via Nx Cloud
# Run this command as early as possible, before dependencies are installed
# Learn more at https://nx.dev/ci/reference/nx-cloud-cli#npx-nxcloud-startcirun
# Connect your workspace by running "nx connect" and uncomment this
# - run: pnpm dlx nx-cloud start-ci-run --distribute-on="3 linux-medium-js" --stop-agents-after="e2e-ci"

- run: pnpm install --frozen-lockfile
Expand Down Expand Up @@ -230,22 +232,62 @@ The Nx Agents feature

To enable Nx Agents, make sure the following line is uncommented in the `.circleci/config.yml` file.

```yml {% fileName=".circleci/config.yml" highlightLines=[3] %}
# Connect your workspace on nx.app and uncomment this to enable task distribution.
# The "--stop-agents-after" is optional, but allows idle agents to shut down once the "e2e-ci" targets have been requested
- run: pnpm dlx nx-cloud start-ci-run --distribute-on="3 linux-medium-js" --stop-agents-after="e2e-ci"
```yml {% fileName=".circleci/config.yml" highlightLines=["21"] %}
version: 2.1
orbs:
nx: nrwl/[email protected]
jobs:
main:
docker:
- image: cimg/node:lts-browsers
steps:
- checkout
- run:
name: Install PNPM
command: npm install --prefix=$HOME/.local -g pnpm@8
# This enables task distribution via Nx Cloud
# Run this command as early as possible, before dependencies are installed
# Learn more at https://nx.dev/ci/reference/nx-cloud-cli#npx-nxcloud-startcirun
# Connect your workspace by running "nx connect" and uncomment this
- run: pnpm dlx nx-cloud start-ci-run --distribute-on="3 linux-medium-js" --stop-agents-after="e2e-ci"
- run: pnpm install --frozen-lockfile
- nx/set-shas:
main-branch-name: 'main'
# Prepend any command with "nx-cloud record --" to record its logs to Nx Cloud
# - run: pnpm exec nx-cloud record -- echo Hello World
- run: pnpm exec nx affected --base=$NX_BASE --head=$NX_HEAD -t lint test build
workflows:
version: 2
ci:
jobs:
- main
```

We recommend you add this line right after you check out the repo, before installing node modules.

- `nx-cloud start-ci-run --distribute-on="3 linux-medium-js` lets Nx know that all the tasks after this line should using Nx Agents and that Nx Cloud should use 5 instances of the `linux-medium-js` launch template. See the separate reference on how to [configure a custom launch template](/ci/reference/launch-templates).
- `nx-cloud start-ci-run --distribute-on="3 linux-medium-js` lets Nx know that all the tasks after this line should use Nx Agents and that Nx Cloud should use three instances of the `linux-medium-js` launch template. See the separate reference on how to [configure a custom launch template](/ci/reference/launch-templates).
- `--stop-agents-after="e2e-ci"` lets Nx Cloud know which line is the last command in this pipeline. Once there are no more e2e tasks for an agent to run, Nx Cloud will automatically shut them down. This way you're not wasting money on idle agents while a particularly long e2e task is running on a single agent.

Try it out by creating a new PR with the above changes.

```shell
git checkout -b enable-distribution
git commit -am 'enable task distribution'
```

![](/nx-cloud/tutorial/github-pr-distribution.avif)

Once Circle CI starts, you can click on the Nx Cloud report to see what tasks agents are executing in real time.

![Circle CI showing multiple DTE agents](/nx-cloud/tutorial/nx-cloud-agents-in-progress.png)
![](/nx-cloud/tutorial/nx-cloud-distribution.avif)

With this pipeline configuration in place, no matter how large the repository scales, Nx Cloud will adjust and distribute tasks across agents in the optimal way. If CI pipelines start to slow down, just add some agents. One of the main advantages is that this pipeline definition is declarative. We tell Nx what commands to run, but not how to distribute them. That way even if our monorepo structure changes and evolves over time, the distribution will be taken care of by Nx Cloud.

Expand Down
52 changes: 44 additions & 8 deletions docs/nx-cloud/tutorial/github-actions.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,9 +60,9 @@ pnpm nx generate ci-workflow --ci=github

This generator creates a `.github/workflows/ci.yml` file that contains a CI pipeline that will run the `lint`, `test`, `build` and `e2e` tasks for projects that are affected by any given PR.

The key lines in the CI pipeline are highlighted in this excerpt:
The key lines in the CI pipeline are:

```yml {% fileName=".github/workflows/ci.yml" highlightLines=["31-33"] %}
```yml {% fileName=".github/workflows/ci.yml" highlightLines=["29-32"] %}
name: CI
# ...
jobs:
Expand Down Expand Up @@ -197,22 +197,58 @@ The Nx Agents feature

To enable Nx Agents, make sure the following line is uncommented in the `.github/workflows/ci.yml` file.

```yml {% fileName=".github/workflows/ci.yml" highlightLines=[3] %}
# Connect your workspace on nx.app and uncomment this to enable task distribution.
# The "--stop-agents-after" is optional, but allows idle agents to shut down once the "e2e-ci" targets have been requested
- run: pnpm dlx nx-cloud start-ci-run --distribute-on="3 linux-medium-js" --stop-agents-after="e2e-ci"
```yml {% fileName=".github/workflows/ci.yml" highlightLines=[19] %}
name: CI
# ...
jobs:
main:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: pnpm/action-setup@v2
with:
version: 8
# This enables task distribution via Nx Cloud
# Run this command as early as possible, before dependencies are installed
# Learn more at https://nx.dev/ci/reference/nx-cloud-cli#npx-nxcloud-startcirun
# Connect your workspace by running "nx connect" and uncomment this
- run: pnpm dlx nx-cloud start-ci-run --distribute-on="3 linux-medium-js" --stop-agents-after="e2e-ci"
- uses: actions/setup-node@v3
with:
node-version: 20
cache: 'pnpm'
- run: pnpm install --frozen-lockfile
- uses: nrwl/nx-set-shas@v4
# Prepend any command with "nx-cloud record --" to record its logs to Nx Cloud
# - run: pnpm exec nx-cloud record -- echo Hello World
# Nx Affected runs only tasks affected by the changes in this PR/commit. Learn more: https://nx.dev/ci/features/affected
- run: pnpm exec nx affected -t lint test build
```

We recommend you add this line right after you check out the repo, before installing node modules.

- `nx-cloud start-ci-run --distribute-on="3 linux-medium-js` lets Nx know that all the tasks after this line should using Nx Agents and that Nx Cloud should use 5 instances of the `linux-medium-js` launch template. See the separate reference on how to [configure a custom launch template](/ci/reference/launch-templates).
- `nx-cloud start-ci-run --distribute-on="3 linux-medium-js` lets Nx know that all the tasks after this line should use Nx Agents and that Nx Cloud should use three instances of the `linux-medium-js` launch template. See the separate reference on how to [configure a custom launch template](/ci/reference/launch-templates).
- `--stop-agents-after="e2e-ci"` lets Nx Cloud know which line is the last command in this pipeline. Once there are no more e2e tasks for an agent to run, Nx Cloud will automatically shut them down. This way you're not wasting money on idle agents while a particularly long e2e task is running on a single agent.

Try it out by creating a new PR with the above changes.

```shell
git checkout -b enable-distribution
git commit -am 'enable task distribution'
```

![](/nx-cloud/tutorial/github-pr-distribution.avif)

Once GitHub Actions starts, you can click on the Nx Cloud report to see what tasks agents are executing in real time.

![GitHub Actions showing multiple DTE agents](/nx-cloud/tutorial/nx-cloud-agents-in-progress.png)
![](/nx-cloud/tutorial/nx-cloud-distribution.avif)

With this pipeline configuration in place, no matter how large the repository scales, Nx Cloud will adjust and distribute tasks across agents in the optimal way. If CI pipelines start to slow down, just add some agents. One of the main advantages is that this pipeline definition is declarative. We tell Nx what commands to run, but not how to distribute them. That way even if our monorepo structure changes and evolves over time, the distribution will be taken care of by Nx Cloud.

Expand Down
Binary file not shown.
Binary file not shown.
Binary file added docs/nx-cloud/tutorial/nx-cloud-distribution.avif
Binary file not shown.

0 comments on commit f6a6ba7

Please sign in to comment.