Skip to content

Commit

Permalink
[ADP-3173] Reinstate integration tests for macOS (#4429)
Browse files Browse the repository at this point in the history
This pull request reinstates the macOS integration tests at pre-release
granularity.

### Comments

The integration tests on macOS are not free of flakiness (e.g. running
out of file descriptors), but they can succeed:

<img width="543" alt="macos-integration-tests"
src="https://github.com/cardano-foundation/cardano-wallet/assets/564592/f2710c1c-6eae-4e4f-ac44-c828b696af1c">

For this reason, I propose to run them at pre-release granularity, not
at pre-merge granularity.

### Issue number

ADP-3173
  • Loading branch information
HeinrichApfelmus authored Feb 19, 2024
2 parents 4ad2d2a + 7cee054 commit cbc2569
Show file tree
Hide file tree
Showing 2 changed files with 30 additions and 18 deletions.
25 changes: 13 additions & 12 deletions .buildkite/nightly.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,62 +3,63 @@ env:
NIX_PATH: "channel:nixos-21.11"

# Per-host variables - shared across containers on host
macos: "x86_64-darwin"
macos: "aarch64-darwin"
linux: "x86_64-linux"

steps:
- label: 'Check auto-generated Nix on macos'
- label: 'Check auto-generated Nix on macOS'
key: macos-nix
commands:
- './nix/regenerate.sh'
agents:
system: ${macos}
if: 0 == 1 # Suspended until we have a macos build agent
queue: "cardano-wallet"

# ADP-2522 - Fix integration tests on MacOS
- label: 'Run integration tests on macos'
- label: 'Run integration tests on macOS'
key: macos-tests-integration
depends_on: macos-nix
command: 'GC_DONT_GC=1 nix build -L .#ci.${macos}.tests.run.integration'
command: nix shell 'nixpkgs#just' -c just babbage-integration-tests
agents:
system: ${macos}
if: 0 == 1 # Suspended until we have a macos build agent
queue: "cardano-wallet"

- label: 'Restore benchmark - cardano mainnet'
command: "./.buildkite/bench-restore.sh mainnet"
timeout_in_minutes: 1200
agents:
system: x86_64-linux
system: ${linux}
queue: adrestia-bench
if: 'build.env("step") == null || build.env("step") =~ /restore-mainnet/'

- label: 'API benchmark'
command: "./.buildkite/bench-api.sh"
timeout_in_minutes: 210
agents:
system: x86_64-linux
system: ${linux}
queue: adrestia-bench
if: 'build.env("step") == null || build.env("step") =~ /bench-api/'

- label: 'Database benchmark'
command: "./.buildkite/bench-db.sh"
timeout_in_minutes: 240
agents:
system: x86_64-linux
system: ${linux}
queue: adrestia-bench
if: 'build.env("step") == null || build.env("step") =~ /bench-db/'

- label: 'Latency benchmark'
command: "./.buildkite/bench-latency.sh"
timeout_in_minutes: 120
agents:
system: x86_64-linux
system: ${linux}
queue: adrestia-bench
if: 'build.env("step") == null || build.env("step") =~ /bench-latency/'

- label: 'Memory benchmark'
command: "./.buildkite/bench-memory.sh"
timeout_in_minutes: 30
agents:
system: x86_64-linux
system: ${linux}
queue: adrestia-bench
if: 'build.env("step") == null || build.env("step") =~ /bench-memory/'

Expand Down
23 changes: 17 additions & 6 deletions .buildkite/pipeline.yml
Original file line number Diff line number Diff line change
Expand Up @@ -184,29 +184,37 @@ steps:
system: ${macos}

- label: 'Run unit tests (macOS, x86_64)'
key: macos-tests-unit
key: macos-intel-tests-run-unit
depends_on: macos-nix
command: 'nix build -L .#ci.x86_64-darwin.tests.run.unit'
agents:
system: ${macos}

- label: 'Run unit tests (macOS, arm64)'
key: macos-arm64-tests-run-unit
depends_on: macos-nix
key: macos-build-tests-arm64
command: 'nix build -L .#ci.aarch64-darwin.tests.run.unit'
agents:
system: ${macos}

- label: 'Build integration tests (macOS, arm64)'
key: macos-arm64-tests-build-integration
depends_on: macos-nix
command: nix build -L .#packages.aarch64-darwin.integration-exe
agents:
system: ${macos}
queue: "cardano-wallet"

- label: 'Build package (macOS, x86_64)'
key: macos-package
key: macos-intel-package
depends_on: macos-nix
command: 'nix build -o result/macos-intel .#packages.x86_64-darwin.ci.artifacts.macos-intel.release'
artifact_paths: [ "./result/macos-intel/**" ]
agents:
system: ${macos}

- label: 'Build package (macOS, arm64)'
key: macos-package-arm64
key: macos-arm64-package
depends_on: macos-nix
command: 'nix build -o result/macos-silicon .#packages.aarch64-darwin.ci.artifacts.macos-silicon.release'
artifact_paths: [ "./result/macos-silicon/**" ]
Expand Down Expand Up @@ -285,15 +293,18 @@ steps:
- linux-tests-unit
- linux-tests-integration-babbage
- linux-tests-integration-conway
- macos-arm64-tests-run-unit
- macos-arm64-tests-build-integration
- macos-intel-tests-run-unit
- cabal-configure
- code-format
- hlint
- openapi
- lint-bash
- hls
- build-benchmarks
- macos-tests-unit
- macos-package
- macos-arm64-package
- macos-intel-package
- linux-package
- windows-package
- windows-testing-bundle
Expand Down

0 comments on commit cbc2569

Please sign in to comment.