Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Fixes to cloud benchmarks (no-changelog) #10634

Merged
merged 3 commits into from
Sep 2, 2024
Merged

Conversation

tomi
Copy link
Contributor

@tomi tomi commented Sep 2, 2024

Summary

Contains the following changes:

  1. fix(benchmark): Separate container data into own host folders

So different postgres instances have their own data instead of
reusing it from the previous run. Also fixes the permission issues
in the data directory. If we let docker create the mounted dirs,
they will be owned by root:root, and the containers won't have
access to them. Hence we create them ourselves and run the containers
using the current user and group

  1. refactor(benchmark): Use ssh instead of az ssh vm

While the az ssh vm is more secure, it has multiple issues:

  • The access token expires in 10 mins, causing test failures:
    ERROR: AADSTS700024: Client assertion is not within its valid time range
  • It does not propagate the exit code, but instead always exits with 0
  • It can't do scp, so we need to clone the repo. This makes local development more cumbersome
  • It logs all kinds of unnecessary stuff into stdout

Hence this commits replaces az ssh vm with ssh and scp. Since the benchmark
environment is a throwaway env without any critical data, it's not the biggest deals.
We keep the private key hidden throughout the run anyways, so it's not leaked unless
someone gains access to the GH runner.

  1. fix(benchmark): Exit with 1 if the run script fails

Review / Merge checklist

  • PR title and summary are descriptive. (conventions)
  • Docs updated or follow-up ticket created.
  • Tests included.
  • PR Labeled with release/backport (if the PR is an urgent fix that needs to be backported)

tomi added 3 commits September 2, 2024 13:43
So different postgres instances have their own data instead of
reusing it from the previous run. Also fixes the permission issues
in the data directory. If we let docker create the mounted dirs,
they will be owned by root:root, and the containers won't have
access to them. Hence we create them ourselves and run the containers
using the current user and group
While the `az ssh vm` is more secure, it has multiple issues:
- The access token expires in 10 mins, causing test failures:
  `ERROR: AADSTS700024: Client assertion is not within its valid time range`
- It does not propagate the exit code, but instead always exits with 0
- It can't do `scp`, so we need to clone the repo. This makes local development more cumbersome
- It logs all kinds of unnecessary stuff into stdout

Hence this commits replaces `az ssh vm` with `ssh` and `scp`. Since the benchmark
environment is a throwaway env without any critical data, it's not the biggest deals.
We keep the private key hidden throughout the run anyways, so it's not leaked unless
someone gains access to the GH runner.
@tomi tomi changed the title Fix cloud benchmarks fix: Fixes to cloud benchmarks (no-changelog) Sep 2, 2024
@n8n-assistant n8n-assistant bot added the n8n team Authored by the n8n team label Sep 2, 2024
Copy link

cypress bot commented Sep 2, 2024

n8n    Run #6700

Run Properties:  status check passed Passed #6700  •  git commit a03a797174: 🌳 🖥️ browsers:node18.12.0-chrome107 🤖 tomi 🗃️ e2e/*
Project n8n
Branch Review fix-cloud-benchmarks
Run status status check passed Passed #6700
Run duration 04m 38s
Commit git commit a03a797174: 🌳 🖥️ browsers:node18.12.0-chrome107 🤖 tomi 🗃️ e2e/*
Committer Tomi Turtiainen
View all properties for this run ↗︎

Test results
Tests that failed  Failures 0
Tests that were flaky  Flaky 0
Tests that did not run due to a developer annotating a test with .skip  Pending 0
Tests that did not run due to a failure in a mocha hook  Skipped 0
Tests that passed  Passing 422
View all changes introduced in this branch ↗︎

Copy link
Contributor

github-actions bot commented Sep 2, 2024

✅ All Cypress E2E specs passed

@tomi tomi merged commit afc4d4e into master Sep 2, 2024
33 checks passed
@tomi tomi deleted the fix-cloud-benchmarks branch September 2, 2024 11:58
MiloradFilipovic added a commit that referenced this pull request Sep 2, 2024
* master:
  fix(core): Flush responses for ai streaming endpoints (#10633)
  fix: Re-enable infra provisioning and teardown (no-changelog) (#10636)
  feat(core): Execution curation (#10342)
  fix(Webhook Node): Add tests for utils (no-changelog) (#10613)
  fix: Fixes to cloud benchmarks (no-changelog) (#10634)
  test: Add JS CodeNode benchmark scenario (#10632)
  refactor(editor): Migrate `MainSidebar.vue` to composition API (no-changelog) (#10538)
  build: Fix `cli` nodemon config (#10628)
  docs: Add 'benchmark' scope to PR Title Conventions documentation (#10624)
  ci: Fixes to benchmarks in cloud (#10626)
  test(editor): Increase test coverage for users settings page and modal (#10623)
  fix(Wait Node): Append n8n attribution option (#10585)
  refactor(editor): Migrate `NodeSettings.vue` to composition API (#10545)
  fix(editor): Add pinned data only to manual executions in execution view (#10605)
  ci: Omit benchmark scope commits from changelog (no-changelog) (#10618)
  fix: Disable errors obfuscation (no-changelog) (#10617)
  ci: Fix script name in gh workflow (#10619)
  ci: Fix nightly image not being pushed to ghcr (#10620)
@janober
Copy link
Member

janober commented Sep 5, 2024

Got released with [email protected]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
n8n team Authored by the n8n team Released
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants