[performance] use journey own ftr config to run scalability test #152596
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dmlemeshko
added
release_note:skip
|
@elasticmachine merge upstream |
|
Testing PR in scalability pipeline |
i did a retry to make sure that jest failure was not related to this. It didn't look related, but it doesn't hurt. Will start the actual review now and will leave merging to you on Monday. |
💚 Build Succeeded
Metrics [docs]Unknown metric groupsESLint disabled line counts
Total ESLint disabled count
History
To update your PR or re-run it, just comment with: |
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Mar 7, 2023
…stic#152596) While debugging scalability testing failure for `cloud_security_dashboard` journey, I found that we hardcoded base FTR config to `x-pack/performance/journeys/login.ts` and main issue is that Kibana is not started properly. This PR makes few changes: - update `kbn-performance-testing-dataset-extractor` to save journey path as `configPath` so it can be later used to start ES/Kibana in the scalability run with the same configuration it was run for the single user journey run. - update scalability entry configuration to read base FTR config from generated scalability json file (`configPath` property) How to test: - make sure to clone the latest [kibana-load-testing](https://github.com/elastic/kibana-load-testing) repo and build it `mvn clean test-compile` - from kibana root directory run any api capacity test ``` node scripts/run_scalability.js --journey-path x-pack/test/scalability/apis/api.core.capabilities.json ``` Expected result: logs should display ``` debg Loading config file from x-pack/performance/journeys/login.ts ``` - download the latest artifacts from [buildkite](https://buildkite.com/elastic/kibana-performance-data-set-extraction/builds/171#0186a342-9dea-4a9b-bbe4-c96449563269), find `cloud_security_dashboard-<uuid>.json` - from kibana root directory run scalability test for `cloud_security_dashboard` journey ``` node scripts/run_scalability.js --journey-path <path to cloud_security_dashboard-<uuid>.json> ``` Expected result: logs should display ``` debg Loading config file from x-pack/performance/journeys/cloud_security_dashboard.ts ``` Co-authored-by: Kibana Machine <[email protected]> (cherry picked from commit bc44f52)
💔 Some backports could not be created
Note: Successful backport PRs will be merged automatically after passing CI. Manual backportTo create the backport manually run: Questions ?Please refer to the Backport tool documentation |
kibanamachine
added a commit
that referenced
this pull request
Mar 7, 2023
#152596) (#152806) # Backport This will backport the following commits from `main` to `8.7`: - [[performance] use journey own ftr config to run scalability test (#152596)](#152596) <!--- Backport version: 8.9.7 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Dzmitry Lemechko","email":"[email protected]"},"sourceCommit":{"committedDate":"2023-03-07T12:20:21Z","message":"[performance] use journey own ftr config to run scalability test (#152596)\n\nWhile debugging scalability testing failure for\r\n`cloud_security_dashboard` journey, I found that we hardcoded base FTR\r\nconfig to `x-pack/performance/journeys/login.ts` and main issue is that\r\nKibana is not started properly.\r\n\r\nThis PR makes few changes:\r\n- update `kbn-performance-testing-dataset-extractor` to save journey\r\npath as `configPath` so it can be later used to start ES/Kibana in the\r\nscalability run with the same configuration it was run for the single\r\nuser journey run.\r\n- update scalability entry configuration to read base FTR config from\r\ngenerated scalability json file (`configPath` property)\r\n\r\nHow to test:\r\n- make sure to clone the latest\r\n[kibana-load-testing](https://github.com/elastic/kibana-load-testing)\r\nrepo and build it `mvn clean test-compile`\r\n- from kibana root directory run any api capacity test\r\n```\r\nnode scripts/run_scalability.js --journey-path x-pack/test/scalability/apis/api.core.capabilities.json\r\n```\r\nExpected result: logs should display\r\n```\r\ndebg Loading config file from x-pack/performance/journeys/login.ts\r\n```\r\n- download the latest artifacts from\r\n[buildkite](https://buildkite.com/elastic/kibana-performance-data-set-extraction/builds/171#0186a342-9dea-4a9b-bbe4-c96449563269),\r\nfind `cloud_security_dashboard-<uuid>.json`\r\n- from kibana root directory run scalability test for\r\n`cloud_security_dashboard` journey\r\n```\r\nnode scripts/run_scalability.js --journey-path <path to cloud_security_dashboard-<uuid>.json>\r\n```\r\nExpected result: logs should display \r\n```\r\ndebg Loading config file from x-pack/performance/journeys/cloud_security_dashboard.ts\r\n```\r\n\r\nCo-authored-by: Kibana Machine <[email protected]>","sha":"bc44f524ac547f0211486c125b999fdd94bb03bf","branchLabelMapping":{"^v8.8.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","wg:performance","v8.8.0","v8.6.3","v8.7.1"],"number":152596,"url":"https://github.com/elastic/kibana/pull/152596","mergeCommit":{"message":"[performance] use journey own ftr config to run scalability test (#152596)\n\nWhile debugging scalability testing failure for\r\n`cloud_security_dashboard` journey, I found that we hardcoded base FTR\r\nconfig to `x-pack/performance/journeys/login.ts` and main issue is that\r\nKibana is not started properly.\r\n\r\nThis PR makes few changes:\r\n- update `kbn-performance-testing-dataset-extractor` to save journey\r\npath as `configPath` so it can be later used to start ES/Kibana in the\r\nscalability run with the same configuration it was run for the single\r\nuser journey run.\r\n- update scalability entry configuration to read base FTR config from\r\ngenerated scalability json file (`configPath` property)\r\n\r\nHow to test:\r\n- make sure to clone the latest\r\n[kibana-load-testing](https://github.com/elastic/kibana-load-testing)\r\nrepo and build it `mvn clean test-compile`\r\n- from kibana root directory run any api capacity test\r\n```\r\nnode scripts/run_scalability.js --journey-path x-pack/test/scalability/apis/api.core.capabilities.json\r\n```\r\nExpected result: logs should display\r\n```\r\ndebg Loading config file from x-pack/performance/journeys/login.ts\r\n```\r\n- download the latest artifacts from\r\n[buildkite](https://buildkite.com/elastic/kibana-performance-data-set-extraction/builds/171#0186a342-9dea-4a9b-bbe4-c96449563269),\r\nfind `cloud_security_dashboard-<uuid>.json`\r\n- from kibana root directory run scalability test for\r\n`cloud_security_dashboard` journey\r\n```\r\nnode scripts/run_scalability.js --journey-path <path to cloud_security_dashboard-<uuid>.json>\r\n```\r\nExpected result: logs should display \r\n```\r\ndebg Loading config file from x-pack/performance/journeys/cloud_security_dashboard.ts\r\n```\r\n\r\nCo-authored-by: Kibana Machine <[email protected]>","sha":"bc44f524ac547f0211486c125b999fdd94bb03bf"}},"sourceBranch":"main","suggestedTargetBranches":["8.6","8.7"],"targetPullRequestStates":[{"branch":"main","label":"v8.8.0","labelRegex":"^v8.8.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/152596","number":152596,"mergeCommit":{"message":"[performance] use journey own ftr config to run scalability test (#152596)\n\nWhile debugging scalability testing failure for\r\n`cloud_security_dashboard` journey, I found that we hardcoded base FTR\r\nconfig to `x-pack/performance/journeys/login.ts` and main issue is that\r\nKibana is not started properly.\r\n\r\nThis PR makes few changes:\r\n- update `kbn-performance-testing-dataset-extractor` to save journey\r\npath as `configPath` so it can be later used to start ES/Kibana in the\r\nscalability run with the same configuration it was run for the single\r\nuser journey run.\r\n- update scalability entry configuration to read base FTR config from\r\ngenerated scalability json file (`configPath` property)\r\n\r\nHow to test:\r\n- make sure to clone the latest\r\n[kibana-load-testing](https://github.com/elastic/kibana-load-testing)\r\nrepo and build it `mvn clean test-compile`\r\n- from kibana root directory run any api capacity test\r\n```\r\nnode scripts/run_scalability.js --journey-path x-pack/test/scalability/apis/api.core.capabilities.json\r\n```\r\nExpected result: logs should display\r\n```\r\ndebg Loading config file from x-pack/performance/journeys/login.ts\r\n```\r\n- download the latest artifacts from\r\n[buildkite](https://buildkite.com/elastic/kibana-performance-data-set-extraction/builds/171#0186a342-9dea-4a9b-bbe4-c96449563269),\r\nfind `cloud_security_dashboard-<uuid>.json`\r\n- from kibana root directory run scalability test for\r\n`cloud_security_dashboard` journey\r\n```\r\nnode scripts/run_scalability.js --journey-path <path to cloud_security_dashboard-<uuid>.json>\r\n```\r\nExpected result: logs should display \r\n```\r\ndebg Loading config file from x-pack/performance/journeys/cloud_security_dashboard.ts\r\n```\r\n\r\nCo-authored-by: Kibana Machine <[email protected]>","sha":"bc44f524ac547f0211486c125b999fdd94bb03bf"}},{"branch":"8.6","label":"v8.6.3","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.7","label":"v8.7.1","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Dzmitry Lemechko <[email protected]>
bmorelli25
pushed a commit
to bmorelli25/kibana
that referenced
this pull request
Mar 10, 2023
…stic#152596) While debugging scalability testing failure for `cloud_security_dashboard` journey, I found that we hardcoded base FTR config to `x-pack/performance/journeys/login.ts` and main issue is that Kibana is not started properly. This PR makes few changes: - update `kbn-performance-testing-dataset-extractor` to save journey path as `configPath` so it can be later used to start ES/Kibana in the scalability run with the same configuration it was run for the single user journey run. - update scalability entry configuration to read base FTR config from generated scalability json file (`configPath` property) How to test: - make sure to clone the latest [kibana-load-testing](https://github.com/elastic/kibana-load-testing) repo and build it `mvn clean test-compile` - from kibana root directory run any api capacity test ``` node scripts/run_scalability.js --journey-path x-pack/test/scalability/apis/api.core.capabilities.json ``` Expected result: logs should display ``` debg Loading config file from x-pack/performance/journeys/login.ts ``` - download the latest artifacts from [buildkite](https://buildkite.com/elastic/kibana-performance-data-set-extraction/builds/171#0186a342-9dea-4a9b-bbe4-c96449563269), find `cloud_security_dashboard-<uuid>.json` - from kibana root directory run scalability test for `cloud_security_dashboard` journey ``` node scripts/run_scalability.js --journey-path <path to cloud_security_dashboard-<uuid>.json> ``` Expected result: logs should display ``` debg Loading config file from x-pack/performance/journeys/cloud_security_dashboard.ts ``` Co-authored-by: Kibana Machine <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
While debugging scalability testing failure for
cloud_security_dashboardjourney, I found that we hardcoded base FTR config tox-pack/performance/journeys/login.tsand main issue is that Kibana is not started properly.This PR makes few changes:
kbn-performance-testing-dataset-extractorto save journey path asconfigPathso it can be later used to start ES/Kibana in the scalability run with the same configuration it was run for the single user journey run.configPathproperty)How to test:
mvn clean test-compileExpected result: logs should display
cloud_security_dashboard-<uuid>.jsoncloud_security_dashboardjourneyExpected result: logs should display