Skip to content

Commit

Permalink
[8.7] [performance] use journey own ftr config to run scalability test (
Browse files Browse the repository at this point in the history 
#152596) (#152806)

# Backport

This will backport the following commits from `main` to `8.7`:
- [[performance] use journey own ftr config to run scalability test
(#152596)](#152596)

<!--- Backport version: 8.9.7 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Dzmitry
Lemechko","email":"[email protected]"},"sourceCommit":{"committedDate":"2023-03-07T12:20:21Z","message":"[performance]
use journey own ftr config to run scalability test (#152596)\n\nWhile
debugging scalability testing failure for\r\n`cloud_security_dashboard`
journey, I found that we hardcoded base FTR\r\nconfig to
`x-pack/performance/journeys/login.ts` and main issue is that\r\nKibana
is not started properly.\r\n\r\nThis PR makes few changes:\r\n- update
`kbn-performance-testing-dataset-extractor` to save journey\r\npath as
`configPath` so it can be later used to start ES/Kibana in
the\r\nscalability run with the same configuration it was run for the
single\r\nuser journey run.\r\n- update scalability entry configuration
to read base FTR config from\r\ngenerated scalability json file
(`configPath` property)\r\n\r\nHow to test:\r\n- make sure to clone the
latest\r\n[kibana-load-testing](https://github.com/elastic/kibana-load-testing)\r\nrepo
and build it `mvn clean test-compile`\r\n- from kibana root directory
run any api capacity test\r\n```\r\nnode scripts/run_scalability.js
--journey-path
x-pack/test/scalability/apis/api.core.capabilities.json\r\n```\r\nExpected
result: logs should display\r\n```\r\ndebg Loading config file from
x-pack/performance/journeys/login.ts\r\n```\r\n- download the latest
artifacts
from\r\n[buildkite](https://buildkite.com/elastic/kibana-performance-data-set-extraction/builds/171#0186a342-9dea-4a9b-bbe4-c96449563269),\r\nfind
`cloud_security_dashboard-<uuid>.json`\r\n- from kibana root directory
run scalability test for\r\n`cloud_security_dashboard`
journey\r\n```\r\nnode scripts/run_scalability.js --journey-path <path
to cloud_security_dashboard-<uuid>.json>\r\n```\r\nExpected result: logs
should display \r\n```\r\ndebg Loading config file from
x-pack/performance/journeys/cloud_security_dashboard.ts\r\n```\r\n\r\nCo-authored-by:
Kibana Machine
<[email protected]>","sha":"bc44f524ac547f0211486c125b999fdd94bb03bf","branchLabelMapping":{"^v8.8.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","wg:performance","v8.8.0","v8.6.3","v8.7.1"],"number":152596,"url":"https://github.com/elastic/kibana/pull/152596","mergeCommit":{"message":"[performance]
use journey own ftr config to run scalability test (#152596)\n\nWhile
debugging scalability testing failure for\r\n`cloud_security_dashboard`
journey, I found that we hardcoded base FTR\r\nconfig to
`x-pack/performance/journeys/login.ts` and main issue is that\r\nKibana
is not started properly.\r\n\r\nThis PR makes few changes:\r\n- update
`kbn-performance-testing-dataset-extractor` to save journey\r\npath as
`configPath` so it can be later used to start ES/Kibana in
the\r\nscalability run with the same configuration it was run for the
single\r\nuser journey run.\r\n- update scalability entry configuration
to read base FTR config from\r\ngenerated scalability json file
(`configPath` property)\r\n\r\nHow to test:\r\n- make sure to clone the
latest\r\n[kibana-load-testing](https://github.com/elastic/kibana-load-testing)\r\nrepo
and build it `mvn clean test-compile`\r\n- from kibana root directory
run any api capacity test\r\n```\r\nnode scripts/run_scalability.js
--journey-path
x-pack/test/scalability/apis/api.core.capabilities.json\r\n```\r\nExpected
result: logs should display\r\n```\r\ndebg Loading config file from
x-pack/performance/journeys/login.ts\r\n```\r\n- download the latest
artifacts
from\r\n[buildkite](https://buildkite.com/elastic/kibana-performance-data-set-extraction/builds/171#0186a342-9dea-4a9b-bbe4-c96449563269),\r\nfind
`cloud_security_dashboard-<uuid>.json`\r\n- from kibana root directory
run scalability test for\r\n`cloud_security_dashboard`
journey\r\n```\r\nnode scripts/run_scalability.js --journey-path <path
to cloud_security_dashboard-<uuid>.json>\r\n```\r\nExpected result: logs
should display \r\n```\r\ndebg Loading config file from
x-pack/performance/journeys/cloud_security_dashboard.ts\r\n```\r\n\r\nCo-authored-by:
Kibana Machine
<[email protected]>","sha":"bc44f524ac547f0211486c125b999fdd94bb03bf"}},"sourceBranch":"main","suggestedTargetBranches":["8.6","8.7"],"targetPullRequestStates":[{"branch":"main","label":"v8.8.0","labelRegex":"^v8.8.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/152596","number":152596,"mergeCommit":{"message":"[performance]
use journey own ftr config to run scalability test (#152596)\n\nWhile
debugging scalability testing failure for\r\n`cloud_security_dashboard`
journey, I found that we hardcoded base FTR\r\nconfig to
`x-pack/performance/journeys/login.ts` and main issue is that\r\nKibana
is not started properly.\r\n\r\nThis PR makes few changes:\r\n- update
`kbn-performance-testing-dataset-extractor` to save journey\r\npath as
`configPath` so it can be later used to start ES/Kibana in
the\r\nscalability run with the same configuration it was run for the
single\r\nuser journey run.\r\n- update scalability entry configuration
to read base FTR config from\r\ngenerated scalability json file
(`configPath` property)\r\n\r\nHow to test:\r\n- make sure to clone the
latest\r\n[kibana-load-testing](https://github.com/elastic/kibana-load-testing)\r\nrepo
and build it `mvn clean test-compile`\r\n- from kibana root directory
run any api capacity test\r\n```\r\nnode scripts/run_scalability.js
--journey-path
x-pack/test/scalability/apis/api.core.capabilities.json\r\n```\r\nExpected
result: logs should display\r\n```\r\ndebg Loading config file from
x-pack/performance/journeys/login.ts\r\n```\r\n- download the latest
artifacts
from\r\n[buildkite](https://buildkite.com/elastic/kibana-performance-data-set-extraction/builds/171#0186a342-9dea-4a9b-bbe4-c96449563269),\r\nfind
`cloud_security_dashboard-<uuid>.json`\r\n- from kibana root directory
run scalability test for\r\n`cloud_security_dashboard`
journey\r\n```\r\nnode scripts/run_scalability.js --journey-path <path
to cloud_security_dashboard-<uuid>.json>\r\n```\r\nExpected result: logs
should display \r\n```\r\ndebg Loading config file from
x-pack/performance/journeys/cloud_security_dashboard.ts\r\n```\r\n\r\nCo-authored-by:
Kibana Machine
<[email protected]>","sha":"bc44f524ac547f0211486c125b999fdd94bb03bf"}},{"branch":"8.6","label":"v8.6.3","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.7","label":"v8.7.1","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Dzmitry Lemechko <[email protected]>
  • Loading branch information
@kibanamachine @dmlemeshko
kibanamachine and dmlemeshko authored Mar 7, 2023
1 parent 84356af commit a74084d
Show file tree
Hide file tree
Showing 5 changed files with 9 additions and 4 deletions.
1 change: 1 addition & 0 deletions packages/kbn-performance-testing-dataset-extractor/src/cli.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,7 @@ export async function runExtractor() {
return extractor({
param: {
journeyName: journey.config.getName(),
configPath,
scalabilitySetup,
testData,
buildId,
Expand Down
5 changes: 4 additions & 1 deletion packages/kbn-performance-testing-dataset-extractor/src/extractor.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,8 @@ export const extractor = async ({ param, client, log }: CLIParams) => {
username: client.username,
password: client.password,
};
const { journeyName, scalabilitySetup, testData, buildId, withoutStaticResources } = param;
const { journeyName, configPath, scalabilitySetup, testData, buildId, withoutStaticResources } =
param;
log.info(
`Searching transactions with 'labels.testBuildId=${buildId}' and 'labels.journeyName=${journeyName}'`
);
Expand Down Expand Up @@ -87,6 +88,7 @@ export const extractor = async ({ param, client, log }: CLIParams) => {
await saveFile(
{
journeyName,
configPath,
kibanaVersion,
scalabilitySetup,
testData,
Expand All @@ -101,6 +103,7 @@ export const extractor = async ({ param, client, log }: CLIParams) => {
await saveFile(
{
journeyName,
configPath,
kibanaVersion,
testData,
streams: esStreams,
Expand Down
1 change: 1 addition & 0 deletions packages/kbn-performance-testing-dataset-extractor/src/types.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ export interface TestData {
export interface CLIParams {
param: {
journeyName: string;
configPath: string;
scalabilitySetup?: ScalabilitySetup;
testData: TestData;
buildId: string;
Expand Down
5 changes: 2 additions & 3 deletions x-pack/test/scalability/config.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -42,10 +42,9 @@ export default async function ({ readConfigFile }: FtrConfigProviderContext) {
}

const journey: ScalabilityJourney = JSON.parse(fs.readFileSync(scalabilityJsonPath, 'utf8'));
const configPath = journey.configPath ?? 'x-pack/performance/journeys/login.ts';

const baseConfig = (
await readConfigFile(require.resolve('../../performance/journeys/login.ts'))
).getAll();
const baseConfig = (await readConfigFile(path.resolve(REPO_ROOT, configPath))).getAll();

return {
...baseConfig,
Expand Down
1 change: 1 addition & 0 deletions x-pack/test/scalability/types.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ import { ScalabilitySetup } from '@kbn/journeys';

export interface ScalabilityJourney {
journeyName: string;
configPath?: string;
scalabilitySetup: ScalabilitySetup;
testData?: {
esArchives: string[];
Expand Down

0 comments on commit a74084d

Please sign in to comment.