Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Reporting/Tests] Improvements for task stability in serverless tests #195841

Merged
merged 25 commits into from
Oct 24, 2024
Merged

[Reporting/Tests] Improvements for task stability in serverless tests #195841

Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
11351b7
fix "management: deletion" test by loading necessary archives
tsullivan Oct 10, 2024
2c3ae7b
Merge branch 'main' into reporting/tests-serverless-improvements-ii
tsullivan Oct 17, 2024
d7a2995
Merge branch 'main' into reporting/tests-serverless-improvements-ii
tsullivan Oct 17, 2024
4402169
Merge branch 'main' into reporting/tests-serverless-improvements-ii
tsullivan Oct 18, 2024
a3ae988
use samlAuth service and cookieHeader for internal API
dmlemeshko Oct 18, 2024
a2a8e1b
Merge branch 'main' into reporting/tests-serverless-improvements-ii
dmlemeshko Oct 18, 2024
46ba5d8
Merge branch 'main' into reporting/tests-serverless-improvements-ii
tsullivan Oct 21, 2024
dce635b
unload es archive and delete saved objects after the test run
tsullivan Oct 21, 2024
11a32e1
Use cookie credentials for internal requests
tsullivan Oct 21, 2024
a5fb38a
Use cookie credentials instead of role credentials
tsullivan Oct 21, 2024
4a4f9c2
Merge branch 'main' into reporting/tests-serverless-improvements-ii
tsullivan Oct 22, 2024
0bea1a5
use api key credentials for datastream management api
tsullivan Oct 22, 2024
2d2c790
Merge branch 'main' into reporting/tests-serverless-improvements-ii
tsullivan Oct 22, 2024
db3fa48
invalidate the api key in the after() block of test
tsullivan Oct 23, 2024
f530496
Merge branch 'main' into reporting/tests-serverless-improvements-ii
tsullivan Oct 23, 2024
b3f9765
Getting the internalReqHeader is not async
tsullivan Oct 23, 2024
5305cdd
Ensure the functional test job is allowed to finish
tsullivan Oct 23, 2024
631e9f1
Fix internalReqHeader
tsullivan Oct 23, 2024
c795446
Fix an instance of waiting for the job to finish
tsullivan Oct 23, 2024
03c5617
Merge branch 'main' into reporting/tests-serverless-improvements-ii
tsullivan Oct 23, 2024
dd877b2
fix a broken test
tsullivan Oct 23, 2024
ab5a79e
Merge branch 'reporting/tests-serverless-improvements-ii' of github.c…
tsullivan Oct 23, 2024
97d4836
Merge branch 'main' into reporting/tests-serverless-improvements-ii
tsullivan Oct 23, 2024
8e4ee0f
remove inessential differences
tsullivan Oct 23, 2024
ba8a7a0
Merge branch 'main' into reporting/tests-serverless-improvements-ii
tsullivan Oct 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 12 additions & 7 deletions x-pack/test_serverless/api_integration/test_suites/common/reporting/csv_v2_esql.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ import expect from '@kbn/expect';
import request from 'supertest';

import { DISCOVER_APP_LOCATOR } from '@kbn/discover-plugin/common';
import { InternalRequestHeader, RoleCredentials } from '@kbn/ftr-common-functional-services';
import { CookieCredentials, InternalRequestHeader } from '@kbn/ftr-common-functional-services';
import type { ReportApiJSON } from '@kbn/reporting-common/types';
import type { JobParamsCsvFromSavedObject } from '@kbn/reporting-export-types-csv-common';
import { FtrProviderContext } from '../../../ftr_provider_context';
Expand All @@ -21,8 +21,8 @@ export default ({ getService }: FtrProviderContext) => {
const log = getService('log');
const reportingAPI = getService('svlReportingApi');
const svlCommonApi = getService('svlCommonApi');
const svlUserManager = getService('svlUserManager');
let roleAuthc: RoleCredentials;
const samlAuth = getService('samlAuth');
let cookieCredentials: CookieCredentials;
let internalReqHeader: InternalRequestHeader;

// Helper function
Expand All @@ -38,7 +38,12 @@ export default ({ getService }: FtrProviderContext) => {
};
log.info(`sending request for query: ${JSON.stringify(job.locatorParams[0].params.query)}`);

return await reportingAPI.createReportJobInternal('csv_v2', job, roleAuthc, internalReqHeader);
return await reportingAPI.createReportJobInternal(
'csv_v2',
job,
cookieCredentials,
internalReqHeader
);
};

describe('CSV Generation from ES|QL', () => {
Expand Down Expand Up @@ -84,7 +89,7 @@ export default ({ getService }: FtrProviderContext) => {
};
before(async () => {
await loadTimelessData();
roleAuthc = await svlUserManager.createM2mApiKeyWithRoleScope('admin');
cookieCredentials = await samlAuth.getM2MApiCookieCredentialsWithRoleScope('admin');
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NOTE: cookie credentials do not need to be invalidated after the test

internalReqHeader = svlCommonApi.getInternalRequestHeader();
});

Expand Down Expand Up @@ -112,7 +117,7 @@ export default ({ getService }: FtrProviderContext) => {
},
],
}));
await reportingAPI.waitForJobToFinish(path, roleAuthc, internalReqHeader);
await reportingAPI.waitForJobToFinish(path, cookieCredentials, internalReqHeader);
response = await supertest.get(path);
csvFile = response.text;
});
Expand Down Expand Up @@ -184,7 +189,7 @@ export default ({ getService }: FtrProviderContext) => {
],
title: 'Untitled discover search',
}));
await reportingAPI.waitForJobToFinish(path, roleAuthc, internalReqHeader);
await reportingAPI.waitForJobToFinish(path, cookieCredentials, internalReqHeader);
response = await supertest.get(path);
csvFile = response.text;
});
Expand Down
16 changes: 11 additions & 5 deletions x-pack/test_serverless/api_integration/test_suites/common/reporting/datastream.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,12 @@
*/

import { expect } from 'expect';
import {
CookieCredentials,
InternalRequestHeader,
RoleCredentials,
} from '@kbn/ftr-common-functional-services';
import { FtrProviderContext } from '../../../ftr_provider_context';
import { InternalRequestHeader, RoleCredentials } from '../../../../shared/services';

export default function ({ getService }: FtrProviderContext) {
const esArchiver = getService('esArchiver');
Expand All @@ -17,6 +21,8 @@ export default function ({ getService }: FtrProviderContext) {
const supertestWithoutAuth = getService('supertestWithoutAuth');
const svlUserManager = getService('svlUserManager');
let roleAuthc: RoleCredentials;
const samlAuth = getService('samlAuth');
let cookieCredentials: CookieCredentials;
let internalReqHeader: InternalRequestHeader;

const archives: Record<string, { data: string; savedObjects: string }> = {
Expand All @@ -30,6 +36,7 @@ export default function ({ getService }: FtrProviderContext) {
const generatedReports = new Set<string>();
before(async () => {
roleAuthc = await svlUserManager.createM2mApiKeyWithRoleScope('admin');
cookieCredentials = await samlAuth.getM2MApiCookieCredentialsWithRoleScope('admin');
internalReqHeader = svlCommonApi.getInternalRequestHeader();

await esArchiver.load(archives.ecommerce.data);
Expand All @@ -48,7 +55,7 @@ export default function ({ getService }: FtrProviderContext) {
title: 'Ecommerce Data',
version: '8.15.0',
},
roleAuthc,
cookieCredentials,
internalReqHeader
);

Expand All @@ -57,19 +64,18 @@ export default function ({ getService }: FtrProviderContext) {

after(async () => {
for (const reportId of generatedReports) {
await reportingAPI.deleteReport(reportId, roleAuthc, internalReqHeader);
await reportingAPI.deleteReport(reportId, cookieCredentials, internalReqHeader);
}

await esArchiver.unload(archives.ecommerce.data);
await kibanaServer.importExport.unload(archives.ecommerce.savedObjects);
await svlUserManager.invalidateM2mApiKeyWithRoleScope(roleAuthc);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tim, you need to keep it and invalidate API key you created in line #38

});

it('uses the datastream configuration', async () => {
const { status, body } = await supertestWithoutAuth
.get(`/api/index_management/data_streams/.kibana-reporting`)
.set(internalReqHeader)
.set(roleAuthc.apiKeyHeader);
.set(roleAuthc.apiKeyHeader); // use API key since the datastream management API is a public endpoint

svlCommonApi.assertResponseStatusCode(200, status, body);

Expand Down
84 changes: 40 additions & 44 deletions x-pack/test_serverless/api_integration/test_suites/common/reporting/generate_csv_discover.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,16 +9,16 @@ import expect from '@kbn/expect';
import type { SortDirection } from '@kbn/data-plugin/common';
import type { JobParamsCSV } from '@kbn/reporting-export-types-csv-common';
import type { Filter } from '@kbn/es-query';
import { InternalRequestHeader, RoleCredentials } from '../../../../shared/services';
import { CookieCredentials, InternalRequestHeader } from '@kbn/ftr-common-functional-services';
import { FtrProviderContext } from '../../../ftr_provider_context';

export default function ({ getService }: FtrProviderContext) {
const esArchiver = getService('esArchiver');
const kibanaServer = getService('kibanaServer');
const reportingAPI = getService('svlReportingApi');
const svlCommonApi = getService('svlCommonApi');
const svlUserManager = getService('svlUserManager');
let roleAuthc: RoleCredentials;
const samlAuth = getService('samlAuth');
let cookieCredentials: CookieCredentials;
let internalReqHeader: InternalRequestHeader;

/*
Expand Down Expand Up @@ -79,7 +79,7 @@ export default function ({ getService }: FtrProviderContext) {
this.timeout(12 * 60 * 1000);

before(async () => {
roleAuthc = await svlUserManager.createM2mApiKeyWithRoleScope('admin');
cookieCredentials = await samlAuth.getM2MApiCookieCredentialsWithRoleScope('admin');
internalReqHeader = svlCommonApi.getInternalRequestHeader();
});

Expand All @@ -90,10 +90,6 @@ export default function ({ getService }: FtrProviderContext) {
});
});

after(async () => {
await svlUserManager.invalidateM2mApiKeyWithRoleScope(roleAuthc);
});

describe('exported CSV', () => {
before(async () => {
await esArchiver.load(archives.ecommerce.data);
Expand Down Expand Up @@ -169,13 +165,13 @@ export default function ({ getService }: FtrProviderContext) {
title: 'Ecommerce Data',
version: '8.14.0',
}),
roleAuthc,
cookieCredentials,
internalReqHeader
);
await reportingAPI.waitForJobToFinish(res.path, roleAuthc, internalReqHeader);
await reportingAPI.waitForJobToFinish(res.path, cookieCredentials, internalReqHeader);
const csvFile = await reportingAPI.getCompletedJobOutput(
res.path,
roleAuthc,
cookieCredentials,
internalReqHeader
);
expect((csvFile as string).length).to.be(124183);
Expand Down Expand Up @@ -212,11 +208,11 @@ export default function ({ getService }: FtrProviderContext) {
title: 'Untitled discover search',
version: '8.14.0',
}),
roleAuthc,
cookieCredentials,
internalReqHeader
);
await reportingAPI.waitForJobToFinish(res.path, roleAuthc, internalReqHeader);
return reportingAPI.getCompletedJobOutput(res.path, roleAuthc, internalReqHeader);
await reportingAPI.waitForJobToFinish(res.path, cookieCredentials, internalReqHeader);
return reportingAPI.getCompletedJobOutput(res.path, cookieCredentials, internalReqHeader);
}

it('includes an unmapped field to the report', async () => {
Expand Down Expand Up @@ -359,13 +355,13 @@ export default function ({ getService }: FtrProviderContext) {
},
},
}),
roleAuthc,
cookieCredentials,
internalReqHeader
);
await reportingAPI.waitForJobToFinish(res.path, roleAuthc, internalReqHeader);
await reportingAPI.waitForJobToFinish(res.path, cookieCredentials, internalReqHeader);
const csvFile = await reportingAPI.getCompletedJobOutput(
res.path,
roleAuthc,
cookieCredentials,
internalReqHeader
);
expect((csvFile as string).length).to.be(1270683);
Expand Down Expand Up @@ -411,13 +407,13 @@ export default function ({ getService }: FtrProviderContext) {
},
},
}),
roleAuthc,
cookieCredentials,
internalReqHeader
);
await reportingAPI.waitForJobToFinish(res.path, roleAuthc, internalReqHeader);
await reportingAPI.waitForJobToFinish(res.path, cookieCredentials, internalReqHeader);
const csvFile = await reportingAPI.getCompletedJobOutput(
res.path,
roleAuthc,
cookieCredentials,
internalReqHeader
);
expect((csvFile as string).length).to.be(918298);
Expand Down Expand Up @@ -469,13 +465,13 @@ export default function ({ getService }: FtrProviderContext) {
},
columns: ['@timestamp', 'clientip', 'extension'],
}),
roleAuthc,
cookieCredentials,
internalReqHeader
);
await reportingAPI.waitForJobToFinish(res.path, roleAuthc, internalReqHeader);
await reportingAPI.waitForJobToFinish(res.path, cookieCredentials, internalReqHeader);
const csvFile = await reportingAPI.getCompletedJobOutput(
res.path,
roleAuthc,
cookieCredentials,
internalReqHeader
);
expect((csvFile as string).length).to.be(3020);
Expand Down Expand Up @@ -515,13 +511,13 @@ export default function ({ getService }: FtrProviderContext) {
},
columns: ['@timestamp', 'clientip', 'extension'],
}),
roleAuthc,
cookieCredentials,
internalReqHeader
);
await reportingAPI.waitForJobToFinish(res.path, roleAuthc, internalReqHeader);
await reportingAPI.waitForJobToFinish(res.path, cookieCredentials, internalReqHeader);
const csvFile = await reportingAPI.getCompletedJobOutput(
res.path,
roleAuthc,
cookieCredentials,
internalReqHeader
);
expect((csvFile as string).length).to.be(3020);
Expand Down Expand Up @@ -555,13 +551,13 @@ export default function ({ getService }: FtrProviderContext) {
},
columns: ['date', 'message'],
}),
roleAuthc,
cookieCredentials,
internalReqHeader
);
await reportingAPI.waitForJobToFinish(res.path, roleAuthc, internalReqHeader);
await reportingAPI.waitForJobToFinish(res.path, cookieCredentials, internalReqHeader);
const csvFile = await reportingAPI.getCompletedJobOutput(
res.path,
roleAuthc,
cookieCredentials,
internalReqHeader
);
expect((csvFile as string).length).to.be(103);
Expand All @@ -584,13 +580,13 @@ export default function ({ getService }: FtrProviderContext) {
},
columns: ['date', 'message'],
}),
roleAuthc,
cookieCredentials,
internalReqHeader
);
await reportingAPI.waitForJobToFinish(res.path, roleAuthc, internalReqHeader);
await reportingAPI.waitForJobToFinish(res.path, cookieCredentials, internalReqHeader);
const csvFile = await reportingAPI.getCompletedJobOutput(
res.path,
roleAuthc,
cookieCredentials,
internalReqHeader
);
expect((csvFile as string).length).to.be(103);
Expand Down Expand Up @@ -627,13 +623,13 @@ export default function ({ getService }: FtrProviderContext) {
},
columns: ['date', 'message', '_id', '_index'],
}),
roleAuthc,
cookieCredentials,
internalReqHeader
);
await reportingAPI.waitForJobToFinish(res.path, roleAuthc, internalReqHeader);
await reportingAPI.waitForJobToFinish(res.path, cookieCredentials, internalReqHeader);
const csvFile = await reportingAPI.getCompletedJobOutput(
res.path,
roleAuthc,
cookieCredentials,
internalReqHeader
);
expect((csvFile as string).length).to.be(134);
Expand All @@ -659,13 +655,13 @@ export default function ({ getService }: FtrProviderContext) {
},
columns: ['name', 'power'],
}),
roleAuthc,
cookieCredentials,
internalReqHeader
);
await reportingAPI.waitForJobToFinish(res.path, roleAuthc, internalReqHeader);
await reportingAPI.waitForJobToFinish(res.path, cookieCredentials, internalReqHeader);
const csvFile = await reportingAPI.getCompletedJobOutput(
res.path,
roleAuthc,
cookieCredentials,
internalReqHeader
);
expect((csvFile as string).length).to.be(274);
Expand Down Expand Up @@ -743,13 +739,13 @@ export default function ({ getService }: FtrProviderContext) {
},
columns: [],
}),
roleAuthc,
cookieCredentials,
internalReqHeader
);
await reportingAPI.waitForJobToFinish(res.path, roleAuthc, internalReqHeader);
await reportingAPI.waitForJobToFinish(res.path, cookieCredentials, internalReqHeader);
const csvFile = await reportingAPI.getCompletedJobOutput(
res.path,
roleAuthc,
cookieCredentials,
internalReqHeader
);
expect((csvFile as string).length).to.be(356);
Expand Down Expand Up @@ -809,13 +805,13 @@ export default function ({ getService }: FtrProviderContext) {
},
},
}),
roleAuthc,
cookieCredentials,
internalReqHeader
);
await reportingAPI.waitForJobToFinish(res.path, roleAuthc, internalReqHeader);
await reportingAPI.waitForJobToFinish(res.path, cookieCredentials, internalReqHeader);
const csvFile = await reportingAPI.getCompletedJobOutput(
res.path,
roleAuthc,
cookieCredentials,
internalReqHeader
);
expect((csvFile as string).length).to.be(4845684);
Expand Down
Loading