Skip to content

Commit

Permalink
[8.9] [FTR] Implement browser network condition utils (elastic#163633) (
Browse files Browse the repository at this point in the history
elastic#163703)

# Backport

This will backport the following commits from `main` to `8.9`:
- [[FTR] Implement browser network condition utils
(elastic#163633)](elastic#163633)

<!--- Backport version: 8.9.7 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Marco Antonio
Ghiani","email":"[email protected]"},"sourceCommit":{"committedDate":"2023-08-11T12:24:06Z","message":"[FTR]
Implement browser network condition utils (elastic#163633)\n\n## 📓
Summary\r\n\r\nThe PR implements some utilities into the `browser`
service to allow\r\ncontrolling the network conditions during the
execution of a functional\r\ntest.\r\n\r\n###
`getNetworkConditions`\r\n\r\nReturns the current network simulation
options. If none conditions are\r\npreviously set, it returns
`undefined`\r\n\r\n**N.B.**: _if the testing environment is not a
Chromium browser, it\r\nthrows an error that can be easily caught to
manually skip the test or\r\nhandle a fallback
scenario._\r\n\r\n\r\n```ts\r\nit('should display a loading skeleton
while loading', async function () {\r\n // Skip the test in case network
condition utils are not available\r\n try {\r\n const networkConditions
= await browser.getNetworkConditions(); // undefined\r\n\r\n await
browser.setNetworkConditions('SLOW_3G');\r\n\r\n const networkConditions
= await browser.getNetworkConditions();\r\n // {\r\n // offline:
false,\r\n // latency: 2000,\r\n // download_throughput: 50000,\r\n //
upload_throughput: 50000,\r\n // }\r\n } catch (error) {\r\n
this.skip();\r\n }\r\n});\r\n```\r\n\r\n###
`setNetworkConditions`\r\n\r\nSet the desired network conditions.\r\nIt
supports different presets that match the [network profiles
provided\r\nby
Chrome\r\ndebugger](https://github.com/ChromeDevTools/devtools-frontend/blob/da276a3faec9769cb55e442f0db77ebdce5cd178/front_end/core/sdk/NetworkManager.ts#L363-L393):\r\n-
`NO_THROTTLING`\r\n- `FAST_3G`\r\n- `SLOW_3G`\r\n- `OFFLINE`\r\n-
`CLOUD_USER` (pre-existing)\r\nIt also accepts ad-hoc options to
configure more specifically the\r\nnetwork conditions.\r\n\r\n**N.B.**:
_if the testing environment is not a Chromium browser, it\r\nthrows an
error that can be easily caught to manually skip the test or\r\nhandle a
fallback scenario._\r\n\r\n```ts\r\nit('should display a loading
skeleton while loading', async function () {\r\n // Skip the test in
case network condition utils are not available\r\n try {\r\n await
browser.setNetworkConditions('NO_THROTTLING');\r\n await
browser.setNetworkConditions('FAST_3G');\r\n await
browser.setNetworkConditions('SLOW_3G');\r\n await
browser.setNetworkConditions('OFFLINE');\r\n await
browser.setNetworkConditions('CLOUD_USER');\r\n await
browser.setNetworkConditions({\r\n offline: false,\r\n latency: 5, //
Additional latency (ms).\r\n download_throughput: 500 * 1024, // Maximal
aggregated download throughput.\r\n upload_throughput: 500 * 1024, //
Maximal aggregated upload throughput.\r\n });\r\n } catch (error) {\r\n
this.skip();\r\n }\r\n});\r\n```\r\n\r\n###
restoreNetworkConditions\r\n\r\nRestore the original network conditions,
setting to `NO_THROTTLING`.\r\nThe native implementation of
`deleteNetworkConditions` exposed by\r\nselenium is unofficial and
didn't consistently work, the recommended\r\napproach by the google dev
tools team is to restore the connection\r\nsetting the no throttling
profile.\r\n\r\n**N.B.**: _if the testing environment is not a Chromium
browser, it\r\nthrows an error that can be easily caught to manually
skip the test or\r\nhandle a fallback
scenario._\r\n\r\n```ts\r\nit('should display a loading skeleton while
loading', async function () {\r\n // Skip the test in case network
condition utils are not available\r\n try {\r\n await
browser.setNetworkConditions('SLOW_3G'); // Slow down network
conditions\r\n \r\n // Do your assertions\r\n\r\n await
browser.restoreNetworkConditions(); // Restore network conditions\r\n }
catch (error) {\r\n this.skip();\r\n
}\r\n});\r\n```\r\n\r\nCo-authored-by: Marco Antonio Ghiani
<[email protected]>\r\nCo-authored-by: Dzmitry Lemechko
<[email protected]>","sha":"aa45152a4e787f36014675b864fcc0fce7bd6758","branchLabelMapping":{"^v8.10.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v8.10.0","v7.17.13","v8.9.2"],"number":163633,"url":"https://github.com/elastic/kibana/pull/163633","mergeCommit":{"message":"[FTR]
Implement browser network condition utils (elastic#163633)\n\n## 📓
Summary\r\n\r\nThe PR implements some utilities into the `browser`
service to allow\r\ncontrolling the network conditions during the
execution of a functional\r\ntest.\r\n\r\n###
`getNetworkConditions`\r\n\r\nReturns the current network simulation
options. If none conditions are\r\npreviously set, it returns
`undefined`\r\n\r\n**N.B.**: _if the testing environment is not a
Chromium browser, it\r\nthrows an error that can be easily caught to
manually skip the test or\r\nhandle a fallback
scenario._\r\n\r\n\r\n```ts\r\nit('should display a loading skeleton
while loading', async function () {\r\n // Skip the test in case network
condition utils are not available\r\n try {\r\n const networkConditions
= await browser.getNetworkConditions(); // undefined\r\n\r\n await
browser.setNetworkConditions('SLOW_3G');\r\n\r\n const networkConditions
= await browser.getNetworkConditions();\r\n // {\r\n // offline:
false,\r\n // latency: 2000,\r\n // download_throughput: 50000,\r\n //
upload_throughput: 50000,\r\n // }\r\n } catch (error) {\r\n
this.skip();\r\n }\r\n});\r\n```\r\n\r\n###
`setNetworkConditions`\r\n\r\nSet the desired network conditions.\r\nIt
supports different presets that match the [network profiles
provided\r\nby
Chrome\r\ndebugger](https://github.com/ChromeDevTools/devtools-frontend/blob/da276a3faec9769cb55e442f0db77ebdce5cd178/front_end/core/sdk/NetworkManager.ts#L363-L393):\r\n-
`NO_THROTTLING`\r\n- `FAST_3G`\r\n- `SLOW_3G`\r\n- `OFFLINE`\r\n-
`CLOUD_USER` (pre-existing)\r\nIt also accepts ad-hoc options to
configure more specifically the\r\nnetwork conditions.\r\n\r\n**N.B.**:
_if the testing environment is not a Chromium browser, it\r\nthrows an
error that can be easily caught to manually skip the test or\r\nhandle a
fallback scenario._\r\n\r\n```ts\r\nit('should display a loading
skeleton while loading', async function () {\r\n // Skip the test in
case network condition utils are not available\r\n try {\r\n await
browser.setNetworkConditions('NO_THROTTLING');\r\n await
browser.setNetworkConditions('FAST_3G');\r\n await
browser.setNetworkConditions('SLOW_3G');\r\n await
browser.setNetworkConditions('OFFLINE');\r\n await
browser.setNetworkConditions('CLOUD_USER');\r\n await
browser.setNetworkConditions({\r\n offline: false,\r\n latency: 5, //
Additional latency (ms).\r\n download_throughput: 500 * 1024, // Maximal
aggregated download throughput.\r\n upload_throughput: 500 * 1024, //
Maximal aggregated upload throughput.\r\n });\r\n } catch (error) {\r\n
this.skip();\r\n }\r\n});\r\n```\r\n\r\n###
restoreNetworkConditions\r\n\r\nRestore the original network conditions,
setting to `NO_THROTTLING`.\r\nThe native implementation of
`deleteNetworkConditions` exposed by\r\nselenium is unofficial and
didn't consistently work, the recommended\r\napproach by the google dev
tools team is to restore the connection\r\nsetting the no throttling
profile.\r\n\r\n**N.B.**: _if the testing environment is not a Chromium
browser, it\r\nthrows an error that can be easily caught to manually
skip the test or\r\nhandle a fallback
scenario._\r\n\r\n```ts\r\nit('should display a loading skeleton while
loading', async function () {\r\n // Skip the test in case network
condition utils are not available\r\n try {\r\n await
browser.setNetworkConditions('SLOW_3G'); // Slow down network
conditions\r\n \r\n // Do your assertions\r\n\r\n await
browser.restoreNetworkConditions(); // Restore network conditions\r\n }
catch (error) {\r\n this.skip();\r\n
}\r\n});\r\n```\r\n\r\nCo-authored-by: Marco Antonio Ghiani
<[email protected]>\r\nCo-authored-by: Dzmitry Lemechko
<[email protected]>","sha":"aa45152a4e787f36014675b864fcc0fce7bd6758"}},"sourceBranch":"main","suggestedTargetBranches":["7.17","8.9"],"targetPullRequestStates":[{"branch":"main","label":"v8.10.0","labelRegex":"^v8.10.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/163633","number":163633,"mergeCommit":{"message":"[FTR]
Implement browser network condition utils (elastic#163633)\n\n## 📓
Summary\r\n\r\nThe PR implements some utilities into the `browser`
service to allow\r\ncontrolling the network conditions during the
execution of a functional\r\ntest.\r\n\r\n###
`getNetworkConditions`\r\n\r\nReturns the current network simulation
options. If none conditions are\r\npreviously set, it returns
`undefined`\r\n\r\n**N.B.**: _if the testing environment is not a
Chromium browser, it\r\nthrows an error that can be easily caught to
manually skip the test or\r\nhandle a fallback
scenario._\r\n\r\n\r\n```ts\r\nit('should display a loading skeleton
while loading', async function () {\r\n // Skip the test in case network
condition utils are not available\r\n try {\r\n const networkConditions
= await browser.getNetworkConditions(); // undefined\r\n\r\n await
browser.setNetworkConditions('SLOW_3G');\r\n\r\n const networkConditions
= await browser.getNetworkConditions();\r\n // {\r\n // offline:
false,\r\n // latency: 2000,\r\n // download_throughput: 50000,\r\n //
upload_throughput: 50000,\r\n // }\r\n } catch (error) {\r\n
this.skip();\r\n }\r\n});\r\n```\r\n\r\n###
`setNetworkConditions`\r\n\r\nSet the desired network conditions.\r\nIt
supports different presets that match the [network profiles
provided\r\nby
Chrome\r\ndebugger](https://github.com/ChromeDevTools/devtools-frontend/blob/da276a3faec9769cb55e442f0db77ebdce5cd178/front_end/core/sdk/NetworkManager.ts#L363-L393):\r\n-
`NO_THROTTLING`\r\n- `FAST_3G`\r\n- `SLOW_3G`\r\n- `OFFLINE`\r\n-
`CLOUD_USER` (pre-existing)\r\nIt also accepts ad-hoc options to
configure more specifically the\r\nnetwork conditions.\r\n\r\n**N.B.**:
_if the testing environment is not a Chromium browser, it\r\nthrows an
error that can be easily caught to manually skip the test or\r\nhandle a
fallback scenario._\r\n\r\n```ts\r\nit('should display a loading
skeleton while loading', async function () {\r\n // Skip the test in
case network condition utils are not available\r\n try {\r\n await
browser.setNetworkConditions('NO_THROTTLING');\r\n await
browser.setNetworkConditions('FAST_3G');\r\n await
browser.setNetworkConditions('SLOW_3G');\r\n await
browser.setNetworkConditions('OFFLINE');\r\n await
browser.setNetworkConditions('CLOUD_USER');\r\n await
browser.setNetworkConditions({\r\n offline: false,\r\n latency: 5, //
Additional latency (ms).\r\n download_throughput: 500 * 1024, // Maximal
aggregated download throughput.\r\n upload_throughput: 500 * 1024, //
Maximal aggregated upload throughput.\r\n });\r\n } catch (error) {\r\n
this.skip();\r\n }\r\n});\r\n```\r\n\r\n###
restoreNetworkConditions\r\n\r\nRestore the original network conditions,
setting to `NO_THROTTLING`.\r\nThe native implementation of
`deleteNetworkConditions` exposed by\r\nselenium is unofficial and
didn't consistently work, the recommended\r\napproach by the google dev
tools team is to restore the connection\r\nsetting the no throttling
profile.\r\n\r\n**N.B.**: _if the testing environment is not a Chromium
browser, it\r\nthrows an error that can be easily caught to manually
skip the test or\r\nhandle a fallback
scenario._\r\n\r\n```ts\r\nit('should display a loading skeleton while
loading', async function () {\r\n // Skip the test in case network
condition utils are not available\r\n try {\r\n await
browser.setNetworkConditions('SLOW_3G'); // Slow down network
conditions\r\n \r\n // Do your assertions\r\n\r\n await
browser.restoreNetworkConditions(); // Restore network conditions\r\n }
catch (error) {\r\n this.skip();\r\n
}\r\n});\r\n```\r\n\r\nCo-authored-by: Marco Antonio Ghiani
<[email protected]>\r\nCo-authored-by: Dzmitry Lemechko
<[email protected]>","sha":"aa45152a4e787f36014675b864fcc0fce7bd6758"}},{"branch":"7.17","label":"v7.17.13","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.9","label":"v8.9.2","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Marco Antonio Ghiani <[email protected]>
  • Loading branch information
kibanamachine and tonyghiani authored Aug 11, 2023
1 parent 2bc5386 commit 5c5c2da
Show file tree
Hide file tree
Showing 4 changed files with 117 additions and 29 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ export default function ({ getService, getPageObjects }: FtrProviderContext) {
expect(event.properties.value4).to.be.a('number');
expect(event.properties.value5).to.be.a('number');

if (browser.isChromium) {
if (browser.isChromium()) {
// Kibana Loaded memory
expect(meta).to.have.property('jsHeapSizeLimit');
expect(meta.jsHeapSizeLimit).to.be.a('number');
Expand Down
75 changes: 70 additions & 5 deletions test/functional/services/common/browser.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,15 +7,17 @@
*/

import { setTimeout as setTimeoutAsync } from 'timers/promises';
import { cloneDeepWith } from 'lodash';
import { cloneDeepWith, isString } from 'lodash';
import { Key, Origin, WebDriver } from 'selenium-webdriver';
import { Driver as ChromiumWebDriver } from 'selenium-webdriver/chrome';
import { modifyUrl } from '@kbn/std';

import sharp from 'sharp';
import { NoSuchSessionError } from 'selenium-webdriver/lib/error';
import { WebElementWrapper } from '../lib/web_element_wrapper';
import { FtrProviderContext, FtrService } from '../../ftr_provider_context';
import { Browsers } from '../remote/browsers';
import { NetworkOptions, NetworkProfile, NETWORK_PROFILES } from '../remote/network_profiles';

export type Browser = BrowserService;

Expand All @@ -25,19 +27,20 @@ class BrowserService extends FtrService {
*/
public readonly keys = Key;
public readonly isFirefox: boolean;
public readonly isChromium: boolean;

private readonly log = this.ctx.getService('log');

constructor(
ctx: FtrProviderContext,
public readonly browserType: string,
private readonly driver: WebDriver
protected readonly driver: WebDriver | ChromiumWebDriver
) {
super(ctx);
this.isFirefox = this.browserType === Browsers.Firefox;
this.isChromium =
this.browserType === Browsers.Chrome || this.browserType === Browsers.ChromiumEdge;
}

public isChromium(): this is { driver: ChromiumWebDriver } {
return this.driver instanceof ChromiumWebDriver;
}

/**
Expand Down Expand Up @@ -661,6 +664,68 @@ class BrowserService extends FtrService {
}
}
}

/**
* Get the network simulation for chromium browsers if available.
* https://www.selenium.dev/selenium/docs/api/javascript/module/selenium-webdriver/chrome_exports_Driver.html#getNetworkConditions
*
* @return {Promise<NetworkOptions>}
*/
public async getNetworkConditions() {
if (this.isChromium()) {
return this.driver.getNetworkConditions().catch(() => undefined); // Return undefined instead of throwing if no conditions are set.
} else {
const message =
'WebDriver does not support the .getNetworkConditions method.\nProbably the browser in used is not chromium based.';
this.log.error(message);
throw new Error(message);
}
}

/**
* Delete the network simulation for chromium browsers if available.
*
* @return {Promise<void>}
*/
public async restoreNetworkConditions() {
this.log.debug('Restore network conditions simulation.');
return this.setNetworkConditions('NO_THROTTLING');
}

/**
* Set the network conditions for chromium browsers if available.
*
* __Sample Usage:__
*
* browser.setNetworkConditions('FAST_3G')
* browser.setNetworkConditions('SLOW_3G')
* browser.setNetworkConditions('OFFLINE')
* browser.setNetworkConditions({
* offline: false,
* latency: 5, // Additional latency (ms).
* download_throughput: 500 * 1024, // Maximal aggregated download throughput.
* upload_throughput: 500 * 1024, // Maximal aggregated upload throughput.
* });
*
* https://www.selenium.dev/selenium/docs/api/javascript/module/selenium-webdriver/chrome_exports_Driver.html#setNetworkConditions
*
* @return {Promise<void>}
*/
public async setNetworkConditions(profileOrOptions: NetworkProfile | NetworkOptions) {
const networkOptions = isString(profileOrOptions)
? NETWORK_PROFILES[profileOrOptions]
: profileOrOptions;

if (this.isChromium()) {
this.log.debug(`Set network conditions with profile "${profileOrOptions}".`);
return this.driver.setNetworkConditions(networkOptions);
} else {
const message =
'WebDriver does not support the .setNetworkCondition method.\nProbably the browser in used is not chromium based.';
this.log.error(message);
throw new Error(message);
}
}
}

export async function BrowserProvider(ctx: FtrProviderContext) {
Expand Down
45 changes: 39 additions & 6 deletions test/functional/services/remote/network_profiles.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,17 +6,50 @@
* Side Public License, v 1.
*/

interface NetworkOptions {
DOWNLOAD: number;
UPLOAD: number;
LATENCY: number;
export type NetworkProfile = 'NO_THROTTLING' | 'FAST_3G' | 'SLOW_3G' | 'OFFLINE' | 'CLOUD_USER';

export interface NetworkOptions {
offline: boolean;
latency: number;
download_throughput: number;
upload_throughput: number;
}

const sec = 10 ** 3;
const MBps = 10 ** 6 / 8; // megabyte per second (MB/s) (can be abbreviated as MBps)

// Selenium uses B/s (bytes) for network throttling
// Download (B/s) Upload (B/s) Latency (ms)
export const NETWORK_PROFILES: { [key: string]: NetworkOptions } = {
CLOUD_USER: { DOWNLOAD: 6 * MBps, UPLOAD: 6 * MBps, LATENCY: 0.1 * sec },

export const NETWORK_PROFILES: Record<NetworkProfile, NetworkOptions> = {
NO_THROTTLING: {
offline: false,
latency: 0,
download_throughput: -1,
upload_throughput: -1,
},
FAST_3G: {
offline: false,
latency: 0.56 * sec,
download_throughput: 1.44 * MBps,
upload_throughput: 0.7 * MBps,
},
SLOW_3G: {
offline: false,
latency: 2 * sec,
download_throughput: 0.4 * MBps,
upload_throughput: 0.4 * MBps,
},
OFFLINE: {
offline: true,
latency: 0,
download_throughput: 0,
upload_throughput: 0,
},
CLOUD_USER: {
offline: false,
latency: 0.1 * sec,
download_throughput: 6 * MBps,
upload_throughput: 6 * MBps,
},
};
24 changes: 7 additions & 17 deletions test/functional/services/remote/webdriver.ts
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ import { createStdoutSocket } from './create_stdout_stream';
import { preventParallelCalls } from './prevent_parallel_calls';

import { Browsers } from './browsers';
import { NETWORK_PROFILES } from './network_profiles';
import { NetworkProfile, NETWORK_PROFILES } from './network_profiles';

const throttleOption: string = process.env.TEST_THROTTLE_NETWORK as string;
const headlessBrowser: string = process.env.TEST_BROWSER_HEADLESS as string;
Expand Down Expand Up @@ -300,22 +300,17 @@ async function attemptToCreateCommand(
const { session, consoleLog$ } = await buildDriverInstance();

if (throttleOption === '1' && browserType === 'chrome') {
const { KBN_NETWORK_TEST_PROFILE = 'CLOUD_USER' } = process.env;
const KBN_NETWORK_TEST_PROFILE = (process.env.KBN_NETWORK_TEST_PROFILE ??
'CLOUD_USER') as NetworkProfile;

const profile =
KBN_NETWORK_TEST_PROFILE in Object.keys(NETWORK_PROFILES)
? KBN_NETWORK_TEST_PROFILE
: 'CLOUD_USER';
KBN_NETWORK_TEST_PROFILE in NETWORK_PROFILES ? KBN_NETWORK_TEST_PROFILE : 'CLOUD_USER';

const {
DOWNLOAD: downloadThroughput,
UPLOAD: uploadThroughput,
LATENCY: latency,
} = NETWORK_PROFILES[`${profile}`];
const networkProfileOptions = NETWORK_PROFILES[profile];

// Only chrome supports this option.
log.debug(
`NETWORK THROTTLED with profile ${profile}: ${downloadThroughput} B/s down, ${uploadThroughput} B/s up, ${latency} ms latency.`
`NETWORK THROTTLED with profile ${profile}: ${networkProfileOptions.download_throughput} B/s down, ${networkProfileOptions.upload_throughput} B/s up, ${networkProfileOptions.latency} ms latency.`
);

if (noCache) {
Expand All @@ -326,12 +321,7 @@ async function attemptToCreateCommand(
}

// @ts-expect-error
session.setNetworkConditions({
offline: false,
latency,
download_throughput: downloadThroughput,
upload_throughput: uploadThroughput,
});
session.setNetworkConditions(networkProfileOptions);
}

if (attemptId !== attemptCounter) {
Expand Down

0 comments on commit 5c5c2da

Please sign in to comment.