Skip to content

Commit

Permalink
Merge branch 'master' into unskip-flaky-tests
Browse files Browse the repository at this point in the history
  • Loading branch information
elasticmachine authored Feb 18, 2020
2 parents 4cd9d9e + 76d475a commit 9f866ed
Show file tree
Hide file tree
Showing 109 changed files with 1,066 additions and 528 deletions.
48 changes: 14 additions & 34 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,11 +55,9 @@ Granted that you share your thoughts, we might even be able to come up with crea

First of all, **sorry about that!** We want you to have a great time with Kibana.

Hosting meaningful discussions on GitHub can be challenging. For that reason, we'll sometimes ask that you join us on IRC _([#kibana](https://kiwiirc.com/client/irc.freenode.net/?#kibana) on freenode)_ to chat about your issues. You may also experience **faster response times** when engaging us via IRC.

There's hundreds of open issues and prioritizing what to work on is an important aspect of our daily jobs. We prioritize issues according to impact and difficulty, so some issues can be neglected while we work on more pressing issues.

Feel free to bump your issues if you think they've been neglected for a prolonged period, or just jump on IRC and let us have it!
Feel free to bump your issues if you think they've been neglected for a prolonged period.

### "I want to help!"

Expand Down Expand Up @@ -471,11 +469,11 @@ The following table outlines possible test file locations and how to invoke them

| Test runner | Test location | Runner command (working directory is kibana root) |
| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------- |
| Jest | `src/**/*.test.js`<br>`src/**/*.test.ts` | `node scripts/jest -t regexp [test path]` |
| Jest (integration) | `**/integration_tests/**/*.test.js` | `node scripts/jest_integration -t regexp [test path]` |
| Jest | `src/**/*.test.js`<br>`src/**/*.test.ts` | `yarn test:jest -t regexp [test path]` |
| Jest (integration) | `**/integration_tests/**/*.test.js` | `yarn test:jest_integration -t regexp [test path]` |
| Mocha | `src/**/__tests__/**/*.js`<br>`!src/**/public/__tests__/*.js`<br>`packages/kbn-datemath/test/**/*.js`<br>`packages/kbn-dev-utils/src/**/__tests__/**/*.js`<br>`tasks/**/__tests__/**/*.js` | `node scripts/mocha --grep=regexp [test path]` |
| Functional | `test/*integration/**/config.js`<br>`test/*functional/**/config.js`<br>`test/accessibility/config.js` | `node scripts/functional_tests_server --config test/[directory]/config.js`<br>`node scripts/functional_test_runner --config test/[directory]/config.js --grep=regexp` |
| Karma | `src/**/public/__tests__/*.js` | `npm run test:dev` |
| Functional | `test/*integration/**/config.js`<br>`test/*functional/**/config.js`<br>`test/accessibility/config.js` | `yarn test:ftr:server --config test/[directory]/config.js`<br>`yarn test:ftr:runner --config test/[directory]/config.js --grep=regexp` |
| Karma | `src/**/public/__tests__/*.js` | `yarn test:karma:debug` |

For X-Pack tests located in `x-pack/` see [X-Pack Testing](x-pack/README.md#testing)

Expand All @@ -486,56 +484,38 @@ Test runner arguments:
Examples:
- Run the entire elasticsearch_service test suite:
```
node scripts/jest src/core/server/elasticsearch/elasticsearch_service.test.ts
yarn test:jest src/core/server/elasticsearch/elasticsearch_service.test.ts
```
- Run the jest test case whose description matches `stops both admin and data clients`:
```
node scripts/jest -t 'stops both admin and data clients' src/core/server/elasticsearch/elasticsearch_service.test.ts
yarn test:jest -t 'stops both admin and data clients' src/core/server/elasticsearch/elasticsearch_service.test.ts
```
- Run the api integration test case whose description matches the given string:
```
node scripts/functional_tests_server --config test/api_integration/config.js
node scripts/functional_test_runner --config test/api_integration/config.js --grep='should return 404 if id does not match any sample data sets'
yarn test:ftr:server --config test/api_integration/config.js
yarn test:ftr:runner --config test/api_integration/config.js --grep='should return 404 if id does not match any sample data sets'
```
### Debugging Unit Tests
The standard `yarn test` task runs several sub tasks and can take several minutes to complete, making debugging failures pretty painful. In order to ease the pain specialized tasks provide alternate methods for running the tests.
To execute both server and browser tests, but skip linting, use `yarn test:quick`.
```bash
yarn test:quick
```

Use `yarn test:mocha` when you want to run the mocha tests.

```bash
yarn test:mocha
```

When you'd like to execute individual server-side test files, you can use the command below. Note that this command takes care of configuring Mocha with Babel compilation for you, and you'll be better off avoiding a globally installed `mocha` package. This command is great for development and for quickly identifying bugs.

```bash
node scripts/mocha <file>
```

You could also add the `--debug` option so that `node` is run using the `--debug-brk` flag. You'll need to connect a remote debugger such as [`node-inspector`](https://github.com/node-inspector/node-inspector) to proceed in this mode.
```bash
node scripts/mocha --debug <file>
```

With `yarn test:browser`, you can run only the browser tests. Coverage reports are available for browser tests by running `yarn test:coverage`. You can find the results under the `coverage/` directory that will be created upon completion.
With `yarn test:karma`, you can run only the browser tests. Coverage reports are available for browser tests by running `yarn test:coverage`. You can find the results under the `coverage/` directory that will be created upon completion.

```bash
yarn test:browser
yarn test:karma
```

Using `yarn test:dev` initializes an environment for debugging the browser tests. Includes an dedicated instance of the kibana server for building the test bundle, and a karma server. When running this task the build is optimized for the first time and then a karma-owned instance of the browser is opened. Click the "debug" button to open a new tab that executes the unit tests.
Using `yarn test:karma:debug` initializes an environment for debugging the browser tests. Includes an dedicated instance of the kibana server for building the test bundle, and a karma server. When running this task the build is optimized for the first time and then a karma-owned instance of the browser is opened. Click the "debug" button to open a new tab that executes the unit tests.

```bash
yarn test:dev
yarn test:karma:debug
```

In the screenshot below, you'll notice the URL is `localhost:9876/debug.html`. You can append a `grep` query parameter to this URL and set it to a string value which will be used to exclude tests which don't match. For example, if you changed the URL to `localhost:9876/debug.html?query=my test` and then refreshed the browser, you'd only see tests run which contain "my test" in the test description.
Expand All @@ -551,7 +531,7 @@ To run the tests for just your particular plugin run the following command from

```bash
yarn test:mocha
yarn test:browser --dev # remove the --dev flag to run them once and close
yarn test:karma:debug # remove the debug flag to run them once and close
```

### Automated Accessibility Testing
Expand Down
12 changes: 5 additions & 7 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -40,23 +40,21 @@
"kbn": "node scripts/kbn",
"es": "node scripts/es",
"test": "grunt test",
"test:dev": "grunt test:dev",
"test:quick": "grunt test:quick",
"test:browser": "grunt test:browser",
"test:karma": "grunt test:karma",
"test:karma:debug": "grunt test:karmaDebug",
"test:jest": "node scripts/jest",
"test:mocha": "node scripts/mocha",
"test:mocha:coverage": "grunt test:mochaCoverage",
"test:ui": "node scripts/functional_tests",
"test:ui:server": "node scripts/functional_tests_server",
"test:ui:runner": "node scripts/functional_test_runner",
"test:ftr": "node scripts/functional_tests",
"test:ftr:server": "node scripts/functional_tests_server",
"test:ftr:runner": "node scripts/functional_test_runner",
"test:coverage": "grunt test:coverage",
"typespec": "typings-tester --config x-pack/legacy/plugins/canvas/public/lib/aeroelastic/tsconfig.json x-pack/legacy/plugins/canvas/public/lib/aeroelastic/__fixtures__/typescript/typespec_tests.ts",
"checkLicenses": "node scripts/check_licenses --dev",
"build": "node scripts/build --all-platforms",
"start": "node --trace-warnings --throw-deprecation scripts/kibana --dev",
"debug": "node --nolazy --inspect scripts/kibana --dev",
"debug-break": "node --nolazy --inspect-brk scripts/kibana --dev",
"karma": "karma start",
"lint": "yarn run lint:es && yarn run lint:sass",
"lint:es": "node scripts/eslint",
"lint:sass": "node scripts/sasslint",
Expand Down
2 changes: 1 addition & 1 deletion packages/kbn-plugin-generator/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ Generated plugins receive a handful of scripts that can be used during developme
Build a distributable archive of your plugin.
- `yarn test:browser`
- `yarn test:karma`
Run the browser tests in a real web browser.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -70,8 +70,8 @@ describe(`running the plugin-generator via 'node scripts/generate_plugin.js plug
});

describe(`then running`, () => {
it(`'yarn test:browser' should exit 0`, async () => {
await execa('yarn', ['test:browser'], {
it(`'yarn test:karma' should exit 0`, async () => {
await execa('yarn', ['test:karma'], {
cwd: generatedPath,
env: {
DISABLE_JUNIT_REPORTER: '1',
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,3 @@
## Development

See the [kibana contributing guide](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md) for instructions setting up your development environment.

4 changes: 2 additions & 2 deletions packages/kbn-plugin-helpers/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,8 @@ $ plugin-helpers help
start Start kibana and have it include this plugin
build [options] [files...] Build a distributable archive
test Run the server and browser tests
test:browser [options] Run the browser tests in a real web browser
test:server [files...] Run the server tests using mocha
test:karma [options] Run the browser tests in a real web browser
test:mocha [files...] Run the server tests using mocha

Options:

Expand Down
12 changes: 6 additions & 6 deletions packages/kbn-plugin-helpers/cli.js
Original file line number Diff line number Diff line change
Expand Up @@ -66,24 +66,24 @@ program
.action(createCommanderAction('testAll'));

program
.command('test:browser')
.command('test:karma')
.description('Run the browser tests in a real web browser')
.option('--dev', 'Enable dev mode, keeps the test server running')
.option('-p, --plugins <plugin-ids>', "Manually specify which plugins' test bundles to run")
.on('--help', docs('test/browser'))
.on('--help', docs('test/karma'))
.action(
createCommanderAction('testBrowser', command => ({
createCommanderAction('testKarma', command => ({
dev: Boolean(command.dev),
plugins: command.plugins,
}))
);

program
.command('test:server [files...]')
.command('test:mocha [files...]')
.description('Run the server tests using mocha')
.on('--help', docs('test/server'))
.on('--help', docs('test/mocha'))
.action(
createCommanderAction('testServer', (command, files) => ({
createCommanderAction('testMocha', (command, files) => ({
files: files,
}))
);
Expand Down
2 changes: 1 addition & 1 deletion packages/kbn-plugin-helpers/lib/index.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,6 @@ export function babelRegister(): void;
export function resolveKibanaPath(path: string): string;
export function readFtrConfigFile(path: string): any;
export function run(
task: 'build' | 'start' | 'testAll' | 'testBrowser' | 'testServer' | 'postinstall',
task: 'build' | 'start' | 'testAll' | 'testKarma' | 'testMocha' | 'postinstall',
options: any
): Promise<void>;
8 changes: 4 additions & 4 deletions packages/kbn-plugin-helpers/lib/tasks.js
Original file line number Diff line number Diff line change
Expand Up @@ -20,15 +20,15 @@
const buildTask = require('../tasks/build');
const startTask = require('../tasks/start');
const testAllTask = require('../tasks/test/all');
const testBrowserTask = require('../tasks/test/browser');
const testServerTask = require('../tasks/test/server');
const testKarmaTask = require('../tasks/test/karma');
const testMochaTask = require('../tasks/test/mocha');
const postinstallTask = require('../tasks/postinstall');

module.exports = {
build: buildTask,
start: startTask,
testAll: testAllTask,
testBrowser: testBrowserTask,
testServer: testServerTask,
testKarma: testKarmaTask,
testMocha: testMochaTask,
postinstall: postinstallTask,
};
4 changes: 2 additions & 2 deletions packages/kbn-plugin-helpers/tasks/test/all/README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
Runs both the server and browser tests, in that order.
Runs both the mocha and karma tests, in that order.

This is just a simple caller to both `test/server` and `test/browser`
This is just a simple caller to both `test/mocha` and `test/karma`
4 changes: 2 additions & 2 deletions packages/kbn-plugin-helpers/tasks/test/all/test_all_action.js
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,6 @@
*/

module.exports = function testAllAction(plugin, run) {
run('testServer');
run('testBrowser');
run('testMocha');
run('testKarma');
};
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,12 @@ Browser tests are written just like server tests, they are just executed differe
starting the test runner
========================

Under the covers this command uses the `test:browser` task from kibana. This will execute
Under the covers this command uses the `test:karma` task from kibana. This will execute
your tasks once and exit when complete.

When run with the `--dev` option, the command uses the `test:dev` task from kibana.
This task sets-up a test runner that will watch your code for changes and rebuild your
tests when necessary. You access the test runner through a browser that it starts itself
When run with the `--dev` option, the command uses the `test:karma:debug` task from kibana.
This task sets-up a test runner that will watch your code for changes and rebuild your
tests when necessary. You access the test runner through a browser that it starts itself
(via Karma).

If your plugin consists of a number of internal plugins, you may wish to keep the tests
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,4 +17,4 @@
* under the License.
*/

module.exports = require('./test_server_action');
module.exports = require('./test_karma_action');
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
const execFileSync = require('child_process').execFileSync;
const winCmd = require('../../../lib/win_cmd');

module.exports = function testBrowserAction(plugin, run, options) {
module.exports = function testKarmaAction(plugin, run, options) {
options = options || {};

const kbnServerArgs = ['--kbnServer.plugin-path=' + plugin.root];
Expand All @@ -31,7 +31,7 @@ module.exports = function testBrowserAction(plugin, run, options) {
kbnServerArgs.push('--kbnServer.tests_bundle.pluginId=' + plugin.id);
}

const task = options.dev ? 'test:dev' : 'test:browser';
const task = options.dev ? 'test:karma:debug' : 'test:karma';
const args = [task].concat(kbnServerArgs);
execFileSync(winCmd('yarn'), args, {
cwd: plugin.kibanaRoot,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,14 +20,14 @@ Server tests are written just like browser tests, they are just executed differe
running the tests
=================

Running the server tests is simple, just execute `yarn test:server` in your terminal
Running the server tests is simple, just execute `yarn test:mocha` in your terminal
and all of the tests in your server will be run.

By default, the runner will look for tests in `server/**/__tests__/**/*.js`. If you'd prefer to
use a different collection of globs and files, you can specify them after the `yarn test:server`
use a different collection of globs and files, you can specify them after the `yarn test:mocha`
task, like so:
`yarn test:server 'plugins/myplugins/server/__tests__/**/*.js'`
`yarn test:mocha 'plugins/myplugins/server/__tests__/**/*.js'`
NOTE: quoting the glob pattern is not required, but helps to avoid issues with globbing expansion
in your shell.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,4 +17,4 @@
* under the License.
*/

module.exports = require('./test_browser_action');
module.exports = require('./test_mocha_action');
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,7 @@ import sinon from 'sinon';

describe('EmptyState', () => {
it('should render normally', () => {
const component = shallow(
<EmptyState loadingDataDocUrl="http://www.elastic.co" onRefresh={() => {}} />
);
const component = shallow(<EmptyState onRefresh={() => {}} />);

expect(component).toMatchSnapshot();
});
Expand All @@ -36,9 +34,7 @@ describe('EmptyState', () => {
it('is called when refresh button is clicked', () => {
const onRefreshHandler = sinon.stub();

const component = shallow(
<EmptyState loadingDataDocUrl="http://www.elastic.co" onRefresh={onRefreshHandler} />
);
const component = shallow(<EmptyState onRefresh={onRefreshHandler} />);

component.find('[data-test-subj="refreshIndicesButton"]').simulate('click');

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,13 +18,12 @@
*/

import React from 'react';
import PropTypes from 'prop-types';

import { EuiCallOut, EuiTextColor, EuiLink, EuiButton } from '@elastic/eui';

import { FormattedMessage } from '@kbn/i18n/react';

export const EmptyState = ({ onRefresh }) => (
export const EmptyState = ({ onRefresh }: { onRefresh: () => void }) => (
<div>
<EuiCallOut
color="warning"
Expand Down Expand Up @@ -82,7 +81,3 @@ export const EmptyState = ({ onRefresh }) => (
</EuiCallOut>
</div>
);

EmptyState.propTypes = {
onRefresh: PropTypes.func.isRequired,
};

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit 9f866ed

Please sign in to comment.