Add testing document [skip-ci]

[joshdover]

Summary

Relates to #53135

Adds a new documentation file to cover testing recommendations.

TODO:

• Answer TODO about how to write integration tests for UI code
• Figure out how to test Observables correctly
• Move to asciidocs?

Checklist

Use strikethroughs to remove checklist items you don't feel are applicable to this PR.

• This was checked for cross-browser compatibility, including a check against IE11
• Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
• Documentation was added for features that require explanation or tutorials
• Unit or functional tests were updated or added to match the most common scenarios
• This was checked for keyboard-only and screenreader accessibility

For maintainers

• This was checked for breaking API changes and was labeled appropriately
• This includes a feature addition or change that requires a release note and was labeled appropriately

[joshdover]

This is not ready to merge quite yet, but I'm opening it for review / feedback on the general format.

I would like to get this merged even before all of the different sections are filled out so that it can be useful sooner than later. The other sections can be filled out in other PRs.

[elasticmachine]

Pinging @elastic/kibana-platform (Team:Platform)

[mshustov]

Tests for a public API without a UI-part. For example, license$ in licensing plugin, session control in the security plugin, etc.

[mshustov]

Plugins can test their route handlers via FTR or existing test_utils. We can describe both ways until #51150
Can be done in follow-ups

[joshdover]

Yep that is the plan, to do in follow up PRs

[mshustov]

global API calls as well. that a plugin doesn't redirect the app on a specific url on the mount, for example.

[joshdover]

Core API calls is a good example. For instance, calling core.chrome.setIsVisible() for a full-screen mode, but not unsetting it when unmounted if user is redirected to another app.

Could you explain this example more? "a plugin doesn't redirect the app on a specific url on the mount"

[mshustov]

This example is not possible to test. Probably to document it is enough.

setup(core: CoreStart, plugins: Deps) {
  // this logic would affect the whole Kibana
  plugins.licensing$.subscribe(async license => {
    if(!license.iAvailable) {
      window.href = ..
    });
  });
};

[mshustov]

Well. If a plugin doesn't subscribe manually, it needn't to unsubscribe. We must explain this in the manual.
Given your example.
settings$ is Observable. A plugin cannot complete it. It will be completed if the unsubscription is implemented in the useObservable hook correctly.
The better example could be if a plugin implements polling for a every new settings$ value and implements a subscription manually.

const renderApp = (params: AppMountParameters, core: CoreStart): AppUnmount => {
  const settings$ = core.uiSettings.get$<string>('mysetting1');
  const pollingSubscription = settings$.subscribe(async url => {
    const value = await core.http.fetch(url);
    // ..
  });

  return function cleanup() {
    pollingSubscription.unsubscribe();
  };
};
...
describe('renderApp', () => {
  it('mounts and unmounts UI', () => {
    const params = { element: document.createElement('div'), appBasePath: '/fake/base/path' };
    const core = coreMock.createStart();
    const settings$ = new Rx.Subject<any>();
    core.uiSettings.get$.mockReturnValue(settings$);

    const unmount = renderApp(params, core);
    // Verify some expected DOM element is rendered into the element
    expect(params.element.querySelector('.some-app-class')).not.toBeUndefined();
    // Verify that subscribed to settings$
    expect(settings$.observers.length).toBeGreaterThan(0);

    unmount();
    // Verify the element is empty after unmounting
    expect(params.element.innerHTML).toEqual('');
    // Verify that unsubscribed correctly
    expect(settings$.observers.length).toBe(0);
  });
});

We can extract some of this logic in a custom matcher to be able to write something like this:

expect(settings$).toHaveNoSubscriptions();

[joshdover]

Thanks for the input here, Subjects make this much easier / possible. I think we should encourage this pattern for more fine-grained results but rely on something like drool to make this foolproof.

[mshustov]

imho it makes sense to draw the line between API tests and plugin contract tests. With the NP we should encourage plugins to interacts via interfaces instead of API. It means a significant shift in testing as well.

[joshdover]

Added some guidelines on what this section should detail in the future, let me know if that lines up with what you're thinking.

[mshustov]

Probably we should introduce autotests for memory leak detection. just a handful for the most critical parts https://github.com/samccone/drool and friends

[joshdover]

Using drool to verify app unmounting cleaning would be interesting for sure. It appears we could have a single mechanism that tests each application automatically.

[mshustov]

should I create an issue to investigate this option? it's not a critical problem right now, can be done later

[joshdover]

Some updates pushed based on first round of feedback.

[mshustov]

I think it shouldn't be hidden in Testing docs and deserves a separate file https://github.com/elastic/kibana/pull/53866/files

[flash1293]

Currently there are some karma tests doing visual regression testing (e.g. src/legacy/core_plugins/vis_type_vega/public/__tests__/vega_visualization.js). In a jest/ftr world it's not 100% clear to me where they belong. Is it recommended to cover them with the percy setup or will there be something closer to the current karma runners? Just food for thought, a section about visual regression testing would probably be helpful in some cases.

[mshustov]

Let's merge as-is. We can update the remaining options in follow-ups.