Performance issues with userEvent.type()

[Etheryte]

• @testing-library/user-event version: 12.6.0
• Testing Framework and version: @testing-library/react version 11.2.1
• DOM Environment: jsdom version 16.4.0

Relevant code or config:

describe("demo", () => {
  const text = "2001:db8:ac10:fd01::20";

  test("time type", () => {
    render(<input data-testid="input" type="text" />);
    const input = screen.getByTestId('input');
    const before = new Date();
    userEvent.type(input, text);
    console.log(`type took ${new Date() - before}ms`);
    expect(input.value).toEqual(text);
  });

  test("time delayed type", async () => {
    render(<input data-testid="input" type="text" />);
    const input = screen.getByTestId('input');
    const before = new Date();
    await userEvent.type(input, text, {
      delay: Number.MIN_VALUE,
    });
    console.log(`delayed type took ${new Date() - before}ms`);
    expect(input.value).toEqual(text);
  });

  test("time paste", () => {
    render(<input data-testid="input" type="text" />);
    const input = screen.getByTestId('input');
    const before = new Date();
    userEvent.paste(input, text);
    console.log(`paste took ${new Date() - before}ms`);
    expect(input.value).toEqual(text);
  });
});

Problem description:

Running the above test suite with yarn test --no-cache logs the following values on my machine:

type took 29ms
delayed type took 63ms
paste took 1ms

The results are fairly consistent between reruns (plus-minus a few ms).

The performance difference is starker with longer strings: replacing the input text with const text = "2001:db8:ac10:fd01::20".repeat(100);, the results are as follows:

type took 1380ms
delayed type took 5940ms
paste took 1ms

I understand that some of this difference stems from the letter-by-letter nature of type() which is crucial for certain test scenarios. For the time being however, type() is too slow for many of the tests we run and we've had to opt for paste() instead.

I haven't profiled type()'s internals so I don't know what the source of the performance penalty is.

[ph-fritsche]

I don't think there is much to gain optimizing the test utility code for performance.
After all we're talking about miliseconds in a test environment that can run multiple tests parallel.
The approach here is to trade manual labor (analyzing code) for automated labor (running test containers) by running these utilities in the first place.

If you put in the work and optimize some part of the code, a PR will be welcome.
But keep in mind we'll err on the side of maintainability.

[Etheryte]

I don't think the scale of the problem is as low as you make it to be. If you do ten type() calls within one test you're already getting close to a second and with anything larger you can quickly reach Jest's default timeout of 5 seconds. Choosing to not look into this is fair, but I don't think downplaying the problem is beneficial.

For context, in a perfect world I would expect type() with no delay to be on the same scale as paste(), or at least in the same order of magnitude. Right now it can be several orders of magnitude slower, easily reaching the default time limit.

[ph-fritsche]

Don't get me wrong. I see that you can't type a book in 5sec default timeout.
But I just checked on my local machine. I can repeat your userEvent.type(input, '2001:db8:ac10:fd01::20') 308 times before hitting the timeout.
I don't see any test that would benefit from doing a character by character and event by event simulation of typing more than that.

userEvent.paste provides a completely different abstraction. No keydown..keypress..keyup etc. for each character.
If you want to test your user typing, it is no better than calling fireEvent.input directly.

What userEvent.type does for you on the other hand is exactly this: Simulate every event that would happen if a user went on and typed every character in your input string one by one.

[nickserv]

If this is still a performance issue for anyone, a reproducible example app or performance test using User Event 13 would be helpful.

[Etheryte]

@nickmccurdy Not sure if I misunderstood you, but the example given in the opening comment still yields the same results with @testing-library/user-event 13.2.0.

[nzyoni]

@ph-fritsche

I don't think there is much to gain optimizing the test utility code for performance.
After all we're talking about miliseconds in a test environment that can run multiple tests parallel.

Hi, 😄
In my case, we run our test suites in parallel using jest and we do face time issues with the type operation.
It takes longer than what you have mentioned here, it ranges from 500 ms to 1600 ms depending on the test.

After switching from type to paste we saw a major improvement
For example

1550 ms -> 680 ms
570 ms -> 94 ms

We want to use type in order to test our code/components as a user would use, but, using type makes our tests slow 😞 and as a result, it makes our pipeline slow.

IMO This issue should be addressed.

[ph-fritsche]

As mentioned before paste is a completely different implementation with a completely different behavior. It's not interchangeable at all and comparing the execution time between those two APIs is moot.
If you want to demonstrate the performance of the implementation, you need to record the events it dispatches and replay those calling all the same event handlers.

[danny-pflughoeft]

userEvent.type(input, 'spaces not allowed'); is currently taking 5+ seconds to execute on a simple component. The same call to .paste() is instant.

I think this issue should be reopened. I get that performance of tests is not hyper-critical, but when tests are timing out literally due to a single call to .type(), that's a major, breaking bug.

[ph-fritsche]

If you can provide a reproduction, we'll look into it.
If you keep on comparing the execution time to something completely different, we'll get nowhere.

[AlonMiz]

@ph-fritsche is it reasonable to wait for 5s for typing spaces not allowed?
a regular user would type it much faster.
I experience the same behavior in my codebase with formik forms (even when using FastField), typing is significantly slow. I had to move to paste to alleviate the long test times.

[ph-fritsche]

@AlonMiz Depends.

el.addEventListener('keydown', () => {
  const t0 = (new Date()).getTime()
  while((new Date()).getTime() < t0 + 5000) true
})

And all of a sudden it's reasonable for a single keydown to take 5s.
The fact that doing something completely different does not take 5s changes nothing.

So if you think there is a problem, please help to reproduce the issue.
.paste() does something completely different than .type(). The fact that different APIs which do completely different things perform different is moot.

I'm sure there are some things inside keyboard implementation that could be optimized. If someone wants to put in the work, I'll gladly support the effort. But I don't think there is much to gain and if the difference is miniscule, we'll opt for the more maintainable variant.

I'm pretty sure that there is nothing that allows to optimize in the magnitude that you are suggesting. This is simply because I know the code and it is not that slow in any environment I examined.

So if you guys think this issue should be addressed, please address it!
The first step doing so is providing a reproduction.
Stop demanding other people to try to imagine possible scenarios for performance issues. 😞

[AlonMiz]

i agree with giving a reproducible example.
and i get your main point of "this can be something on your side",
but the main point we are trying to make is that: there are several users that report the same issue, it is not resoved, but the issue is closed. of course it can be related to 3rd party module (like the combination of formik with testing-library), but let's try get to the root cause here, and then try to fix it. 🙏

[ph-fritsche]

@AlonMiz The initial issue report states something between 0.6ms (when repeated 100 times) and 1.3ms (on the first call) execution time per char. This is in the same ballpark of the execution times I got when reproducing the example on my machine with jest+jsdom.
(Updating to v14 the execution time goes up a little bit, probably due to the additional workarounds on the element properties and selection handling, but still nothing I would not consider reasonable. The execution time per char is actually somewhere around 0.0045ms (if you type 2000 characters). The rest of the execution time is calling the API and interpreting code and input (~0.6ms), clicking and selecting (~9ms), checking for pointer-events (~40ms) and awaiting the next microtask respectively the next macrotask when using delay.)
This issue has been closed because we don't consider this an issue. Running any userEvent call is primarily about simulating behavior, so it would only be an issue if you could simulate the same behavior in a substantially shorter amount of time.

If there is another issue with some environment, I don't know. But reports about >270ms per char suggest that this is another issue that is specific to some environment.

[mnbroatch]

Locally this is running for me at 30-50 ms per character. In our test runner? Forget about it. This feature is simply working badly and any attempts to say it isn't that bad is just some lost-in-the-sauce denial.

[ph-fritsche]

@mnbroatch Thanks for your feedback. Have you tried turning it off and on again?

Your detailed information about the environment will certainly allow us reproduce the problem and fix it for you. We'll be working day and night on this.

[sigmike]

Here is an example with MUI's Autocomplete that takes about 1 second to type "united states" in the input: https://codesandbox.io/s/aged-glitter-dq5coo.

But with a basic input it takes about 120 ms: https://codesandbox.io/s/gracious-tereshkova-0qvx0z.

So I guess the timing mostly depends on what's happening when a character is typed.

[JonasBa]

We at Sentry recently started to profile our tests in CI and we have noticed the same thing.

I would just like to emphasize what @sigmike said. While userEvent.type is definitely slower than using keypress, the reality is that it likely plays a much smaller role than you may think.

For example, if you are rerendering the entire tree of the component on each keypress or running some long running function on each user input, then the performance issue will be exacerbated by using .type - as a developer that is something you should know about and is imo of the best features of userEvent library.

I would still always recommend using .type and investigating what (in your application code) makes it slow.

[Asvarox]

I came across this issue after upgrading from userevent 13.2.1 to 14 - multiple tests started timing out on CI. I noticed that userEvent.type (or user.type) took about 4 times longer. For larger forms with a lot of text to be inputted it adds up.

I prepared a simple repro repo with branches userevent13 and userevent14. On my machine, on the former it takes ~10ms to input a longer text while on latter it's ~38ms (eye-balled averages after multiple runs).

• 13 test code https://github.com/Asvarox/slow-userevent/blob/userevent13/src/App.test.js
• 14 test code https://github.com/Asvarox/slow-userevent/blob/userevent14/src/App.test.js

On the real project the slowdown is similar, but the numbers are greater - from ~40ms to ~140ms. On CI it takes 600ms on userevent14 (I didn't measure 13 there).

Should I open a new issue?

[ph-fritsche]

@Asvarox The measurements are inaccurate regarding what exactly is measured here.
The userEvent13 example is synchronous. This does not allow other asynchronous code to run in between your console.time and console.timeEnd.
The userEvent14 example is asynchronous. After each action (keydown, keyup, etc.) the subsequent code is pushed to (at least) the next macrotask (await new Promise(r => setTimeout(r))). This allows other code to be executed in between.
The measurement for asynchronous variant therefore includes other tasks from the event loop that are not part of the userEvent code itself and may or may not be triggered by event handlers on the simulated events.

See these examples on Codesandbox:
Note that the measurement runs in the same event loop that also e.g. handles events on your cursor movement.
Note that the exact execution time heavily depends on the machine and e.g. browser plugins so you can't compare them across different devices / browsers.
Note that the pointer implementation simulates more events and keeps track of the cursor position in the recent version, so I split the type call into the underlying click and keyboard.
userEvent 13 (synchronous): For me it clocks at just over 5ms for the click, 0.56ms for a single character typed and 0.31ms per char for those 17 characters.
(In each of those examples you can see how calling the API and interpreting the input string adds more time than looping over the events for each character.)
userEvent 14 (asynchronous): For me it clocks at little bit under 21ms for the click and 9.3ms for a single character typed and 5.4ms per char for those 17 characters.

Now what looks like a much slower API turns out to not look like it if we don't allow the browser to pick up another macrotask (scheduled by codesandbox or user code or a plugin or the browser itself because e.g. the real mouse moved):
Note that other microtasks might still be executed between different asynchronous parts of the userEvent implementation.
userEvent 14 (with delay: null): For me it clocks at under 6ms for the click, 0.65ms for a single character typed and 0.56ms for those 17 characters.

If you want to accurately measure the performance, we'll have to isolate it and set up a docker container that runs just the userEvent API in a headless browser or node task.
I don't think we'll find anything, but if you want to explore this, please file a new issue as the initial post is about differences between type and paste which is moot.

[iamdench]

Hello! Thank you for your good libs for testing!
I conducted a speed test where I measured the speed of running tests with clearing and filling the input field 50 times.
In one test I used filling using type, and in another using paste. In my case it was a difference of 3 times
You might want to add options in userEvent.type where you can set a flag to simply print or simulate all user events to improve the performance.
I understand that your userEvent.type is now very similar to native and there are certainly merits to it and users who want to use it, but most users would like to see a slightly more simple feature.
Screenshot with my speed test:

[morganp-dev]

Has anyone tried a mix of 'paste' and 'type' so that we get the speed of pasting with the events of typing. Something along these lines:

export const fastUserType = async (element, text) => {
  element.focus();
  userEvent.paste(text.slice(0, text.length - 1));
  await userEvent.type(element, text.slice(-1));
};

This way we only ever type a single character, which hopefully won't take too long.

[rodrigovallades]

Why is this issue closed? userEvent.type is actually unusable in real world scenarios. Simple forms are timing out.

[LeviDeng]

We're still encountering the performance issue with userEvent.type. So I used fireEvent as the alternative, which is something to consider if you must test scenarios with user typing:

import { fireEvent } from "@testing-library/react";

userEventType(element, "text");

function userEventType(
  element: Document | Element | Window | Node,
  input: string,
) {
  fireEvent.click(element);
  input.split("").forEach((char, i) => {
    const value = input.slice(0, i + 1);
    fireEvent.keyDown(element, { key: char });
    fireEvent.keyPress(element, { key: char });
    fireEvent.input(element, { target: { value } });
    fireEvent.keyUp(element, { key: char });
  });
}

[jeremy-daley-kr]

I certainly don't mean this as any disrespect to the purpose of @testing-library packages, but you may need to reconsider the appropriate tooling when you're pushing the bounds of Jest's timeout defaults, and I dare say, @testing-libary's intention to adequately execute within those bounds.

So far I've found that different configurations can lead to significantly different timing outcomes, and those timings start to seem (almost exponentially) inadequate as instances increase.

I'd certainly like to get weigh-in from Jest's own contributors on the matter if some of its major dependents want to redefine what are acceptable for those defaults, but also... As much as @testing-library seems to suggest fireEvent is a "fallback", I think for performance reasons, there's more to it than that.

Most projects have a few use cases for fireEvent, but the majority of the time you should probably use @testing-library/user-event.

I'm not contesting that user-event could be in need for performance optimizations itself, but if your tests don't depend on key-by-key input, perhaps user-event is simply not the right tool for the job.

Consider the following (Typescript) example, and play with any of the values:

• FIELDS - total number of text input fields
• CHARS - number of chars to input per text input field
• defaultHidden - option for getByRole(), etc. to bypass accessibility checks
• skipClick - option for explicitly clicking into an element before then typing in it

import React from 'react'
import {
  render,
  fireEvent,
  configure as rtlConfigure,
} from '@testing-library/react'
import userEvent, {
  Options as UserEventOptions,
} from '@testing-library/user-event'

jest.setTimeout(60000)
rtlConfigure({ defaultHidden: true })

const FIELDS = 10
const CHARS = 1000
const SESSION_CONFIG: UserEventOptions = { skipClick: true }

const fields = Array.from({ length: FIELDS }).map((_, i) => (
  <input type="text" key={i} aria-label={`fake${i}`} />
))
const chars = 'a'.repeat(CHARS)
const MyTest = () => <div>{fields}</div>

describe('MyTest', () => {
  test('userEvent', async () => {
    const session = userEvent.setup(SESSION_CONFIG)
    const { getByRole } = render(<MyTest />)
    for (let i = 0; i < fields.length; i++) {
      await session.type(getByRole('textbox', { name: `fake${i}` }), chars)
    }
  })

  test('fireEvent', async () => {
    const { getByRole } = render(<MyTest />)
    for (let i = 0; i < fields.length; i++) {
      fireEvent.input(getByRole('textbox', { name: `fake${i}` }), {
        target: { value: chars },
      })
    }
  })
})

These are the results of that ^^ (at least on my Mac M1):