Tests are much slower since 22

[trollepierre]

Version

1.0.0-beta.22

Steps to reproduce

VueTestUtils + Jest + snapsnots are slower since beta 22

What is expected?

My tests use to last 70 seconds (900 tests, 70 snapshots) during VTU beta 20.

What is actually happening?

Now they last 170 seconds (900 tests, 70 snapshots) during VTU beta 22

---

I don't know why it is so slow, but I believe this should be fastened

[eddyerburgh]

Yes we should make sure tests are as fast as possible.

Are you able to provide me a repo that demonstrates the slower tests? That way I can investigate perf improvements.

[trollepierre]

Sorry I am not allowed.
Maybe I will create a personal project to help.

[trollepierre]

We have inconsistence when passing tests. (some simple tests needs 8seconds)
Sometimes all tests on our pipeline lasts more than 1 hour
Sometimes all tests on our pipeline lasts 170-250 seconds...
I don't know what happen but I think we will go back to beta 20.

[trollepierre]

This is one test that lasts 8 min on our pipeline https://codesandbox.io/s/308vv8l08p

[trollepierre]

Long tests are fixed in beta 24. I don't know why. Thanks for that.
But I stil have test times that is multiplied by 2. I will investigate

[trollepierre]

After investigation, I notice that Jest RAM consumption has exploded from beta 20 to beta 24.
That provokes long tests, timeout on my Docker, ...
Having a repository with 1000 tests (mixing mount and shallowMount) will be enough to have a good repro (I did it)

[trollepierre]

On that branch,
https://github.com/trollepierre/recontact_travel_blog/tree/vue-test-utils-perf

Tests last 18 seconds on vue-test-utils@^beta 20 and 32 seconds on vue-test-utils@^beta 24

The issue is on that component:
https://github.com/trollepierre/recontact_travel_blog/blob/vue-test-utils-perf/client/src/components/DTP/DatetimePicker.spec.js
(I intentionally duplicate tests for this issue)

[eddyerburgh]

Thanks for the reproduction.

I'm failing to replicate your results on my machine, I actually get a shorter time on beta.24 (~10s) compared to beta.20 (~13s).

I've been profiling the vue-test-utils tests and can't see any obvious expensive function calls. I'll keep investigating, because it's important that Vue Test Utils is fast.

[trollepierre]

Could you limit your RAM use to 2 Go?

We notice a memory leak when executing the tests. Because we can see the RAM consumption increasing on EACH of the Jest process as long as tests are executed.

[trollepierre]

Are you using the good branch? the branch vue-test-utils-perf?
On my Mac with 16 Go of RAM, it's 32seconds.

The time of 10 seconds is close to my time without the thousands of duplicated tests...

[eddyerburgh]

Thanks, I switched branches and profiled. The functions that are taking the longest are internal calls to createLocalVue, and external calls to Vue.use.

I'm going to open a PR to minimize internal createLocalVue calls. I'll continue investigating and improving performance.

[mkopinsky]

I don't know if it's related to this, but when running tests with mocha-webpack --watch, we occasionally see tests slow to a crawl, and node sometimes dies entirely with a stack trace/core dump. This points (in my mind) to a memory leak somewhere, but I have no clue if it's in mocha, mocha-webpack, vue-test-utils, or elsewhere.

I'd be happy to post the stack trace or do some other investigation next time this happens - if you have any debugging/profiling recommendations, I would be appreciative!

[BrianBrenner]

I haven't been noticing tests slowing down, but I only have 300 tests so it might not be noticiable yet, but lately the node heap has been running out of memory. I'm running the tests with using Karma and mocha and this plugin for Vue CLI 3 (https://www.npmjs.com/package/vue-cli-plugin-unit-karma). When running with the watch flag the heap runs out of memory after running two or three times.

[trollepierre]

You just need to duplicate 4-5 times your tests (especially the one with mounting options and snapshots...) to replicate and see the difference... Personally I increased the RAM level on my Gitlab And it works. Le mer. 28 nov. 2018 à 18:14, Brian Brenner <[email protected]> a écrit :

I haven't been noticing tests slowing down, but I only have 300 tests so it might not be noticiable yet, but lately the node heap has been running out of memory. I'm running the tests with using Karma and mocha and this plugin for Vue CLI 3 ( https://www.npmjs.com/package/vue-cli-plugin-unit-karma). When running with the watch flag the heap runs out of memory after running two or three times. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#892 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AH-dRlFYXWd1ygZHgR8q0WhV6RkGM4Moks5uzsRwgaJpZM4VslAS> .

-- Pierre Trollé