yarn 1.10.0 is really slow (compared to 1.9.4)

[kaiyoma]

I'm running version 1.10.0 on Windows 10, after having just upgraded from 1.9.4. All yarn operations are taking noticeably longer than they should. I tried clearing my cache, but that hasn't helped.

As a concrete example, if I run yarn install --pure-lockfile over and over, subsequent invocations take a really long time to complete, even though there's nothing to be done. Version 1.9.4 did not work like this; it always finished within a second or two, saying that there was nothing to be done.

Here's what I see with 1.10.0:

$ yarn install --pure-lockfile
yarn install v1.10.0
[1/4] Resolving packages...
[2/4] Fetching packages...
info [email protected]: The platform "win32" is incompatible with this module.
info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation.
info [email protected]: The platform "win32" is incompatible with this module.
info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation.
[3/4] Linking dependencies...
[4/4] Building fresh packages...
Done in 777.03s.

Kyle Getz@kgetz MINGW64 ~/Work/event-viewer (master)
$ yarn install --pure-lockfile
yarn install v1.10.0
[1/4] Resolving packages...
[2/4] Fetching packages...
info [email protected]: The platform "win32" is incompatible with this module.
info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation.
info [email protected]: The platform "win32" is incompatible with this module.
info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation.
[3/4] Linking dependencies...
[4/4] Building fresh packages...

Done in 280.12s.

Kyle Getz@kgetz MINGW64 ~/Work/event-viewer (master)
$ yarn install --pure-lockfile
yarn install v1.10.0
[1/4] Resolving packages...
[2/4] Fetching packages...
info [email protected]: The platform "win32" is incompatible with this module.
info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation.
info [email protected]: The platform "win32" is incompatible with this module.
info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation.
[3/4] Linking dependencies...
[4/4] Building fresh packages...

Done in 276.44s.

Those runtimes (nearly 5 minutes!) are pretty ridiculous considering nothing has to be done. If I downgrade to 1.9.4, the bad behavior goes away:

$ yarn install --pure-lockfile
yarn install v1.9.4
[1/4] Resolving packages...
success Already up-to-date.
Done in 1.29s.

Kyle Getz@kgetz MINGW64 ~/Work/event-viewer (master)
$ yarn install --pure-lockfile
yarn install v1.9.4
[1/4] Resolving packages...
success Already up-to-date.
Done in 1.18s.

Kyle Getz@kgetz MINGW64 ~/Work/event-viewer (master)
$ yarn install --pure-lockfile
yarn install v1.9.4
[1/4] Resolving packages...
success Already up-to-date.
Done in 1.18s.

Even clearing the cache takes a lot longer in the new version. I think something is very broken here.

[audiolion]

I just tried upgrading, I use yarn workspaces and yarn never correctly resolved all of my workspace packages, I could just keep running yarn && yarn && yarn, it never gets to the point where everything is installed and has similar behavior that @kaiyoma is experiencing

[edmorley]

I believe this is because yarn >=1.10.0 will try to add integrity fields for each package in yarn.lock, if they don't already exist, which means it has to download all the packages again to retrospectively calculate the hashes. (See also: heroku/heroku-buildpack-nodejs#569 (comment))

Unfortunately yarn seems to do this extra work, even when using --frozen-lockfile (I haven't tested with --pure-lockfile) - but then then that flag means yarn.lock never gets updated, so it repeats it all over again the next time it's run, even when the next run should have been a no-op.

In your case, have you tried committing the updated yarn.lock that has the added integrity fields?

[audiolion]

In your case, have you tried committing the updated yarn.lock that has the added integrity fields?

not sure if you are talking to me, but I am doing this locally so version control doesn't matter here and I am not using --pure-lockfile or --frozen-lockfile, just yarn

[kaiyoma]

@edmorley Yes, that was exactly it, thank you! Once I committed the lockfile changes, yarn stopped doing all the extra work. Maybe there could be a helpful message to the user in this scenario? Otherwise it's really not clear why yarn is suddenly struggling to do its normal business.

[ckknight]

Even after updating and committing the yarn.lock file, running either yarn, yarn --pure-lockfile, or yarn --frozen-lockfile continues to do all the extra work with no indication as to why.

[vzaidman]

meanwhile, any workarounds besides using an older version of yarn?

[ckknight]

This is fixed with yarn 1.12 (don't know about 1.11)