Yarn unnecessarily splitting dependencies

[mockdeep]

Do you want to request a feature or report a bug?
This looks like a bug in version 0.27.5. It works as expected on version 0.24.6.

What is the current behavior?
When I run yarn upgrade, yarn splits out dependencies unnecessarily, causing multiple versions to be installed. The version specifications are the same as they were before, but result in two or more dependency versions instead of a single resolved version. Here's a diff from our yarn.lock:

-"jquery@>=1.7.1 <4.0.0", jquery@>=1.8.0, jquery@^1.12.4:
+"jquery@>=1.7.1 <4.0.0", jquery@>=1.8.0:
+  version "3.2.1"
+  resolved "https://registry.yarnpkg.com/jquery/-/jquery-3.2.1.tgz#5c4d9de652af6cd0a770154a631bba12b015c787"
+
+jquery@^1.12.4:
   version "1.12.4"
   resolved "https://registry.yarnpkg.com/jquery/-/jquery-1.12.4.tgz#01e1dfba290fe73deba77ceeacb0f9ba2fec9e0c"

The JQuery version requirements did not change, and version 1.12.4 satisfies all of them, but for some reason yarn decided to add an additional copy of JQuery to satisfy transitive dependencies.

What is the expected behavior?

Yarn should find a single version to satisfy dependencies where possible.

Please mention your node.js, yarn and operating system version.
Node: 8.1.4
Yarn: 0.27.5
Operating system: reproducible on both MacOS and Linux

[BYK]

Hey there! We need a concrete, reproducible scenario so we can investigate the issue and hopefully fix it. Would you mind providing us a sample package.json file or a test repo along with any custom steps that need to be followed to consistently reproduce this.

[mockdeep]

Sure, I was able to simplifiy it down to two dependencies. package.json:

{
  "engines": {
    "node": "8.1.4"
  },
  "dependencies": {
    "jquery": "^1.12.4",
    "jquery-ujs": "^1.2.2"
  }
}

Which for me produces the following yarn.lock:

# THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS FILE DIRECTLY.
# yarn lockfile v1


jquery-ujs@^1.2.2:
  version "1.2.2"
  resolved "https://registry.yarnpkg.com/jquery-ujs/-/jquery-ujs-1.2.2.tgz#6a8ef1020e6b6dda385b90a4bddc128c21c56397"
  dependencies:
    jquery ">=1.8.0"

jquery@>=1.8.0:
  version "3.2.1"
  resolved "https://registry.yarnpkg.com/jquery/-/jquery-3.2.1.tgz#5c4d9de652af6cd0a770154a631bba12b015c787"

jquery@^1.12.4:
  version "1.12.4"
  resolved "https://registry.yarnpkg.com/jquery/-/jquery-1.12.4.tgz#01e1dfba290fe73deba77ceeacb0f9ba2fec9e0c"

[BYK]

@mockdeep this is because jquery-ujs lists jquery as a dependency instead of a peerDependency. I'd either open the issue there or simply go ahead and create a PR that fixes this.

[BYK]

You can find a similar discussion with reference to some others here: #3951

[mockdeep]

@BYK I've read over the comments there but I'm still not quite understanding the rationale for this change in behavior. There are quite a few dependencies in our yarn.lock which expanded out this way and it causes our application to break when we end up requiring in the wrong version of a dependency, both in our direct dependencies and in transitive dependencies down the line. It also causes bloat to our client-side bundle. It seems to me like if there is a valid version that satisfies both dependency and peerDependency, it ought to use that one. I can understand adding an extra version in cases where there is no single version that will satisfy the dependencies, but in many cases it doesn't seem like it makes sense to necessarily require something as a peer dependency. One example is:

yarn add [email protected]

This ends up installing 2 versions of domutils for 2 transitive dependencies, when a single version would suffice.

[BYK]

It seems to me like if there is a valid version that satisfies both dependency and peerDependency, it ought to use that one

I can definitely relate to this and trust me when I say, our aim is to make this a reality when possible. That said there may be situations that it may not be possible to hoist everything up. And we also have some things that we can improve around common version/range resolutions. But from a purely spec-related perspective, what Yarn does is correct. The reason you are running into problems is either your project or its dependencies are relying on unspecified hoisting mechanics which are not guaranteed to exist. The correct way to ensure consistent and shared dependencies across projects is using peerDependencies so your ultimate solution is there.

In the meantime, we will try to make the hoister and resolver better to avoid duplicates but as stated above, this is not something projects can rely on. Neither for yarn nor for npm.