Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automatically deduplicate "overlapping" packages #125712

Closed
jportner opened this issue Feb 15, 2022 · 2 comments · Fixed by #169165
Closed

Automatically deduplicate "overlapping" packages #125712

jportner opened this issue Feb 15, 2022 · 2 comments · Fixed by #169165
Labels
Team:Operations Team label for Operations Team

Comments

@jportner
Copy link
Contributor

jportner commented Feb 15, 2022

Describe the feature:

When upgrading packages, we often make a best effort to take the opportunity deduplicate transitive dependencies, but it's a burdensome, manual process.
It would be nice if we could automate this during yarn kbn bootstrap!

I found yarn-deduplicate that could do the trick for now. We are currently using Yarn v1.22.15. When we eventually upgrade to Yarn 2+, it supports package deduplication natively without any third party tools.

Describe a specific use case for the feature:

Sample PR where this extra effort could be avoided: #125526 (review)

I ran yarn-deduplicate locally and here is the result:

git diff --stat
 yarn.lock | 2190 ++++++++---------------------------------------------------------
 1 file changed, 266 insertions(+), 1924 deletions(-)

After rerunning yarn kbn bootstrap, my node_modules folder size went down from 2.1G to 1.8G. This could also reduce the size of our distributables!

@jportner jportner added the Team:Operations Team label for Operations Team label Feb 15, 2022
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-operations (Team:Operations)

@exalate-issue-sync exalate-issue-sync bot added impact:needs-assessment Product and/or Engineering needs to evaluate the impact of the change. loe:small Small Level of Effort labels Feb 16, 2022
@jportner
Copy link
Contributor Author

jportner commented Feb 24, 2022

Another example why we should implement this--

I had just upgraded a dependency, and then another PR (from another team) introduced an older version of that dependency. I had to submit a third PR to fix it: #126301

@tylersmalley tylersmalley removed loe:small Small Level of Effort impact:needs-assessment Product and/or Engineering needs to evaluate the impact of the change. labels Mar 16, 2022
jbudz added a commit that referenced this issue Oct 19, 2023
- Adds an entrypoint, `node scripts/yarn_deduplicate` for fixing
duplicated packages via npm package `yarn-deduplicate`
- Fixes existing duplicated packages
- Adds a CI check verifying no duplicated packages have been added.  

There are currently two exclusions - all types packages which causes
other checks to break, and axe-core, which does not follow semver.

Closes #125712
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Team:Operations Team label for Operations Team
Projects
None yet
3 participants