Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[rush] Add --from to rush install #2435

Merged
merged 4 commits into from
Jan 29, 2021
Merged

Conversation

wbern
Copy link
Contributor

@wbern wbern commented Jan 13, 2021

My first real PR to rush, finally. ❤️

Summary

I added support for --from-based installs using rush install, currently there is only support for --to, and the change was trivial to make, following pnpm workspaces documentation here: https://pnpm.js.org/en/filtering#filter-lt-package_name-

Fixes #2434

Details

In my org, we want to install/build/test only the changed projects in our CI, which we are accomplishing with the --to flag. However, we see that if we change a widely used package, we want to build and test all the dependents of that package when it gets changed, and publish those packages as well. Hence we have a need for the --from flag for rush install, a flag already present in the other rush commands we use around building things.

How it was tested

I created an alias called testrush locally on my machine and used in our biggest repo, I then did

testrush install --to @org/my-lib

I inspected the output being Scope: 4 out of 124 workspace projects,

then I ran testrush install --to @org/my-lib --from @org/my-lib

Which gave me the output of Scope: 115 of 124 workspace projects.

I also tested testrush install --from @org/my-lib which returned Scope: 112 of 124 workspace projects, so it adds up.

The output from the filtered install in this case is pnpm install --filter ...@org/my-lib --filter @org/my-lib....

Final notes

I know we're changing names of --from/--to to other things later, so open for suggestions how to merge this.

Copy link
Member

@D4N14L D4N14L left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good functionally, small comment on syntax of filter creation logic, but leaving to someone else to approve.

// "<package>..." selects the specified package and all direct and indirect dependencies
.map((p) => p.packageName + '...')
// ..."<package>" selects the specified package and all direct and indirect dependents of that package
.concat(this.options.fromProjects.map((p) => '...' + p.packageName));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, the original PR that brought this functionality in actually had this. I was talked out of it by @octogonz (though I think it may have been a Gitter discussion since I can't find it now...). Main reason it was removed was that I couldn't think of a great justification for how this would be used. So he's likely going to need some convincing :)

Copy link
Contributor Author

@wbern wbern Jan 14, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Our justification is that in order to use the rush build --from flag, we need to also install the deps of those dependents coming up from the --from flag. If package A relies on B, we can't build A when B changes, if we don't install the dependencies for A via the --from flag.

The workaround for now is to do rush install without any filtering, but that is largely inefficient for our large monorepos.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regardless of whether or not upstream projects need to be built, there are plenty of instances where merely referencing the upstream project results in references to the node_modules of the upstream project, so it would not surprise me if, unlike in the rush build case (where the developer may be running it after an earlier rush build), rush install --from is completely unusable if it skips the dependencies of upstream local projects.

Copy link
Member

@D4N14L D4N14L left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Signing off, but merge will likely be gated on @octogonz

Thanks for your changes @wbern!

@iclanton
Copy link
Member

@octogonz - do you have any concerns about this PR? Looks good to @D4N14L and me.

@octogonz
Copy link
Collaborator

The code looks fine. There is a slightly interesting nuance in the design. From the discussion about PR #2422, we said that:

  • rush build --to x builds a transitive closure; whereas
  • rush build --from x does NOT build a transitive closure, so we proposed that: 👈
  • rush build --from x --with-dependencies will build the transitive closure

Whereas if I understand correctly:

  • rush install --to x installs a transitive closure
  • rush install --from x installs a transitive closure 👈

Would this inconsistency be confusing to anyone? (And if so, would we rename --from for rush install? Or would we redefine --from for rush build?)

CC @dmichon-msft

@wbern
Copy link
Contributor Author

wbern commented Jan 26, 2021

@octogonz Honestly I think --from should include everything that's needed. As a colleague of mine said, to and from makes an hourglass, or at least so we thought at first. 🙂

@octogonz
Copy link
Collaborator

In #2422 (comment) we're proposing to use --without-dependencies rather than --with-dependencies for rush build. So I think I'm good with this PR. 👍

@octogonz octogonz merged commit 61294b7 into microsoft:master Jan 29, 2021
@octogonz
Copy link
Collaborator

@wbern This was released with Rush 5.37.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

[rush] Add --from to rush install
5 participants