yarn 2 / Plug N Play Support #2386

Jordan-Hall · 2020-01-27T11:40:31Z

Please make sure you have read the submission guidelines before posting an issue

Prerequisites

Yarn version 2.0.0-rc.27

Expected Behavior

Should work correctly like before

Current Behavior

Build failure

Failure Information (for bugs)

An unhandled exception occurred: Cannot locate the 'node_modules' directory.
See "AppData\Local\Temp\ng-mO3oFO\angular-errors.log" for further details.

Steps to Reproduce

Please provide detailed steps for reproducing the issue.

yarn dlx @yarnpkg/doctor .
yarn install
yarn start

Context

Yarn has finally released a usable version two. If you using webpack 4 you need to enable PlugNPlay which is native to webpack 5

https://yarnpkg.com/advanced/migration
https://yarnpkg.com/advanced/pnpapi

Failure Logs

[error] Error: Cannot locate the 'node_modules' directory.
    at NgccProcessor.findNodeModulesDirectory (C:\dev\lec-migration\.yarn\$$virtual\@ngtools-webpack-virtual-9bdca6ad48\0\cache\@ngtools-webpack-npm-9.0.0-rc.10-6bfaf96715-1.zip\node_modules\@ngtools\webpack\src\ngcc_processor.js:113:15)
    at new NgccProcessor (C:\dev\lec-migration\.yarn\$$virtual\@ngtools-webpack-virtual-9bdca6ad48\0\cache\@ngtools-webpack-npm-9.0.0-rc.10-6bfaf96715-1.zip\node_modules\@ngtools\webpack\src\ngcc_processor.js:31:43)
    at C:\dev\lec-migration\.yarn\$$virtual\@ngtools-webpack-virtual-9bdca6ad48\0\cache\@ngtools-webpack-npm-9.0.0-rc.10-6bfaf96715-1.zip\node_modules\@ngtools\webpack\src\angular_compiler_plugin.js:554:33
    at SyncHook.eval [as call] (eval at create (C:\dev\lec-migration\.yarn\cache\tapable-npm-1.1.3-f1c2843426-1.zip\node_modules\tapable\lib\HookCodeFactory.js:19:10), <anonymous>:7:1)
    at SyncHook.lazyCompileHook (C:\dev\lec-migration\.yarn\cache\tapable-npm-1.1.3-f1c2843426-1.zip\node_modules\tapable\lib\Hook.js:154:20)
    at Object.webpack [as webpackFactory] (C:\dev\lec-migration\.yarn\cache\webpack-npm-4.41.2-efbe0580aa-1.zip\node_modules\webpack\lib\webpack.js:55:30)
    at createWebpack (C:\dev\lec-migration\.yarn\$$virtual\@angular-devkit-build-webpack-virtual-0beaa59de8\0\cache\@angular-devkit-build-webpack-npm-0.900.0-rc.10-7017fe1d56-1.zip\node_modules\@angular-devkit\build-webpack\src\webpack-dev-server\index.js:20:36)
    at Object.runWebpackDevServer (C:\dev\lec-migration\.yarn\$$virtual\@angular-devkit-build-webpack-virtual-0beaa59de8\0\cache\@angular-devkit-build-webpack-npm-0.900.0-rc.10-7017fe1d56-1.zip\node_modules\@angular-devkit\build-webpack\src\webpack-dev-server\index.js:46:12)
    at SwitchMapSubscriber.project (C:\dev\lec-migration\.yarn\$$virtual\@angular-devkit-build-angular-virtual-d79a835d58\0\cache\@angular-devkit-build-angular-npm-0.900.0-rc.10-f83f279b13-1.zip\node_modules\@angular-devkit\build-angular\src\dev-server\index.js:191:32)
    at SwitchMapSubscriber._next (C:\dev\lec-migration\.yarn\cache\rxjs-npm-6.5.3-bc949c0a94-1.zip\node_modules\rxjs\internal\operators\switchMap.js:49:27)
    at SwitchMapSubscriber.Subscriber.next (C:\dev\lec-migration\.yarn\cache\rxjs-npm-6.5.3-bc949c0a94-1.zip\node_modules\rxjs\internal\Subscriber.js:66:18)
    at C:\dev\lec-migration\.yarn\cache\rxjs-npm-6.5.3-bc949c0a94-1.zip\node_modules\rxjs\internal\util\subscribeToPromise.js:7:24
    at processTicksAndRejections (internal/process/task_queues.js:97:5)

Other

Documentation suggested you add the fallowing into .yarnrc.yml
nodeLinker: node-modules

When then produces the following error

Error: Something that got detected as your top-level application (because it doesn't seem to belong to any package) tried to access a package that is not declared in your dependencies       

Required package: @yarnpkg/fslib (via "@yarnpkg/fslib")
Required by: /C:/dev/lec-migration/node_modules/@yarnpkg/pnpify/lib/

The text was updated successfully, but these errors were encountered:

Jordan-Hall · 2020-01-27T12:32:49Z

Angular appears not to be supporting PNP yet. Tracking angular/angular-cli#12465

ElbowBaggins · 2020-03-05T21:27:51Z

According to this angular-cli comment this is no longer exactly the case.

It looks like native PnP mode support is not in yet, but @yarnpkg/pnpify or the node-modules linker strategy should (allegedly) be usable now.

Of course, I have no idea what this means for you guys, maybe this doesn't help you, but it's a thing now.

Jordan-Hall · 2020-04-30T11:39:08Z

@ElbowBaggins CLI still now working. getting command nx not found

Jordan-Hall · 2020-04-30T18:34:19Z

@FrozenPandaz

this issue is now relating to nx cli i believe https://github.com/nrwl/nx/blob/master/packages/cli/lib/init-local.tsit's looking directly in the node_module folder

vaunus · 2020-08-10T14:15:02Z

I am seeing the same issue here and disappointed NX CLI is breaking builds on yarn2 given it is now at version 2.1. Our project fails to build with default yarn2 settings, i.e. pnp enabled, but works as expected in node-modules mode.

> yarn nx serve

/project-path/.pnp.js:34448
    throw firstError;
    ^

Error: Qualified path resolution failed - none of the candidates can be found on the disk.

Source path: /project-path/node_modules/@nrwl/tao/index.js

Looks like this does indeed relate to the cli looking for tao in the node_modules directory as @Jordan-Hall indicated.

Is it possible to get a fix for this any time soon?

Jordan-Hall · 2020-08-14T18:54:05Z

I am seeing the same issue here and disappointed NX CLI is breaking builds on yarn2 given it is now at version 2.1. Our project fails to build with default yarn2 settings, i.e. pnp enabled, but works as expected in node-modules mode.
> yarn nx serve

/project-path/.pnp.js:34448
    throw firstError;
    ^

Error: Qualified path resolution failed - none of the candidates can be found on the disk.

Source path: /project-path/node_modules/@nrwl/tao/index.js
Looks like this does indeed relate to the cli looking for tao in the node_modules directory as @Jordan-Hall indicated.

Is it possible to get a fix for this any time soon?

Its annoying me now but im too ill to look. Ill carry on soon but they appears to be loads of files referencing direct file path grrr

PetarKirov · 2021-01-20T08:34:51Z

@FrozenPandaz @bekos what's the current status of Yarn v2 PnP support?

derHodrig · 2021-02-12T11:23:16Z

Is still in Progress
angular/angular-cli#16980

Jordan-Hall · 2021-02-14T12:16:09Z

@derHodrig this is internal of NX thats the issue atm

github-actions · 2021-03-29T15:11:04Z

This issue has been automatically marked as stale because it hasn't had any recent activity. It will be closed in 14 days if no further activity occurs.
If we missed this issue please reply to keep it active.
Thanks for being a part of the Nx community! 🙏

fguitton · 2021-03-29T19:29:18Z

This is still of interest for ongoing projects

hydrosquall · 2021-05-01T17:30:39Z

https://github.com/nrwl/nx/blob/master/packages/cli/lib/init-local.ts

It looks like this step was solved in #3709

Is there a checklist of any other known current issues besides references to the string node_modules? After excluding test files (.spec.ts), it looks like there are ~79 references to node_modules (some of them strings) spread across several files remaining.

https://sourcegraph.com/search?q=repo:%5Egithub%5C.com/nrwl/nx%24+node_modules+lang:typescript++-file:.*.spec%5C.ts&patternType=literal

KyleBastien · 2021-05-02T02:19:39Z

We've been running an Nx repo with Yarn v2 PNP on loose mode for a few months now, and so far most things have worked for us.

The biggest exception is the nx migrate latest script, which just outright fails on Yarn v2 PNP. My workaround so far has been to switch Yarn v2 to use the node_modules folder, run the migrate latest script, and then switch back to using PNP before committing the result.

I would assume the migrate latest script's problem stem from something either not declaring a dependency correctly, or there being a string referencing a node_modules path. I'm just not sure which out of the above ~79 it might be.

srosato · 2021-05-31T19:29:31Z

@KyleBastien I wonder how you made it work in loose mode, I've added pnpMode: loose to my .yarnrc file, re-ran yarn install, but I still get an error when running

command: yarn nx affected:test

error:

(node:45) [MODULE_NOT_FOUND] Error: @nrwl/devkit tried to access rxjs, but it isn't declared in its dependencies; this makes the require call ambiguous and unsound.

KyleBastien · 2021-06-01T15:59:01Z

@KyleBastien I wonder how you made it work in loose mode, I've added pnpMode: loose to my .yarnrc file, re-ran yarn install, but I still get an error when running

command: yarn nx affected:test

error:
(node:45) [MODULE_NOT_FOUND] Error: @nrwl/devkit tried to access rxjs, but it isn't declared in its dependencies; this makes the require call ambiguous and unsound.

Note: All of the @nrwl package extensions here are unnecessary after Nx 12.6.2, because they were added to the respective package.json's

You can fix this by utilizing packageExtensions in your .yarnrc.yml file. Mine currently looks like:

packageExtensions:
  # unnecessary after 12.6
  '@nrwl/tao@*':
    dependencies:
      yargs: '15.4.1'
  # unnecessary after 12.6
  '@nrwl/jest@*':
    dependencies:
      identity-obj-proxy: '3.0.0'
  # unnecessary after 12.6
  '@nrwl/devkit@*':
    dependencies:
      rxjs: 6.6.6
  babel-plugin-const-enum@*:
    dependencies:
      '@babel/generator': ^7.9.6

Note: These should probably be added to the package.json's of the respective @nrwl packages. But Yarn does give you the tools to declare the dependencies yourself, so you're unblocked.

srosato · 2021-06-07T21:52:21Z

Thank you, trying it out, although I agree that we should not have to do this

marckassay · 2021-07-30T14:27:26Z

My workaround so far has been to switch Yarn v2 to use the node_modules folder, run the migrate latest script, and then switch back to using PNP before committing the result.

@KyleBastien You mentioned "switch", what exactly do you mean? I assume setting nodeLinker: node-modules in .yarnrc.yml, but anything else? And also, is nx installed globally for this to work? Thanks.

KyleBastien · 2021-07-30T16:48:52Z

@marckassay Here's the steps I follow to get this to work:

Remove pnpMode: loose from .yarnrc.yml
Add nodeLinker: node-modules to .yarnrc.yml
Run yarn to install all packages in node_modules
Run node ./node_modules/@nrwl/tao/index.js migrate latest
Run yarn again to install any changed that the migrate script made to package.json
Run node ./node_modules/@nrwl/tao/index.js migrate --run-migrations
Remove the migrations.json file
Remove nodeLinker: node-modules from .yarnrc.yml
Add pnpMode: loose to .yarnrc.yml
Run yarn again to remove the node_modules directory, and add back the .pnp.cjs file

Lonli-Lokli · 2021-08-05T15:15:36Z

Yarn 3 is already available...

Peeja · 2022-12-23T21:30:48Z

To be clear, as I understand it, Nx absolutely supports Yarn 2+, just not PnP mode. PnP is one of the main developments in Yarn 2, but the traditional node_modules mode is still very much supported.

It would be fantastic to have PnP support in Nx, but it's not as dire as only supporting Yarn 1, which as @tyteen4a03 says is no longer in active development.

tyteen4a03 · 2022-12-23T21:32:08Z

@Peeja I struggled to get Yarn2 to work even in node linking mode.

Peeja · 2022-12-23T21:41:42Z

@tyteen4a03 Ah, that seems more concerning. I haven't tried myself in some time, so I was going by the comments above. But I wonder if conflating the issues is part of what keeps this from moving forward.

AgentEnder · 2022-12-23T22:07:45Z

iirc yarn2 with node modules linker should absolutely work at this point. If anyone is actively experiencing issues in latest please open a new report.

slikts · 2023-02-14T13:55:17Z

When getting used to the zero-installs workflow with Yarn PnP, switching back to node_modules is a bit painful, so it'd be really nice to have this feature.

ild0tt0re · 2023-04-01T13:59:05Z

Is there any roadmap on the work required to implement this feature?

doughlass · 2023-05-30T15:13:48Z

Is there any roadmap on the work required to implement this feature?

Apparently not.

vire · 2023-06-05T13:52:58Z

FYI 16.2.1 works for me with yarn2 - tho I'm using own webpack via run command, but then the next in 16.3.2 is broken as it calls npx next build

nemonemi · 2023-06-05T14:28:06Z

@vire when you say, it worked for you with yarn2, do you mean the Plug N Play version or the node_modules version?

vire · 2023-06-05T14:45:06Z

@vire when you say, it worked for you with yarn2, do you mean the Plug N Play version or the node_modules version?

Plug'n'Play version, without node_modules

amartincolby · 2023-07-07T04:49:07Z

So after reading through this, I assume that the conclusion is, Yarn PnP is not supported and there is no timeframe in which it will be supported? I am willing to transition to Pnpm in service of switching to Nx, but are there at least plans to make a future transition to PnP easy?

meeroslav · 2023-07-28T19:45:47Z

I'm happy to announce that we are working on the PnP support. A lot of the issues have been resolved already (you can try it out with the latest beta).

There are two known issues that we need to fix:

Swc compiler fix from @nx/js was fixed but didn't make it on time for the beta cut
NextJS build fix was reverted due to some issues with our own docs app

If you can, try this beta and let us know if you find any issues (apart from those listed above).

sushruth · 2023-07-28T22:21:54Z

@meeroslav this is the best news I have heard all year. Yarn pnp is not perfect but it is a great option to choose in the current JS ecosystem.

amartincolby · 2023-07-30T04:32:23Z

This could not have come at a better time. I was just balancing pure Yarn Workspaces vs Nx specifically because of this issue.

meeroslav · 2023-07-31T09:20:20Z

Swc compiler is fixed in the latest 16.6.0-beta.8

amartincolby · 2023-08-04T05:46:13Z

How do we enable Plug-n-Play? If I generate a workspace with yarn v3, it successfully uses yarn Berry but does not generate a .pnp.cjs file. It still defaults to using nodeLinker:node-modules.

meeroslav · 2023-08-07T21:44:20Z

You can use this reference:
https://nx.dev/recipes/tips-n-tricks/yarn-pnp

We do not provide a flag for create-nx-workspace to enable PnP automatically.

meeroslav · 2023-08-07T21:45:39Z

Currently, our @nx/angular package is the only one not supporting PnP due to third-party blockers.

If you detect any issues with any other package, please report it here.

meeroslav · 2023-08-11T16:31:32Z

The latest beta has all the changes related to Yarn PnP support. This concludes this feature request.

If you find any problems with Yarn PnP, please open a new issue. Thank you for your patience!

github-actions · 2023-09-11T00:07:03Z

This issue has been closed for more than 30 days. If this issue is still occuring, please open a new issue with more recent context.

Jordan-Hall changed the title ~~yarn 2 Support~~ yarn 2 / Plug N Play Support Jan 27, 2020

FrozenPandaz added blocked: third-party scope: misc Misc issues type: feature labels Jan 30, 2020

Jordan-Hall mentioned this issue Aug 14, 2020

feat(core): add support for yarn 2 #3522

Closed

This was referenced Oct 30, 2020

Do not use require(path.join(workspace.dir type requires - can help some folks already using yarn 2 #3637

Closed

Yarn 2 support #3201

Closed

github-actions bot added the stale label Mar 29, 2021

github-actions bot removed the stale label Mar 29, 2021

KyleBastien mentioned this issue Jul 2, 2021

chore(core): add missing dependencies to devkit and jest package.json #6233

Merged

marckassay mentioned this issue Jul 30, 2021

Unable to successfully execute: nx migrate #6509

Closed

AgentEnder mentioned this issue Mar 2, 2023

Unable to run barebones NestJS app with yarn pnp #15406

Closed

leosvelperez mentioned this issue Jun 28, 2023

Nx16 - Angular 16 not updated to the latest with warning (yarn 3) #17642

Closed

4 tasks

ed-jinyoung-park mentioned this issue Jul 17, 2023

모노레포에 대한 이해 FEStudy-boostcamp/FEStudy#2

Open

meeroslav self-assigned this Jul 28, 2023

meeroslav added scope: core core nx functionality and removed blocked: third-party scope: misc Misc issues labels Jul 31, 2023

meeroslav closed this as completed Aug 11, 2023

github-actions bot added the outdated label Sep 11, 2023

github-actions bot locked as resolved and limited conversation to collaborators Sep 11, 2023

yarn 2 / Plug N Play Support #2386

yarn 2 / Plug N Play Support #2386

Comments

Jordan-Hall commented Jan 27, 2020 • edited Loading

Prerequisites

Expected Behavior

Current Behavior

Failure Information (for bugs)

Steps to Reproduce

Context

Failure Logs

Other

Jordan-Hall commented Jan 27, 2020

ElbowBaggins commented Mar 5, 2020

Jordan-Hall commented Apr 30, 2020

Jordan-Hall commented Apr 30, 2020

vaunus commented Aug 10, 2020

Jordan-Hall commented Aug 14, 2020

PetarKirov commented Jan 20, 2021

derHodrig commented Feb 12, 2021

Jordan-Hall commented Feb 14, 2021

github-actions bot commented Mar 29, 2021

fguitton commented Mar 29, 2021

hydrosquall commented May 1, 2021

KyleBastien commented May 2, 2021

srosato commented May 31, 2021 • edited Loading

KyleBastien commented Jun 1, 2021 • edited Loading

srosato commented Jun 7, 2021

marckassay commented Jul 30, 2021 • edited Loading

KyleBastien commented Jul 30, 2021

Lonli-Lokli commented Aug 5, 2021

Peeja commented Dec 23, 2022

tyteen4a03 commented Dec 23, 2022

Peeja commented Dec 23, 2022

AgentEnder commented Dec 23, 2022

slikts commented Feb 14, 2023

ild0tt0re commented Apr 1, 2023

doughlass commented May 30, 2023

vire commented Jun 5, 2023

nemonemi commented Jun 5, 2023 • edited Loading

vire commented Jun 5, 2023

amartincolby commented Jul 7, 2023

meeroslav commented Jul 28, 2023

sushruth commented Jul 28, 2023

amartincolby commented Jul 30, 2023

meeroslav commented Jul 31, 2023

amartincolby commented Aug 4, 2023

meeroslav commented Aug 7, 2023

meeroslav commented Aug 7, 2023

meeroslav commented Aug 11, 2023

github-actions bot commented Sep 11, 2023

Jordan-Hall commented Jan 27, 2020 •

edited

Loading

srosato commented May 31, 2021 •

edited

Loading

KyleBastien commented Jun 1, 2021 •

edited

Loading

marckassay commented Jul 30, 2021 •

edited

Loading

nemonemi commented Jun 5, 2023 •

edited

Loading