Re: Fix plugins not being found when no node_modules exists #289

benatshippabo · 2021-11-30T01:51:46Z

Description

Fixes #170.

As I mentioned in the issue, Yarn 2 uses a different package system that does not have node_modules. As such, relying on node_modules should be discouraged.

Yarn suggests using the resolve package, and this is the approach I have taken.

Please let me know if any further changes are required. I tried to conform to idioms of the code, but for testing in particular I wasn't sure what the guidelines were.

Technical Details

Uses native require.resolve to resolve dependency package's path instead of the third party resolve package. Addresses @mdonnalley's comment #171 (comment)

For Yarn 2+ zero install users who needed this issue resolved

you may still have to use yarn patch to add main: lib/index.js inside the package.json of oclif plugins that still fail to resolve

Context

#171

This test should pass when it has been updated to use resolve

Assume resolve returns an entry file not a directory

salesforce-cla · 2021-11-30T01:51:49Z

Thanks for the contribution! Before we can merge this, we need @benatshippabo to sign the Salesforce.com Contributor License Agreement.

codecov-commenter · 2021-11-30T01:53:15Z

Codecov Report

Merging #289 (58b6ad7) into master (07a0670) will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff           @@
##           master    #289   +/-   ##
======================================
  Coverage    0.00%   0.00%           
======================================
  Files           2       2           
  Lines          12      12           
  Branches        3       3           
======================================
  Misses         12      12

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 07a0670...58b6ad7. Read the comment docs.

benatshippabo · 2021-11-30T01:56:01Z

I signed the CLA, but it seems that the CI check is still failing 🤷‍♂️ Here is the error id 7lp4kni03

mdonnalley · 2021-11-30T16:02:49Z

@benatshippabo I tested this locally with sfdx-cli and I'm still seeing the same issue:

~/repos/salesforcecli/sfdx-cli [main] : bin/run auth:list
(node:90678) Error Plugin: sfdx-cli: could not find package.json with {
  type: 'link',
  root: '/Users/mdonnalley/repos/salesforcecli/plugin-auth',
  name: '@salesforce/plugin-auth',
  tag: undefined
}
module: @oclif/[email protected]
task: loadPlugins
plugin: sfdx-cli
root: /Users/mdonnalley/repos/salesforcecli/sfdx-cli
See more details with DEBUG=*
(Use `node --trace-warnings ...` to show where the warning was created)
(node:90678) Error Plugin: sfdx-cli: could not find package.json with {
  type: 'dev',
  root: '/Users/mdonnalley/repos/salesforcecli/sfdx-cli',
  name: '@oclif/plugin-command-snapshot'
}
module: @oclif/[email protected]
task: loadPlugins
plugin: sfdx-cli
root: /Users/mdonnalley/repos/salesforcecli/sfdx-cli
See more details with DEBUG=*
(node:90678) Error Plugin: sfdx-cli: could not find package.json with {
  type: 'dev',
  root: '/Users/mdonnalley/repos/salesforcecli/sfdx-cli',
  name: '@salesforce/plugin-release-management'
}
module: @oclif/[email protected]
task: loadPlugins
plugin: sfdx-cli
root: /Users/mdonnalley/repos/salesforcecli/sfdx-cli
See more details with DEBUG=*
(node:90678) Error Plugin: sfdx-cli: could not find package.json with {
  type: 'core',
  root: '/Users/mdonnalley/repos/salesforcecli/sfdx-cli',
  name: '@oclif/plugin-autocomplete'
}
module: @oclif/[email protected]
task: loadPlugins
plugin: sfdx-cli
root: /Users/mdonnalley/repos/salesforcecli/sfdx-cli
See more details with DEBUG=*
(node:90678) Error Plugin: sfdx-cli: could not find package.json with {
  type: 'core',
  root: '/Users/mdonnalley/repos/salesforcecli/sfdx-cli',
  name: '@oclif/plugin-commands'
}
module: @oclif/[email protected]
task: loadPlugins
plugin: sfdx-cli
root: /Users/mdonnalley/repos/salesforcecli/sfdx-cli
See more details with DEBUG=*
(node:90678) Error Plugin: sfdx-cli: could not find package.json with {
  type: 'core',
  root: '/Users/mdonnalley/repos/salesforcecli/sfdx-cli',
  name: '@oclif/plugin-help'
}
module: @oclif/[email protected]
task: loadPlugins
plugin: sfdx-cli
root: /Users/mdonnalley/repos/salesforcecli/sfdx-cli
See more details with DEBUG=*
(node:90678) Error Plugin: sfdx-cli: could not find package.json with {
  type: 'core',
  root: '/Users/mdonnalley/repos/salesforcecli/sfdx-cli',
  name: '@oclif/plugin-not-found'
}
module: @oclif/[email protected]
task: loadPlugins
plugin: sfdx-cli
root: /Users/mdonnalley/repos/salesforcecli/sfdx-cli
See more details with DEBUG=*
(node:90678) Error Plugin: sfdx-cli: could not find package.json with {
  type: 'core',
  root: '/Users/mdonnalley/repos/salesforcecli/sfdx-cli',
  name: '@oclif/plugin-plugins'
}
module: @oclif/[email protected]
task: loadPlugins
plugin: sfdx-cli
root: /Users/mdonnalley/repos/salesforcecli/sfdx-cli
See more details with DEBUG=*
(node:90678) Error Plugin: sfdx-cli: could not find package.json with {
  type: 'core',
  root: '/Users/mdonnalley/repos/salesforcecli/sfdx-cli',
  name: '@oclif/plugin-update'
}
module: @oclif/[email protected]
task: loadPlugins
plugin: sfdx-cli
root: /Users/mdonnalley/repos/salesforcecli/sfdx-cli
See more details with DEBUG=*
(node:90678) Error Plugin: sfdx-cli: could not find package.json with {
  type: 'core',
  root: '/Users/mdonnalley/repos/salesforcecli/sfdx-cli',
  name: '@oclif/plugin-warn-if-update-available'
}
module: @oclif/[email protected]
task: loadPlugins
plugin: sfdx-cli
root: /Users/mdonnalley/repos/salesforcecli/sfdx-cli
See more details with DEBUG=*
(node:90678) Error Plugin: sfdx-cli: could not find package.json with {
  type: 'core',
  root: '/Users/mdonnalley/repos/salesforcecli/sfdx-cli',
  name: '@oclif/plugin-which'
}
module: @oclif/[email protected]
task: loadPlugins
plugin: sfdx-cli
root: /Users/mdonnalley/repos/salesforcecli/sfdx-cli
See more details with DEBUG=*
(node:90678) Error Plugin: sfdx-cli: could not find package.json with {
  type: 'core',
  root: '/Users/mdonnalley/repos/salesforcecli/sfdx-cli',
  name: '@salesforce/plugin-alias'
}
module: @oclif/[email protected]
task: loadPlugins
plugin: sfdx-cli
root: /Users/mdonnalley/repos/salesforcecli/sfdx-cli
See more details with DEBUG=*
(node:90678) Error Plugin: sfdx-cli: could not find package.json with {
  type: 'core',
  root: '/Users/mdonnalley/repos/salesforcecli/sfdx-cli',
  name: '@salesforce/plugin-apex'
}
module: @oclif/[email protected]
task: loadPlugins
plugin: sfdx-cli
root: /Users/mdonnalley/repos/salesforcecli/sfdx-cli
See more details with DEBUG=*
(node:90678) Error Plugin: sfdx-cli: could not find package.json with {
  type: 'core',
  root: '/Users/mdonnalley/repos/salesforcecli/sfdx-cli',
  name: '@salesforce/plugin-auth'
}
module: @oclif/[email protected]
task: loadPlugins
plugin: sfdx-cli
root: /Users/mdonnalley/repos/salesforcecli/sfdx-cli
See more details with DEBUG=*
(node:90678) Error Plugin: sfdx-cli: could not find package.json with {
  type: 'core',
  root: '/Users/mdonnalley/repos/salesforcecli/sfdx-cli',
  name: '@salesforce/plugin-config'
}
module: @oclif/[email protected]
task: loadPlugins
plugin: sfdx-cli
root: /Users/mdonnalley/repos/salesforcecli/sfdx-cli
See more details with DEBUG=*
(node:90678) Error Plugin: sfdx-cli: could not find package.json with {
  type: 'core',
  root: '/Users/mdonnalley/repos/salesforcecli/sfdx-cli',
  name: '@salesforce/plugin-custom-metadata'
}
module: @oclif/[email protected]
task: loadPlugins
plugin: sfdx-cli
root: /Users/mdonnalley/repos/salesforcecli/sfdx-cli
See more details with DEBUG=*
(node:90678) Error Plugin: sfdx-cli: could not find package.json with {
  type: 'core',
  root: '/Users/mdonnalley/repos/salesforcecli/sfdx-cli',
  name: '@salesforce/plugin-data'
}
module: @oclif/[email protected]
task: loadPlugins
plugin: sfdx-cli
root: /Users/mdonnalley/repos/salesforcecli/sfdx-cli
See more details with DEBUG=*
(node:90678) Error Plugin: sfdx-cli: could not find package.json with {
  type: 'core',
  root: '/Users/mdonnalley/repos/salesforcecli/sfdx-cli',
  name: '@salesforce/plugin-generator'
}
module: @oclif/[email protected]
task: loadPlugins
plugin: sfdx-cli
root: /Users/mdonnalley/repos/salesforcecli/sfdx-cli
See more details with DEBUG=*
(node:90678) Error Plugin: sfdx-cli: could not find package.json with {
  type: 'core',
  root: '/Users/mdonnalley/repos/salesforcecli/sfdx-cli',
  name: '@salesforce/plugin-limits'
}
module: @oclif/[email protected]
task: loadPlugins
plugin: sfdx-cli
root: /Users/mdonnalley/repos/salesforcecli/sfdx-cli
See more details with DEBUG=*
(node:90678) Error Plugin: sfdx-cli: could not find package.json with {
  type: 'core',
  root: '/Users/mdonnalley/repos/salesforcecli/sfdx-cli',
  name: '@salesforce/plugin-org'
}
module: @oclif/[email protected]
task: loadPlugins
plugin: sfdx-cli
root: /Users/mdonnalley/repos/salesforcecli/sfdx-cli
See more details with DEBUG=*
(node:90678) Error Plugin: sfdx-cli: could not find package.json with {
  type: 'core',
  root: '/Users/mdonnalley/repos/salesforcecli/sfdx-cli',
  name: '@salesforce/plugin-source'
}
module: @oclif/[email protected]
task: loadPlugins
plugin: sfdx-cli
root: /Users/mdonnalley/repos/salesforcecli/sfdx-cli
See more details with DEBUG=*
(node:90678) Error Plugin: sfdx-cli: could not find package.json with {
  type: 'core',
  root: '/Users/mdonnalley/repos/salesforcecli/sfdx-cli',
  name: '@salesforce/plugin-schema'
}
module: @oclif/[email protected]
task: loadPlugins
plugin: sfdx-cli
root: /Users/mdonnalley/repos/salesforcecli/sfdx-cli
See more details with DEBUG=*
(node:90678) Error Plugin: sfdx-cli: could not find package.json with {
  type: 'core',
  root: '/Users/mdonnalley/repos/salesforcecli/sfdx-cli',
  name: '@salesforce/plugin-telemetry'
}
module: @oclif/[email protected]
task: loadPlugins
plugin: sfdx-cli
root: /Users/mdonnalley/repos/salesforcecli/sfdx-cli
See more details with DEBUG=*
(node:90678) Error Plugin: sfdx-cli: could not find package.json with {
  type: 'core',
  root: '/Users/mdonnalley/repos/salesforcecli/sfdx-cli',
  name: '@salesforce/plugin-templates'
}
module: @oclif/[email protected]
task: loadPlugins
plugin: sfdx-cli
root: /Users/mdonnalley/repos/salesforcecli/sfdx-cli
See more details with DEBUG=*
(node:90678) Error Plugin: sfdx-cli: could not find package.json with {
  type: 'core',
  root: '/Users/mdonnalley/repos/salesforcecli/sfdx-cli',
  name: '@salesforce/plugin-trust'
}
module: @oclif/[email protected]
task: loadPlugins
plugin: sfdx-cli
root: /Users/mdonnalley/repos/salesforcecli/sfdx-cli
See more details with DEBUG=*
(node:90678) Error Plugin: sfdx-cli: could not find package.json with {
  type: 'core',
  root: '/Users/mdonnalley/repos/salesforcecli/sfdx-cli',
  name: '@salesforce/plugin-user'
}
module: @oclif/[email protected]
task: loadPlugins
plugin: sfdx-cli
root: /Users/mdonnalley/repos/salesforcecli/sfdx-cli
See more details with DEBUG=*
(node:90678) Error Plugin: sfdx-cli: could not find package.json with {
  type: 'core',
  root: '/Users/mdonnalley/repos/salesforcecli/sfdx-cli',
  name: '@salesforce/sfdx-plugin-lwc-test'
}
module: @oclif/[email protected]
task: loadPlugins
plugin: sfdx-cli
root: /Users/mdonnalley/repos/salesforcecli/sfdx-cli
See more details with DEBUG=*
(node:90678) Error Plugin: sfdx-cli: could not find package.json with {
  type: 'core',
  root: '/Users/mdonnalley/repos/salesforcecli/sfdx-cli',
  name: 'salesforce-alm'
}
module: @oclif/[email protected]
task: loadPlugins
plugin: sfdx-cli
root: /Users/mdonnalley/repos/salesforcecli/sfdx-cli
See more details with DEBUG=*
 ›   Error: command auth:list not found

benatshippabo · 2021-12-01T00:16:58Z

@benatshippabo I tested this locally with sfdx-cli and I'm still seeing the same issue:

Thanks for the prompt feedback @mdonnalley! I'm not sure how to test this against that repository, ~~but I think I found the problem. Could you please re-test it?~~

edit:
nvm, I don't think the new commit makes a difference since it all gets resolved in the up function anyways, going to revert the commit.

edit 2:
referenced #171 (comment) to make some edits, hopefully this fixes it @mdonnalley could you please re-test? 🙏

mdonnalley · 2021-12-01T16:12:23Z

@benatshippabo I'm still seeing the same error with your latest changes.

You can test against sfdx-cli like this:

git clone [email protected]:salesforcecli/sfdx-cli.git
cd sfdx-cli
yarn link @oclif/config
bin/run config:list

Just make sure to compile your changes before running bin/run config:list again

benatshippabo · 2021-12-01T17:57:20Z

@benatshippabo I'm still seeing the same error with your latest changes.

You can test against sfdx-cli like this:
git clone [email protected]:salesforcecli/sfdx-cli.git
cd sfdx-cli
yarn link @oclif/config
bin/run config:list
Just make sure to compile your changes before running bin/run config:list again

Thanks for the test steps.

I just figured out the issue! The issue is that the package.json for those missing plugins do not specify the main field. Because it is missing that field in the package.json, the node module resolution algorithm will look at the root level for index.js (see LOAD_AS_DIRECTORY(X) step 2 and LOAD_INDEX(X)). We can test this by trying to use the node module resolution to try finding it in the node repl:

# at the root directory of sfdx
yarn
node -e "console.log(require.resolve('@oclif/parser'))"
node -e "console.log(require.resolve('@oclif/plugin-commands'))"

If we inspect each of those package's package.json we can see that the compiled index.js is put inside the lib folder, but from the example, only @oclif/parser specifies main and that is why require.resolve is able to find @oclif/parser and not @oclif/plugin-commands.

We can test the fix by adding main: "lib/index.js" inside node_module/@oclif/plugin-commands/package.json and then rerunning:

# we can see here that it will now resolve properly 😃
node -e "console.log(require.resolve('@oclif/plugin-commands'))"

I think this will require some minor updates to all the missing plugins so that is is compliant with the node module resolution algorithm.

List of plugins missing the `main` field in `package.json`

'@oclif/plugin-command-snapshot'
'@salesforce/plugin-release-management'
'@oclif/plugin-autocomplete'
'@oclif/plugin-update'
'@oclif/plugin-warn-if-update-available'
'@oclif/plugin-which'
'@salesforce/plugin-alias'
'@salesforce/plugin-apex'
'@salesforce/plugin-auth'
'@salesforce/plugin-config'
'@salesforce/plugin-custom-metadata'
'@salesforce/plugin-data'
'@salesforce/plugin-limits'
'@salesforce/plugin-org'
'@salesforce/plugin-source'
'@salesforce/plugin-schema'
'@salesforce/plugin-telemetry'
'@salesforce/plugin-templates'
'@salesforce/plugin-trust'
'@salesforce/plugin-user'
'@salesforce/sfdx-plugin-lwc-test'
'@oclif/plugin-warn-if-update-available'

This will probably require some effort to update these plugins. Thoughts @mdonnalley?

mdonnalley · 2021-12-01T18:16:38Z

@benatshippabo Good find!

Unfortunately, if we require that plugins must specify the main field then this would be a breaking change, which we're not willing to do on this package since it will soon be in maintenance mode.

Is it possible to fallback to the old behavior whenever a module isn't found?

benatshippabo · 2021-12-02T01:45:32Z

src/plugin.ts

-    }
-    yield from
-  }
+async function findRootLegacy(name: string | undefined, root: string): Promise<string | undefined> {


@benatshippabo Good find!

Unfortunately, if we require that plugins must specify the main field then this would be a breaking change, which we're not willing to do on this package since it will soon be in maintenance mode.

That's unfortunate, maybe we can see these changes once the package that will supersede this one is released?

Is it possible to fallback to the old behavior whenever a module isn't found?

Done! @mdonnalley I re-added the legacy dependency resolution function as a fallback

mdonnalley · 2021-12-02T16:37:53Z

Thanks @benatshippabo

Testing is going well so far. I'm going to test it out a bit more before merging though

oclif/core is the heir apparent to this project so it'd be great if you could port these changes over there as well. It should be a fairly straightforward copy and paste since most of this code is unchanged: https://github.com/oclif/core/blob/main/src/config/plugin.ts#L41

I would prefer to keep the legacy fallback to persevere backwards compatibility between plugins that use oclif/config and plugins that use oclif/core. However, I will add a note about adding the main field to the soon-to-be-published migration guide. That way we can drop the legacy code path if/when we do a major version bump of oclif/core

cromwellryan · 2021-12-02T17:17:21Z

@benatshippabo - thanks so much for your thorough and thoughtful work on this PR. This comment is amazing. 🙌

benatshippabo · 2021-12-03T01:29:15Z

Thanks @benatshippabo

Testing is going well so far. I'm going to test it out a bit more before merging though

Awesome! Thanks for being so responsive for this pull request.

oclif/core is the heir apparent to this project so it'd be great if you could port these changes over there as well. It should be a fairly straightforward copy and paste since most of this code is unchanged: https://github.com/oclif/core/blob/main/src/config/plugin.ts#L41

I would prefer to keep the legacy fallback to persevere backwards compatibility between plugins that use oclif/config and plugins that use oclif/core. However, I will add a note about adding the main field to the soon-to-be-published migration guide. That way we can drop the legacy code path if/when we do a major version bump of oclif/core

Done oclif/core#309

dylanpomeroy · 2021-12-07T19:00:40Z

Hi @mdonnalley

Was there a release cut for this merge? If not - do you know when we can expect a version to come out with these changes?

Thanks!

dylanpomeroy · 2021-12-13T20:11:40Z

@mdonnalley bump on the above ^^ any way we can get a tag version to use with this change?

Thanks!

benatshippabo · 2021-12-14T16:47:49Z

@mdonnalley bump on the above ^^ any way we can get a tag version to use with this change?

Thanks!

The latest version 1.18.2 has this fix

TPHRyan and others added 5 commits August 30, 2020 23:10

Implement test for plugin load

ee785b2

This test should pass when it has been updated to use resolve

Update findRoot to use resolve function

a067b51

Fix findRoot not working due to incorrect assumptions

7925d35

Assume resolve returns an entry file not a directory

use require.resolve instead of resolve package

3abfe02

remove resolve types

ce633d0

salesforce-cla bot added the cla:missing label Nov 30, 2021

benatshippabo mentioned this pull request Nov 30, 2021

Fix plugins not being found when no node_modules exists #171

Merged

return directory instead of resolved file

81a00f5

salesforce-cla bot added cla:signed and removed cla:missing labels Dec 1, 2021

benatshippabo added 2 commits November 30, 2021 18:10

undo previous commit

c34e109

adds root to one of the starting resolution paths

e69ebd4

fallback to legacy dependency resolution if not found

12f8d90

benatshippabo commented Dec 2, 2021

View reviewed changes

grammar

58b6ad7

mdonnalley self-requested a review December 2, 2021 16:47

mdonnalley approved these changes Dec 2, 2021

View reviewed changes

benatshippabo mentioned this pull request Dec 3, 2021

chore(sync): push @oclif/config dependency resolution changes upstream oclif/core#309

Merged

mdonnalley merged commit 208b2d7 into oclif:master Dec 6, 2021

Nicoowr mentioned this pull request Jan 6, 2022

Yarn V2 compatibility apollographql/apollo-tooling#1850

Open

danieltroger mentioned this pull request Oct 3, 2023

Oclif can't find plugins when using yarn pnp/yarn 4 oclif/core#806

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Re: Fix plugins not being found when no node_modules exists #289

Re: Fix plugins not being found when no node_modules exists #289

benatshippabo commented Nov 30, 2021 •

edited

Loading

salesforce-cla bot commented Nov 30, 2021

codecov-commenter commented Nov 30, 2021 •

edited

Loading

benatshippabo commented Nov 30, 2021 •

edited

Loading

mdonnalley commented Nov 30, 2021

benatshippabo commented Dec 1, 2021 •

edited

Loading

mdonnalley commented Dec 1, 2021

benatshippabo commented Dec 1, 2021

mdonnalley commented Dec 1, 2021

benatshippabo Dec 2, 2021 •

edited

Loading

mdonnalley commented Dec 2, 2021

cromwellryan commented Dec 2, 2021

benatshippabo commented Dec 3, 2021

dylanpomeroy commented Dec 7, 2021 •

edited

Loading

dylanpomeroy commented Dec 13, 2021

benatshippabo commented Dec 14, 2021

Re: Fix plugins not being found when no node_modules exists #289

Re: Fix plugins not being found when no node_modules exists #289

Conversation

benatshippabo commented Nov 30, 2021 • edited Loading

Description

Technical Details

For Yarn 2+ zero install users who needed this issue resolved

Context

salesforce-cla bot commented Nov 30, 2021

codecov-commenter commented Nov 30, 2021 • edited Loading

Codecov Report

benatshippabo commented Nov 30, 2021 • edited Loading

mdonnalley commented Nov 30, 2021

benatshippabo commented Dec 1, 2021 • edited Loading

mdonnalley commented Dec 1, 2021

benatshippabo commented Dec 1, 2021

mdonnalley commented Dec 1, 2021

benatshippabo Dec 2, 2021 • edited Loading

Choose a reason for hiding this comment

mdonnalley commented Dec 2, 2021

cromwellryan commented Dec 2, 2021

benatshippabo commented Dec 3, 2021

dylanpomeroy commented Dec 7, 2021 • edited Loading

dylanpomeroy commented Dec 13, 2021

benatshippabo commented Dec 14, 2021

benatshippabo commented Nov 30, 2021 •

edited

Loading

codecov-commenter commented Nov 30, 2021 •

edited

Loading

benatshippabo commented Nov 30, 2021 •

edited

Loading

benatshippabo commented Dec 1, 2021 •

edited

Loading

benatshippabo Dec 2, 2021 •

edited

Loading

dylanpomeroy commented Dec 7, 2021 •

edited

Loading