Improve Local Nx Plugins Loading

[jogelin]

Description:

In Nx, you can generate a specialized library called an Nx Plugin, which includes powerful built-in features such as:

• Generators
• Executors
• Migrations
• Inferred Configurations

When it comes to loading plugins for inferred configurations, Nx compiles and loads these plugins at runtime. However, there’s a limitation: Nx will only load the entry point specified in the main property of the plugin’s Project Configuration.

Because of that limitation, some use cases cannot be covered!

Current

❌ Cannot Use a Secondary Entry Point for Local Nx Plugins

If you need to share some functions that are unrelated to the same use case, it is recommended to create separate entry points and use the one relevant to your use case. This way, you won’t import libraries that you don’t need. This is important to avoid unnecessary compilation, especially when loading multiple plugins locally.

However, it is not possible to expose the createNodes() API in a secondary entry point for a Local Nx Plugin.

For example, if you create a separate plugin.ts file to expose your createNodes():

and declare it in your tsconfig.base.json like:

{
    "paths": {
          "@org/my-lib/plugin": ["libs/my-lib/plugin.ts"],
     }
}

and try to access it in your nx.json like this:

  "plugins": [
    {
      "plugin": "@org/my-lib/plugin"
    }
  ],

Nx will not load your libs/my-lib/plugin.ts. Instead, it will fall back to the main entry point specified in your project configuration, typically libs/my-lib/src/index.ts.

❌ Cannot Use Inferred Configuration for Local Nx Plugins

When using Nx Plugins in a monorepo, you may end up with many plugins and similar configurations. In other projects, inferring configurations can be useful, and the same principle applies to Nx Plugins.

For example, you might have an additional plugin responsible for managing all other plugins, similar to how Nx does with @nx/plugin:

  "plugins": [
    {
      "plugin": "@org/my-plugin/plugin"
    }
  ],

However, Nx will fail in this case. To load a plugin, Nx requires reading the main property from your project configuration, but since this is part of your inferred configuration, it won’t be available in your project.json.

Expected:

✅ Load Nx Plugins from the Entry Point Specified in the tsconfig Path

I expect that if I declare a path pointing to a different entry point like this:

{
    "paths": {
          "@org/my-lib/plugin": ["libs/my-lib/plugin.ts"],
          "@org/my-lib/*": ["libs/my-lib/*"],
     }
}

Nx should allow loading the plugin both locally and in distributed repositories using:

  "plugins": [
    {
      "plugin": "@org/my-lib/plugin"
    }
  ],

✅ Allow Inferred Configuration for Nx Plugin Projects

I expect that I can use inferred configurations for any type of project, including Nx Plugin projects.

Suggestions

• I am uncertain which use case is being addressed by defaulting to the main entry point. If the path is found in tsconfig.base.json, shouldn’t Nx use it directly without loading the project configuration?

• When searching for the path matching the plugin import, could we support wildcards instead of requiring perfect matches? This would reduce the need for declaring numerous paths.

Hacky solution

It is possible to patch Nx to support these use cases by bypassing the main property and avoiding the fallback to src/index.ts.

Here’s a diff that outlines the changes:

diff --git a/node_modules/nx/src/project-graph/plugins/loader.js b/node_modules/nx/src/project-graph/plugins/loader.js
index ace1ad5..d678688 100644
--- a/node_modules/nx/src/project-graph/plugins/loader.js
+++ b/node_modules/nx/src/project-graph/plugins/loader.js
@@ -83,16 +83,16 @@ function registerPluginTSTranspiler() {
     };
 }
 function lookupLocalPlugin(importPath, projects, root = workspace_root_1.workspaceRoot) {
-    const projectConfig = findNxProjectForImportPath(importPath, projects, root);
+    const { tsConfigPath, projectConfig } = findNxProjectForImportPath(importPath, projects, root);
     if (!projectConfig) {
         return null;
     }
-    return { path: path.join(root, projectConfig.root), projectConfig };
+    return { path: path.join(root, projectConfig.root), tsConfigPath, projectConfig };
 }
 function findNxProjectForImportPath(importPath, projects, root = workspace_root_1.workspaceRoot) {
     const tsConfigPaths = readTsConfigPaths(root);
     const possiblePaths = tsConfigPaths[importPath]?.map((p) => (0, path_1.normalizePath)(path.relative(root, path.join(root, p))));
     if (possiblePaths?.length) {
         const projectRootMappings = new Map();
         const projectNameMap = new Map();
         for (const projectRoot in projects) {
@@ -103,7 +103,7 @@ function findNxProjectForImportPath(importPath, projects, root = workspace_root_
         for (const tsConfigPath of possiblePaths) {
             const nxProject = (0, find_project_for_path_1.findProjectForPath)(tsConfigPath, projectRootMappings);
             if (nxProject) {
-                return projectNameMap.get(nxProject);
+                return { tsConfigPath, projectConfig: projectNameMap.get(nxProject) };
             }
         }
         logger_1.logger.verbose('Unable to find local plugin', possiblePaths, projectRootMappings);
@@ -152,8 +152,9 @@ function getPluginPathAndName(moduleName, paths, projects, root) {
             const plugin = resolveLocalNxPlugin(moduleName, projects, root);
             if (plugin) {
                 registerTSTranspiler = true;
-                const main = readPluginMainFromProjectConfiguration(plugin.projectConfig);
-                pluginPath = main ? path.join(root, main) : plugin.path;
+                pluginPath = plugin.tsConfigPath;
             }
             else {
                 logger_1.logger.error(`Plugin listed in \`nx.json\` not found: ${moduleName}`);

Last thought

I see that Nx follows this approach internally. For example, in the @nx/angular plugin, generators and executors are exposed through separate entry points.

@AgentEnder, I remember you mentioned the potential need to review how plugins are loaded. In the meantime, do you think it makes sense to adapt Nx’s code as suggested in the patch above?

[RobbyRabbitman]

I have a secondary entry point in "src/plugin" and it works locally. https://github.com/RobbyRabbitman/nx-plus/tree/demo/local-plugin. I could not create a stackblitz, somehow nx is not working in there, but i created a branch to fork/clone/download.

[jogelin]

It is working because you are exporting the createNodesV2 in your main file, in your packages/web-test-runner/src/index.ts.

As specified above, from your tsconfig path:

      "@robby-rabbitman/nx-plus-web-test-runner/plugin": [
        "packages/web-test-runner/src/plugin.ts"
      ]

Nx will load the related project configuration and use the path from the main property which is pointing to packages/web-test-runner/src/index.ts and not your plugin.ts

I did the test on your demo.

1. Without any modification:
   

2. Removing createNodesV2:
   
   and run:
   

As you can see, the target is not here.

[RobbyRabbitman]

Alright, that was a happy little accident :D

In general, should you export a plugin via a secondary entry point?

[jogelin]

I am creating libraries that are exporting entries for differents use cases:

• generators
• executors
• plugins
• shared configurations
• utilities

I don't want to mix them because that means you'll import code you don't need.

If you expose too much code in your index.ts and your plugin is using it, that means all of that code will be compiled at runtime which can make the generation of the graph slower.

Btw, if you publish your plugin, it will work because Nx code will first try require('@org/my-lib/plugin') and because it is available in the node_modules, it is ok. However, to resolve local plugin, it uses a different path.

[RobbyRabbitman]

ok, seems logic to me. I think I will also remove the plugin from the index.ts, since I don't need it locally. Hopefully there is a solution for that in the future though :D

[jogelin]

Related to that, this is the approach I recommend for the moment:

Current:

Improved:

[RobbyRabbitman]

I am using exactly this approach with local plugins - Each plugin is a nx project with 1 file for the plugin itself and 1 file for unit testing the plugin

[RobbyRabbitman]

@jogelin do u have a working example of a local plugin with ts project references instead of paths?

[RobbyRabbitman]

ok i found a solution - invoke tsc directly (without nx cli) e.g. in a post install hook tsc --build path/to/tsconfig and make sure to set type: commonjs, main: ... and types: ...