feat(js): introduce rust-based typescript dependency processor

[FrozenPandaz]

Current Behavior

Nx uses the typescript npm package to analyze dependencies between files.

Expected Behavior

Nx uses the swc Rust crates to analyze dependencies between files.

The new dependency analysis is faster and also handles more cases. Most notably, require.resolve('dep') will now also result in a dependency while it did not before. Some dependencies which were Static, might now be Dynamic or the other way around, but the new way should be more accurate.

This new dependency is enabled via NX_NATIVE_TS_DEPS environment variable:

• The default remains the same and is still typescript right now. This will change to the new one in the near future.
• NX_NATIVE_TS_DEPS=true will use the new native code.
• NX_NATIVE_TS_DEPS=debug will use the new native code, but also analyze the files using the typescript files so the 2 can be compared.

The new dependency analysis uses the Lexer to create our own parser. It's compared in unit tests to the results from swc_ecma_dep_graph::analyze_dependencies(&module, &comments); to ensure parity with SWC.

Testing

nx

- Time for 'build typescript dependencies' 630.4862889945507
+ Time for 'build typescript dependencies' 302.4862850010395

+ webpack -> npm:swc-loader (packages/webpack/src/executors/webpack/webpack.impl.ts)
+ webpack -> npm:@swc/core (packages/webpack/src/executors/webpack/webpack.impl.ts)
+ webpack -> npm:source-map-loader (packages/webpack/src/utils/with-nx.ts)
+ js -> npm:@babel/preset-env (packages/js/babel.ts)
+ webpack -> npm:swc-loader (packages/webpack/src/utils/with-nx.ts)
+ webpack -> npm:sass-loader (packages/webpack/src/utils/with-web.ts)
+ webpack -> npm:less-loader (packages/webpack/src/utils/with-web.ts)
+ webpack -> npm:stylus-loader (packages/webpack/src/utils/with-web.ts)
+ webpack -> npm:style-loader (packages/webpack/src/utils/with-web.ts)
+ webpack -> npm:css-loader (packages/webpack/src/utils/with-web.ts)
+ webpack -> npm:postcss-loader (packages/webpack/src/utils/with-web.ts)
+ playwright -> npm:@playwright/test/cli (packages/playwright/src/executors/playwright/playwright.ts)
+ js -> npm:@babel/preset-typescript (packages/js/babel.ts)
+ js -> npm:@babel/plugin-transform-runtime (packages/js/babel.ts)
+ js -> npm:@babel/runtime (packages/js/babel.ts)
+ js -> npm:babel-plugin-macros (packages/js/babel.ts)
+ js -> npm:babel-plugin-transform-typescript-metadata (packages/js/babel.ts)
+ js -> npm:@babel/plugin-proposal-decorators (packages/js/babel.ts)
+ js -> npm:@babel/plugin-proposal-class-properties (packages/js/babel.ts)
+ js -> npm:babel-plugin-const-enum (packages/js/babel.ts)
+ js -> npm:source-map-support (packages/js/src/executors/node/node.impl.ts)
+ js -> npm:verdaccio (packages/js/src/executors/verdaccio/verdaccio.impl.ts)
+ js -> nx (packages/js/src/plugins/jest/start-local-registry.ts)
+ react-native -> npm:react-native/cli.js (packages/react-native/src/executors/build-android/build-android.impl.ts)
+ react-native -> npm:react-native/cli.js (packages/react-native/src/executors/build-ios/build-ios.impl.ts)
+ react-native -> npm:react-native/cli.js (packages/react-native/src/executors/bundle/bundle.impl.ts)
+ react-native -> npm:react-native/cli.js (packages/react-native/src/executors/run-android/run-android.impl.ts)
+ react-native -> npm:react-native/cli.js (packages/react-native/src/executors/run-ios/run-ios.impl.ts)
+ react-native -> npm:react-native/cli.js (packages/react-native/src/executors/start/start.impl.ts)
+ plugin -> nx (packages/plugin/src/utils/testing-utils/nx-project.ts)
+ nx -> angular (packages/nx/src/adapter/angular-json.ts)
+ nx -> npm:@nrwl/angular (packages/nx/src/adapter/angular-json.ts)
+ nx -> npm:enquirer (packages/nx/src/utils/params.ts)
+ eslint-rules -> npm:@typescript-eslint/parser (tools/eslint-rules/rules/valid-schema-description.spec.ts)
+ nx -> npm:typescript (packages/nx/src/plugins/js/project-graph/build-dependencies/build-explicit-typescript-and-package-json-dependencies.ts)
+ @nx/nx-source -> npm:identity-obj-proxy (scripts/patched-jest-resolver.js)
+ react -> cypress (packages/react/src/generators/storybook-configuration/configuration.ts)
+ jest -> npm:identity-obj-proxy (packages/jest/plugins/resolver.ts)
+ expo -> webpack (packages/expo/plugins/with-nx-webpack.ts)
+ expo -> npm:eas-cli/bin/run (packages/expo/src/executors/build-list/build-list.impl.ts)
+ expo -> npm:eas-cli/bin/run (packages/expo/src/executors/build/build.impl.ts)
+ expo -> npm:@expo/cli/build/bin/cli (packages/expo/src/executors/export/export.impl.ts)
+ expo -> npm:@expo/cli/build/bin/cli (packages/expo/src/executors/install/install.impl.ts)
+ expo -> npm:@expo/cli/build/bin/cli (packages/expo/src/executors/prebuild/prebuild.impl.ts)
+ expo -> npm:@expo/cli/build/bin/cli (packages/expo/src/executors/run/run.impl.ts)
+ expo -> npm:@expo/cli/build/bin/cli (packages/expo/src/executors/start/start.impl.ts)
+ expo -> npm:eas-cli/bin/run (packages/expo/src/executors/submit/submit.impl.ts)
+ expo -> npm:eas-cli/bin/run (packages/expo/src/executors/update/update.impl.ts)
+ react -> js (packages/react/babel.ts)
+ react -> npm:@babel/preset-react (packages/react/babel.ts)
+ react -> npm:style-loader (packages/react/plugins/component-testing/webpack-fallback.ts)
+ react -> npm:swc-loader (packages/react/plugins/component-testing/webpack-fallback.ts)
+ react -> npm:babel-loader (packages/react/plugins/component-testing/webpack-fallback.ts)
+ react -> npm:css-loader (packages/react/plugins/component-testing/webpack-fallback.ts)
+ react -> npm:sass-loader (packages/react/plugins/component-testing/webpack-fallback.ts)
+ react -> npm:less-loader (packages/react/plugins/component-testing/webpack-fallback.ts)
+ react -> npm:stylus-loader (packages/react/plugins/component-testing/webpack-fallback.ts)
+ react -> npm:react-refresh (packages/react/plugins/with-react.ts)
+ react -> npm:@svgr/webpack (packages/react/plugins/with-react.ts)
+ react -> npm:file-loader (packages/react/plugins/with-react.ts)
+ devkit -> npm:@schematics/angular (packages/devkit/src/utils/invoke-nx-generator.spec.ts)
+ next -> npm:@babel/plugin-proposal-decorators (packages/next/babel.ts)
+ next -> npm:@svgr/webpack (packages/next/plugins/with-nx.ts)
+ next -> npm:url-loader (packages/next/plugins/with-nx.ts)
+ next -> npm:next (packages/next/src/executors/server/server.impl.ts)
- workspace -> npm:@nx (packages/workspace/src/generators/preset/preset.ts)
+ rollup -> npm:babel-plugin-transform-async-to-promises (packages/rollup/src/executors/rollup/rollup.impl.ts)
+ angular -> npm:@storybook/angular/package.json (packages/angular/src/generators/storybook-configuration/lib/assert-compatible-storybook-version.ts)
+ angular -> npm:typescript (packages/angular/src/generators/library/lib/add-load-children.ts)
+ angular -> npm:tailwindcss (packages/angular/src/executors/utilities/tailwindcss.ts)

Boiled down to just workspace projects:

+ nx -> angular
+ expo -> webpack

Most of these differences are from require.resolve being handled now.

The only problematic one is: + nx -> angular (packages/nx/src/adapter/angular-json.ts) which is accurate but now introduces a circular dependency. This one is now ignored.

nx-examples

- Time for 'build typescript dependencies' 378.22046199440956
+ Time for 'build typescript dependencies' 4.6858730018138885

Absolutely no difference in dependencies

ocean (Internal Repo)

- Time for 'build typescript dependencies' 447.4808270037174
+ Time for 'build typescript dependencies' 126.27237199246883

Only difference in dependencies is due to a require.resolve

Related Issue(s)

Fixes #

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
nx-dev	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jul 18, 2023 8:44pm

[nx-cloud]

☁️ Nx Cloud Report

CI is running/has finished running commands for commit 04e3b48. As they complete they will appear below. Click to see the status, the terminal output, and the build insights.

📂 See all runs for this branch

---

✅ Successfully ran 1 target

• nx affected -t e2e-macos --parallel=1 --base=36cd6b0485325db0f6147320b04e3ffcd41373f4 --head=04e3b48207cab6c369608e68874ff79f2ab0bdf0

---

Sent with 💌 from NxCloud.

[github-actions]

This pull request has already been merged/closed. If you experience issues related to these changes, please open a new issue referencing this pull request.