Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

module: add --experimental-strip-types #2

Closed
wants to merge 1 commit into from

Conversation

marco-ippolito
Copy link
Owner

@marco-ippolito marco-ippolito commented Jul 1, 2024

It is possible to execute TypeScript files by by setting the experimental flag --experimental-strip-types.
Node.js will transpile TypeScript source code into JavaScript source code as esnext ES Modules.
During the transpilation process, no type checking is performed, and types are discarded.

Motivation

I believe enabling users to execute TypeScript files is crucial to move the ecosystem forward, it has been requested on all the surveys, and it simply cannot be ignored. We must acknowledge users want to run node foo.ts without installing external dependencies or loaders.

Why type stripping

Type stripping as the name suggest, means removing all the types, transform the input in a esm JavaScript module.

const foo: string = "foo";

Becomes:

const foo = "foo";

Note that NO module type transformation will be performed (.ts => .cjs 🚫 🙅🏼 ).

Other runtimes also perform transformation of some TypeScript only features into JavaScript, for example enums, which do not exists in JavaScript.
At least initially in this PR no trasformation is performed, meaning that using Enum will not be possible.
I believe as a project we don't want to vendor TypeScript directly and I would like to push the ecosystem to STANDARDIZE these features (example: https://github.com/tc39/proposal-decorators) so that they become part of the language.

Why I chose @swc/wasm-typescript:

Because of simplicity.
I have considered other tools like swc/core and esbuild but they require either rust or go to be added to the toolchain.
@swc/wasm-typescript its a small package with a wasm and a js file to bind it.
swc is currently used by Deno for the same purpose, it's battle tested.
In the future I see this being implemented in native layer.
Currently why just npm pack @swc/wasm-typescript, perform some transformation and add it to our sources.
This is not the standard way we operate, we want to have the sources under versioning to be able to hotfix and monitor possible security issues.
This could be solved by vendoring swc and build the wasm ourselves.
I will do that in a immediate future iteration.

Why not TSC (typescript.js)

typescript.js is a (at the time of writing this) a ~8.5MB and it does not perform type-stripping, but only transformation.

Limitations

TypeScript only features

By design Node.js will not perform transformations on TypeScript only features.
This means Enum, experimentalDecorators, namespaces and other TS only features are not supported.
You can set isolatedModules in your tsconfig.json to ensure that you are not using TypeScript only features.

Importing modules without .ts extension

Currently importing modules without .ts extension is not supported.

  import { foo } from 'foo'; // it will not work
  import { foo } from './foo.ts'; // it will work

Only files with the extension .ts are supported, meaning it will not support .mts, .cts, or .tsx syntax.

REPL

Executing source code in the REPL with TypeScript is not supported.
This also applies to --eval, --print, --check and inspect.

Source maps

Currently source maps are not supported, this could be trivial to add.

No benchmarks

This PR does not have performance in mind, as its a first iteration so no benchmarks and no bundle size measurements are performed

Next Steps

Short term

  • Add @swc/wasm-typescript sources and build the wasm ourselves
  • Support extensionless imports
  • Improve UX, performance, etc...

Long term

  • Move to @swc/core as a native implementation.

@marco-ippolito marco-ippolito force-pushed the feat/typescript-support branch 4 times, most recently from 9333ed4 to d5a23b0 Compare July 1, 2024 19:04
@marco-ippolito marco-ippolito changed the title Feat/typescript support loader: create typescript loader Jul 1, 2024
@marco-ippolito marco-ippolito force-pushed the feat/typescript-support branch 6 times, most recently from b1ac849 to fa32fc9 Compare July 1, 2024 19:35
@aduh95

This comment was marked as resolved.

@marco-ippolito marco-ippolito force-pushed the feat/typescript-support branch 3 times, most recently from cf1d7f5 to f507952 Compare July 1, 2024 21:08
@marco-ippolito marco-ippolito changed the title loader: create typescript loader module: add --experimental-typescript Jul 1, 2024
@AugustinMauroy

This comment has been minimized.

@marco-ippolito

This comment was marked as resolved.

@AugustinMauroy
Copy link

It's a bit more complex to set up, but couldn't we just get the .wasm file from the package and do the js implementation in our code base?

@AugustinMauroy
Copy link

Slow (?)

if it's slow it's should not from SWC
REF: https://swc.rs/docs/benchmarks

@marco-ippolito

This comment was marked as outdated.

@marco-ippolito
Copy link
Owner Author

Slow (?)

if it's slow it's should not from SWC REF: https://swc.rs/docs/benchmarks

I mean this implementation is not refined for performance

kdy1 added a commit to swc-project/swc that referenced this pull request Jul 3, 2024
**Description:**

This PR adds a Wasm binding which is only capable of stripping TypeScript types.


**Related issue:**

 - marco-ippolito/node#2
@marco-ippolito
Copy link
Owner Author

We use wasm-pack, and wasm is quite undocumented because we have a fallback script that automatically fallbacks to Wasm from @swc/core and it was the main usage.

But it seems like the CJS loader is not a good fit for node.js. What's the ideal form of the Wasm package?

Since we are going to bundle it the binary, we cant just do readFile or load it dinamically, the wasm must be in the form of a buffer stored in a variable. In the PR I converted it to

const WASM_BODY = Buffer.from(<the wasm>)

@marco-ippolito
Copy link
Owner Author

For --eval and stdin, it should be rather trivial to add support; I would expect --check to be also rather trivial. It can certainly happen in a follow up though.

Regarding --print is out of scope as long as we don't support it for ESM and we only support transpiling TS to ESM.

I'm surprised you say it doesn't work with node inspect, I guess it's also a limitation with the module hook API?

Simply untested, so I preferred to do that in a follow up pr

@kdy1
Copy link

kdy1 commented Jul 3, 2024

I added some tests, published @swc/[email protected] , and updated the documentation

As requested at nodejs/loaders#208 (comment), it only strips type and does not touch enum or something like that.

If required, I can modify it to make the behavior configurable via option.

@marco-ippolito
Copy link
Owner Author

I added some tests, published @swc/[email protected] , and updated the documentation

As requested at nodejs/loaders#208 (comment), it only strips type and does not touch enum or something like that.

If required, I can modify it to make the behavior configurable via option.

Just to clarify, it will throw when enums are encountered?

@marco-ippolito marco-ippolito changed the title module: add --experimental-typescript module: add --experimental-strip-type Jul 3, 2024
@marco-ippolito marco-ippolito changed the title module: add --experimental-strip-type module: add --experimental-strip-types Jul 3, 2024
@marco-ippolito
Copy link
Owner Author

marco-ippolito commented Jul 3, 2024

I added some tests, published @swc/[email protected] , and updated the documentation

As requested at nodejs/loaders#208 (comment), it only strips type and does not touch enum or something like that.

If required, I can modify it to make the behavior configurable via option.

It seems the package is not working as I expected, like @swc/wasm, seems more for the browser @swc/wasm-typescript
@swc/wasm is working fine, except the reading wasm file

@marco-ippolito marco-ippolito changed the title module: add --experimental-strip-types module: add --experimental-typescript Jul 3, 2024
@marco-ippolito
Copy link
Owner Author

I will rename to --experimental-strip-types once we have swc working

@kdy1
Copy link

kdy1 commented Jul 4, 2024

I published a new version using nodejs target, and I verified that

import { transform } from "@swc/wasm-typescript";

const { code } = await transform(
    `const a = 1;
    type Foo = number;`, {});
console.log(code);

emits const a = 1;.

By the way, should it throw when it encounters an enum? It now preserves enums. Trying to execute the output will fail anyway, but I can add some early errors for it.

@marco-ippolito
Copy link
Owner Author

marco-ippolito commented Jul 4, 2024

I published a new version using nodejs target, and I verified that

import { transform } from "@swc/wasm-typescript";

const { code } = await transform(
    `const a = 1;
    type Foo = number;`, {});
console.log(code);

emits const a = 1;.

By the way, should it throw when it encounters an enum? It now preserves enums. Trying to execute the output will fail anyway, but I can add some early errors for it.

Thank you so much! Yes errors should be throw during transpilation, otherwise they will be syntax errors that are hard to wrap and to make them intellegible

const originalContent = fs.readFileSync(outputFile);

fs.writeFileSync(outputFile, `'use strict'\n
const { Buffer } = require('node:buffer');\n
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kdy1 I created this script to insert the wasm as byte array in the wasm.js file because we cannot perform a readFile or require

Copy link

@kdy1 kdy1 Jul 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thats amazing! Can we have the same linting error for namespaces?

@marco-ippolito marco-ippolito changed the title module: add --experimental-typescript module: add --experimental-strip-types Jul 4, 2024
@marco-ippolito marco-ippolito force-pushed the feat/typescript-support branch 8 times, most recently from 5ed84e5 to ba740b6 Compare July 4, 2024 15:02
@marco-ippolito
Copy link
Owner Author

Moved to the node repo
nodejs#53725

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants