Tools comparisson (dts-bundle-generator, api-extractor, rollup-plugin-dts, npm-dts)

[octogonz]

@timocov FYI we've been working on a tool API Extractor that seems to have similar goals to dts-bundle-generator. It might be interesting to compare the features and goals of the two projects, and see if anything can be shared between them.

[timocov]

Hi @pgonzal,

thank you for your interest in the project!

After slight looking up I see the next differences:

1. dts-bundle-generator does not require configure anything - you can just run it via command line and it'll just work. But you're able to configure some stuff if you need it (see https://github.com/timocov/dts-bundle-generator#usage).
2. To run dts-bundle-generator you don't need to compile your project - it'll do it by itself while generating (and it does it in the memory, so no unnecessary output will be written to your drive).
3. It looks like api-extractor requires you export all interfaces/functions/const/etc from your entry point if you want to export them, and doesn't follow dependencies, right? If so, this is difference also, because dts-bundle-generator will follow dependencies and put them all into result file.
4. api-extractor requires add tag to an export member (@public, @internal, @beta).
5. Installing api-extractor adds @types/node to the project and it may affect the build (in my case it did it). I'm not sure that it's a good idea to put it to dependencies instead of devDependencies.
6. api-extractor requires TypeScript version 3.1.x, but dts-bundle-generator works with any versions of TypeScript starts with 2.6.1.
7. It looks like configuring api-extractor might be painful. I still have an error (at least it looks red) The "mainDtsRollupPath" setting was not specified. In this case, the package.json "typings" field must point to a file under the "publishFolder": ./dist, but I've specified typings field in the package.json file and don't know what it actually wants from me.
8. With dts-bundle-generator you can generate dts bundle for several entry points (example), it looks like api-extractor supports only one entry point per project.

Correct me if I were wrong somewhere about api-extractor. Also, it would be nice to see what do you think about it and what differences (or pros and cons) you see between projects.

[octogonz]

Thanks for writing up this comparison! It's very interesting.

2. To run dts-bundle-generator you don't need to compile your project - it'll do it by itself while generating (and it does it in the memory, so no unnecessary output will be written to your drive).

Right, API Extractor processes .d.ts files instead of .ts files. This supports the usage of preprocessors (e.g. a .js -> .d.ts generator), and also simplifies the processing that we do, since .d.ts files are in a simplified normalized form.

3. It looks like api-extractor requires you export all interfaces/functions/const/etc from your entry point if you want to export them, and doesn't follow dependencies, right? If so, this is difference also, because dts-bundle-generator will follow dependencies and put them all into result file.

What do you mean by "follow dependencies"? For locally referenced files import { X } from './X'; it does follow and collapse these. For external NPM packages import { X } from 'package'; it emits the import statement in the rollup .d.ts. Inlining such a declaration seems like it would be incorrect behavior.

4. api-extractor requires add tag to an export member (@public, @internal, @beta).

This requirement can be disabled in the config file.

5. Installing api-extractor adds @types/node to the project and it may affect the build (in my case it did it). I'm not sure that it's a good idea to put it to dependencies instead of devDependencies.

Wow, thanks for pointing this out! We overlooked it for a long time. :-)

6. api-extractor requires TypeScript version 3.1.x, but dts-bundle-generator works with any versions of TypeScript starts with 2.6.1.

API Extractor installs with a specific (very recent) TypeScript package that it uses for its own analysis, but this is typically different from the version that your project builds with. For example when processing a 2.6.1 project, it can use the 2.6.1 runtime library typings while still invoking the 3.1 engine.

7. It looks like configuring api-extractor might be painful. I still have an error (at least it looks red) The "mainDtsRollupPath" setting was not specified. In this case, the package.json "typings" field must point to a file under the "publishFolder": ./dist, but I've specified typings field in the package.json file and don't know what it actually wants from me.

Yeah the configuration is a weak area, since in our own usage at Microsoft, all the projects are in a huge monorepo with a standardized configuration, so nobody customizes the configuration directly. The external community has complained about this, and it's a high priority to improve the configuration and make it more flexible.

8. With dts-bundle-generator you can generate dts bundle for several entry points (example), it looks like api-extractor supports only one entry point per project.

That's true. The analyzer engine partially models path-based entry points, but they are very difficult to support in general. (How should a documentation web site represent them? When generating a .d.ts rollup, where do you emit common local declarations that are shared by two different entry points?)

[timocov]

Right, API Extractor processes .d.ts files instead of .ts files.

AFAIK some dts-bundle-generator's users use it with their d.ts files and it works fine. Actually there is no restrictions to use it only with .ts files, but it is just simpler in a common case.

For locally referenced files import { X } from './X'; it does follow and collapse these.

Oh, great! It is quite possible that I just didn't get it work in a proper way 🙁

This requirement can be disabled in the config file.

👌 Will know, thanks!

For example when processing a 2.6.1 project, it can use the 2.6.1 runtime library typings while still invoking the 3.1 engine.

In my case, I've tried to run in a project with 3.2.1 TypeScript, and it tried to use lib files from 3.1.6, but they're incompatible and I get a lot or errors about it. As temporarily fix I've enabled skipLibCheck compiler option, but this isn't what I want to have in the compiler options 🙂

The external community has complained about this

Another, but the similar opinion about configuration - https://github.com/Swatinem/rollup-plugin-dts#why.

How should a documentation web site represent them?

What do you mean "documentation web site"? It is about the documentation site, which your tool generates? If so, I don't know the answer 🙁 In my case I just pointing to separate (not main) file in docs (see JS file example).

When generating a .d.ts rollup, where do you emit common local declarations that are shared by two different entry points?

I haven't think about it yet deeply, but it is possible, that it is not a big problem in a common case. In a project where I'm working on we generate 3 different d.ts bundles (and there is a lot of code in a common between them), and it seems that everything is ok for now with that.

[timocov]

where do you emit common local declarations that are shared by two different entry points?

It sounds like a good new issue 🙂 #69

[timocov]

@pgonzal also we can compare generated output. I think a test cases would be fine for that (for example run dts-bundle-generator against api-extractor's tests and vice versa). It is possible that we'll find something interesting here 🙂

[Swatinem]

Hi, and let me also chime in as the maintainer of rollup-plugin-dts

As I have also mentioned in my README, I have tried api-extractor as well, but my biggest hurdle was also the configuration as @timocov also mentioned. In the end I couldn’t get it to work at all and just gave up.

So I have nothing to actually compare the output quality and differences, like @timocov asked me to here: Swatinem/rollup-plugin-dts#5

But apart from the configuration issue that was already mentioned, my biggest beef is that it requires me to create intermediate output in the form of pre-rollup .d.ts files (point 2. of @timocov’s list), which goes against how I would like my library workflows to work. (use tsc for typechecking only, and use a bundler like rollup / webpack to actually generate the output; I don’t want to have to manage/clean any intermediate build artifacts)

Also concerning 4., I recently found out that tsc has a compilerOptions for that which I plan to rely on at some point (Swatinem/rollup-plugin-dts#2)

[octogonz]

compilerOptions.stripInternal is very limited. Last time I tried it, if you did this...

/**
 * For more info, please email [email protected]
 */
export class Base {
}
export class Child extends Base {
}

...it would strip Base because it found the @internal substring in there, resulting in a Child definition that does not compile. Whereas API Extractor relies on the TSDoc grammar to understand that @internal is not a tag in this context. Also, if something is marked as @internal that is referenced by an @public or @beta declaration, API Extractor will detect this and issue a warning about inconsistent visibility.

As I have also mentioned in my README, I have tried api-extractor as well, but my biggest hurdle was also the configuration as @timocov also mentioned.

This is useful feedback. Thanks!

But apart from the configuration issue that was already mentioned, my biggest beef is that it requires me to create intermediate output in the form of pre-rollup .d.ts files

It would be possible to pass these as in-memory data structures rather than writing them to disk. Someone else proposed this in microsoft/rushstack#1010. (It's not a very high priority for me personally, since from an opposite perspective I find it frustrating to work with a "black box" where you can't easily inspect the intermediary stages of the build. Also, I haven't seen significant perf benefits from skipping disk writes; if anything it would make incremental builds less feasible. But this feature is useful, so we should implement it.)

[Swatinem]

...it would strip Base because it found the @internal substring in there

puh, that is definitely something that has to be fixed upstream then.

from an opposite perspective I find it frustrating to work with a "black box" where you can't easily inspect the intermediary stages of the build. Also, I haven't seen significant perf benefits from skipping disk writes; if anything it would make incremental builds less feasible. But this feature is useful, so we should implement it.

Those are definitely valid points. Luckily, when you create small focused libraries, perf is not a problem.

Regarding configuration, I personally would like things to be as zero-config as possible.
rollup-plugin-dts comes extremely close to that, since this is basically the only thing you need:
https://github.com/Swatinem/rollup-plugin-dts/blob/c1ee0801753e78bb76b2eae0903836b993a983a7/rollup.config.js#L32-L39

Also related to that, the intermediate artifacts themselves are not the problem. Its rather that you have to manage those yourself.

To my own shame, I also didn’t yet understand how to correctly work with TS project references. At work we have a codebase of ~2700 ts files (a lot of those come from external dependencies) and quickly growing. Typechecking times (and IDE performance) are already a pain that I would like to solve somehow.

[aleclarson]

I just found dts-generator by SitePen.

cc @dylans Could you explain how your package differs from the others mentioned in this thread? 😄

[Swatinem]

BTW, I also did a really quick research around this topic and I found https://github.com/TypeStrong/dts-bundle, but the latest commit was 2 years ago so this seems pretty dead to me.
I also saw a webpack plugin for types, but can’t remember what it was.

[timocov]

There is also https://github.com/nomaed/dts-builder.

[vytenisu]

Hello there!

Let me also hop in with npm-dts module I have put together to make it easy with some use cases.

Basically I was a user of dts-generator and encountered a problem at some point (SitePen/dts-generator#121). After trying to solve it for a while I decided to write a simple tool myself which would have less mandatory command-line arguments and would work for my use case.

I tried to make this tool as simple as possible to make it easy to maintain. Added logging with configurable log levels to make it easier to understand what is happening. And well, tried to keep it's source, CLI and docs as tidy as it was viable.

My use case was to pre-bundle several npm libraries (excluding sub-dependencies of course) and generate index.d.ts file for each of those library bundles so that I would still have full TS support when consuming them even though initial src was not uploaded during publishing.

Quick summary based on discussion points above:

• Configuration - kept to absolutely minimum/simple set of options. For my case instead of "dts-generator --name ter --project ./ --out index.d.ts --main ter/index" i could simply write "npm-dts generate". It generates index.d.ts (very similar to dts-generator) which is automatically picked up by TS when importing.

• Project compilation during generation - project is compiled to system tmp directory (selected using tmp package based on OS) using TSC. Temporary files are cleaned once done. Basically tool generates typings using TSC itself and then it concatenates + adds some additional stuff resulting in index.d.ts file.

• Following dependencies - npm-dts does not follow/inline dependencies (on purpose) - it simply uses what is generated by TSC. Not sure why I would need to follow dependencies (at least for my use case). Every library exports it's own index.d.ts so if type is found in sub-dependency, that library has it's own index.d.ts and everything works just fine. Library should not bundle-in sub-dependencies because consumers of such library risk of having duplicated code and large bundles as a result. Therefor bundling typings together also does not seem to be necessary.

• Adding custom tags in code (like @public, etc.) - not needed and not supported. Code is parsed using TSC. Whatever TypeScript understands, npm-dts should also.

• Creation of additional files/folders in target project - npm-dts only generates index.d.ts. All temporary files are stored in tmp of the system and is deleted at once after operation is complete. In case it would miserably fail for whatever reason, files would be deleted next time.

• Should it be dep or dev-dep - it should be dev-dependency in most cases unless you would be extending the script with some additional features of your own by wrapping it.

• What TypeScript version is used? - npm-dts uses same TS version as target project. I have not tested which versions of TS are working tbh but as long as this command line is supported "tsc --declaration --emitDeclarationOnly --declarationDir .........", it should be ok.

• Do I need to specify which files to bundle? - no. npm-dts extracts typings from all ts files and concatenates them into a single index.d.ts.

• Number of entry points - npm-dts is for single-entry-point libraries. Entry point should be index.ts by default (sort-of convention) but can be overridden using command line options.

It also has a webpack plugin and can be executed from another JS script as a dependency. If it works for you, feel free to use it. If not - use other generators :) Aaaand feel free to leave feedback, especially if something does not work as expected.

[gavar]

Hi, and let me also chime in as the maintainer of rollup-plugin-dts

As I have also mentioned in my README, I have tried api-extractor as well, but my biggest hurdle was also the configuration as @timocov also mentioned. In the end I couldn’t get it to work at all and just gave up.

So I have nothing to actually compare the output quality and differences, like @timocov asked me to here: Swatinem/rollup-plugin-dts#5

But apart from the configuration issue that was already mentioned, my biggest beef is that it requires me to create intermediate output in the form of pre-rollup .d.ts files (point 2. of @timocov’s list), which goes against how I would like my library workflows to work. (use tsc for typechecking only, and use a bundler like rollup / webpack to actually generate the output; I don’t want to have to manage/clean any intermediate build artifacts)

Also concerning 4., I recently found out that tsc has a compilerOptions for that which I plan to rely on at some point (Swatinem/rollup-plugin-dts#2)

rollup-plugin-dts is the best option I found, IMHO.
The only missing part I spent a lot of time on was how to properly use it with rollup-plugin-typescript2
I would suggest minimal config below - rollup execute arrays of configs sequentially (which I haven't found documented well), so it first builds code to common directory and then creates a .d.ts bundle from it.

import dts from "rollup-plugin-dts";
import rts2 from "rollup-plugin-typescript2";

export default [
  { // TypeScript
    input: "src/index.ts",
    output: [
      {file: "lib/index.js", format: "cjs"},
      {file: "lib/index.mjs", format: "esm"},
    ],
    plugins: [
      rts2({
        cacheRoot: "out/.rts2_cache",
        useTsconfigDeclarationDir: true,
        tsconfigOverride: {
          compilerOptions: {
            declarationDir: "out/.dts",
          },
        },
      }),
    ],
  },
  { // TypeScript Type Definitions
    input: "out/.dts/index.d.ts",
    plugins: [dts()],
    output: {file: "lib/index.d.ts", format: "esm"},
  },
];

PS:
I've made a package out of it
GitHub: https://github.com/gavar/wrench/tree/beta/packages/roll
NPM: https://www.npmjs.com/package/@wrench/roll

No docs yet, only comments in .d.ts, but it basically relies on the package.json structure to bundle a NodeJS library: entry point, .d.ts bundle and executable bin scripts.

Of course it use itself to build itself.

[timocov]

rollup-plugin-dts is the best option I found, IMHO.

@gavar can you please tell what features/abilities you didn't find in other tools (like api-extractor or dts-bundle-generator), but they are in rollup-plugin-dts? I believe it might be helpful for other people who'll see this thread.

[Swatinem]

I also recently found a lot more issues with my tool, which are due to the way typescript works:

• in TS, you can override and “monkeypath” things in the global scope, which is not supported, since my tools only works with things you explicitly import/export.
• in TS, you have declaration merging, the most obvious one is when you define overrides for one function, but you can also merge functions with namespace declarations, which is the typescript way to attach static properties to functions. When you have multiple declarations with the same name, things can blow up in surprising ways -_-

I still hope that this usecase will be implemented in typescript properly, which would make all of these tools obsolete.

[adventure-yunfei]

User for api-extractor here.
I have to say, if consider dts rollup usage only, api-extractor may not support much edge cases as other tools listed here, with the saming effort. It is limited by its design, re-creating nearly an entire intermediate ast tree. api-extractor has to handle each syntax case that it wants to support, while other dts rollup tools may only handle cases that they don't want to support. And it's true that I met so many unsupported cases that I had to fork the repo to fix.
But api-extractor is still a very nice tool. The huge advantage which seems not to be mentioned here, is that api-extractor supports different kind of outputs, and dts rollup is only one of them. The analyzer in api-extractor extract intermediate ast tree from the source to hide details, that's why multiple generators are allowed. It is supported by its design.
For now, api-extractor tool system supports (see api-extractor site for details):

• dts rollup
• api document generation
• api review

My recomendation:

• if you only need dts rollup feature, and wish it support as many cases as possible, pick a dts rollup tool (with fast response for issues, or well-designed codes. Trust me, you will still get some edge cases that they don't support.).
• if you're also suprised with other features around d.ts, pick api-extracor

[shigma]

I would really like to use dts-bundle-generator but I used a monorepo, which means the bundler I choose should support composite project. It seems that dts-bunder-generator doesn't. I wonder which library above does.

[loreanvictor]

Good thread. However, I am searching for the proper tool for a situation where I need to bundle all types from external dependencies as well, with proper name-conflict resolution. Since these are areas where all the mentioned tools seem to be outright avoiding or not guaranteeing proper functionality, can I ask for any recommendations / suggestions?

[timocov]

where I need to bundle all types from external dependencies as well

@loreanvictor when and where it would be needed? Can you please elaborate why do you need this (with examples if possible)?

[loreanvictor]

@timocov I need it for this project. Basically, I'm wrapping some TypeScript functions automatically into API endpoints, and then transferring the type definitions of these functions to the client-side for auto-generating an SDK for said API. The type definitions will sit alongside some auto-generated .js code that basically calls the API endpoint corresponding to each of the aforementioned functions (API functions), making the functions look exactly as they were defined on the API code (while the implementation is different).

This works pretty smoothly except for situations where the API functions rely on types from dependencies, as in this situation the client would also need to have the dependencies installed (or at least, the corresponding types installed). Current work-around is avoiding such types in the API functions, which is not ideal as it bars re-using recurring types. The alternative would be to also include all types that the API functions are dependent on in the bundle automatically.

P.s. on revisiting my comment I see that it can be read as "collecting all types exported from all dependencies in a blanket manner and add them to the bundle", but thats not what I meant. I basically need to create type-bundles with zero external dependencies.

[Swatinem]

So I think rollup-plugin-dts should be up for the task. Although I absolutely do not recommend it, I know that a few people use it to bundle up all external dependencies as well, and it will happily take care of name conflicts and everything.

[adventure-yunfei]

Good thread. However, I am searching for the proper tool for a situation where I need to bundle all types from external dependencies as well, with proper name-conflict resolution. Since these are areas where all the mentioned tools seem to be outright avoiding or not guaranteeing proper functionality, can I ask for any recommendations / suggestions?

api-extractor supports that with bundledPackages option.

[loreanvictor]

Although I absolutely do not recommend it

@Swatinem may I ask why?

[loreanvictor]

Good thread. However, I am searching for the proper tool for a situation where I need to bundle all types from external dependencies as well, with proper name-conflict resolution. Since these are areas where all the mentioned tools seem to be outright avoiding or not guaranteeing proper functionality, can I ask for any recommendations / suggestions?

api-extractor supports that with bundledPackages option.

@adventure-yunfei yeah it seems that similar to dts-bundle-generator, you can only specify a few select external libraries and it will bundle types exported from them (i.e. I could not find the "all" option).

[Swatinem]

@Swatinem may I ask why?

Its just that some @types packages are manually written and have some really strange idioms in them. What I don’t support, and quite a lot of @types use for example is export = … syntax. Also bundling foreign code is basically against the whole idea of having packages and sharing/deduping dependencies.

[loreanvictor]

@Swatinem may I ask why?

Its just that some @types packages are manually written and have some really strange idioms in them. What I don’t support, and quite a lot of @types use for example is export = … syntax. Also bundling foreign code is basically against the whole idea of having packages and sharing/deduping dependencies.

makes sense. tbh, if there was a way to just install types of a package, the cleanest solution for my case would also be simply installing all types as explicit dependencies of the client SDK instead of bundling them all together.

question though: since this pitfall affects mostly @types packages, and since @types packages can actually be safely installed in the client environment as explicit dependencies, is there a way to exclude paths / libraries / etc from rollup-plugin-dts's "respectExternal"?

[Swatinem]

when using respectExternal, you can use the normal external setting from rollup.

[surol]

Hello!

Let me introduce a new member of the family: rollup-plugin-flat-dts. Probably the dumbest one :), but good reliable worker.

Instead of trying to be "smart" while merging .d.ts files, it simply flattens them in this way:

1. Requests TypeScript compiler to generate a single .d.ts file (with "module": "None" and other options).
2. Replaces module names of declare module "path/to/file" {...} statements with package name.
3. Adjusts imports, exports, and nested module declarations accordingly.

This algorithm applies severe limitations to the code base:

1. Every exported symbol from every file is treated as exported from the package.
   To disable this the exported symbol has to be marked internal. Either with @internal jsdoc comment, or with internal plugin option.
2. No way to rename the exported symbols.
3. A default export is only allowed at the top level of the package or entry point.

Surprisingly, I had to adjust only one project's structure out of a few dozens I maintain. The rest of the projects already suit the requirements. So, these limitations did not cause any troubles.

Multiple Entry Points

One important feature causing me to develop this tool is a multiple entry points support.

Some tools like API extractor do not support it intentionally. rollup-plugin-ts declares support of multiple Rollup entry points, but:

1. Rollup entry point is not the same as package one.
2. rollup-plugin-ts generates additional chunks and sometimes fails to reexport from them correctly.

In contrast, rollup-plugin-flat-dts allows to specify which type definitions belong to which module. And then either merge them to one file, or put different module type definitions to different files. No extra files generated.

Thus, it is possible to establish a one-to-one connection between package entry point and its type definition. This allows IDEs to recognize the imported modules and enable correct autocompletion for them. Also, this prevents incorrect imports. The latter often happens when generating a type definition per source file with tsc.

Conclusion

In exchange for some limitations, the plugin emits predictable output and grants control over contents of generated type definitions files. I hope this makes it a good tool for library authors like myself.

[bhovhannes]

@Swatinem thank you very much for rollup-plugin-dts!
It worked perfectly for bundling types generated from JSDoc comments in JS code.

First I was trying api-extractor, but it does not support imports in JSDoc comments like this:

/**
 * Returns an Observable
 * @returns {import('rxjs').Observable}
 */
export function getItems() {}

Another downside of api-extractor for me was the requirement to put @internal on methods I don't export from entrypoint, otherwise their declarations kept appearing in dts bundle.
Again, rollup-plugin-dts correctly understands which methods are exported by package entrypoint and omits declarations of other methods from dts bundle, which was exactly what I wanted.

[Nis5l]

How would I go about bundling @types packages, for example bundling all imports and references from @types/lodash into one .d.ts file.
I pretty much tried all the mentioned tools, but with no success...

[Nis5l]

Thanks a lot for the response.
To get types, autocompletion etc. for imports in the monaco editor you can add type definitions using addExtraLib, passing the content as a string.
My plan was, if possible, to bundle the whole @type package into one file, starting from the index.d.ts and adding all the additional files that are imported or referenced recursively.
Then I could simply add the content of the file using addExtraLib.

[timocov]

I'm sorry but I still don't get the idea. Is it possible to create an example of what you would like to get eventually?

[hassnian]

@Nis5l Did you solve the issue? , I'm trying to do the same

[Nis5l]

No I haven't solved it,
ultimately I would like to combine all type files of a package into one so i will be able to import it.
I can import moment because it already only has one type definition file.
I can't import lodash because it has multiple files which in this example are referenced, but could also be imported.

[timocov]

@Nis5l as for your example you can try to re-export everything from the library and mark a library you'd like to bundle as a "inlinded" one:

import * as lodash from 'lodash';
export { lodash };

 ./node_modules/.bin/dts-bundle-generator index.ts --external-inlines=lodash

In case of lodash it fails with a few errors right now:

index.d.ts(1,32): error TS2315: Type 'Partial' is not generic.
index.d.ts(5782,30): error TS2314: Generic type 'Function<T>' requires 1 type argument(s).

but I think it might be fixed by fixing the types itself because they use the same names as from the default lib.

I hope this might help.