Support for project references

[Gerrit0]

Version 0.20.0-beta.26 introduces an initial attempt at support for "solution style" tsconfig files, aka project references.

How this works

When TypeDoc is given a tsconfig file containing project references, it will create a ts.Program for each of those references (in addition to the root program). To find each provided entry point, it will look at each program until it finds one that contains that file.

For the most part, this should just work, but if you have a tsconfig which only contains references, you might see unexpected behavior where types are not linked, even if another entry point exports them.

As an example, in the project-references-demo project, npx typedoc zoo/zoo.ts animals/index.ts will result in documentation where the createZoo function returns Array<Dog>, but Dog is not linked, even though animals/index.ts exports it. The cause of this is that the root tsconfig.json looks like this:

{
  "files": [],
  "references": [
    {
      "path": "./core"
    },
    {
      "path": "./animals"
    },
    {
      "path": "./zoo"
    }
  ]
}

When TypeDoc looks for the entry points, it finds animals/index.ts in the second listed program before looking at ./zoo, while the Dog type used in createZoo is from the third program. TypeDoc relies on TypeScript to link types, so is unable to resolve types across programs, resulting in a broken link.

There are a couple possible fixes for this:

1. Specify the "largest" projects first - if the tsconfig specified ./zoo first, then TypeDoc would find ./animals/index.ts in the zoo project, and links would work.
2. Specify the tsconfig only for the root project expected to contain all symbols npx typedoc --tsconfig zoo/tsconfig.json zoo/zoo.ts animals/index.ts works as expected. This is usually the better solution since it allows TypeDoc to skip creating any unnecessary projects, making the type checking potentially significantly faster.

---

Once 0.20 releases, I will put all of this info on the website, but for now... an issue works. Feel free to ask questions below!

Cross linking for visibility: #923, #1403, #1364

[davidfirst]

@Gerrit0 , why not using the createSolutionBuilderHost API which supports project-references, and then no need to create ts.program for each one of the projects? Maybe that's your suggestion # 2?

[Gerrit0]

I assume you mean createSolutionBuilder since the host doesn't appear to give me anything useful... besides the same createProgram function I pass it (which defaults to something that's nearly identical to what typedoc does)

createSolutionBuilder is the right API if you are writing your own version of tsc - but typedoc needs a ts.Program for conversion. The getNextInvalidatedProject function returns an object with a getProgram method - but it seems like an abuse of that API to call getNextInvalidatedProject until it returns undefined and then do work.

Having stepped away from it for a day, I think typedoc can avoid creating a program for each reference unless the tsconfig appears to be a solution style tsconfig, which should help some.

    interface SolutionBuilder<T extends BuilderProgram> {
        build(project?: string, cancellationToken?: CancellationToken): ExitStatus;
        clean(project?: string): ExitStatus;
        buildReferences(project: string, cancellationToken?: CancellationToken): ExitStatus;
        cleanReferences(project?: string): ExitStatus;
        getNextInvalidatedProject(cancellationToken?: CancellationToken): InvalidatedProject<T> | undefined;
    }

[davidfirst]

Yes, I meant createSolutionBuilder, and correct, you can get the Program by doing something like this:

const solutionBuilder = ts.createSolutionBuilder(host, [rootDir]);
while (nextProject = solutionBuilder.getNextInvalidatedProject()) {
    const program = nextProject.getProgram();
    // do something with the program
    nextProject.done();
}

We do something similar at Bit for compiling multiple packages. For us, it improved the performance dramatically especially when there are multiple dependencies between the packages.

Not sure if it helps for your specific use case though.

[Gerrit0]

I did a bit of experimenting - and I don't think it offers much benefit for TypeDoc unfortunately. The performance increases you saw were likely because TypeScript was able to mark dependencies as complete and avoid compiling them... but for TypeDoc, even if a project has been compiled, we still need to get its program, and the solution builder won't give it to us without { force: true }

[Gerrit0]

jupyterlab recently upgraded typedoc to use 0.20, which has revealed some problems with the implementation here:

1. Exports from project references are documented as originating in their declaration files. This makes sense - since they do... but is problematic since it breaks links to the source file on GitHub.
2. A "solution style" tsconfig with 86 projects is horrible. jupyterlab doesn't use one, thankfully, but I tried it as an experiment, (put "files": [] in the tsconfigdoc.json file) and TypeDoc chewed up 12gb of ram getting a program for each of the projects... and then failed to compile some of them.

I'm not sure of the fixes for either of these yet, but wanted to document them while still fresh in my mind.

[Gerrit0]

There are now docs for this on the site - https://typedoc.org/guides/project-references/