Module conversion work

[elibarzilay]

This issue is to keep track of the module conversion work, which is ongoing at #46567:

• Update the old conversion script to working state.
  Done, keep track of the code in a new fork to working state.
• Script the generation so it can be continuously rebased on the tip: done in the run-transform script.
• Make the tests run, similar to 1c3944343ee8 in Wesley's PR: included in the current PR, and generated via the script.
• Fix treatment of newlines, mainly a bug that changes CRLF to LF if running on linux.
• Make the diff more readable
  • Pre-script that unindents the code in a separate commit, which avoids a huge mass of mostly no-op changes.
  • Drop the many empty-line removals from the conversion diff. Done by another script that filters the resulting patch before applying it.
  • Possibly re-implement the conversion using ts-morph to make it more well-behaved wrt unnecessary changes (adding/removing newlines, still some comments get dropped, minor issues like changing ,s to ;s or dropping a suffix , in argument lists).
• Bundle results
  • Decide on a bundler: most likely esbuild, since it fast enough to make multiple bundles (eg, for testing and for distributing) possible.
  • Need to decide how to use it:
    • Can run on the generated JS: will probably be sensitive to the generated format.
      Main problem: the modules are expressed indirectly which is more likely to cause problems.
    • A possibly better option is to run directly from the TS sources. This has a big issue of not dog-fooding tsc, but that could possibly still be used for testing (but see below).
  • Should also re-do the src/tsserverlibrary and src/typescriptServices stubs.
  • Some other files that need to be sorted out:
    • src/instrumenter (including src/loggedIO/tsconfig-tsc-instrumented.json)
    • src/services/exportAsModule.ts
• Running the tests takes around 1.5x more time with the modules.
  • This might be resolved if running from a bundle, but if this and the actual bundles are done with esbuild then there's no dogfooding at all.
• There are a few tests that still fail (12) -- should be resolved when there are bundles.
• Make the parallel tests work. (This depends on the resolution of the above, since it's an issue with how the tests are implemented wrt the global describe binding.)
• API extractor.
• Make the linter happy.
• Possibly a cleanup step, like getting rid of re-exporting modules
  for direct imports, and related (like *Impl.ts files).

[ahejlsberg]

Running esbuild directly on the TS sources likely isn't an option because const enum inlining isn't implemented by esbuild (or at least isn't implemented for references across modules). See here.

[andrewbranch]

That shouldn’t be a problem for local testing since const enum usage will still work without inlining via preserveConstEnums. For our final distributed bundle and perhaps for testing in CI, I think we want to use tsc as much as possible. But if using esbuild locally could let us run a test with near-zero delay, I think that would be awesome.

[jakebailey]

Closing in favor of #49332 (where I'll do something similar to this issue).