[FEATURE] absolute->relative module path transformation

[nakamorichi]

Problem

tsc does not support transforming absolute module paths into relative paths for modules located outside node_modules. In client side apps, this is often not an issue because people tend to use Webpack or similar tool for transformations and bundling, but for TypeScript apps targeting Node.js, this is an issue because there is usually no need for complex transformations and bundling, and adding additional build steps and tools on top of tsc only for path transformation is cumbersome.

Example input (es2015 style):

import { myModule } from 'myModuleRoot/a/b/my_module';

Example output (CommonJS style):

const myModule = require('./a/b/my_module');

My personal opinion is that relative module paths (import { ... } from '../../xxx/yyy';) are an abomination and make it difficult to figure out and change application structure. The possibility of using absolute paths would also be a major benefit of using TypeScript for Node.js apps.

Solution

Compiler options for tsc similar to Webpack's resolve.modules.

Could this be achieved for example with existing baseUrl and paths options and adding a new --rewriteAbsolute option?

Related

#5039
#12954

[aluanhaddad]

My personal opinion is that relative module paths (import { ... } from '../../xxx/yyy';) are an abomination and make it difficult to figure out and change application structure.

While I heartily agree with your sentiment, I think rewriting such imports would open up a can of worms that would ultimately break or complicate an insanely large number of tools and workflows that rely on the emitted JavaScript using the same module specifiers. Even under a flag, I think it will introduce a lot of complexity.

I hate relative paths that go up, it is just awful for maintainability, but my two cents is that this needs to be done on the NodeJS side, not the transpiler side. Of course it's extremely unlikely that will ever happen...

[ikokostya]

This issue can be solved if add node_modules directory with symlink to src directory:

project_root/
  node_modules/  <-- external modules here
  src/
     node_modules/  <-- keep this folder in git
        src -> ../src  <-- symlink to src
     a/
        b/
           c.ts
     d.ts
  tsconfig.json

In this case you can use the following import in c.ts:

import * as d from 'src/d';

instead

import * as d from '../../d';

All will work in typescript and commonjs. You just need to add exclude field in tsconfig.json:

{
    "exclude": [
        "src/node_modules"
    ]
}

[nakamorichi]

@aluanhaddad I've been using path rewrite (Webpack) on client-side since I began writing React apps. How would it cause problems on the server side if it doesn't cause problems on the client side? Editors already support setting module roots, and linters (ESLint, TSLint) seem to be fine also.

To the maintainers of TypeScript: Please add support for custom module roots so that we can get rid of the awful and unmaintainable import paths.

[nakamorichi]

@ikokostya Many tools have special treatment for node_modules, and it is always treated as a folder for external code. Putting app code into node_modules may solve the import path problem, but introduces potential other problems. It is also ugly solution that shouldn't, in my opinion, be recommended to anyone.

[ikokostya]

Many tools have special treatment for node_modules, and it is always treated as a folder for external code.

Could you provide example of such tools? In my example all external code are placed in node_modules in project_root.

Putting app code into node_modules may solve the import path problem, but introduces potential other problems.

Which problems?

It is also ugly solution that shouldn't, in my opinion, be recommended to anyone.

Maybe you should read this

• https://gist.github.com/branneman/8048520
• http://stackoverflow.com/questions/10860244/how-to-make-the-require-in-node-js-to-be-always-relative-to-the-root-folder-of-t#24630974

And why it's ugly? It uses standard way for loading from node_modules.

[aluanhaddad]

@Kitanotori perhaps I misunderstood your suggestion, TypeScript supports this feature with the --baseUrl flag.
My point was that NodeJS doesn't support it and that TypeScript should not attempt to provide the future on top of NodeJS by rewriting paths in output code.

@ikokostya

Could you provide example of such tools?

There are too many to count but TypeScript is definitely an example of such a tool.

I think putting application code in a node_modules folder is a very ill-advised hack.

[nakamorichi]

@aluanhaddad --baseUrl setting enables to transpile the app, but what's the point of being able to transpile successfully if you can't execute it? ts-node does not seem to support --baseUrl, so I think it is inaccurate to say that TypeScript supports custom absolute paths.

@ikokostya Thanks for the links. It seems that setting NODE_PATH in the startup script is the best way currently. I think I will go with that approach.

[aluanhaddad]

@Kitanotori

@aluanhaddad --baseUrl setting enables to transpile the app, but what's the point of being able to transpile successfully if you can't execute it? ts-node does not seem to support --baseUrl, so I think it is inaccurate to say that TypeScript supports custom absolute paths.

The point is that it does execute perfectly in environments that support that. RequireJS, SystemJS, and even Webpack support setting a base URL.

What I'm trying to say is that the issue is on the NodeJS side. TypeScript provides base URL configuration to integrate with and take advantage of those other tools.

[RyanCavanaugh]

Our general take on this is that you should write the import path that works at runtime, and set your TS flags to satisfy the compiler's module resolution step, rather than writing the import that works out-of-the-box for TS and then trying to have some other step "fix" the paths to what works at runtime.

We have about a billion flags that affect module resolutions already (baseUrl, path mapping, rootDir, outDir, etc). Trying to rewrite import paths "correctly" under all these schemes would be an endless nightmare.

[nakamorichi]

@RyanCavanaugh Sorry to hear that. If Webpack team was able to deliver such feature, I thought TypeScript team could also - considering that TypeScript even has major corporate backing. It's a shame that people are forced to add another build step on top of tsc for such a widely needed feature.

[morlay]

Still hope TypeScript could support this.
Or Just make it easy to create a plugin, so we can build something like babel-plugin-module-resolver to make it work. (ref: #11441)

[azarus]

So anyone got any solution or a really nice workaround without babel to make this happen ?

Edit
I've ended up with a custom transform script using gulp & tsify & gulp-typescript
https://gist.github.com/azarus/f369ee2ab0283ba0793b0ccf0e9ec590
Browserify & Gulp samples included.
So anyone got any solution or a really nice workaround without babel to make this happen ?

[nakamorichi]

@azarus I went with the solution of adding this to my top level file:

import * as AppModulePath from 'app-module-path';
AppModulePath.addPath(__dirname);

The downside is that if the loading order differs from expected, the app crashes. Better solution would be to have a compiler plugin for converting absolute paths to relative paths. However, I'm not sure if TypeScript has any kind of plugin feature.

[nakamorichi]

I created a post-build script for converting absolute paths to relative (doesn't yet have way to set the module root paths, but one can easily implement such): https://gist.github.com/Kitanotori/86c906b2c4032d4426b750d232a4133b

I was thinking of having the module roots being set in package.json via moduleRoots array containing path strings relative to package.json. I wonder what kind of risks there are in this kind of approach?

[azarus]

I've ended up with my own post build script too,
https://gist.github.com/azarus/f369ee2ab0283ba0793b0ccf0e9ec590
Browserify & Gulp samples included.

It acutally uses the tsconfig.json paths and baseUrl setup :)
I am gonna make a npm package from this during the weekend, i just haven't had time to properly test the script.

[kayjtea]

I have this in my tsconfig:

{
  "compilerOptions": {
    ...
    "module": "commonjs",
    "moduleResolution": "node",
    "rootDir": "src",
    "baseUrl": "src",
    "paths": {
      "~/*": ["*"]
    }
  },

I can't remember if the other options above are necessary, but "paths" will allow this:

import {fundManagersApi} from "~/core/api/fund-managers.api";

The core directory is at src/core.

And then I use the npm package "module-alias" and do:

require('module-alias').addAlias("~", __dirname + "/src");

At the top of my top level file.

Webstorm understands the paths option and will auto-import correctly even.

[vakhtang]

I use tsconfig-paths to handle this during runtime. I'm not aware of any performance implications or issues otherwise. The one objection I could see is monkey-patching Node's module resolution strategy.

[0x80]

@kayjtea I don't think module and moduleResolution are related. Here's a slightly simpeler version that seems to work:

    "baseUrl": "./",
    "paths": {
      "~/*": [
        "src/*"
      ]
    },

Personally, I prefer writing @src, and for this you can use:

 "baseUrl": "./",
    "paths": {
      "@src/*": [
        "src/*"
      ]
    },

In VSCode I don't seem to have to do anything with my top-level files or npm module-alias.

--- edit ---
Ah damn it. I see now that the paths don't work at runtime work if you don't do more then make the compiler and VSCode happy.

[azarus]

Well i solved this sort of problem by splitting my apllication into npm modules and linking them during development. Symlinks are generated and i find it much easier to solve the relative path hell. However theres still a great need to properly support importing modules by absolute/root path! Any other solution just seems to be band-aid for a pain that needs pain killer.

…

On Jan 6, 2018 22:21, "Thijs Koerselman" ***@***.***> wrote: @kayjtea <https://github.com/kayjtea> I don't think module and moduleResolution are related. Here's a slightly simpeler version that seems to work: "baseUrl": "./", "paths": { "~/*": [ "src/*" ] }, — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#15479 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFlNc1RXUW0Hx4D4MF-UMQVMQQ06CouCks5tH49bgaJpZM4NMg4-> .

[0x80]

@azarus I think I might have to go that route too.

I have two repo's that are also depending on roughly the same set of helpers and services. And I am getting tired of copying code between the two.

[azarus]

@0x80 i ran into a similar problem when had to share model definitions between 5 different services. I solved this by creating a shared folder that everyone were able to access if needed.
Setting up an NPM link is also easy and less hassle than you would think.
But i am sad that proper solution is still not in place.

[duffman]

Use tspath (https://www.npmjs.com/package/tspath) run it in the project folder after compile, no config needed, it uses the tsconfig to figure things out (a recent version of node is needed)

[0x80]

@duffman Thanks for the tip! I'll check it out 👍

[stereobooster]

In case you want to configure create-react-app project add:

{
  "compilerOptions": {
    //... other react-scripts-ts options
    "paths": {
      "src/*": ["*"]
    },
    "baseUrl": "src"
  },

Is equivalent to NODE_PATH=./ in .env in standard c-r-a project.

[RacerDelux]

Another issue is when you use typescript with the msbuild typescript package.
I don't have NPM, I don't use NPM build pipeline, I don't use webpack.
I just want the output javascript files to append the base url (as an option).
What is the argument against this setting in compilerOptions?:
"appendBaseUrl" : true or false

This would allow us to use relative paths just fine.
import { Something } from "global.js";
Would become:
import { Something } from "./global.js";
With these options in tsconfig:
"compilerOptions": { "baseUrl": "./", "appendBaseUrl": true }

[loopingz]

So I find this issue interesting annoying,

I do understand the concept of not modifying the JS but.:

When I have a target with es5, you transpile every await/async, import etc to the promise/require, the generated code does not look at all like my .ts files.
Yet on this one to just add the extension at the end of the import, seems to violate the typescript principle.

I must admit I am confused

[ArcticLampyrid]

For anyone who may concern this:
I've found a custom transformer: LeDDGroup/typescript-transform-paths
We can load it with ts-loader or awesome-typescript-loader by wrting such config:

const createTsTransformPaths = require('typescript-transform-paths').default;
{
    test: /\.tsx?$/,
    use: "ts-loader",
    use: {
        loader: "ts-loader",
        options: {
            getCustomTransformers: (program) => ({
                before: [createTsTransformPaths(program, {})],
                afterDeclarations: [createTsTransformPaths(program, {
                    afterDeclarations: true
                })]
            })
        }
    },
    exclude: /node_modules/
}

[joecarl]

Design goal number 7 says

Preserve runtime behavior of all JavaScript code.

When you write the JavaScript line

var n = 10;

TypeScript says "This is JavaScript code" and will always emit

var n = 10;

When you write JavaScript code, TypeScript says that you wrote JavaScript code, and the behavior of this is preserved.

I'm quoting this comment here because I come from this other issue

I know the issue discussed here is not exactly about what I'm going to say but here it goes:

Look at this typical import statement:

import MyClass from '../utils/MyClass'

Is that javascript code? Well, yes it is. But the problem here is that each language will interpret this line differently!

And when you transpile code shouldn't it be in a way so the transpiled code is interpreted exactly in the same way the original code was?

It's not about changing JavaScript semantics but rather aligning TypeScript's behavior with JavaScript to ensure consistent interpretation.

I don't know, maybe the followingTypescript dev team principle should be reconsidered.

We are not going to implement features, even under a commandline flag, that imply changing JS semantics by rewriting it during emit

[RyanCavanaugh]

It's not about changing JavaScript semantics but rather aligning TypeScript's behavior with JavaScript to ensure consistent interpretation.

This is exactly why we don't modify the paths - so that it's never a potential source of disagreement between you, TypeScript, and the runtime.

The new module documentation covers this in exhaustive detail and I'd recommend reading it.

I don't know, maybe the followingTypescript dev team principle should be reconsidered.

The problem here is not that we haven't thought about it over the hundreds of comments we've posted and thousands of comments we've read.

[RacerDelux]

@RyanCavanaugh If your goal is to stick to the JS standards, then please do that.
This means you should by default assume any import is referring to an es6 module, not a node module, and treat it as such.
I get that when typescript came out we didn't have modules outside of node, but we do now.

As it stands, by not wanting to disagree with the runtime, you mean "do not disagree with the node runtime".

We NEED the ability to tell the compiler we are running in a node or es environment, as the way imports are handled in those two environments are fundamentally different.

[adrianstephens]

I don't know if anyone looks at old, closed issues, but I suppose this goes here.

In my ignorance of this controversy, I implemented a flag to rewrite the paths (#60723) after many, increasingly dodgy, attempts at recommended workarounds. The glorious feeling of finally being able to stop fighting vscode and tsc was very short lived when I became aware of just how adamantly forbidden it was.

I thought my use case must be very unusual, because of course tsc should be doing this, but this thread has disabused me.

The dogma is that JS must not be modified; but at least in my scenario JS requires relative paths for everything, and I want the JS to be modified. I suppose I could preprocess the TS to fix the paths, or I could postprocess the JS, but the tsconfig paths and rootUri already do exactly what I want - but then it skips the final logical step and throws the information away.

I know how long it took me to try and get around this apparent oversight (before giving up), and we can multiply that by the many thousands of other people facing the same situation. I have read the arguments against modifying the paths and I'm sure there are cases where they make sense, but this is not one of them. Typescript exists to improve the experience and safety of developing javascript, and enabling non-relative paths would absolutely add to that mission.

[RacerDelux]

I know how long it took me to try and get around this apparent oversight (before giving up), and we can multiply that by the many thousands of other people facing the same situation. I have read the arguments against modifying the paths and I'm sure there are cases where they make sense, but this is not one of them. Typescript exists to improve the experience and safety of developing javascript, and enabling non-relative paths would absolutely add to that mission.

Welcome to the fight ha.

Sadly I don't see them moving on this any time soon. I ended up integrating vite into my .net build runtime to get around this issue.
Is it ok that their philosophy is to force us to these measures? No. Will it change? Probably not.

[adrianstephens]

@RacerDelux at least we have a support group
"I am Adrian, and I tried to use non-relative paths in typescript"

I'm going to use my forked version for now because I can be stubborn too!

[ArcticLampyrid]

For everyone who met this, I recommand to use https://github.com/LeDDGroup/typescript-transform-paths, which seems more friendly to maintainers than making a fork directly.