[Feature]: Support transpiling external modules

[airjp73]

What is the new or updated feature that you are suggesting?

It would be great to have the ability to have the remix compiler transpile local modules when working in a monorepo. Essentially this would be a remix equivalent to next-transpile-modules.

One way to get this done currently is via a patch outlined in this gist. In the patch, there's a new config option in remix.config.js.

I believe this issue is also related to #1398. Looking at the compiler code, if we had the ability the bundle all server dependencies that might also solve the issue.

The exact implementation is not super important to me -- I'm more interested in the ability to do this than the api itself.

Why should this feature be included?

It would be very useful for monorepos containing a remix app to be able to have the remix compiler transpile local modules from the monorepo. This makes it really simple & easy to create internal packages to share across multiple apps without the added friction of setting up a build step for each one.

This change also doesn't require opening up esbuild configuration options directly. The patch does touch remix's custom externals plugin, but this shouldn't impact remix's ability to switch compilers. If the remix team decided to switch another compiler, this could be done without breaking the api for this feature (though the functionality would have to be replicated with the other compiler).

[hannadrehman]

i m having same issue. need an ability to transpile local modules in a monorepo.

[hannadrehman]

thanks to this solution given to overide esbuild config i was able to build my local packages.

i created esbuild-overrides.js

added following code to allow custom packages

const esbuild = require('esbuild');
const Module = require('module');

function manualExternalsPlugin() {
  return {
    name: 'manual-externals-overide',
    setup(build) {
      build.onResolve(
        {
          filter: /@YourNamespaceOrPackageName/,
        },
        (args) => {
          return {
            external: false,
            namespace: args.path,
          };
        },
      );
    },
  };
}

const originalRequire = Module.prototype.require;
const originalBuild = esbuild.build;

function build(options) {
  if (options.platform === 'node') {
    const { plugins } = options;
    const externalPlugin = plugins.find(
      (plugin) => plugin.name === 'manual-externals',
    );
    const localPlugins = plugins.filter(
      (plugin) => plugin.name !== 'manual-externals',
    );
    localPlugins.push(manualExternalsPlugin());
    localPlugins.push(externalPlugin);
    return originalBuild({
      ...options,
      plugins: localPlugins,
    });
  }
  return originalBuild({
    ...options,
  });
}

Module.prototype.require = function (id) {
  // when remix requires esbuild, it will get our modified build function from above
  if (id === 'esbuild') {
    return { ...esbuild, build };
  }
  return originalRequire.apply(this, arguments);
};

then added following scripts

"scrips": {
      "dev:patched": "NODE_OPTIONS='-r ./esbuild-overrides' remix dev",
     "build:patched": "NODE_OPTIONS='-r ./esbuild-overrides' remix build"
}

[cj]

fyi - transpile modules can be added to remix by just updating a few lines of code I came up with this a few months ago https://gist.github.com/cj/c6d7a3b1fcf238073951762747c201ba you can now just add transpileModules to your remix.config.js i.e.:

/**
 * @type {import('@remix-run/dev/config').AppConfig}
 */
module.exports = {
  appDirectory: 'app',
  browserBuildDirectory: './public/build',
  publicPath: '/build/',
  serverBuildDirectory: './server/build',
  devServerPort: 8002,
  transpileModules: ['ui', 'lib', 'env'],
}

[cj]

@airjp73 @hannadrehman I created a pull request here #1806

[hannadrehman]

@cj this is great. thanks.

[badeAdebayo]

thanks to this solution given to overide esbuild config i was able to build my local packages.

i created esbuild-overrides.js

added following code to allow custom packages

const esbuild = require('esbuild');
const Module = require('module');

function manualExternalsPlugin() {
  return {
    name: 'manual-externals-overide',
    setup(build) {
      build.onResolve(
        {
          filter: /@YourNamespaceOrPackageName/,
        },
        (args) => {
          return {
            external: false,
            namespace: args.path,
          };
        },
      );
    },
  };
}

const originalRequire = Module.prototype.require;
const originalBuild = esbuild.build;

function build(options) {
  if (options.platform === 'node') {
    const { plugins } = options;
    const externalPlugin = plugins.find(
      (plugin) => plugin.name === 'manual-externals',
    );
    const localPlugins = plugins.filter(
      (plugin) => plugin.name !== 'manual-externals',
    );
    localPlugins.push(manualExternalsPlugin());
    localPlugins.push(externalPlugin);
    return originalBuild({
      ...options,
      plugins: localPlugins,
    });
  }
  return originalBuild({
    ...options,
  });
}

Module.prototype.require = function (id) {
  // when remix requires esbuild, it will get our modified build function from above
  if (id === 'esbuild') {
    return { ...esbuild, build };
  }
  return originalRequire.apply(this, arguments);
};

then added following scripts

"scrips": {
      "dev:patched": "NODE_OPTIONS='-r ./esbuild-overrides' remix dev",
     "build:patched": "NODE_OPTIONS='-r ./esbuild-overrides' remix build"
}

This did not work for me. I get an error SyntaxError: Unexpected token 'export' it looks like esbuild does not transpile the typescript file before loading it. any idea on how to get it to do that from esbuild?

[adonaicandido]

This feature has been implemented in the latest release, although would be also helpful to watch those files and hot reload the browser.

Someone currently has a workaround #here.

[machour]

Implemented.
If the new implementation still doesn't fit your needs, please open a new issue 🙏🏼

[penx]

This feature has been implemented in the latest release
@adonaicandido

Implemented.
@machour

Which feature is this in reference to? #1806 was closed without merge.

transpileModules was the originally proposed config setting. I can see a reference to serverDependenciesToBundle from #1839 but that doesn't sound like the right setting and doesn't seem to be working for me (transpiling typescript from another package in a monorepo).

If I add

 serverDependenciesToBundle: ["@my-namespace/other-package"],

I still get

Error: Cannot find module '@my-namespace/other-package'

(Where the entry point to this package is a ts file)

Edit: not sure what I was doing wrong earlier, serverDependenciesToBundle is working for me now

[garretteklof]

I cannot get this to work. I'm working with pnpm in a turborepo monorepo. I have the transpileModules setting set in Remix to include my package .. but I get a SyntaxError: Unexpected token 'export' .. when it's main field is pointed to a .ts file .. both have ES2019 as targets .. if I include the package in serverDependenciesToBundle .. I get a different error where it can't find the path I setup in the tsconfig.json ..

I'm trying to port over an existing Next.js application where it was working using next-transpile-modules. Any guidance would be much appreciated.

Edit: I see now that transpileModules is not actually supported. It was a little confusing since there is a majority of 🚀 on that comment 😅 With that said, the question regarding serverDependenciesToBundle still stands. It's not recognizing the tsconfig.json for transpiled packages.. which may be intentional but now how next-transpile-modules functioned. Is there a way around using custom paths for transpiled packages apart from using relative directories in the Remix app tsconfig.json? I also notice that Remix warns about dependencies not being installed that are dependencies in transpiled packages.. What's the workflow I should be adopting for using Remix in a monorepo? Every package has its own build step?

[ianwremmel]

@garretteklof not sure if this will help with your issue, but I just ran into it: I'm working in a monorepo and remix is one of many packages laid out via npm workspaces. I had serverDependenciesToBundle set to be an array of strings of the package names of each package defined in the repo. That stopped working after a recent update. I was able to fix it by changing those strings to RegExps. I believe it's matching individual filenames rather than the package.json names that worked previously.

[osdiab]

I am getting this warning in the console (though the dev server seems to work nonetheless):

The path "@shared/ui/styles" is imported in app/routes/index.tsx but "@shared/ui/styles" was not found in your node_modules. Did you forget to install it?

If I change the import in the route to import styles from "@shared/ui/styles.ts" then the warning goes away, but TypeScript says I shouldn't be importing a file with the ts extension. I'm not bothering transpiling the shared UI library since it's only being consumed inside my monorepo and I can control the transpilation.

I had added serverDependenciesToBundle: [/^@shared\//] to my remix config, doesn't seem to be affecting things at a glance. Thoughts on how to get rid of this warning?