Next.js webpack throws loader error when importing external packages

[floroz]

What version of Next.js are you using?

12.0.3

What version of Node.js are you using?

16.13.0

What browser are you using?

Chrome, Safari

What operating system are you using?

macOS

How are you deploying your application?

Vercel

DISCLAIMER: Thank you for all the amazing work you do for the open-source community, providing us with great tools such as Next.js!

The issue

We are migrating our React application to Next.js within our monorepo project, but we have encountered a number of issues when trying to import UI components and styles from outside the Next.js root directory.

Observations on the issue

• consuming packages outside the root directory of a Next.js application, under a workspace managed repository, seems to only be possible by using the experimental.externalDir flag.
• There isn't a clear way to allow the use of external CSS modules or stylesheets without modifying the Next.js webpack's configuration.

Relates to:

• Importing correctly scoped css from node_modules outside of _app #12079
• RFC: Global CSS Imports #27953
• importing .css from node_modules from within a @shared/ui package  martpie/next-transpile-modules#122
• Experimental feature for allowing importing Typescript files outside of the root directory #22867

Expected Behavior

We want separate Next.js applications to consume and share the same UI components from outside their root directories.

In our example, we want admin and shop to consume and share the same components from libs/ui.

To Reproduce

The setup

• Clone this project: https://github.com/floroz/nextjs-monorepo-issue

For the purpose of illustration, I've setup an example repository that uses npm workspaces for managing the symlinks between the dependencies.

The project structure of this demo closely resembles (in principles) what we currently have in our repository.

├── apps
│   ├── admin
│   │   ├── node_modules
│   │   ├── pages
│   │   └── public
│   └── shop
│       ├── node_modules
│       ├── pages
│       └── public
├── libs
│   ├── shared
│   │   └── src
│   └── ui
│       └── src
├── package-lock.json
└── package.json

Scenario

1. In apps/shop/pages/index.tsx (a Next.js application) we try to import and consume a button coming from the shared UI layer.

import { Button } from "@tundra/ui";

export default function Home() {
  return (
    <div>
      <h1>Hello World</h1>
      <Button />
    </div>
  );
}

2. Once we try to start the application, we see this error:

../../libs/ui/src/button.tsx
Module parse failed: Unexpected token (7:9)
You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders

3. At which point, I enable the experimental.externalDir flag in the next.config.js and restart the application

4. The error we now see is:

error - ../../libs/ui/src/button.module.css
CSS Modules cannot be imported from within node_modules.
Read more: https://nextjs.org/docs/messages/css-modules-npm
Location: ../../libs/ui/src/button.tsx

5. If we now go into the libs/ui/src/button.tsx and comment out the CSS Modules import, and reload the application

import React from "react";
import { randomString } from "@tundra/shared";
// import styles from "./button.module.css";
const styles = { button: "button" };

export const Button = () => {
  return <div className={styles.button}>{randomString()}</div>;
};

6. it now renders the button correctly (without the styles being applied).

[floroz]

UPDATE: I have looked into next-transpile-modules and @nrwl/next plugins:

• https://github.dev/martpie/next-transpile-modules/blob/c5f548e9adafdc9c5afb8a97f15a92f92bdcb266/src/next-transpile-modules.js
• https://github.dev/nrwl/nx/blob/bea4ba22b189f4e89a92beeda32b44d35580df16/packages/next/plugins/with-nx.ts

I have put together a custom plugin to re-use the logic of those libraries to fit my use case:

plugins/withExternalLibs.js

/**
 * This plugins allows transpilation of external modules (either from node_modules, or folders outside the Next.js root directory).
 * Its logic is mostly a copy/paste/rearrange from `next-transpile-module` (https://github.com/martpie/next-transpile-modules) and the `withNx` plugins (https://github.com/nrwl/nx/blob/master/packages/next/plugins/with-nx.ts)
 * @param {string[]} modules - an array of external packages and libs either in node_modules, or outside Next.js root directory (monorepo packages)
 */
function withExternalLibs(modules) {
    return function (nextConfig) {
        /**
         * In collaboration with Vercel themselves, it's advised to set the "experimental-serverless-trace" target
         * if we detect that the build is running on Vercel to allow for the most ergonomic configuration for Vercel users.
         */
        if (process.env.NOW_BUILDER) {
            console.log(
                'withExternalLibs() plugin: Detected Vercel build environment, applying "experimental-serverless-trace" target',
            );
            nextConfig.target = 'experimental-serverless-trace';
        }

        const userWebpack = nextConfig.webpack || ((x) => x);

        return {
            eslint: {
                ignoreDuringBuilds: true,
                ...(nextConfig.eslint ?? {}),
            },
            ...nextConfig,
            webpack: (config, options) => {
                /*
                 * Update babel to support our monorepo setup.
                 * The 'upward' mode allows the root babel.config.json and per-project .babelrc files to be picked up.
                 */
                options.defaultLoaders.babel.options.babelrc = true;
                options.defaultLoaders.babel.options.rootMode = 'upward';

                /*
                 * Modify the Next.js webpack config to allow external libs to use css modules.
                 * Note: This would be easier if Next.js exposes css-loader and sass-loader on `defaultLoaders`.
                 */

                // Include external libs in css/sass loaders
                const includes = [...modules];

                // grab the CSS loaders
                const nextCssLoaders = config.module.rules.find((rule) => typeof rule.oneOf === 'object');

                // if no loaders are found, webpack config is not as expected
                if (!nextCssLoaders) return config;

                // add external modules to CSS Modules loader
                addExternalLibsToCSSModulesLoader(includes, nextCssLoaders);

                // add external modules to SCSS/SASS Modules loader
                addExternalLibsToSASSModulesLoader(includes, nextCssLoaders);

                // add external modules to CSS Global loader
                addExternalLibsToCSSGlobalLoader(includes, nextCssLoaders, options.isServer);

                // remove CSS loader errors for the declared external modules
                excludeExternalLibsFromCustomErrorLoader(includes, nextCssLoaders);

                return userWebpack(config, options);
            },
        };
    };
}

function regexEqual(x, y) {
    return (
        x instanceof RegExp &&
        y instanceof RegExp &&
        x.source === y.source &&
        x.global === y.global &&
        x.ignoreCase === y.ignoreCase &&
        x.multiline === y.multiline
    );
}

/**
 * Modify css modules loader to enable module support for external libs
 * @param {string[]} externalLibs - list of external libs and modules to include in the loaders
 * @param {*} nextCssLoaders - Next.js CSS Loaders exposed in the Webpack config
 */
function addExternalLibsToCSSModulesLoader(externalLibs, nextCssLoaders) {
    const nextCssModulesLoader = nextCssLoaders.oneOf.find(
        (rule) => rule.sideEffects === false && regexEqual(rule.test, /\.module\.css$/),
    );
    // Might not be found if Next.js webpack config changes in the future
    if (nextCssModulesLoader) {
        nextCssModulesLoader.issuer.or = nextCssModulesLoader.issuer.and
            ? nextCssModulesLoader.issuer.and.concat(externalLibs)
            : externalLibs;
        delete nextCssModulesLoader.issuer.and;
    }
}

/**
 * Modify scss modules loader to enable module support for external libs
 * @param {string[]} externalLibs - list of external libs and modules to include in the loaders
 * @param {*} nextCssLoaders - Next.js CSS Loaders exposed in the Webpack config
 */
function addExternalLibsToSASSModulesLoader(externalLibs, nextCssLoaders) {
    const nextSassLoader = nextCssLoaders.oneOf.find(
        (rule) => rule.sideEffects === false && regexEqual(rule.test, /\.module\.(scss|sass)$/),
    );
    // Might not be found if Next.js webpack config changes in the future
    if (nextSassLoader) {
        nextSassLoader.issuer.or = nextSassLoader.issuer.and
            ? nextSassLoader.issuer.and.concat(externalLibs)
            : externalLibs;
        delete nextSassLoader.issuer.and;
    }
}
/**
 * Modify error loader to ignore css/scss modules used by external libs
 * @param {string[]} externalLibs - list of external libs and modules to include in the loaders
 * @param {*} nextCssLoaders - Next.js CSS Loaders exposed in the Webpack config
 */
function excludeExternalLibsFromCustomErrorLoader(externalLibs, nextCssLoaders) {
    const nextErrorCssModuleLoader = nextCssLoaders.oneOf.find(
        (rule) =>
            rule.use &&
            rule.use.loader === 'error-loader' &&
            rule.use.options &&
            (rule.use.options.reason ===
                'CSS Modules \u001b[1mcannot\u001b[22m be imported from within \u001b[1mnode_modules\u001b[22m.\n' +
                    'Read more: https://err.sh/next.js/css-modules-npm' ||
                rule.use.options.reason ===
                    'CSS Modules cannot be imported from within node_modules.\nRead more: https://err.sh/next.js/css-modules-npm'),
    );
    // Might not be found if Next.js webpack config changes in the future
    if (nextErrorCssModuleLoader) {
        nextErrorCssModuleLoader.exclude = externalLibs;
    }
}

/**
 * Modify Global CSS loader to enable module support for external libs
 * @param {string[]} modules -  list of external libs and modules to include in the loaders
 * @param {*} nextCssLoaders - Next.js CSS Loaders exposed in the Webpack config
 * @param {boolean} isServer - webpack flag for server side rendering
 */
function addExternalLibsToCSSGlobalLoader(modules, nextCssLoaders, isServer) {
    const nextGlobalCssLoader = nextCssLoaders.oneOf.find(
        (rule) => rule.sideEffects === true && regexEqual(rule.test, /(?<!\.module)\.css$/),
    );

    if (nextGlobalCssLoader) {
        nextGlobalCssLoader.issuer = { or: [...modules, nextGlobalCssLoader.issuer] };
        nextGlobalCssLoader.include = { or: [...modules, nextGlobalCssLoader.include] };
    } else if (!isServer) {
        // Note that Next.js ignores global CSS imports on the server
        console.warn('withExternalLibs - could not find default CSS rule, global CSS imports may not work');
    }

    const nextGlobalSassLoader = nextCssLoaders.oneOf.find(
        (rule) => rule.sideEffects === true && regexEqual(rule.test, /(?<!\.module)\.(scss|sass)$/),
    );

    // FIXME: SASS works only when using a custom _app.js file.
    // See https://github.com/vercel/next.js/blob/24c3929ec46edfef8fb7462a17edc767a90b5d2b/packages/next/build/webpack/config/blocks/css/index.ts#L211
    if (nextGlobalSassLoader) {
        nextGlobalSassLoader.issuer = { or: [...modules, nextGlobalSassLoader.issuer] };
    } else if (!isServer) {
        // Note that Next.js ignores global SASS imports on the server
        console.info('withExternalLibs - global SASS imports only work with a custom _app.js file');
    }
}

module.exports = withExternalLibs;

The issues resolved by this plugin are:

1. can import CSS/SCSS Modules from outside Next.js root;
2. can import CSS Global stylesheets from outside Next.js root;

The issue NOT resolved is:

1. cannot import SCSS Global stylesheets from outside the Next.js root;

Summary

The use of the experimental.externalDir = true combined with the custom plugin allows full transpilation of external files, with the sole exception of Global SCSS stylesheet.

I am still not sure I exactly understand the technical limitation (will need to dig in further) but the main questions for me to address is:

• Are there plans to expand the experimental.externalDir flag to also include *.module.s?css$ files?
  • If the answer is NO, can the ergonomics of accessing the CSS Loaders in the webpack configuration be improved to support these use cases?

[floroz]

I am closing this as it seems to be a constraint by design not to include CSS modules and forcing adoption of next-transpile-modules for monorepos.

[balazsorban44]

This issue has been automatically locked due to no recent activity. If you are running into a similar issue, please create a new issue with the steps to reproduce. Thank you.