Symlink behaviour

[subhero24]

If I add a symlink in my src directory to another directory, and then include files from that path, create-react-app gives an error:

Module parse failed: Unexpected token
You may need an appropriate loader to handle this file type.

I assume create-react-app wants to have this import already built/transpiled. I would expect this to be the case for imports outside the src directory (node_modules for example). But since the symlink resides in the src directory, I would assume create-react-app would fetch these files as if they were truly in src directory.

Is this a bug, or expected behaviour? It makes it really hard to extract common components.

[Timer]

It sounds like a bug, but this should be working. Can you give more details to reproduce this easily & some test files? A repo with README would be fantastic.

[subhero24]

I created a repo here.

This was just initialisation with CRA under MacOS High Sierra and then symlinking in terminal with:

cd src && ln -s ../symlink

[Timer]

Thanks!

[nuthinking]

I actually have only problems with Symlinks when building for deployment: #3650

On dev it works well.

[bradfordlemley]

@BrunoVanDamme Can you elaborate on your use case? (Is it related to trying to share source between cra-apps and/or monorepo?)

[subhero24]

I am trying to share components between projects, so I would like to have these components in another directory and symlink to them from all the cra-apps where i would like to use them.

[bradfordlemley]

@BrunoVanDamme Have a look at #1333 which is for supporting source sharing via monorepo manager (lerna and/or yarn workspace).

I proposed a more generic source-sharing solution in #3436, but it has some downsides vs using a monorepo manager: (1) required some cra-configuration to specify allowable source paths, and (2) managing dependencies of those shared components is a job best left to some monorepo manager anyways...so, my favor has turned more toward using monorepo w/ managers for sharing source between apps. Curious if there's a reason for not using lerna and/or yarn workspace?

[subhero24]

As I understand yarn workspaces uses symlinks in the node_modules directory to other locations. My understanding is that CRA does not process files from node_modules, so I would have to do the transpiling on these external components myself. Is this correct? I would expect CRA to process the files symlinked from src as these files appear as if they where in the project itself.

[bradfordlemley]

Correct, currently CRA does not process those files, but #1333 tracks the feature request to make CRA process those files.

I think PR 3741 is pretty close to completing that feature, but there are some (minor?) open questions about how it should work. See questions in #1333 (comment) -- any feedback you could provide in that thread would be awesome and helpful in building consensus as to how it should work.

[gaearon]

@bradfordlemley Thanks a lot for jumping on that issue btw. I'm sorry repo infra changes are being a bit disruptive right now; hopefully we can review them soon!

[gaearon]

We also merged Jest 22 into next branch so you might want to use that as a base.

[subhero24]

@bradfordlemley Thanks for clarifying. I am not knowledgeable enough on this topic, but I don't understand where the technical difficulties are in handling these symlinks. (I can imagine processing the dependencies in node_modules is a whole different matter, as not all dependencies should be handled the same way). Why isn't just resolving these symlinks a simple solution to this problem?

[bradfordlemley]

I don't think it's really a technical issue (although the comments in #3695 maybe show that it's not as simple as it seems), maybe more about encouraging maintainable practices.

If you symlink from under /src, you have to import via relative path (unless you do some other magic with NODE_MODULES) and then maintain various symlinks if you change locations, and then there's defining, installing, and resolving dependencies for your external source. These are things that can make your build fragile, but monorepo managers seem to do those things well.

There is a bit of a learning curve to the monorepo managers, but maybe worth it in the long run, seems like the direction many folks are headed (???), btw, they also help with de-duplicating dependencies.

[bradfordlemley]

All that said, it is really easy to just create a symlink from under /src and seems like a reasonable expectation that it should be treated as if it was in /src, so hope others will chime in on this discussion.

[1st]

I also interesting in building two separate ReactJS projects. Both of them are using the same shared library or UI Components. I created a symlink in both projects to this shared UI Components directory.

When I'm trying to import something from that UI Components directory - it tells the same error message as in the original message:

Module parse failed: Unexpected token (20:18)
You may need an appropriate loader to handle this file type.

It happening because code isn't viable by a watcher from create-react-app that is watching for changes in one of my my projects.

Can someone give me a hint how to solve this problem for now? Because I potentially can copy he entire directory to both projects, and write some watcher that will do it for me on the regular basis. But it looks ugly solution. I wish to use the same shared directory between two projects and include "raw" files that then will be compiled.

Need your help.

Thank you,
Anton

[1st]

BTW, here webpack/webpack#1643 I found some information about webpack and symlinks. It looks like webpack can find our files but doesn't run a compilation process for them.

[bradfordlemley]

@1st Have you looked at monorepo support in 2.0? See 2.0 roadmap for updates and on how to use the alpha builds.

Sharing code that way has several advantages over manual symlinks, but mainly allows your shared code to be truly modularized with its own dependencies, etc. Manual symlinks like you're requesting aren't supported in 1.0 or 2.0 yet -- if you find manual symlinks preferable, it'd be great to give the reasons here.

[1st]

Finally I found a workaround. You can run a watch process to compile your shared library and use it as symlink in your project. Here is my gist.

[anton-g]

I have a similar issue as @ali-wetrill except I haven't set up a symlink but rather uses the project reference feature in typescript. So in my clients tsconfig I've added this:

"references": [{ "path": "../common" }]

Interfaces and types work fine, but enums break with this error:

Module not found: You attempted to import ../../../../common/src/models/message-type which falls outside of the project src/ directory. Relative imports outside of src/ are not supported.

[mako-taco]

It's not a problem with enums. Everything else you've tried to export is valid javascript AND valid typescript. Try writing a function with a type signature; I expect you'll see the same issues.

Forcing webpackConfig.resolve.symlink = false solves the issue.

[rmarfil3]

@mako-taco I have a similar problem as @ali-wetrill. I added webpackConfig.resolve.symlink = false like you said using customize-cra and it no longer resolves to the actual path. However, it's still showing the same error about "keyword enum is reserved, may need an appropriate loader".

[vicary]

It seems customize-cra doesn't help symlink paths by postcss-loader et al.

[PeledYuval]

Here's the Solution we've arrived to at Vim (no, not the text editor :) ) for working with CRA + symlinked components in a monorepo:

1. Install craco (allows for config overrides in create-react-app. This might seem like exactly the opposite of what CRA wants to be - and it is - but sometimes CRA is too-opinionated. Between having CRA and minimal craco config and having to manage a complete webpack+typescript+jest+babel stack, I'd choose the former)
2. Create a craco.config.js file in your project root. Use the code below as a starting point for it
3. In your package.json, change your start, build, test scripts to craco start, craco build, craco test

Portions of our craco.config.js:

const cracoLessPlugin = require('craco-less');
const path = require('path');
const { whenProd } = require('@craco/craco');

/* Allows importing code from other packages in a monorepo. Explanation:
When you use lerna / yarn workspaces to import a package, you create a symlink in node_modules to
that package's location. By default Webpack resolves those symlinks to the package's actual path,
which makes some create-react-app plugins and compilers fail (in prod builds) because you're only
allowed to import things from ./src or from node_modules
 */
const disableSymlinkResolution = {
  plugin: {
    overrideWebpackConfig: ({ webpackConfig }) => {
      webpackConfig.resolve.symlinks = false;
      return webpackConfig;
    },
  },
};

const webpackSingleModulesResolution = {
  alias: {
    react$: path.resolve(__dirname, 'node_modules/react'),
    'react-dom$': path.resolve(__dirname, 'node_modules/react-dom'),
    'react-router-dom$': path.resolve(__dirname, 'node_modules/react-router-dom'),
  },
};

const jestSingleModuleResolution = {
  moduleNameMapper: {
    '^react$': '<rootDir>/node_modules/react',
    '^react-dom$': '<rootDir>/node_modules/react-dom',
    '^react-router-dom$': '<rootDir>/node_modules/react-router-dom',
  },
};

module.exports = {
  plugins: [...whenProd(() => [disableSymlinkResolution], [])],
  webpack: webpackSingleModulesResolution,
  jest: {
    configure: {
      jestSingleModuleResolution,
    },
  },
};

Sadly this does require manual management of each broken import. Over a few months we haven't seen too many of these, so this is fine for now.

Please excuse syntax errors if any, this is edited from our real configuration for brevity.

[harryjubb]

Had this issue with symlinking a TS file, where non-TS syntax works, but on adding any TS syntax, get the "Unexpected token" error.

In my case so far, adding CRACO with their instructions, with the following simple craco.config.js, has worked:

module.exports = {
  webpack: {
    configure: (webpackConfig, { env, paths }) => ({
      ...webpackConfig,
      resolve: {
        ...webpackConfig.resolve,
        symlinks: false
      }
    })
  }
}

Presumably when #7993 is merged this fix will no longer be needed.

[nitinkatyal1314]

With {symlinks: false} the hot reloading does not work. Changing the src code for which link was created is not hot reloaded in the browser. Is there a solution to this?

[dudulasry]

@nitinkatyal1314 The hot reload functionality doesn't work for me either, when disabling the symlinks. Does anyone know how to solve it?

[romgrk-comparative]

For googlers ending up here, there is a (better) alternative to { symlinks: false }. This is especially useful if you're using pnpm (which you should) because pnpm symlinks all modules. The alternative is to modify the webpack config to add the other directories to the include list of the loader that should process the file, which is the issue that symlinks: false work-arounds without really solving. Below is an example using CRACO, but a similar modification will work for anything else that let's you modifiy the webpack config.

/* craco.config.js */
const path = require('path')

const updateWebpackConfig = {
  overrideWebpackConfig: ({ webpackConfig }) => {

    // This is a bit brittle, but this retrieves the `babel-loader` for me.
    const loader = webpackConfig.module.rules[1].oneOf[2]
    loader.include = [
      path.join(__dirname, 'src'),
      path.join(__dirname, '../backend/src'), // This is the directory containing the symlinked-to files
    ]

    return webpackConfig;
  }
}

module.exports = {
  plugins: [
    { plugin: updateWebpackConfig, options: {} }
  ]
}

/cc @romgrk

[benvium]

@romgrk-comparative This really helped me :-). Thanks. I improved the 'bit brittle' part by using built-in utilities from craco to grab babel-loader without 'magic numbers' in the code.

const path = require('path');
const {getLoader, loaderByName} = require('@craco/craco');

// Relative paths to shared folders.
// IMPORTANT: If you want to directly import these (no symlink) these folders must also be added to tsconfig.json {"compilerOptions": "rootDirs": [..]}
const SRC_LOCATIONS = [
 'src',
  '../../shared/src',
];

const updateWebpackConfig = {
  overrideWebpackConfig: ({webpackConfig}) => {

    // Get hold of the babel-loader, so we can add shared folders to it, ensuring that they get compiled too
    const {match:{loader}} = getLoader(webpackConfig, loaderByName("babel-loader"));

    loader.include = SRC_LOCATIONS.map(p => path.join(__dirname, p)),

    return webpackConfig;
  }
}

module.exports = {
  plugins: [
    { plugin: updateWebpackConfig, options: {} }
  ]
}

[RandScullard]

The solution from @romgrk-comparative and @benvium is working great for me now, but I had a lot of trouble getting started because of a quirk in the Windows mklink command. If you're on Windows, read on...

When you're creating a symlink, the mklink command remembers the case of the destination folder as you typed it. I had a folder named C:\RDS\ATWebsites\ATClientShared\src and I typed:

mklink /J shared \rds\ATWebsites\ATClientShared\src

Note that I didn't bother upper-casing the RDS part of the path. (Why would I, when Windows pathnames are case-insensitive... right?)

The resulting symlink remembered the lower-case rds and since the actual source file pathnames start with upper-case RDS, none of them matched the loader.include and I got the "unexpected token" error from webpack. 😭

I'll never get back the hours I wasted figuring this out, but maybe I can save someone else the trouble.

[TeemuKoivisto]

I don't really understand what's going on with this symlinking soup. All I know my CRA app crashes due to library being imported twice in my pnpm monorepo. The fixes shown above didn't really work but using aliases did:

const path = require('path')

const updateWebpackConfig = {
  overrideWebpackConfig: ({ webpackConfig }) => {
    // As described in https://github.com/facebook/create-react-app/issues/3547
    // there are some issues with how CRA treats symlinks which pnpm heavily uses.
    // This hack should fix this issue (loading prosemirror-model twice) for now
    webpackConfig.resolve.alias = {
      'prosemirror-model$': path.resolve(__dirname, '../editor/node_modules/prosemirror-model'),
    }
    return webpackConfig;
  }
}

module.exports = {
  plugins: [
    { plugin: updateWebpackConfig, options: {} }
  ]
}

Basically I'm aliasing the library to the main module that uses it. Not really scalable but I hope I'll be able to contain it to only a few broken modules.

[RandScullard]

I'm using another method to solve the duplicate package problem. Replace @TeemuKoivisto's webpackConfig.resolve.alias line with:

webpackConfig.resolve.modules = webpackConfig.resolve.modules.filter(mod => mod !== "node_modules")

Explanation: CRA configures this modules array to include two entries: node_modules and the full path of ClientApp/node_modules. The line of code above removes node_modules. Now webpack will only resolve modules from your main ClientApp/node_modules folder tree and not from any of your "sibling" shared code projects. This means no duplicated modules, and no need to add modules one by one to your craco config.

Note: You will need to npm install in your main ClientApp the union of all packages you've installed in your shared code projects. (In @TeemuKoivisto's example above, prosemirror-model needs to be installed in the main app even if it is only used in ../editor.) In my project this is just a few packages - YMMV. You can try the config line above and run webpack to immediately see what you need to install.

Note 2: It's important to prevent all module duplication even if it doesn't cause a runtime error, because it can significantly increase the size of your webpack bundle. (You can use source-map-explorer to do a before-and-after comparison.)

[TeemuKoivisto]

@RandScullard hi and no I don't. It works fine with the dependency being just inside editor (which in turn imports another workspace package that uses prosemirror-model). Anyway, installing the deps for the CRA would be even more annoying than just adding aliases so I'll let my hack stand in my repo. But thanks though for the explanation!

[RandScullard]

I just had to add another thing to my overrideWebpackConfig for my TypeScript project:

const tscheckerPlugin = getPlugin(webpackConfig, pluginByName("ForkTsCheckerWebpackPlugin"))
tscheckerPlugin.match.options.reportFiles.splice(0)

Explanation: ForkTsCheckerWebpackPlugin does the TypeScript type checking in the webpack build. The default webpack config sets the reportFiles option for this plugin to filter out errors from anywhere outside the local project¹. As a result I did not get any TypeScript compiler errors from my shared project, only from my main project. My solution is just to clear out the reportFiles array so there is no filtering of errors. (This might be too broad a brush for your project and you may find it preferable to modify the array to leave some of the filters in place.)

---

¹ This is the default value for reportFiles:

    [
            '../**/src/**/*.{ts,tsx}',
            '**/src/**/*.{ts,tsx}',
            '!**/src/**/__tests__/**',
            '!**/src/**/?(*.)(spec|test).*',
            '!**/src/setupProxy.*',
            '!**/src/setupTests.*',
    ]

[mcqj]

All this to try to get symlinks to work the way the OS designers intended them to work?

Surely, this needs re-evaluation on the part of CRA? If an application developer is creating symlinks, he/she probably has a good reason - I don't think its right to second guess them.

[RandScullard]

All this to try to get symlinks to work the way the OS designers intended them to work?

FWIW, I used the techniques described in this issue to get my shared module working without symlinks. Rather than symlinking my shared module into my main project, I have it linked in my package.json like so:

"atclient-shared": "file:../../ATClientShared",

[mcqj]

Thanks @RandScullard - yes, there are workarounds but they shouldn't really be necessary. I sometimes use a similar workaround to the one you describe but HMR doesn't work; package manager caching gets in the way. I have to delete node_modules and/or clean caches when updating components - all very unproductive.

[hasan-aa]

Hi @RandScullard,
Just wanted to confirm, are you using CRA v5.0.0?

Since I've upgraded my code to v5.0.0 none of the above workarounds seem to work.

To get rid of the duplicates, I had to remove duplicated packages from the main app and rely on the nested dependencies of linked packages.

[RandScullard]

@hasan-aa I am not using CRA v5, I am still on 4.0.3. I am waiting to upgrade until craco fully supports v5. (dilanx/craco#378)

I have taken a look at the changes to webpack.config.js in CRA v5 and I can tell you that the workarounds above will definitely have to be modified to be compatible with the new config. I believe they still can be made to work, but not exactly as shown above.

[hasan-aa]

Hi @RandScullard Thanks for the clarification.

It turned out the caching behavior was making it almost impossible to debug. If the previous build was creating duplicate modules, even if you fix the configuration and build again, it was still resolving the wrong modules due to the cache I believe.

Once I enable memory cache, it became debuggable and now it's working fine.
config.resolve.cache = true; (memory cache)

By the way, my solution is to move 'node_modules' to second position in the config.resolve.modules array so that the project node_modules folder takes the precedence:

before:

config.resolve.modules:["node_modules", "path/to/project/node_modules"]

after

config.resolve.modules:["path/to/project/node_modules", "node_modules"]

[Alevale]

For the ones hoping for a fix for webpack and when you npm link a package you will have to do the following.
It seems like when you have a linked package npm link "myPackage" (you are creating a symlink) this does not get properly resolved by the following line

const resolveApp = relativePath => path.resolve(appDirectory, relativePath);

To fix npm link packages not updating on your main directory you have to do the following:

1. Add the following to follow symlinks (which have been created) under /config/webpack.config.dev.js

{
    resolve: {
        symlinks: true,
    }
}

2. Under the module rules for JS, add the paths of your package to the "include" (see this example) /config/webpack.config.dev.js

{
  test: /\.(js|jsx|mjs)$/,
  include: [paths.appSrc, paths.myPackage],
  use: {
    ...
  }
}           

3. Add the following code to resolve the symlink to the right path /config/paths.js (the resolveRealPathApp will also work in a production build as it will just get the normal package, and not follow the symlinked one)

const resolveRealPathApp = relativePath => fs.realpathSync(process.cwd() + relativePath)

4. Add the following to the export section of the /config/paths.js file

module.exports = {
  ...
  myPackage: resolveRealPathApp('/node_modules/myPackage')
};

Catches:
Verify that you are not ignoring the folders in the /config/webpackDevServer.config.js file

watchOptions: {
  ignored: ignoredFiles(paths.appSrc),
}

[joelwestland]

Has any progress been made on this issue? I was using symlinks as a way to share interfaces between my Firebase Cloud Functions project and its React frontend project, but was surprised to find that as soon as I added enums everything broke (similar to the issue that @ali-wetrill was having above). Unfortunately, all the workarounds above seem to rely on unmaintained stuff, with react-app-rewired, customize-cra, and craco all not supporting CRA 5.0. If not, does anyone have any alternative suggestions for how to achieve the same thing?

[e40]

It's a decades old trick to symlinks to a single source directory for multiple architecture builds. This issue has broken that for me. 👎

[muvaf]

FWIW, I've added a small cp -a ../shared src/ command to all my NPM commands in package.json and added the src/shared under every project to their .gitignore file to work around the issue. From CRA's perspective, it's just a file and you still get to maintain a single copy for multiple projects.

[a-i-joe]

FWIW, I've added a small cp -a ../shared src/ command to all my NPM commands...

I've elaborated on this solution by using watch to monitor the files (the source and destination) and rerun rm -rf and copy commands to sync destination with source if they are changed in either location (rsync is not universally available). I then run this in parallel with react scripts like start and test, which support live loading. This means the live coding aspect is fully inclusive of the shared files and the files are protected from being edited in the wrong place. React picks up the overwriting of the files in destination as file edits and refreshes them as you'd hope.

npm install --save-dev watch

This is what my script look like:

  "scripts": {
    "watch-shared": "watch \"rm -rf src/shared && cp -a ../other-project/shared/ src/\" ../other-project/shared src/shared --ext ts",
    "sync-shared": "rm -rf src/shared && cp -a ../other-project/shared/ src/",
    "start": "run-p watch-shared start-react",
    "start-react": "react-scripts start",
    "build": "run-s sync-shared build-react",
    "build-react": "react-scripts build",
    "test": "run-p watch-shared test-react",
    "test-react": "react-scripts test",
    "eject": "react-scripts eject"
  }

run-s and run-p come from npm install --save-dev npm-run-all

I don't recommend trying to re-use the "sync-shared" script from "watch-shared", because running a node script in from watch may restart everything.

[raymi]

On linux-like systems you might also be able to use mount --bind instead of copy & watch to link your shared directory.