Missing webpack dependency when using output standalone + custom server #64031

maximelebastard · 2024-04-03T16:00:26Z

Link to the code that reproduces this issue

https://github.com/maximelebastard/nextjs-issue-custom-server

To Reproduce

The issue is related to using the output: "standalone" with a custom server. See example repository.

Clone the reproducing repository
Run npm install then npm run build and npm run start
When starting the app, an error will be triggered due to a missing webpack file on a webpack require (see the error below)
To see the code triggering this issue, see this commit: maximelebastard/nextjs-issue-custom-server@c47de3a

Current vs. Expected behavior

Expected:

I create a recent NextJS project (14.1.4) using a recent Node version (v20.12)
I want to deploy my app using Docker, therefore I use the output: "standalone" configuration in next.config.js
I want to use some logging middlewares, therefore I use a custom server.js
When I run npm run build and copy my custom server code to the build dir, the app is properly built and the file is properly copied
When requiring next from server.js, all the dependencies are present and the server starts

Actual:

1-4 are the same as expected
.next/standalone/node_modules/next/dist/compiled/webpack/build5.js does not exists in the built app
When requiring next from server.js, I'm getting the error below because webpack tries to require build5.js

> node .next/standalone/custom-server.js

node:internal/modules/cjs/loader:1143
  const err = new Error(message);
              ^

Error: Cannot find module './bundle5'
Require stack:
- /Users/maxime/workspace/issue-bundle5-nextjs/.next/standalone/node_modules/next/dist/compiled/webpack/webpack.js
- /Users/maxime/workspace/issue-bundle5-nextjs/.next/standalone/node_modules/next/dist/server/config-utils.js
- /Users/maxime/workspace/issue-bundle5-nextjs/.next/standalone/node_modules/next/dist/server/config.js
- /Users/maxime/workspace/issue-bundle5-nextjs/.next/standalone/node_modules/next/dist/server/next.js
- /Users/maxime/workspace/issue-bundle5-nextjs/.next/standalone/custom-server.js
    at Module._resolveFilename (node:internal/modules/cjs/loader:1143:15)
    at /Users/maxime/workspace/issue-bundle5-nextjs/.next/standalone/node_modules/next/dist/server/require-hook.js:55:36
    at Module._load (node:internal/modules/cjs/loader:984:27)
    at Module.require (node:internal/modules/cjs/loader:1231:19)
    at mod.require (/Users/maxime/workspace/issue-bundle5-nextjs/.next/standalone/node_modules/next/dist/server/require-hook.js:65:28)
    at require (node:internal/modules/helpers:179:18)
    at exports.init (/Users/maxime/workspace/issue-bundle5-nextjs/.next/standalone/node_modules/next/dist/compiled/webpack/webpack.js:24:28)
    at loadWebpackHook (/Users/maxime/workspace/issue-bundle5-nextjs/.next/standalone/node_modules/next/dist/server/config-utils.js:18:5)
    at loadConfig (/Users/maxime/workspace/issue-bundle5-nextjs/.next/standalone/node_modules/next/dist/server/config.js:642:46)
    at initialize (/Users/maxime/workspace/issue-bundle5-nextjs/.next/standalone/node_modules/next/dist/server/lib/router-server.js:55:46) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/Users/maxime/workspace/issue-bundle5-nextjs/.next/standalone/node_modules/next/dist/compiled/webpack/webpack.js',
    '/Users/maxime/workspace/issue-bundle5-nextjs/.next/standalone/node_modules/next/dist/server/config-utils.js',
    '/Users/maxime/workspace/issue-bundle5-nextjs/.next/standalone/node_modules/next/dist/server/config.js',
    '/Users/maxime/workspace/issue-bundle5-nextjs/.next/standalone/node_modules/next/dist/server/next.js',
    '/Users/maxime/workspace/issue-bundle5-nextjs/.next/standalone/custom-server.js'
  ]
}

Provide environment information

Operating System:
  Platform: darwin
  Arch: arm64
  Version: Darwin Kernel Version 23.1.0: Mon Oct  9 21:27:24 PDT 2023; root:xnu-10002.41.9~6/RELEASE_ARM64_T6000
Binaries:
  Node: 20.12.0
  npm: 10.5.0
  Yarn: N/A
  pnpm: N/A
Relevant Packages:
  next: 14.1.4
  eslint-config-next: N/A
  react: 18.2.0
  react-dom: 18.2.0
  typescript: N/A
Next.js Config:
  output: standalone

Which area(s) are affected? (Select all that apply)

Standalone mode (output: "standalone")

Which stage(s) are affected? (Select all that apply)

next start (local)

Additional context

A workaround I found is to disable standalone output.
That probably disables tree shaking and relies on the complete node_modules directory.
A big drawback is that it makes a heavy docker image.

The text was updated successfully, but these errors were encountered:

clementzhao · 2024-04-26T06:22:12Z

I have same issue. And I found the server.js which compiled with standalone can works. So I just copy these 2lines, and my custom server.js also works fine.

const nextConfig = {which from compiled server.js}
process.env.__NEXT_PRIVATE_STANDALONE_CONFIG = JSON.stringify(nextConfig)

hi-zp · 2024-04-29T02:35:32Z

Here's my code, and while it works successfully, I'm not sure it's going to be a hidden problem

import { createServer } from 'node:http';
import { parse } from 'node:url';
import NextServer from 'next/dist/server/next-server';
// @ts-ignore
import { config } from './.next/required-server-files.json';
import { NextConfig } from 'next';

const port = parseInt(process.env.PORT || '9000', 10);
const hostname = '0.0.0.0';
const dev = process.env.NODE_ENV !== 'production';

const app = new NextServer({
  hostname: 'localhost',
  port,
  dir: process.cwd(),
  dev,
  conf: {
    ...(config as NextConfig),
  },
});

const handle = app.getRequestHandler();

app.prepare().then(() => {
  createServer((req, res) => {
    try {
      const parsedUrl = parse(req.url!, true);
     handle(req, res, parsedUrl);
    } catch (err) {
      console.error('Error occurred handling', req.url, err);
      res.statusCode = 500;
      res.end('internal server error');
    }
  })
    .once('error', (err) => {
      logger.error(err);
      process.exit(1);
    })
    .listen(port, () => {
      console.log(
        `> Server listening at http://${hostname}:${port} as ${
          dev ? 'development' : process.env.NODE_ENV
        }`,
      );
    });
});

wq93 · 2024-04-30T06:45:20Z

I've met, too, frustrating

Node.js v18.20.2
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
yarn run v1.22.19
$ NODE_ENV=production node ./server/main.js
node:internal/modules/cjs/loader:1140
  const err = new Error(message);
              ^

Error: Cannot find module './bundle5'
Require stack:

yunsii · 2024-07-04T03:35:08Z

Already 3 months passed, there is still no any progress? 🫠

nectoscorp · 2024-07-15T15:09:04Z

I confirmed that this works for me: #64031 (comment)
So AWS lambda, I am using the following to extract the JSON:
sed -n 12p .next/standalone/server.js | cut -c 20-

then require it inside my lambda handler file.

If I had more time I will figure out which particular parameter in the config made it work by commenting them out one at a time.

yunsii · 2024-07-16T01:39:00Z

I have same issue. And I found the server.js which compiled with standalone can works. So I just copy these 2lines, and my custom server.js also works fine.
const nextConfig = {which from compiled server.js}
process.env.__NEXT_PRIVATE_STANDALONE_CONFIG = JSON.stringify(nextConfig)

Thanks @nectoscorp

After I tested, it works 😂

nectoscorp · 2024-07-16T02:29:14Z

I wish the nextConfig json was just saved on a file in like .next/standalone/next.config.json so we don't have to resort to these hacks. But obviously, the best is solution is to find out why bundle5 couldn't be found.

Brandawg93 · 2024-09-09T20:12:01Z

Having the same issue. This seems to work well for me to allow for both local dev development and docker deployment:

const dev = process.env.NODE_ENV !== 'production'

if (!dev) {
  // eslint-disable-next-line @typescript-eslint/no-require-imports
  const { config } = require('./.next/required-server-files.json')
  process.env.__NEXT_PRIVATE_STANDALONE_CONFIG = JSON.stringify(config)
}

flawnn · 2024-09-25T09:07:04Z

Having the same issue. This seems to work well for me to allow for both local dev development and docker deployment:

const dev = process.env.NODE_ENV !== 'production'

if (!dev) {
  // eslint-disable-next-line @typescript-eslint/no-require-imports
  const { config } = require('./.next/required-server-files.json')
  process.env.__NEXT_PRIVATE_STANDALONE_CONFIG = JSON.stringify(config)
}

The ES6-conform version:

import("./.next/required-server-files.json")
        .then(({ default: { config } }) => {
        process.env.__NEXT_PRIVATE_STANDALONE_CONFIG = JSON.stringify(config);
        const app = next({ dev });
        return app.prepare().then(() => startServer(app) // startup func);
    })
        .catch((error) => {
        console.error("Failed to load required-server-files.json:", error);
        process.exit(1);
    });

SujithPriyamRajan · 2024-10-04T08:29:46Z

I have the same issue but surprisingly in PM2 as local & VM's was working fine

Can anyone summarize why this issue is ??

/** @type {import('next').NextConfig} */

const nextConfig = {
    output: 'standalone',
    typescript: {
        ignoreBuildErrors: true
    },
    images: { unoptimized: true },
    experimental: {
        serverActions: {
            serverActions: true
        },
        missingSuspenseWithCSRBailout: false
    },
    distDir: 'out',
    rewrites: async () => {
        return [
            {
                source: '/_next/static/:path*',
                destination: '/static/:path*'
            }
        ];
    }
};
export default nextConfig;

aczekajski · 2024-10-09T09:14:46Z

I found a universal way if adding missing dependencies to standalone output that I guess can also be used for custom server.

You can have a look here:
#66327 (comment)

MJez29 · 2024-11-19T18:21:51Z

I also ran into this issue and came up with the exact same fix

I put up a PR to fix this in next.js: #72966

## Fixing a bug ### Related issues fixes #64031 ### Why is this change needed Per the Next.js [docs](https://nextjs.org/docs/pages/api-reference/next-config-js/output) > Next.js' production server is also traced for its needed files and output at .next/next-server.js.nft.json which can be leveraged in production. When I tried doing this my [custom server](https://nextjs.org/docs/pages/building-your-application/configuring/custom-server) would crash on startup because of this codepath https://github.com/vercel/next.js/blob/canary/packages/next/src/server/config.ts#L1022 My first attempt at fixing this involved changing ``` if (!process.env.__NEXT_PRIVATE_STANDALONE_CONFIG) { ``` to ``` if (phase === PHASE_PRODUCTION_SERVER) { ``` but this is not guaranteed to work because the next.config.js and it's required dependencies may not be present causing the server to use the default config The best solution is to mimic standalone mode as closely as possible ### What does this change do This change allows a custom server running in production with only required files to pass in a preloaded next config (most likely from `required-server-files.json`) This is very similar to how the [standalone server works](https://github.com/vercel/next.js/blob/bc49287063d6afd50a7361566b14f1ab6c26b136/packages/next/src/build/utils.ts#L2142) ### Testing I'd like to get an initial round of feedback from the next team before I dive into writing an integration test. I patched nextjs in my repo and this solution worked

maximelebastard added the bug Issue was opened via the bug report template. label Apr 3, 2024

github-actions bot added the Output (export/standalone) Related to the the output option in `next.config.js`. label Apr 3, 2024

maximelebastard changed the title ~~Custom server is not usable anymore in production~~ Missing webpack dependency when using output standalone + custom server Apr 3, 2024

This comment has been minimized.

Sign in to view

MJez29 mentioned this issue Nov 19, 2024

Support custom servers with minimal node modules #72966

Merged

gaojude closed this as completed in #72966 Nov 19, 2024

gaojude reopened this Nov 20, 2024

gaojude mentioned this issue Nov 20, 2024

Revert "Support custom servers with minimal node modules" #73003

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Missing webpack dependency when using output standalone + custom server #64031

Missing webpack dependency when using output standalone + custom server #64031

maximelebastard commented Apr 3, 2024 •

edited

Loading

This comment has been minimized.

clementzhao commented Apr 26, 2024

hi-zp commented Apr 29, 2024

wq93 commented Apr 30, 2024

yunsii commented Jul 4, 2024

This comment has been minimized.

nectoscorp commented Jul 15, 2024 •

edited

Loading

yunsii commented Jul 16, 2024

nectoscorp commented Jul 16, 2024 •

edited

Loading

Brandawg93 commented Sep 9, 2024

flawnn commented Sep 25, 2024

SujithPriyamRajan commented Oct 4, 2024

aczekajski commented Oct 9, 2024

MJez29 commented Nov 19, 2024

Missing webpack dependency when using output standalone + custom server #64031

Missing webpack dependency when using output standalone + custom server #64031

Comments

maximelebastard commented Apr 3, 2024 • edited Loading

Link to the code that reproduces this issue

To Reproduce

Current vs. Expected behavior

Provide environment information

Which area(s) are affected? (Select all that apply)

Which stage(s) are affected? (Select all that apply)

Additional context

This comment has been minimized.

clementzhao commented Apr 26, 2024

hi-zp commented Apr 29, 2024

wq93 commented Apr 30, 2024

yunsii commented Jul 4, 2024

This comment has been minimized.

nectoscorp commented Jul 15, 2024 • edited Loading

yunsii commented Jul 16, 2024

nectoscorp commented Jul 16, 2024 • edited Loading

Brandawg93 commented Sep 9, 2024

flawnn commented Sep 25, 2024

SujithPriyamRajan commented Oct 4, 2024

aczekajski commented Oct 9, 2024

MJez29 commented Nov 19, 2024

maximelebastard commented Apr 3, 2024 •

edited

Loading

nectoscorp commented Jul 15, 2024 •

edited

Loading

nectoscorp commented Jul 16, 2024 •

edited

Loading