How to ignore a folder when running "next build"

[andy-asi]

When we run "next build" in our production server, it takes +20mins for the process to finish.
The problem for this delay is that we have a folder in the public/ directory called "uploads" and it has about 30,000 folders and files. This is our project hierarchy:

• ... other folders
• public/
  • ... other folders
  • uploads/ #### << here
    • ... (huge number of files)
• pages/

is there a way to prevent the build command from going through each folder and file inside the uploads folder because they are only used for serving static pictures and assets.

maybe adding an argument inside "next.config.js" to exclude specific folders?

thank you for your time!

[andy-asi]

does anyone have a solution for this?

[andy-asi]

still waiting for a solution

[janus-reith]

While I aggree such a feature could help migitating this, I guess it would be better to solve the underlying issues for now.
Why do you run "next build" on the production server from the very same folder things are served from?
Also, how are you doing version history with that - Is that folder simply ignored from your version history and you pull in changes?
I would try to separate that out somehow.

One thing you could do is switch to docker, and preferabbly, build the image which already contains the output of next build somewhere outside of that production server. Then you could mount that public folder as a docker volume.

[jaysinghcp]

facing same issue

[jguffey]

In the past I've done something like host user uploads in a AWS S3 bucket, instead of allowing users to upload assets to my source code. I think this is a safer approach and decouples user-uploaded files from the production server. I've only done this in Python, (Using this module) but I'm sure there's a parallel module in Node.

Good luck!

[protometa]

I also need to be able to tell Next to ignore certain files in the build process.

[timneutkens]

This issue was fixed on next@canary, please try it out.

[iKettles]

@timneutkens What exactly was fixed? Is there now a way to exclude files/folders from being included in production builds?

[timneutkens]

There was a bug where we would traverse the whole public directory to check if files conflict with routes in pages, this was fixed.

[iKettles]

Great, thanks for clearing that up.

[eric-burel]

We still have a similar issue with any code colocated with sources, eg __test__, storybook stories, jest tests etc. The bugfix answers the precise question but not the question from the title.

For reference I need that to be able to remove test dependencies from my build and have slimmer docker images.

[rtrembecky]

I think I'm talking about the different part of this problem. Collocating tests and stories with pages is one thing, I'm actually using pageExtensions now as well.
The other thing is to prevent next.js build from even looking at the test and story files - At first, I was scared they might be included in the build, but I guess that's not the case. However, the first step of the build by default is typecheck, so I think whether it could be noticeably optimized by excluding some files explicitly in tsconfig.build.json.

[OzzieOrca]

Oh, gotcha. I didn't even think about that and am probably blindly type checking everything... Are you sure it's running typechecks on those files? Guess that'd be easy to test by breaking some type. I'd be curious to hear what other people are doing. Your multiple tsconfig.json files sounds like a valid solution... Might not be pretty.

[rtrembecky]

Just checked it and verified - I put a broken ts file in the cypress/ folder and ran next build two times:

• with tsconfigPath: "tsconfig.build.json", in next.config.js and having "exclude": ["cypress"] in that tsconfig - typecheck was successful ✅
• without tsconfigPath in next.config.js - typecheck failed on the broken file ❌

[rtrembecky]

So it seems if you're not interested in typechecking tests/stories/config files during next build, the tsconfigPath is the way to go.

[eric-burel]

Nice to see tsconfigPath, it's a long awaited feature! For typechecking everything I go with a tsc as a CI step, it's a better pattern than typechecking during build.

[ctjlewis]

My workaround for this was adding the redundant folder to exclude in tsconfig.json, or overriding the include field to include only the directories you want.

You can still have your IDE type-check the directories you don't want Next to check by adding a local tsconfig:

.
├── nextDontCompile
│   ├── index.tsx
│   └── tsconfig.json (local)
├── src
│   ├── index.tsx
│   └── utils.ts
└── tsconfig.json (Next)

This seems messy, but it is ultimately what you want here: Your Firebase functions or whatever are really a separate project that are not part of the root-level Next project, and so it makes sense to ignore them.

You can also extend the root-level config to prevent having to duplicate your settings:

# ./nextDontCompile/tsconfig.json

{
  "extends": "../tsconfig.json",
  ...
}

[artooras]

I would like to second a need for an option to ignore a directory when running next build. I tried following the above recommendations of simply reorganizing the project, however, this sometimes causes other issues. For instance, my Next.js project uses a number of Firebase services - auth, database, functions - along with emulators. If I move the functions folder outside of the root directory, emulators don't work properly anymore. And yes, it's likely possible to reconfigure the emulators to work even with such a setup, but it's not a mainstream use case, it certainly isn't well (if at all) documented, and would cost me a day or more researching, experimenting, etc. It would just be nice to just have an option for next build to ignore some directories in root and thus allow developers more flexibility in organizing their projects.

[ctjlewis]

@eric-burel You could create a local tsconfig in that directory, right? So that VS Code knows to type-check them for you, but also they're not tied into the root-level configuration, so Next won't touch them. See my example from above:

.
├── nextDontCompile
│   ├── index.tsx
│   └── tsconfig.json (local)
├── src
│   ├── index.tsx
│   └── utils.ts
└── tsconfig.json (Next)

I've been working in this part of the codebase recently and it literally just runs the equivalent of tsc --noEmit in the root directory, so if you tell TypeScript you don't want to include those things in the root project, you can avoid type-checking them.

Does this sound right? It's clear that there needs to be a help article or something on this at the very least and probably logic changes, just thinking out loud so I can consider the best way to approach this.

[eric-burel]

Yeah I must I haven't digged folders outside of src much, as my issue is more with collocated files (tests, stories), so multiple tsconfig should do it. It's quite hard to debug, but 2 tsconfig might do it in your case. tsc --listFiles is the way to go to test this.

[ctjlewis]

Agree. I have made the argument to the Next team that src/ structure should be the default: This would simplify everything greatly by relying on the rootDir field. Currently in draft at #27667.

I will spend a few hours trying to write docs or otherwise make it easier for users to understand how to approach this, since it's not very intuitive at all.

Also IIRC, the logic excludes test/ by default, as one of the few TS overrides.

[artooras]

Thanks for the suggestion @ctjlewis , this did indeed solve my issue. In my case it was also OK to exclude in the root's tsconfig as I was excluding the functions folder (which has it's own tsconfig and compiles locally anyway) and folders like tests, emulator-data, cypress, etc.

[ctjlewis]

cc @timneutkens—will send in docs to make approaching this clearer, see also draft #27667.

[coltenkrauter]

How can I ignore a folder when running next build? I didn't see an answer above, did I miss something?

[Xananax]

In theory, this should work... But it doesn't:

1. add it to the ignore section of tsconfig.json:

   ...
     "exclude": [
       "node_modules",
       "myDir"
   ]

   potentially also add a new tsconfig.json with and "extends": "../tsconfig.json" in the ignored directory
2. Add the following to next.config.js:

//@ts-check

const rulesToProcess = [/\.m?js/, /\.(js|cjs|mjs)$/].map(String);
const dirToIgnore = /myDir/;

/** @type {import('next').NextConfig} */
const nextConfig = {
  reactStrictMode: true,
  pageExtensions: ["js", "jsx", "ts", "tsx", "md", "mdx"],
  ...,
  /**
   *
   * @param {import('webpack').Configuration} config
   * @param {import('next/dist/server/config-shared').WebpackConfigContext} context
   * @returns {import('webpack').Configuration}
   */
  webpack: (config) => {
    config.module.rules = config.module.rules.map((rule) => {
      if (rule !== "..." && rulesToProcess.indexOf(String(rule.test)) > -1) {
        rule.exclude = [dirToIgnore];
      }
      return rule;
    });
    return config;
  },
};

export default nextConfig;

In my case, I am using next.config.mjs and I was trying to suppress the warning (#30870, #33693, #27650). Ignoring the directory was convenient for me as it only contains build files, but alas I wasn't able to make it work.
I worked around the issue by suppressing webpack warnings, as described in #30870 (comment), but I'd rather do it more cleanly.

Still, I leave this here in the hopes it puts someone on some kind of correct path.

[luixo]

Made a poor man's patch to implement the feature of ignoring pages on build:

diff --git a/dist/build/entries.js b/dist/build/entries.js
index a455fef61988ef553069946b1138ca8dec7e9918..12d4682124c8a089d7b98cfbe77b39a2e4a53c92 100644
--- a/dist/build/entries.js
+++ b/dist/build/entries.js
@@ -153,7 +153,7 @@ function getPageFilePath({ absolutePagePath , pagesDir , appDir , rootDir  }) {
     }
     return require.resolve(absolutePagePath);
 }
-function createPagesMapping({ isDev , pageExtensions , pagePaths , pagesType , pagesDir  }) {
+function createPagesMapping({ isDev , pageExtensions , ignorePages , pagePaths , pagesType , pagesDir  }) {
     const isAppRoute = pagesType === "app";
     const previousPages = {};
     const pages = pagePaths.reduce((result, pagePath)=>{
@@ -161,6 +161,10 @@ function createPagesMapping({ isDev , pageExtensions , pagePaths , pagesType , p
         if (pagePath.endsWith(".d.ts") && pageExtensions.includes("ts")) {
             return result;
         }
+        const ignorePagesArray = Array.isArray(ignorePages) ? ignorePages : ignorePages ? [ignorePages] : [];
+        if (ignorePagesArray.some((regex) => regex.test(pagePath))) {
+            return result;
+        }
         let pageKey = getPageFromPath(pagePath, pageExtensions);
         if (isAppRoute) {
             pageKey = pageKey.replace(/%5F/g, "_");
diff --git a/dist/build/index.js b/dist/build/index.js
index 667a9efe1a4c24ba087a5d7a5ac8e8e6fb15ca10..ad1d5fd36d7732b229a8fa67110ea936870736dd 100644
--- a/dist/build/index.js
+++ b/dist/build/index.js
@@ -275,6 +275,7 @@ async function build(dir, reactProductionProfiling = false, debugOutput = false,
             const mappedPages = nextBuildSpan.traceChild("create-pages-mapping").traceFn(()=>(0, _entries.createPagesMapping)({
                     isDev: false,
                     pageExtensions: config.pageExtensions,
+                    ignorePages: config.ignorePages,
                     pagesType: "pages",
                     pagePaths: pagesPaths,
                     pagesDir
diff --git a/dist/server/config.js b/dist/server/config.js
index e9232bcb5dc81256f80ca82076384e68d009301a..2aa9c55635fe0f424cd195b64c99615658a55ddd 100644
--- a/dist/server/config.js
+++ b/dist/server/config.js
@@ -172,6 +172,21 @@ function assignDefaults(dir, userConfig, silent = false) {
                 }
             });
         }
+        if (key === "ignorePages") {
+            if (Array.isArray(value)) {
+                if (!value.length) {
+                    throw new Error(`Specified ignorePages is an empty array. Please update it with the relevant RegExps or remove it.`);
+                }
+                value.forEach((regex)=>{
+                    if (!(regex instanceof RegExp)) {
+                        throw new Error(`Specified ignorePages is not an array of RegExps, found "${regex}" of type "${typeof regex}". Please update this config or remove it.`);
+                    }
+                });
+            } else if (!(value instanceof RegExp)) {
+                throw new Error(`Specified ignorePages is neither an array nor a RegExp, found "${value}". Please update this config or remove it.`);
+            }
+            
+        }
         if (!!value && value.constructor === Object) {
             currentConfig[key] = {
                 ..._configshared.defaultConfig[key],

Also, you'll have to dig into dist/next-config-validate.js which is compiled beforehand for some reason, find pageExtensions and insert before it something like ignorePages:{anyOf: [{instanceof: "RegExp"},{minItems:1,type:"array"}]} (didn't include the patch here as it's replacing the whole one-line file).

This patch doesn't work in dev environment, you will still be able to open an ignored page while developing.

The ignorePages option accept RegExp or an array of RegExps.
I personally use it to ignore tests that are co-located with pages: ignorePages: /\.spec\.tsx?$/

[luixo]

An updated patch for Next.js 14.0.3 (also I switched to string | string[] in ignorePages type as it seems like maintainers prefer to keep it serializable).
Also, no need to lookup prebuilt next-config-validate.js anymore.

My config prop now looks like this: ignorePages: "\\.spec\\.tsx?$".

# see https://github.com/vercel/next.js/discussions/11113#discussioncomment-7083801
diff --git a/dist/build/entries.js b/dist/build/entries.js
index c759abdd2419661dd658c0f6741eca30e8000042..ce25d5a6e4a098391ff86d019e4b58dd47f41448 100644
--- a/dist/build/entries.js
+++ b/dist/build/entries.js
@@ -177,7 +177,7 @@ function getPageFilePath({ absolutePagePath, pagesDir, appDir, rootDir }) {
     }
     return require.resolve(absolutePagePath);
 }
-function createPagesMapping({ isDev, pageExtensions, pagePaths, pagesType, pagesDir }) {
+function createPagesMapping({ isDev, pageExtensions, ignorePages, pagePaths, pagesType, pagesDir }) {
     const isAppRoute = pagesType === "app";
     const previousPages = {};
     const pages = pagePaths.reduce((result, pagePath)=>{
@@ -185,6 +185,10 @@ function createPagesMapping({ isDev, pageExtensions, pagePaths, pagesType, pages
         if (pagePath.endsWith(".d.ts") && pageExtensions.includes("ts")) {
             return result;
         }
+        const ignorePagesArray = Array.isArray(ignorePages) ? ignorePages : ignorePages ? [ignorePages] : [];
+        if (ignorePagesArray.some((regex) => new RegExp(regex).test(pagePath))) {
+            return result;
+        }
         let pageKey = getPageFromPath(pagePath, pageExtensions);
         if (isAppRoute) {
             pageKey = pageKey.replace(/%5F/g, "_");
diff --git a/dist/build/index.js b/dist/build/index.js
index 353eaa712e453b4b8185cf8b9bf799a36194c6fe..04ce0b11247dc691711643c2a641fbbe65d6aa03 100644
--- a/dist/build/index.js
+++ b/dist/build/index.js
@@ -329,6 +329,7 @@ async function build(dir, reactProductionProfiling = false, debugOutput = false,
             const mappedPages = nextBuildSpan.traceChild("create-pages-mapping").traceFn(()=>(0, _entries.createPagesMapping)({
                     isDev: false,
                     pageExtensions: config.pageExtensions,
+                    ignorePages: config.ignorePages,
                     pagesType: "pages",
                     pagePaths: pagesPaths,
                     pagesDir
diff --git a/dist/server/config-schema.js b/dist/server/config-schema.js
index dae2dec14daae7feb32329109104a54574783a30..f21da9c215e33f78e0cdd4e0bb34c094f3d2962a 100644
--- a/dist/server/config-schema.js
+++ b/dist/server/config-schema.js
@@ -407,6 +407,10 @@ const configSchema = _zod.z.lazy(()=>_zod.z.strictObject({
             "export"
         ]).optional(),
         outputFileTracing: _zod.z.boolean().optional(),
+        ignorePages: _zod.z.union([
+            _zod.z.string(),
+            _zod.z.array(_zod.z.string()).min(1)
+        ]).optional(),
         pageExtensions: _zod.z.array(_zod.z.string()).min(1).optional(),
         poweredByHeader: _zod.z.boolean().optional(),
         productionBrowserSourceMaps: _zod.z.boolean().optional(),
diff --git a/dist/server/config-shared.d.ts b/dist/server/config-shared.d.ts
index 2d93f521495a5c24478714a19d8089487d106b9b..21051445d0937006572701ea70277909e590b175 100644
--- a/dist/server/config-shared.d.ts
+++ b/dist/server/config-shared.d.ts
@@ -488,6 +488,11 @@ export interface NextConfig extends Record<string, any> {
      *
      */
     outputFileTracing?: boolean;
+    /**
+     * Regex that define pages to ignore while building
+     *
+     */
+    ignorePages?: string | string[];
     /**
      * Timeout after waiting to generate static pages in seconds
      *
diff --git a/dist/server/config.js b/dist/server/config.js
index 541fa7863a2017619bce7d11a40de9cd7f017310..fbde5170cd49c391e2003732d17810a4d6eb2006 100644
--- a/dist/server/config.js
+++ b/dist/server/config.js
@@ -234,6 +234,21 @@ function assignDefaults(dir, userConfig, silent) {
                 }
             });
         }
+        if (key === "ignorePages") {
+            if (Array.isArray(value)) {
+                if (!value.length) {
+                    throw new Error(`Specified ignorePages is an empty array. Please update it with the relevant strings or remove it.`);
+                }
+                value.forEach((element)=>{
+                    if (typeof element !== 'string') {
+                        throw new Error(`Specified ignorePages is not an array of string, found "${element}" of type "${typeof element}". Please update this config or remove it.`);
+                    }
+                });
+            } else if (typeof value !== 'string') {
+                throw new Error(`Specified ignorePages is neither an array nor a string, found "${value}" of type "${typeof value}". Please update this config or remove it.`);
+            }
+            
+        }
         if (!!value && value.constructor === Object) {
             currentConfig[key] = {
                 ..._configshared.defaultConfig[key],

[JasonSimms]

Bump for visibility i still dont see this working with next.config.js exclude. ( "next": "14.2.5" )

[Virous77]

@JasonSimms if you are using typescript just add folder name in tsconfig.json - "exclude": ["node_modules", "not-build"] like that.

[volfcan]

doesn't work

[kristian-nst]

Why would i want to exclude files (while im coding?)
We just wanna exclude it when building.