Next 13 fetch in server component stuck forever if response is larger than ~15kb

[Escapado]

Verify canary release

• I verified that the issue exists in the latest Next.js canary release

Provide environment information

Operating System:
  Platform: linux
  Arch: x64
  Version: Ubuntu 20.04.0 LTS Wed Oct 26 2022 14:44:18 GMT+0200 (Central European Summer Time)
Binaries:
  Node: 16.14.2
  npm: 7.17.0
  Yarn: 1.22.19
  pnpm: 7.13.6
Relevant packages:
  next: 13.0.0
  eslint-config-next: N/A
  react: 18.2.0
  react-dom: 18.2.0

What browser are you using? (if relevant)

No response

How are you deploying your application? (if relevant)

No response

Describe the Bug

When fetching data as described in the docs, the fetch function gets stuck and never resolves, if the payload reaches a certain size.

I created a minimal stackblitz and a mock api to reproduce the problem. If I fetch 75 posts from the mock api it will never resolve, if I only fetch 50 it works fine. Since I was able to reproduce this problem with multiple mock apis and even when using something like supabase to fetch some data I think it might be a size issue? Here the difference is 14kb vs 20kb for the two requests.

EDIT:
I tried to nail the problem down further. fetch does indeed resolve but .json() does not. Does this have to do with the highWaterMark buffer size limit on nodejs streams?

Expected Behavior

Fetch should not get stuck forever.

Link to reproduction

https://stackblitz.com/edit/nextjs-vv7ryj?file=app/page.tsx

To Reproduce

• Open the stackblitz
• Comment out line 3 and comment in in line 4 in pages.tsx to fetch 75 instead of 50 posts
• See that fetch never resolves

[klarstrup]

Sounds like a reprisal of this type of issue:

• Presence of middleware prevents access to raw request bodies greater than or equal to 16,384 bytes (16 KiB) #39262
• fix(stream): Allows body larger than 16 KiB with middleware #41270

[renomureza]

@Escapado I think line 4 i.e. this API https://apimocha.com/next13/75posts returns an invalid response body for JSON. they don't have an opening array tag "[".

I had the same problem even though the API was returning valid JSON.

But, works fine when I use axios.

[Escapado]

Thanks @renomureza. I fixed that. However the issue still stands it's reproducible if you for example try the following mock apis:
https://jsonplaceholder.typicode.com/posts
https://jsonplaceholder.typicode.com/comments

[renomureza]

@Escapado Instead of using fetch try using axios, it worked for me.

[t3dotgg]

This hurts me deeply

[balazsorban44]

Thanks for reporting, I did some digging, and the issue seems to be node-fetch-specific.

Currently, on the server, Next.js uses node-fetch to polyfill fetch when Node.js <18 is used. Above Node.js 18, or when the experimental: {enableUndici: true} option in next.config.js is present, we will use undici instead, which does not have this problem.

So current workarounds are:

1. Use Node.js 18 or above, if possible
2. Add the following to next.config.js:

module.exports = {
  experimental: {
    enableUndici: true
  }
}

[srini-leanfolks]

using node-fetch solved this

[nowaythatworked]

using node-fetch solved this

What version did you use? As @balazsorban44 mentioned, Next.js also uses node-fetch under the hood. Perhaps they are using a different version?

[srini-leanfolks]

@nowaythatworked node-fetch v3.2.10

[icyJoseph]

#41988 I see now.... the payload on this one is about 300kb

[timneutkens]

Investigated this one on my flight back from Next.js Conf, it's a known issue with cloning responses of fetch() in node-fetch and I could reproduce it with standalone node-fetch as well (outside of Next.js) both in the latest 2.x and 3.x release. Since we're moving to using undici anyway I've made it so that appDir enabled it by default. Note that this does mean that appDir now requires Node.js 16.8 or later.

[icyJoseph]

When deploying to Vercel, on [email protected], I still see this problem.

• The enableUndici flag is still needed, even locally though
• In Vercel, when the flag is active and with [email protected], the build process goes swell, but then at runtime I get timeouts on functions that use fetch within Server Components. API routes work fine. Node version is the highest Vercel allows.

[GET] /pokemon/1
11:01:15:13
Fetching pokemon: 1
Got JSON for specie
2022-11-01T10:01:26.263Z 171204fd-e67e-4df5-8281-033e63eb019f Task timed out after 10.01 seconds

• This is the page
• Actual function

Could the problem be that this function lives at lib/ and not within app/? Locally all works wonderfully though.

Using Node 16.X ~

[ervwalter]

FYI to @timneutkens, I can confirm that this is still broken when deploying to Vercel (same as what @icyJoseph is seeing). Neither manually setting enableUndici nor using [email protected] makes a difference.

Server Components that fetch large payloads hang until the serverless function times out.

I added some debug output and determined that my Vercel deployment is running Node.js v16.16.0.

[swamidass]

Same problem here. Still broken on deploy,but works fine in dev now.

The max version of node on vercel is 16.6. Perhaps now is a time to enable node 18?

[ervwalter]

FYI, I just confirmed that this Vercel deployment aspect is fixed in v13.0.2-canary.1 via #42382

[BWBama85]

I am seeing this warning now but do not have this option configured: #42441

[ijjk]

Hi this should be updated in v13.0.2 please update and give it a try!

[swamidass]

I can confirm it is fixed. Thanks!

[github-actions]

This closed issue has been automatically locked because it had no new activity for a month. If you are running into a similar issue, please create a new issue with the steps to reproduce. Thank you.