[Bug?]: Couldn't allocate enough memory on Windows x86

[Brooooooklyn]

Self-service

• I'd be willing to implement a fix

Describe the bug

Couldn't allocate enough memory during installation

To reproduce

https://github.com/Brooooooklyn/Image/runs/4786431120?check_suite_focus=true

Environment

Os: Windows 32 bit
Node: 16.13.2

Additional context

No response

[Brooooooklyn]

https://github.com/napi-rs/napi-rs/runs/4919807837?check_suite_focus=true very easy to reproduce

[julianhille]

im having the exact same issue.

[natew]

I'm getting this issue using UTM/windows arm64 on a mac m1. Could we get the upholded label. I've tried upping memory up to 14GB but something tells me it's not the VM memory but perhaps shell or node configuration, because it happens very early in the install process.

Reproduction steps:

npx create-tamagui-app@latest
cd my-app
yarn

[natew]

Tried going all the way up to node --max-old-space-size=9900 C:\Users\n8\AppData\Roaming\npm\node_modules\yarn\bin\yarn.js but still get the errors, so thinking this isn't actually a system OOM thing.

Tried on VMWare and Parallels as well, same thing. Haven't tried x86 yet.

[Kuinox]

Hello, I'm hitting this issue on Appveyors VMs which has node 18.12.0 (x86).
I can't reproduce the issue locally, but I made a minimal reproduction using appveyor:

https://ci.appveyor.com/project/Kuinox/appveyor-yarn-crash/builds/45458965

Build started
git clone -q --branch=main https://github.com/Kuinox/appveyor-yarn-crash.git C:\projects\appveyor-yarn-crash
git checkout -qf 41c3ac79c0a03c933a1cc12dc22835e53e17ef9d
node .yarn/releases/yarn-3.2.2.cjs
➤ YN0000: ┌ Resolution step
➤ YN0000: └ Completed
➤ YN0000: ┌ Fetch step
➤ YN0001: │ Error: Couldn't allocate enough memory
    at ZipFS.allocateBuffer ([worker eval]:1:42071)
    at ZipFS.allocateSource ([worker eval]:1:42507)
    at ZipFS.setFileSource ([worker eval]:1:42781)
    at ZipFS.writeFileSync ([worker eval]:1:47716)
    at extractArchiveTo ([worker eval]:1:463001)
    at async MessagePort.<anonymous> ([worker eval]:1:464372)
➤ YN0013: │ No packages were cached - 11 packages had to be fetched
➤ YN0000: └ Completed in 5s 109ms
➤ YN0000: Failed with errors in 5s 215ms
Command exited with code 1

As you can see, I'm not doing anything weird:
https://github.com/Kuinox/appveyor-yarn-crash
The crash is not 100% reproductible on this minimal reproduction, but it happens every single time on my regular projects (I suspect adding more dependencies will increase the chance of the bug being triggered).

[Kuinox]

I just reproduced it in a Windows 11 Hyper-V VM on my desktop.
I reproduced it with Node 18.2.1 (x86), but I can't reproduce it in Node 19 (x86).

Edit:
I suspect something very bad is happening. I stopped to be able to reproduce the issue, so I restarted the VM, and now it fails in a very different way:

Edit 2:
Cleaning the cache helps reproducing the issue more reliably.

With this new knowledge, I managed to reproduce without any VM, simply by cleaning the cache (cache clean --all) before running the restore with node 18 (x86).

[bcrosnier]

GitHub Actions also fails with "Couldn't allocate enough memory" when using yarn on Node x86 (which is the default on AppVeyor).

x64 works as expected.

[julianhille]

i never could fix it although digged very deep into the source of yarn. We started having it when using more and more workspace packages.
And then i just switched back to npm, it was a day of pain by changing all the scripts but worth it.

Whoever has that issue locally, look at the process monitor and search for the node one.
I witnessed some sort of recursive process calling.
1 node process with 16 MB, having a child yarn with 98 MB ram, having a child with another 67MB and then start all over at 16MB -> 98MB -> 67 MB until infinity.

[cxcorp]

I'm able to reproduce this every time by using docker buildx build --platform linux/arm64 or linux/arm/v7. Something's up with the ZipFS (maybe wasm on ARM?).

As a workaround, I tried reducing the maxOpenFiles for every ZipOpenFs instance, which worked. The build didn't run out of memory, but oh boy was it slow. Only modifying the node-modules linker's maxOpenFiles also worked.

So to make it work, I just searched and replaced every maxOpenFiles:80 in .yarn/releases/yarn-3.6.0.cjs to maxOpenFiles:4 or maxOpenFiles:10 or something low until it worked.

[ltfschoen]

I'm able to reproduce this every time by using docker buildx build --platform linux/arm64 or linux/arm/v7. Something's up with the ZipFS (maybe wasm on ARM?).

As a workaround, I tried reducing the maxOpenFiles for every ZipOpenFs instance, which worked. The build didn't run out of memory, but oh boy was it slow. Only modifying the node-modules linker's maxOpenFiles also worked.

So to make it work, I just searched and replaced every maxOpenFiles:80 in .yarn/releases/yarn-3.6.0.cjs to maxOpenFiles:4 or maxOpenFiles:10 or something low until it worked.

i'm getting the same error Couldn't allocate enough memory in a fresh Ubuntu 23.04 Docker container using aarch64 and Node.js 20.6.1 and Yarn 3.6.3.

i'm using a Macbook Pro Max M2 and already configured Docker for Mac Settings to use up to 40 out of my 64Gb of memory

i tried your solution by running

sed -i 's/maxOpenFiles:80/maxOpenFiles:10/g' .yarn/releases/yarn-3.6.3.cjs

then tried installing dependencies again with the following

time yarn install --network-timeout 1000000000

then i still get error

➤ YN0001: │ Error: Couldn't allocate enough memory
    at ZipFS.allocateBuffer ([worker eval]:1:40610)
    at ZipFS.allocateSource ([worker eval]:1:41045)
    at ZipFS.setFileSource ([worker eval]:1:41318)
    at ZipFS.writeFileSync ([worker eval]:1:46409)
    at extractArchiveTo ([worker eval]:1:466938)
    at async MessagePort.<anonymous> ([worker eval]:1:468587)
➤ YN0000: └ Completed in 16s 266ms
➤ YN0000: Failed with errors in 45s 772ms

real	0m46.520s
user	0m22.146s
sys	0m7.962s

so i changed maxOpenFiles from 10 to 4 by running

sed -i 's/maxOpenFiles:10/maxOpenFiles:4/g' .yarn/releases/yarn-3.6.3.cjs

then tried installing dependencies again with the following again

time yarn install --network-timeout 1000000000

and got same error

➤ YN0001: │ Error: Couldn't allocate enough memory
    at ZipFS.allocateBuffer ([worker eval]:1:40610)
    at ZipFS.allocateSource ([worker eval]:1:41045)
    at ZipFS.setFileSource ([worker eval]:1:41318)
    at ZipFS.writeFileSync ([worker eval]:1:46409)
    at extractArchiveTo ([worker eval]:1:466938)
    at async MessagePort.<anonymous> ([worker eval]:1:468587)
➤ YN0000: └ Completed in 15s 289ms
➤ YN0000: Failed with errors in 42s 942ms

real	0m43.719s
user	0m21.505s
sys	0m7.707s

so lastly i changed maxOpenFiles from 4 to 1 by running the following and checking the file had changed

sed -i 's/maxOpenFiles:4/maxOpenFiles:1/g' .yarn/releases/yarn-3.6.3.cjs

then tried installing dependencies again with the following again

time yarn install --network-timeout 1000000000

and got same error

➤ YN0001: │ Error: Couldn't allocate enough memory
    at ZipFS.allocateBuffer ([worker eval]:1:40610)
    at ZipFS.allocateSource ([worker eval]:1:41045)
    at ZipFS.setFileSource ([worker eval]:1:41318)
    at ZipFS.writeFileSync ([worker eval]:1:46409)
    at extractArchiveTo ([worker eval]:1:466938)
    at async MessagePort.<anonymous> ([worker eval]:1:468587)
➤ YN0000: └ Completed in 14s 861ms
➤ YN0000: Failed with errors in 42s 292ms

real	0m43.046s
user	0m21.005s
sys	0m6.910s

[FezVrasta]

This is happening to me on yarn 4.2.1, on macOS, with the following yarnrc:

nodeLinker: pnp

supportedArchitectures:
  cpu:
    - x86
    - x64
    - arm64
  libc:
    - current
    - glibc
    - musl
  os:
    - darwin
    - linux
    - win32

If I remove the musl libc entry it works, but I need that for Docker.

➤ YN0001: │ Error: Failed to open the cache entry for @next/swc-linux-x64-musl@npm:14.2.3: Couldn't allocate enough memory

Has anyone ever found a solution?

[tomisykora]

I also had issue with not enough memory on macOS 14.5 with node 20.15.1 and yarn 4.3.1:

➤ YN0001: │ Error: Couldn't allocate enough memory
    at ZipFS.allocateBuffer ([worker eval]:1:40610)
    at ZipFS.allocateSource ([worker eval]:1:41045)
    at ZipFS.setFileSource ([worker eval]:1:41318)
    at ZipFS.writeFileSync ([worker eval]:1:46409)
    at extractArchiveTo ([worker eval]:1:466938)
    at async MessagePort.<anonymous> ([worker eval]:1:468587)

I was able to mitigate the issue by using yarn 3.8.3 and by adding this to my .zprofile:

export NODE_OPTIONS="--max-old-space-size=4096"

[ergsolo]

It happens not only in windows and macos, but on linux too
#3580