thread 'tokio-runtime-worker' has overflowed its stack

[jonmmease]

Checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pixi, using pixi --version.

Reproducible example

It's been a few days since I last updated a package with pixi for the VegaFusion repository, but when I tried it today I started getting this error.

Check out VegaFusion repo, then attempt to update rust to 1.72.0:

pixi add rust=1.72.0

jonmmease@Hex-Jon-Mease vegafusion % pixi add rust=1.72.0
⠒ fetching latest repodata
  ✔ conda-forge/osx-arm6 [00:00:00] Using cache
    conda-forge/osx-64   [00:00:02] [────────────────────]      0 B @ 0 B/s   
    conda-forge/linux-64 [00:00:02] [────────────────────]      0 B @ 0 B/s   
    conda-forge/win-64   [00:00:02] [━━━━━───────────────] 4.23 MiB @ 1.50 MiB/s
  ✔ conda-forge/noarch   [00:00:00] Using cache                                                                                                                           
thread 'tokio-runtime-worker' has overflowed its stack
fatal runtime error: stack overflow
zsh: abort      pixi add rust=1.72.0

It always happens when the conda-forge/win-64 bar get's to about 25%. I wonder if there's something that's changed about the conda-forge windows repo metadata recently.

This is on an Apple M1 Pro with macOS 13.5.2 (22G91)

Issue description

above

Expected behavior

No crash

[baszalmstra]

Thanks for the report! I will take a look at this asap!

[baszalmstra]

I traced the issue to the json_patch crate that rattler uses to apply JLAP patches when fetching repodata. The crate uses a lot of recursion which leads to this issue.

I created a PR in the crate to hopefully fix the issue:

idubrov/json-patch#28

[jonmmease]

Thanks for digging into this and making the upstream fix. Depending on the timeline for json-patch to accept the PR and release an update, I also wanted to point out that it's possible to increase the tokio thread stack size in case we need that as a band-aid in the meantime. See:

https://github.com/hex-inc/vegafusion/blob/2ed4925ec536531d9bb344dab2f017405433265c/vegafusion-runtime/src/tokio_runtime.rs#L9

[tony-p]

@jonmmease A temporary fix for some of us also seems to be clearing the cache.

So pixi info and remove the given cache directory

[jonmmease]

Clearing the cache got me unblocked. Thanks!

[baszalmstra]

pixi 0.3.0 which will be released very soon includes the json-patch fix.

[vsoch]

I'm not sure if this is the same thing, but this error starting appearing in my container build just now:

Apologies for the screen shot, I can't copy the text with buildx running! I wound up needing to reinstall pixi, but then it didn't like this block:

[system-requirements]
unix = true

I tried changing to linux = true per the instruction, it wanted a string (wasn't sure what) so I deleted it entirely, and that seems to be building.

[wolfv]

Hmm your error looks more like a network (or disc space?) issue, unfortunately. Could that be?

[vsoch]

I don't think so - I tried it 3-4 times and it reproduced, and it fixed when I re-installed pixi in the container. For context, this was a previous built (the environment was ready to go) and I was updating the environment.

[ruben-arts]

We removed unix from the system requirements.

[vsoch]

We removed unix from the system requirements.

Ha, yes I grepped that 😉