Devnet RPC node crashed with repeated unsafe unwraps

[danpaul000]

Problem

Devnet RPC node went down 3 times due to unsafe unwraps then appeared to stop re-starting itself. Appears to be working now after I manually stopped it, pulled over a newer snapshot, then restarted.

journalctl_first_crash.txt

syslog_unwrap_logs.txt

[mvines]

It's failing to create threads, which is so weird:

solana/ledger/src/sigverify_shreds.rs

Line 32 in 01697a9

.unwrap();

Nothing obvious in dmesg or syslog about this. I've rebooted the machine 🤞

[danpaul000]

Looks like it's still failing in a similar manner, and in a service restart cycle right now

[mvines]

Debugged this down to #9383, with #9384 as a temporary workaround