You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
To initialize the runtime we have a bunch of scripts triggered by busybox init or by compute_ctl. Failures in the initialization might go unnoticed and trigger other initialization steps to fail.
TLDR: an incompatible tunefs binary failed as a part of the mounts.sh; script terminated early, aborting next mounts steps. This went unnoticed, until the compute failed on a workload.
DoD
Report the failures through alerts/early exit.
Make steps as independent as possible - failure in one feature should not fail other features.
Implementation ideas
We can use neonvm-daemon to do all runtime bootstraping, thus controlling every step and reporting failures/failing healthcheck.
The text was updated successfully, but these errors were encountered:
Problem description / Motivation
To initialize the runtime we have a bunch of scripts triggered by busybox init or by compute_ctl. Failures in the initialization might go unnoticed and trigger other initialization steps to fail.
This issue is a follow-up for this investigation: https://neondb.slack.com/archives/C06SJG60FRB/p1727851630611959
TLDR: an incompatible
tunefs
binary failed as a part of themounts.sh
; script terminated early, aborting next mounts steps. This went unnoticed, until the compute failed on a workload.DoD
Implementation ideas
We can use neonvm-daemon to do all runtime bootstraping, thus controlling every step and reporting failures/failing healthcheck.
The text was updated successfully, but these errors were encountered: