Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SIGETRM during shutdown forces shutdown #1623

Closed
dimalit opened this issue Aug 17, 2023 · 3 comments · Fixed by #1695
Closed

SIGETRM during shutdown forces shutdown #1623

dimalit opened this issue Aug 17, 2023 · 3 comments · Fixed by #1695
Assignees
Labels
bug Something isn't working epic:node-rotation
Milestone

Comments

@dimalit
Copy link
Contributor

dimalit commented Aug 17, 2023

When skaled is shutting down by internal cause, it counts this event as though it was external SIGTERM. This leads to forced shutdown if real SIGTERM is received during shutdown procedure. Need not to count internal exit request as SIGTERM.

@PolinaKiporenko PolinaKiporenko added epic:node-rotation bug Something isn't working labels Aug 17, 2023
@PolinaKiporenko PolinaKiporenko added this to the SKALE 2.3 milestone Aug 17, 2023
@DmytroNazarenko DmytroNazarenko moved this from To Do to Ready For Pickup in SKALE Engineering 🚀 Sep 5, 2023
@PolinaKiporenko PolinaKiporenko moved this from Ready For Pickup to In Progress in SKALE Engineering 🚀 Sep 7, 2023
@PolinaKiporenko PolinaKiporenko moved this from In Progress to Ready For Pickup in SKALE Engineering 🚀 Sep 14, 2023
@dimalit dimalit moved this from Ready For Pickup to In Progress in SKALE Engineering 🚀 Oct 16, 2023
@dimalit dimalit linked a pull request Oct 17, 2023 that will close this issue
@dimalit dimalit moved this from In Progress to Blocked in SKALE Engineering 🚀 Oct 19, 2023
@dimalit dimalit moved this from Blocked to Code Review in SKALE Engineering 🚀 Oct 31, 2023
@PolinaKiporenko PolinaKiporenko moved this from Code Review to Ready For Release Candidate in SKALE Engineering 🚀 Nov 22, 2023
@PolinaKiporenko PolinaKiporenko moved this from Ready For Release Candidate to Code Review in SKALE Engineering 🚀 Nov 22, 2023
@github-project-automation github-project-automation bot moved this from Code Review to Ready For Pickup in SKALE Engineering 🚀 Nov 22, 2023
@PolinaKiporenko PolinaKiporenko moved this from Ready For Pickup to Code Review in SKALE Engineering 🚀 Nov 22, 2023
@github-project-automation github-project-automation bot moved this from Code Review to Ready For Release Candidate in SKALE Engineering 🚀 Nov 22, 2023
@DmytroNazarenko
Copy link
Collaborator

skaled: 3.18.0-beta.0

@DmytroNazarenko DmytroNazarenko moved this from Ready For Release Candidate to Merged To Release Candidate in SKALE Engineering 🚀 Jan 23, 2024
@EvgeniyZZ EvgeniyZZ moved this from Merged To Release Candidate to QA in SKALE Engineering 🚀 Jan 23, 2024
@oleksandrSydorenkoJ
Copy link

oleksandrSydorenkoJ commented Feb 14, 2024

skaled: 3.18.0-beta.1

Cases:

  • Node rotation
  • State root mismatch error
  • Consensus inactive timeout (3 hours)
  • Attached volume issues (no space left on the device)

Node Rotation

[2024-02-13 16:41:47.953] [6:consensus] [info] 29320:Consensus exiting: exitGracefully called by skaled
Internal exit initiated. Signal -1. Unknown signal type.
[2024-02-13 16:41:48.155] [config] [info] Node exit called

State root mismatch error

2024-02-14 15:00:34.099364   FATAL STATE ROOT MISMATCH ERROR: current state root 30...58 is not equal to arrived state root 32...67 with block ID #35386, /data_dir cleanup is recommended, exiting with code 200...

Internal exit initiated. Signal -1. Unknown signal type.
[2024-02-14 15:00:34.366] [6:main] [info] 35385:Exiting on block boundary for block 35385

@oleksandrSydorenkoJ
Copy link

Verified on Regression network
skaled: 3.18.0-beta.1

Consensus shutdown will be fixed in a separate issue skalenetwork/skale-consensus#812

@EvgeniyZZ EvgeniyZZ moved this from QA to Done in SKALE Engineering 🚀 Feb 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working epic:node-rotation
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

4 participants