Skip to content

Commit

Permalink
make polkadot die graciously (#2056)
Browse files Browse the repository at this point in the history
While investigating some db migrations that make the node startup fail,
I noticed that the node wasn't exiting and that the log file were
growing exponentially, until my whole system was freezing and that makes
it really hard to actually find why it was failing in the first place.

E.g:
```
 ls -lh /tmp/zombie-01a04c2a2c0265d85f6440cf01c0f44a_-51319-uyggzuD4wEpV/bob.log
 32,6G oct 27 11:16 /tmp/zombie-01a04c2a2c0265d85f6440cf01c0f44a_-51319-uyggzuD4wEpV/bob.log
```

This was happening because the following errors were being printed
continously without the subsystem main loop exiting:

From dispute-coordinator:
```
WARN tokio-runtime-worker parachain::dispute-coordinator: error=Subsystem(Generated(Context("Signal channel is terminated and empty.")))
```

From availability recovery:
```
Erasure task channel closed. Node shutting down ?
```

Signed-off-by: Alexandru Gheorghe <[email protected]>
  • Loading branch information
alexggh authored and s0me0ne-unkn0wn committed Oct 29, 2023
1 parent 33f3eb3 commit 919984e
Show file tree
Hide file tree
Showing 2 changed files with 3 additions and 1 deletion.
3 changes: 2 additions & 1 deletion polkadot/node/core/dispute-coordinator/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@

use std::sync::Arc;

use error::FatalError;
use futures::FutureExt;

use gum::CandidateHash;
Expand Down Expand Up @@ -431,7 +432,7 @@ impl DisputeCoordinatorSubsystem {
#[overseer::contextbounds(DisputeCoordinator, prefix = self::overseer)]
async fn wait_for_first_leaf<Context>(ctx: &mut Context) -> Result<Option<ActivatedLeaf>> {
loop {
match ctx.recv().await? {
match ctx.recv().await.map_err(FatalError::SubsystemReceive)? {
FromOrchestra::Signal(OverseerSignal::Conclude) => return Ok(None),
FromOrchestra::Signal(OverseerSignal::ActiveLeaves(update)) => {
if let Some(activated) = update.activated {
Expand Down
1 change: 1 addition & 0 deletions polkadot/node/network/availability-recovery/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -822,6 +822,7 @@ async fn erasure_task_thread(
target: LOG_TARGET,
"Erasure task channel closed. Node shutting down ?",
);
break
},
}
}
Expand Down

0 comments on commit 919984e

Please sign in to comment.