Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Blocks may be never finalised on network start #10558

Open
Longarithm opened this issue Feb 2, 2024 · 1 comment
Open

Blocks may be never finalised on network start #10558

Longarithm opened this issue Feb 2, 2024 · 1 comment
Labels
A-stateless-validation Area: stateless validation T-core Team: issues relevant to the core team

Comments

@Longarithm
Copy link
Member

Longarithm commented Feb 2, 2024

On StatelessNet start @khorolets discovered that there are no blocks on chain: https://near.zulipchat.com/#narrow/stream/407237-pagoda.2Fcore.2Fstateless-validation/topic/block.20finality.20issues/near/419233634

The issue persisted for ~3 hours and got autoresolved then, and we don't know why. It was too early for epoch transition. This doesn't look relevant only to the protocol change introduced by stateless validation. Debug logs show that no block endorsements were being sent, only the skips, which led to skipping absolutely all blocks on even heights.

Also this was reproduced on localnet with 1-2 nodes. However, adversenet testing didn't show that.

I don't think it will happen again for statelessnet as more nodes will improve network quality, but it could be nice to understand what happens inside Doomslug::get_timer and the reason behind the issue.

Debug branch ab20f8c
2.6 GB logs https://drive.google.com/file/d/1pYc9pZcyCAcAUjloYRzxRwkIFhJEyRPp/view?usp=drive_link

@andrei-near
Copy link
Contributor

We have this situation in Statelessnet
near_final_block_height 119519718
near_block_height_head 119525916

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-stateless-validation Area: stateless validation T-core Team: issues relevant to the core team
Projects
None yet
Development

No branches or pull requests

2 participants