Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Merged by Bors] - Don't return errors when fork choice fails #3370

Closed
wants to merge 4 commits into from

Conversation

paulhauner
Copy link
Member

@paulhauner paulhauner commented Jul 25, 2022

Issue Addressed

NA

Proposed Changes

There are scenarios where the only viable head will have an invalid execution payload, in this scenario the get_head function on proto_array will return an error. We must recover from this scenario by importing blocks from the network.

This PR stops BeaconChain::recompute_head from returning an error so that we can't accidentally start down-scoring peers or aborting block import just because the current head has an invalid payload.

Reviewer Notes

The following changes are included:

  1. Allow fork_choice.get_head to fail gracefully in BeaconChain::process_block when trying to update the early_attester_cache; simply don't add the block to the cache rather than aborting the entire process.
  2. Don't return an error from BeaconChain::recompute_head_at_current_slot and BeaconChain::recompute_head to defensively prevent calling functions from aborting any process just because the fork choice function failed to run.
    • This should have practically no effect, since most callers were still continuing if recomputing the head failed.
    • The outlier is that the API will return 200 rather than a 500 when fork choice fails.
  3. Add the ProtoArrayForkChoice::set_all_blocks_to_optimistic function to recover from the scenario where we've rebooted and the persisted fork choice has an invalid head.

@paulhauner paulhauner changed the title Don' Don't return errors when fork choice fails Jul 25, 2022
@paulhauner paulhauner added the work-in-progress PR is a work-in-progress label Jul 25, 2022
@paulhauner paulhauner changed the base branch from stable to unstable July 25, 2022 07:17
@paulhauner paulhauner added the v2.5.0 Required for Goerli merge release label Jul 26, 2022
@paulhauner paulhauner force-pushed the invalid-head branch 2 times, most recently from 35d64c7 to ab0f7f2 Compare July 27, 2022 05:31
@paulhauner paulhauner added ready-for-review The code is ready for review and removed work-in-progress PR is a work-in-progress labels Jul 27, 2022
@realbigsean realbigsean marked this pull request as ready for review July 27, 2022 13:40
Copy link
Member

@realbigsean realbigsean left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM (one typo)

Copy link
Member

@michaelsproul michaelsproul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, modulo some small comment fixes

beacon_node/beacon_chain/src/test_utils.rs Outdated Show resolved Hide resolved
beacon_node/beacon_chain/tests/payload_invalidation.rs Outdated Show resolved Hide resolved
@michaelsproul michaelsproul added waiting-on-author The reviewer has suggested changes and awaits thier implementation. and removed ready-for-review The code is ready for review labels Jul 28, 2022
Don't return an error from fork choice

Fix more tests

Add test

Fix beacon processor test

Improve test

Fix block import

Rebuild from finalized state if `get_head` fails

Rebuild from finalized state if `get_head` fails

Revert changes to `ForkChoice`

Add failing test

Revert "Rebuild from finalized state if `get_head` fails"

This reverts commit 17cf507.

Add `set_all_blocks_to_optimistic`

Remove code fragment

Fix tests
@michaelsproul michaelsproul changed the base branch from unstable to stable July 28, 2022 12:32
@michaelsproul michaelsproul changed the base branch from stable to unstable July 28, 2022 12:32
@michaelsproul
Copy link
Member

This looks ready to merge, I noticed that my corrections from above got reverted but that's no biggie, we can merge without them

@michaelsproul michaelsproul added ready-for-merge This PR is ready to merge. and removed waiting-on-author The reviewer has suggested changes and awaits thier implementation. labels Jul 28, 2022
@paulhauner
Copy link
Member Author

bors r+

@bors
Copy link

bors bot commented Jul 28, 2022

🕐 Waiting for PR status (Github check) to be set, probably by CI. Bors will automatically try to run when all required PR statuses are set.

bors bot pushed a commit that referenced this pull request Jul 28, 2022
## Issue Addressed

NA

## Proposed Changes

There are scenarios where the only viable head will have an invalid execution payload, in this scenario the `get_head` function on `proto_array` will return an error. We must recover from this scenario by importing blocks from the network.

This PR stops `BeaconChain::recompute_head` from returning an error so that we can't accidentally start down-scoring peers or aborting block import just because the current head has an invalid payload.

## Reviewer Notes

The following changes are included:

1. Allow `fork_choice.get_head` to fail gracefully in `BeaconChain::process_block` when trying to update the `early_attester_cache`; simply don't add the block to the cache rather than aborting the entire process.
1. Don't return an error from `BeaconChain::recompute_head_at_current_slot` and `BeaconChain::recompute_head` to defensively prevent calling functions from aborting any process just because the fork choice function failed to run.
    - This should have practically no effect, since most callers were still continuing if recomputing the head failed.
    - The outlier is that the API will return 200 rather than a 500 when fork choice fails.
1. Add the `ProtoArrayForkChoice::set_all_blocks_to_optimistic` function to recover from the scenario where we've rebooted and the persisted fork choice has an invalid head.
@bors bors bot changed the title Don't return errors when fork choice fails [Merged by Bors] - Don't return errors when fork choice fails Jul 28, 2022
@bors bors bot closed this Jul 28, 2022
bors bot pushed a commit that referenced this pull request Jul 30, 2022
## Issue Addressed

NA

## Proposed Changes

This PR will make Lighthouse return blocks with invalid payloads via the API with `execution_optimistic = true`. This seems a bit awkward, however I think it's better than returning a 404 or some other error.

Let's consider the case where the only possible head is invalid (#3370 deals with this). In such a scenario all of the duties endpoints will start failing because the head is invalid. I think it would be better if the duties endpoints continue to work, because it's likely that even though the head is invalid the duties are still based upon valid blocks and we want the VC to have them cached. There's no risk to the VC here because we won't actually produce an attestation pointing to an invalid head.

Ultimately, I don't think it's particularly important for us to distinguish between optimistic and invalid blocks on the API. Neither should be trusted and the only *real* reason that we track this is so we can try and fork around the invalid blocks.


## Additional Info

- ~~Blocked on #3370~~
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready-for-merge This PR is ready to merge. v2.5.0 Required for Goerli merge release
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants