Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Splitstore: Implement cold object reification #6868

Closed
wants to merge 25 commits into from

Conversation

vyzo
Copy link
Contributor

@vyzo vyzo commented Jul 25, 2021

On top of #6854
Closes #6726

This adds configurable cold object reification in the splitstore. We can do that now that #6705 is resolved.

Rationale: It has been observed that the hot range in my discard store still contains some references to the cold store (containing only the original snapshot data), even after rewarming up with #6867; afacit these are objects that get recreated by the VM and occurs-checked with Has, which avoids a write.

This patch adds a policy for cold object reification, with the following options:

  • 0/default no cold object reification.
  • 1 reification when there is a view.
  • 2 reification when there is a view or a get.

The separation of view from get comes from the observation that state computations access objects with view to unmarshal, while chain exchange uses get to transmit over the network. Thus a setting of 1 should be safe; a setting of 2 becomes relevant only when we have gc for the coldstore, but it's there to future proof.

@vyzo vyzo requested review from Stebalien and raulk July 25, 2021 12:06
@vyzo vyzo requested a review from a team as a code owner July 25, 2021 12:06
@jacobheun jacobheun added epic/splitstore team/ignite Issues and PRs being tracked by Team Ignite at Protocol Labs labels Jul 26, 2021
@vyzo vyzo marked this pull request as draft July 26, 2021 20:01
@vyzo
Copy link
Contributor Author

vyzo commented Jul 26, 2021

Turning this back to draft, as I am not convinced this is the right approach.

Base automatically changed from feat/moving-gc to master July 27, 2021 20:27
@vyzo
Copy link
Contributor Author

vyzo commented Jul 28, 2021

Following discussion with @Stebalien it appears that this is the correct course of action:

  • Implement a specialized view of the splitstore that recursively reifes all hotstore misses
  • Use the view in the vm (and other places that need to reify objects)

This will ensure that we reify objects only when we really need to, and avoid accidental reification.
It will also obviate the need for a policy and the (hacky) distinction of get-vs-view.

In this light I am closing this pr; will open a new one with the desired implementation.

@vyzo vyzo closed this Jul 28, 2021
@vyzo vyzo deleted the feat/splitstore-cold-reification branch July 28, 2021 08:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
team/ignite Issues and PRs being tracked by Team Ignite at Protocol Labs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Splitstore: reification of cold objects
2 participants