Skip to content
This repository has been archived by the owner on Jan 13, 2025. It is now read-only.

StatusDeque split into separate objects with their own root checkpoint strategy #2613

Merged
merged 7 commits into from
Jan 31, 2019

Conversation

aeyakovenko
Copy link
Member

@aeyakovenko aeyakovenko commented Jan 31, 2019

Problem

StatusDeque was doing multiple jobs, tracking last_ids and signature statuses.

Summary of Changes

  • Split StatusDeque into 2 objects.
  • Use a bloom filter for storing Ok transaction results, reducing memory overhead for a 1m transactions to 4mb per block.
  • Separate concurrency for each object
  • Support for checkpointing necessary for forking

Fixes # (part of #2289)

* LastIdQueue tracks last_ids
* StatusCache keeps track of signature statuses
* StatusCache stores success as a bit in a bloom filter
* Overhead for 1m Ok transactions is 4mb in memory
* Less concurency between the objects, last_id and status_cache are read and written to at different points in the pipeline
* Each object has its own strategy for merging into the root checkpoint
@codecov
Copy link

codecov bot commented Jan 31, 2019

Codecov Report

Merging #2613 into master will increase coverage by 0.2%.
The diff coverage is 93.5%.

@@           Coverage Diff            @@
##           master   #2613     +/-   ##
========================================
+ Coverage    77.6%   77.8%   +0.2%     
========================================
  Files         112     113      +1     
  Lines       18180   18251     +71     
========================================
+ Hits        14112   14208     +96     
+ Misses       4068    4043     -25

@aeyakovenko aeyakovenko merged commit 2754cee into solana-labs:master Jan 31, 2019
wen-coding pushed a commit to wen-coding/solana that referenced this pull request Aug 16, 2024
When hot-swapping identity, ContactInfo.outset should be updated so that
the new ContactInfo overrides older node with the same pubkey.
CriesofCarrots pushed a commit to CriesofCarrots/solana that referenced this pull request Aug 27, 2024
…lana-labs#2666) (solana-labs#2681)

* customizes override logic for gossip ContactInfo (solana-labs#2579)

If there are two running instances of the same node, we want the
ContactInfo with more recent start time to be propagated through
gossip regardless of wallclocks.

The commit adds custom override logic for ContactInfo to first compare
by outset timestamp.

* updates ContactInfo.outset when hot-swapping identity (solana-labs#2613)

When hot-swapping identity, ContactInfo.outset should be updated so that
the new ContactInfo overrides older node with the same pubkey.

* patches bug causing false duplicate nodes error (solana-labs#2666)

The bootstrap code during the validator start pushes a contact-info with
more recent timestamp to gossip. If the node is staked the contact-info
lingers in gossip causing false duplicate node instances when the fully
initialized node joins gossip later on.

The commit refreshes the timestamp on contact-info so that it overrides
the one pushed by bootstrap and avoid false duplicates error.

---------

Co-authored-by: behzad nouri <[email protected]>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants