You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, our storage engine uses rocksdb's default options to construct a bloom filter, which only checks the existence of "hashkey+sortkey". For the situation where users scan over the range of hashkey (multigetrange, hashscan), the bloom filter makes no optimization if the hashkey does not exist. And rocksdb will search the corresponding SST data block, until it confirms the inexistence.
For point lookups (multiget/get), only the whole key blooms are used. For range queries (multigetrange, hashscan, unorderedscan+hashkey prefix filtering), rocksdb responds without searching SST after the hashkey misses in the bloom filter.
One other improvement is to use hash-based memtable. Since Pegasus doesn't support cross-partition ordering, this implementation can be taken into consideration.
The text was updated successfully, but these errors were encountered:
Currently, our storage engine uses rocksdb's default options to construct a bloom filter, which only checks the existence of "hashkey+sortkey". For the situation where users scan over the range of hashkey (
multigetrange
,hashscan
), the bloom filter makes no optimization if the hashkey does not exist. And rocksdb will search the corresponding SST data block, until it confirms the inexistence.So what can we do with it? RocksDB provides an option called prefix seek mode. Its major improvement is adding prefix (the hashkey) as well as the whole key to the bloom filter. (https://github.com/facebook/rocksdb/wiki/RocksDB-Bloom-Filter#prefix-vs-whole-key)
For point lookups (multiget/get), only the whole key blooms are used. For range queries (multigetrange, hashscan, unorderedscan+hashkey prefix filtering), rocksdb responds without searching SST after the hashkey misses in the bloom filter.
One other improvement is to use hash-based memtable. Since Pegasus doesn't support cross-partition ordering, this implementation can be taken into consideration.
The text was updated successfully, but these errors were encountered: