Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

115-offloadable-bloom-filter #121

Merged
merged 57 commits into from
Nov 10, 2021
Merged

115-offloadable-bloom-filter #121

merged 57 commits into from
Nov 10, 2021

Conversation

vovac12
Copy link
Contributor

@vovac12 vovac12 commented Aug 15, 2021

Closes #115

is a performance hit in this version, so I decided to postpone it.
This generic implementation may be found and is discussed here:
#101)

This reverts commit e156056.
Base automatically changed from b+-tree-on-disk-index to master September 28, 2021 12:15
@ikopylov
Copy link
Member

@vovac12 please, fix merge conflicts

@vovac12 vovac12 requested review from piakushin and idruzhitskiy and removed request for piakushin November 2, 2021 17:08
src/blob/index/core.rs Outdated Show resolved Hide resolved
src/blob/index/core.rs Outdated Show resolved Hide resolved
src/storage/core.rs Outdated Show resolved Hide resolved
src/storage/core.rs Outdated Show resolved Hide resolved
src/blob/index/bloom.rs Show resolved Hide resolved
src/blob/index/bloom.rs Show resolved Hide resolved
src/blob/index/bloom.rs Show resolved Hide resolved
src/blob/index/bloom.rs Show resolved Hide resolved
if i >= self.header.meta_size as u64 {
return Err(anyhow::anyhow!("read meta out of range"));
}
let mut buf = vec![0; 1];
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can use just an array here

if !self.range_filter.contains(key) {
FilterResult::NotContains
} else if self.params.bloom_is_on && !self.bloom_filter.contains(key) {
} else if self.params.bloom_is_on
&& matches!(self.check_bloom_key(key).await, Ok(Some(false)))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It works fine. But we need to investigate errors somehow. And errors would be just dropped here. Maybe it makes sense to return Result<FilterResult> here

if !self.range_filter.contains(key) {
FilterResult::NotContains
} else if self.params.bloom_is_on
&& self.bloom_filter.contains_in_memory(key) == Some(false)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can squash this condition with previous

src/blob/index/core.rs Show resolved Hide resolved
/// Returns memory allocated for bloom filter buffer
pub async fn filter_memory_allocated(&self) -> usize {
let safe = self.inner.safe.read().await;
let active = if let Some(ablob) = safe.active_blob.as_ref() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be shorter with map_or()

@piakushin piakushin merged commit a4e99f2 into master Nov 10, 2021
@piakushin piakushin deleted the 115-offloadable-bloom-filter branch November 10, 2021 09:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Make bloom filter offloadable from memory
5 participants