Memory useage keeps on high level and wouldn't be released by storage process

[Sophie-Xie]

Describe the bug (required)

After importing data, memory useage keeps on high level with bloom filter, which often results in OOM in some scenario -- huge and full data import everytime.

In out internal tests, prefix bloom filter couldn't reduce the memory usage obviously instead of whole key bloom filter with a 8 billion dataset.

Your Environments (required)

• OS: uname -a
• Compiler: g++ --version or clang++ --version
• CPU: lscpu
• Commit id (e.g. a3ffc7d8)

How To Reproduce(required)

Steps to reproduce the behavior:

1. Step 1
2. Step 2
3. Step 3

Expected behavior

Additional context

[carrey-feng]

Is there any new progress on this issue

[wey-gu]

Is there any new progress on this issue

It seems this issue was stuck lacking reproducing environment, are you facing similar issues? If so, could you please help provide some information on this?

@Sophie-Xie

[huzhi915]

Is there any new progress on this issue

It seems this issue was stuck lacking reproducing environment, are you facing similar issues? If so, could you please help provide some information on this?

@Sophie-Xie

this issure can be fixed by configuring the items "enable_rocksdb_prefix_filtering" and "enable_partitioned_index_filter" of nebula-storaged.conf file, close it plz.

[carrey-feng]

this issure can be fixed by configuring the items "enable_rocksdb_prefix_filtering" and "enable_partitioned_index_filter" of nebula-storaged.conf file, close it plz.

Need rebuild index after modify enable_rocksdb_prefix_filtering ？

[wey-gu]

this issure can be fixed by configuring the items "enable_rocksdb_prefix_filtering" and "enable_partitioned_index_filter" of nebula-storaged.conf file, close it plz.

Need rebuild index after modify enable_rocksdb_prefix_filtering ？

Rebuild of the index is not needed, I think those two configurations are cache related.

@huzhi915 please correct me if wrong.

Thanks.

[huzhi915]

this issure can be fixed by configuring the items "enable_rocksdb_prefix_filtering" and "enable_partitioned_index_filter" of nebula-storaged.conf file, close it plz.

Need rebuild index after modify enable_rocksdb_prefix_filtering ？

Rebuild of the index is not needed, I think those two configurations are cache related.

@huzhi915 please correct me if wrong.

Thanks.

exactly.

[Sophie-Xie]

Talked it offline, now it's work.