Assert failure in block_prefix_index

[killme2008]

Note: Please use Issues only for bug reports. For questions, discussions, feature requests, etc. post to dev group: https://www.facebook.com/groups/rocksdb.dev

Expected behavior

Restart the service, should not cause any crash.

Actual behavior

Assertion failed: (prefixes_per_bucket[i] == nullptr), function Finish, file table/block_prefix_index.cc, line 132.

Steps to reproduce the behavior

On MacOS, do not happen on linux system.

1. open a db with column families and options：

    private static BlockBasedTableConfig createTableConfig() {
        BlockBasedTableConfig tconfig = new BlockBasedTableConfig(). //
                setIndexType(IndexType.kHashSearch). // use hash search(btree) for prefix scan.
                setBlockSize(4 * SizeUnit.KB).//
                setFilter(new BloomFilter(16, false)). //
                setCacheIndexAndFilterBlocks(true). //
                setBlockCacheSize(512 * SizeUnit.MB). //
                setCacheNumShardBits(8);
        return tconfig;
    }


   public static DBOptions createDBOptions() {

        //Turn based on https://github.com/facebook/rocksdb/wiki/RocksDB-Tuning-Guide
        DBOptions options = new DBOptions();
        return options.setCreateIfMissing(true). //
                setCreateMissingColumnFamilies(true). //
                setMaxOpenFiles(-1). //
                setMaxLogFileSize(MAX_LOG_FILE_SIZE). //
                setMaxBackgroundFlushes(1). //
                setMaxBackgroundCompactions(1);

    }

    public static ColumnFamilyOptions createColumnFamilyOptions() {
        BlockBasedTableConfig tconfig = createTableConfig();
        ColumnFamilyOptions options = new ColumnFamilyOptions();
        return options.setMaxWriteBufferNumber(2). //
                useFixedLengthPrefixExtractor(8). //
                setTableFormatConfig(tconfig). //
                setCompressionType(CompressionType.LZ4_COMPRESSION). // 
                setCompactionStyle(CompactionStyle.LEVEL). //
                optimizeLevelStyleCompaction(). //
                setLevel0FileNumCompactionTrigger(10). //
                setLevel0SlowdownWritesTrigger(20). //
                setLevel0StopWritesTrigger(40). //
                setWriteBufferSize(64 * SizeUnit.MB). //
                setTargetFileSizeBase(64 * SizeUnit.MB). //
                setMaxBytesForLevelBase(512 * SizeUnit.MB). //
                setMergeOperator(new StringAppendOperator()). //
                setMemtablePrefixBloomSizeRatio(0.125);

    }

2. Write/Read some data, then stop the application(NOT graceful close rocksdb)
3. Restart the application ,then it crashed:

Assertion failed: (prefixes_per_bucket[i] == nullptr), function Finish, file table/block_prefix_index.cc, line 132.

[maysamyabandeh]

@killme2008 Thanks for reporting this. The assert statement is valid and this seems to be a case of data corruption. Still RocksDB should report data corruption instead of failing on the assert statement. Did your test check all the return statuses after the restart? Did any indicate a corruption? Also can you reproduce it consistently or it was one time incident?

[maysamyabandeh]

@killme2008 any thought?

[killme2008]

@maysamyabandeh

Did your test check all the return statuses after the restart?

I am sorry ,i am not sure what you mean.

Did any indicate a corruption? Also can you reproduce it consistently or it was one time incident?

No, i don't see any corruption error ,and it can be reproduced every time on Mac.I doubt it may has some relationship with my db/column options,but i am not sure.I will try it with different options next week.