In-memory mode creates physical tables (in the current directory)

[damz]

It looks like the in-memory mode creates physical tables in the current directory. This is because *DB.handleFlushTask does not seem to have a special case for the in-memory mode.

Steps to Reproduce the issue

• Run go test . and watch SST files being created and destroyed in the current directory.

[yashkothari42]

Hey @damz

I think this is happening because there isn't any check handling the case when InMemory is true but KeepL0InMemory is false. This causes badger to flush L0 on disks. This scenario is happening here.

badger/levels_test.go

Lines 772 to 779 in dd332b0

	t.Run("with KeepL0InMemory", func(t *testing.T) {
	opt.KeepL0InMemory = true
	test(t, &opt)
	})
	t.Run("with L0 on disk", func(t *testing.T) {
	opt.KeepL0InMemory = false
	test(t, &opt)
	})

@ashish-goswami, is this intended?

I guess there are two ways to handle it

1. return Error while opening db if we encounter this case
2. force KeepL0InMemory to be true when InMemory is true.

[damz]

Indeed, the easiest fix would be to make this check db.opt.InMemory || db.opt.KeepL0InMemory:

badger/db.go

Lines 1023 to 1029 in dd332b0

	if db.opt.KeepL0InMemory {
	tbl, err := table.OpenInMemoryTable(tableData, fileID, &bopts)
	if err != nil {
	return errors.Wrapf(err, "failed to open table in memory")
	}
	return db.lc.addLevel0Table(tbl)
	}

[jarifibrahim]

The code @damz pointed out is the correct place to fix this. This is a low hanging fruit and I'll leave it for new contributors.

Thank you @damz and @itsyashkothari !

[yashkothari42]

Hi @jarifibrahim

Wouldn't the solution provided result in inconsistency in logic.

For example here

badger/level_handler.go

Lines 191 to 195 in dd332b0

	// Return false only if L0 is in memory and number of tables is more than number of
	// ZeroTableStall. For on disk L0, we should just add the tables to the level.
	if s.db.opt.KeepL0InMemory && len(s.tables) >= s.db.opt.NumLevelZeroTablesStall {
	return false
	}

KeepL0InMemory==false but actually L0 is in memory.

These are my proposed solutions -
1.

if !opt.KeepL0InMemory && opt.InMemory {
        return nil,errors.Errorf("Can not have KeepL0InMemory false and InMemory true.") 
}

opt.KeepL0InMemory = opt.KeepL0InMemory || opt.InMemory 

I would love to submit PR for this

[jarifibrahim]

Approach 1 would mean the user has to set two options to enable in-memory mode. This doesn't seem like a good user experience.

opt.KeepL0InMemory = opt.KeepL0InMemory || opt.InMemory

This would result in the least amount of changes. Let's go with this approach.