DB::CompressionCodecFactory::getStaticCodec(const CompressionSetting &): Assertion `inserted' failed

[JinheLin]

Error

gtests_dbms: /home/jenkins/agent/workspace/tiflash-build-common/tiflash/dbms/src/IO/Compression/CompressionCodecFactory.cpp:97: static CompressionCodecPtr DB::CompressionCodecFactory::getStaticCodec(const CompressionSetting &): Assertion `inserted' failed.

tiflash/dbms/src/IO/Compression/CompressionCodecFactory.cpp

Lines 81 to 99 in 5500b35

	template <>
	CompressionCodecPtr CompressionCodecFactory::getStaticCodec<CompressionCodecLZ4>(const CompressionSetting & setting)
	{
	static constexpr auto MAX_LZ4_MAP_SIZE = 10;
	static std::shared_mutex lz4_mutex;
	static std::unordered_map<int, CompressionCodecPtr> lz4_map(MAX_LZ4_MAP_SIZE);
	{
	std::shared_lock lock(lz4_mutex);
	auto it = lz4_map.find(setting.level);
	if (it != lz4_map.end())
	return it->second;
	}
	std::unique_lock lock(lz4_mutex);
	if (lz4_map.size() >= MAX_LZ4_MAP_SIZE)
	lz4_map.clear();
	auto [it, inserted] = lz4_map.emplace(setting.level, std::make_shared<CompressionCodecLZ4>(setting.level));
	assert(inserted);
	return it->second;
	}

1. Two threads with the same level acquire shared lock of lz4_mutex and read lz4_map concurrently and both failed because codec of level not exist.
2. Thread1 acquire unique lock of lz4_mutex, insert codec of level and assert(inserted) successfully.
3. Thread2 acquire unique lock of lz4_mutex, but insert codec of level failed, and assert(inserted) will crash the process in debug mode.

[JinheLin]

Seems the level of compression is determined by dt_compression_level:

tiflash/dbms/src/Interpreters/Settings.h

Line 218 in cf5a204

M(SettingInt64, dt_compression_level, 1, "The compression level.") \

Can we just create a static object when the server start? @Lloyd-Pottiger

[Lloyd-Pottiger]

I have fixed this in #9312.

Can we just create a static object when the server start? @Lloyd-Pottiger

No, dt_compression_level needs to be dynamically changeable.