Reduce the number of files to avoid inodes exhausting

[JinheLin]

Enhancement

There is a case that a customer has more than 40000 tables or partitions.
Each table or partition has about 70 - 80 columns.
Since TiFlash will generate at most 5 files (dat, mrk, null.dat, null.mrk, idx) for each column, it will eventually generate 350 - 400 small files for each table/partition in initialization.

In the end, inodes of ext4 filesystem is exhausted.

[JinheLin]

Reducing the number of files of TiFlash can be decomposed into two sub problems.

1. Reducing the number of files of empty table.

• When TiFlash creates an empty table, it will create an empty segment which will create many empty files. So, it is import ant to reducing the number of files of empty table.
• Option 1: we can fix this problem by not creating empty segment when an empty table is created. Creating the first segment only when the first write request is coming.
  • However, if there is a small amount of data is written, the data will only be saved in Delta/PageStorage, but a Stable/DMFile with many empty files will still be created.
• Option2: creating Stable/DMFile only when it has data.
  • A lot of code assumes that stable is not null, which has a significant impact to codebase.

2. Merging small files into larger one.

• Merging packStat, packProperty and meta into one meta data file. This feature is already supported in S3 mode, when STORAGE_FORMAT_CURRENT.dm_file is DMFileFormat::V3. We need to make this feature support op mode.
• TiFlash will generate up to 5 files (dat, mrk, null. dat, null. mrk, idx) for each column. For simplicity, we can consider merging small files such as mrk, null.dat, null.mrk, and idx.