Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Compression: Support enabling lightweight compression #9312

Merged
merged 12 commits into from
Sep 6, 2024
Merged

Compression: Support enabling lightweight compression #9312

merged 12 commits into from
Sep 6, 2024

Conversation

Lloyd-Pottiger
Copy link
Contributor

@Lloyd-Pottiger Lloyd-Pottiger commented Aug 12, 2024 •
edited by JaySon-Huang
Loading

What problem does this PR solve?

Issue Number: close #8982, close #9336

Problem Summary:

What is changed and how it works?

Compression: Support enabling lightweight compression
  1. fullstack test
  2. support use lightweight compression in DMFile
  3. metric
image

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Add a new option for configuration `profiles.default.dt_compression_method`: lightweight (case insensitive) which can help improve the read performance and data size may be smaller.

@ti-chi-bot ti-chi-bot bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Aug 12, 2024
@Lloyd-Pottiger Lloyd-Pottiger changed the title Compression: Support enable lightweight compression Compression: Support enabling lightweight compression Aug 12, 2024
@Lloyd-Pottiger
Copy link
Contributor Author

Lloyd-Pottiger commented Aug 12, 2024 •
edited
Loading 

$ ./dbms/bench_dbms --benchmark_filter="CodecSingle*"         
2024-08-12T16:26:11+08:00
Running ./dbms/bench_dbms
Run on (72 X 3300.03 MHz CPU s)
CPU Caches:
  L1 Data 32 KiB (x36)
  L1 Instruction 32 KiB (x36)
  L2 Unified 1024 KiB (x36)
  L3 Unified 25344 KiB (x2)
Load Average: 14.53, 19.11, 17.60
--------------------------------------------------------------------------------------------------------------------------------
Benchmark                                                                                      Time             CPU   Iterations
--------------------------------------------------------------------------------------------------------------------------------
CodecSingleWriteSameValueUInt8LZ4/tests::SameValueGenerator(128)                          153757 ns        60776 ns        11502
CodecSingleWriteSameValueUInt8Lightweight/tests::SameValueGenerator(128)                  149884 ns        62701 ns        11489
CodecSingleWriteSameValueUInt16LZ4/tests::SameValueGenerator(128)                         148737 ns        55795 ns        13129
CodecSingleWriteSameValueUInt16Lightweight/tests::SameValueGenerator(128)                 145025 ns        57799 ns        11418
CodecSingleWriteSameValueUInt32LZ4/tests::SameValueGenerator(128)                         143569 ns        56761 ns        12103
CodecSingleWriteSameValueUInt32Lightweight/tests::SameValueGenerator(128)                 150501 ns        62172 ns        11125
CodecSingleWriteSameValueUInt64LZ4/tests::SameValueGenerator(128)                         146883 ns        62221 ns        11045
CodecSingleWriteSameValueUInt64Lightweight/tests::SameValueGenerator(128)                 152306 ns        61762 ns        11097
CodecSingleWriteSequentialUInt8LZ4/tests::SequentialGenerator(2)                          145188 ns        54837 ns        13180
CodecSingleWriteSequentialUInt8Lightweight/tests::SequentialGenerator(2)                  157273 ns        66011 ns        11166
CodecSingleWriteSequentialUInt16LZ4/tests::SequentialGenerator(2)                         165280 ns        67583 ns        10330
CodecSingleWriteSequentialUInt16Lightweight/tests::SequentialGenerator(2)                 158097 ns        72335 ns         9225
CodecSingleWriteSequentialUInt32LZ4/tests::SequentialGenerator(2)                         195465 ns        86991 ns         7755
CodecSingleWriteSequentialUInt32Lightweight/tests::SequentialGenerator(2)                 164868 ns        77014 ns         8936
CodecSingleWriteSequentialUInt64LZ4/tests::SequentialGenerator(2)                         270918 ns       170553 ns         4053
CodecSingleWriteSequentialUInt64Lightweight/tests::SequentialGenerator(2)                 157595 ns        70715 ns         9659
CodecSingleWriteSequentialReverseUInt8LZ4/tests::SequentialGenerator(-2)                  141709 ns        52900 ns        13473
CodecSingleWriteSequentialReverseUInt8Lightweight/tests::SequentialGenerator(-2)          152278 ns        64754 ns        11265
CodecSingleWriteSequentialReverseUInt16LZ4/tests::SequentialGenerator(-2)                 164498 ns        67767 ns        11091
CodecSingleWriteSequentialReverseUInt16Lightweight/tests::SequentialGenerator(-2)         160262 ns        76008 ns         9361
CodecSingleWriteSequentialReverseUInt32LZ4/tests::SequentialGenerator(-2)                 182718 ns        82438 ns         8181
CodecSingleWriteSequentialReverseUInt32Lightweight/tests::SequentialGenerator(-2)         162765 ns        74835 ns         9482
CodecSingleWriteSequentialReverseUInt64LZ4/tests::SequentialGenerator(-2)                 270342 ns       169957 ns         4146
CodecSingleWriteSequentialReverseUInt64Lightweight/tests::SequentialGenerator(-2)         161405 ns        75648 ns         9020
CodecSingleWriteMonotonicUInt8LZ4/tests::MonotonicGenerator()                             154131 ns        60954 ns        12132
CodecSingleWriteMonotonicUInt8Lightweight/tests::MonotonicGenerator()                     173593 ns        73648 ns         9528
CodecSingleWriteMonotonicUInt16LZ4/tests::MonotonicGenerator()                            169647 ns        68153 ns        10276
CodecSingleWriteMonotonicUInt16Lightweight/tests::MonotonicGenerator()                    189014 ns        91910 ns         7925
CodecSingleWriteMonotonicUInt32LZ4/tests::MonotonicGenerator()                            227417 ns       125291 ns         5181
CodecSingleWriteMonotonicUInt32Lightweight/tests::MonotonicGenerator()                    178301 ns        87313 ns         7298
CodecSingleWriteMonotonicUInt64LZ4/tests::MonotonicGenerator()                            274385 ns       172988 ns         4188
CodecSingleWriteMonotonicUInt64Lightweight/tests::MonotonicGenerator()                    178037 ns        85646 ns         8176
CodecSingleWriteMonotonicReverseUInt8LZ4/tests::MonotonicGenerator(-1)                    150067 ns        59703 ns        11457
CodecSingleWriteMonotonicReverseUInt8Lightweight/tests::MonotonicGenerator(-1)            160130 ns        71529 ns        10164
CodecSingleWriteMonotonicReverseUInt16LZ4/tests::MonotonicGenerator(-1)                   163385 ns        66672 ns        10230
CodecSingleWriteMonotonicReverseUInt16Lightweight/tests::MonotonicGenerator(-1)           196112 ns        96928 ns         7493
CodecSingleWriteMonotonicReverseUInt32LZ4/tests::MonotonicGenerator(-1)                   224015 ns       124720 ns         5484
CodecSingleWriteMonotonicReverseUInt32Lightweight/tests::MonotonicGenerator(-1)           177407 ns        88889 ns         7854
CodecSingleWriteMonotonicReverseUInt64LZ4/tests::MonotonicGenerator(-1)                   265325 ns       169654 ns         3937
CodecSingleWriteMonotonicReverseUInt64Lightweight/tests::MonotonicGenerator(-1)           176963 ns        90837 ns         7785
CodecSingleWriteMinMaxUInt8LZ4/tests::MinMaxGenerator()                                   139044 ns        51733 ns        13354
CodecSingleWriteMinMaxUInt8Lightweight/tests::MinMaxGenerator()                           146375 ns        62138 ns        10221
CodecSingleWriteMinMaxUInt16LZ4/tests::MinMaxGenerator()                                  136482 ns        52640 ns        13689
CodecSingleWriteMinMaxUInt16Lightweight/tests::MinMaxGenerator()                          160292 ns        75461 ns         9211
CodecSingleWriteMinMaxUInt32LZ4/tests::MinMaxGenerator()                                  138460 ns        55317 ns        12755
CodecSingleWriteMinMaxUInt32Lightweight/tests::MinMaxGenerator()                          165157 ns        81067 ns         8696
CodecSingleWriteMinMaxUInt64LZ4/tests::MinMaxGenerator()                                  143630 ns        60614 ns        10941
CodecSingleWriteMinMaxUInt64Lightweight/tests::MinMaxGenerator()                          167070 ns        82648 ns         8674
CodecSingleWriteRandomUInt8LZ4/tests::RandomGenerator<UInt8>(0)                           151741 ns        61590 ns        11802
CodecSingleWriteRandomUInt8Lightweight/tests::RandomGenerator<UInt8>(0)                   174187 ns        81512 ns         8471
CodecSingleWriteRandomUInt16LZ4/tests::RandomGenerator<UInt16>(0)                         166525 ns        69516 ns         9926
CodecSingleWriteRandomUInt16Lightweight/tests::RandomGenerator<UInt16>(0)                 196934 ns        98294 ns         6557
CodecSingleWriteRandomUInt32LZ4/tests::RandomGenerator<UInt32>(0)                         186654 ns        81931 ns         8467
CodecSingleWriteRandomUInt32Lightweight/tests::RandomGenerator<UInt32>(0)                 220727 ns       117468 ns         5987
CodecSingleWriteRandomUInt64LZ4/tests::RandomGenerator<UInt64>(0)                         220699 ns       116389 ns         5752
CodecSingleWriteRandomUInt64Lightweight/tests::RandomGenerator<UInt64>(0)                 251935 ns       147726 ns         4912
CodecSingleWriteSmallRandomUInt8LZ4/tests::RandomGenerator<UInt8>(0, 0, 16)               155814 ns        64514 ns        10886
CodecSingleWriteSmallRandomUInt8Lightweight/tests::RandomGenerator<UInt8>(0, 0, 16)       180154 ns        83587 ns         8175
CodecSingleWriteSmallRandomUInt16LZ4/tests::RandomGenerator<UInt16>(0, 0, 16)             186090 ns        96176 ns         7017
CodecSingleWriteSmallRandomUInt16Lightweight/tests::RandomGenerator<UInt16>(0, 0, 16)     219221 ns       130410 ns         5217
CodecSingleWriteSmallRandomUInt32LZ4/tests::RandomGenerator<UInt32>(0, 0, 16)             238441 ns       142270 ns         4928
CodecSingleWriteSmallRandomUInt32Lightweight/tests::RandomGenerator<UInt32>(0, 0, 16)     184300 ns        94414 ns         7260
CodecSingleWriteSmallRandomUInt64LZ4/tests::RandomGenerator<UInt64>(0, 0, 16)             247245 ns       150864 ns         4608
CodecSingleWriteSmallRandomUInt64Lightweight/tests::RandomGenerator<UInt64>(0, 0, 16)     183801 ns        97446 ns         7091
CodecSingleWriteRepeatUInt8LZ4/tests::RepeatGenerator<UInt8>(0)                           151206 ns        63583 ns        10999
CodecSingleWriteRepeatUInt8Lightweight/tests::RepeatGenerator<UInt8>(0)                   157959 ns        74665 ns         9202
CodecSingleWriteRepeatUInt16LZ4/tests::RepeatGenerator<UInt16>(0)                         155850 ns        68193 ns         9997
CodecSingleWriteRepeatUInt16Lightweight/tests::RepeatGenerator<UInt16>(0)                 175013 ns        87923 ns         7916
CodecSingleWriteRepeatUInt32LZ4/tests::RepeatGenerator<UInt32>(0)                         172967 ns        78635 ns         9141
CodecSingleWriteRepeatUInt32Lightweight/tests::RepeatGenerator<UInt32>(0)                 179412 ns        92225 ns         7604
CodecSingleWriteRepeatUInt64LZ4/tests::RepeatGenerator<UInt64>(0)                         184073 ns        91497 ns         7563
CodecSingleWriteRepeatUInt64Lightweight/tests::RepeatGenerator<UInt64>(0)                 176042 ns        91601 ns         7624
CodecSingleReadSameValueUInt8LZ4/tests::SameValueGenerator(128)                             5209 ns         5123 ns       137146
CodecSingleReadSameValueUInt8Lightweight/tests::SameValueGenerator(128)                     3753 ns         3667 ns       184974
CodecSingleReadSameValueUInt16LZ4/tests::SameValueGenerator(128)                            6902 ns         6810 ns       103964
CodecSingleReadSameValueUInt16Lightweight/tests::SameValueGenerator(128)                    4154 ns         4059 ns       173535
CodecSingleReadSameValueUInt32LZ4/tests::SameValueGenerator(128)                           10369 ns        10255 ns        68437
CodecSingleReadSameValueUInt32Lightweight/tests::SameValueGenerator(128)                    4582 ns         4496 ns       153206
CodecSingleReadSameValueUInt64LZ4/tests::SameValueGenerator(128)                           16495 ns        16349 ns        42651
CodecSingleReadSameValueUInt64Lightweight/tests::SameValueGenerator(128)                    6524 ns         6436 ns       109297
CodecSingleReadSequentialUInt8LZ4/tests::SequentialGenerator(2)                             3940 ns         3860 ns       184909
CodecSingleReadSequentialUInt8Lightweight/tests::SequentialGenerator(2)                     3869 ns         3788 ns       182107
CodecSingleReadSequentialUInt16LZ4/tests::SequentialGenerator(2)                            5810 ns         5714 ns       119202
CodecSingleReadSequentialUInt16Lightweight/tests::SequentialGenerator(2)                    4264 ns         4176 ns       171169
CodecSingleReadSequentialUInt32LZ4/tests::SequentialGenerator(2)                            8566 ns         8455 ns        80589
CodecSingleReadSequentialUInt32Lightweight/tests::SequentialGenerator(2)                    4591 ns         4496 ns       156301
CodecSingleReadSequentialUInt64LZ4/tests::SequentialGenerator(2)                           70182 ns        69791 ns        10055
CodecSingleReadSequentialUInt64Lightweight/tests::SequentialGenerator(2)                    6268 ns         6172 ns       111335
CodecSingleReadSequentialReverseUInt8LZ4/tests::SequentialGenerator(-2)                     3902 ns         3830 ns       185203
CodecSingleReadSequentialReverseUInt8Lightweight/tests::SequentialGenerator(-2)             3847 ns         3764 ns       181450
CodecSingleReadSequentialReverseUInt16LZ4/tests::SequentialGenerator(-2)                    5952 ns         5855 ns       121603
CodecSingleReadSequentialReverseUInt16Lightweight/tests::SequentialGenerator(-2)            4126 ns         4042 ns       172005
CodecSingleReadSequentialReverseUInt32LZ4/tests::SequentialGenerator(-2)                    8125 ns         8033 ns        87546
CodecSingleReadSequentialReverseUInt32Lightweight/tests::SequentialGenerator(-2)            4597 ns         4514 ns       156062
CodecSingleReadSequentialReverseUInt64LZ4/tests::SequentialGenerator(-2)                   70070 ns        69775 ns        10052
CodecSingleReadSequentialReverseUInt64Lightweight/tests::SequentialGenerator(-2)            6244 ns         6161 ns       113214
CodecSingleReadMonotonicUInt8LZ4/tests::MonotonicGenerator()                                4631 ns         4554 ns       151451
CodecSingleReadMonotonicUInt8Lightweight/tests::MonotonicGenerator()                        4578 ns         4496 ns       153992
CodecSingleReadMonotonicUInt16LZ4/tests::MonotonicGenerator()                               5739 ns         5655 ns       123161
CodecSingleReadMonotonicUInt16Lightweight/tests::MonotonicGenerator()                       5796 ns         5696 ns       123391
CodecSingleReadMonotonicUInt32LZ4/tests::MonotonicGenerator()                              11636 ns        11536 ns        60006
CodecSingleReadMonotonicUInt32Lightweight/tests::MonotonicGenerator()                       9568 ns         9459 ns        74211
CodecSingleReadMonotonicUInt64LZ4/tests::MonotonicGenerator()                              62528 ns        62321 ns        11170
CodecSingleReadMonotonicUInt64Lightweight/tests::MonotonicGenerator()                      12506 ns        12399 ns        56846
CodecSingleReadMonotonicReverseUInt8LZ4/tests::MonotonicGenerator(-1)                       4635 ns         4547 ns       147511
CodecSingleReadMonotonicReverseUInt8Lightweight/tests::MonotonicGenerator(-1)               4650 ns         4572 ns       153594
CodecSingleReadMonotonicReverseUInt16LZ4/tests::MonotonicGenerator(-1)                      5810 ns         5721 ns       119840
CodecSingleReadMonotonicReverseUInt16Lightweight/tests::MonotonicGenerator(-1)              5750 ns         5661 ns       113639
CodecSingleReadMonotonicReverseUInt32LZ4/tests::MonotonicGenerator(-1)                     11503 ns        11409 ns        60824
CodecSingleReadMonotonicReverseUInt32Lightweight/tests::MonotonicGenerator(-1)             10016 ns         9925 ns        69834
CodecSingleReadMonotonicReverseUInt64LZ4/tests::MonotonicGenerator(-1)                     62789 ns        62565 ns        11214
CodecSingleReadMonotonicReverseUInt64Lightweight/tests::MonotonicGenerator(-1)             14079 ns        13947 ns        50923
CodecSingleReadMinMaxUInt8LZ4/tests::MinMaxGenerator()                                      5160 ns         5072 ns       126251
CodecSingleReadMinMaxUInt8Lightweight/tests::MinMaxGenerator()                              5148 ns         5061 ns       139030
CodecSingleReadMinMaxUInt16LZ4/tests::MinMaxGenerator()                                     6869 ns         6780 ns       103018
CodecSingleReadMinMaxUInt16Lightweight/tests::MinMaxGenerator()                             7009 ns         6924 ns       103979
CodecSingleReadMinMaxUInt32LZ4/tests::MinMaxGenerator()                                     9654 ns         9561 ns        72951
CodecSingleReadMinMaxUInt32Lightweight/tests::MinMaxGenerator()                             9676 ns         9557 ns        70932
CodecSingleReadMinMaxUInt64LZ4/tests::MinMaxGenerator()                                    13169 ns        13048 ns        54162
CodecSingleReadMinMaxUInt64Lightweight/tests::MinMaxGenerator()                            14698 ns        14571 ns        48674
CodecSingleReadRandomUInt8LZ4/tests::RandomGenerator<UInt8>(0)                              4677 ns         4585 ns       152407
CodecSingleReadRandomUInt8Lightweight/tests::RandomGenerator<UInt8>(0)                      4603 ns         4523 ns       152982
CodecSingleReadRandomUInt16LZ4/tests::RandomGenerator<UInt16>(0)                            5819 ns         5733 ns       113951
CodecSingleReadRandomUInt16Lightweight/tests::RandomGenerator<UInt16>(0)                    5779 ns         5695 ns       121982
CodecSingleReadRandomUInt32LZ4/tests::RandomGenerator<UInt32>(0)                            8288 ns         8186 ns        85139
CodecSingleReadRandomUInt32Lightweight/tests::RandomGenerator<UInt32>(0)                    8230 ns         8115 ns        86273
CodecSingleReadRandomUInt64LZ4/tests::RandomGenerator<UInt64>(0)                           13574 ns        13440 ns        51936
CodecSingleReadRandomUInt64Lightweight/tests::RandomGenerator<UInt64>(0)                   13537 ns        13415 ns        50957
CodecSingleReadSmallRandomUInt8LZ4/tests::RandomGenerator<UInt8>(0, 0, 16)                  4873 ns         4794 ns       145066
CodecSingleReadSmallRandomUInt8Lightweight/tests::RandomGenerator<UInt8>(0, 0, 16)          4848 ns         4762 ns       147287
CodecSingleReadSmallRandomUInt16LZ4/tests::RandomGenerator<UInt16>(0, 0, 16)               17131 ns        16999 ns        41122
CodecSingleReadSmallRandomUInt16Lightweight/tests::RandomGenerator<UInt16>(0, 0, 16)       17083 ns        16935 ns        41527
CodecSingleReadSmallRandomUInt32LZ4/tests::RandomGenerator<UInt32>(0, 0, 16)               36512 ns        36317 ns        19243
CodecSingleReadSmallRandomUInt32Lightweight/tests::RandomGenerator<UInt32>(0, 0, 16)        6709 ns         6596 ns       106313
CodecSingleReadSmallRandomUInt64LZ4/tests::RandomGenerator<UInt64>(0, 0, 16)               36046 ns        35875 ns        20675
CodecSingleReadSmallRandomUInt64Lightweight/tests::RandomGenerator<UInt64>(0, 0, 16)        8597 ns         8497 ns        82801
CodecSingleReadRepeatUInt8LZ4/tests::RepeatGenerator<UInt8>(0)                              8006 ns         7908 ns        87967
CodecSingleReadRepeatUInt8Lightweight/tests::RepeatGenerator<UInt8>(0)                      8212 ns         8086 ns        74980
CodecSingleReadRepeatUInt16LZ4/tests::RepeatGenerator<UInt16>(0)                            9682 ns         9561 ns        72618
CodecSingleReadRepeatUInt16Lightweight/tests::RepeatGenerator<UInt16>(0)                    8012 ns         7915 ns        88283
CodecSingleReadRepeatUInt32LZ4/tests::RepeatGenerator<UInt32>(0)                           13848 ns        13722 ns        51416
CodecSingleReadRepeatUInt32Lightweight/tests::RepeatGenerator<UInt32>(0)                    7653 ns         7553 ns        89664
CodecSingleReadRepeatUInt64LZ4/tests::RepeatGenerator<UInt64>(0)                           17809 ns        17694 ns        39306
CodecSingleReadRepeatUInt64Lightweight/tests::RepeatGenerator<UInt64>(0)                    9487 ns         9375 ns        74912

From the microbenchmark, the write performance of each block will decrease at most (1/186654 - 1/220727) / (1/186654) * 100% = 15.4%

CodecSingleWriteRandomUInt32LZ4/tests::RandomGenerator<UInt32>(0)                         186654 ns        81931 ns         8467
CodecSingleWriteRandomUInt32Lightweight/tests::RandomGenerator<UInt32>(0)                 220727 ns       117468 ns         5987

And the read performance of each block will increase by 0 ~ 10x.

CodecSingleReadSequentialReverseUInt64LZ4/tests::SequentialGenerator(-2)                   70070 ns        69775 ns        10052
CodecSingleReadSequentialReverseUInt64Lightweight/tests::SequentialGenerator(-2)            6244 ns         6161 ns       113214

@Lloyd-Pottiger Lloyd-Pottiger force-pushed the enabling-lightweight branch 2 times, most recently from eebb240 to d0d4588 Compare August 20, 2024 03:08
@Lloyd-Pottiger Lloyd-Pottiger mentioned this pull request Aug 22, 2024
Closed
@ti-chi-bot ti-chi-bot bot added needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Aug 25, 2024
JaySon-Huang
JaySon-Huang reviewed Sep 5, 2024
View reviewed changes
dbms/src/IO/Compression/CompressionCodecDeflateQpl.h Outdated Show resolved Hide resolved
dbms/src/Interpreters/Settings.h Outdated
Comment on lines 217 to 218
M(SettingCompressionMethod, dt_compression_method, CompressionMethod::Lightweight, "The method of data compression when writing.") \
M(SettingInt64, dt_compression_level, 3, "The compression level.") \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better seperate this enabling by default in another PR

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK

tests/docker/config/tiflash_dt_lightweight_compression.toml Outdated Show resolved Hide resolved
tests/docker/config/tiflash_dt_lightweight_compression.toml Outdated Show resolved Hide resolved
tests/docker/config/tiflash_dt_lightweight_compression.toml Outdated Show resolved Hide resolved
Lloyd-Pottiger and others added 3 commits September 6, 2024 10:23
@Lloyd-Pottiger
Test lightweight (#14)
35e90b6 
* Support enable lightweight compression

Signed-off-by: Lloyd-Pottiger <[email protected]>
@Lloyd-Pottiger
remove deltalz4 & optimize deltafor
d27d1f1 
Signed-off-by: Lloyd-Pottiger <[email protected]>
@Lloyd-Pottiger
address comments
5a3d7e6 
Signed-off-by: Lloyd-Pottiger <[email protected]>
@Lloyd-Pottiger
fix level
35d4765 
Signed-off-by: Lloyd-Pottiger <[email protected]>
@Lloyd-Pottiger
fix level
17effd9 
Signed-off-by: Lloyd-Pottiger <[email protected]>
@Lloyd-Pottiger
fix comments
1d61d85 
Signed-off-by: Lloyd-Pottiger <[email protected]>
JaySon-Huang
JaySon-Huang reviewed Sep 6, 2024
View reviewed changes
Copy link
Contributor

@JaySon-Huang JaySon-Huang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rest LGTM

dbms/src/Interpreters/SettingsCommon.h Outdated Show resolved Hide resolved
dbms/src/IO/Compression/EncodingUtil.cpp Outdated Show resolved Hide resolved
dbms/src/IO/Compression/EncodingUtil.cpp Show resolved Hide resolved
dbms/src/IO/Compression/EncodingUtil.cpp Show resolved Hide resolved
@Lloyd-Pottiger
address comments
50fd258 
Signed-off-by: Lloyd-Pottiger <[email protected]>
Lloyd-Pottiger and others added 4 commits September 6, 2024 17:38
@Lloyd-Pottiger
add comments
3619535 
Signed-off-by: Lloyd-Pottiger <[email protected]>
@Lloyd-Pottiger
Merge branch 'master' into enabling-lightweight
1f882f3
@Lloyd-Pottiger
format
c8e3149 
Signed-off-by: Lloyd-Pottiger <[email protected]>
@Lloyd-Pottiger
format
2734f9e 
Signed-off-by: Lloyd-Pottiger <[email protected]>
@JinheLin
Copy link
Contributor

JinheLin commented Sep 6, 2024

Rest LGTM

@ti-chi-bot ti-chi-bot bot added needs-1-more-lgtm Indicates a PR needs 1 more LGTM. approved labels Sep 6, 2024
JaySon-Huang
JaySon-Huang approved these changes Sep 6, 2024
View reviewed changes
Copy link
Contributor

@JaySon-Huang JaySon-Huang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Sep 6, 2024
@ti-chi-bot ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Sep 6, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: JaySon-Huang, JinheLin

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [JaySon-Huang,JinheLin]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Sep 6, 2024

[LGTM Timeline notifier]

Timeline:

  • 2024-09-06 13:17:54.992451765 +0000 UTC m=+16744.732875702: ☑️ agreed by JinheLin.
  • 2024-09-06 13:27:03.609870244 +0000 UTC m=+17293.350294200: ☑️ agreed by JaySon-Huang.

@ti-chi-bot ti-chi-bot bot merged commit 627fd1f into pingcap:master Sep 6, 2024
5 checks passed
@Lloyd-Pottiger Lloyd-Pottiger deleted the enabling-lightweight branch September 7, 2024 10:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@JinheLin JinheLin JinheLin approved these changes

@JaySon-Huang JaySon-Huang JaySon-Huang approved these changes

@CalvinNeo CalvinNeo Awaiting requested review from CalvinNeo

Assignees
No one assigned
Labels
approved lgtm release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
3 participants
@Lloyd-Pottiger @JinheLin @JaySon-Huang