test: Add tests case when deltaMergeBySplit + { flushCache | split } occurs simultaneously (#5454)

[ti-chi-bot]

This is an automated cherry-pick of #5454

What problem does this PR solve?

Issue Number: ref #5409

Problem Summary: For now we lack some tests to test the behavior when multiple operations occurred at the same time.

What is changed and how it works?

1. Introduced a new facility called SyncPoint, which can be used to control the arbitrary business logical flow to suspend and continue.

2. Added or modified existing ManualCompaction tests based on the new SyncPoint:

   • Modified: BasicManualCompactTest/DuplicatedLogicalId

     Previously this test uses sleep(500ms) to expect specific code logic is finished. Now this sleep is substituted with SyncPoint, so that expected code logic is always finished and the test can continue without waiting for useless time.

   • Added: DeltaMergeStoreMergeDeltaBySegmentTest/RetryByFlushCache

   • Added: DeltaMergeStoreMergeDeltaBySegmentTest/RetryBySplit

     The two new cases above tests whether everything works fine when there are mergeDeltaBySplit and flushCache (or split) happening simultaneously, based on the SyncPoint.

3. Added more guards in gtest_main:

   • Failures are now turned into exceptions by default, so that when writing ASSERT in sub-functions it will break the test immediately. This is particularly useful in SyncPoint tests, as when there are failures in previous sync point steps, it is very likely that the rest step will not work as expected, result in infinite wait. By turning into exceptions, we these test cases fail fast.

   • Added stack output when the program is unexpected terminated in fault signals like SIG_ABORT (and also std::terminate). This helps locate the code of triggering these faults.

Check List

Tests

• Unit test
• Integration test
• Manual test (add detailed scripts or steps below)
• No code

Side effects

• Performance regression: Consumes more CPU
• Performance regression: Consumes more Memory
• Breaking backward compatibility

Documentation

• Affects user behaviors
• Contains syntax changes
• Contains variable changes
• Contains experimental features
• Changes MySQL compatibility

Release note

None

[ti-chi-bot]

[REVIEW NOTIFICATION]

This pull request has been approved by:

• breezewish
• lidezhu

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

[breezewish]

/run-all-tests

[breezewish]

Note: a test case in #5454 is disabled in this cherry pick: c522c2b

As 6da631c is not cherry picked, and flush will actually block each other instead of fail fast immediately.

[sre-bot]

Coverage for changed files

Filename                                                     Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Common/FailPoint.cpp                                             452                99    78.10%           6                 0   100.00%          56                 8    85.71%         150                62    58.67%
Common/SyncPoint/Ctl.cpp                                         132                31    76.52%           7                 0   100.00%          62                 4    93.55%          58                33    43.10%
Common/SyncPoint/Ctl.h                                             2                 0   100.00%           2                 0   100.00%           8                 0   100.00%           0                 0         -
Common/SyncPoint/ScopeGuard.cpp                                    8                 1    87.50%           5                 0   100.00%          18                 1    94.44%           2                 1    50.00%
Common/SyncPoint/ScopeGuard.h                                      1                 0   100.00%           1                 0   100.00%           3                 0   100.00%           0                 0         -
Common/SyncPoint/SyncChannel.h                                    23                 4    82.61%           9                 0   100.00%          82                 2    97.56%          16                 8    50.00%
Flash/Management/ManualCompact.cpp                               134                54    59.70%           9                 0   100.00%         166                23    86.14%          66                32    51.52%
Flash/Management/tests/gtest_manual_compact.cpp                  154                78    49.35%          16                 0   100.00%         246                 4    98.37%          54                45    16.67%
Storages/DeltaMerge/Delta/ColumnFileFlushTask.cpp                 38                 3    92.11%           3                 0   100.00%          60                 5    91.67%          22                 3    86.36%
Storages/DeltaMerge/DeltaMergeStore.cpp                         1469               511    65.21%          67                 5    92.54%        2053               464    77.40%         846               389    54.02%
Storages/DeltaMerge/Segment.cpp                                  689               212    69.23%          47                 3    93.62%        1163               266    77.13%         364               195    46.43%
Storages/DeltaMerge/tests/MultiSegmentTestUtil.h                  43                 3    93.02%           6                 0   100.00%          84                 0   100.00%          12                 3    75.00%
Storages/DeltaMerge/tests/gtest_dm_delta_merge_store.cpp        1181               323    72.65%          52                 4    92.31%        3052               151    95.05%         576               188    67.36%
TestUtils/TiFlashTestBasic.h                                       4                 0   100.00%           2                 0   100.00%          12                 0   100.00%           2                 0   100.00%
TestUtils/gtests_dbms_main.cpp                                     7                 4    42.86%           4                 2    50.00%          32                12    62.50%           4                 2    50.00%

Files which contain no functions:
Common/SyncPoint/SyncPoint.h                                       0                 0         -           0                 0         -           0                 0         -           0                 0         -
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                                           4337              1323    69.50%         236                14    94.07%        7097               940    86.75%        2172               961    55.76%

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
18291      9784             46.51%    205180  98021        52.23%

full coverage report (for internal network access only)

[breezewish]

/merge

[ti-chi-bot]

@breezewish: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

[ti-chi-bot]

This pull request has been accepted and is ready to merge.

Commit hash: c522c2b

[sre-bot]

Coverage for changed files

Filename                                                     Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Common/FailPoint.cpp                                             452                99    78.10%           6                 0   100.00%          56                 8    85.71%         150                62    58.67%
Common/SyncPoint/Ctl.cpp                                         132                31    76.52%           7                 0   100.00%          62                 4    93.55%          58                33    43.10%
Common/SyncPoint/Ctl.h                                             2                 0   100.00%           2                 0   100.00%           8                 0   100.00%           0                 0         -
Common/SyncPoint/ScopeGuard.cpp                                    8                 1    87.50%           5                 0   100.00%          18                 1    94.44%           2                 1    50.00%
Common/SyncPoint/ScopeGuard.h                                      1                 0   100.00%           1                 0   100.00%           3                 0   100.00%           0                 0         -
Common/SyncPoint/SyncChannel.h                                    23                 4    82.61%           9                 0   100.00%          82                 2    97.56%          16                 8    50.00%
Flash/Management/ManualCompact.cpp                               134                54    59.70%           9                 0   100.00%         166                23    86.14%          66                32    51.52%
Flash/Management/tests/gtest_manual_compact.cpp                  154                78    49.35%          16                 0   100.00%         246                 4    98.37%          54                45    16.67%
Storages/DeltaMerge/Delta/ColumnFileFlushTask.cpp                 38                 3    92.11%           3                 0   100.00%          60                 5    91.67%          22                 3    86.36%
Storages/DeltaMerge/DeltaMergeStore.cpp                         1469               511    65.21%          67                 5    92.54%        2053               464    77.40%         846               387    54.26%
Storages/DeltaMerge/Segment.cpp                                  689               212    69.23%          47                 3    93.62%        1163               266    77.13%         364               195    46.43%
Storages/DeltaMerge/tests/MultiSegmentTestUtil.h                  43                 3    93.02%           6                 0   100.00%          84                 0   100.00%          12                 3    75.00%
Storages/DeltaMerge/tests/gtest_dm_delta_merge_store.cpp        1181               323    72.65%          52                 4    92.31%        3052               151    95.05%         576               188    67.36%
TestUtils/TiFlashTestBasic.h                                       4                 0   100.00%           2                 0   100.00%          12                 0   100.00%           2                 0   100.00%
TestUtils/gtests_dbms_main.cpp                                     7                 4    42.86%           4                 2    50.00%          32                12    62.50%           4                 2    50.00%

Files which contain no functions:
Common/SyncPoint/SyncPoint.h                                       0                 0         -           0                 0         -           0                 0         -           0                 0         -
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                                           4337              1323    69.50%         236                14    94.07%        7097               940    86.75%        2172               959    55.85%

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
18291      9784             46.51%    205180  98032        52.22%

full coverage report (for internal network access only)