MPP Unit Test don't support semi join

[ywqzzy]

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

TEST_F(ComputeServerRunner, runFineGrainedShuffleJoinTest)
try
{
    startServers(3);
    constexpr size_t join_type_num = 7;
    constexpr tipb::JoinType join_types[join_type_num] = {
        tipb::JoinType::TypeInnerJoin,
        tipb::JoinType::TypeLeftOuterJoin,
        tipb::JoinType::TypeRightOuterJoin,
        tipb::JoinType::TypeSemiJoin,
        tipb::JoinType::TypeAntiSemiJoin,
        tipb::JoinType::TypeLeftOuterSemiJoin,
        tipb::JoinType::TypeAntiLeftOuterSemiJoin,
    };
    // fine-grained shuffle is enabled.
    constexpr uint64_t enable = 8;
    constexpr uint64_t disable = 0;

    for (auto join_type : join_types)
    {
        auto properties = DB::tests::getDAGPropertiesForTest(serverNum());
        auto request = context
                           .scan("test_db", "l_table_2")
                           .join(context.scan("test_db", "r_table_2"), join_type, {col("s1"), col("s2")}, disable)；
               
        const auto expected_cols = buildAndExecuteMPPTasks(request);

        auto request2 = context
                            .scan("test_db", "l_table_2")
                            .join(context.scan("test_db", "r_table_2"), join_type, {col("s1"), col("s2")}, enable)；
                           
        auto tasks = request2.buildMPPTasks(context, properties);
        for (auto & task : tasks)
        {
            setFineGrainedShuffleForExchange(const_cast<tipb::Executor &>(task.dag_request->root_executor()));
        }
        const auto actual_cols = executeMPPTasks(tasks, properties, MockComputeServerManager::instance().getServerConfigMap());
        ASSERT_COLUMNS_EQ_UR(expected_cols, actual_cols);
    }
}
CATCH

2. What did you expect to see? (Required)

Test pass

3. What did you see instead (Required)

Abort

4. What is your TiFlash version? (Required)

[ywqzzy]

The mpp task test called function columnPrune, then the columnPrune remove the columns that need to use for semi join.

[ywqzzy]

There are 3 ways to solve it:

1. disable column prune for semi join.
2. add extra projection for the test to make sure all columns are exist in MockTableScan.
3. refine the column prune process for semi join in UT framework.

[ywqzzy]

There are 3 ways to solve it:

1. disable column prune for semi join.
2. add extra projection for the test to make sure all columns are exist in MockTableScan.
3. refine the column prune process for semi join in UT framework.

The second way could be as follows:

TEST_F(ComputeServerRunner, runFineGrainedShuffleJoinTest)
try
{
    startServers(3);
    constexpr size_t join_type_num = 7;
    constexpr tipb::JoinType join_types[join_type_num] = {
        tipb::JoinType::TypeInnerJoin,
        tipb::JoinType::TypeLeftOuterJoin,
        tipb::JoinType::TypeRightOuterJoin,
        tipb::JoinType::TypeSemiJoin,
        tipb::JoinType::TypeAntiSemiJoin,
        tipb::JoinType::TypeLeftOuterSemiJoin,
        tipb::JoinType::TypeAntiLeftOuterSemiJoin,
    };
    // fine-grained shuffle is enabled.
    constexpr uint64_t enable = 8;
    constexpr uint64_t disable = 0;

    for (auto join_type : join_types)
    {
        auto properties = DB::tests::getDAGPropertiesForTest(serverNum());
        auto request = context
                           .scan("test_db", "l_table_2")
                           .join(context.scan("test_db", "r_table_2"), join_type, {col("s1"), col("s2")}, disable)
                           .project({col("l_table_2.s1"), col("l_table_2.s2"), col("l_table_2.s3")});
        const auto expected_cols = buildAndExecuteMPPTasks(request);

        auto request2 = context
                            .scan("test_db", "l_table_2")
                            .join(context.scan("test_db", "r_table_2"), join_type, {col("s1"), col("s2")}, enable)
                            .project({col("l_table_2.s1"), col("l_table_2.s2"), col("l_table_2.s3")});
        auto tasks = request2.buildMPPTasks(context, properties);
        for (auto & task : tasks)
        {
            setFineGrainedShuffleForExchange(const_cast<tipb::Executor &>(task.dag_request->root_executor()));
        }
        const auto actual_cols = executeMPPTasks(tasks, properties, MockComputeServerManager::instance().getServerConfigMap());
        ASSERT_COLUMNS_EQ_UR(expected_cols, actual_cols);
    }
}
CATCH

[zanmato1984]

It's not a bug (it's a XXX!). Adjusting label.

[ywqzzy]

It's not a bug (it's a XXX!). Adjusting label.

Get!