perf(streaming): compact useless intermediate result in stream chunk

[st1page]

Background

Some streaming operators can produce useless intermediate results in a stream chunk. And because under out process model, we can compact ops with the same stream key in the same epoch. We can reduce them. Even for those slightly reducing could matters before some other operators with amplification such as join and HopWindow( 2x amplification does not matter but 100x amplification is far better than 200x). here are some examples.

1. Project update some field

create table t(a int, b int, k int primiary key);
create materialize view mv as select a, k from t;
insert into t values (1, 1, 1), (2, 2, 2);
UPDATE SET b = 10 FROM T;

even the update will not change the value of the mv, the changes will still be sent to the down stream (IIRC it is found by @TennyZhuang )

2. In discussion with some users, the first case can appear in some source or cdc connector(update a extra filed and give update with the same old value and new value)

3. outer join
   if outer join does not do any optimization. It will amplify anyway. the outer's two op (U-, old),(U+,new) change to (U-, old), (U+,NULL), (U-, NULL), (U+,new)

feature

we should have common utils to compact ops with the same stream key in the same epoch, even only on the chunk based. It should only change the visibility and ops to prevent unnecessary over head

Tracking

• feat(stream): compact stream chunk by dropping useless rows in sink #11070
• do check on adjacent operations to compact useless updates in stream scan #13409.

[xx01cyx]

#10853 is actually a case of 2 and can be resolved by this. cc @fuyufjh