[WIP][db2] The first draft of DB2-CDC #450
Conversation
| tEnv.executeSql(sourceDDL); | ||
| tEnv.executeSql(sinkDDL); | ||
|
|
||
| // async submit job |
There was a problem hiding this comment.
Why do we use this group by query for verification? Would not it be clearer to use only a few db updates (let's say 2 inserts, 1 update, 1 delete) and check that the related changes are captured? For me it seems a bit complicated to verify the output of this test.
There was a problem hiding this comment.
The reason I have come up with this is because for me it seems that debezium produces multiple updates for a simple insert query. Could you please double check this change I have applied on top of yours?
Could not figure it out why the second update shows up.
kisstom@bf7fe76
Also in the debezium log I can see the following, suggesting that it sends two records instead of one :
[debezium-engine] INFO io.debezium.connector.common.BaseSourceTask - 2 records sent during previous 00:00:10.16, last recorded offset: {transaction_id=null, event_serial_no=1, commit_lsn=00000000:00001581:0000000000046f82, change_lsn=00000000:00000000:0000000004412ada}
There was a problem hiding this comment.
Sorry for the late reply @kisstom , I understand this test is mainly for checking
- The source cdc can receive changelog
- Flink SQL can process these changelog well
There was a problem hiding this comment.
it seems that debezium produces multiple updates for a simple insert query.
It looks strange, are you sure it is like this？
|
go go go！ |
|
希望2.2支持DB2-CDC |
|
go go go！Hope 2.2 support db2 cdc！ |
|
@leonardBang Hello, do you need to compile and configure DB2 server-side UDF to use this? |
|
Hey @leonardBang, why is this PR still in WIP and not merged back to master? Based on this work we were able to perform CDC for Db2 and it looked fine. |
Hey @kisstom. The priority of this PR is relatively low. From the issue list, there are very few user requirement for db2 cdc. Thus we spent effort on other database support like SqlServer and TiDB. |
|
|
The first draft that offers DB2-CDC to capture DB2 table changes.