DM(syncer): syncer may lost dml under special sharding ddls

[WizardXiao]

What did you do?

1. open gtid
2. pessimistic shard mode
3. two sub-table in one source
4. sql like this:
   tb1 --> ddl1
   tb1 --> dml (lost)
   tb1 --> ddl2
   tb2 --> ddl1
   tb2 --> ddl2

What did you expect to see?

Don't lost tb1's dml.

What did you see instead?

lost tb1's dml

Versions of the cluster

DM version (run dmctl -V or dm-worker -V or dm-master -V):

v5.3 --> master

Upstream MySQL/MariaDB server version:

5.7.22-log

Downstream TiDB cluster version (execute SELECT tidb_version(); in a MySQL client):

5.7.25-TiDB-v5.3.0

How did you deploy DM: tiup or manually?

manually

Other interesting information (system version, hardware config, etc):

>
>

current status of DM cluster (execute query-status <task-name> in dmctl)

(paste current status of DM cluster here)

[GMHDBJD]

ping @lance6716

[lance6716]

If this bug need not to be cherry-pick to earlier version, I guess it's earier to fix when use

tiflow/dm/syncer/binlog_locations.go

Lines 30 to 45 in c48f208

	type locationRecorder struct {
	// +-------------+
	// ... |current event| ...
	// ^ +-------------+ ^
	// | |
	// curStartLocation curEndLocation
	// there may be more events between curStartLocation and curEndLocation due to the limitation of binlog or
	// implementation of DM, but in such scenario, those events should always belong to one transaction.
	// When curStartLocation is equal to curEndLocation, it means current event is not a data change.
	//
	// curStartLocation is used when
	// - display a meaningful location
	// - match the injected location by handle-error
	// - save table checkpoint of DML
	// curEndLocation is used when
	// - handle end location of DDL, when save table checkpoint or shard-resync

😂 not likely that we don't cherry-pick

[GMHDBJD]

😂 not likely that we don't cherry-pick

/(ㄒoㄒ)/~~

[WizardXiao]

After tb1-ddl1 and tb2-ddl1 has been resolved, tb1-dml will be synced, and we will check wether tb1-dml < tb1-ddl2 (active ddl), but caculate tb1-ddl2's position use last_position‘s gtid which is the same as tb1-dml, when use gtid, we will filter tb1-dml