binlog_syncer: reduce the times of Clone of MySQLGTIDSet to speed up #746
Conversation
GMHDBJD
commented
Nov 29, 2022
GMHDBJD
commented
- add DiscardGTIDSet to allow user not to maintain Gset for QueryEvent/XIDEvent
- add prevMySQLGTIDEvent to calculate prevGSet instead of clone currGSet
| case *MariadbGTIDEvent: | ||
| if b.prevGset == nil { | ||
| break | ||
| } | ||
| GTID := event.GTID | ||
| err := advanceCurrentGtidSet(uuid.Nil, 0, GTID.DomainID, GTID.ServerID, GTID.SequenceNumber) | ||
| prev := b.currGset.Clone() |
There was a problem hiding this comment.
why we don't do it for MariaDB?
There was a problem hiding this comment.
OK seems MariaDB has a simpler representation of GTID, we can leave this improvement to future when we found a bottleneck
| @@ -120,6 +120,8 @@ type BinlogSyncerConfig struct { | |||
| Dialer client.Dialer | |||
|
|
|||
| RowsEventDecodeFunc func(*RowsEvent, []byte) error | |||
|
|
|||
| DiscardGTIDSet bool | |||
There was a problem hiding this comment.
I think the DiscardGTIDSet should also skip maintaining prevGset and currGset, only skipping output is no benefit to performance
There was a problem hiding this comment.
prevGset and currGset need to maintain for reconnect. Skip output reduce the clone operation
| b.currGset.(*MysqlGTIDSet).AddGTID(u, event.GNO) | ||
| if b.prevMySQLGTIDEvent != nil { | ||
| u, _ = uuid.FromBytes(b.prevMySQLGTIDEvent.SID) | ||
| b.prevGset.(*MysqlGTIDSet).AddGTID(u, b.prevMySQLGTIDEvent.GNO) |
There was a problem hiding this comment.
should we also consider the case "right after reconnect we will see same gtid as we saw before, thus currGset will not get changed"? (but I don't know when it will happen)
There was a problem hiding this comment.
That won't happen, because we will leave the currSet empty when we retry, and the mariadb comment will just stay as it is.
go-mysql/replication/binlogsyncer.go
Line 664 in 7685eee
