panic: The CommitTs must be greater than the resolvedTs

[amyangfei]

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. What did you do? If possible, provide a recipe for reproducing the error.

   The same test environment and test procedure with https://github.com/pingcap/ticdc/issues/1599

2. What did you expect to see?

   replication normally

3. What did you see instead?

panic: The CommitTs must be greater than the resolvedTs

goroutine 1368 [running]:
go.uber.org/zap/zapcore.(*CheckedEntry).Write(0xc0004fe180, 0xc00fb98b00, 0x4, 0x4)
        go.uber.org/[email protected]/zapcore/entry.go:234 +0x585
go.uber.org/zap.(*Logger).Panic(0xc002f197a0, 0x2c5a49b, 0x30, 0xc00fb98b00, 0x4, 0x4)
        go.uber.org/[email protected]/logger.go:226 +0x85
github.com/pingcap/log.Panic(0x2c5a49b, 0x30, 0xc00fb98b00, 0x4, 0x4)
        github.com/pingcap/[email protected]/global.go:50 +0x10b
github.com/pingcap/ticdc/cdc/kv.(*eventFeedSession).singleEventFeed(0xc00313a900, 0x30b7b20, 0xc0034c21c0, 0x74, 0xc0036143e0, 0x12, 0x12, 0xc003614400, 0x12, 0x12, ...)
        github.com/pingcap/ticdc/cdc/kv/client.go:1503 +0x1ba2
github.com/pingcap/ticdc/cdc/kv.(*eventFeedSession).partialRegionFeed(0xc00313a900, 0x30b7b20, 0xc0034c21c0, 0xc005f7a0b0, 0xc000a5aa50, 0x0, 0x0)
        github.com/pingcap/ticdc/cdc/kv/client.go:883 +0x12b
github.com/pingcap/ticdc/cdc/kv.(*eventFeedSession).sendRegionChangeEvent.func1(0x4417d6, 0x2d3ed28)
        github.com/pingcap/ticdc/cdc/kv/client.go:1248 +0x4e
golang.org/x/sync/errgroup.(*Group).Go.func1(0xc000f88420, 0xc00d767410)
        golang.org/x/[email protected]/errgroup/errgroup.go:57 +0x59
created by golang.org/x/sync/errgroup.(*Group).Go
        golang.org/x/[email protected]/errgroup/errgroup.go:54 +0x66

4. Versions of the cluster

   • Upstream TiDB cluster version (execute SELECT tidb_version(); in a MySQL client):

TiKV
Release Version:   5.0.0-rc.x
Edition:           Community
Git Commit Hash:   21e73d6d9403e54e4ee635cfe0b37f4e944e1bc6
Git Commit Branch: zixiong-ticdc-flow-control-pr-prepare
UTC Build Time:    2021-03-25 07:07:54
Rust Version:      rustc 1.51.0-nightly (bc39d4d9c 2021-01-15)
Enable Features:   jemalloc mem-profiling portable sse protobuf-codec test-engines-rocksdb cloud-aws cloud-gcp
Profile:           release

- TiCDC version (execute `cdc version`):

master@f86e3022303d2a09e9158439b8a5b84618f0c095

[amyangfei]

This is more likely a bug from temporary TiKV version and we haven't reproduced it for a long time, close the issue now