unit-test failed

[zz-jason]

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. What did you do?

found in #11469

https://internal.pingcap.net/idc-jenkins/job/tidb_ghpr_unit_test/4644/display/redirect

2. What did you expect to see?

It should pass stably.

3. What did you see instead?

In the log file internal.pingcap.net.txt, I see an assertion failure:

[2019-07-29T01:37:24.845Z] [2019/07/29 09:36:46.300 +08:00] [ERROR] [mvcc_leveldb.go:767] ["ASSERTION FAIL!!!"] [mutation="op:Del key:\"t\\200\\000\\000\\000\\000\\000\\000\\031_r\\200\\000\\000\\000\\000\\000\\000\\025\" assertion:Exist "] [stack="github.com/pingcap/tidb/store/       mockstore/mocktikv.prewriteMutation
    /home/jenkins/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/store/mockstore/mocktikv/mvcc_leveldb.go:767
github.com/pingcap/tidb/store/mockstore/mocktikv.(*MVCCLevelDB).Prewrite
    /home/jenkins/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/store/mockstore/mocktikv/mvcc_leveldb.go:658
github.com/pingcap/tidb/store/mockstore/mocktikv.(*rpcHandler).handleKvPrewrite
    /home/jenkins/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/store/mockstore/mocktikv/rpc.go:295
github.com/pingcap/tidb/store/mockstore/mocktikv.(*RPCClient).SendRequest
    /home/jenkins/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/store/mockstore/mocktikv/rpc.go:694
github.com/pingcap/tidb/store/tikv.(*RegionRequestSender).sendReqToRegion
    /home/jenkins/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/store/tikv/region_request.go:145
github.com/pingcap/tidb/store/tikv.(*RegionRequestSender).SendReqCtx
    /home/jenkins/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/store/tikv/region_request.go:116
github.com/pingcap/tidb/store/tikv.(*RegionRequestSender).SendReq
    /home/jenkins/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/store/tikv/region_request.go:72
github.com/pingcap/tidb/store/tikv.(*tikvStore).SendReq
    /home/jenkins/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/store/tikv/kv.go:367
github.com/pingcap/tidb/store/tikv.(*twoPhaseCommitter).prewriteSingleBatch
    /home/jenkins/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/store/tikv/2pc.go:497
github.com/pingcap/tidb/store/tikv.(*twoPhaseCommitter).doActionOnBatches
    /home/jenkins/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/store/tikv/2pc.go:385
github.com/pingcap/tidb/store/tikv.(*twoPhaseCommitter).doActionOnKeys
    /home/jenkins/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/store/tikv/2pc.go:361
github.com/pingcap/tidb/store/tikv.(*twoPhaseCommitter).prewriteKeys
    /home/jenkins/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/store/tikv/2pc.go:830
github.com/pingcap/tidb/store/tikv.(*twoPhaseCommitter).execute
    /home/jenkins/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/store/tikv/2pc.go:890
github.com/pingcap/tidb/store/tikv.(*twoPhaseCommitter).executeAndWriteFinishBinlog
    /home/jenkins/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/store/tikv/2pc.go:850
github.com/pingcap/tidb/store/tikv.(*tikvTxn).Commit
    /home/jenkins/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/store/tikv/txn.go:297
github.com/pingcap/tidb/session.(*TxnState).Commit
    /home/jenkins/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/session/txn.go:197
github.com/pingcap/tidb/session.(*session).doCommit
    /home/jenkins/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/session/session.go:392
github.com/pingcap/tidb/session.(*session).doCommitWithRetry
    /home/jenkins/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/session/session.go:402
github.com/pingcap/tidb/session.(*session).CommitTxn
    /home/jenkins/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/session/session.go:467
github.com/pingcap/tidb/session.finishStmt
    /home/jenkins/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/session/tidb.go:172
github.com/pingcap/tidb/session.runStmt
    /home/jenkins/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/session/tidb.go:240
github.com/pingcap/tidb/session.(*session).executeStatement
    /home/jenkins/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/session/session.go:957
github.com/pingcap/tidb/session.(*session).execute
    /home/jenkins/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/session/session.go:1066
github.com/pingcap/tidb/session.(*session).Execute
    /home/jenkins/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/session/session.go:994
github.com/pingcap/tidb/ddl/util.RemoveFromGCDeleteRange
    /home/jenkins/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/ddl/util/util.go:118
github.com/pingcap/tidb/ddl/util.CompleteDeleteRange
    /home/jenkins/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/ddl/util/util.go:112
github.com/pingcap/tidb/ddl.(*delRange).doTask
    /home/jenkins/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/ddl/delete_range.go:223
github.com/pingcap/tidb/ddl.(*delRange).doDelRangeWork
    /home/jenkins/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/ddl/delete_range.go:177
github.com/pingcap/tidb/ddl.(*delRange).startEmulator
    /home/jenkins/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/ddl/delete_range.go:141"]^M

4. What version of TiDB are you using (tidb-server -V or run select tidb_version(); on TiDB)?

[winkyao]

@amyangfei is investigating this issue.

[amyangfei]

the error message is

[2019-07-29T01:40:51.374Z] 
[2019-07-29T01:40:51.374Z] ----------------------------------------------------------------------
[2019-07-29T01:40:51.374Z] FAIL: http_handler_test.go:548: HTTPHandlerTestSuite.TestGetSchema
[2019-07-29T01:40:51.374Z] 
[2019-07-29T01:40:51.374Z] http_handler_test.go:589:
[2019-07-29T01:40:51.374Z]     c.Assert(lt[0].Name.L, Equals, "test")
[2019-07-29T01:40:51.374Z] ... obtained string = "pt"
[2019-07-29T01:40:51.374Z] ... expected string = "test"
[2019-07-29T01:40:51.374Z] 
[2019-07-29T01:40:51.374Z] [2019/07/29 09:40:23.836 +08:00] [INFO] [http_status.go:263] ["listen failed"] [error="http: Server closed"]

[jackysp]

#11495 fixed it.

[jackysp]

@tiancaiamao It seems "ASSERTION FAIL!!!" appeared again. PTAL

[sre-bot]

Dear contributor, the first Performance challenge program has been closed. But you can still contribute to this issue and we appreciate your contribution.

[jackysp]

Dear contributor, the first Performance challenge program has been closed. But you can still contribute to this issue and we appreciate your contribution.

???