roachtest: add another error to jepsen allowlist

[tbg]

Release note: None

[cockroach-teamcity]

This change is 

[andreimatei]

Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @andreimatei and @tbg)

---

pkg/cmd/roachtest/jepsen.go, line 253 at r1 (raw file):

				// lock=true stat=PENDING rts=1592384065.123218152 ,0 wto=false
				// max=1592384065.128230720,0
				`-e "TransactionRetryWithProtoRefreshError"`,

mmm for this one it might be worth asking ben if it's easy to put a retry to whatever schema change causes it

[tbg]

cc @bdarnell

[bdarnell]

Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @andreimatei)

---

pkg/cmd/roachtest/jepsen.go, line 253 at r1 (raw file):

Previously, andreimatei (Andrei Matei) wrote…

mmm for this one it might be worth asking ben if it's easy to put a retry to whatever schema change causes it

It's really important that simple schema changes like the ones jepsen does not be subject to random retry errors like this. We've put a lot of work into this in the past. Have we had a regression in this area? (ABORT_REASON_NEW_LEASE_PREVENTS_TXN is new to me)

[andreimatei]

I was complaining about some apparent regression in the area in #37751 (comment)
Don't know if it's exactly what we're seeing here or not... Maybe not.
The question here is why didn't automatic retries kick in, if Jepsen only does simple things.

The underlying reasons for ABORT_REASON_NEW_LEASE_PREVENTS_TXN we've always had. The reason code is new because I've carved out the case where the timestamp cache reset was caused by a new lease, as opposed to other reasons.

[bdarnell]

The question here is why didn't automatic retries kick in, if Jepsen only does simple things.

Yeah. Specifically, I'm pretty sure jepsen only does its schema changes in implicit transactions (or maybe as the first and only statement in an explicit transaction) so it should be retried on the server side.

Assuming this was discovered in #49360, multiregister's DDL statements are trivial

[tbg]

Yes, it was discovered there. Not frequently - a few times over >400 runs.

[tbg]

Do we even know that this was a schema change? Could the error have come from a regular txn operation in jepsen or would they not surface to that level?

[bdarnell]

During normal operation jepsen catches nearly all errors. Most of the time when an error bubbles up to this level, it's from a schema change. But the jepsen logs should tell us exactly where it's coming from. Do you have a pointer to a log showing this problem?