vtctlclient SwitchReads with -dry_run results in unexpected lock errors

[aquarapid]

Version: git hash 68f6713 (7/24/2020)

Scenario:

• Run the local/examples up to (but not including) 304_switch_reads.sh
• Now, run SwitchReads from 304_switch_reads.sh manually, but with the -dry_run flag, i.e.:

$ vtctlclient -server localhost:15999 SwitchReads -dry_run -tablet_type=rdonly customer.cust2cust
E0727 09:57:37.837502   26799 main.go:64] E0727 16:57:37.837152 traffic_switcher.go:168] switchShardReads failed: Code: INVALID_ARGUMENT
keyspace customer is not locked (no locksInfo)
E0727 09:57:37.838051   26799 main.go:67] remote error: rpc error: code = Unknown desc = keyspace customer is not locked (no locksInfo)

• Note the locksInfo error

This seems to be expected, since we cannot take the lock in dry-run mode; but can be confusing (and scary) to users.

Expected behavior: Should run without errors if everything checks out

[rohit-nayak-ps]

Fixed via #6497

[vahe]

@rohit-nayak-ps @aquarapid It seems like the fix #6497 was never merged. We just came across the same bug.

[mattlord]

@rohit-nayak-ps @aquarapid It seems like the fix #6497 was never merged. We just came across the same bug.

Hi @vahe !

Can you please provide the command you ran and the output you saw? I know that since this we've done work in the dry run to ensure that the actual command would likely work in #10058. So I want to be sure you're seeing the same issue mentioned here and not simply a dry run failing / returning an error (which is valid).

Thanks!

[vahe]

@mattlord We ran:

vtctlclient Reshard -- --dry_run --tablet_types='rdonly,replica' switchtraffic ourkeyspacename.users2shards

Output:

E0210 17:12:33.599764    1223 main.go:94] E0210 17:12:33.599198 traffic_switcher.go:373] switchShardReads failed: Code: INVALID_ARGUMENT
keyspace ourkeyspacename is not locked (no locksInfo)
E0210 17:12:33.600623    1223 main.go:94] E0210 17:12:33.599365 vtctl.go:2433]
keyspace ourkeyspacename is not locked (no locksInfo)

We're on v14. It looks like the code is still there even on current main. Namely the call to switchShardReads should be on sw instead of ts: https://github.com/vitessio/vitess/pull/6497/files#diff-a071a972abd45af831ed93cc6447a87b83cee112f1c7d3bb0bdd592979d80c91R167.

[rohit-nayak-ps]

fixed via #12992