Wait for master until a timeout

[filonenko-mikhail]

• Setup cartridge cluster with failover
• Setup separated crud-router and crud-storage
• Make a write load
• Switch master under load
• One time

LuajitError: ...rk/starwars/.rocks/share/tarantool/crud/common/utils.lua:35: attempt to index field 'master' (a nil value)

[Totktonada]

It seems, it was crud-0.0.4. The relevant code is the following:

crud/crud/common/utils.lua

Lines 33 to 37 in 1fa7ffc

	function utils.get_space(space_name, replicasets)
	local replicaset = select(2, next(replicasets))
	local space = replicaset.master.conn.space[space_name]
	return space
	end

The code remain unchanged since 0.0.4.

It seems logical that there is a time frame, when the old master is already demoted, but the new one is not promoted yet. Also I see that vshard contains 'whether there is a master?' checks that confirms my guess.

We should not lean on master presence unconditionally and should report a proper error if there is no master. The error should give a user hint that it is a transient error and the request can be safely retried.

BTW, crud has no autoretries / retry strategy settings?

[Totktonada]

BTW, as I see, vshard's call automatically waits for master (until a timeout). It seems, it would be good to use the same approach.

NB: Don't forget to substract time we waiting for a space from net.box request timeout.