IsAcquired not getting the lock sometimes #32
Comments
|
That's odd - are you able to check redis while this is occurring and see:
If the key doesn't exist, but redlock fails to get a lock, that would possibly indicate a communication or timeout issue with redis, although you should see something in the logs if you're logging at debug level. Given that the lock is unavailable for 20 mins to 2 hours, it seems likely that either another process is holding the lock during that period (if it was an abandoned lock it should expire after the timeout - 60 seconds in your case). Also, you don't need to explicitly dispose the redis lock after the |
|
Is there a way to know programatically who is holding the lock. If so, is there a way to force release the lock? |
|
There isn't currently a way to programmatically know who has the lock (although I suppose RedLock could be altered to include some identifying information in the lock value). And there isn't a way to programatically force release a lock from within RedLock (because that's generally a bad idea). You could just delete the redis key though (however, if it exists then presumably something created it, or is keeping it alive, so it would probably be best to try and figure out why it's still there). |
|
I did a trace for a lockid 06935f4d-e327-467a-9924-e648ef74e279. Even if the key is acquired on the 1st attempt, redlock is trying again to get it and finally giving up. I am puzzled why it should do so if it acquires the lock on 1st attempt? |
|
Those logs seem to be ordered quite strange - descending by timestamp, but in ascending blocks for each second? At any rate, if I'm reading them correctly it appears that it tries repeatedly (but fails) to get a lock until eventually it managed to do so at 09:32:45.000 ( Do you know whether the redis server was available during the time it was failing, and if so, whether there was already an existing key Are you able to supply some code that reliably reproduces the problem? |
|
Yes, Redis is available all the time. This is our prod environment and I can never reproduce in Test or Dev. The logs are just in descending order. Yes, there is a key myKey in Redis, which the application acquires to lock Redis to perform cache manipulation. |
|
Those logs appear to be consistent with the key in redis already existing when the attempt to acquire the lock occurs. When you call If a key with that name does already exist, it will retry up to two more times with a small random sleep in between (to avoid two processes racing for the same lock continually blocking each other), before declaring that it was not able to acquire the lock. Since you're doing a blocking wait in this case ( The key value (lock id) that it uses (e.g. In order to solve this I think you need to find out what is holding the redis key during the time that you aren't able to acquire the lock. Some possibilities:
|
|
If you're still having trouble with this, I've pushed a pre-release package version See #24 for more details. |
doesnt it suppose to lock the same key i want to write to from another process ? from on process i read from key and do some work, from another process i write to the same key |
|
The key is used to lock a particular resource for one process only. If one process holds the lock key in redis, another process attempting to acquire a lock with the same key will fail (or be blocked until it is available). |
|
Hi @samcook |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
We are using Redis Windows version (latest). We are using Redlock.NET to lock the DBs. Everyday, for about 30mnts to 2 hours no one gets the lock. This is not very consistent. Works most of the time, expect when those 30mnts to 2 hour.
I don't see any issues with Redis on logfiles.
We have the lock expiry set to 1 mnt, but the lock doesn't seem to be expiring.
Settings:
Expiry: 60 sec, Retry: 1 sec, Wait: 20 sec
Redlock.NET version: 1.7.5
StackExchange.Redis: 1.2.6.0
Code:
Logs in descending order:
More Logs of not acquiring locks
The text was updated successfully, but these errors were encountered: