Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No logging or rethrowing on GetReplicasForService in ConnectionMultiplexer #1512

Closed
ThomasTosik opened this issue Jun 29, 2020 · 6 comments
Closed

Comments

@ThomasTosik
Copy link

Hi!

I was trying out the new Sentinel support and had some issues.
Exception thrown: 'System.NullReferenceException' in StackExchange.Redis.dll: 'Object reference not set to an instance of an object.' Stack trace: at StackExchange.Redis.ConnectionMultiplexer.GetSentinelMasterConnection(ConfigurationOptions config, TextWriter log) in /_/src/StackExchange.Redis/ConnectionMultiplexer.cs:line 2401

I started to dig and found out that i had no chance to get to the real issues due to missing logging or rethrowing.

Is this intentional? After a tedious debug session i got at least my exception that i was looking for...

@mgravell
Copy link
Collaborator

"intentional" is complex, but I'm going to assume "probably not", but since the code here is parallel, logging becomes... trickier; what is the scenario that caused this to fail in this way, out of curiosity?

@ThomasTosik
Copy link
Author

So the exception that i got was:
Exception thrown: 'StackExchange.Redis.RedisConnectionException' in StackExchange.Redis.dll: 'Unexpected response to SENTINEL: MultiBulk: 0 items' Stack trace: at StackExchange.Redis.ConnectionMultiplexer.ExecuteSyncImpl[T](Message message, ResultProcessor`1 processor, ServerEndPoint server) in /_/src/StackExchange.Redis/ConnectionMultiplexer.cs:line 2826

Some background: I was using the bitnami redis/sentinel setup in my docker tests. I had only one redis master (first naive try). After i added a real redis slave this issue was gone.

@azharmahmood
Copy link

Hello, I am getting same error, I have only one remote sentinel master in my test environment. So is the issue resolved by adding a sentinel slave? Please confirm?

@ThomasTosik
Copy link
Author

Hello, I am getting same error, I have only one remote sentinel master in my test environment. So is the issue resolved by adding a sentinel slave? Please confirm?

I got it to work with one Sentinel, one Redis Master and one Redis Slave.

NickCraver pushed a commit that referenced this issue Jul 12, 2020
The assumption in the result processor was that it must have results to be valid, but that's not correct. If a sentinel has 0 replicas (only 1 master), that's a perfectly valid state. This adjusts the result processor logic to be more correct.
@NickCraver
Copy link
Collaborator

Thanks for the additional info! The root of the problem here is in SentinelGetSentinelAddressesProcessor I believe, specifically:

if (endPoints.Count > 0)
{
SetResult(message, endPoints.ToArray());
return true;
}
return false;

The result processor treats "no endpoints" as an invalid state, e.g. in this case where there simply are no replicas. I pushed up a PR for this in #1527.

@NickCraver
Copy link
Collaborator

Fixed via #1527

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants