Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Attach topology retrieval exceptions when Lettuce cannot retrieve a topology update #1024

Closed
stillerrr opened this issue Apr 26, 2019 · 9 comments
Labels
type: feature A new feature
Milestone

Comments

@stillerrr
Copy link

stillerrr commented Apr 26, 2019

Feature Request

throw the "NOAUTH" exception instead of “Cannot retrieve initial cluster partitions”, in other words, clarify why lettuce can't initialize the partitions.

Is your feature request related to a problem? Please describe

if a cluster has password, but we don's configure it for lettuce RedisClusterClient.
when the RedisClusterClient would initialize the cluster partitions, lettuce would create a new connection. and then report the exception "Cannot retrieve initial cluster partitions", just some WARN logs that report "NOAUTH".
here is the error:

Caused by: io.lettuce.core.RedisException: Cannot retrieve initial cluster partitions from initial URIs [RedisURI [host='127.0.0.1', port=2882], RedisURI [host='127.0.0.1', port=2881]]
        at io.lettuce.core.cluster.RedisClusterClient.doLoadPartitions(RedisClusterClient.java:874)
        at io.lettuce.core.cluster.RedisClusterClient.loadPartitions(RedisClusterClient.java:844)
        at io.lettuce.core.cluster.RedisClusterClient.initializePartitions(RedisClusterClient.java:819)
        at io.lettuce.core.cluster.RedisClusterClient.connect(RedisClusterClient.java:345)

Describe the solution you'd like

add the info why lettuce cannot retrieve initial cluster partitions in the exception, connection refuse or "NOAUTH" or others.

@stillerrr stillerrr added the type: feature A new feature label Apr 26, 2019
@mp911de
Copy link
Collaborator

mp911de commented Apr 26, 2019

Redis Cluster partition retrieval is attempted from multiple hosts and hence multiple errors can happen. Lettuce does not fail on the first error but attempts to retrieve the topology from other nodes. Since each Redis node can be configured individually, one node can be configured with a wrong password while other nodes might succeed.

We attach the underlying cause as suppressed exceptions to RedisException: Cannot retrieve initial cluster partitions. Each topology retrieval attempt can fail with an individual cause (e.g. connection refused, unresolvable host, …).

Just reporting NOAUTH is a bit misleading because we do not have the context. Let me know what you think about how we could improve the debugging experience here.

@mp911de mp911de added the status: waiting-for-feedback We need additional information before we can continue label Apr 26, 2019
@stillerrr
Copy link
Author

if the cluster has no password, but I configure it for lettuce, the exception message that lettuce reports contains error message "no password is set", I can check the exception and try to create a connection without password.

if the cluster has password, but I don't configure it, the initial exception is just "Cannot retrieve initial cluster partitions", it is confused because I only know that Iettuce can't initialize partitions without any reason. So there is nothing I can do in my code to handle this situation like reloading password

@mp911de
Copy link
Collaborator

mp911de commented Apr 26, 2019

You can traverse exception hierarchy and inspect suppressed exceptions (Throwable.getSuppressed()) Have you tried this already?

@stillerrr
Copy link
Author

yes, I do.
but I cannot find "NOAUTH", but in first situation, I can find the "no password is set"

the doLoadPartitions() method in RedisClusterClient throws a new RedisException if the partition is empty, but this exception has no other info.

@mp911de mp911de added status: feedback-provided Feedback has been provided and removed status: waiting-for-feedback We need additional information before we can continue labels Apr 29, 2019
@mp911de mp911de removed the status: feedback-provided Feedback has been provided label May 19, 2019
@mp911de mp911de changed the title throw the "NOAUTH" exception instead of “Cannot retrieve initial cluster partitions” Attach topology retrieval exceptions when Lettuce cannot retrieve a topology update May 19, 2019
@mp911de mp911de added this to the 5.1.7 milestone May 19, 2019
mp911de added a commit that referenced this issue May 19, 2019
…opology update #1024

We now fail earlier with an exception when topology retrieval has failed. We also attach individual failures as cause/suppressed exceptions to add context for error tracing.
mp911de added a commit that referenced this issue May 19, 2019
…opology update #1024

We now fail earlier with an exception when topology retrieval has failed. We also attach individual failures as cause/suppressed exceptions to add context for error tracing.
@mp911de
Copy link
Collaborator

mp911de commented May 19, 2019

That's in place now and backported to 5.1.7.

@mp911de mp911de closed this as completed May 19, 2019
@roottraveller
Copy link

Hi @mp911de
I am facing a similar issue. what are the actual cause and possible fix?

@mp911de
Copy link
Collaborator

mp911de commented Jul 25, 2019

@roottraveller please file a new ticket containing debug information such as a stack trace, a thread dump and describing your case. We cannot derive any specific information from a generic "It does not work".

@roottraveller
Copy link

roottraveller commented Jul 25, 2019

Thanks, @mp911de
I have created a separate bug ticket #1088
any help will be appreciated.

@mp911de
Copy link
Collaborator

mp911de commented Jul 25, 2019

Thanks a lot!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: feature A new feature
Projects
None yet
Development

No branches or pull requests

3 participants