Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exception thrown during RedisClusterClient shutdown #1800

Closed
jdeppe-pivotal opened this issue Jul 9, 2021 · 4 comments
Closed

Exception thrown during RedisClusterClient shutdown #1800

jdeppe-pivotal opened this issue Jul 9, 2021 · 4 comments
Labels
type: bug A general bug
Milestone

Comments

@jdeppe-pivotal
Copy link

jdeppe-pivotal commented Jul 9, 2021

Bug Report

Current Behavior

While using lettuce against a cluster where nodes are being stopped and restarted, the following exception is thrown when the test shuts down the RedisClusterClient

    java.util.NoSuchElementException
        at java.util.concurrent.ConcurrentHashMap$KeyIterator.next(ConcurrentHashMap.java:3453)
        at io.lettuce.core.AbstractRedisClient.closeResources(AbstractRedisClient.java:551)
        at io.lettuce.core.AbstractRedisClient.shutdownAsync(AbstractRedisClient.java:540)
        at io.lettuce.core.cluster.RedisClusterClient.shutdownAsync(RedisClusterClient.java:1058)
        at io.lettuce.core.AbstractRedisClient.shutdown(AbstractRedisClient.java:504)
        at io.lettuce.core.AbstractRedisClient.shutdown(AbstractRedisClient.java:472)
        at org.apache.geode.redis.internal.executor.CrashAndNoRepeatDUnitTest.cleanup(CrashAndNoRepeatDUnitTest.java:110)```

Input Code

The RedisClusterClient is configured as follows:

    clusterClient = RedisClusterClient.create("redis://localhost:" + redisServerPort1);

    ClusterTopologyRefreshOptions refreshOptions =
        ClusterTopologyRefreshOptions.builder()
            .enableAllAdaptiveRefreshTriggers()
            .enablePeriodicRefresh(Duration.ofSeconds(5))
            .build();

    clusterClient.setOptions(ClusterClientOptions.builder()
        .topologyRefreshOptions(refreshOptions)
        .validateClusterNodeMembership(false)
        .build());

Expected behavior/code

shutdown should not produce any errors.

Environment

  • Lettuce version(s): 6.1.3.RELEASE
  • Redis version: custom
@mp911de
Copy link
Collaborator

mp911de commented Jul 9, 2021

I'm currently not sure where the bug is. shutdown isn't thread-safe to be called by multiple threads concurrently. We could copy over the closeable resources to an independent list instead of operating on the actual holder.

@jdeppe-pivotal
Copy link
Author

Just to be clear - shutdown is only being called by a single thread.

@mp911de mp911de added the type: bug A general bug label Jul 12, 2021
@mp911de mp911de added this to the 6.0.7 milestone Jul 13, 2021
mp911de added a commit that referenced this issue Jul 13, 2021
…ces #1800

shutdown now uses copies of mutable collections to process shutdown signals to avoid concurrent modification exceptions.
mp911de added a commit that referenced this issue Jul 13, 2021
…ces #1800

shutdown now uses copies of mutable collections to process shutdown signals to avoid concurrent modification exceptions.
mp911de added a commit that referenced this issue Jul 13, 2021
…ces #1800

shutdown now uses copies of mutable collections to process shutdown signals to avoid concurrent modification exceptions.
@mp911de
Copy link
Collaborator

mp911de commented Jul 13, 2021

That's fixed now. Care to run tests against the latest snapshots (via https://oss.sonatype.org/content/repositories/snapshots/)?

@jdeppe-pivotal
Copy link
Author

Looks like it's good now. Thanks again @mp911de !!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: bug A general bug
Projects
None yet
Development

No branches or pull requests

2 participants