Releases · redis/lettuce

11 Feb 12:45

mp911de

3.4.1.Final

ae7ea91

3.4.1.Final

lettuce 3.4.1 is a bugfix release to fix issues with Redis Cluster and thread interrupts.

Fixes

NPE when issuing multi-exec against cluster #187 (thanks to @rovarghe)
ClusterTopologyRefresh fails with NPE on password-secured Cluster nodes if password is not set #189
Redis Cluster Node connections are not authenticated in a password protected Cluster (lettuce 3.4) #190
Set interrupted bit after catching InterruptedException #192

Javadoc: http://redis.paluch.biz/docs/api/releases/3.4.1.Final/

Assets 10

29 Jan 20:31

mp911de

4.1.Final

94a3b12

4.1.Final

lettuce 4.1 is here. This release contains numerous features and bugfixes. Lettuce 4.1 introduces reusable client-resources, an EventBus, client metrics, and support for newly introduced commands. This version works with Redis 3.2 RC3 but Redis expects a change in the format of CLUSTER NODES. So watch out for a new release of lettuce as soon as Redis 3.2 RC4 or a final is released.

lettuce is available in two major versions. The 3.x stream and the 4.x stream. Both streams are maintained.

After this release, the 4.x branch will be promoted to the default branch.
Following rules should give a guidance for the stream in which a particular change is done:

Changes affecting both streams

New Redis commands (such as HSTRLEN)
Bugfixes

Changes for the 4.x stream only

New Redis paradigms
Enriching the API (such as multi-key command execution in the Cluster API)
Technical improvements to the client (such as the Reactive API)

The 3.x stream will be maintained at least until end of 2016.

Reusable ClientResources

lettuce requires a threading infrastructure to operate. Threads are grouped within EventLoopGroups and are expensive resources. They need to be spun up and shut down. Prior to this release, each instance of RedisClient and RedisClusterClient created its own EventLoopGroups and EventExecutorGroup. If you had two instances of RedisClient or RedisClusterClient, lettuce used up to 2 * (Runtime.getRuntime().availableProcessors() * 4 * 3) threads.

Two things changed now:

Lettuce uses at least 3 threads but at most the number of available processors Runtime.getRuntime().availableProcessors() * 3 threads.
EventLoopGroups are hosted within ClientResources and can be reused across multiple RedisClient or RedisClusterClient instances

By default, each RedisClient and RedisClusterClient instance have their own, dedicated ClientResources. Shared ClientResources can be supplied upon client creation (see below). In general, it is a good idea to reuse instances of ClientResources across multiple clients.

Shared client resources are required to be shutdown once they are no longer used.

You can create instances using two different patterns:

The create() factory method

By using the create() method on DefaultClientResources you create ClientResources with default settings:

ClientResources res = DefaultClientResources.create();

RedisClient client = RedisClient.create(res);
RedisClusterClient clusterClient = RedisClusterClient.create(res, seedUris);
...
client.shutdown();
clusterClient.shutdown();

res.shutdown();

This approach fits the most needs.

Resources builder

You can build instances of DefaultClientResources by using the embedded builder. It is designed to configure the resources to your needs. The builder accepts the configuration in a fluent fashion and then creates the ClientResources at the end:

ClientResources res = new DefaultClientResources.Builder().
                        .ioThreadPoolSize(4)
                        .computationThreadPoolSize(4)
                        .build();

RedisClient client = RedisClient.create(res);
RedisClusterClient clusterClient = RedisClusterClient.create(res, seedUris);
...
client.shutdown();
clusterClient.shutdown();

res.shutdown();

A RedisClient and RedisClusterClient can be created without passing ClientResources upon creation. The resources are exclusive to the client and are managed itself by the client. When calling shutdown() of the client instance ClientResources are shut down.

`create` methods to construct the client

lettuce 4.1 introduces new create methods to create client instances. The create methods replace the deprecated constructors of RedisClient and RedisClusterClient. The create methods come with various signatures to support a conslidated style of client creation:

Create a client

RedisClient client = RedisClient.create();
RedisClusterClient clusterClient = RedisClusterClient.create(seedUris);
...

Create a client using shared ClientResources

ClientResources res = DefaultClientResources.create();
RedisClient client = RedisClient.create(res);
RedisClusterClient clusterClient = RedisClusterClient.create(res, seedUris);
...

EventBus and Client Events

lettuce introduces a new event bus to publish events. The previous client events were restricted to Connected, Disconnected and ExceptionCaught and required structural changes in the event listener. With the event bus any event type can be published. To listen to client events you need to subscribe to the EventBus which is available from the ClientResources.

RedisClient client = RedisClient.create();
EventBus eventBus = client.getResources().eventBus();

Subscription subscription = eventBus.get()
                .filter(redisEvent -> redisEvent instanceof ConnectedEvent)
                .cast(ConnectedEvent.class)
                .subscribe(e -> System.out.println(e.localAddress()));

...
subscription.unsubscribe();
client.shutdown();

The event bus uses rx-java to publish events. Following events are available:

ConnectedEvent
ConnectionActivatedEvent
DisconnectedEvent
ConnectionDeactivatedEvent
ClusterTopologyChangedEvent
CommandLatencyEvent (see Command Latency Metrics for further details)

All of the built-in events carry additional details (see JavaDoc for details).

Events are published within the scope of the EventBus instance that is by default part of the ClientResources.
The event bus will be used by multiple client instances if client resources are shared.

Listeners implementing the RedisConnectionStateListener still work.

Command Latency Metrics

Command latency metrics give insight about command execution and latencies. Metrics are collected for every completed command and are enabled by default.

Each command is tracked with:

Execution count
Latency to first response (min, max, percentiles)
Latency to complete (min, max, percentiles)

Command latencies are tracked on remote endpoint (distinction by host and port or socket path) and command type level
(GET, SET, ...). It is possible to track command latencies on a per-connection level (see DefaultCommandLatencyCollectorOptions).

Command latencies are transported using Events on the EventBus. The EventBus can be obtained from the ClientResources of the client instance. Please keep in mind that the EventBus is used for various event types. Filter on the event type if you're interested only in particular event types.

RedisClient client = RedisClient.create();
EventBus eventBus = client.getResources().eventBus();

Subscription subscription = eventBus.get()
                .filter(redisEvent -> redisEvent instanceof CommandLatencyEvent)
                .cast(CommandLatencyEvent.class)
                .subscribe(e -> System.out.println(e.getLatencies()));

The EventBus uses rx-java to publish events. This example prints the received latencies to stdout. The interval and
the collection of command latency metrics can be configured in the ClientResources.

Disabling command latency metrics

To disable metrics collection, use own ClientResources with a disabled DefaultCommandLatencyCollectorOptions:

ClientResources res = new DefaultClientResources
        .Builder()
        .commandLatencyCollectorOptions(DefaultCommandLatencyCollectorOptions.disabled())
        .build();

RedisClient client = RedisClient.create(res);

Master/Slave connections

Redis nodes can be operated in a Master/Slave setup to achieve availability and performance. Master/Slave setups can be run either Standalone or managed using Redis Sentinel. Lettuce allows to use slave nodes for read operations by using the MasterSlave API that supports both Master/Slave setups:

Redis Standalone Master/Slave (no failover)
Redis Sentinel Master/Slave (Sentinel-managed failover)

The resulting connection uses in any case the primary connection-point to dispatch non-read operations.

Redis Sentinel

Master/Slave with Redis Sentinel is very similar to regular Redis Sentinel operations. When the master fails over, a slave is promoted by Redis Sentinel to the new master and the client obtains the new topology from Redis Sentinel.

Connections to Master/Slave require one or more Redis Sentinel connection points and a master name.
The primary connection point is the Sentinel monitored master node.

Example

RedisURI sentinelUri = RedisURI.Builder.sentinel("sentinel-host", 26379, "master-name").build();
RedisClient client = RedisClient.create();

StatefulRedisMasterSlaveConnection<String, String> connection = MasterSlave.connect(client,
            new Utf8StringCodec(), sentinelUri);

connection.setReadFrom(ReadFrom.SLAVE);

connection.sync().get("key");

connection.close();
client.shutdown();

Redis Standalone

Master/Slave with Redis Standalone is very similar to regular Redis Standalone operations. A Redis Standalone Master/Slave setup is static and provides no built-in failover. Slaves are read from the Redis Master node's INFO command.

Connecting to Redis Standalone Master/Slave nodes requires to connect use the Redis Master for the RedisURI. The node used within the RedisURI is the primary connection point.

RedisURI masterUri = RedisURI.Builder.redis("master-host", 6379).build();
RedisClient client = RedisClient.create();

StatefulRedisMasterSlaveConnection<String, String> connection = MasterSlave.connect(client,
            new Utf8StringCodec(), masterUri);

connection.setReadFrom(ReadFrom.SLAVE);

connection.sync().get("key");

connection.close();
client.shutdown();

Reusable ClientResources

Two things changed now:

Lettuce uses at least 3 threads but at most the number of
available processors Runtime.getRuntime().availableProcessors() * 3 threads.
EventLoopGroups are hosted within ClientResources
and can be reused across multiple RedisClient or RedisClusterClient instances

Shared client resources are required to be shutdown once they are no longer used.

You can create instances using two different patterns:

The create() factory method

By using the create() method on DefaultClientResources you create ClientResources with default settings:

ClientResources res = DefaultClientResources.create();

RedisClient client = RedisClient.create(res);
RedisClusterClient clusterClient = RedisClusterClient.create(res, seedUris);
...
client.shutdown();
clusterClient.shutdown();

res.shutdown();

This approach fits the most needs.

Resources builder

ClientResources res = new DefaultClientResources.Builder().
                        .ioThreadPoolSize(4)
                        .computationThreadPoolSize(4)
                        .build();

RedisClient client = RedisClient.create(res);
RedisClusterClient clusterClient = RedisClusterClient.create(res, seedUris);
...
client.shutdown();
clusterClient.shutdown();

res.shutdown();

`create` methods to construct the client

Create a client

RedisClient client = RedisClient.create();
RedisClusterClient clusterClient = RedisClusterClient.create(seedUris);
...

*Create a client using shared ClientResources *

ClientResources res = DefaultClientResources.create();
RedisClient client = RedisClient.create(res);
RedisClusterClient clusterClient = RedisClusterClient.create(res, seedUris);
...

EventBus and Client Events

RedisClient client = RedisClient.create();
EventBus eventBus = client.getResources().eventBus();

Subscription subscription = eventBus.get()
                .filter(redisEvent -> redisEvent instanceof ConnectedEvent)
                .cast(ConnectedEvent.class)
                .subscribe(e -> System.out.println(e.localAddress()));

...
subscription.unsubscribe();
client.shutdown();

The event bus uses rx-java to publish events. Following events are available:

ConnectedEvent
ConnectionActivatedEvent
DisconnectedEvent
ConnectionDeactivatedEvent
ClusterTopologyChangedEvent
CommandLatencyEvent (see Command Latency Metrics for further details)

All of the built-in events carry additional details (see JavaDoc for details).

Events are published within the scope of the EventBus instance that is by default part of the ClientResources. The event bus will be used by multiple client instances if client resources are shared.

Listeners implementing the RedisConnectionStateListener still work.

Command Latency Metrics

Command latency metrics give insight about command execution and latencies. Metrics are collected for every completed command and are enabled by default.

Each command is tracked with:

Execution count
Latency to first response (min, max, percentiles)
Latency to complete (min, max, percentiles)

RedisClient client = RedisClient.create();
EventBus eventBus = client.getResources().eventBus();

Subscription subscription = eventBus.get()
                .filter(redisEvent -> redisEvent instanceof CommandLatencyEvent)
                .cast(CommandLatencyEvent.class)
                .subscribe(e -> System.out.println(e.getLatencies()));

The EventBus uses rx-java to publish events. This example prints the received latencies to stdout. The interval and the collection of command latency metrics can be configured in the ClientResources.

Disabling command latency metrics

To disable metrics collection, use own ClientResources with a disabled DefaultCommandLatencyCollectorOptions:

ClientResources res = new DefaultClientResources
        .Builder()
        .commandLatencyCollectorOptions(DefaultCommandLatencyCollectorOptions.disabled())
        .build();

RedisClient client = RedisClient.create(res);

ReadFrom Settings/Redis Cluster slave reads

The ReadFrom setting describes how lettuce routes read operations to the members of a Redis Cluster.

By default, lettuce routes its read operations to the master node. Reading from the master returns the most recent version of the data because write operations are issued to the single master node. Reading from masters guarantees strong consistency.

The ReadFrom setting can be set to one of the following presets:

MASTER Default mode. Read from the current master node.
MASTER_PREFERRED Read from the master, but if it is unavailable, read from slave nodes.
SLAVE Read from slave nodes.
NEAREST Read from any node of the cluster with the lowest latency.

Custom read settings can be implemented by extending the com.lambdaworks.redis.ReadFrom class.

Updated dependencies

netty 4.0.28.Final -> 4.0.34.Final

Enhancements

Support CLUSTER SETSLOT STABLE command #160
Support FLUSHALL [ASYNC]/FLUSHDB [ASYNC]/UNLINK commands #146
Adjust logging when running into Exceptions (exceptionCaught()) #140
Implement an EventBus system to publish events and metrics #124 (Thanks to @pulse00)
Implement a CompressionCodec for GZIP and Deflate compression #127
Provide a reusable client configuration for ThreadPools and other expensive resources #110
Use much faster JDK utility for converting an int to a byte sequence #163 (Thanks to @CodingFabian)
Cluster ReadFrom #114
Allow limiting the request queue size #115
Extend support for CLUSTER commands #111
Dispatch CLUSTER commands based on the slot #112
Support changed CLUSTER SLOTS #183

Fixes

Do not cache InetSocketAddress/SocketAddress in RedisURI #144
pfmerge invokes PFADD instead of PFMERGE #158 (Thanks to @christophstrobl)
Fix set with args method signature #159 (Thanks to @joshdurbin)
fix NOAUTH error when connecting to a cluster with password protection #171 (Thanks to @LiuFL)
Enable P...

Assets 10

11 Dec 06:58

mp911de

4.0.2.Final

4dc9bab

4.0.2.Final

This is a bugfix release for lettuce 4.0.1.Final.

Fixes

pfmerge invokes PFADD instead of PFMERGE #158 (Thanks to @christophstrobl)
Fix set with args method signature #159 (Thanks to @joshdurbin)

Assets 10

11 Dec 06:55

mp911de

3.3.2.Final

069b197

3.3.2.Final

This is a bugfix release for lettuce 3.3.1.Final.

Fixes

pfmerge invokes PFADD instead of PFMERGE #158 (Thanks to @christophstrobl)
Fix set with args method signature #159 (Thanks to @joshdurbin)

Assets 10

30 Nov 19:50

mp911de

4.0.1.Final

dfb0ba4

4.0.1.Final

This is a bugfix release for lettuce 4.0.Final to fix a bug in the Redis Cluster API when using Geo commands.

Fixes

Cluster API does not implement the Geo commands interface #154 (thanks to @IdanFridman)

Javadoc: http://redis.paluch.biz/docs/api/releases/4.0.1.Final/

Assets 10

02 Oct 13:38

mp911de

4.0.Final

4c743c5

4.0.Final

This release is a major release that introduces numerous changes like stateful connections, the reactive API and many more. Lettuce 4.0 includes all features from lettuce 3.3.

Highlights of lettuce 4.0.Final

Reactive API
Stateful connections
Cross-slot command execution
Node Selection API/Execution of commands on multiple cluster nodes
ReadFrom Settings/Redis Cluster slave reads
Custom commands

This release contains some breaking changes. You may want to consult the wiki at Migration from 3.x to 4.x to check the migration guide.

All Redis...Connection and Redis...AsyncConnection interfaces are deprecated and replaced by new ...Commands interfaces.

The cluster API was extended to run a command on multiple nodes and invocation to multi-key commands DEL, MGET, MSET and MSETNX perform automatic pipelining if the keys belong to different slots/masters.

A couple of changes are breaking changes, and you need most likely to adopt your code to use lettuce 4.0. lettuce 4.0 dropped Java 6 and 7 support and requires Java 8 to run. lettuce 4.0 needs Java 8 and cannot be used with Java 6 or 7.

Reactive API

Lettuce provides since its genesis two API's: Sync and async.
Now comes a third one: Reactive API.
The reactive API uses rx-java and every command on the reactive API returns an Observable.

StatefulRedisConnection<String, String> connection = client.connect();
RedisReactiveCommands<String, String> reactive = stateful.reactive();

Observable<String> observable = reactiveApi.keys("*");
observable.subscribe();

// more sophisticated

reactive.keys("*").flatMap(reactive::del).subscribe();

The command is only executed on a subscription to the Observable.
Commands returning a collection, such as Lists or Sets, return Observables that provide the element type. The reactive API covers the same commands as the synchronous/asynchronous API's.

Rearchitecting the API

Before 4.0, connection resources (sockets, events) were bound to the particular API.
If one wanted to use a different API, he had to create another connection to Redis. This coupling is loosed now. By calling connect() you will no longer get a synchronous connection, you will get a StatefulRedisConnection with the access to the synchronous, asynchronous and reactive API.
All commands are executed using netty and it does not matter, from which API you come.

3.x code:

RedisConnection connection = client.connect();

4.x code:

StatefulRedisConnection stateful = client.connect();

RedisCommands commands = stateful.sync();

// still working
RedisAsyncConnection connection = stateful.connectAsync();

The other connect methods like connectAsync and connectSentinelAsync are
deprecated but remain unchanged.

Breaking changes in connect methods are:

RedisClient.connect (provides a StatefulRedisConnection)
RedisClient.connectPubSub (provides a StatefulRedisPubSubConnection)
RedisClusterClient.connectCluster (provides a StatefulRedisClusterConnection)

New connect methods:

RedisClient.connectSentinel (provides a StatefulRedisSentinelConnection)

New segregated command interfaces

Starting with reactive API's, another 13 interfaces will be provided with lettuce.
This change increases the count of types within the com.lambdaworks.redis package and the package gets messier again. In combination with the stateful connection the original ...Connection or ...AsyncConnection interfaces no longer reflect the real purpose. New Commands interfaces provide the same functionality and are located in api.sync and api.async packages (respective cluster.api.sync, cluster.api.async and so on for the Redis Cluster client and PubSub).

The following interfaces are deprecated and substituted by new ...Commands interfaces:

RedisHashesAsyncConnection
RedisHashesConnection
RedisHLLAsyncConnection
RedisHLLConnection
RedisKeysAsyncConnection
RedisKeysConnection
RedisListsAsyncConnection
RedisListsConnection
RedisScriptingAsyncConnection
RedisScriptingConnection
RedisSentinelAsyncConnection
RedisServerAsyncConnection
RedisServerConnection
RedisSetsAsyncConnection
RedisSetsConnection
RedisSortedSetsAsyncConnection
RedisSortedSetsConnection
RedisStringsAsyncConnection
RedisStringsConnection
RedisClusterConnection
RedisConnection
RedisClusterAsyncConnection
RedisAsyncConnection
BaseRedisConnection
BaseRedisAsyncConnection

See Migration from 3.x to 4.x for the migration matrix.

New API's

StatefulClusterConnection
StatefulRedisPubSubConnection
StatefulClusterConnection
StatefulRedisSentinelConnection
RedisPubSubAsyncConnection and RedisPubSubConnection

API Changes

readOnly and readWrite changed from String return type to RedisFuture<String>. The connection state is maintained by the future completion.
Moved CommandOutput from com.lambdaworks.redis.protocol to com.lambdaworks.redis.output
Moved SetArgs from com.lambdaworks.redis.protocol to com.lambdaworks.redis
All connections are AutoCloseable so you can handle connections using try-with-resources.
RedisFutures are based on CompleteableFuture and throw now any occurred exception when accessing the value using get().

Exceptions are passed down the CompletionStages.

Client factory methods

RedisClient and RedisClusterClient provide factory methods (create) to create new instances. This is, to get flexibility by reducing the number of constructors and in preparation for reusable ClientResources which will be added with lettuce 4.1.

The constructors of RedisClient and RedisClusterClient are deprecated and will be removed in future versions.

RedisClient client = RedisClient.create();

RedisClient client = RedisClient.create("redis://localhost/");

RedisClient client = RedisClient.create(RedisURI.Builder.redis("localhost", 6379).build());


RedisClusterClient client = RedisClusterClient.create("redis://localhost/");

RedisClusterClient client = RedisClusterClient.create(RedisURI.Builder.redis("localhost", 6379).build());

Cross-slot command execution

Regular Redis Cluster commands are limited to single-slot keys, basically either single key commands or multi-key commands that share the same hash slot of their keys.

The cross slot limitation can be mitigated by using the advanced cluster API for some multi-key commands. Commands that operate on keys with different slots are decomposed into multiple commands.
The single commands are fired in a fork/join fashion. The commands are issued concurrently to avoid
synchronous chaining. Results are synchronized before the command is completed (from a user perspective).

Following commands are supported for cross-slot command execution:

DEL: Delete the KEYs from the affected cluster. Returns the number of keys that were removed
MGET`: Get the values of all given KEYs. Returns the values in the order of the keys.
MSET: Set multiple key/value pairs for all given KEYs. Returns always OK.

Cross-slot command execution is available on the following APIs:

RedisAdvancedClusterCommands
RedisAdvancedClusterAsyncCommands
RedisAdvancedClusterReactiveCommands

Node Selection API/Execution of commands on multiple cluster nodes

The advanced cluster API allows to select nodes and run commands on the node selection. This API is subject to incompatible changes in a future release. The API is exempt from any compatibility guarantees made by lettuce. The current state implies nothing about the quality or performance of the API in question, only the fact that it is not "API-frozen". All commands are sent concurrently to the Redis nodes, meaning you do not have to wait until the commands have finished to trigger the next command. That behavior is independent of the API you're using. The Node Selection API is available from the synchronous and asynchronous command interfaces:

Asynchronous

RedisAdvancedClusterAsyncCommands<String, String> async = clusterClient.connect().async();
AsyncNodeSelection<String, String> slaves = async.slaves();

AsyncExecutions<List<String>> executions = slaves.commands().keys("*");
executions.stream().forEach(result -> result.thenAccept(keys -> System.out.println(keys)));

Synchronous

RedisAdvancedClusterCommands<String, String> sync = clusterClient.connect().sync();
NodeSelection<String, String> slaves = sync.slaves();

Executions<List<String>> executions = slaves.commands().keys("*");
executions.stream().forEach(value -> System.out.println(value));

Commands are dispatched to the nodes within the selection, the result (CompletionStage or the value) is available from the AsyncExecutions/Executions result holders.
This API is a technical preview, so your feedback is highly appreciated.

ReadFrom Settings/Redis Cluster slave reads

The ReadFrom setting describes how lettuce routes read operations to the members of a Redis Cluster.

The ReadFrom setting can be set to one of the following presets:

MASTER Default mode. Read from the current master node.
MASTER_PREFERRED Read from the master, but if it is unavailable, read from slave nodes.
SLAVE Read from slave nodes.
NEAREST Read from any node of the cluster w...

Assets 10

02 Oct 06:03

mp911de

3.3.1.Final

1b873fa

3.3.1.Final

lettuce 3.3.1.Final is a maintenance release that fixes several problems reported
by the community.

Fixes

GEOADD passes long/lat parameters in the wrong order to Redis #134 (thanks to @IdanFridman)
Strip username from URI userinfo when creating a RedisURI with a username #131 (thanks to @jsiebens)
GeoArgs not evaluated when calling georadiusbymember(...) #142 (thanks to @codeparity)
Guava's CacheBuilder missing in shaded jar #143

API Docs: http://redis.paluch.biz/docs/api/releases/3.3.1.Final/

Assets 10

03 Sep 08:36

mp911de

4.0.Beta2

df8badd

4.0.Beta2 Pre-release

Pre-release

This release is a major release that introduces numerous changes like stateful connections, the reactive API and many more. Lettuce 4.0 includes all features from lettuce 3.3.

Highlights of lettuce 4.0.Beta2

Reactive API
Stateful connections
Cross-slot command execution
Node Selection API/Execution of commands on multiple cluster nodes
ReadFrom Settings/Redis Cluster slave reads
Custom commands

This release contains some breaking changes. You may want to consult the wiki at Migration from 3.x to 4.x to check the migration guide.

All Redis...Connection and Redis...AsyncConnection interfaces are deprecated and replaced by new ...Commands interfaces.

The cluster API was extended to run a command on multiple nodes and invocation to multi-key commands DEL, MGET, MSET and MSETNX perform automatic pipelining if the keys belong to different slots/masters.

A couple of changes are breaking changes, and you need most likely to adopt your code to use lettuce 4.0. lettuce 4.0 dropped Java 6 and 7 support and requires Java 8 to run.

lettuce 4.0 needs Java 8 and cannot be used with Java 6 or 7.

If you need any support, meet lettuce at:

Google Group: https://groups.google.com/d/forum/lettuce-redis-client-users
Gitter: https://gitter.im/mp911de/lettuce
Github Issues: https://github.com/mp911de/lettuce/issues

Reactive API

Lettuce provides since its genesis two API's: Sync and async. Now comes a third one: Reactive.
The reactive API uses rx-java and every command on the reactive API returns an Observable.

StatefulRedisConnection<String, String> connection = client.connect();
RedisReactiveCommands<String, String> reactive = stateful.reactive();

Observable<String> observable = reactiveApi.keys("*");
observable.subscribe();

// more sophisticated

reactive.keys("*").flatMap(reactive::del).subscribe();

The command is only executed on a subscription to the Observable. Commands returning a collection, such as Lists or Sets, return Observables that provide the element type. The reactive API covers the same commands as the synchronous/asynchronous API's.

Rearchitecting the API

Before 4.0, connection resources (sockets, events) were bound to the particular API. If one wanted to use a different API, he had to create another connection to Redis. This coupling is loosed now. By calling connect() you will no longer get a synchronous connection, you will get a StatefulRedisConnection with the access to the synchronous, asynchronous and reactive API.
All commands are executed using netty and it does not matter, from which API you come.

3.x code:

RedisConnection connection = client.connect();

4.x code:

StatefulRedisConnection stateful = client.connect();

RedisCommands commands = stateful.sync();

// still working
RedisConnection connection = stateful.sync();

The other connect methods like connectAsync and connectSentinelAsync are deprecated but remain unchanged.

Breaking changes in connect methods are:

RedisClient.connect (provides a StatefulRedisConnection)
RedisClient.connectPubSub (provides a StatefulRedisPubSubConnection)
RedisClusterClient.connectCluster (provides a StatefulRedisClusterConnection)

New connect methods:

RedisClient.connectSentinel (provides a StatefulRedisSentinelConnection)

New segregated command interfaces

Starting with reactive API's, another 13 interfaces will be provided with lettuce. This change increases the count of types within the com.lambdaworks.redis package and the package gets messier again. In combination with the stateful connection the original ...Connection or ...AsyncConnection interfaces no longer reflect the real purpose. New Commands interfaces provide the same functionality and are located in api.sync and api.async packages (respective cluster.api.sync, cluster.api.async and so on for the Redis Cluster client and PubSub).

The following interfaces are deprecated and substituted by new ...Commands interfaces:

RedisHashesAsyncConnection
RedisHashesConnection
RedisHLLAsyncConnection
RedisHLLConnection
RedisKeysAsyncConnection
RedisKeysConnection
RedisListsAsyncConnection
RedisListsConnection
RedisScriptingAsyncConnection
RedisScriptingConnection
RedisSentinelAsyncConnection
RedisServerAsyncConnection
RedisServerConnection
RedisSetsAsyncConnection
RedisSetsConnection
RedisSortedSetsAsyncConnection
RedisSortedSetsConnection
RedisStringsAsyncConnection
RedisStringsConnection
RedisClusterConnection
RedisConnection
RedisClusterAsyncConnection
RedisAsyncConnection
BaseRedisConnection
BaseRedisAsyncConnection

See Migration guide for the migration matrix.

New API's

StatefulClusterConnection
StatefulRedisPubSubConnection
StatefulClusterConnection
StatefulRedisSentinelConnection
RedisPubSubAsyncConnection and RedisPubSubConnection

API Changes

readOnly and readWrite changed from String return type to RedisFuture<String>. The connection state is maintained by the future completion.
Moved CommandOutput from com.lambdaworks.redis.protocol to com.lambdaworks.redis.output
Moved SetArgs from com.lambdaworks.redis.protocol to com.lambdaworks.redis
All connections are AutoCloseable so you can handle connections using try-with-resources.
RedisFutures are based on CompleteableFuture and throw now any occurred exception when accessing the value using get().
Exceptions are passed down the CompletionStages.

Cross-slot command execution

Regular Redis Cluster commands are limited to single-slot keys, basically either single key commands or multi-key commands that share the same hash slot of their keys.

The cross slot limitation can be mitigated by using the advanced cluster API for some multi-key commands. Commands that operate on keys with different slots are decomposed into multiple commands. The single commands are fired in a fork/join fashion. The commands are issued concurrently to avoid synchronous chaining. Results are synchronized before the command is completed (from a user perspective).

Following commands are supported for cross-slot command execution:

DEL: Delete the KEYs from the affected cluster. Returns the number of keys that were removed
MGET: Get the values of all given KEYs. Returns the values in the order of the keys.
MSET: Set multiple key/value pairs for all given KEYs. Returns always OK.

Cross-slot command execution is available on the following APIs:

RedisAdvancedClusterCommands
RedisAdvancedClusterAsyncCommands
RedisAdvancedClusterReactiveCommands

Node Selection API/Execution of commands on multiple cluster nodes

Asynchronous

RedisAdvancedClusterAsyncCommands<String, String> async = clusterClient.connect().async();
AsyncNodeSelection<String, String> slaves = connection.slaves();

AsyncExecutions<List<String>> executions = slaves.commands().keys("*");
executions.stream().forEach(result -> result.thenAccept(keys -> System.out.println(keys)));

Synchronous

RedisAdvancedClusterCommands<String, String> async = clusterClient.connect().sync();
NodeSelection<String, String> slaves = connection.slaves();

Executions<List<String>> executions = slaves.commands().keys("*");
executions.stream().forEach(value -> System.out.println(value));

Commands are dispatched to the nodes within the selection, the result (CompletionStage or the value) is available from the AsyncExecutions/Executions result holders. This API is a technical preview, so your feedback is highly appreciated.

ReadFrom Settings/Redis Cluster slave reads

The ReadFrom setting describes how lettuce routes read operations to the members of a Redis Cluster.

The ReadFrom setting can be set to one of the following presets:

MASTER Default mode. Read from the current master node.
MASTER_PREFERRED Read from the master, but if it is unavailable, read from slave nodes.
SLAVE Read from slave nodes.
NEAREST Read from any node of the cluster with the lowest latency.

Custom read settings can be implemented by extending the com.lambdaworks.redis.ReadFrom class.

Custom commands

Lettuce covers nearly all Redis commands. Redis development is an ongoing process, and some commands are not covered by lettuce meaning there are use cases that require invocation of custom commands or custom outputs. lettuce 4.x allows you to trigger own commands. That API is used by lettuce itself to dispatch commands and requires some knowledge of how commands are constructed and dispatched within lettuce.

StatefulRe...

Assets 10

27 Aug 15:05

mp911de

3.3.Final

8679420

3.3.Final

lettuce 3.3 introduces a variety of changes: It features changes to cluster support, new Geo commands for the upcoming Redis 3.2 release and allows batching, an extension to pipelining.

The documentation within the lettuce wiki was overhauled.
Meet the very new wiki at https://github.com/mp911de/lettuce/wiki

The test stability issue was addressed by running infinite testing jobs. Instabilities were caused due to missing synchronization and relying too much on timing. However, the real life teaches us, not to rely on time, rather on hard facts whether a cluster node is available, or a command is written out to the transport. The tests are improved and fail less often.

I'm very excited to present you lettuce 3.3.Final. Read on for the details or jump to the end to find the summary and the download links.

Redis Cluster: Node connections

lettuce's cluster support is enhanced by a couple of features. Users can obtain a connection to the particular cluster nodes by specifying either the nodeId or host and port. The new methods are available on two new interfaces:

RedisAdvancedClusterConnection
RedisAdvancedClusterAsyncConnection

Example code:

RedisAdvancedClusterAsyncConnection<String, String> connection = clusterClient.connectClusterAsync();
RedisClusterAsyncConnection<String, String> nodeConnection = connection
                           .getConnection("adf7f86efa42d903bcd93c5bce72397fe52e91bb");

...

RedisClusterAsyncConnection<String, String> nodeConnection = connection.getConnection("localhost", 7379);

You are free to operate on these connections. Connections can be bound to specific hosts or nodeId's. Connections bound to a nodeId will always stick to the nodeId, even if the nodeId is handled by a different host. Requests to unknown nodeId's or host/ports that are not part of the cluster are rejected.
Do not close the connections. Otherwise, unpredictable behavior will occur. Keep also in mind, that the node connections are used by the cluster connection itself to perform cluster operations: If you block one connection all other users of the cluster connection might be affected.

The cluster client handles ASK redirection the same way as MOVED redirection. It is transparent to the client. Dispatching the commands between the particular cluster connections was optimized, and the performance was improved.

Cluster topology view refreshing

Another cluster-related feature is the cluster topology view refresh. Reloading the partitions was possible since lettuce 3.0. Version 3.3 enabled the reloading on a regular basis in the background. The background update will close connections to nodeId's/hosts that are no longer part of the cluster. You can enable the refresh job (disabled by default) and specify the interval (defaults to 60 seconds). The refresh can be configured in the ClusterClientOptions.

Example code:

clusterClient.setOptions(new ClusterClientOptions.Builder()
                                 .refreshClusterView(true)
                                 .refreshPeriod(5, TimeUnit.SECONDS)
                                 .build());

RedisAdvancedClusterAsyncConnection<String, String> clusterConnection = clusterClient.connectClusterAsync();

Pipelining/Batching

Lettuce operates using pipelining described in http://redis.io/topics/pipelining.

What is different now? lettuce performs a flush after each command invocation on the transport. This is fine for the most use cases, but flushing can become a limiting factor when bulk loading, or you need batching.

Asynchronous connections allow you to disable the auto-flush behavior and give control over flushing the queued commands:

Example code:

RedisAsyncConnection<String, String> connection = client.connectAsync();
connection.setAutoFlushCommands(false);

connection.set("key", "value");
connection.set("key2", "value2");

connection.flushCommands(); // send the two SET commands out to the transport

connection.setAutoFlushCommands(true);

Why on the asynchronous connections only? The asynchronous connections return already a handle to the result, the synchronous API does not. Adding another API would require to duplicate all interfaces and increase complexity. Pipelining/Batching can improve your throughput. Pipelining also works with Redis Cluster and is not available on pooled connections.

Codecs

lettuce 3.3 uses a dedicated String codec instance for each String-encoded connection (Standalone) instead of sharing one String codec for all connections. Cluster connections share a String codec between the internal connections.

A new ByteArrayCodec ships with lettuce 3.3 that allows byte array connections
without the need to create an own codec.

Geo commands

This release supports the Geo-commands of the upcoming Redis 3.2 release. The Geo-API allows to maintain a set (backed by a Redis sorted set) of Geo locations described by WGS84 coordinates. You can add and query set members using the new Geo-API. Use ZREM to remove members from the Geo set until redis/redis#2674 is resolved.

The design of the Geo-API within lettuce, differs from other APIs in lettuce. The response structures of GEORADIUS depend on the command input, and there are other languages that fit better into the Redis response structure patterns. The static type checking within Java would only allow a List<Object> (or Object)
which contains nested Lists and Maps carrying the data. You would have to cast the elements to maps or lists and access then again the nested elements. Working with Lists and Maps in Java is less convenient compared to JavaScript or Ruby.

The Geo-API provides GeoCoordinates and GeoWithin types that allow direct access to the response values such as distance or the coordinate points.

Example code:

redis.geoadd(key, 8.6638775, 49.5282537, "Weinheim",
                  8.3796281, 48.9978127, "Office tower",
                  8.665351, 49.553302, "Train station");

Set<String> georadius = redis.georadius(key, 8.6582861, 49.5285695,
                                               5, GeoArgs.Unit.km);

// georadius contains "Weinheim" and "Train station"

Double distance = redis.geodist(key, "Weinheim", "Train station", GeoArgs.Unit.km);

// distance ≈ 2.78km

GeoArgs geoArgs = new GeoArgs().withHash()
                               .withCoordinates()
                               .withDistance()
                               .withCount(1)
                               .desc();

List<GeoWithin<String>> georadiusWithArgs = redis.georadius(key,
                                                        8.665351, 49.553302,
                                                        5, GeoArgs.Unit.km,
                                                        geoArgs);

// georadiusWithArgs contains "Weinheim" and "Train station"
// ordered descending by distance and containing distance/coordinates

Command execution reliability

A new document describes Command execution reliability in the context of lettuce and reconnects. In general, lettuce supports at-least-once and at-most-once semantics. The mode of operations is bound to the auto-reconnect flag in the client options.

If auto-reconnect is enabled, at-least-once semantics applies, at-most-once if auto-reconnect is disabled.

At-least-once and at-most-once are not new to lettuce. The documentation just explains how these semantics apply to lettuce.

Enhancements

Provide access to cluster connection using the advanced cluster API #71
Cluster connection failover when cluster topology changes #97
NodeId-bound cluster connections enhancement #104
Implement at-least-once for cluster connections #105
Pipelining for lettuce (or: flush after n commands) #92
Decouple ConnectionWatchdog reconnect from timer thread #100
Improve performance in 3.3 #90
Add checks for arguments #69
Use dedicated string codecs on String connections and add ByteArrayCodec #70
Tests for command reliability docs #98
Expose RedisClient.connect(RedisCodec, RedisURI) and RedisClient.connectAsync(RedisCodec, RedisURI) #108
Add close stale connections and strict cluster member check flags to ClusterClientOptions #109

Commands

Adopt variadic EXISTS and DEBUG HTSTATS #103
Support geo commands in lettuce 3.3 #86
Support NX|XX|CH|INCR options in ZADD #74
Add support for COUNTKEYSINSLOT #107
Add support for HSTRLEN #117

Fixes

Synchronization/cross thread visibility of variables #94
Check channel state before calling Channel.close.syncUninterruptibly #113
Stop the HashedWheelTimer before shutting down EventLoopGroups #123

Other

Rework of the wiki

http://redis.paluch.biz/docs/api/releases/3.3.Final/

Assets 10

Releases: redis/lettuce

3.4.1.Final

Fixes

4.1.Final

Reusable ClientResources

create methods to construct the client

EventBus and Client Events

Command Latency Metrics

Disabling command latency metrics

Master/Slave connections

Redis Sentinel

Redis Standalone

3.4.Final

Reusable ClientResources

create methods to construct the client

EventBus and Client Events

Command Latency Metrics

Disabling command latency metrics

ReadFrom Settings/Redis Cluster slave reads

Updated dependencies

Enhancements

Fixes

4.0.2.Final

Fixes

3.3.2.Final

Fixes

4.0.1.Final

Fixes

4.0.Final

Reactive API

Rearchitecting the API

New segregated command interfaces

New API's

API Changes

Client factory methods

Cross-slot command execution

Node Selection API/Execution of commands on multiple cluster nodes

ReadFrom Settings/Redis Cluster slave reads

3.3.1.Final

Fixes

4.0.Beta2

Reactive API

Rearchitecting the API

New segregated command interfaces

New API's

API Changes

Cross-slot command execution

Node Selection API/Execution of commands on multiple cluster nodes

ReadFrom Settings/Redis Cluster slave reads

Custom commands

3.3.Final

Redis Cluster: Node connections

Cluster topology view refreshing

Pipelining/Batching

Codecs

Geo commands

Command execution reliability

Enhancements

Commands

Fixes

Other

`create` methods to construct the client

`create` methods to construct the client