Skip to content

Commit

Permalink
Merge pull request #34486 from cescoffier/redis-custom-codec
Browse files Browse the repository at this point in the history
Implement Redis custom codec support
  • Loading branch information
machi1990 authored Jul 4, 2023
2 parents 3734e34 + 6e7d4a1 commit 798f1ce
Show file tree
Hide file tree
Showing 20 changed files with 405 additions and 87 deletions.
82 changes: 60 additions & 22 deletions docs/src/main/asciidoc/redis-reference.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@ Typically, we recommend:
This extension provides imperative and reactive APIs and low-level and high-level (type-safe) clients.


== Installation

If you want to use this extension, you need to add the `io.quarkus:quarkus-redis` extension first.
Expand Down Expand Up @@ -86,7 +85,6 @@ To help you select the suitable API for you, here are some recommendations:
* If you have existing Vert.x code, use `io.vertx.redis.client.RedisAPI`
* If you need to emit custom commands, you can either use the data sources (reactive or imperative) or the `io.vertx.mutiny.redis.client.Redis`.


== Default and named clients

This extension lets you configure a _default_ Redis client/data sources or _named_ ones.
Expand All @@ -95,7 +93,6 @@ The latter is essential when you need to connect to multiple Redis instances.
The default connection is configured using the `quarkus.redis.*` properties.
For example, to configure the default Redis client, use:


[source,properties]
----
quarkus.redis.hosts=redis://localhost/
Expand All @@ -116,7 +113,6 @@ public class RedisExample {

TIP: In general, you inject a single one; the previous snippet is just an example.


_Named_ clients are configured using the `quarkus.redis.<name>.*` properties:

[source,properties]
Expand Down Expand Up @@ -318,6 +314,7 @@ public class MyRedisService {
}
}
----

<1> Inject the `RedisDataSource` in the constructor
<2> Creates the `HashCommands` object.
This object has three type parameters: the type of the key, the type of the field, and the type of the member
Expand All @@ -327,10 +324,51 @@ This object has three type parameters: the type of the key, the type of the fiel
=== Serialization and Deserialization

The data source APIs handle the serialization and deserialization automatically.
When a non-standard type is used, the object is serialized into JSON and deserialized from JSON.
By default, non-standard types are serialized into JSON and deserialized from JSON.
In this case, `quarkus-jackson` is used.

To store binary data, use `byte[]`.
=== Binary

To store or retrieve binary data, use `byte[]`.

=== Custom codec

You can register custom codec by providing a CDI _bean_ implementing the `io.quarkus.redis.datasource.codecs.Codec` interface:

[source,java]
----
import java.lang.reflect.Type;
import java.nio.charset.StandardCharsets;
import jakarta.enterprise.context.ApplicationScoped;
import io.quarkus.redis.datasource.codecs.Codec;
@ApplicationScoped
public class PersonCodec implements Codec {
@Override
public boolean canHandle(Type clazz) {
return clazz.equals(Person.class);
}
@Override
public byte[] encode(Object item) {
var p = (Person) item;
return (p.firstName + ";" + p.lastName.toUpperCase()).getBytes(StandardCharsets.UTF_8);
}
@Override
public Object decode(byte[] item) {
var value = new String(item, StandardCharsets.UTF_8);
var segments = value.split(";");
return new Person(segments[0], segments[1]);
}
}
----

The `canHandle` method is called to check if the codec can handle a specific type.
The parameter received in the `encode` method matches that type.
The object returned by the `decode` method must also match that type.

=== The `value` group

Expand Down Expand Up @@ -443,6 +481,7 @@ public static class MyRedisCounter {
}
----

<1> Retrieve the commands.
This time we will manipulate `Long` values
<2> Retrieve the counter associated with the given `key`.
Expand All @@ -464,7 +503,7 @@ These features are available from the `pubsub` group.

The following snippets shows how a _cache_ can emit a `Notification` after every `set`, and how a subscriber can receive the notification.

[source, java]
[source,java]
----
public static final class Notification {
public String key;
Expand Down Expand Up @@ -542,7 +581,7 @@ When that consumer returns, the transaction is _executed_.

The following snippet shows how to create a transaction executing two related _writes_:

[source, java]
[source,java]
----
@Inject RedisDataSource ds;
Expand All @@ -560,7 +599,7 @@ The returned `TransactionResult` lets you retrieve the result of each command.

When using the reactive variant of the data source, the passed callback is a `Function<ReactiveTransactionalRedisDataSource, Uni<Void>>`:

[source, java]
[source,java]
----
@Inject ReactiveRedisDataSource ds;
Expand All @@ -577,7 +616,7 @@ Transaction execution can be conditioned by _keys_.
When a passed key gets modified during the execution of a transaction, the transaction is discarded.
The keys are passed as `String` as a second parameter to the `withTransaction` method:

[source, java]
[source,java]
----
TransactionResult result = ds.withTransaction(tx -> {
TransactionalHashCommands<String, String, String> hash = tx.hash(String.class);
Expand Down Expand Up @@ -608,7 +647,7 @@ EXEC

For example, if you need to update a value in a hash only if the field exists, you will use the following API:

[source, java]
[source,java]
----
OptimisticLockingTransactionResult<Boolean> result = blocking.withTransaction(ds -> {
// The pre-transaction block:
Expand Down Expand Up @@ -641,7 +680,7 @@ The transaction is aborted if the pre-transaction block throws an exception (or

To execute a custom command, or a command not supported by the API, use the following approach:

[source, java]
[source,java]
----
@Inject ReactiveRedisDataSource ds;
Expand All @@ -667,7 +706,7 @@ On startup, you can configure the Redis client to preload data into the Redis da

Specify the _load script_ you want to load using:

[source, properties]
[source,properties]
----
quarkus.redis.load-script=import.redis # import.redis is the default in dev mode, no-file is the default in production mode
quarkus.redis.my-redis.load-script=actors.redis, movies.redis
Expand All @@ -682,7 +721,7 @@ In the case of a list, the data is imported in the list order (for example, firs

The `.redis` file follows a _one command per line_ format:

[source, text]
[source,text]
----
# Line starting with # and -- are ignored, as well as empty lines
Expand All @@ -701,7 +740,7 @@ Quarkus batches all the commands from a single file and sends all the commands.
The loading process fails if there is any error, but the previous instructions may have been executed.
To avoid that, you can wrap your command into a Redis _transaction_:

[source, text]
[source,text]
----
-- Run inside a transaction
MULTI
Expand All @@ -725,7 +764,7 @@ You can force to import even if there is data using the `quarkus.redis.load-only
As mentioned above, in dev and test modes, Quarkus tries to import data by looking for the `src/main/resources/import.redis`.
This behavior is disabled in _prod_ mode, and if you want to import even in production, add:

[source, properties]
[source,properties]
----
%prod.quarkus.redis.load-script=import.redis
----
Expand Down Expand Up @@ -772,8 +811,8 @@ public class ExampleRedisHostProvider implements RedisHostsProvider {
----

The host provider can be used to configure the redis client like shown below
[source,properties,indent=0]

[source,properties,indent=0]
----
quarkus.redis.hosts-provider-name=hosts-provider
----
Expand All @@ -783,7 +822,7 @@ quarkus.redis.hosts-provider-name=hosts-provider
You can expose a bean implementing the `io.quarkus.redis.client.RedisOptionsCustomizer` interface to customize the Redis client options.
The bean is called for each configured Redis client:

[source, java]
[source,java]
----
@ApplicationScoped
public static class MyExampleCustomizer implements RedisOptionsCustomizer {
Expand All @@ -800,7 +839,6 @@ public static class MyExampleCustomizer implements RedisOptionsCustomizer {
}
----


=== Dev Services

See xref:redis-dev-services.adoc[Redis Dev Service].
Expand All @@ -814,7 +852,7 @@ Micrometer collects the metrics of all the Redis clients implemented by the appl

As an example, if you export the metrics to Prometheus, you will get:

[source, text]
[source,text]
----
# HELP redis_commands_duration_seconds The duration of the operations (commands of batches
# TYPE redis_commands_duration_seconds summary
Expand Down Expand Up @@ -871,12 +909,12 @@ The metrics contain both the Redis connection pool metrics (`redis_pool_*`) and

To disable the Redis client metrics when `quarkus-micrometer` is used, add the following property to the application configuration:

[source, properties]
[source,properties]
----
quarkus.micrometer.binder.redis.enabled=false
----

[[redis-configuration-reference]]
== Configuration reference

include::{generated-dir}/config/quarkus-redis-client.adoc[opts=optional, leveloffset=+1]
include::{generated-dir}/config/quarkus-redis-client.adoc[opts=optional,leveloffset=+1]
2 changes: 1 addition & 1 deletion extensions/redis-client/deployment/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-resteasy-mutiny-deployment</artifactId>
<artifactId>quarkus-resteasy-reactive-jackson-deployment</artifactId>
<scope>test</scope>
</dependency>
<dependency>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,11 @@

import org.eclipse.microprofile.config.ConfigProvider;
import org.jboss.jandex.AnnotationInstance;
import org.jboss.jandex.ClassInfo;
import org.jboss.jandex.DotName;
import org.jboss.jandex.IndexView;

import io.quarkus.arc.deployment.AdditionalBeanBuildItem;
import io.quarkus.arc.deployment.BeanArchiveIndexBuildItem;
import io.quarkus.arc.deployment.BeanDiscoveryFinishedBuildItem;
import io.quarkus.arc.deployment.SyntheticBeanBuildItem;
Expand All @@ -23,9 +25,11 @@
import io.quarkus.deployment.annotations.BuildStep;
import io.quarkus.deployment.annotations.ExecutionTime;
import io.quarkus.deployment.annotations.Record;
import io.quarkus.deployment.builditem.CombinedIndexBuildItem;
import io.quarkus.deployment.builditem.ShutdownContextBuildItem;
import io.quarkus.redis.datasource.ReactiveRedisDataSource;
import io.quarkus.redis.datasource.RedisDataSource;
import io.quarkus.redis.datasource.codecs.Codec;
import io.quarkus.redis.runtime.client.RedisClientRecorder;
import io.quarkus.vertx.deployment.VertxBuildItem;

Expand Down Expand Up @@ -65,6 +69,15 @@ public void detectUsage(BuildProducer<RequestedRedisClientBuildItem> request,
}
}

@BuildStep
public void makeCodecsUnremovable(CombinedIndexBuildItem index, BuildProducer<AdditionalBeanBuildItem> producer) {
producer.produce(AdditionalBeanBuildItem.unremovableOf(Codec.class));

for (ClassInfo implementor : index.getIndex().getAllKnownImplementors(Codec.class.getName())) {
producer.produce(AdditionalBeanBuildItem.unremovableOf(implementor.name().toString()));
}
}

@BuildStep
@Record(ExecutionTime.RUNTIME_INIT)
public void init(RedisClientRecorder recorder,
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
package io.quarkus.redis.client.deployment.datasource;

import static org.assertj.core.api.Assertions.assertThat;

import java.lang.reflect.Type;
import java.nio.charset.StandardCharsets;
import java.util.UUID;

import jakarta.enterprise.context.ApplicationScoped;
import jakarta.inject.Inject;

import org.jboss.shrinkwrap.api.ShrinkWrap;
import org.jboss.shrinkwrap.api.spec.JavaArchive;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.RegisterExtension;

import io.quarkus.redis.client.deployment.RedisTestResource;
import io.quarkus.redis.datasource.RedisDataSource;
import io.quarkus.redis.datasource.codecs.Codec;
import io.quarkus.redis.datasource.codecs.Codecs;
import io.quarkus.redis.datasource.hash.HashCommands;
import io.quarkus.test.QuarkusUnitTest;
import io.quarkus.test.common.QuarkusTestResource;

@QuarkusTestResource(RedisTestResource.class)
public class CustomCodecTest {

@RegisterExtension
static final QuarkusUnitTest unitTest = new QuarkusUnitTest()
.setArchiveProducer(
() -> ShrinkWrap.create(JavaArchive.class).addClass(Jedi.class).addClass(MyCustomCodec.class))
.overrideConfigKey("quarkus.redis.hosts", "${quarkus.redis.tr}");

@Inject
RedisDataSource ds;

@Test
void testCustomCodecs() {
String key1 = UUID.randomUUID().toString();
// Check that the codec is registered
assertThat(Codecs.getDefaultCodecFor(Jedi.class)).isInstanceOf(MyCustomCodec.class);

HashCommands<String, String, Jedi> hash1 = ds.hash(Jedi.class);
hash1.hset(key1, "test", new Jedi("luke", "skywalker"));
var retrieved = hash1.hget(key1, "test");
assertThat(retrieved.firstName).isEqualTo("luke");
assertThat(retrieved.lastName).isEqualTo("SKYWALKER");

HashCommands<String, Jedi, String> hash2 = ds.hash(String.class, Jedi.class, String.class);
hash2.hset(key1, new Jedi("luke", "skywalker"), "test");
var retrieved2 = hash2.hget(key1, new Jedi("luke", "skywalker"));
assertThat(retrieved2).isEqualTo("test");

HashCommands<Jedi, String, String> hash3 = ds.hash(Jedi.class, String.class, String.class);
hash3.hset(new Jedi("luke", "skywalker"), "key", "value");
var retrieved3 = hash3.hget(new Jedi("luke", "skywalker"), "key");
assertThat(retrieved3).isEqualTo("value");
}

@ApplicationScoped
public static class MyCustomCodec implements Codec {

@Override
public boolean canHandle(Type clazz) {
return clazz.equals(Jedi.class);
}

@Override
public byte[] encode(Object item) {
var jedi = (Jedi) item;
return (jedi.firstName + ";" + jedi.lastName).getBytes(StandardCharsets.UTF_8);
}

@Override
public Object decode(byte[] item) {
String s = new String(item, StandardCharsets.UTF_8);
String[] strings = s.split(";");
return new Jedi(strings[0], strings[1].toUpperCase());
}
}

public static class Jedi {
public final String firstName;
public final String lastName;

public Jedi(String firstName, String lastName) {
this.firstName = firstName;
this.lastName = lastName;
}
}

}
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ default List<String> toArgs() {
* @param encoder an optional encoder to encode some of the values
* @return the list of arguments, encoded as a list of String.
*/
default <T> List<String> toArgs(Codec<T> encoder) {
default List<String> toArgs(Codec encoder) {
return Collections.emptyList();
}

Expand Down
Loading

0 comments on commit 798f1ce

Please sign in to comment.