Binary to Java String to Redis Lua and back again

[ceckhardt]

Hi all,

I am having trouble figuring out an exact encoding issue when sending binary as strings to EVAL through lettuce, vs using the lettuce command apis.

Specifically, if I call set(K, V) with the lettuce api where V is an object which uses the byte[] codec, vs calling EVAL with a Lua script that calls set using strings that are US_ASCII encoded, the results in redis are different.

Does anyone know the exact combination of encoding / StringCodec configurations that when using EVAL, the resulting binary will be the same as calling set with a byte array codec?

Thanks

[mp911de]

Can you provide a reproducer that shows the problem you're trying to solve?

[ceckhardt]

It's difficult to create a simple repro, as there are a lot of parts. I'm taking protobuf objects, and using the internal .toByteString() method to get a utf8, or a us_ascii string before sending it to eval. When I compare the results of what is stored in redis using eval vs the lettuce set() api, there are odd bytes that don't match each other.

I'll also mentioned that if I keep eval out of the test, i.e. pass the object through string codec's encodeValue, and byte codec's decodeValue and vice versa the problem doesn't exist. My guess it is something happens to the encoding when it is written to the socket, or processed by redis (Lua).

What I ended up doing is changing from string codec to the byte codec, and the problem went away. I just can't pass strings (v) to eval and need to grab byte arrays from everything I send.

[mp911de]

When using StringCodec, then the String encoding always interferes with the binary payload. You would need to use the ByteArrayCodec to preserve the binary content. You can also create a custom codec RedisCodec<String, byte[]> to conveniently use strings as keys and byte arrays for values. Your codec can delegate encode/decode methods to the string and byte array codec for easier implementation.