-
Notifications
You must be signed in to change notification settings - Fork 3.9k
Home
Check out the ReleaseNotes.
In order to have Jedis as a dependency in your application you can:
That is very easy, you just need to:
git clone git://github.com/xetorthio/jedis.git
Before you package it using maven, for the tests to succeed, you need to have two Redis server instances running. Simply use the two Redis conf files in the conf directory with your up-to-date redis build. On two separate command lines, just run:
redis-server jedis/conf/redis.conf
redis-server jedis/conf/redis2.conf
(one is on the default port (6379) and the other on 6380, both with "foobared" as password)
then finally run
mvn package
to build, run the tests and package.
Just go to the Downloads section and use the latest Jedis JAR available.
You will also need to download Apache Commons http://commons.apache.org/pool/download_pool.cgi
Jedis is also distributed as a Maven Dependency through Sonatype. To configure that just add the following XML snippet to your pom.xml file.
<dependency>
<groupId>redis.clients</groupId>
<artifactId>jedis</artifactId>
<version>2.0.0</version>
<type>jar</type>
<scope>compile</scope>
</dependency>You shouldn't use the same instance from different threads because you'll have strange errors. And sometimes creating lots of Jedis instances is not good enough because it means lots of sockets and connections, which leads to strange errors as well. A single Jedis instance is not threadsafe! To avoid these problems, you should use JedisPool, which is a threadsafe pool of network connections. You can use the pool to reliably create several Jedis instances, given you return the Jedis instance to the pool when done. This way you can overcome those strange errors and achieve great performance.
To use it, init a pool:
JedisPool pool = new JedisPool(new JedisPoolConfig(), "localhost");You can store the pool somewhere statically, it is thread-safe.
JedisPoolConfig includes a number of helpful Redis-specific connection pooling defaults. For example, Jedis with JedisPoolConfig will close a connection after 300 seconds if it has not been returned.
You use it by:
Jedis jedis = pool.getResource();
try {
/// ... do stuff here ... for example
jedis.set("foo", "bar");
String foobar = jedis.get("foo");
jedis.zadd("sose", 0, "car"); jedis.zadd("sose", 0, "bike");
Set<String> sose = jedis.zrange("sose", 0, -1);
} finally {
/// ... it's important to return the Jedis instance to the pool once you've finished using it
pool.returnResource(jedis);
}
/// ... when closing your application:
pool.destroy();###Transactions To do transactions in Jedis, you have to wrap operations in a transaction block, very similar to pipelining:
jedis.watch (key1, key2, ...);
BinaryTransaction t = jedis.multi();
t.set("foo", "bar");
t.exec();Note: when you have any method that returns values, you have to do like this:
t.get("foo");
t.hgetAll("car");
List<Object> all = t.exec();
String result1 = SafeEncoder.encode(all.get(1)); // get the result of the first get in the transaction.Note 2: From version 2.0 there is a much improved support for transactions. It provides a convenient access the results of the transaction without the need to extract them of the List as showed above. Note that a Response Object does not contain the result before t.exec() is called (it is a kind of a Future).
Transaction t = jedis.multi();
t.set("fool", "bar");
Response<String> result1 = t.get("fool");
t.zadd("foo", 1, "barowitch"); t.zadd("foo", 0, "barinsky"); t.zadd("foo", 0, "barikoviev");
Response<Set<String>> sose = t.zrange("foo", 0, -1); // get the entire sortedset
t.exec(); // dont forget it
String foolbar = result1.get()); // use Response.get() to retrieve things from a Response
int soseSize = sose.get().size(); // on sose.get() you can directly call Set methods!
// List<Object> allResults = t.exec(); // you could still get all results at once, as before
int wontwork = allResults.get(5).size(); // but this won't work to access the set. see aboveSometimes you need to send a bunch of different commands. A very cool way to do that, and have better performance than doing it the naive way, is to use pipelining. This way you send commands without waiting for response, and you actually read the responses at the end, which is faster.
Here is how to do it (very similar to transactions):
Pipeline p = jedis.pipelined();
p.set("foo", "bar");
p.get("foo");
List<Object> results = p.exec();
String result1 = SafeEncoder.encode(results.get(1)); // get the result of the first get in the pipeline.From version 1.5.3 there is a much more convenient way for creating pipelines, without the need to deal with positions and conversions after p.exec() :
Pipeline p = jedis.pipelined();
p.set("fool", "bar");
p.zadd("foo", 1, "barowitch"); p.zadd("foo", 0, "barinsky"); p.zadd("foo", 0, "barikoviev");
Response<String> pipeString = p.get("fool");
Response<Set<String>> sose = p.zrange("foo", 0, -1);
p.sync();
int soseSize = sose.get().size();
Set<String> setBack = sose.get();For more explanations see code comments in the transaction section.
To subscribe to a channel in Redis, create an instance of JedisPubSub and call subscribe on the Jedis instance:
class MyListener extends JedisPubSub {
public void onMessage(String channel, String message) {
}
public void onSubscribe(String channel, int subscribedChannels) {
}
public void onUnsubscribe(String channel, int subscribedChannels) {
}
public void onPSubscribe(String pattern, int subscribedChannels) {
}
public void onPUnsubscribe(String pattern, int subscribedChannels) {
}
public void onPMessage(String pattern, String channel,
String message) {
}
}
MyListener l = new MyListener();
jedis.subscribe(l, "foo");Note that subscribe is a blocking operation operation because it will poll Redis for responses on the thread that calls subscribe. A single JedisPubSub instance can be used to subscribe to multiple channels. You can call subscribe or psubscribe on an existing JedisPubSub instance to change your subscriptions.
####Motivation In the normal Redis master-slave approach, generally there is one master that serves write requests, and many slaves that serve read requests. This means, the user has to take care of effectively distributing the load on the slaves. Furthermore, only reads scale with the number of slaves, but writes do not, since there can be only one master! With ShardedJedis you achieve scalability for both reads and writes. Sharding uses a technique called "consistent hashing" and assigns the keys equally on a set of redis servers according to some hash algorithm (md5 and murmur, the latter being less standard, but faster). A node like this is then called a "shard". A further advantage is that each shards only needs to have RAM 1/n the size of the total dataset (for n being the number of participating slaves).
####The downside Since each shard is a separate master, sharding has limited functionality: i.e. you cannot use transactions, pipelining, pub/sub, especially not across shards! However, generally it is feasible to do a not allowed operation, as long as the concerned keys are on the same shard (check / ask the forum). You can influence which key go to which shard by keytags (see below). A further downside is that in the current standard implementation, shards cannot be added or removed from a running ShardedJedis. If you need this feature, there is an experimental reimplementation of ShardedJedis which allows adding and removing shards of a running ShardedJedis: yaourt - dynamic sharding implementation
If you want easy load distribution of ShardedJedis, but still need transactions/pipelining/pubsub etc, you can also mix the normal and the sharded approach: define a master as normal Jedis, the others as sharded Jedis. Then make all the shards slaveof master. In your application, direct your write requests to the master, the read requests to ShardedJedis. Your writes don't scale anymore, but you gain good read distribution, and you have transactions/pipelining/pubsub simply using the master. Dataset should fit in RAM of master. Remember that you can improve performance of the master a lot, if you let the slaves do the persistance for the master!
####Alternatives
There is another, very simple but effective strategy for distribution: "RoundRobin" simply cycles through the available slaves. The advantage is that no hashing/mapping occurs, and distribution is independent on the key name, hence more equal especially if particular keys are used more frequently at certain times. In future versions, it will allow to add and remove slaves at runtime, tuneable redundancy, and simultaneous multi-slave download of i.e. bigger lists/sortedsets. This is not possible with ShardedJedis. The downside is that unlike ShardedJedis RoundRobin does not allow scaling of writes, it only manages the master-slave setting. Furthermore, each slave has to manage the whole dataset, hence for optimal performance, RAM should be 1x, not 1/n of total dataset size. Since it is not yet part of official jedis, you have to build Jedis from source and add "RoundRobinPool" before make. Just as with JedisPool, call getRessource() on it to get a Jedis instance and don't forget returnResource after usage. Note, you only distribute the load by cycling through the slaves if you do get/returnResource frequently. For convenience and other reasons, consider UnifiedJedis.
The utility class UnifiedJedis abstracts away the pool operations and also greatly simplifies the general usage of Jedis: you use it just as a fully operational Jedis, but under the hood it delegates writes to the master, and reads to cycling Jedis instances using RoundRobinPool. Besides making distribution transparent, it also avoids difficult to track errors: if by negligence you direct a write process to a slave, there is no error, the slave accepts and saves it just fine, but the change is not replicated and eventually overwritten. This can no longer happen. Furthermore, since it calls get/returnResource for each single Jedis method of both master and RoundRobin, exhaustion of the pool by forgetting or delaying returnResource is avoided, and load is distributed very equally. You have to build Jedis from source and add RoundRobinPool and UnifiedJedis class files before make. Note that for advanced functions like transactions/pipeline and pubsub, you currently you have to obtain a Jedis instance of the master manually like so: Jedis master = uj.getWritePool().getResource() and after usage: uj.getWritePool().returnResource(master); UnifiedJedis maybe one day adapted for ShardedJedis too.
Sometime later 2011, there will be first versions of "redis cluster" which will be a much improved Sharded Jedis and should give back some if not all of the Redis functionalities you cannot have with shardedJedis. If you want to know more about redis cluster, youtube has a presentation of Salvatore Sanfilippo (the creator of Redis).
Here is the general proceeding:
List<JedisShardInfo> shards = new ArrayList<JedisShardInfo>();
JedisShardInfo si = new JedisShardInfo("localhost", 6379);
si.setPassword("foobared");
shards.add(si);
si = new JedisShardInfo("localhost", 6380);
si.setPassword("foobared");
shards.add(si);Then, there are two ways of using ShardedJedis. Direct connections or by using ShardedJedisPool. For reliable operation, the latter has to be used in a multithreaded environment.
ShardedJedis jedis = new ShardedJedis(shards);
jedis.set("a", "foo");
jedis.disconnect;ShardedJedisPool pool = new ShardedJedisPool(new Config(), shards);
ShardedJedis jedis = pool.getResource();
jedis.set("a", "foo");
.... // do your work here
pool.returnResource(jedis);
.... // a few moments later
ShardedJedis jedis2 = pool.getResource();
jedis.set("z", "bar");
pool.returnResource(jedis);
pool.destroy();pool.returnResource should be called as soon as you are finished using jedis in a particular moment. If you don't, the pool may get slower after a while. getResource and returnResource are fast, since no new connection have to be created. Creation and destruction of a pool are slower, since theses are the actual network connections. Forgetting pool.destroy keeps the connection open until timeout is reached.
ShardInfo si = jedis.getShardInfo(key);
si.getHost/getPort/getPassword/getTimeout/getNameWhat you need is something called "keytags", and they are supported by Jedis. To work with keytags you just need to set a pattern when you instance ShardedJedis. For example:
ShardedJedis jedis = new ShardedJedis(shards,
ShardedJedis.DEFAULT_KEY_TAG_PATTERN);You can create your own pattern if you want. The default pattern is {}, this means that whatever goes inside curly brackets will be used to determine the shard.
So for example:
jedis.set("foo{bar}", "12345");and
jedis.set("car{bar}", "877878");will go to the same shard.
To use the monitor command you can do something like the following:
new Thread(new Runnable() {
public void run() {
Jedis j = new Jedis("localhost");
for (int i = 0; i < 100; i++) {
j.incr("foobared");
try {
Thread.sleep(200);
} catch (InterruptedException e) {
}
}
j.disconnect();
}
}).start();
jedis.monitor(new JedisMonitor() {
public void onCommand(String command) {
System.out.println(command);
}
});Redis/Jedis talks a lot about Strings. And here http://redis.io/topics/internals it says Strings are the basic building block of Redis. However, this stress on strings may be misleading. Redis' "String" refer to the C char type (8 bit), which is incompatible with Java Strings (16-bit). Redis sees only 8-bit blocks of data of predefined length, so normally it doesn't interpret the data (it's "binary safe"). Therefore in Java, byte[] data is "native", whereas Strings have to be encoded before being sent, and decoded after being retrieved by the SafeEncoder. This can have a considerable performance impact. In short: if you have binary data, don't encode it into String, but use the binary versions.
A Redis network consists of redis servers, which can be either masters or slaves. Slaves are synchronized to the master (master/slave replication). However, master and slaves look identical to a client, and slaves do accept write requests, but they will not be propagated "up-hill" and could eventually be overwritten by the master. It makes sense to route reads to slaves, and write demands to the master. Furthermore, being a slave doesn't prevent from being considered master by another slave.