Skip to content

Commit

Permalink
Split cluster support in a separate gem
Browse files Browse the repository at this point in the history
  • Loading branch information
byroot committed Aug 17, 2022
1 parent 93e5dc8 commit f18566d
Show file tree
Hide file tree
Showing 44 changed files with 573 additions and 402 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -197,7 +197,7 @@ jobs:
LOW_TIMEOUT: "0.14"
DRIVER: ruby
REDIS_BRANCH: "7.0"
REDIS_CLUSTER: "true"
BUNDLE_GEMFILE: redis_cluster/Gemfile
steps:
- name: Check out code
uses: actions/checkout@v3
Expand Down
4 changes: 4 additions & 0 deletions .rubocop.yml
Original file line number Diff line number Diff line change
Expand Up @@ -148,3 +148,7 @@ Style/SymbolProc:

Bundler/OrderedGems:
Enabled: false

Gemspec/RequiredRubyVersion:
Exclude:
- redis_cluster/redis_cluster.gemspec
1 change: 0 additions & 1 deletion Gemfile
Original file line number Diff line number Diff line change
Expand Up @@ -11,4 +11,3 @@ gem 'mocha'

gem 'redis-client', github: 'redis-rb/redis-client'
gem 'hiredis-client'
gem 'redis-cluster-client', github: 'redis-rb/redis-cluster-client' if ENV['REDIS_CLUSTER']
54 changes: 1 addition & 53 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -116,59 +116,7 @@ redis = Redis.new(name: 'mymaster', sentinels: SENTINELS, role: :master)

## Cluster support

`redis-rb` supports [clustering](https://redis.io/topics/cluster-spec).

```ruby
# Nodes can be passed to the client as an array of connection URLs.
nodes = (7000..7005).map { |port| "redis://127.0.0.1:#{port}" }
redis = Redis.new(cluster: nodes)

# You can also specify the options as a Hash. The options are the same as for a single server connection.
(7000..7005).map { |port| { host: '127.0.0.1', port: port } }
```

You can also specify only a subset of the nodes, and the client will discover the missing ones using the [CLUSTER NODES](https://redis.io/commands/cluster-nodes) command.

```ruby
Redis.new(cluster: %w[redis://127.0.0.1:7000])
```

If you want [the connection to be able to read from any replica](https://redis.io/commands/readonly), you must pass the `replica: true`. Note that this connection won't be usable to write keys.

```ruby
Redis.new(cluster: nodes, replica: true)
```

The calling code is responsible for [avoiding cross slot commands](https://redis.io/topics/cluster-spec#keys-distribution-model).

```ruby
redis = Redis.new(cluster: %w[redis://127.0.0.1:7000])

redis.mget('key1', 'key2')
#=> Redis::CommandError (CROSSSLOT Keys in request don't hash to the same slot)

redis.mget('{key}1', '{key}2')
#=> [nil, nil]
```

* The client automatically reconnects after a failover occurred, but the caller is responsible for handling errors while it is happening.
* The client support permanent node failures, and will reroute requests to promoted slaves.
* The client supports `MOVED` and `ASK` redirections transparently.

## Cluster mode with SSL/TLS
Since Redis can return FQDN of nodes in reply to client since `7.*` with CLUSTER commands, we can use cluster feature with SSL/TLS connection like this:

```ruby
Redis.new(cluster: %w[rediss://foo.example.com:6379])
```

On the other hand, in Redis versions prior to `6.*`, you can specify options like the following if cluster mode is enabled and client has to connect to nodes via single endpoint with SSL/TLS.

```ruby
Redis.new(cluster: %w[rediss://foo-endpoint.example.com:6379], fixed_hostname: 'foo-endpoint.example.com')
```

In case of the above architecture, if you don't pass the `fixed_hostname` option to the client and servers return IP addresses of nodes, the client may fail to verify certificates.
[Clustering](https://redis.io/topics/cluster-spec). is supported via the [`redis_cluster` gem](redis_cluster/).

## Storing objects

Expand Down
11 changes: 10 additions & 1 deletion Rakefile
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
# frozen_string_literal: true

require 'bundler/gem_tasks'
Bundler::GemHelper.install_tasks(dir: "redis_cluster", name: "redis_cluster")

require 'rake/testtask'

namespace :test do
groups = %i(redis distributed sentinel cluster)
groups = %i(redis distributed sentinel)
groups.each do |group|
Rake::TestTask.new(group) do |t|
t.libs << "test"
Expand All @@ -18,6 +20,13 @@ namespace :test do
unless lost_tests.empty?
abort "The following test files are in no group:\n#{lost_tests.join("\n")}"
end

Rake::TestTask.new(:cluster) do |t|
t.libs << "redis_cluster/test" << "test"
t.libs << "redis_cluster/lib" << "lib"
t.test_files = FileList["redis_cluster/test/**/*_test.rb"]
t.options = '-v' if ENV['CI'] || ENV['VERBOSE']
end
end

task test: ["test:redis", "test:distributed", "test:sentinel", "test:cluster"]
Expand Down
2 changes: 1 addition & 1 deletion bin/cluster_creator
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ puts ARGV.join(" ")
require 'bundler/setup'

$LOAD_PATH.unshift(File.expand_path('../lib', __dir__))
require_relative '../test/support/cluster/orchestrator'
require_relative '../redis_cluster/test/support/orchestrator'

urls = ARGV.map { |host_port| "redis://#{host_port}" }
orchestrator = ClusterOrchestrator.new(urls, timeout: 3.0)
Expand Down
68 changes: 28 additions & 40 deletions lib/redis.rb
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,6 @@ class Redis
BASE_PATH = __dir__
Deprecated = Class.new(StandardError)

autoload :ClusterClient, "redis/cluster_client"

class << self
attr_accessor :silence_deprecations, :raise_deprecations

Expand Down Expand Up @@ -50,12 +48,6 @@ def deprecate!(message)
# @option options [Boolean] :inherit_socket (false) Whether to use socket in forked process or not
# @option options [String] :name The name of the server group to connect to.
# @option options [Array] :sentinels List of sentinels to contact
# @option options [Symbol] :role (:master) Role to fetch via Sentinel, either `:master` or `:slave`
# @option options [Array<String, Hash{Symbol => String, Integer}>] :cluster List of cluster nodes to contact
# @option options [Boolean] :replica Whether to use readonly replica nodes in Redis Cluster or not
# @option options [String] :fixed_hostname Specify a FQDN if cluster mode enabled and
# client has to connect nodes via single endpoint with SSL/TLS
# @option options [Class] :connector Class of custom connector
#
# @return [Redis] a new client instance
def initialize(options = {})
Expand All @@ -68,34 +60,7 @@ def initialize(options = {})
inherit_socket = @options.delete(:inherit_socket)
@subscription_client = nil

@client = if @cluster_mode = options.key?(:cluster)
@options[:nodes] ||= @options.delete(:cluster)
cluster_config = RedisClient.cluster(**@options, protocol: 2, client_implementation: ClusterClient)
begin
cluster_config.new_client
rescue ::RedisClient::Error => error
raise ClusterClient::ERROR_MAPPING.fetch(error.class), error.message, error.backtrace
end
elsif @options.key?(:sentinels)
if url = @options.delete(:url)
uri = URI.parse(url)
if !@options.key?(:name) && uri.host
@options[:name] = uri.host
end

if !@options.key?(:password) && uri.password && !uri.password.empty?
@options[:password] = uri.password
end

if !@options.key?(:username) && uri.user && !uri.user.empty?
@options[:username] = uri.user
end
end

Client.sentinel(**@options).new_client
else
Client.config(**@options).new_client
end
@client = initialize_client(@options)
@client.inherit_socket! if inherit_socket
end

Expand Down Expand Up @@ -145,10 +110,6 @@ def dup
end

def connection
if @cluster_mode
raise NotImplementedError, "Redis::Cluster doesn't implement #connection"
end

{
host: @client.host,
port: @client.port,
Expand All @@ -160,6 +121,33 @@ def connection

private

def initialize_client(options)
if options.key?(:cluster)
raise "Redis Cluster support was moved to the `redis_cluster` gem."
end

if options.key?(:sentinels)
if url = options.delete(:url)
uri = URI.parse(url)
if !options.key?(:name) && uri.host
options[:name] = uri.host
end

if !options.key?(:password) && uri.password && !uri.password.empty?
options[:password] = uri.password
end

if !options.key?(:username) && uri.user && !uri.user.empty?
options[:username] = uri.user
end
end

Client.sentinel(**options).new_client
else
Client.config(**options).new_client
end
end

def synchronize
@monitor.synchronize { yield(@client) }
end
Expand Down
92 changes: 0 additions & 92 deletions lib/redis/cluster_client.rb

This file was deleted.

19 changes: 1 addition & 18 deletions lib/redis/commands/cluster.rb
Original file line number Diff line number Diff line change
Expand Up @@ -12,24 +12,7 @@ module Cluster
#
# @return [Object] depends on the subcommand
def cluster(subcommand, *args)
subcommand = subcommand.to_s.downcase
block = case subcommand
when 'slots'
HashifyClusterSlots
when 'nodes'
HashifyClusterNodes
when 'slaves'
HashifyClusterSlaves
when 'info'
HashifyInfo
else
Noop
end

# @see https://github.com/antirez/redis/blob/unstable/src/redis-trib.rb#L127 raw reply expected
block = Noop unless @cluster_mode

send_command([:cluster, subcommand] + args, &block)
send_command([:cluster, subcommand] + args)
end

# Sends `ASKING` command to random node and returns its reply.
Expand Down
36 changes: 0 additions & 36 deletions lib/redis/errors.rb
Original file line number Diff line number Diff line change
Expand Up @@ -52,40 +52,4 @@ class InheritedError < BaseConnectionError
# Raised when client options are invalid.
class InvalidClientOptionError < BaseError
end

class Cluster
# Raised when client connected to redis as cluster mode
# and failed to fetch cluster state information by commands.
class InitialSetupError < BaseError
end

# Raised when client connected to redis as cluster mode
# and some cluster subcommands were called.
class OrchestrationCommandNotSupported < BaseError
def initialize(command, subcommand = '')
str = [command, subcommand].map(&:to_s).reject(&:empty?).join(' ').upcase
msg = "#{str} command should be used with care "\
'only by applications orchestrating Redis Cluster, like redis-trib, '\
'and the command if used out of the right context can leave the cluster '\
'in a wrong state or cause data loss.'
super(msg)
end
end

# Raised when error occurs on any node of cluster.
class CommandErrorCollection < BaseError
attr_reader :errors

# @param errors [Hash{String => Redis::CommandError}]
# @param error_message [String]
def initialize(errors, error_message = 'Command errors were replied on any node')
@errors = errors
super(error_message)
end
end

# Raised when cluster client can't select node.
class AmbiguousNodeError < BaseError
end
end
end
Empty file added redis_cluster/CHANGELOG.md
Empty file.
11 changes: 11 additions & 0 deletions redis_cluster/Gemfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# frozen_string_literal: true

source 'https://rubygems.org'

gemspec

gem 'minitest'
gem 'rake'
gem 'rubocop', '~> 1.25.1'
gem 'mocha'
gem 'redis-cluster-client', github: 'redis-rb/redis-cluster-client'
Loading

0 comments on commit f18566d

Please sign in to comment.