Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make command for redis and sentinel configurable #127

Merged
merged 2 commits into from
Mar 6, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
81 changes: 54 additions & 27 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,39 +75,41 @@ In order to have persistence, a `PersistentVolumeClaim` usage is allowed. The fu
**IMPORTANT**: By default, the persistent volume claims will be deleted when the Redis Failover is. If this is not the expected usage, a `keepAfterDeletion` flag can be added under the `storage` section of Redis. [An example is given](example/redisfailover/persistent-storage-no-pvc-deletion.yaml).

### NodeAffinity and Tolerations

You can use NodeAffinity and Tolerations to deploy Pods to isolated groups of Nodes

Example:

```yaml
apiVersion: v1
items:
- apiVersion: storage.spotahome.com/v1alpha2
kind: RedisFailover
metadata:
name: redis
spec:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: kops.k8s.io/instancegroup
operator: In
values:
- productionnodes
hardAntiAffinity: false
redis: null
sentinel:
replicas: 3
resources:
limits:
memory: 100Mi
requests:
cpu: 100m
tolerations:
- effect: NoExecute
key: dedicated
operator: Equal
value: production
- apiVersion: storage.spotahome.com/v1alpha2
kind: RedisFailover
metadata:
name: redis
spec:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: kops.k8s.io/instancegroup
operator: In
values:
- productionnodes
hardAntiAffinity: false
redis: null
sentinel:
replicas: 3
resources:
limits:
memory: 100Mi
requests:
cpu: 100m
tolerations:
- effect: NoExecute
key: dedicated
operator: Equal
value: production
kind: List
```

Expand Down Expand Up @@ -145,6 +147,31 @@ This behavior is configurable, creating a configmap and indicating to use it. An

**Important**: the configmap has to be in the same namespace. The configmap has to have a `shutdown.sh` data, containing the script.

### Custom command

By default, redis and sentinel will be called with de basic command, giving the configuration file:

- Redis: `redis-server /redis/redis.conf`
- Sentinel: `redis-server /redis/sentinel.conf --sentinel`

If necessary, this command can be changed with the `command` option inside redis/sentinel spec:

```yaml
sentinel:
command:
- "redis-server"
- "/redis/sentinel.conf"
- "--sentinel"
- "--protected-mode"
- "no"
redis:
customConfig:
- "redis-server"
- "/redis/redis.conf"
- "--protected-mode"
- "no"
```

### Connection

In order to connect to the redis-failover and use it, a [Sentinel-ready](https://redis.io/topics/sentinel-clients) library has to be used. This will connect through the Sentinel service to the Redis node working as a master.
Expand Down
2 changes: 2 additions & 0 deletions api/redisfailover/v1alpha2/types.go
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ type RedisSettings struct {
Image string `json:"image,omitempty"`
Version string `json:"version,omitempty"`
CustomConfig []string `json:"customConfig,omitempty"`
Command []string `json:"command,omitempty"`
ShutdownConfigMap string `json:"shutdownConfigMap,omitempty"`
Storage RedisStorage `json:"storage,omitempty"`
}
Expand All @@ -56,6 +57,7 @@ type SentinelSettings struct {
Replicas int32 `json:"replicas,omitempty"`
Resources RedisFailoverResources `json:"resources,omitempty"`
CustomConfig []string `json:"customConfig,omitempty"`
Command []string `json:"command,omitempty"`
}

// RedisFailoverResources sets the limits and requests for a container
Expand Down
34 changes: 25 additions & 9 deletions operator/redisfailover/service/generator.go
Original file line number Diff line number Diff line change
Expand Up @@ -159,6 +159,7 @@ func generateRedisStatefulSet(rf *redisfailoverv1alpha2.RedisFailover, labels ma

spec := rf.Spec
redisImage := getRedisImage(rf)
redisCommand := getRedisCommand(rf)
resources := getRedisResources(spec)
labels = util.MergeLabels(labels, generateLabels(redisRoleName, rf.Name))
volumeMounts := getRedisVolumeMounts(rf)
Expand Down Expand Up @@ -203,10 +204,7 @@ func generateRedisStatefulSet(rf *redisfailoverv1alpha2.RedisFailover, labels ma
},
},
VolumeMounts: volumeMounts,
Command: []string{
"redis-server",
fmt.Sprintf("/redis/%s", redisConfigFileName),
},
Command: redisCommand,
ReadinessProbe: &corev1.Probe{
InitialDelaySeconds: graceTime,
TimeoutSeconds: 5,
Expand Down Expand Up @@ -274,6 +272,7 @@ func generateSentinelDeployment(rf *redisfailoverv1alpha2.RedisFailover, labels

spec := rf.Spec
redisImage := getRedisImage(rf)
sentinelCommand := getSentinelCommand(rf)
resources := getSentinelResources(spec)
labels = util.MergeLabels(labels, generateLabels(sentinelRoleName, rf.Name))

Expand Down Expand Up @@ -349,11 +348,7 @@ func generateSentinelDeployment(rf *redisfailoverv1alpha2.RedisFailover, labels
MountPath: "/redis",
},
},
Command: []string{
"redis-server",
fmt.Sprintf("/redis/%s", sentinelConfigFileName),
"--sentinel",
},
Command: sentinelCommand,
ReadinessProbe: &corev1.Probe{
InitialDelaySeconds: graceTime,
TimeoutSeconds: 5,
Expand Down Expand Up @@ -653,3 +648,24 @@ func getRedisDataVolumeName(rf *redisfailoverv1alpha2.RedisFailover) string {
return redisStorageVolumeName
}
}

func getRedisCommand(rf *redisfailoverv1alpha2.RedisFailover) []string {
if len(rf.Spec.Redis.Command) > 0 {
return rf.Spec.Redis.Command
}
return []string{
"redis-server",
fmt.Sprintf("/redis/%s", redisConfigFileName),
}
}

func getSentinelCommand(rf *redisfailoverv1alpha2.RedisFailover) []string {
if len(rf.Spec.Sentinel.Command) > 0 {
return rf.Spec.Sentinel.Command
}
return []string{
"redis-server",
fmt.Sprintf("/redis/%s", sentinelConfigFileName),
"--sentinel",
}
}
103 changes: 103 additions & 0 deletions operator/redisfailover/service/generator_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -452,3 +452,106 @@ func TestRedisStatefulSetStorageGeneration(t *testing.T) {
assert.NoError(err)
}
}

func TestRedisStatefulSetCommands(t *testing.T) {
tests := []struct {
name string
givenCommands []string
expectedCommands []string
}{
{
name: "Default values",
givenCommands: []string{},
expectedCommands: []string{
"redis-server",
"/redis/redis.conf",
},
},
{
name: "Given commands should be used in redis container",
givenCommands: []string{
"test",
"command",
},
expectedCommands: []string{
"test",
"command",
},
},
}

for _, test := range tests {
assert := assert.New(t)

// Generate a default RedisFailover and attaching the required storage
rf := generateRF()
rf.Spec.Redis.Command = test.givenCommands

gotCommands := []string{}

ms := &mK8SService.Services{}
ms.On("CreateOrUpdatePodDisruptionBudget", namespace, mock.Anything).Once().Return(nil, nil)
ms.On("CreateOrUpdateStatefulSet", namespace, mock.Anything).Once().Run(func(args mock.Arguments) {
ss := args.Get(1).(*appsv1beta2.StatefulSet)
gotCommands = ss.Spec.Template.Spec.Containers[0].Command
}).Return(nil)

client := rfservice.NewRedisFailoverKubeClient(ms, log.Dummy)
err := client.EnsureRedisStatefulset(rf, nil, []metav1.OwnerReference{})

assert.Equal(test.expectedCommands, gotCommands)
assert.NoError(err)
}
}

func TestSentinelDeploymentCommands(t *testing.T) {
tests := []struct {
name string
givenCommands []string
expectedCommands []string
}{
{
name: "Default values",
givenCommands: []string{},
expectedCommands: []string{
"redis-server",
"/redis/sentinel.conf",
"--sentinel",
},
},
{
name: "Given commands should be used in sentinel container",
givenCommands: []string{
"test",
"command",
},
expectedCommands: []string{
"test",
"command",
},
},
}

for _, test := range tests {
assert := assert.New(t)

// Generate a default RedisFailover and attaching the required storage
rf := generateRF()
rf.Spec.Sentinel.Command = test.givenCommands

gotCommands := []string{}

ms := &mK8SService.Services{}
ms.On("CreateOrUpdatePodDisruptionBudget", namespace, mock.Anything).Once().Return(nil, nil)
ms.On("CreateOrUpdateDeployment", namespace, mock.Anything).Once().Run(func(args mock.Arguments) {
d := args.Get(1).(*appsv1beta2.Deployment)
gotCommands = d.Spec.Template.Spec.Containers[0].Command
}).Return(nil)

client := rfservice.NewRedisFailoverKubeClient(ms, log.Dummy)
err := client.EnsureSentinelDeployment(rf, nil, []metav1.OwnerReference{})

assert.Equal(test.expectedCommands, gotCommands)
assert.NoError(err)
}
}