Skip to content

Commit

Permalink
Issue 220: Added support for more zookeeper configuration options (#262)
Browse files Browse the repository at this point in the history 
* Added support for more zookeeper configuration options

Signed-off-by: anisha.kj <[email protected]>

* Fixed the comments

Signed-off-by: anisha.kj <[email protected]>

* Added misising fields in crd

Signed-off-by: anisha.kj <[email protected]>

* Set default value for session timeout

Signed-off-by: anisha.kj <[email protected]>
  • Loading branch information
@anishakj
anishakj authored Oct 22, 2020
1 parent 6cef7c5 commit cc11ea6
Show file tree
Hide file tree
Showing 7 changed files with 235 additions and 0 deletions.
53 changes: 53 additions & 0 deletions charts/zookeeper-operator/templates/zookeeper.pravega.io_zookeeperclusters_crd.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -69,11 +69,55 @@ spec:
is provided required default values will be provided, and optional
values will be excluded.
properties:
autoPurgePurgeInterval:
description: "The time interval in hours for which the purge task
has to be triggered \n Disabled by default"
type: integer
autoPurgeSnapRetainCount:
description: "Retain the snapshots according to retain count \n
The default value is 3"
type: integer
commitLogCount:
description: "Zookeeper maintains an in-memory list of last committed
requests for fast synchronization with followers \n The default
value is 500"
type: integer
globalOutstandingLimit:
description: "Clients can submit requests faster than ZooKeeper
can process them, especially if there are a lot of clients. Zookeeper
will throttle Clients so that requests won't exceed global outstanding
limit. \n The default value is 1000"
type: integer
initLimit:
description: "InitLimit is the amount of time, in ticks, to allow
followers to connect and sync to a leader. \n Default value is
10."
type: integer
maxClientCnxns:
description: "Limits the number of concurrent connections that a
single client, identified by IP address, may make to a single
member of the ZooKeeper ensemble. \n The default value is 60"
type: integer
maxCnxns:
description: "Limits the total number of concurrent connections
that can be made to a zookeeper server \n The defult value is
0, indicating no limit"
type: integer
maxSessionTimeout:
description: "The maximum session timeout in milliseconds that the
server will allow the client to negotiate. \n The default value
is 40000"
type: integer
minSessionTimeout:
description: "The minimum session timeout in milliseconds that the
server will allow the client to negotiate \n The default value
is 4000"
type: integer
preAllocSize:
description: "To avoid seeks ZooKeeper allocates space in the transaction
log file in blocks of preAllocSize kilobytes \n The default value
is 64M"
type: integer
quorumListenOnAllIPs:
description: "QuorumListenOnAllIPs when set to true the ZooKeeper
server will listen for connections from its peers on all available
Expand All @@ -82,6 +126,15 @@ spec:
the ZAB protocol and the Fast Leader Election protocol. \n The
default value is false."
type: boolean
snapCount:
description: "ZooKeeper records its transactions using snapshots
and a transaction log The number of transactions recorded in the
transaction log before a snapshot can be taken is determined by
snapCount \n The default value is 100,000"
type: integer
snapSizeLimitInKb:
description: "Snapshot size limit in Kb \n The defult value is 4GB"
type: integer
syncLimit:
description: "SyncLimit is the amount of time, in ticks, to allow
followers to sync with Zookeeper. \n The default value is 2."
Expand Down
11 changes: 11 additions & 0 deletions charts/zookeeper/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,17 @@ The following table lists the configurable parameters of the zookeeper chart and
| `config.initLimit` | Amount of time (in ticks) to allow followers to connect and sync to a leader | `10` |
| `config.tickTime` | Length of a single tick which is the basic time unit used by Zookeeper (measured in milliseconds) | `2000` |
| `config.syncLimit` | Amount of time (in ticks) to allow followers to sync with Zookeeper | `2` |
| `config.globalOutstandingLimit` | Max limit for outstanding requests | `1000` |
| `config.preAllocSize` | PreAllocSize in kilobytes | `65536` |
| `config.snapCount` | The number of transactions recorded in the transaction log before a snapshot can be taken | `100000` |
| `config.commitLogCount` | The number of committed requests in memory | `500`
| `config.snapSizeLimitInKb` | SnapSizeLimitInKb | `4194304` |
| `config.maxCnxns` | The total number of concurrent connections that can be made to a zookeeper server | `0` |
| `config.maxClientCnxns` | The number of concurrent connections that a single client | `60` |
| `config.minSessionTimeout` | The minimum session timeout in milliseconds that the server will allow the client to negotiate | `4000` |
| `config.maxSessionTimeout` | The maximum session timeout in milliseconds that the server will allow the client to negotiate | `40000` |
| `config.autoPurgeSnapRetainCount` | The number of snapshots to be retained | `3`
| `config.autoPurgePurgeInterval` | The time interval in hours for which the purge task has to be triggered | `0`
| `config.quorumListenOnAllIPs` | Whether Zookeeper server will listen for connections from its peers on all available IP addresses | `false` |
| `storageType` | Type of storage that can be used it can take either ephemeral or persistence as value | `persistence` |
| `persistence.reclaimPolicy` | Reclaim policy for persistent volumes | `Delete` |
Expand Down
53 changes: 53 additions & 0 deletions deploy/crds/zookeeper.pravega.io_zookeeperclusters_crd.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -68,11 +68,55 @@ spec:
is provided required default values will be provided, and optional
values will be excluded.
properties:
autoPurgePurgeInterval:
description: "The time interval in hours for which the purge task
has to be triggered \n Disabled by default"
type: integer
autoPurgeSnapRetainCount:
description: "Retain the snapshots according to retain count \n
The default value is 3"
type: integer
commitLogCount:
description: "Zookeeper maintains an in-memory list of last committed
requests for fast synchronization with followers \n The default
value is 500"
type: integer
globalOutstandingLimit:
description: "Clients can submit requests faster than ZooKeeper
can process them, especially if there are a lot of clients. Zookeeper
will throttle Clients so that requests won't exceed global outstanding
limit. \n The default value is 1000"
type: integer
initLimit:
description: "InitLimit is the amount of time, in ticks, to allow
followers to connect and sync to a leader. \n Default value is
10."
type: integer
maxClientCnxns:
description: "Limits the number of concurrent connections that a
single client, identified by IP address, may make to a single
member of the ZooKeeper ensemble. \n The default value is 60"
type: integer
maxCnxns:
description: "Limits the total number of concurrent connections
that can be made to a zookeeper server \n The defult value is
0, indicating no limit"
type: integer
maxSessionTimeout:
description: "The maximum session timeout in milliseconds that the
server will allow the client to negotiate. \n The default value
is 40000"
type: integer
minSessionTimeout:
description: "The minimum session timeout in milliseconds that the
server will allow the client to negotiate \n The default value
is 4000"
type: integer
preAllocSize:
description: "To avoid seeks ZooKeeper allocates space in the transaction
log file in blocks of preAllocSize kilobytes \n The default value
is 64M"
type: integer
quorumListenOnAllIPs:
description: "QuorumListenOnAllIPs when set to true the ZooKeeper
server will listen for connections from its peers on all available
Expand All @@ -81,6 +125,15 @@ spec:
the ZAB protocol and the Fast Leader Election protocol. \n The
default value is false."
type: boolean
snapCount:
description: "ZooKeeper records its transactions using snapshots
and a transaction log The number of transactions recorded in the
transaction log before a snapshot can be taken is determined by
snapCount \n The default value is 100,000"
type: integer
snapSizeLimitInKb:
description: "Snapshot size limit in Kb \n The defult value is 4GB"
type: integer
syncLimit:
description: "SyncLimit is the amount of time, in ticks, to allow
followers to sync with Zookeeper. \n The default value is 2."
Expand Down
1 change: 1 addition & 0 deletions go.mod
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ require (
k8s.io/api v0.17.5
k8s.io/apimachinery v0.17.5
k8s.io/client-go v12.0.0+incompatible
k8s.io/code-generator v0.19.3 // indirect
sigs.k8s.io/controller-runtime v0.5.2
)

Expand Down
2 changes: 2 additions & 0 deletions go.sum
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1137,10 +1137,12 @@ k8s.io/autoscaler v0.0.0-20190607113959-1b4f1855cb8e/go.mod h1:QEXezc9uKPT91dwqh
k8s.io/cli-runtime v0.17.5/go.mod h1:MgU0RZdbJoDThMLacP4ik4W7qpI0wOf2uiMyzVvB/BE=
k8s.io/client-go v0.17.5 h1:Sm/9AQ415xPAX42JLKbJZnreXFgD2rVfDUDwOTm0gzA=
k8s.io/client-go v0.17.5/go.mod h1:S8uZpBpjJJdEH/fEyxcqg7Rn0P5jH+ilkgBHjriSmNo=
k8s.io/code-generator v0.17.6-beta.0 h1:1TWQ5C1MpYdLLs/xQwqRYuPAe26IitM6n3bqsj0QjFc=
k8s.io/code-generator v0.17.6-beta.0/go.mod h1:qdiSCSTKtS+3WtPelj2h57fylSQcPUlhMVm+TD9Dvqc=
k8s.io/component-base v0.17.5/go.mod h1:cZQAW1AUbBjD1lh+e/krbiIpqGz6fipI+vHslOBbuHE=
k8s.io/gengo v0.0.0-20190128074634-0689ccc1d7d6/go.mod h1:ezvh/TsK7cY6rbqRK0oQQ8IAqLxYwwyPxAX1Pzy0ii0=
k8s.io/gengo v0.0.0-20190822140433-26a664648505/go.mod h1:ezvh/TsK7cY6rbqRK0oQQ8IAqLxYwwyPxAX1Pzy0ii0=
k8s.io/gengo v0.0.0-20191010091904-7fa3014cb28f h1:eW/6wVuHNZgQJmFesyAxu0cvj0WAHHUuGaLbPcmNY3Q=
k8s.io/gengo v0.0.0-20191010091904-7fa3014cb28f/go.mod h1:ezvh/TsK7cY6rbqRK0oQQ8IAqLxYwwyPxAX1Pzy0ii0=
k8s.io/helm v2.16.3+incompatible/go.mod h1:LZzlS4LQBHfciFOurYBFkCMTaZ0D1l+p0teMg7TSULI=
k8s.io/klog v0.0.0-20181102134211-b9b56d5dfc92/go.mod h1:Gq+BEi5rUBO/HRz0bTSXDUcqjScdoY3a9IHpCEIOOfk=
Expand Down
104 changes: 104 additions & 0 deletions pkg/apis/zookeeper/v1beta1/zookeepercluster_types.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -68,9 +68,11 @@ type ZookeeperClusterSpec struct {
//It can take either Ephemeral or persistence
//Default StorageType is Persistence storage
StorageType string `json:"storageType,omitempty"`

// Persistence is the configuration for zookeeper persistent layer.
// PersistentVolumeClaimSpec and VolumeReclaimPolicy can be specified in here.
Persistence *Persistence `json:"persistence,omitempty"`

// Ephemeral is the configuration which helps create ephemeral storage
// At anypoint only one of Persistence or Ephemeral should be present in the manifest
Ephemeral *Ephemeral `json:"ephemeral,omitempty"`
Expand Down Expand Up @@ -412,6 +414,71 @@ type ZookeeperConfig struct {
// The default value is 2.
SyncLimit int `json:"syncLimit,omitempty"`

// Clients can submit requests faster than ZooKeeper can process them, especially
// if there are a lot of clients. Zookeeper will throttle Clients so that requests
// won't exceed global outstanding limit.
//
// The default value is 1000
GlobalOutstandingLimit int `json:"globalOutstandingLimit,omitempty"`

// To avoid seeks ZooKeeper allocates space in the transaction log file in
// blocks of preAllocSize kilobytes
//
// The default value is 64M
PreAllocSize int `json:"preAllocSize,omitempty"`

// ZooKeeper records its transactions using snapshots and a transaction log
// The number of transactions recorded in the transaction log before a snapshot
// can be taken is determined by snapCount
//
// The default value is 100,000
SnapCount int `json:"snapCount,omitempty"`

// Zookeeper maintains an in-memory list of last committed requests for fast
// synchronization with followers
//
// The default value is 500
CommitLogCount int `json:"commitLogCount,omitempty"`

// Snapshot size limit in Kb
//
// The defult value is 4GB
SnapSizeLimitInKb int `json:"snapSizeLimitInKb,omitempty"`

// Limits the total number of concurrent connections that can be made to a
//zookeeper server
//
// The defult value is 0, indicating no limit
MaxCnxns int `json:"maxCnxns,omitempty"`

// Limits the number of concurrent connections that a single client, identified
// by IP address, may make to a single member of the ZooKeeper ensemble.
//
// The default value is 60
MaxClientCnxns int `json:"maxClientCnxns,omitempty"`

// The minimum session timeout in milliseconds that the server will allow the
// client to negotiate
//
// The default value is 4000
MinSessionTimeout int `json:"minSessionTimeout,omitempty"`

// The maximum session timeout in milliseconds that the server will allow the
// client to negotiate.
//
// The default value is 40000
MaxSessionTimeout int `json:"maxSessionTimeout,omitempty"`

// Retain the snapshots according to retain count
//
// The default value is 3
AutoPurgeSnapRetainCount int `json:"autoPurgeSnapRetainCount,omitempty"`

// The time interval in hours for which the purge task has to be triggered
//
// Disabled by default
AutoPurgePurgeInterval int `json:"autoPurgePurgeInterval,omitempty"`

// QuorumListenOnAllIPs when set to true the ZooKeeper server will listen for
// connections from its peers on all available IP addresses, and not only the
// address configured in the server list of the configuration file. It affects
Expand All @@ -434,6 +501,43 @@ func (c *ZookeeperConfig) withDefaults() (changed bool) {
changed = true
c.SyncLimit = 2
}
if c.GlobalOutstandingLimit == 0 {
changed = true
c.GlobalOutstandingLimit = 1000
}
if c.PreAllocSize == 0 {
changed = true
c.PreAllocSize = 65536
}
if c.SnapCount == 0 {
changed = true
c.SnapCount = 10000
}
if c.CommitLogCount == 0 {
changed = true
c.CommitLogCount = 500
}
if c.SnapSizeLimitInKb == 0 {
changed = true
c.SnapSizeLimitInKb = 4194304
}
if c.MaxClientCnxns == 0 {
changed = true
c.MaxClientCnxns = 60
}
if c.MinSessionTimeout == 0 {
changed = true
c.MinSessionTimeout = 2 * c.TickTime
}
if c.MaxSessionTimeout == 0 {
changed = true
c.MaxSessionTimeout = 20 * c.TickTime
}
if c.AutoPurgeSnapRetainCount == 0 {
changed = true
c.AutoPurgeSnapRetainCount = 3
}

return changed
}

Expand Down
11 changes: 11 additions & 0 deletions pkg/zk/generators.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -233,6 +233,17 @@ func makeZkConfigString(s v1beta1.ZookeeperClusterSpec) string {
"initLimit=" + strconv.Itoa(s.Conf.InitLimit) + "\n" +
"syncLimit=" + strconv.Itoa(s.Conf.SyncLimit) + "\n" +
"tickTime=" + strconv.Itoa(s.Conf.TickTime) + "\n" +
"globalOutstandingLimit=" + strconv.Itoa(s.Conf.GlobalOutstandingLimit) + "\n" +
"preAllocSize=" + strconv.Itoa(s.Conf.PreAllocSize) + "\n" +
"snapCount=" + strconv.Itoa(s.Conf.SnapCount) + "\n" +
"commitLogCount=" + strconv.Itoa(s.Conf.CommitLogCount) + "\n" +
"snapSizeLimitInKb=" + strconv.Itoa(s.Conf.SnapSizeLimitInKb) + "\n" +
"maxCnxns=" + strconv.Itoa(s.Conf.MaxCnxns) + "\n" +
"maxClientCnxns=" + strconv.Itoa(s.Conf.MaxClientCnxns) + "\n" +
"minSessionTimeout=" + strconv.Itoa(s.Conf.MinSessionTimeout) + "\n" +
"maxSessionTimeout=" + strconv.Itoa(s.Conf.MaxSessionTimeout) + "\n" +
"autopurge.snapRetainCount=" + strconv.Itoa(s.Conf.AutoPurgeSnapRetainCount) + "\n" +
"autopurge.purgeInterval=" + strconv.Itoa(s.Conf.AutoPurgePurgeInterval) + "\n" +
"quorumListenOnAllIPs=" + strconv.FormatBool(s.Conf.QuorumListenOnAllIPs) + "\n" +
"dynamicConfigFile=/data/zoo.cfg.dynamic\n"
}
Expand Down

0 comments on commit cc11ea6

Please sign in to comment.