Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue 220: Added support for more zookeeper configuration options #262

Merged
merged 5 commits into from
Oct 22, 2020
Merged

Issue 220: Added support for more zookeeper configuration options #262

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
b8b23c0
Added support for more zookeeper configuration options
anishakj Oct 20, 2020
cc098c0
Fixed the comments
anishakj Oct 20, 2020
1cc8421
Merge branch 'master' of https://github.com/pravega/zookeeper-operato…
anishakj Oct 20, 2020
ffd6ab8
Added misising fields in crd
anishakj Oct 20, 2020
2cc8592
Set default value for session timeout
anishakj Oct 20, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 53 additions & 0 deletions charts/zookeeper-operator/templates/zookeeper.pravega.io_zookeeperclusters_crd.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,11 +69,55 @@ spec:
is provided required default values will be provided, and optional
values will be excluded.
properties:
autoPurgePurgeInterval:
description: "The time interval in hours for which the purge task
has to be triggered \n Disabled by default"
type: integer
autoPurgeSnapRetainCount:
description: "Retain the snapshots according to retain count \n
The default value is 3"
type: integer
commitLogCount:
description: "Zookeeper maintains an in-memory list of last committed
requests for fast synchronization with followers \n The default
value is 500"
type: integer
globalOutstandingLimit:
description: "Clients can submit requests faster than ZooKeeper
can process them, especially if there are a lot of clients. Zookeeper
will throttle Clients so that requests won't exceed global outstanding
limit. \n The default value is 1000"
type: integer
initLimit:
description: "InitLimit is the amount of time, in ticks, to allow
followers to connect and sync to a leader. \n Default value is
10."
type: integer
maxClientCnxns:
description: "Limits the number of concurrent connections that a
single client, identified by IP address, may make to a single
member of the ZooKeeper ensemble. \n The default value is 60"
type: integer
maxCnxns:
description: "Limits the total number of concurrent connections
that can be made to a zookeeper server \n The defult value is
0, indicating no limit"
type: integer
maxSessionTimeout:
description: "The maximum session timeout in milliseconds that the
server will allow the client to negotiate. \n The default value
is 40000"
type: integer
minSessionTimeout:
description: "The minimum session timeout in milliseconds that the
server will allow the client to negotiate \n The default value
is 4000"
type: integer
preAllocSize:
description: "To avoid seeks ZooKeeper allocates space in the transaction
log file in blocks of preAllocSize kilobytes \n The default value
is 64M"
type: integer
quorumListenOnAllIPs:
description: "QuorumListenOnAllIPs when set to true the ZooKeeper
server will listen for connections from its peers on all available
Expand All @@ -82,6 +126,15 @@ spec:
the ZAB protocol and the Fast Leader Election protocol. \n The
default value is false."
type: boolean
snapCount:
description: "ZooKeeper records its transactions using snapshots
and a transaction log The number of transactions recorded in the
transaction log before a snapshot can be taken is determined by
snapCount \n The default value is 100,000"
type: integer
snapSizeLimitInKb:
description: "Snapshot size limit in Kb \n The defult value is 4GB"
type: integer
syncLimit:
description: "SyncLimit is the amount of time, in ticks, to allow
followers to sync with Zookeeper. \n The default value is 2."
Expand Down
11 changes: 11 additions & 0 deletions charts/zookeeper/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,17 @@ The following table lists the configurable parameters of the zookeeper chart and
| `config.initLimit` | Amount of time (in ticks) to allow followers to connect and sync to a leader | `10` |
| `config.tickTime` | Length of a single tick which is the basic time unit used by Zookeeper (measured in milliseconds) | `2000` |
| `config.syncLimit` | Amount of time (in ticks) to allow followers to sync with Zookeeper | `2` |
| `config.globalOutstandingLimit` | Max limit for outstanding requests | `1000` |
| `config.preAllocSize` | PreAllocSize in kilobytes | `65536` |
| `config.snapCount` | The number of transactions recorded in the transaction log before a snapshot can be taken | `100000` |
| `config.commitLogCount` | The number of committed requests in memory | `500`
| `config.snapSizeLimitInKb` | SnapSizeLimitInKb | `4194304` |
| `config.maxCnxns` | The total number of concurrent connections that can be made to a zookeeper server | `0` |
| `config.maxClientCnxns` | The number of concurrent connections that a single client | `60` |
| `config.minSessionTimeout` | The minimum session timeout in milliseconds that the server will allow the client to negotiate | `4000` |
| `config.maxSessionTimeout` | The maximum session timeout in milliseconds that the server will allow the client to negotiate | `40000` |
| `config.autoPurgeSnapRetainCount` | The number of snapshots to be retained | `3`
| `config.autoPurgePurgeInterval` | The time interval in hours for which the purge task has to be triggered | `0`
| `config.quorumListenOnAllIPs` | Whether Zookeeper server will listen for connections from its peers on all available IP addresses | `false` |
| `storageType` | Type of storage that can be used it can take either ephemeral or persistence as value | `persistence` |
| `persistence.reclaimPolicy` | Reclaim policy for persistent volumes | `Delete` |
Expand Down
53 changes: 53 additions & 0 deletions deploy/crds/zookeeper.pravega.io_zookeeperclusters_crd.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,11 +68,55 @@ spec:
is provided required default values will be provided, and optional
values will be excluded.
properties:
autoPurgePurgeInterval:
description: "The time interval in hours for which the purge task
has to be triggered \n Disabled by default"
type: integer
autoPurgeSnapRetainCount:
description: "Retain the snapshots according to retain count \n
The default value is 3"
type: integer
commitLogCount:
description: "Zookeeper maintains an in-memory list of last committed
requests for fast synchronization with followers \n The default
value is 500"
type: integer
globalOutstandingLimit:
description: "Clients can submit requests faster than ZooKeeper
can process them, especially if there are a lot of clients. Zookeeper
will throttle Clients so that requests won't exceed global outstanding
limit. \n The default value is 1000"
type: integer
initLimit:
description: "InitLimit is the amount of time, in ticks, to allow
followers to connect and sync to a leader. \n Default value is
10."
type: integer
maxClientCnxns:
description: "Limits the number of concurrent connections that a
single client, identified by IP address, may make to a single
member of the ZooKeeper ensemble. \n The default value is 60"
type: integer
maxCnxns:
description: "Limits the total number of concurrent connections
that can be made to a zookeeper server \n The defult value is
0, indicating no limit"
type: integer
maxSessionTimeout:
description: "The maximum session timeout in milliseconds that the
server will allow the client to negotiate. \n The default value
is 40000"
type: integer
minSessionTimeout:
description: "The minimum session timeout in milliseconds that the
server will allow the client to negotiate \n The default value
is 4000"
type: integer
preAllocSize:
description: "To avoid seeks ZooKeeper allocates space in the transaction
log file in blocks of preAllocSize kilobytes \n The default value
is 64M"
type: integer
quorumListenOnAllIPs:
description: "QuorumListenOnAllIPs when set to true the ZooKeeper
server will listen for connections from its peers on all available
Expand All @@ -81,6 +125,15 @@ spec:
the ZAB protocol and the Fast Leader Election protocol. \n The
default value is false."
type: boolean
snapCount:
description: "ZooKeeper records its transactions using snapshots
and a transaction log The number of transactions recorded in the
transaction log before a snapshot can be taken is determined by
snapCount \n The default value is 100,000"
type: integer
snapSizeLimitInKb:
description: "Snapshot size limit in Kb \n The defult value is 4GB"
type: integer
syncLimit:
description: "SyncLimit is the amount of time, in ticks, to allow
followers to sync with Zookeeper. \n The default value is 2."
Expand Down
1 change: 1 addition & 0 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ require (
k8s.io/api v0.17.5
k8s.io/apimachinery v0.17.5
k8s.io/client-go v12.0.0+incompatible
k8s.io/code-generator v0.19.3 // indirect
sigs.k8s.io/controller-runtime v0.5.2
)

Expand Down
2 changes: 2 additions & 0 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1137,10 +1137,12 @@ k8s.io/autoscaler v0.0.0-20190607113959-1b4f1855cb8e/go.mod h1:QEXezc9uKPT91dwqh
k8s.io/cli-runtime v0.17.5/go.mod h1:MgU0RZdbJoDThMLacP4ik4W7qpI0wOf2uiMyzVvB/BE=
k8s.io/client-go v0.17.5 h1:Sm/9AQ415xPAX42JLKbJZnreXFgD2rVfDUDwOTm0gzA=
k8s.io/client-go v0.17.5/go.mod h1:S8uZpBpjJJdEH/fEyxcqg7Rn0P5jH+ilkgBHjriSmNo=
k8s.io/code-generator v0.17.6-beta.0 h1:1TWQ5C1MpYdLLs/xQwqRYuPAe26IitM6n3bqsj0QjFc=
k8s.io/code-generator v0.17.6-beta.0/go.mod h1:qdiSCSTKtS+3WtPelj2h57fylSQcPUlhMVm+TD9Dvqc=
k8s.io/component-base v0.17.5/go.mod h1:cZQAW1AUbBjD1lh+e/krbiIpqGz6fipI+vHslOBbuHE=
k8s.io/gengo v0.0.0-20190128074634-0689ccc1d7d6/go.mod h1:ezvh/TsK7cY6rbqRK0oQQ8IAqLxYwwyPxAX1Pzy0ii0=
k8s.io/gengo v0.0.0-20190822140433-26a664648505/go.mod h1:ezvh/TsK7cY6rbqRK0oQQ8IAqLxYwwyPxAX1Pzy0ii0=
k8s.io/gengo v0.0.0-20191010091904-7fa3014cb28f h1:eW/6wVuHNZgQJmFesyAxu0cvj0WAHHUuGaLbPcmNY3Q=
k8s.io/gengo v0.0.0-20191010091904-7fa3014cb28f/go.mod h1:ezvh/TsK7cY6rbqRK0oQQ8IAqLxYwwyPxAX1Pzy0ii0=
k8s.io/helm v2.16.3+incompatible/go.mod h1:LZzlS4LQBHfciFOurYBFkCMTaZ0D1l+p0teMg7TSULI=
k8s.io/klog v0.0.0-20181102134211-b9b56d5dfc92/go.mod h1:Gq+BEi5rUBO/HRz0bTSXDUcqjScdoY3a9IHpCEIOOfk=
Expand Down
104 changes: 104 additions & 0 deletions pkg/apis/zookeeper/v1beta1/zookeepercluster_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,9 +68,11 @@ type ZookeeperClusterSpec struct {
//It can take either Ephemeral or persistence
//Default StorageType is Persistence storage
StorageType string `json:"storageType,omitempty"`

// Persistence is the configuration for zookeeper persistent layer.
// PersistentVolumeClaimSpec and VolumeReclaimPolicy can be specified in here.
Persistence *Persistence `json:"persistence,omitempty"`

// Ephemeral is the configuration which helps create ephemeral storage
// At anypoint only one of Persistence or Ephemeral should be present in the manifest
Ephemeral *Ephemeral `json:"ephemeral,omitempty"`
Expand Down Expand Up @@ -412,6 +414,71 @@ type ZookeeperConfig struct {
// The default value is 2.
SyncLimit int `json:"syncLimit,omitempty"`

// Clients can submit requests faster than ZooKeeper can process them, especially
// if there are a lot of clients. Zookeeper will throttle Clients so that requests
// won't exceed global outstanding limit.
//
// The default value is 1000
GlobalOutstandingLimit int `json:"globalOutstandingLimit,omitempty"`

// To avoid seeks ZooKeeper allocates space in the transaction log file in
// blocks of preAllocSize kilobytes
//
anishakj marked this conversation as resolved.
Show resolved Hide resolved
// The default value is 64M
PreAllocSize int `json:"preAllocSize,omitempty"`

// ZooKeeper records its transactions using snapshots and a transaction log
// The number of transactions recorded in the transaction log before a snapshot
// can be taken is determined by snapCount
//
// The default value is 100,000
SnapCount int `json:"snapCount,omitempty"`

// Zookeeper maintains an in-memory list of last committed requests for fast
// synchronization with followers
//
// The default value is 500
CommitLogCount int `json:"commitLogCount,omitempty"`

// Snapshot size limit in Kb
//
// The defult value is 4GB
SnapSizeLimitInKb int `json:"snapSizeLimitInKb,omitempty"`

// Limits the total number of concurrent connections that can be made to a
//zookeeper server
//
// The defult value is 0, indicating no limit
MaxCnxns int `json:"maxCnxns,omitempty"`

// Limits the number of concurrent connections that a single client, identified
// by IP address, may make to a single member of the ZooKeeper ensemble.
//
// The default value is 60
MaxClientCnxns int `json:"maxClientCnxns,omitempty"`

// The minimum session timeout in milliseconds that the server will allow the
// client to negotiate
//
// The default value is 4000
MinSessionTimeout int `json:"minSessionTimeout,omitempty"`

// The maximum session timeout in milliseconds that the server will allow the
// client to negotiate.
//
// The default value is 40000
MaxSessionTimeout int `json:"maxSessionTimeout,omitempty"`

// Retain the snapshots according to retain count
//
// The default value is 3
AutoPurgeSnapRetainCount int `json:"autoPurgeSnapRetainCount,omitempty"`

// The time interval in hours for which the purge task has to be triggered
//
// Disabled by default
AutoPurgePurgeInterval int `json:"autoPurgePurgeInterval,omitempty"`

// QuorumListenOnAllIPs when set to true the ZooKeeper server will listen for
// connections from its peers on all available IP addresses, and not only the
// address configured in the server list of the configuration file. It affects
Expand All @@ -434,6 +501,43 @@ func (c *ZookeeperConfig) withDefaults() (changed bool) {
changed = true
c.SyncLimit = 2
}
if c.GlobalOutstandingLimit == 0 {
changed = true
c.GlobalOutstandingLimit = 1000
}
if c.PreAllocSize == 0 {
changed = true
c.PreAllocSize = 65536
}
if c.SnapCount == 0 {
changed = true
c.SnapCount = 10000
}
if c.CommitLogCount == 0 {
changed = true
c.CommitLogCount = 500
}
if c.SnapSizeLimitInKb == 0 {
changed = true
c.SnapSizeLimitInKb = 4194304
}
if c.MaxClientCnxns == 0 {
changed = true
c.MaxClientCnxns = 60
}
if c.MinSessionTimeout == 0 {
changed = true
c.MinSessionTimeout = 2 * c.TickTime
}
if c.MaxSessionTimeout == 0 {
changed = true
c.MaxSessionTimeout = 20 * c.TickTime
}
if c.AutoPurgeSnapRetainCount == 0 {
changed = true
c.AutoPurgeSnapRetainCount = 3
}

return changed
}

Expand Down
11 changes: 11 additions & 0 deletions pkg/zk/generators.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -233,6 +233,17 @@ func makeZkConfigString(s v1beta1.ZookeeperClusterSpec) string {
"initLimit=" + strconv.Itoa(s.Conf.InitLimit) + "\n" +
"syncLimit=" + strconv.Itoa(s.Conf.SyncLimit) + "\n" +
"tickTime=" + strconv.Itoa(s.Conf.TickTime) + "\n" +
"globalOutstandingLimit=" + strconv.Itoa(s.Conf.GlobalOutstandingLimit) + "\n" +
"preAllocSize=" + strconv.Itoa(s.Conf.PreAllocSize) + "\n" +
"snapCount=" + strconv.Itoa(s.Conf.SnapCount) + "\n" +
"commitLogCount=" + strconv.Itoa(s.Conf.CommitLogCount) + "\n" +
"snapSizeLimitInKb=" + strconv.Itoa(s.Conf.SnapSizeLimitInKb) + "\n" +
"maxCnxns=" + strconv.Itoa(s.Conf.MaxCnxns) + "\n" +
"maxClientCnxns=" + strconv.Itoa(s.Conf.MaxClientCnxns) + "\n" +
"minSessionTimeout=" + strconv.Itoa(s.Conf.MinSessionTimeout) + "\n" +
"maxSessionTimeout=" + strconv.Itoa(s.Conf.MaxSessionTimeout) + "\n" +
"autopurge.snapRetainCount=" + strconv.Itoa(s.Conf.AutoPurgeSnapRetainCount) + "\n" +
"autopurge.purgeInterval=" + strconv.Itoa(s.Conf.AutoPurgePurgeInterval) + "\n" +
"quorumListenOnAllIPs=" + strconv.FormatBool(s.Conf.QuorumListenOnAllIPs) + "\n" +
"dynamicConfigFile=/data/zoo.cfg.dynamic\n"
}
Expand Down