Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue 63: Added support for ephemeral storage #215

Merged
merged 26 commits into from
Aug 14, 2020
Merged

Issue 63: Added support for ephemeral storage #215

Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
2f022a7
added support for ephemeral storage
Jul 8, 2020
8e02a92
changed emepheral ete
Jul 8, 2020
ba3e241
fixed test
Jul 8, 2020
17f5dff
fixed format
Jul 8, 2020
e549b2d
doc changes and ut added
Jul 9, 2020
dada168
fixed doc
Jul 9, 2020
5f20b7e
changed charts
Jul 10, 2020
143a52b
addressed review comments
Jul 15, 2020
5b747c7
fixed ete
Jul 15, 2020
cb7c588
Merge branch 'master' of https://github.com/pravega/zookeeper-operato…
Jul 15, 2020
fefa364
changed readme file
Jul 16, 2020
df5ca92
addressed comments
Jul 16, 2020
e663fc7
fixed charts
Jul 16, 2020
023271f
addressed comments
Jul 17, 2020
4967437
changed default storage to persistence
Jul 17, 2020
5560fa5
changed doc
Jul 23, 2020
ebe8cca
changed in zookeeper.yaml
Jul 23, 2020
0b5d7c9
addressed comments
Jul 24, 2020
6c5e449
addressed comments
Jul 24, 2020
b369dd8
pulled changes from master
Jul 24, 2020
11cf150
changed structure for ephemeral storage
Jul 28, 2020
942e972
doc added
Aug 3, 2020
2d44c1e
fixed readme
Aug 3, 2020
6b4259f
addressed comments
Aug 13, 2020
0cad266
Merge branch 'master' of https://github.com/pravega/zookeeper-operato…
Aug 13, 2020
1db7f35
addressed comment
Aug 14, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ The project is currently alpha. While no breaking API changes are currently plan
* [Usage](#usage)
* [Installation of the Operator](#install-the-operator)
* [Deploy a sample Zookeeper Cluster](#deploy-a-sample-zookeeper-cluster)
* [Deploy a sample ZooKeeper Cluster with Ephemeral Storage](#Deploy-a-sample-zookeeper-cluster-with-ephemeral-storage)
pbelgundi marked this conversation as resolved.
Show resolved Hide resolved
* [Deploy a sample Zookeeper Cluster to a cluster using Istio](#deploy-a-sample-zookeeper-cluster-with-istio)
* [Upgrade a Zookeeper Cluster](#upgrade-a-zookeeper-cluster)
* [Uninstall the Zookeeper Cluster](#uninstall-the-zookeeper-cluster)
Expand Down Expand Up @@ -131,6 +132,36 @@ svc/example-client ClusterIP 10.31.243.173 <none> 2181/TCP
svc/example-headless ClusterIP None <none> 2888/TCP,3888/TCP 2m
```

### Deploy a sample Zookeeper cluster with Ephemeral storage

Create a Yaml file called `zk.yaml` with the following content to install a 3-node Zookeeper cluster.

```yaml
apiVersion: "zookeeper.pravega.io/v1beta1"
kind: "ZookeeperCluster"
metadata:
name: "example"
spec:
replicas: 3
storage:
ephemeral:
enabled: true
```

```
$ kubectl create -f zk.yaml
```

After a couple of minutes, all cluster members should become ready.

```
$ kubectl get zk

NAME REPLICAS READY REPLICAS VERSION DESIRED VERSION INTERNAL ENDPOINT EXTERNAL ENDPOINT AGE
example 3 3 0.2.7 0.2.7 10.100.200.18:2181 N/A 94s
```
>Note: User should only provide either persistence or ephemeral in the spec, if none of the values is specified default is persistence
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be "persistent" and not "persistence" storage.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have rephrased this statement.


### Deploy a sample Zookeeper cluster with Istio
Create a Yaml file called `zk-with-istio.yaml` with the following content to install a 3-node Zookeeper cluster.

Expand Down
4 changes: 4 additions & 0 deletions charts/zookeeper/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,10 @@ The following table lists the configurable parameters of the Zookeeper chart and
| `config.tickTime` | Length of a single tick which is the basic time unit used by Zookeeper (measured in milliseconds) | `2000` |
| `config.syncLimit` | Amount of time (in ticks) to allow followers to sync with Zookeeper | `2` |
| `config.quorumListenOnAllIPs` | Whether Zookeeper server will listen for connections from its peers on all available IP addresses | `false` |
| `storageType.type` | Type of storage that can be used it can take either ephemeral or persistence as value | `persistence` |
| `persistence.reclaimPolicy` | Reclaim policy for persistent volumes | `Delete` |
| `persistence.storageClassName` | Storage class for persistent volumes | `standard` |
| `persistence.volumeSize` | Size of the volume requested for persistent volumes | `20Gi` |
| `ephemeral.enabled` | whether ephemeral storage is enabled or not | `false` |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line is no longer needed, correct?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have removed it

| `ephemeral.emptydirvolumesource.medium` | What type of storage medium should back the directory. | `""` |
| `ephemeral.emptydirvolumesource.sizeLimit` | Total amount of local storage required for the EmptyDir volume. | `nil` |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nil is not a valid default value in helm. Please leave it empty or give an empty string.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

26 changes: 19 additions & 7 deletions charts/zookeeper/templates/zookeeper.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,10 +43,22 @@ spec:
syncLimit: {{ .Values.config.syncLimit }}
quorumListenOnAllIPs: {{ .Values.config.quorumListenOnAllIPs }}
{{- end }}
persistence:
reclaimPolicy: {{ .Values.persistence.reclaimPolicy }}
spec:
storageClassName: {{ .Values.persistence.storageClassName }}
resources:
requests:
storage: {{ .Values.persistence.volumeSize }}
storage:
{{- if eq .Values.storageType.type "ephemeral" }}
ephemeral:
enabled: {{ .Values.ephemeral.enabled }}
{{- if .Values.ephemeral.emptydirvolumesource }}
emptydirvolumesource:
medium: {{ .Values.ephemeral.emptydirvolumesource.medium }}
sizeLimit: {{ .Values.ephemeral.emptydirvolumesource.sizeLimit }}
{{- end }}
{{- end }}
{{- if eq .Values.storageType.type "persistence" }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be part of {{- else }} and not as part of a separate if block. Only then would this be treated as the default even when no value is provided for storageType.type

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

persistence:
reclaimPolicy: {{ .Values.persistence.reclaimPolicy }}
spec:
storageClassName: {{ .Values.persistence.storageClassName }}
resources:
requests:
storage: {{ .Values.persistence.volumeSize }}
{{- end }}
21 changes: 16 additions & 5 deletions charts/zookeeper/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,13 +25,24 @@ config: {}
# tickTime: 2000
# syncLimit: 2
# quorumListenOnAllIPs: false
storageType:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rename this strorage and make persistence and ephemeral as part of this block

storage:
   type: persistence
   persistence:
      storageClassName: standard
      reclaimPolicy: Delete
      volumeSize: 20Gi
    ephemeral:
       emptydirvolumesource: {}
          medium: Memory
          sizeLimit: 15Gi

## configure the storage backend type
## accepted values : persistence/ephemeral
## default option is
type: persistence

persistence:
storageClassName: standard
## specifying reclaim policy for PersistentVolumes
## accepted values - Delete / Retain
reclaimPolicy: Delete
volumeSize: 20Gi
storageClassName: standard
# specifying reclaim policy for PersistentVolumes
# accepted values - Delete / Retain
reclaimPolicy: Delete
volumeSize: 20Gi

ephemeral:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add one more level of indentation so that the blocks persistence and ephemeral come within the storage block

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

enabled: true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please remove the enabled field from here. Whether ephemeral will be enabled/not will be determined by the value of storage.type field

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

#emptydirvolumesource: {}
#medium: ""/Memory
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of mentioning Memory as an accepted value for medium as /Memory, please add a comment in the previous line which lists the accepted values, and keep just the default value here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

#sizeLimit: 15Gi

hooks:
image:
Expand Down
4 changes: 4 additions & 0 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,18 @@ module github.com/pravega/zookeeper-operator
go 1.13

require (
4d63.com/gochecknoglobals v0.0.0-20190306162314-7c3491d2b6ec // indirect
4d63.com/gochecknoinits v0.0.0-20200108094044-eb73b47b9fc4 // indirect
github.com/ghodss/yaml v1.0.1-0.20190212211648-25d852aebe32
github.com/go-logr/logr v0.1.0
github.com/gordonklaus/ineffassign v0.0.0-20200309095847-7953dde2c7bf // indirect
github.com/onsi/ginkgo v1.12.0
github.com/onsi/gomega v1.9.0
github.com/opennota/check v0.0.0-20180911053232-0c771f5545ff // indirect
github.com/operator-framework/operator-sdk v0.17.0
github.com/pkg/errors v0.9.1
github.com/samuel/go-zookeeper v0.0.0-20190923202752-2cc03de413da
github.com/securego/gosec v0.0.0-20200401082031-e946c8c39989 // indirect
k8s.io/api v0.17.5
k8s.io/apimachinery v0.17.5
k8s.io/client-go v12.0.0+incompatible
Expand Down
19 changes: 19 additions & 0 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
4d63.com/gochecknoglobals v0.0.0-20190306162314-7c3491d2b6ec h1:LArU+LLj2RRPagtrxuBE+xs+zJzOCWvX99TIoGac1gE=
4d63.com/gochecknoglobals v0.0.0-20190306162314-7c3491d2b6ec/go.mod h1:Sk40JNJmh0koZukOjJfaBNLZazbZthFfHnLHIcZNS6A=
4d63.com/gochecknoinits v0.0.0-20200108094044-eb73b47b9fc4 h1:bf5qocEKjrY58JO2GwywfLsb1199lIVs7qHkiplwHy0=
4d63.com/gochecknoinits v0.0.0-20200108094044-eb73b47b9fc4/go.mod h1:4o1i5aXtIF5tJFt3UD1knCVmWOXg7fLYdHVu6jeNcnM=
bazil.org/fuse v0.0.0-20160811212531-371fbbdaa898/go.mod h1:Xbm+BRKSBEpa4q4hTSxohYNQpsxXPbPry4JJWOB3LB8=
bitbucket.org/bertimus9/systemstat v0.0.0-20180207000608-0eeff89b0690/go.mod h1:Ulb78X89vxKYgdL24HMTiXYHlyHEvruOj1ZPlqeNEZM=
bou.ke/monkey v1.0.1/go.mod h1:FgHuK96Rv2Nlf+0u1OOVDpCMdsWyOFmeeketDHE7LIg=
Expand Down Expand Up @@ -461,6 +465,8 @@ github.com/gophercloud/gophercloud v0.6.0 h1:Xb2lcqZtml1XjgYZxbeayEemq7ASbeTp09m
github.com/gophercloud/gophercloud v0.6.0/go.mod h1:GICNByuaEBibcjmjvI7QvYJSZEbGkcYwAR7EZK2WMqM=
github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY=
github.com/gopherjs/gopherjs v0.0.0-20191106031601-ce3c9ade29de/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY=
github.com/gordonklaus/ineffassign v0.0.0-20200309095847-7953dde2c7bf h1:vc7Dmrk4JwS0ZPS6WZvWlwDflgDTA26jItmbSj83nug=
github.com/gordonklaus/ineffassign v0.0.0-20200309095847-7953dde2c7bf/go.mod h1:cuNKsD1zp2v6XfE/orVX2QE1LC+i254ceGcVeDT3pTU=
github.com/gorilla/context v1.1.1/go.mod h1:kBGZzfjB9CEq2AlWe17Uuf7NDRt0dE0s8S51q0aT7Yg=
github.com/gorilla/handlers v0.0.0-20150720190736-60c7bfde3e33/go.mod h1:Qkdc/uu4tH4g6mTK6auzZ766c4CA0Ng8+o/OAirnOIQ=
github.com/gorilla/mux v1.6.2/go.mod h1:1lud6UwP+6orDFRuTfBEV8e9/aOM/c4fVVCaMa2zaAs=
Expand Down Expand Up @@ -567,6 +573,7 @@ github.com/kardianos/osext v0.0.0-20190222173326-2bc1f35cddc0/go.mod h1:1NbS8ALr
github.com/karrick/godirwalk v1.7.5/go.mod h1:2c9FRhkDxdIbgkOnCEvnSWs71Bhugbl46shStcFDJ34=
github.com/karrick/godirwalk v1.10.12/go.mod h1:RoGL9dQei4vP9ilrpETWE8CLOZ1kiN0LhBygSwrAsHA=
github.com/kisielk/errcheck v1.1.0/go.mod h1:EZBBE59ingxPouuu3KfxchcWSUPOHkagtvWXihfKN4Q=
github.com/kisielk/errcheck v1.2.0 h1:reN85Pxc5larApoH1keMBiu2GWtPqXQ1nc9gx+jOU+E=
github.com/kisielk/errcheck v1.2.0/go.mod h1:/BMXB+zMLi60iA8Vv6Ksmxu/1UDYcXs4uQLJ+jE2L00=
github.com/kisielk/gotool v0.0.0-20161130080628-0de1eaf82fa3/go.mod h1:jxZFDH7ILpTPQTk+E2s+z4CUas9lVNjIuKR4c5/zKgM=
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
Expand All @@ -583,13 +590,15 @@ github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORN
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/pty v1.1.3/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/pty v1.1.5/go.mod h1:9r2w37qlBe7rQ6e1fg1S/9xpWHSnaqNdHD3WcMdbPDA=
github.com/kr/pty v1.1.8/go.mod h1:O1sed60cT9XZ5uDucP5qwvh+TE3NnUj51EiZO/lmSfw=
github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/kshvakov/clickhouse v1.3.5/go.mod h1:DMzX7FxRymoNkVgizH0DWAL8Cur7wHLgx3MUnGwJqpE=
github.com/kylelemons/godebug v0.0.0-20160406211939-eadb3ce320cb/go.mod h1:B69LEHPfb2qLo0BaaOLcbitczOKLWTsrBG9LczfCD4k=
github.com/kylelemons/godebug v0.0.0-20170820004349-d65d576e9348/go.mod h1:B69LEHPfb2qLo0BaaOLcbitczOKLWTsrBG9LczfCD4k=
github.com/leanovate/gopter v0.2.4/go.mod h1:gNcbPWNEWRe4lm+bycKqxUYoH5uoVje5SkOJ3uoLer8=
github.com/lib/pq v1.0.0/go.mod h1:5WUZQaWbwv1U+lTReE5YruASi9Al49XbQIvNi/34Woo=
github.com/lib/pq v1.2.0/go.mod h1:5WUZQaWbwv1U+lTReE5YruASi9Al49XbQIvNi/34Woo=
github.com/libopenstorage/openstorage v1.0.0/go.mod h1:Sp1sIObHjat1BeXhfMqLZ14wnOzEhNx2YQedreMcUyc=
github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de/go.mod h1:zAbeS9B/r2mtpb6U+EI2rYA5OAXxsYw6wTamcNW+zcE=
github.com/lightstep/lightstep-tracer-common/golang/gogo v0.0.0-20190605223551-bc2310a04743/go.mod h1:qklhhLq1aX+mtWk9cPHPzaBjWImj5ULL6C7HFJtXQMM=
Expand Down Expand Up @@ -681,6 +690,7 @@ github.com/mohae/deepcopy v0.0.0-20170603005431-491d3605edfb/go.mod h1:TaXosZuwd
github.com/morikuni/aec v0.0.0-20170113033406-39771216ff4c/go.mod h1:BbKIizmSmc5MMPqRYbxO4ZU0S0+P200+tUnFx7PXmsc=
github.com/morikuni/aec v1.0.0/go.mod h1:BbKIizmSmc5MMPqRYbxO4ZU0S0+P200+tUnFx7PXmsc=
github.com/mozilla/tls-observatory v0.0.0-20180409132520-8791a200eb40/go.mod h1:SrKMQvPiws7F7iqYp8/TX+IhxCYhzr6N/1yb8cwHsGk=
github.com/mozilla/tls-observatory v0.0.0-20200317151703-4fa42e1c2dee/go.mod h1:SrKMQvPiws7F7iqYp8/TX+IhxCYhzr6N/1yb8cwHsGk=
github.com/mozillazg/go-cos v0.13.0/go.mod h1:Zp6DvvXn0RUOXGJ2chmWt2bLEqRAnJnS3DnAZsJsoaE=
github.com/mozillazg/go-httpheader v0.2.1/go.mod h1:jJ8xECTlalr6ValeXYdOF8fFUISeBAdw6E61aqQma60=
github.com/mrunalp/fileutils v0.0.0-20171103030105-7d4729fb3618/go.mod h1:x8F1gnqOkIEiO4rqoeEEEqQbo7HjGMTvyoq3gej4iT0=
Expand All @@ -695,6 +705,8 @@ github.com/naoina/go-stringutil v0.1.0/go.mod h1:XJ2SJL9jCtBh+P9q5btrd/Ylo8XwT/h
github.com/naoina/toml v0.1.1/go.mod h1:NBIhNtsFMo3G2szEBne+bO4gS192HuIYRqfvOWb4i1E=
github.com/nbutton23/zxcvbn-go v0.0.0-20160627004424-a22cb81b2ecd/go.mod h1:o96djdrsSGy3AWPyBgZMAGfxZNfgntdJG+11KU4QvbU=
github.com/nbutton23/zxcvbn-go v0.0.0-20171102151520-eafdab6b0663/go.mod h1:o96djdrsSGy3AWPyBgZMAGfxZNfgntdJG+11KU4QvbU=
github.com/nbutton23/zxcvbn-go v0.0.0-20180912185939-ae427f1e4c1d h1:AREM5mwr4u1ORQBMvzfzBgpsctsbQikCVpvC+tX285E=
github.com/nbutton23/zxcvbn-go v0.0.0-20180912185939-ae427f1e4c1d/go.mod h1:o96djdrsSGy3AWPyBgZMAGfxZNfgntdJG+11KU4QvbU=
github.com/ncw/swift v1.0.47/go.mod h1:23YIA4yWVnGwv2dQlN4bB7egfYX6YLn0Yo/S6zZO/ZM=
github.com/oklog/run v1.0.0/go.mod h1:dlhp/R75TPv97u0XWUtDeV/lRKWPKSdTuV0TZvrmrQA=
github.com/oklog/ulid v0.0.0-20170117200651-66bb6560562f/go.mod h1:CirwcVhetQ6Lv90oh/F+FBtV6XMibvdAFo93nm5qn4U=
Expand Down Expand Up @@ -850,6 +862,8 @@ github.com/satori/go.uuid v1.2.0/go.mod h1:dA0hQrYB0VpLJoorglMZABFdXlWrHn1NEOzdh
github.com/sclevine/spec v1.2.0/go.mod h1:W4J29eT/Kzv7/b9IWLB055Z+qvVC9vt0Arko24q7p+U=
github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529/go.mod h1:DxrIzT+xaE7yg65j358z/aeFdxmN0P9QXhEzd20vsDc=
github.com/seccomp/libseccomp-golang v0.9.1/go.mod h1:GbW5+tmTXfcxTToHLXlScSlAvWlF4P2Ca7zGrPiEpWo=
github.com/securego/gosec v0.0.0-20200401082031-e946c8c39989 h1:rq2/kILQnPtq5oL4+IAjgVOjh5e2yj2aaCYi7squEvI=
github.com/securego/gosec v0.0.0-20200401082031-e946c8c39989/go.mod h1:i9l/TNj+yDFh9SZXUTvspXTjbFXgZGP/UvhU1S65A4A=
github.com/sergi/go-diff v1.0.0 h1:Kpca3qRNrduNnOQeazBd0ysaKrUJiIuISHxogkT9RPQ=
github.com/sergi/go-diff v1.0.0/go.mod h1:0CfEIISq7TuYL3j771MWULgwwjU+GofnZX9QAmXWZgo=
github.com/shirou/gopsutil v0.0.0-20180427012116-c95755e4bcd7/go.mod h1:5b4v6he4MtMOwMlS0TUMTu2PcXUg8+E1lC7eC3UO/RA=
Expand Down Expand Up @@ -948,6 +962,7 @@ github.com/xlab/handysort v0.0.0-20150421192137-fb3537ed64a1/go.mod h1:QcJo0QPSf
github.com/xlab/treeprint v0.0.0-20180616005107-d6fb6747feb6/go.mod h1:ce1O1j6UtZfjr22oyGxGLbauSBp2YVXpARAosm7dHBg=
github.com/xordataexchange/crypt v0.0.3-0.20170626215501-b2862e3d0a77/go.mod h1:aYKd//L2LvnjZzWKhF00oedf4jCCReLcmhLdhm1A27Q=
github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yvasiyarov/go-metrics v0.0.0-20140926110328-57bccd1ccd43/go.mod h1:aX5oPXxHm3bOH+xeAttToC8pqch2ScQN/JoXYupl6xs=
github.com/yvasiyarov/gorelic v0.0.0-20141212073537-a9bba5b9ab50/go.mod h1:NUSPSUX/bi6SeDMUh6brw0nXpxHnc96TguQh0+r/ssA=
github.com/yvasiyarov/newrelic_platform_go v0.0.0-20140908184405-b21fdbd4370f/go.mod h1:GlGEuHIJweS1mbCqG+7vt2nvWLzLLnRHbXz5JKd/Qbg=
Expand Down Expand Up @@ -1029,6 +1044,7 @@ golang.org/x/lint v0.0.0-20190409202823-959b441ac422/go.mod h1:6SW0HCj/g11FgYtHl
golang.org/x/lint v0.0.0-20190909230951-414d861bb4ac/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/lint v0.0.0-20190930215403-16217165b5de h1:5hukYrvBGR8/eNkX5mdUezrA6JiaEZDtJb9Ei+1LlBs=
golang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/lint v0.0.0-20191125180803-fdd1cda4f05f/go.mod h1:5qLYkcX4OjUUV8bRuDixDT3tpyyb+LUpUlRWLxfhWrs=
golang.org/x/mobile v0.0.0-20190312151609-d3739f865fa6/go.mod h1:z+o9i4GpDbdi3rU15maQ/Ox0txvL9dWGYEHz965HBQE=
golang.org/x/mobile v0.0.0-20190719004257-d2bd2a29d028/go.mod h1:E/iHnbuqvinMTCcRqshq8CkpyQDoeVncDDYHnLhea+o=
golang.org/x/mod v0.0.0-20190513183733-4bf6d317e70e/go.mod h1:mXi4GBBbnImb6dmsKGUJ2LatrhH/nqhxcFungHvyanc=
Expand Down Expand Up @@ -1203,9 +1219,12 @@ golang.org/x/tools v0.0.0-20191111182352-50fa39b762bc/go.mod h1:b+2E5dAYhXwXZwtn
golang.org/x/tools v0.0.0-20191113191852-77e3bb0ad9e7/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191115202509-3a792d9c32b2/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191125144606-a911d9008d1f/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20200115044656-831fdb1e1868/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200327195553-82bb89366a1e h1:qCZ8SbsZMjT0OuDPCEBxgLZic4NMj8Gj4vNXiTVRAaA=
golang.org/x/tools v0.0.0-20200327195553-82bb89366a1e/go.mod h1:Sl4aGygMT6LrqrWclx+PTx3U+LnKx/seiNR+3G19Ar8=
golang.org/x/tools v0.0.0-20200331202046-9d5940d49312 h1:2PHG+Ia3gK1K2kjxZnSylizb//eyaMG8gDFbOG7wLV8=
golang.org/x/tools v0.0.0-20200331202046-9d5940d49312/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543 h1:E7g+9GITq07hpfrRu66IVDexMakfv52eLZ2CXBWiKr4=
Expand Down
2 changes: 1 addition & 1 deletion pkg/apis/zookeeper/v1beta1/deepcopy_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ var _ = Describe("ZookeeperCluster DeepCopy", func() {
z2.Spec.Ports[0].ContainerPort = p.ContainerPort
z1.SetAnnotations(m)
z2.Spec.Pod.Annotations = z1.Spec.Pod.Annotations
z2.Spec.Persistence = z1.Spec.Persistence.DeepCopy()
z2.Spec.Storage.Persistence = z1.Spec.Storage.Persistence.DeepCopy()
z1.Spec.Pod.DeepCopyInto(&z2.Spec.Pod)
z1.Status.Members.Ready = []string{"zk-0", "zk-1"}
z1.Status.Members.Unready = []string{"zk-2"}
Expand Down
39 changes: 30 additions & 9 deletions pkg/apis/zookeeper/v1beta1/zookeepercluster_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,13 +59,11 @@ type ZookeeperClusterSpec struct {
Ports []v1.ContainerPort `json:"ports,omitempty"`

// Pod defines the policy to create pod for the zookeeper cluster.
//
// Updating the Pod does not take effect on any existing pods.
Pod PodPolicy `json:"pod,omitempty"`

// Persistence is the configuration for zookeeper persistent layer.
// PersistentVolumeClaimSpec and VolumeReclaimPolicy can be specified in here.
Persistence *Persistence `json:"persistence,omitempty"`
//storage is used to define which storagetype we will be using
Storage *Storage `json:"storage,omitempty"`

// Conf is the zookeeper configuration, which will be used to generate the
// static zookeeper configuration. If no configuration is provided required
Expand Down Expand Up @@ -157,11 +155,12 @@ func (s *ZookeeperClusterSpec) withDefaults(z *ZookeeperCluster) (changed bool)
if s.Pod.withDefaults(z) {
changed = true
}
if s.Persistence == nil {
s.Persistence = &Persistence{}
if s.Storage == nil || (s.Storage.Ephemeral == nil && s.Storage.Persistence == nil) {
s.Storage = &Storage{}
s.Storage.Persistence = &Persistence{}
changed = true
}
if s.Persistence.withDefaults() {
if s.Storage != nil && s.Storage.Persistence != nil && s.Storage.Persistence.withDefaults() {
changed = true
}
return changed
Expand Down Expand Up @@ -383,17 +382,39 @@ func (c *ZookeeperConfig) withDefaults() (changed bool) {
return changed
}

type Storage struct {
//Default storage is Persistence storage
// Persistence is the configuration for zookeeper persistent layer.
// PersistentVolumeClaimSpec and VolumeReclaimPolicy can be specified in here.
Persistence *Persistence `json:"persistence,omitempty"`

// Ephemeral is the configuration which helps create ephemeral storage
// At anypoint only one of Persistence or Ephemeral should be present in the manifest
Ephemeral *Ephemeral `json:"ephemeral,omitempty"`
}

type Persistence struct {
// VolumeReclaimPolicy is a zookeeper operator configuration. If it's set to Delete,
// the corresponding PVCs will be deleted by the operator when zookeeper cluster is deleted.
// The default value is Retain.
VolumeReclaimPolicy VolumeReclaimPolicy `json:"reclaimPolicy,omitempty"`
// PersistentVolumeClaimSpec is the spec to describe PVC for the container
// This field is optional. If no PVC spec, stateful containers will use
// emptyDir as volume.
// This field is optional. If no PVC is specified by default persistentvolume
// will get created.
PersistentVolumeClaimSpec v1.PersistentVolumeClaimSpec `json:"spec,omitempty"`
}

type Ephemeral struct {
// Enabled specifies whether or not ephemeral storage is enabled
// By default, ephemeral is not enabled
Enabled bool `json:"enabled"`
//EmptyDirVolumeSource is optional and this will create the emptydir volume
//It has two parameters Medium and SizeLimit which are optional as well
//Medium specifies What type of storage medium should back this directory.
//SizeLimit specifies Total amount of local storage required for this EmptyDir volume.
EmptyDirVolumeSource v1.EmptyDirVolumeSource `json:"emptydirvolumesource,omitempty"`
}

func (p *Persistence) withDefaults() (changed bool) {
if !p.VolumeReclaimPolicy.isValid() {
changed = true
Expand Down
6 changes: 3 additions & 3 deletions pkg/apis/zookeeper/v1beta1/zookeepercluster_types_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -90,12 +90,12 @@ var _ = Describe("ZookeeperCluster Types", func() {
var z1 v1beta1.ZookeeperCluster
BeforeEach(func() {
z1 = *z.DeepCopy()
z1.Spec.Persistence.VolumeReclaimPolicy = "Delete"
z1.Spec.Storage.Persistence.VolumeReclaimPolicy = "Delete"
z1.WithDefaults()
})

It("should set the Volumeclaimpolicy to Delete", func() {
Ω(fmt.Sprintf("%s", z1.Spec.Persistence.VolumeReclaimPolicy)).To(Equal("Delete"))
Ω(fmt.Sprintf("%s", z1.Spec.Storage.Persistence.VolumeReclaimPolicy)).To(Equal("Delete"))
})
})

Expand Down Expand Up @@ -204,7 +204,7 @@ var _ = Describe("ZookeeperCluster Types", func() {
var p corev1.PersistentVolumeClaimSpec

BeforeEach(func() {
p = z.Spec.Persistence.PersistentVolumeClaimSpec
p = z.Spec.Storage.Persistence.PersistentVolumeClaimSpec
})

It("should be an RWO volume", func() {
Expand Down
Loading