Skip to content

Commit

Permalink
Merge pull request #61 from andyzhangx/storage-class-support
Browse files Browse the repository at this point in the history
feat: Storage class support
  • Loading branch information
andyzhangx authored Jul 11, 2020
2 parents a04e886 + b28ed77 commit 733634b
Show file tree
Hide file tree
Showing 18 changed files with 284 additions and 27 deletions.
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ smb-windows:

.PHONY: container
container: smb
docker build --no-cache -t $(IMAGE_TAG) -f ./pkg/smbplugin/Dockerfile .
docker build --no-cache -t $(IMAGE_TAG) -f ./pkg/smbplugin/dev.Dockerfile .

.PHONY: smb-container
smb-container:
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ Please refer to [`smb.csi.k8s.io` driver parameters](./docs/driver-parameters.md
Please refer to [install SMB CSI driver](./docs/install-csi-driver-master.md)

### Examples
- [Set up a SMB server deployment on a Kubernetes cluster](./deploy/example/smb-provisioner/)
- [Set up a Samba Server deployment on a Kubernetes cluster](./deploy/example/smb-provisioner/)
- [Basic usage](./deploy/example/e2e_usage.md)
- [Windows](./deploy/example/windows)

Expand Down
100 changes: 100 additions & 0 deletions deploy/csi-smb-controller.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
---
kind: Deployment
apiVersion: apps/v1
metadata:
name: csi-smb-controller
namespace: kube-system
spec:
replicas: 2
selector:
matchLabels:
app: csi-smb-controller
template:
metadata:
labels:
app: csi-smb-controller
spec:
serviceAccountName: csi-smb-controller-sa
nodeSelector:
kubernetes.io/os: linux
priorityClassName: system-cluster-critical
tolerations:
- key: "node-role.kubernetes.io/master"
operator: "Equal"
value: "true"
effect: "NoSchedule"
containers:
- name: csi-provisioner
image: mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v1.4.0
args:
- "-v=5"
- "--csi-address=$(ADDRESS)"
- "--enable-leader-election"
- "--leader-election-type=leases"
env:
- name: ADDRESS
value: /csi/csi.sock
volumeMounts:
- mountPath: /csi
name: socket-dir
resources:
limits:
cpu: 1
memory: 1Gi
requests:
cpu: 10m
memory: 20Mi
- name: liveness-probe
image: mcr.microsoft.com/oss/kubernetes-csi/livenessprobe:v1.1.0
args:
- --csi-address=/csi/csi.sock
- --connection-timeout=3s
- --health-port=29632
- --v=5
volumeMounts:
- name: socket-dir
mountPath: /csi
resources:
limits:
cpu: 1
memory: 1Gi
requests:
cpu: 10m
memory: 20Mi
- name: smb
image: mcr.microsoft.com/k8s/csi/smb-csi:latest
imagePullPolicy: IfNotPresent
args:
- "--v=5"
- "--endpoint=$(CSI_ENDPOINT)"
ports:
- containerPort: 29632
name: healthz
protocol: TCP
- containerPort: 29634
name: metrics
protocol: TCP
livenessProbe:
failureThreshold: 5
httpGet:
path: /healthz
port: healthz
initialDelaySeconds: 30
timeoutSeconds: 10
periodSeconds: 30
env:
- name: CSI_ENDPOINT
value: unix:///csi/csi.sock
volumeMounts:
- mountPath: /csi
name: socket-dir
resources:
limits:
cpu: 1
memory: 1Gi
requests:
cpu: 10m
memory: 20Mi
volumes:
- name: socket-dir
emptyDir: {}
10 changes: 5 additions & 5 deletions deploy/example/e2e_usage.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
## CSI driver E2E usage example
### Prerequisite
- [Set up a SMB server on a Kubernetes cluster](./smb-provisioner/)
- [Set up a Samba Server on a Kubernetes cluster](./smb-provisioner/)

#### 1. Create PV/PVC bound with SMB share
- Use `kubectl create secret` to create `smbcreds` with SMB username, password
Expand All @@ -9,7 +9,7 @@ kubectl create secret generic smbcreds --from-literal username=USERNAME --from-l
```
> add `--from-literal domain=DOMAIN-NAME` for domain support
- Create a smb CSI PV, download [`pv-smb-csi.yaml`](https://raw.githubusercontent.com/kubernetes-csi/csi-driver-smb/master/deploy/example/pv-smb-csi.yaml) file and edit `source` in `volumeAttributes`
- Create a smb CSI PV, download [`pv-smb.yaml`](https://raw.githubusercontent.com/kubernetes-csi/csi-driver-smb/master/deploy/example/pv-smb.yaml) file and edit `source` in `volumeAttributes`
```yaml
apiVersion: v1
kind: PersistentVolume
Expand Down Expand Up @@ -38,12 +38,12 @@ spec:
> For [Azure File](https://docs.microsoft.com/en-us/azure/storage/files/), format of `source`: `//accountname.file.core.windows.net/sharename`

```console
kubectl create -f pv-smb-csi.yaml
kubectl create -f pv-smb.yaml
```

- Create a PVC
```console
kubectl create -f https://raw.githubusercontent.com/kubernetes-csi/csi-driver-smb/master/deploy/example/pvc-smb-csi-static.yaml
kubectl create -f https://raw.githubusercontent.com/kubernetes-csi/csi-driver-smb/master/deploy/example/pvc-smb-static.yaml
```
- make sure pvc is created and in `Bound` status finally
```console
Expand All @@ -70,4 +70,4 @@ In the above example, there is a `/mnt/smb` directory mounted as cifs filesystem
### 2.2 Create a deployment on Windows
```
kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/csi-driver-smb/master/deploy/example/windows/deployment.yaml
```
```
File renamed without changes.
12 changes: 12 additions & 0 deletions deploy/example/pvc-smb-csi.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: pvc-smb
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 10Gi
storageClassName: smb
File renamed without changes.
14 changes: 7 additions & 7 deletions deploy/example/smb-provisioner/README.md
Original file line number Diff line number Diff line change
@@ -1,26 +1,26 @@
## Set up a SMB server and a deployment to access SMB server on a Kubernetes cluster
## Set up a Samba Server and a deployment to access Samba Server on a Kubernetes cluster
This page will show you how to:
- set up a SMB server deployment on a Kubernetes cluster, the file share data is stored on local disk.
- set up a deployment to access SMB server on a Kubernetes cluster
- set up a Samba Server deployment on a Kubernetes cluster, the file share data is stored on local disk.
- set up a deployment to access Samba Server on a Kubernetes cluster

### Stey-by-step
- Use `kubectl create secret` to create `smbcreds` with SMB username, password
```console
kubectl create secret generic smbcreds --from-literal username=USERNAME --from-literal password="PASSWORD"
```

- Create a SMB server deployment
- Create a Samba Server deployment
> modify `/smbshare-volume` in deployment to specify different path for smb share data store
```console
kubectl create -f https://raw.githubusercontent.com/kubernetes-csi/csi-driver-smb/master/deploy/example/smb-provisioner/smb-server.yaml
```

After deployment, a new service `smb-server` is created, file share path is `//smb-server.default.svc.cluster.local/share`

- Create a deployment to access above SMB server
- Create a deployment to access above Samba Server
```console
kubectl create -f https://raw.githubusercontent.com/kubernetes-csi/csi-driver-smb/master/deploy/example/smb-provisioner/pv-smb-csi.yaml
kubectl create -f https://raw.githubusercontent.com/kubernetes-csi/csi-driver-smb/master/deploy/example/pvc-smb-csi-static.yaml
kubectl create -f https://raw.githubusercontent.com/kubernetes-csi/csi-driver-smb/master/deploy/example/smb-provisioner/pv-smb.yaml
kubectl create -f https://raw.githubusercontent.com/kubernetes-csi/csi-driver-smb/master/deploy/example/pvc-smb-static.yaml
kubectl create -f https://raw.githubusercontent.com/kubernetes-csi/csi-driver-smb/master/deploy/example/deployment.yaml
```

Expand Down
40 changes: 40 additions & 0 deletions deploy/example/statefulset.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: statefulset-smb
labels:
app: nginx
spec:
serviceName: statefulset-smb
replicas: 1
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: statefulset-smb
image: nginx
command:
- "/bin/sh"
- "-c"
- while true; do echo $(date) >> /mnt/smb/outfile; sleep 1; done
volumeMounts:
- name: persistent-storage
mountPath: /mnt/smb
updateStrategy:
type: RollingUpdate
selector:
matchLabels:
app: nginx
volumeClaimTemplates:
- metadata:
name: persistent-storage
annotations:
volume.beta.kubernetes.io/storage-class: smb
spec:
accessModes: ["ReadWriteOnce"]
resources:
requests:
storage: 10Gi
17 changes: 17 additions & 0 deletions deploy/example/storageclass-smb.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: smb
provisioner: smb.csi.k8s.io
parameters:
source: "//smb-server.default.svc.cluster.local/share"
csi.storage.k8s.io/node-stage-secret-name: "smbcreds"
csi.storage.k8s.io/node-stage-secret-namespace: "default"
reclaimPolicy: Retain # only retain is supported
volumeBindingMode: Immediate
mountOptions:
- dir_mode=0777
- file_mode=0777
- uid=1001
- gid=1001
2 changes: 2 additions & 0 deletions deploy/install-driver.sh
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,9 @@ if [ $ver != "master" ]; then
fi

echo "Installing SMB CSI driver, version: $ver ..."
kubectl apply -f $repo/rbac-csi-smb-controller.yaml
kubectl apply -f $repo/csi-smb-driver.yaml
kubectl apply -f $repo/csi-smb-controller.yaml
kubectl apply -f $repo/csi-smb-node.yaml
kubectl apply -f $repo/csi-smb-node-windows.yaml
echo 'SMB CSI driver installed successfully.'
49 changes: 49 additions & 0 deletions deploy/rbac-csi-smb-controller.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: csi-smb-controller-sa
namespace: kube-system

---

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: smb-external-provisioner-role
rules:
- apiGroups: [""]
resources: ["persistentvolumes"]
verbs: ["get", "list", "watch", "create", "delete"]
- apiGroups: [""]
resources: ["persistentvolumeclaims"]
verbs: ["get", "list", "watch", "update"]
- apiGroups: ["storage.k8s.io"]
resources: ["storageclasses"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["events"]
verbs: ["get", "list", "watch", "create", "update", "patch"]
- apiGroups: ["storage.k8s.io"]
resources: ["csinodes"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["nodes"]
verbs: ["get", "list", "watch"]
- apiGroups: ["coordination.k8s.io"]
resources: ["leases"]
verbs: ["get", "list", "watch", "create", "update", "patch"]
---

kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: smb-csi-provisioner-binding
subjects:
- kind: ServiceAccount
name: csi-smb-controller-sa
namespace: kube-system
roleRef:
kind: ClusterRole
name: smb-external-provisioner-role
apiGroup: rbac.authorization.k8s.io
6 changes: 4 additions & 2 deletions deploy/uninstall-driver.sh
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ if [[ "$#" -gt 0 ]]; then
ver="$1"
fi

repo="https://raw.githubusercontent.com/kubernetes-csi/csi-driver-smb/master/deploy"
repo="https://raw.githubusercontent.com/kubernetes-csi/csi-driver-smb/$ver/deploy"
if [[ "$#" -gt 1 ]]; then
if [[ "$2" == *"local"* ]]; then
echo "use local deploy"
Expand All @@ -34,7 +34,9 @@ if [ $ver != "master" ]; then
fi

echo "Uninstalling SMB CSI driver, version: $ver ..."
kubectl delete -f $repo/csi-smb-driver.yaml --ignore-not-found
kubectl delete -f $repo/csi-smb-controller.yaml --ignore-not-found
kubectl delete -f $repo/csi-smb-node.yaml --ignore-not-found
kubectl delete -f $repo/csi-smb-node-windows.yaml --ignore-not-found
kubectl delete -f $repo/csi-smb-driver.yaml --ignore-not-found
kubectl delete -f $repo/rbac-csi-smb-controller.yaml --ignore-not-found
echo 'Uninstalled SMB CSI driver successfully.'
20 changes: 14 additions & 6 deletions docs/driver-parameters.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,19 @@
## `smb.csi.k8s.io` driver parameters
This driver only supports static provisioning
> This driver only supports static provisioning, bring your own Samba server before using this driver.
### Storage Class Usage
> get a quick example [here](../deploy/example/storageclass-smb.yaml)
- Static Provision(use existing smb)
> get a quick example [here](../deploy/example/pv-smb-csi.yaml)
Name | Meaning | Available Value | Mandatory | Default value
--- | --- | --- | --- | ---
source | Samba Server address | `//smb-server-address/sharename`(for [Azure File](https://docs.microsoft.com/en-us/azure/storage/files/storage-files-introduction), format is `//accountname.file.core.windows.net/filesharename`) | Yes |
csi.storage.k8s.io/node-stage-secret-name | secret name that stores `username`, `password`(`domain` is optional) | existing secret name | Yes |
csi.storage.k8s.io/node-stage-secret-namespace | namespace where the secret is | k8s namespace | Yes |

### PV/PVC Usage
> get a quick example [here](../deploy/example/pv-smb.yaml)
Name | Meaning | Available Value | Mandatory | Default value
--- | --- | --- | --- | ---
volumeAttributes.source | SMB server address | `//IP/smb-server/directory`(for azure file, format is `//accountname.file.core.windows.net/filesharename`) | Yes |
nodeStageSecretRef.name | secret name that stores `username`, `password` | existing secret name | Yes |
nodeStageSecretRef.namespace | namespace where the secret is | k8s namespace | No | `default`
volumeAttributes.source | Samba Server address | `//smb-server-address/sharename`(for [Azure File](https://docs.microsoft.com/en-us/azure/storage/files/storage-files-introduction), format is `//accountname.file.core.windows.net/filesharename`) | Yes |
nodeStageSecretRef.name | secret name that stores `username`, `password`(`domain` is optional) | existing secret name | Yes |
nodeStageSecretRef.namespace | namespace where the secret is | k8s namespace | Yes |
9 changes: 6 additions & 3 deletions pkg/smb/controllerserver.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,22 +25,25 @@ import (
"k8s.io/klog/v2"
)

// CreateVolume not implemented, only for sanity test pass
// CreateVolume only supports static provisioning, no create volume action
func (d *Driver) CreateVolume(ctx context.Context, req *csi.CreateVolumeRequest) (*csi.CreateVolumeResponse, error) {
klog.V(2).Infof("CreateVolume called with request %+v", *req)
volumeCapabilities := req.GetVolumeCapabilities()
if len(volumeCapabilities) == 0 {
return nil, status.Error(codes.InvalidArgument, "CreateVolume Volume capabilities must be provided")
}
return &csi.CreateVolumeResponse{
Volume: &csi.Volume{
VolumeId: "volumeID",
VolumeId: req.GetName(),
CapacityBytes: req.GetCapacityRange().GetRequiredBytes(),
VolumeContext: req.GetParameters(),
},
}, nil
}

// DeleteVolume not implemented, only for sanity test pass
// DeleteVolume only supports static provisioning, no delete volume action
func (d *Driver) DeleteVolume(ctx context.Context, req *csi.DeleteVolumeRequest) (*csi.DeleteVolumeResponse, error) {
klog.V(2).Infof("DeleteVolume called with request %v", *req)
if len(req.GetVolumeId()) == 0 {
return nil, status.Error(codes.InvalidArgument, "Volume ID missing in request")
}
Expand Down
Loading

0 comments on commit 733634b

Please sign in to comment.