Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feat.]Raw Block Volume Support for LocalPV #1536

Merged
merged 9 commits into from
Dec 6, 2019

Conversation

rahulchheda
Copy link
Contributor

@rahulchheda rahulchheda commented Nov 28, 2019

What this PR does / why we need it:

  • Added Raw Block Volume Support for LocalPV
  • To use this the PVC spec.volumeMode should be set to Block. For eg:
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: my-pvc
spec:
  accessModes:
    - ReadWriteOnce
  volumeMode: Block
  storageClassName: openebs-device
  resources:
    requests:
      storage: 10Gi

  • In return the PV would look like:
apiVersion: v1
kind: PersistentVolume
metadata:
  annotations:
    local.openebs.io/blockdeviceclaim: bdc-pvc-aeb55a2e-11d9-11ea-adb6-42010a800fc1
    pv.kubernetes.io/provisioned-by: openebs.io/local
  creationTimestamp: "2019-11-28T12:22:15Z"
  finalizers:
  - kubernetes.io/pv-protection
  labels:
    openebs.io/cas-type: local-device
  name: pvc-aeb55a2e-11d9-11ea-adb6-42010a800fc1
  resourceVersion: "2622843"
  selfLink: /api/v1/persistentvolumes/pvc-aeb55a2e-11d9-11ea-adb6-42010a800fc1
  uid: b64950c6-11d9-11ea-adb6-42010a800fc1
spec:
  accessModes:
  - ReadWriteOnce
  capacity:
    storage: 10Gi
  claimRef:
    apiVersion: v1
    kind: PersistentVolumeClaim
    name: my-pvc
    namespace: default
    resourceVersion: "2622808"
    uid: aeb55a2e-11d9-11ea-adb6-42010a800fc1
  local:
    fsType: ""
    path: /dev/sdb
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - gke-standard-cluster-4-default-pool-ad827b74-rc2t
  persistentVolumeReclaimPolicy: Delete
  storageClassName: openebs-device
  volumeMode: Block
status:
  phase: Bound

Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #

Special notes for your reviewer:

Checklist:

  • Fixes #
  • Labelled this PR & related issue with documentation tag
  • PR messages has document related information
  • Labelled this PR & related issue with breaking-changes tag
  • PR messages has breaking changes related information
  • Labelled this PR & related issue with requires-upgrade tag
  • PR messages has upgrade related information
  • Commit has unit tests
  • Commit has integration tests

@rahulchheda rahulchheda changed the title Raw volume [Feat.]Raw Block Volume Support for LocalPV Nov 28, 2019
@kmova kmova self-requested a review November 29, 2019 06:50
path := bd.Spec.FileSystem.Mountpoint

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this change is not required - as the blkpath is already returned by the function. Also, block path should be based on dev links - because after restart the path can change.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

path is over-ridden blkPath, below this function is called. blkPath is now based on dev links.

Copy link
Contributor

@akhilerm akhilerm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Provided some comments

@@ -90,7 +93,7 @@ func (p *Provisioner) ProvisionBlockDevice(opts pvController.VolumeOptions, volu
WithAnnotations(volAnnotations).
WithReclaimPolicy(opts.PersistentVolumeReclaimPolicy).
WithAccessModes(pvc.Spec.AccessModes).
WithVolumeMode(fs).
WithVolumeMode(volumeMode).
WithCapacityQty(pvc.Spec.Resources.Requests[v1.ResourceName(v1.ResourceStorage)]).
WithLocalHostPathFormat(path, fsType).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is the fsType required here?
In Raw Block mode, only the path is required in LocalVolumeSource.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In case of block mode will FSType be ignored? Does path contain block path?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, path is over-ridden by the blkPath.

Copy link

@ghost ghost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found some fixes!

P.S. share your ideas, feedbacks or issues with us at https://github.com/fixmie/feedback (this message will be removed after the beta stage).

When("remove finalizer", func() {
It("finalizer should come back after provisioner restart", func() {
bdcName := "bdc-pvc-" + string(pvcObj.GetUID())
bdcObj, err := ops.BDCClient.WithNamespace(string(artifacts.OpenebsNamespace)).Get(bdcName,

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shadow: declaration of "err" shadows declaration at line 38 (from govet)

Copy link
Contributor

@kmova kmova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested a few code changes for refactor. That will be good to take in this PR.
Overall the functionality looks good.

@rahulchheda
Copy link
Contributor Author

BDD logs for better visibility :

rahul@rahul-ThinkPad-E490:~/go-dev/src/github.com/openebs/maya/tests/localpv$ ginkgo -v -- -kubeconfig=/home/rahul/.kube/config
Running Suite: Test application deployment
==========================================
Random Seed: 1575620756
Will run 26 of 26 specs

STEP: waiting for openebs-localpv-provisioner pod to come into running state
STEP: waiting for openebs-ndm-operator pod to come into running state
STEP: building a namespace
STEP: creating above namespace
TEST HOSTPATH LOCAL PV when pvc with storageclass openebs-hostpath is created 
  should create a pvc 
  /home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostpath_test.go:47
STEP: building a pvc
STEP: creating above pvc
•
------------------------------
TEST HOSTPATH LOCAL PV when deployment with busybox image is created 
  should create a deployment and a running pod
  /home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostpath_test.go:78
STEP: building a deployment
STEP: creating above deployment
STEP: verifying pod count as 1

• [SLOW TEST:11.666 seconds]
TEST HOSTPATH LOCAL PV
/home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostpath_test.go:32
  when deployment with busybox image is created
  /home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostpath_test.go:77
    should create a deployment and a running pod
    /home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostpath_test.go:78
------------------------------
TEST HOSTPATH LOCAL PV when deployment is deleted 
  should not have any deployment or running pod
  /home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostpath_test.go:140
STEP: deleting above deployment
STEP: verifying pod count as 0

• [SLOW TEST:32.768 seconds]
TEST HOSTPATH LOCAL PV
/home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostpath_test.go:32
  when deployment is deleted
  /home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostpath_test.go:139
    should not have any deployment or running pod
    /home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostpath_test.go:140
------------------------------
TEST HOSTPATH LOCAL PV when pvc with storageclass openebs-hostpath is deleted  
  should delete the pvc
  /home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostpath_test.go:159
STEP: deleting above pvc
•
------------------------------
[-ve] TEST HOSTDEVICE LOCAL PV when existing pvc with storageclass openebs-device is created 
  should create a pvc
  /home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostdevice_test.go:386
STEP: building a pvc
STEP: creating above pvc
•
------------------------------
[-ve] TEST HOSTDEVICE LOCAL PV when existing deployment with busybox image is created 
  should create a deployment and a running pod
  /home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostdevice_test.go:414
STEP: building a deployment
STEP: creating above deployment
STEP: verifying pod count as 1

• [SLOW TEST:11.475 seconds]
[-ve] TEST HOSTDEVICE LOCAL PV
/home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostdevice_test.go:363
  when existing deployment with busybox image is created
  /home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostdevice_test.go:413
    should create a deployment and a running pod
    /home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostdevice_test.go:414
------------------------------
[-ve] TEST HOSTDEVICE LOCAL PV when another pvc with storageclass openebs-device is created 
  should create a pvc 
  /home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostdevice_test.go:476
STEP: building a pvc
STEP: creating above pvc
•
------------------------------
[-ve] TEST HOSTDEVICE LOCAL PV when another deployment with busybox image and above pvc is created 
  should not create a deployment and a running pod
  /home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostdevice_test.go:504
STEP: building a deployment
STEP: creating above deployment
STEP: verifying pod count as 0
•
------------------------------
[-ve] TEST HOSTDEVICE LOCAL PV when above deployment is deleted 
  should not have any deployment or running pod
  /home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostdevice_test.go:566
STEP: deleting above deployment
STEP: verifying pod count as 0
•
------------------------------
[-ve] TEST HOSTDEVICE LOCAL PV when above pvc with storageclass openebs-device is deleted  
  should delete the pvc
  /home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostdevice_test.go:585
STEP: deleting above pvc
•
------------------------------
[-ve] TEST HOSTDEVICE LOCAL PV when existing deployment is deleted 
  should not have any deployment or running pod
  /home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostdevice_test.go:600
STEP: deleting above deployment
STEP: verifying pod count as 0
•
------------------------------
[-ve] TEST HOSTDEVICE LOCAL PV when existing pvc with storageclass openebs-device is deleted  
  should delete the pvc
  /home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostdevice_test.go:620
STEP: deleting above pvc
•
------------------------------
TEST HOSTDEVICE LOCAL PV when pvc with storageclass openebs-device is created 
  should create a pvc 
  /home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostdevice_test.go:50
STEP: building a pvc
STEP: creating above pvc
•
------------------------------
TEST HOSTDEVICE LOCAL PV when deployment with busybox image is created 
  should create a deployment and a running pod
  /home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostdevice_test.go:81
STEP: building a deployment
STEP: creating above deployment
STEP: verifying pod count as 1

• [SLOW TEST:22.715 seconds]
TEST HOSTDEVICE LOCAL PV
/home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostdevice_test.go:35
  when deployment with busybox image is created
  /home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostdevice_test.go:80
    should create a deployment and a running pod
    /home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostdevice_test.go:81
------------------------------
TEST HOSTDEVICE LOCAL PV when remove finalizer 
  finalizer should come back after provisioner restart
  /home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostdevice_test.go:142

• [SLOW TEST:6.782 seconds]
TEST HOSTDEVICE LOCAL PV
/home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostdevice_test.go:35
  when remove finalizer
  /home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostdevice_test.go:141
    finalizer should come back after provisioner restart
    /home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostdevice_test.go:142
------------------------------
TEST HOSTDEVICE LOCAL PV when deployment is deleted 
  should not have any deployment or running pod
  /home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostdevice_test.go:163
STEP: deleting above deployment
STEP: verifying pod count as 0

• [SLOW TEST:43.221 seconds]
TEST HOSTDEVICE LOCAL PV
/home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostdevice_test.go:35
  when deployment is deleted
  /home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostdevice_test.go:162
    should not have any deployment or running pod
    /home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostdevice_test.go:163
------------------------------
TEST HOSTDEVICE LOCAL PV when pvc with storageclass openebs-device is deleted  
  should delete the pvc
  /home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostdevice_test.go:182
STEP: deleting above pvc
•
------------------------------
TEST HOSTDEVICE LOCAL PV WITH VOLUMEMODE AS BLOCK when pvc with storageclass openebs-device, and volumeMode as Block, is created 
  should create a pvc 
  /home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostdevice_test.go:213
STEP: building a pvc
STEP: creating above pvc
•
------------------------------
TEST HOSTDEVICE LOCAL PV WITH VOLUMEMODE AS BLOCK when deployment with busybox image is created 
  should create a deployment and a running pod
  /home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostdevice_test.go:246
STEP: building a deployment
STEP: creating above deployment
STEP: verifying pod count as 1

• [SLOW TEST:11.468 seconds]
TEST HOSTDEVICE LOCAL PV WITH VOLUMEMODE AS BLOCK
/home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostdevice_test.go:198
  when deployment with busybox image is created
  /home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostdevice_test.go:245
    should create a deployment and a running pod
    /home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostdevice_test.go:246
------------------------------
TEST HOSTDEVICE LOCAL PV WITH VOLUMEMODE AS BLOCK when remove finalizer 
  finalizer should come back after provisioner restart
  /home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostdevice_test.go:308

• [SLOW TEST:7.168 seconds]
TEST HOSTDEVICE LOCAL PV WITH VOLUMEMODE AS BLOCK
/home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostdevice_test.go:198
  when remove finalizer
  /home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostdevice_test.go:307
    finalizer should come back after provisioner restart
    /home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostdevice_test.go:308
------------------------------
TEST HOSTDEVICE LOCAL PV WITH VOLUMEMODE AS BLOCK when deployment is deleted 
  should not have any deployment or running pod
  /home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostdevice_test.go:329
STEP: deleting above deployment
STEP: verifying pod count as 0

• [SLOW TEST:32.768 seconds]
TEST HOSTDEVICE LOCAL PV WITH VOLUMEMODE AS BLOCK
/home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostdevice_test.go:198
  when deployment is deleted
  /home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostdevice_test.go:328
    should not have any deployment or running pod
    /home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostdevice_test.go:329
------------------------------
TEST HOSTDEVICE LOCAL PV WITH VOLUMEMODE AS BLOCK when pvc with storageclass openebs-device is deleted  
  should delete the pvc
  /home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostdevice_test.go:348
STEP: deleting above pvc
•
------------------------------
[-ve] TEST HOSTDEVICE LOCAL PV WITH VOLUMEMODE AS BLOCK  when existing pvc with storageclass openebs-device is created 
  should create a pvc
  /home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostdevice_test.go:659
STEP: building a pvc
STEP: creating above pvc
•
------------------------------
[-ve] TEST HOSTDEVICE LOCAL PV WITH VOLUMEMODE AS BLOCK  when existing deployment with busybox image is created 
  should create a deployment but should be unable to get a running pod, with PVC volumeMode set to Block,but added as volumeMount in Deployment
  /home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostdevice_test.go:687
STEP: building a deployment, with volume Mount for a Block volumeMode PVC
STEP: creating above deployment
STEP: verifying pvc status as bound
STEP: verifying pod count as 0

• [SLOW TEST:182.099 seconds]
[-ve] TEST HOSTDEVICE LOCAL PV WITH VOLUMEMODE AS BLOCK 
/home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostdevice_test.go:635
  when existing deployment with busybox image is created
  /home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostdevice_test.go:686
    should create a deployment but should be unable to get a running pod, with PVC volumeMode set to Block,but added as volumeMount in Deployment
    /home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostdevice_test.go:687
------------------------------
[-ve] TEST HOSTDEVICE LOCAL PV WITH VOLUMEMODE AS BLOCK  when above deployment is deleted 
  should not have any deployment or running pod
  /home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostdevice_test.go:752
STEP: deleting above deployment
STEP: verifying pod count as 0
•
------------------------------
[-ve] TEST HOSTDEVICE LOCAL PV WITH VOLUMEMODE AS BLOCK  when existing pvc with storageclass openebs-device is deleted  
  should delete the pvc
  /home/rahul/go-dev/src/github.com/openebs/maya/tests/localpv/hostdevice_test.go:771
STEP: deleting above pvc
•STEP: deleting namespace

Ran 26 of 26 Specs in 372.072 seconds
SUCCESS! -- 26 Passed | 0 Failed | 0 Pending | 0 Skipped
PASS

Ginkgo ran 1 suite in 6m15.507395539s
Test Suite Passed

Copy link
Contributor

@akhilerm akhilerm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.
Thanks for taking this up @rahulchheda

Copy link
Contributor

@prateekpandey14 prateekpandey14 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lgtm

Rahul M Chheda added 8 commits December 6, 2019 18:33
- Added Builder Function for support & checking Block Device
- Returned true, for support of the Block Device
- Provided the path for the raw Block device, if provided

Signed-off-by: Rahul M Chheda <[email protected]>
…upport, and some other minor changes highlighter below

  - Added containerBuilder function (funcName: WithVolumeDevices Builder)
  - Added pvcBuilder function (funcname: WithVolumeMode Builder)
  - Added 2 BDD TestCases :
    - [+ve] Basic Raw Block Volume Support
    - [-ve] If PVC volumeMode is "Block", but mountPath is provided in Deployment PodSpecTemplate

Signed-off-by: Rahul M Chheda <[email protected]>
    - localPv-provisioner removed if-else case for providing blockdevice path or mountPath
    - localPV-provisioner change PV volumeMode type to "Block", if PVC has it.
    - Added appropraite comments in the BDD

Signed-off-by: Rahul M Chheda <[email protected]>
@prateekpandey14 prateekpandey14 merged commit 799dc6c into openebs-archive:master Dec 6, 2019
shubham14bajpai pushed a commit to shubham14bajpai/maya that referenced this pull request Dec 27, 2019
- Added Builder Function for support & checking Block Device
- Returned true, for support of the Block Device
- Provided the path for the raw Block device, if provided
 - Added containerBuilder function (funcName: WithVolumeDevices Builder)
  - Added pvcBuilder function (funcname: WithVolumeMode Builder)
  - Added 2 BDD TestCases :
    - [+ve] Basic Raw Block Volume Support
    - [-ve] If PVC volumeMode is "Block", but mountPath is provided in Deployment 
      PodSpecTemplate

Signed-off-by: Rahul M Chheda <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants