merge mock and hostpath driver

[pohly]

Both mock and hostpath driver are used for testing, just in different configurations and with different capabilities of the driver. It's not clear whether there actually is a reason to maintain two different drivers. Perhaps they should be merged into one.

Existing capabilities:

• check credentials (mock)
• optional attach (mock)
• snapshots (both?)
• error injection via JavaScript hooks (mock - may become obsolete via e2e storage: embedded csi mock driver kubernetes/kubernetes#97069)
• raw block volumes (hostpath)
• actual storage of data, including snapshots (hostpath)
• simulating capacity (hostpath)
• multi-node deployment (hostpath)

Missing capabilities:

• support non-default kubelet path (hostpath.csi.k8s.io not found in the list of registered CSI drivers #71) and use that to avoid leaking loop devices (number of loop devices is fixed and unpredictable kubernetes-sigs/kind#1248)

[pohly]

We discussed this and came to the conclusion that as long as we only need to maintain one driver it doesn't matter too much which one we pick. However, the hostpath driver seems to be a better starting point because it is more widely used (several Prow jobs), is more realistic (actual mounts) and more complete (raw block support), and can be released independently because it is in its own repo. The mock driver currently must be released together with csi-sanity.

So here's a tentative plan:

• move functionality from the mock driver to hostpath:
  • JSON call logging (https://github.com/kubernetes-csi/csi-test/blob/c4bffc15d5e956e1dee5ba88d1c1e007960774c2/driver/driver.go#L216-L243)
  • proxy (kubernetes-csi/csi-test@ef07baf)
  • implement ControllerPublishVolume and ControllerUnpublishVolume (

    csi-driver-host-path/pkg/hostpath/controllerserver.go

    Lines 256 to 262 in 2d6e697

    	func (hp *hostPath) ControllerPublishVolume(ctx context.Context, req *csi.ControllerPublishVolumeRequest) (*csi.ControllerPublishVolumeResponse, error) {
    	return nil, status.Error(codes.Unimplemented, "")
    	}
    	func (hp *hostPath) ControllerUnpublishVolume(ctx context.Context, req *csi.ControllerUnpublishVolumeRequest) (*csi.ControllerUnpublishVolumeResponse, error) {
    	return nil, status.Error(codes.Unimplemented, "")
    	}

    ,

    csi-driver-host-path/pkg/hostpath/controllerserver.go

    Lines 256 to 262 in 2d6e697

    	func (hp *hostPath) ControllerPublishVolume(ctx context.Context, req *csi.ControllerPublishVolumeRequest) (*csi.ControllerPublishVolumeResponse, error) {
    	return nil, status.Error(codes.Unimplemented, "")
    	}
    	func (hp *hostPath) ControllerUnpublishVolume(ctx context.Context, req *csi.ControllerUnpublishVolumeRequest) (*csi.ControllerUnpublishVolumeResponse, error) {
    	return nil, status.Error(codes.Unimplemented, "")
    	}

    ), with checking that the volume lifecycle is followed by the CO (CreateVolume -> ControllerPublishVolume -> NodeStageVolume -> NodePublishVolume -> NodeUnpublishVolume -> NodeUnstageVolume -> ControllerUnpublishVolume -> DeleteVolume). This must be an optional feature of the driver, to support testing with and with external-attacher.
• replace Kubernetes E2E tests that currently use mock driver with that enhanced hostpath driver or with embedded error injection (for credential checking)
• vendor the enhanced hostpath driver into Kubernetes to get rid of the mock driver fork
• remove mock driver from csi-test

[avalluri]

With #269 and the below changes in Kubernetes e2e, I could able to run mock csi storage tests:

diff --git a/test/e2e/storage/drivers/csi.go b/test/e2e/storage/drivers/csi.go
index 68c3705d607..06a93f4f658 100644
--- a/test/e2e/storage/drivers/csi.go
+++ b/test/e2e/storage/drivers/csi.go
@@ -553,10 +553,10 @@ func (m *mockCSIDriver) PrepareTest(f *framework.Framework) (*storageframework.P
        } else {
                // When using the mock driver inside the cluster it has to be reconfigured
                // via command line parameters.
-               containerArgs = append(containerArgs, "--name=csi-mock-"+f.UniqueName)
+               containerArgs = append(containerArgs, "--drivername=csi-mock-"+f.UniqueName)
 
-               if !m.attachable {
-                       containerArgs = append(containerArgs, "--disable-attach")
+               if m.attachable {
+                       containerArgs = append(containerArgs, "--enable-attach")
                }
 
                if m.enableTopology {
diff --git a/test/e2e/testing-manifests/storage-csi/mock/csi-mock-driver.yaml b/test/e2e/testing-manifests/storage-csi/mock/csi-mock-driver.yaml
index fe3f062afb4..ad9280c4bdd 100644
--- a/test/e2e/testing-manifests/storage-csi/mock/csi-mock-driver.yaml
+++ b/test/e2e/testing-manifests/storage-csi/mock/csi-mock-driver.yaml
@@ -15,7 +15,7 @@ spec:
       serviceAccountName: csi-mock
       containers:
         - name: csi-provisioner
-          image: k8s.gcr.io/sig-storage/csi-provisioner:v2.1.0
+          image: k8s.gcr.io/sig-storage/csi-provisioner:v2.1.1
           args:
             - "--csi-address=$(ADDRESS)"
             # Topology support is needed for the pod rescheduling test
@@ -53,10 +53,12 @@ spec:
           - mountPath: /registration
             name: registration-dir
         - name: mock
-          image: k8s.gcr.io/sig-storage/mock-driver:v4.1.0
+          image: localhost/hostpathplugin:latest
           args:
-            - "--name=mock.storage.k8s.io"
-            - "-v=3" # enabled the gRPC call logging
+            - "--drivername=mock.storage.k8s.io"
+            - "--nodeid=$(KUBE_NODE_NAME)"
+            - "--endpoint=/csi/csi.sock"
+            - "-v=5" # enabled the gRPC call logging
           env:
             - name: CSI_ENDPOINT
               value: /csi/csi.sock
@@ -74,6 +76,10 @@ spec:
               name: kubelet-pods-dir
             - mountPath: /var/lib/kubelet/plugins/kubernetes.io/csi
               name: kubelet-csi-dir
+            - mountPath: /csi-data-dir
+              name: csi-data-dir
+            - mountPath: /dev
+              name: dev-dir
       volumes:
         - hostPath:
             path: /var/lib/kubelet/plugins/csi-mock
@@ -93,3 +99,12 @@ spec:
             path: /var/lib/kubelet/plugins_registry
             type: Directory
           name: registration-dir
+        - hostPath:
+            # 'path' is where PV data is persisted on host.
+            path: /var/lib/csi-hostpath-data/
+            type: DirectoryOrCreate
+          name: csi-data-dir
+        - hostPath:
+            path: /dev
+            type: Directory
+          name: dev-dir
diff --git a/test/e2e/testing-manifests/storage-csi/mock/csi-mock-proxy.yaml b/test/e2e/testing-manifests/storage-csi/mock/csi-mock-proxy.yaml
index 7892392931e..abf5d8e1f26 100644
--- a/test/e2e/testing-manifests/storage-csi/mock/csi-mock-proxy.yaml
+++ b/test/e2e/testing-manifests/storage-csi/mock/csi-mock-proxy.yaml
@@ -15,7 +15,7 @@ spec:
       serviceAccountName: csi-mock
       containers:
         - name: csi-provisioner
-          image: k8s.gcr.io/sig-storage/csi-provisioner:v2.1.0
+          image: k8s.gcr.io/sig-storage/csi-provisioner:v2.1.1
           args:
             - "--csi-address=$(ADDRESS)"
             # Topology support is needed for the pod rescheduling test
@@ -53,16 +53,23 @@ spec:
           - mountPath: /registration
             name: registration-dir
         - name: mock
-          image: k8s.gcr.io/sig-storage/mock-driver:v4.1.0
+          image: localhost/hostpathplugin:latest
           args:
             # -v3 shows when connections get established. Higher log levels print information about
             # transferred bytes, but cannot print message content (no gRPC parsing), so this is usually
             # not interesting.
-            - -v=3
+            - -v=5
+            - -nodeid=$(KUBE_NODE_NAME)
+            - -endpoint=/csi/csi.sock
             - -proxy-endpoint=tcp://:9000
           env:
             - name: CSI_ENDPOINT
               value: /csi/csi.sock
+            - name: KUBE_NODE_NAME
+              valueFrom:
+                fieldRef:
+                  apiVersion: v1
+                  fieldPath: spec.nodeName
           ports:
             - containerPort: 9000
               name: socat

$ kubetest --provider=local --test --test_args="--ginkgo.focus='\[sig-storage\] CSI mock volume' --test.v"

Mar 29 12:49:33.605: INFO: Running AfterSuite actions on node 1
{"msg":"Test Suite completed","total":38,"completed":38,"skipped":5707,"failed":0}

Ran 38 of 5745 Specs in 3896.168 seconds
SUCCESS! -- 38 Passed | 0 Failed | 0 Pending | 5707 Skipped
--- PASS: TestE2E (3896.18s)
=== RUN   TestViperConfig
--- PASS: TestViperConfig (0.00s)
PASS

Ginkgo ran 1 suite in 1h4m57.73308344s
Test Suite Passed
2021/03/29 12:49:33 process.go:155: Step './hack/ginkgo-e2e.sh --ginkgo.focus=\[sig-storage\].CSI.mock.volume --test.v' finished in 1h4m57.833580413s

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale

[pohly]

/reopen
/assign

All functionality was moved over into csi-driver-host-path and only that driver is used now.

However, we still need to remove the mock driver source code and then at some point do a csi-test release without it.

[k8s-triage-robot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle rotten

[pohly]

/remove-lifecycle rotten