Add csi-s3 storage class as big data passing storage

[Tomcli]

Which issue is resolved by this Pull Request:
Resolves #569

Description of your changes:

• Add CSI-S3 storage class third party plugins. (Translated from Datashim's CSI S3 helm chart to kustomize)
• Add storage class configuration to big data passing (default to s3 storage), this way it doesn't need to provision new cloud storage for every new PVC.
• Use in conjunction with Add artifact mid path to big data passing #751 to store the artifact in a centralized S3 artifact storage such as minio and COS.

Environment tested:

• Python Version (use python --version):
• Tekton Version (use tkn version):
• Kubernetes Version (use kubectl version):
• OS (e.g. from /etc/os-release):

Checklist:

• The title for your pull request (PR) should follow our title convention. Learn more about the pull request title convention used in this repository.

[google-oss-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Tomcli

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [Tomcli]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[Tomcli]

@Udiknedormin This is the open source S3 CSI storage we use for mounting S3 storage. If there are other storages you want to use for mounting S3, please let us know. Thanks.

[Tomcli]

@ScrapCodes can you put lgtm here if this PR looks good to you?

[Tomcli]

/hold
the datashim team has some conflicts with this deployment. Need to rename the storageclass.

[ScrapCodes]

Hi @Tomcli ,

Somehow, I am not succesfull in installing on IBM Cloud k8s cluster.

So, tried on a standalone instance.

After succesfully applying the following commands,

kubectl apply -k cluster-scoped-resources/
kubectl apply -k "env/platform-agnostic/" 
kubectl wait crd/applications.app.k8s.io --for condition=established --timeout=60s

And trying to run the big_data_passing pipeline locally, I am getting POds in pending

          Message:               pod status "PodScheduled":"False"; message: "0/1 nodes are available: 1 pod has unbound immediate PersistentVolumeClaims."

May be CSI-s3 does not work on a standalone instance.

(.venv) 1 prashantsharma@prashant:~/go/pkg/src/github.com/kubeflow/pipelines$ kubectl logs -l app=csi-provisioner-s3  -n kubeflow
W1025 15:48:57.371227       1 connection.go:172] Still connecting to unix:///csi/csi.sock
W1025 15:49:07.370712       1 connection.go:172] Still connecting to unix:///csi/csi.sock
W1025 15:49:17.370715       1 connection.go:172] Still connecting to unix:///csi/csi.sock
W1025 15:49:27.370715       1 connection.go:172] Still connecting to unix:///csi/csi.sock
W1025 15:49:37.370727       1 connection.go:172] Still connecting to unix:///csi/csi.sock
W1025 15:49:47.370692       1 connection.go:172] Still connecting to unix:///csi/csi.sock
W1025 15:49:57.370711       1 connection.go:172] Still connecting to unix:///csi/csi.sock
W1025 15:50:07.370688       1 connection.go:172] Still connecting to unix:///csi/csi.sock
W1025 15:50:17.371190       1 connection.go:172] Still connecting to unix:///csi/csi.sock
W1025 15:50:27.370714       1 connection.go:172] Still connecting to unix:///csi/csi.sock

[Tomcli]

/unhold

[Tomcli]

kubectl apply -k "env/platform-agnostic/"

Sorry with the late reply. kubectl apply -k "env/platform-agnostic/" should work if you are using a kubectl client that is v1.20+. I updated that as part of the developer readme.

[ScrapCodes]

Overall looks good to me !

Here we changed the default storage class to kfp-csi-s3, but the manifests installs a storage class csi-s3. Do you think we can add this point to documentation ?
Or can we change the manifests to install a storage class kfp-csi-s3 by default.

Looks like a I missed a change, LGTM!!

[google-oss-prow]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ScrapCodes, Tomcli

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [ScrapCodes,Tomcli]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[ScrapCodes]

/hold

[ScrapCodes]

/lgtm

Feel free to unhold, if you think this is good to go.

[ScrapCodes]

/unhold

[ScrapCodes]

@Tomcli
Is it normal to see a very high capacity for the PVC

k get pvc
NAME             STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
pvc-0fc0312d1f   Bound    pvc-9b269ff0-9efb-41eb-a714-812be702d950   9314Gi     RWO            kfp-csi-s3     18s

Even though the configured capacity is:

env | grep -i default
DEFAULT_ACCESSMODES=ReadWriteOnce
DEFAULT_STORAGE_CLASS=kfp-csi-s3
DEFAULT_STORAGE_SIZE=2Gi

[Tomcli]

@Tomcli Is it normal to see a very high capacity for the PVC

k get pvc
NAME             STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
pvc-0fc0312d1f   Bound    pvc-9b269ff0-9efb-41eb-a714-812be702d950   9314Gi     RWO            kfp-csi-s3     18s

Even though the configured capacity is:

env | grep -i default
DEFAULT_ACCESSMODES=ReadWriteOnce
DEFAULT_STORAGE_CLASS=kfp-csi-s3
DEFAULT_STORAGE_SIZE=2Gi

I think that's the default behavior of the S3 CSI because the S3 storage can be expanded without updating the PVC. (E.g. when using IBM Cloud object storage). So they used the maximum capacity for pvc as default.