Change node-level idempotency to volume-target

[jacobwolfaws]

Is this a bug fix or adding new feature?
bug fix

What is this PR about? / Why do we need it?
This PR changes the behavior of Node-level idempotency for the CSI Driver. Currently, the node-level operations are idempotent using volumeID. This behavior prevents two pods from mounting the same volume in parallel. When one pod is stuck pending/terminating (mounting/unmounting), this prevents all other pods from mounting or unmounting the volume. Since each pod has a unique target:volume, we use the target in the inflight key to ensure that idempotency is pod-volume-level, not node-volume-level.

What testing is done?

• Added sanity tests to ensure idempotency is at pod-volume-level
• Added sleep to NodePublishVolume to confirm that multiple keys being registered and that one pod mounting doesn't block another

I1106 22:22:16.296515       1 node.go:89] "NodePublishVolume: called with" args={"volume_id":"fs-0a981b1c41788abf3","target_path":"/var/lib/kubelet/pods/2e4617af-cc02-48dc-bfd6-d692b5cf376e/volumes/kubernetes.io~csi/pvc-ef3c54f2-6d96-4725-b37d-ff722b8c61ad/mount","volume_capability":{"AccessType":{"Mount":{"mount_flags":["flock"]}},"access_mode":{"mode":5}},"volume_context":{"dnsname":"fs-0a981b1c41788abf3.fsx.us-east-1.amazonaws.com","mountname":"vh3n5bev","storage.kubernetes.io/csiProvisionerIdentity":"1699308706070-3089-fsx.csi.aws.com"}}
I1106 22:22:16.296544       1 inflight.go:63] "Inflight map: " inflightMap={"fs-0a981b1c41788abf3-/var/lib/kubelet/pods/2e4617af-cc02-48dc-bfd6-d692b5cf376e/volumes/kubernetes.io~csi/pvc-ef3c54f2-6d96-4725-b37d-ff722b8c61ad/mount":true,"fs-0a981b1c41788abf3-/var/lib/kubelet/pods/69de3cb1-48d7-43bf-b99b-b442c51a147a/volumes/kubernetes.io~csi/pvc-ef3c54f2-6d96-4725-b37d-ff722b8c61ad/mount":true}

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jacobwolfaws, torredil

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [jacobwolfaws,torredil]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[khoang98]

/lgtm