-
Notifications
You must be signed in to change notification settings - Fork 2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CSI: disambiguate or merge create and register specs #11899
Comments
Hi @NomAnor! For clarity, let's use the following nomenclature, taken from Nomad's struct definition (ref
The CSI RPC we're talking about is the When the Nomad servers construct the client RPC request for validation (ref
The expectation is that the user provides the The reason for the split in behavior is unfortunately historical. Originally Nomad didn't support a
No doubt there's some issue here in the documentation, but it's not that folks are using the External ID as Nomad ID. The AWS EBS and AWS EFS plugins that we test in our nightly E2E test have different External ID and Nomad ID (ex. AWS's ID
It's not required in the spec that When we use the That being said, it's very unfortunate that the |
I complete missed the I seems that If a plugin does not support volume creation nomad must call Having only a single command to create a volume in nomad, wether a new one or an existing one, seems for me to be a good solution. The user then has to either supply Maybe if you want to deprecate the Reporting a warning or an error when |
It occurs to me that if we merge |
I've retitled this issue to more accurately reflect what I want to do with this one. I'm working on a big stretch of CSI work including a lot of polishing, so it seems like this would be good to resolve within that block of work. |
I'm going to mark this as closed by #12167, which did not merge the create and register spec, but tightened up the behavior to be much less surprising. We'll continue the work to handle safe re-registration in #10324 and other future feature expansions. Thanks again for opening this issue @NomAnor, it really helped clarify some of the work to be done here. |
I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues. |
Nomad version
Nomad v1.2.3
Issue
If I try to register an already existing volume the
ValidateVolumeCapabilities
call is using the nomad volume id instead of thevolume_id
.As far as I understand it, the
ValidateVolumeCapabilities
RPC should be used by the CO to verify that the volume has the expected capabilities after it is created. To do so the CO sends thevolume_id
. This is the internal (to the CSI plugin) id returned from theCreateVolume
call. Nomad instead calls it with the volume id that is internal only to nomad (specified in the volume file).The different ids are a bit confusing. There are three "ids" involved when using CSI volumes in nomad:
volume_id
: Internal id used by the plugin to distinguish the volumes. Set by the plugin on volume creation. Opaque string to nomad. The user can't control this.In my case
volume_id
is the uuid of the lvm volumes and my plugin only uses the name to provide indempotency (currently by using it as the lv name, but I might change that to tags).Even when registering an already existing volume, nomad should call
CreateVolume
because that seems to be the only way to get avolume_id
for a volume name from the CSI plugin. The call is indempotent based on the name so no new volume would be created.The lack of a similar issues seems to suggest that other users use the storage provider id as the nomad volume id. If that is intended the documentation should clearly state so. But this is incompatible with volume creation because the storage provider id can't be known before (it is required in the volume definition).
Volume definition
Plugin logs
This is the request when registering the already existing volume:
This is the request when creating the already existing volume with the same file:
The cli then show the
volume_id
You can then see all three ids in the ui:
![volume](https://user-images.githubusercontent.com/8003824/150539265-ae5d275f-2b4f-4c7a-902f-9354e0fb12d9.png)
The text was updated successfully, but these errors were encountered: