azure: agent-less ephemeral disk support

[arithx]

Related: #65

Two of the main benefits of WALA (https://github.com/Azure/WALinuxAgent), for immutable hosts, are checking in to Azure to confirm the machine has finished creation (see more info in the comments of #65) and ephemeral disks support.

Ephemeral disks are not pre-formatted and most disks have a single partition unless they're >4TB. Which disk is the ephemeral disk must be performed and the method of detection differs between gen 1 & gen 2 VMs (cloud-init has udev rules to handle both gens). Additionally this check is going to have to run each boot as a new ephemeral disk can be given to a VM if it is moved to a new host.

[lucab]

@arithx in CL we carry https://github.com/coreos/init/blob/master/udev/rules.d/66-azure-storage.rules and WALA has https://github.com/Azure/WALinuxAgent/blob/master/config/66-azure-storage.rules (which seems to be the same). Assuming we will carry that on agent-less FCOS too, which other config/binaries are we missing for ephemeral disks? Are we concerned with detection/symlink only or do we have something higher-level to configure and support?

[arithx]

@lucab My understanding is that we'd want to the detection/symlink as well as the formatting (if they aren't pre-formatted).

As far as the udev rules I haven't personally looked into it. They do look like they cover both the gen 1 ID based disks & gen 2 specific SCSI controller cases though.

@bgilbert might have some additional thoughts.

[dustymabe]

@lucab My understanding is that we'd want to the detection/symlink as well as the formatting (if they aren't pre-formatted).

is there a reason we couldn't just rely on the user to know/handle this part using ignition-disks?

[arithx]

@dustymabe a new ephemeral disk that is not formatted can happen when the machine moves to a new host on Azure which could happen during a reboot.

…

On Dec 13, 2018, at 13:54, Dusty Mabe ***@***.***> wrote: @lucab My understanding is that we'd want to the detection/symlink as well as the formatting (if they aren't pre-formatted). is there a reason we couldn't just rely on the user to know/handle this part using ignition-disks? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[dustymabe]

@dustymabe a new ephemeral disk that is not formatted can happen when the machine moves to a new host on Azure which could happen during a reboot.

sure, but doesn't that represent a change the user requested ? (i.e. "Give me a new ephemeral disk"). It seems like we should be able to get by with leaving this to the user. The two questions I would ask are:

• What do people on other cloud platforms do in this same situation?
• Will it be a huge drop in user experience that we don't automatically do something here whereas other OS that include the agent do do something here.

[arithx]

I don’t think that a user would have to request that. In most cloud providers they will shift offline machines around to different hosts if they are trying to rebalance. If Azure does this as well there could be a scenario where the user reboots for say an update and Azure rebalances and moves the VM to a new host thus giving it a new ephemeral disk.

…

On Dec 13, 2018, at 14:06, Dusty Mabe ***@***.***> wrote: @dustymabe a new ephemeral disk that is not formatted can happen when the machine moves to a new host on Azure which could happen during a reboot. sure, but doesn't that represent a change the user requested ? (i.e. "Give me a new ephemeral disk"). It seems like we should be able to get by with leaving this to the user. The two questions I would ask are: What do people on other cloud platforms do in this same situation? Will it be a huge drop in user experience that we don't automatically do something here whereas other OS that include the agent do do something here. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[dustymabe]

If Azure does this as well there could be a scenario where the user reboots for say an update and Azure rebalances and moves the VM to a new host thus giving it a new ephemeral disk.

What does Azure do here differently, than say AWS ?

[arithx]

AWS can move machines to different hosts while they're offline. I believe the main difference is probably just that the ephemeral disks on AWS aren't NTFS formatted by default.

[dustymabe]

Would one option here be to wait until someone from the community identifies it as a need?

[arithx]

Personally I think it would be fair to ship FCOS without ephemeral disks support initially.

[ajeddeloh]

Personally I think it would be fair to ship FCOS without ephemeral disks support initially.

Agreed. When we do want to add support we could add a systemd unit that depends on the well known path(s) of the ephemeral disks and reformats them if they are ntfs. We could ship that unit by default via Ignition then let users mask it.

@arithx Did you ever find out if there will be more than one ephemeral disk?

[arithx]

@ajeddeloh I did not. I briefly searched through to see if I could find more info but nothing popped out.

[dustymabe]

It seems like we are circling on not shipping this functionality. If a user requests it we can revisit.

@arithx - can we open a PR to the design doc to document this decision?

[dustymabe]

with #102 should we close this out as decided?

[arithx]

@dustymabe yeah, we cover the decision in that PR.