Physical Disk Lifecycle: Tracking Issue

[smklein]

This is a tracking issue for a few related issues I'm actively working on regarding Physical Disk lifecycles.

In broad terms, we need to do the following:

• Decouple the notion of a "control plane physical disk" from a "physical disk in inventory". They're currently overloaded in the single "Physical Disk" database table, upserted by sleds through the Nexus API, but need to be decoupled to properly manage their lifecycles.
• Allow Nexus to control more of the U.2 storage explicitly, so it can understand the fall-out of expunging disks.
• Refactor "when disks are actually being formatted" within the Sled Agent, and defer this process until their corresponding "Control plane objects" are created.
• Provide opportunities for Nexus to answer ambiguous questions about disks, like "should a new disk be adopted by the system", or "what do we do with a U.2 that has non-conforming partitions"?

Some additional background context can be found on https://github.com/oxidecomputer/rfd/pull/712 , which is an addendum to RFD 457.

In specific issues:

• Add physical disk information to the Nexus inventory subsystem #5150: We need to know physical disks exist in Nexus before they can be processed further.
• Let Nexus control "when Physical Disks, Zpools, and Datasets are created", rather than relying on self-reporting from sleds #5151: Once we know physical disks exist, we can manage their lifecycle explicitly. This will also change interactions with RSS, which will probably need to also take control of this provisioning too.
• Let Nexus decide how to deal with old partitions on new U.2s #5152: Once Nexus can provision new physical disks, it can also take charge of decisions like "what do we do with weirdly-formatted disks".
• [nexus] Make OmicronZoneConfig include selection of zone root filesystem #5048: This is a pre-requisite for knowing "what could die when we expunge a disk"
• Add notion of physical disk "policy" and "state" to physical disks in the control plane #5153: These are the levers to actually initiate the expungement process
• Sled Agent should durably track which disks are "part of control plane", usable for internal storage #5154
• Expose Sled Agent API to start/stop using physical disks (U.2s) #5155
• [sled-storage] Remove "self-managed" physical disks #5328
• Expunge disks when sleds are expunged #5369
• Add endpoint to expunge disks (without whole-sled expungement) #5370
• expunge sled: planner must not use physical disks being removed #5372

[davepacheco]

@smklein is #5150 the only remaining issue here?

[smklein]

As far as I know. I can close this larger issue and track #5150 separately, if you want?

[davepacheco]

Sure!