From ca0e1ea8586a60da0c8311a847b0c5d8e92de1ee Mon Sep 17 00:00:00 2001 From: Rain Date: Wed, 24 Jul 2024 19:22:28 -0700 Subject: [PATCH] [nexus/sled-agent] move some types around, straighten up the dependency graph (#6138) Reorganize some of our data structures and simplify the dependency graph. In particular, there's now just one copy of `OmicronZonesConfig` and related types. This is _mostly_ code reorganization, but there are a couple of interesting bits in here. Scroll down to the Questions section for more. ## Motivation ### 1. The necessity of sled-agent-types The primary driver behind this change was turning sled-agent's API into a trait -- breaking the sled-agent/nexus dependency cycle was one of the main motivations for that work. To recall, we currently have this logical cycle in the graph ([reference](https://rfd.shared.oxide.computer/rfd/0479#circular_deps)): ![Screenshot from 2024-07-21 21-03-02](https://github.com/user-attachments/assets/9227d036-ab37-4b1e-b4ae-e390a84e829b) With the work, we're attempting to move somewhere closer to this graph: ![Screenshot from 2024-07-21 21-07-30](https://github.com/user-attachments/assets/4d93e98e-cfdc-46dc-9244-7fca8b72d5d1) But to make that happen, some common types need to be moved into a `sled-agent-types` crate, so that the upcoming `sled-agent-api` crate can depend on them. Why not put them in `sled-agent-api` itself? Because: * In `sled-agent-client`, we often end up normalizing to a shared type -- so we either want to implement `From` conversions between these types and `sled-agent-client` types, or to `replace` them outright. So one of them has to depend on the other. * If there's even one type that we want to `replace`, the dependency direction must be `sled-agent-client` -> `sled-agent-api`. * But the client depending on the API trait is exactly the kind of circular dependency we're trying to avoid. * Another reason for trying to avoid a circular dependency here is that it has the potential to leave types "floating in the ether", just circulating via the schema with no definition in Rust. (This actually is a risk today with the way dependencies are set up!) So the only feasible alternative is to put these types in a shared crate. This crate is `sled-agent-types`, which we recently introduced in https://github.com/oxidecomputer/omicron/pull/6122. ### 2. sled-agent-types <-> nexus-types For types shared between Nexus and sled-agent, where should we put them? The options are: * **omicron-common**: this is where we've been putting these types in the past, and is not a terrible place to put them. However, if those types pull in extra dependencies it becomes a bit more problematic, because omicron-common is depended on by several workspaces outside omicron. * A secondary option would be putting these types behind an optional feature. But that would cause a lot of unnecessary rebuilds of omicron-common and all of its dependents (essentially splitting the build unit universe into two), while the option we eventually chose does not. * **sled-agent-types** or **nexus-types**: we could put them in one or the other, but we'd have to introduce a dependency between them at some point, and they're both logically peers -- neither direction seems like the right one. * **nexus-config**: This depends on `tokio-postgres` (and was actually split out of omicron-common for that reason). It wouldn't be great for nexus-client or sled-agent-client to transitively pull in tokio-postgres. * **a new crate**: This is the option we've chosen. It does add the complexity of another crate, but resolves all of the previous points. We're calling this **nexus-sled-agent-shared** to signal that it sits on top of omicron-common and below nexus-types and sled-agent-types. Something to consider is whether we want to have two clients in omicron, one for internal consumers with more dependencies and one for external consumers with fewer ones. I'm not saying we should do this; it's just something to keep at the back of one's head. ## The changes There are a number of closely-related changes here -- most of them had to be atomic with this one. A description of the changes and the rationale for them: ### 1. Zone config types These are `OmicronZonesConfig`, `OmicronZoneConfig`, `OmicronZoneType`, and `OmicronZoneDataset`. In this PR, I've moved them from `sled-agent` to `nexus-sled-agent-shared`. * These types are an important part of the sled-agent API, and `nexus-types` previously used the progenitor-generated copy of them. * However that has had some issues for us, e.g. https://github.com/oxidecomputer/omicron/issues/4988. * Personally speaking, having two copies of such a complex type floating around has been suboptimal for me while I've been writing blueprint code. For example, previously, ctrl-clicking on `OmicronZonesConfig` in vscode always pointed me at the progenitor macro which wasn't helpful. Now, it points at the definition in `nexus-sled-agent-shared`. * There's an extra transitive dependency this introduces, which is why they're in `nexus-sled-agent-shared`. There's one incident that I found along the way that I believe is a **bug fix**. We have rules for DNS server IP addresses: * External DNS servers can be IPv4 or IPv6 * Internal DNS servers are always v6 But the Progenitor-generated copy of `OmicronZonesConfig` uses strings to store address/port pairs. As a result, in some cases we managed to store an IPv4 address for an internal DNS server. With this PR, this case is now detected. (I think it would have been ultimately rejected at some point, but it's possible for the bad DNS server to end up being persisted.) ### 2. `ZoneKind` There were previously two dataless enums describing the kind of Omicron zone floating around. 1. [`ZoneKind` in `sled-agent-client`](https://github.com/oxidecomputer/omicron/blob/26959bfc1a073c91a920bd521da13a980d137e3f/clients/sled-agent-client/src/lib.rs#L87), used by Nexus: exactly `OmicronZoneType` without any associated data. 2. [`ZoneType` in `sled-agent`](https://github.com/oxidecomputer/omicron/blob/26959bfc1a073c91a920bd521da13a980d137e3f/sled-agent/src/params.rs#L265), used by sled-agent -- identifying the substring used within the service (notably with `BoundaryNtp` and `InternalNtp` combined into one). While looking around, I also noticed that there were actually four different string representations in use for Omicron zones: 1. the zone prefix 2. the service prefix 3. a prefix for the `Name` instance 4. a string used in reports and error messages. Here, 1-3 are pretty similar to each other (they combine `InternalNtp` and `BoundaryNtp` down to one, and are generated by sled-agent's `ZoneType`). There is also a `ZoneType` in nexus-db-model that corresponds to `ZoneKind`. Based on this, I decided to combine it all down into one `ZoneKind` enum, with 4 functions returning 4 string representations: one each for 1-4. * There's no `Display` impl -- users must choose the representation they want, which I think is correct. * I've tried to carefully verify that there are **no behavior changes** introduced here. * An alternative would be to have a dedicated representation for 1-3 along with one for 4, but I tried it out and it seemed too unwieldy in practice. (For example, a few places want to use the zone prefix, but then report errors using the report string. Having it be one enum with four representations in string-land makes this easier. I do hope we won't ever need a fifth, though.) ### 3. Physical disk config `OmicronPhysicalDiskConfig` and `OmicronPhysicalDisksConfig` are shared types used by both Nexus and sled-agent. I moved them from `sled-storage` to `omicron-common`, next to `DiskIdentity` (which was already in `omicron-common` and which these types use). * Previously, `nexus-types` used the `sled-agent-client`-generated copy of these types -- but as discussed in the motivation section, that is no longer an option. * Why not `nexus-sled-agent-shared`? Well, `sled-storage` also needs these types, and I wanted to avoid sled-storage depending on the additional dependencies in `nexus-sled-agent-shared`. * I also used `replace` to patch them in so that there's just one copy of these types throughout sled-agent and Nexus. ### 4. Recovery silo types These are two types, `RecoverySiloConfig` and `Certificate`, that are primarily related to Nexus and that `sled-agent` was using via `nexus-client`. Users of these types needed to be moved into sled-agent-types, which (as described in the motivation section) cannot depend on nexus-client. So I've moved them into shared crates. * `Certificate` is a simple type that can go into `omicron-common`'s `api::internal::nexus`. * `RecoverySiloConfig` depends on `omicron-passwords`, so it goes in `nexus-sled-agent-shared`. We also patch these types in `nexus-client`. One **behavior change** as a result is that password hash strength is now validated on **both the client and the server.** I think this is okay, but it's worth mentioning explicitly. ### 5. `UserId` So this is a really interesting one. It needs to be moved into a shared location because `RecoverySiloConfig` depends on it. But during this process I noticed that we weren't doing any validation on it in clients like wicketd. So this counts as a **behavior change**: `UserId` is now validated on **both the client and the server**. Note that `UserId` is exposed by the Nexus external API. But this change itself has no impact on the external API (this can be seen by the fact that `nexus.json` doesn't have any changes in it) (I realized that the name should probably be something closer to `LocalUserId`, but that would be a breaking change to the external API so I've deferred it. Happy to change the name if it makes sense, though.) ### 6. Inventory types These are `Inventory`, `InventoryDisk` and `InventoryZpool`. I've moved them to `nexus-sled-agent-shared` as a replacement for the previous scheme where we were using client-generated versions of them. ### 7. `DiskVariant`, `SledRole`, other misc types These are simple types depended upon by others that have been moved into shared crates -- locations chosen mostly based on which code depends on them. ## OpenAPI changes There are some changes to our OpenAPI specifications. * Most of the changes are to `description` instances now that they're no longer going through Progenitor twice. * There are some validation changes, as documented above. ## Questions - [x] What should the name of the new crate be? - Resolved in 2024-07-23 meeting: **nexus-sled-agent-shared**. - [x] Verify that the behavior changes documented above are all reasonable. ## Other notes It would be nice to have tooling which asserts that e.g. none of the `-types` crates ever depend on `-client` crates. Haven't written this tooling though. Previously I did similar things using [guppy](https://github.com/guppy-rs/guppy), but that's only available as a library and can take a while to build (the total number of build units for `xtask` goes from 254 to 279, which is significant). Turning this into a binary would solve this but requires some work. --- Cargo.lock | 32 +- Cargo.toml | 13 +- clients/nexus-client/Cargo.toml | 1 + clients/nexus-client/src/lib.rs | 5 + clients/sled-agent-client/Cargo.toml | 1 + clients/sled-agent-client/src/lib.rs | 194 +------ common/src/api/external/mod.rs | 123 +++-- common/src/api/internal/nexus.rs | 15 + common/src/api/internal/shared.rs | 29 ++ common/src/disk.rs | 89 ++++ dev-tools/omdb/src/bin/omdb/db.rs | 6 +- dev-tools/reconfigurator-cli/Cargo.toml | 1 + dev-tools/reconfigurator-cli/src/main.rs | 6 +- end-to-end-tests/src/helpers/ctx.rs | 2 +- installinator/src/hardware.rs | 2 +- nexus-sled-agent-shared/Cargo.toml | 17 + nexus-sled-agent-shared/README.md | 58 +++ nexus-sled-agent-shared/src/inventory.rs | 464 +++++++++++++++++ nexus-sled-agent-shared/src/lib.rs | 11 + nexus-sled-agent-shared/src/recovery_silo.rs | 16 + nexus/Cargo.toml | 1 + nexus/db-model/Cargo.toml | 1 + nexus/db-model/src/dataset_kind.rs | 22 +- nexus/db-model/src/deployment.rs | 4 +- nexus/db-model/src/external_ip.rs | 4 +- nexus/db-model/src/inventory.rs | 75 ++- nexus/db-model/src/network_interface.rs | 27 +- nexus/db-model/src/omicron_zone_config.rs | 89 ++-- nexus/db-model/src/sled.rs | 5 +- nexus/db-queries/Cargo.toml | 1 + .../src/db/datastore/external_ip.rs | 2 +- .../db-queries/src/db/datastore/inventory.rs | 4 +- .../src/db/datastore/physical_disk.rs | 15 +- nexus/db-queries/src/db/datastore/rack.rs | 21 +- .../db-queries/src/db/queries/external_ip.rs | 2 +- nexus/inventory/Cargo.toml | 1 + nexus/inventory/src/builder.rs | 24 +- nexus/inventory/src/collector.rs | 16 +- nexus/inventory/src/examples.rs | 52 +- nexus/reconfigurator/execution/Cargo.toml | 1 + .../reconfigurator/execution/src/datasets.rs | 2 +- nexus/reconfigurator/execution/src/dns.rs | 2 +- .../execution/src/external_networking.rs | 39 +- .../execution/src/omicron_zones.rs | 8 +- nexus/reconfigurator/planning/Cargo.toml | 1 + .../planning/src/blueprint_builder/builder.rs | 8 +- nexus/reconfigurator/planning/src/planner.rs | 4 +- .../src/planner/omicron_zone_placement.rs | 4 +- nexus/reconfigurator/planning/src/system.rs | 42 +- .../background/tasks/blueprint_execution.rs | 2 +- .../tasks/crdb_node_id_collector.rs | 3 +- .../background/tasks/sync_service_zone_nat.rs | 24 +- nexus/src/app/sled.rs | 3 +- nexus/src/lib.rs | 4 +- nexus/test-interface/Cargo.toml | 1 + nexus/test-interface/src/lib.rs | 6 +- nexus/test-utils/Cargo.toml | 1 + nexus/test-utils/src/lib.rs | 12 +- nexus/test-utils/src/resource_helpers.rs | 2 +- nexus/tests/integration_tests/certificates.rs | 6 +- nexus/tests/integration_tests/endpoints.rs | 3 +- .../tests/integration_tests/password_login.rs | 14 +- nexus/tests/integration_tests/rack.rs | 2 +- nexus/tests/integration_tests/silos.rs | 10 +- nexus/types/Cargo.toml | 5 +- nexus/types/src/deployment.rs | 162 +----- nexus/types/src/deployment/blueprint_diff.rs | 12 +- nexus/types/src/deployment/zone_type.rs | 41 +- nexus/types/src/external_api/params.rs | 56 +- nexus/types/src/internal_api/params.rs | 70 +-- nexus/types/src/inventory.rs | 28 +- openapi/bootstrap-agent.json | 10 +- openapi/nexus-internal.json | 8 +- openapi/nexus.json | 4 +- openapi/sled-agent.json | 2 +- schema/rss-sled-plan.json | 10 +- sled-agent/Cargo.toml | 1 + sled-agent/src/bin/sled-agent-sim.rs | 33 +- sled-agent/src/dump_setup.rs | 2 +- sled-agent/src/http_entrypoints.rs | 8 +- sled-agent/src/nexus.rs | 27 +- sled-agent/src/params.rs | 490 ++---------------- sled-agent/src/rack_setup/plan/service.rs | 44 +- sled-agent/src/rack_setup/service.rs | 56 +- sled-agent/src/services.rs | 50 +- sled-agent/src/sim/http_entrypoints.rs | 6 +- sled-agent/src/sim/server.rs | 17 +- sled-agent/src/sim/sled_agent.rs | 16 +- sled-agent/src/sim/storage.rs | 4 +- sled-agent/src/sled_agent.rs | 20 +- sled-agent/types/Cargo.toml | 7 +- sled-agent/types/src/rack_init.rs | 11 +- sled-hardware/src/disk.rs | 32 +- sled-hardware/src/illumos/mod.rs | 6 +- sled-hardware/src/illumos/partitions.rs | 4 +- sled-hardware/src/non_illumos/mod.rs | 6 +- sled-storage/src/dataset.rs | 48 +- sled-storage/src/disk.rs | 66 +-- sled-storage/src/manager.rs | 11 +- sled-storage/src/manager_test_harness.rs | 7 +- sled-storage/src/resources.rs | 12 +- wicketd/src/rss_config.rs | 2 +- 102 files changed, 1525 insertions(+), 1524 deletions(-) create mode 100644 nexus-sled-agent-shared/Cargo.toml create mode 100644 nexus-sled-agent-shared/README.md create mode 100644 nexus-sled-agent-shared/src/inventory.rs create mode 100644 nexus-sled-agent-shared/src/lib.rs create mode 100644 nexus-sled-agent-shared/src/recovery_silo.rs diff --git a/Cargo.lock b/Cargo.lock index 7831b5701c..75f9b22c2a 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -4554,6 +4554,7 @@ version = "0.1.0" dependencies = [ "chrono", "futures", + "nexus-sled-agent-shared", "nexus-types", "omicron-common", "omicron-passwords", @@ -4623,6 +4624,7 @@ dependencies = [ "newtype_derive", "nexus-config", "nexus-defaults", + "nexus-sled-agent-shared", "nexus-types", "omicron-certificates", "omicron-common", @@ -4682,6 +4684,7 @@ dependencies = [ "nexus-db-model", "nexus-inventory", "nexus-reconfigurator-planning", + "nexus-sled-agent-shared", "nexus-test-utils", "nexus-types", "omicron-common", @@ -4766,6 +4769,7 @@ dependencies = [ "gateway-client", "gateway-messages", "gateway-test-utils", + "nexus-sled-agent-shared", "nexus-types", "omicron-common", "omicron-sled-agent", @@ -4857,6 +4861,7 @@ dependencies = [ "nexus-networking", "nexus-reconfigurator-planning", "nexus-reconfigurator-preparation", + "nexus-sled-agent-shared", "nexus-test-utils", "nexus-test-utils-macros", "nexus-types", @@ -4891,6 +4896,7 @@ dependencies = [ "maplit", "nexus-config", "nexus-inventory", + "nexus-sled-agent-shared", "nexus-types", "omicron-common", "omicron-test-utils", @@ -4951,12 +4957,27 @@ dependencies = [ "uuid", ] +[[package]] +name = "nexus-sled-agent-shared" +version = "0.1.0" +dependencies = [ + "omicron-common", + "omicron-passwords", + "omicron-uuid-kinds", + "omicron-workspace-hack", + "schemars", + "serde", + "sled-hardware-types", + "uuid", +] + [[package]] name = "nexus-test-interface" version = "0.1.0" dependencies = [ "async-trait", "nexus-config", + "nexus-sled-agent-shared", "nexus-types", "omicron-common", "omicron-workspace-hack", @@ -4987,6 +5008,7 @@ dependencies = [ "internal-dns", "nexus-config", "nexus-db-queries", + "nexus-sled-agent-shared", "nexus-test-interface", "nexus-types", "omicron-common", @@ -5036,6 +5058,7 @@ dependencies = [ "humantime", "ipnetwork", "newtype-uuid", + "nexus-sled-agent-shared", "omicron-common", "omicron-passwords", "omicron-uuid-kinds", @@ -5048,7 +5071,6 @@ dependencies = [ "serde", "serde_json", "serde_with", - "sled-agent-client", "slog", "slog-error-chain", "steno", @@ -5594,6 +5616,7 @@ dependencies = [ "nexus-reconfigurator-planning", "nexus-reconfigurator-preparation", "nexus-saga-recovery", + "nexus-sled-agent-shared", "nexus-test-interface", "nexus-test-utils", "nexus-test-utils-macros", @@ -5864,6 +5887,7 @@ dependencies = [ "mg-admin-client", "nexus-client", "nexus-config", + "nexus-sled-agent-shared", "nexus-types", "omicron-common", "omicron-ddm-admin-client", @@ -7748,6 +7772,7 @@ dependencies = [ "nexus-reconfigurator-execution", "nexus-reconfigurator-planning", "nexus-reconfigurator-preparation", + "nexus-sled-agent-shared", "nexus-test-utils", "nexus-test-utils-macros", "nexus-types", @@ -8940,6 +8965,7 @@ dependencies = [ "anyhow", "async-trait", "chrono", + "nexus-sled-agent-shared", "omicron-common", "omicron-uuid-kinds", "omicron-workspace-hack", @@ -8962,7 +8988,7 @@ dependencies = [ "bootstore", "camino", "camino-tempfile", - "nexus-client", + "nexus-sled-agent-shared", "omicron-common", "omicron-test-utils", "omicron-uuid-kinds", @@ -8973,9 +8999,11 @@ dependencies = [ "serde", "serde_json", "sled-hardware-types", + "sled-storage", "slog", "thiserror", "toml 0.8.15", + "uuid", ] [[package]] diff --git a/Cargo.toml b/Cargo.toml index 8cc8260f66..fddf0ab37d 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -43,6 +43,7 @@ members = [ "key-manager", "nexus", "nexus-config", + "nexus-sled-agent-shared", "nexus/authz-macros", "nexus/auth", "nexus/db-fixed-data", @@ -145,6 +146,7 @@ default-members = [ "key-manager", "nexus", "nexus-config", + "nexus-sled-agent-shared", "nexus/authz-macros", "nexus/auth", "nexus/db-fixed-data", @@ -377,29 +379,30 @@ nexus-reconfigurator-execution = { path = "nexus/reconfigurator/execution" } nexus-reconfigurator-planning = { path = "nexus/reconfigurator/planning" } nexus-reconfigurator-preparation = { path = "nexus/reconfigurator/preparation" } nexus-saga-recovery = { path = "nexus/saga-recovery" } -omicron-certificates = { path = "certificates" } -omicron-passwords = { path = "passwords" } -omicron-workspace-hack = "0.1.0" -oxlog = { path = "dev-tools/oxlog" } -oxnet = { git = "https://github.com/oxidecomputer/oxnet" } +nexus-sled-agent-shared = { path = "nexus-sled-agent-shared" } nexus-test-interface = { path = "nexus/test-interface" } nexus-test-utils-macros = { path = "nexus/test-utils-macros" } nexus-test-utils = { path = "nexus/test-utils" } nexus-types = { path = "nexus/types" } num-integer = "0.1.46" num = { version = "0.4.3", default-features = false, features = [ "libm" ] } +omicron-certificates = { path = "certificates" } omicron-cockroach-admin = { path = "cockroach-admin" } omicron-common = { path = "common" } omicron-gateway = { path = "gateway" } omicron-nexus = { path = "nexus" } omicron-omdb = { path = "dev-tools/omdb" } omicron-package = { path = "package" } +omicron-passwords = { path = "passwords" } omicron-rpaths = { path = "rpaths" } omicron-sled-agent = { path = "sled-agent" } omicron-test-utils = { path = "test-utils" } +omicron-workspace-hack = "0.1.0" omicron-zone-package = "0.11.0" oxide-client = { path = "clients/oxide-client" } oxide-vpc = { git = "https://github.com/oxidecomputer/opte", rev = "3dc9a3dd8d3c623f0cf2c659c7119ce0c026a96d", features = [ "api", "std" ] } +oxlog = { path = "dev-tools/oxlog" } +oxnet = { git = "https://github.com/oxidecomputer/oxnet" } once_cell = "1.19.0" openapi-lint = { git = "https://github.com/oxidecomputer/openapi-lint", branch = "main" } openapiv3 = "2.0.0" diff --git a/clients/nexus-client/Cargo.toml b/clients/nexus-client/Cargo.toml index 1b64fa24d1..bd4473d826 100644 --- a/clients/nexus-client/Cargo.toml +++ b/clients/nexus-client/Cargo.toml @@ -11,6 +11,7 @@ workspace = true chrono.workspace = true futures.workspace = true nexus-types.workspace = true +nexus-sled-agent-shared.workspace = true omicron-common.workspace = true omicron-passwords.workspace = true oxnet.workspace = true diff --git a/clients/nexus-client/src/lib.rs b/clients/nexus-client/src/lib.rs index fc64c48a63..162c3f4dbf 100644 --- a/clients/nexus-client/src/lib.rs +++ b/clients/nexus-client/src/lib.rs @@ -29,6 +29,8 @@ progenitor::generate_api!( // as "blueprint" this way, but we have really useful functionality // (e.g., diff'ing) that's implemented on our local type. Blueprint = nexus_types::deployment::Blueprint, + Certificate = omicron_common::api::internal::nexus::Certificate, + DatasetKind = omicron_common::api::internal::shared::DatasetKind, Generation = omicron_common::api::external::Generation, ImportExportPolicy = omicron_common::api::external::ImportExportPolicy, MacAddr = omicron_common::api::external::MacAddr, @@ -36,6 +38,9 @@ progenitor::generate_api!( NetworkInterface = omicron_common::api::internal::shared::NetworkInterface, NetworkInterfaceKind = omicron_common::api::internal::shared::NetworkInterfaceKind, NewPasswordHash = omicron_passwords::NewPasswordHash, + OmicronPhysicalDiskConfig = nexus_types::disk::OmicronPhysicalDiskConfig, + OmicronPhysicalDisksConfig = nexus_types::disk::OmicronPhysicalDisksConfig, + RecoverySiloConfig = nexus_sled_agent_shared::recovery_silo::RecoverySiloConfig, TypedUuidForCollectionKind = omicron_uuid_kinds::CollectionUuid, TypedUuidForDownstairsKind = omicron_uuid_kinds::TypedUuid, TypedUuidForPropolisKind = omicron_uuid_kinds::TypedUuid, diff --git a/clients/sled-agent-client/Cargo.toml b/clients/sled-agent-client/Cargo.toml index 11cc5adfd7..770588d6b4 100644 --- a/clients/sled-agent-client/Cargo.toml +++ b/clients/sled-agent-client/Cargo.toml @@ -11,6 +11,7 @@ workspace = true anyhow.workspace = true async-trait.workspace = true chrono.workspace = true +nexus-sled-agent-shared.workspace = true omicron-common.workspace = true omicron-uuid-kinds.workspace = true omicron-workspace-hack.workspace = true diff --git a/clients/sled-agent-client/src/lib.rs b/clients/sled-agent-client/src/lib.rs index 8a63cecd4f..4e7a4a72db 100644 --- a/clients/sled-agent-client/src/lib.rs +++ b/clients/sled-agent-client/src/lib.rs @@ -4,14 +4,8 @@ //! Interface for making API requests to a Sled Agent -use anyhow::Context; use async_trait::async_trait; -use omicron_common::api::internal::shared::NetworkInterface; use std::convert::TryFrom; -use std::fmt; -use std::hash::Hash; -use std::net::IpAddr; -use std::net::SocketAddr; use uuid::Uuid; progenitor::generate_api!( @@ -42,13 +36,24 @@ progenitor::generate_api!( "oxnet" = "0.1.0", }, replace = { + Baseboard = nexus_sled_agent_shared::inventory::Baseboard, ByteCount = omicron_common::api::external::ByteCount, DiskIdentity = omicron_common::disk::DiskIdentity, + DiskVariant = omicron_common::disk::DiskVariant, Generation = omicron_common::api::external::Generation, ImportExportPolicy = omicron_common::api::external::ImportExportPolicy, + Inventory = nexus_sled_agent_shared::inventory::Inventory, + InventoryDisk = nexus_sled_agent_shared::inventory::InventoryDisk, + InventoryZpool = nexus_sled_agent_shared::inventory::InventoryZpool, MacAddr = omicron_common::api::external::MacAddr, Name = omicron_common::api::external::Name, NetworkInterface = omicron_common::api::internal::shared::NetworkInterface, + OmicronPhysicalDiskConfig = omicron_common::disk::OmicronPhysicalDiskConfig, + OmicronPhysicalDisksConfig = omicron_common::disk::OmicronPhysicalDisksConfig, + OmicronZoneConfig = nexus_sled_agent_shared::inventory::OmicronZoneConfig, + OmicronZoneDataset = nexus_sled_agent_shared::inventory::OmicronZoneDataset, + OmicronZoneType = nexus_sled_agent_shared::inventory::OmicronZoneType, + OmicronZonesConfig = nexus_sled_agent_shared::inventory::OmicronZonesConfig, PortFec = omicron_common::api::internal::shared::PortFec, PortSpeed = omicron_common::api::internal::shared::PortSpeed, RouterId = omicron_common::api::internal::shared::RouterId, @@ -56,6 +61,7 @@ progenitor::generate_api!( ResolvedVpcRouteSet = omicron_common::api::internal::shared::ResolvedVpcRouteSet, RouterTarget = omicron_common::api::internal::shared::RouterTarget, RouterVersion = omicron_common::api::internal::shared::RouterVersion, + SledRole = nexus_sled_agent_shared::inventory::SledRole, SourceNatConfig = omicron_common::api::internal::shared::SourceNatConfig, SwitchLocation = omicron_common::api::external::SwitchLocation, TypedUuidForInstanceKind = omicron_uuid_kinds::InstanceUuid, @@ -67,182 +73,6 @@ progenitor::generate_api!( } ); -// We cannot easily configure progenitor to derive `Eq` on all the client- -// generated types because some have floats and other types that can't impl -// `Eq`. We impl it explicitly for a few types on which we need it. -impl Eq for types::OmicronPhysicalDisksConfig {} -impl Eq for types::OmicronZonesConfig {} -impl Eq for types::OmicronZoneConfig {} -impl Eq for types::OmicronZoneType {} -impl Eq for types::OmicronZoneDataset {} - -/// Like [`types::OmicronZoneType`], but without any associated data. -/// -/// We have a few enums of this form floating around. This particular one is -/// meant to correspond exactly 1:1 with `OmicronZoneType`. -/// -/// The [`fmt::Display`] impl for this type is a human-readable label, meant -/// for testing and reporting. -#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, PartialOrd, Ord)] -pub enum ZoneKind { - BoundaryNtp, - Clickhouse, - ClickhouseKeeper, - CockroachDb, - Crucible, - CruciblePantry, - ExternalDns, - InternalDns, - InternalNtp, - Nexus, - Oximeter, -} - -impl fmt::Display for ZoneKind { - fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { - match self { - ZoneKind::BoundaryNtp => write!(f, "boundary_ntp"), - ZoneKind::Clickhouse => write!(f, "clickhouse"), - ZoneKind::ClickhouseKeeper => write!(f, "clickhouse_keeper"), - ZoneKind::CockroachDb => write!(f, "cockroach_db"), - ZoneKind::Crucible => write!(f, "crucible"), - ZoneKind::CruciblePantry => write!(f, "crucible_pantry"), - ZoneKind::ExternalDns => write!(f, "external_dns"), - ZoneKind::InternalDns => write!(f, "internal_dns"), - ZoneKind::InternalNtp => write!(f, "internal_ntp"), - ZoneKind::Nexus => write!(f, "nexus"), - ZoneKind::Oximeter => write!(f, "oximeter"), - } - } -} - -impl types::OmicronZoneType { - /// Returns the [`ZoneKind`] corresponding to this variant. - pub fn kind(&self) -> ZoneKind { - match self { - types::OmicronZoneType::BoundaryNtp { .. } => ZoneKind::BoundaryNtp, - types::OmicronZoneType::Clickhouse { .. } => ZoneKind::Clickhouse, - types::OmicronZoneType::ClickhouseKeeper { .. } => { - ZoneKind::ClickhouseKeeper - } - types::OmicronZoneType::CockroachDb { .. } => ZoneKind::CockroachDb, - types::OmicronZoneType::Crucible { .. } => ZoneKind::Crucible, - types::OmicronZoneType::CruciblePantry { .. } => { - ZoneKind::CruciblePantry - } - types::OmicronZoneType::ExternalDns { .. } => ZoneKind::ExternalDns, - types::OmicronZoneType::InternalDns { .. } => ZoneKind::InternalDns, - types::OmicronZoneType::InternalNtp { .. } => ZoneKind::InternalNtp, - types::OmicronZoneType::Nexus { .. } => ZoneKind::Nexus, - types::OmicronZoneType::Oximeter { .. } => ZoneKind::Oximeter, - } - } - - /// Identifies whether this is an NTP zone - pub fn is_ntp(&self) -> bool { - match self { - types::OmicronZoneType::BoundaryNtp { .. } - | types::OmicronZoneType::InternalNtp { .. } => true, - - types::OmicronZoneType::Clickhouse { .. } - | types::OmicronZoneType::ClickhouseKeeper { .. } - | types::OmicronZoneType::CockroachDb { .. } - | types::OmicronZoneType::Crucible { .. } - | types::OmicronZoneType::CruciblePantry { .. } - | types::OmicronZoneType::ExternalDns { .. } - | types::OmicronZoneType::InternalDns { .. } - | types::OmicronZoneType::Nexus { .. } - | types::OmicronZoneType::Oximeter { .. } => false, - } - } - - /// Identifies whether this is a Nexus zone - pub fn is_nexus(&self) -> bool { - match self { - types::OmicronZoneType::Nexus { .. } => true, - - types::OmicronZoneType::BoundaryNtp { .. } - | types::OmicronZoneType::InternalNtp { .. } - | types::OmicronZoneType::Clickhouse { .. } - | types::OmicronZoneType::ClickhouseKeeper { .. } - | types::OmicronZoneType::CockroachDb { .. } - | types::OmicronZoneType::Crucible { .. } - | types::OmicronZoneType::CruciblePantry { .. } - | types::OmicronZoneType::ExternalDns { .. } - | types::OmicronZoneType::InternalDns { .. } - | types::OmicronZoneType::Oximeter { .. } => false, - } - } - - /// Identifies whether this a Crucible (not Crucible pantry) zone - pub fn is_crucible(&self) -> bool { - match self { - types::OmicronZoneType::Crucible { .. } => true, - - types::OmicronZoneType::BoundaryNtp { .. } - | types::OmicronZoneType::InternalNtp { .. } - | types::OmicronZoneType::Clickhouse { .. } - | types::OmicronZoneType::ClickhouseKeeper { .. } - | types::OmicronZoneType::CockroachDb { .. } - | types::OmicronZoneType::CruciblePantry { .. } - | types::OmicronZoneType::ExternalDns { .. } - | types::OmicronZoneType::InternalDns { .. } - | types::OmicronZoneType::Nexus { .. } - | types::OmicronZoneType::Oximeter { .. } => false, - } - } - - /// This zone's external IP - pub fn external_ip(&self) -> anyhow::Result> { - match self { - types::OmicronZoneType::Nexus { external_ip, .. } => { - Ok(Some(*external_ip)) - } - - types::OmicronZoneType::ExternalDns { dns_address, .. } => { - let dns_address = - dns_address.parse::().with_context(|| { - format!( - "failed to parse ExternalDns address {dns_address}" - ) - })?; - Ok(Some(dns_address.ip())) - } - - types::OmicronZoneType::BoundaryNtp { snat_cfg, .. } => { - Ok(Some(snat_cfg.ip)) - } - - types::OmicronZoneType::InternalNtp { .. } - | types::OmicronZoneType::Clickhouse { .. } - | types::OmicronZoneType::ClickhouseKeeper { .. } - | types::OmicronZoneType::CockroachDb { .. } - | types::OmicronZoneType::Crucible { .. } - | types::OmicronZoneType::CruciblePantry { .. } - | types::OmicronZoneType::InternalDns { .. } - | types::OmicronZoneType::Oximeter { .. } => Ok(None), - } - } - - /// The service vNIC providing external connectivity to this zone - pub fn service_vnic(&self) -> Option<&NetworkInterface> { - match self { - types::OmicronZoneType::Nexus { nic, .. } - | types::OmicronZoneType::ExternalDns { nic, .. } - | types::OmicronZoneType::BoundaryNtp { nic, .. } => Some(nic), - - types::OmicronZoneType::InternalNtp { .. } - | types::OmicronZoneType::Clickhouse { .. } - | types::OmicronZoneType::ClickhouseKeeper { .. } - | types::OmicronZoneType::CockroachDb { .. } - | types::OmicronZoneType::Crucible { .. } - | types::OmicronZoneType::CruciblePantry { .. } - | types::OmicronZoneType::InternalDns { .. } - | types::OmicronZoneType::Oximeter { .. } => None, - } - } -} - impl omicron_common::api::external::ClientError for types::Error { fn message(&self) -> String { self.message.clone() diff --git a/common/src/api/external/mod.rs b/common/src/api/external/mod.rs index 2397cd15f8..e7d4c2f899 100644 --- a/common/src/api/external/mod.rs +++ b/common/src/api/external/mod.rs @@ -296,38 +296,20 @@ impl JsonSchema for Name { fn json_schema( _: &mut schemars::gen::SchemaGenerator, ) -> schemars::schema::Schema { - schemars::schema::SchemaObject { - metadata: Some(Box::new(schemars::schema::Metadata { - title: Some( - "A name unique within the parent collection".to_string(), - ), - description: Some( - "Names must begin with a lower case ASCII letter, be \ - composed exclusively of lowercase ASCII, uppercase \ - ASCII, numbers, and '-', and may not end with a '-'. \ - Names cannot be a UUID, but they may contain a UUID. \ - They can be at most 63 characters long.".to_string(), - ), - ..Default::default() - })), - instance_type: Some(schemars::schema::InstanceType::String.into()), - string: Some(Box::new(schemars::schema::StringValidation { - max_length: Some(63), - min_length: Some(1), - pattern: Some( - concat!( - r#"^"#, - // Cannot match a UUID - r#"(?![0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$)"#, - r#"^[a-z]([a-zA-Z0-9-]*[a-zA-Z0-9]+)?"#, - r#"$"#, - ) + name_schema(schemars::schema::Metadata { + title: Some( + "A name unique within the parent collection".to_string(), + ), + description: Some( + "Names must begin with a lower case ASCII letter, be \ + composed exclusively of lowercase ASCII, uppercase \ + ASCII, numbers, and '-', and may not end with a '-'. \ + Names cannot be a UUID, but they may contain a UUID. \ + They can be at most 63 characters long." .to_string(), - ) - })), + ), ..Default::default() - } - .into() + }) } } @@ -399,6 +381,87 @@ impl JsonSchema for NameOrId { } } +/// A username for a local-only user. +#[derive(Clone, Debug, PartialEq, Eq, Deserialize, Serialize)] +#[serde(try_from = "String")] +pub struct UserId(String); + +impl AsRef for UserId { + fn as_ref(&self) -> &str { + self.0.as_ref() + } +} + +impl FromStr for UserId { + type Err = String; + fn from_str(value: &str) -> Result { + UserId::try_from(String::from(value)) + } +} + +/// Used to impl `Deserialize` +impl TryFrom for UserId { + type Error = String; + fn try_from(value: String) -> Result { + // Mostly, this validation exists to cap the input size. The specific + // length is not critical here. For convenience and consistency, we use + // the same rules as `Name`. + let _ = Name::try_from(value.clone())?; + Ok(UserId(value)) + } +} + +impl JsonSchema for UserId { + fn schema_name() -> String { + "UserId".to_string() + } + + fn json_schema( + _: &mut schemars::gen::SchemaGenerator, + ) -> schemars::schema::Schema { + name_schema(schemars::schema::Metadata { + title: Some("A username for a local-only user".to_string()), + description: Some( + "Usernames must begin with a lower case ASCII letter, be \ + composed exclusively of lowercase ASCII, uppercase ASCII, \ + numbers, and '-', and may not end with a '-'. Usernames \ + cannot be a UUID, but they may contain a UUID. They can be at \ + most 63 characters long." + .to_string(), + ), + ..Default::default() + }) + } +} + +fn name_schema( + metadata: schemars::schema::Metadata, +) -> schemars::schema::Schema { + schemars::schema::SchemaObject { + metadata: Some(Box::new(metadata)), + instance_type: Some(schemars::schema::InstanceType::String.into()), + string: Some(Box::new(schemars::schema::StringValidation { + max_length: Some(63), + min_length: Some(1), + pattern: Some( + concat!( + r#"^"#, + // Cannot match a UUID + concat!( + r#"(?![0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}"#, + r#"-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$)"#, + ), + r#"^[a-z]([a-zA-Z0-9-]*[a-zA-Z0-9]+)?"#, + r#"$"#, + ) + .to_string(), + ), + })), + ..Default::default() + } + .into() +} + // TODO: remove wrapper for semver::Version once this PR goes through // https://github.com/GREsau/schemars/pull/195 #[derive( diff --git a/common/src/api/internal/nexus.rs b/common/src/api/internal/nexus.rs index d82d9a980a..d4ed1773f6 100644 --- a/common/src/api/internal/nexus.rs +++ b/common/src/api/internal/nexus.rs @@ -23,6 +23,21 @@ use std::time::Duration; use strum::{EnumIter, IntoEnumIterator}; use uuid::Uuid; +#[derive(Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)] +pub struct Certificate { + pub cert: String, + pub key: String, +} + +impl std::fmt::Debug for Certificate { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("Certificate") + .field("cert", &self.cert) + .field("key", &"") + .finish() + } +} + /// Runtime state of the Disk, which includes its attach state and some minimal /// metadata #[derive(Clone, Debug, Deserialize, Serialize, JsonSchema)] diff --git a/common/src/api/internal/shared.rs b/common/src/api/internal/shared.rs index 24bb339112..e457d08fb2 100644 --- a/common/src/api/internal/shared.rs +++ b/common/src/api/internal/shared.rs @@ -702,6 +702,35 @@ pub struct ResolvedVpcRouteSet { pub routes: HashSet, } +/// Describes the purpose of the dataset. +#[derive( + Debug, Serialize, Deserialize, JsonSchema, Clone, Copy, PartialEq, Eq, +)] +#[serde(rename_all = "snake_case")] +pub enum DatasetKind { + Crucible, + Cockroach, + Clickhouse, + ClickhouseKeeper, + ExternalDns, + InternalDns, +} + +impl fmt::Display for DatasetKind { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + use DatasetKind::*; + let s = match self { + Crucible => "crucible", + Cockroach => "cockroach", + Clickhouse => "clickhouse", + ClickhouseKeeper => "clickhouse_keeper", + ExternalDns => "external_dns", + InternalDns => "internal_dns", + }; + write!(f, "{}", s) + } +} + /// Identifiers for a single sled. /// /// This is intended primarily to be used in timeseries, to identify diff --git a/common/src/disk.rs b/common/src/disk.rs index c6d60c5140..4b4cd2e69d 100644 --- a/common/src/disk.rs +++ b/common/src/disk.rs @@ -4,8 +4,70 @@ //! Disk related types shared among crates +use omicron_uuid_kinds::ZpoolUuid; use schemars::JsonSchema; use serde::{Deserialize, Serialize}; +use uuid::Uuid; + +use crate::{ + api::external::Generation, ledger::Ledgerable, zpool_name::ZpoolKind, +}; + +#[derive( + Clone, + Debug, + Deserialize, + Serialize, + JsonSchema, + PartialEq, + Eq, + Hash, + PartialOrd, + Ord, +)] +pub struct OmicronPhysicalDiskConfig { + pub identity: DiskIdentity, + pub id: Uuid, + pub pool_id: ZpoolUuid, +} + +#[derive( + Clone, Debug, Deserialize, Serialize, JsonSchema, PartialEq, Eq, Hash, +)] +pub struct OmicronPhysicalDisksConfig { + /// generation number of this configuration + /// + /// This generation number is owned by the control plane (i.e., RSS or + /// Nexus, depending on whether RSS-to-Nexus handoff has happened). It + /// should not be bumped within Sled Agent. + /// + /// Sled Agent rejects attempts to set the configuration to a generation + /// older than the one it's currently running. + pub generation: Generation, + + pub disks: Vec, +} + +impl Default for OmicronPhysicalDisksConfig { + fn default() -> Self { + Self { generation: Generation::new(), disks: vec![] } + } +} + +impl Ledgerable for OmicronPhysicalDisksConfig { + fn is_newer_than(&self, other: &OmicronPhysicalDisksConfig) -> bool { + self.generation > other.generation + } + + // No need to do this, the generation number is provided externally. + fn generation_bump(&mut self) {} +} + +impl OmicronPhysicalDisksConfig { + pub fn new() -> Self { + Self { generation: Generation::new(), disks: vec![] } + } +} /// Uniquely identifies a disk. #[derive( @@ -25,3 +87,30 @@ pub struct DiskIdentity { pub model: String, pub serial: String, } + +#[derive( + Debug, + Clone, + Copy, + PartialEq, + Eq, + Hash, + Serialize, + Deserialize, + JsonSchema, + Ord, + PartialOrd, +)] +pub enum DiskVariant { + U2, + M2, +} + +impl From for DiskVariant { + fn from(kind: ZpoolKind) -> DiskVariant { + match kind { + ZpoolKind::External => DiskVariant::U2, + ZpoolKind::Internal => DiskVariant::M2, + } + } +} diff --git a/dev-tools/omdb/src/bin/omdb/db.rs b/dev-tools/omdb/src/bin/omdb/db.rs index 98669ddc06..78c68d8121 100644 --- a/dev-tools/omdb/src/bin/omdb/db.rs +++ b/dev-tools/omdb/src/bin/omdb/db.rs @@ -3737,7 +3737,11 @@ fn inv_collection_print_sleds(collection: &Collection) { println!(" ZONES FOUND"); for z in &zones.zones.zones { - println!(" zone {} (type {})", z.id, z.zone_type.kind()); + println!( + " zone {} (type {})", + z.id, + z.zone_type.kind().report_str() + ); } } else { println!(" warning: no zone information found"); diff --git a/dev-tools/reconfigurator-cli/Cargo.toml b/dev-tools/reconfigurator-cli/Cargo.toml index 5edf9d0ef8..f6b225a5c9 100644 --- a/dev-tools/reconfigurator-cli/Cargo.toml +++ b/dev-tools/reconfigurator-cli/Cargo.toml @@ -21,6 +21,7 @@ humantime.workspace = true indexmap.workspace = true nexus-reconfigurator-planning.workspace = true nexus-reconfigurator-execution.workspace = true +nexus-sled-agent-shared.workspace = true nexus-types.workspace = true omicron-common.workspace = true omicron-uuid-kinds.workspace = true diff --git a/dev-tools/reconfigurator-cli/src/main.rs b/dev-tools/reconfigurator-cli/src/main.rs index 3234a3fdc2..983dde412d 100644 --- a/dev-tools/reconfigurator-cli/src/main.rs +++ b/dev-tools/reconfigurator-cli/src/main.rs @@ -20,16 +20,16 @@ use nexus_reconfigurator_planning::planner::Planner; use nexus_reconfigurator_planning::system::{ SledBuilder, SledHwInventory, SystemDescription, }; +use nexus_sled_agent_shared::inventory::OmicronZonesConfig; +use nexus_sled_agent_shared::inventory::SledRole; +use nexus_sled_agent_shared::inventory::ZoneKind; use nexus_types::deployment::BlueprintZoneFilter; use nexus_types::deployment::OmicronZoneNic; use nexus_types::deployment::PlanningInput; use nexus_types::deployment::SledFilter; -use nexus_types::deployment::ZoneKind; use nexus_types::deployment::{Blueprint, UnstableReconfiguratorState}; use nexus_types::internal_api::params::DnsConfigParams; use nexus_types::inventory::Collection; -use nexus_types::inventory::OmicronZonesConfig; -use nexus_types::inventory::SledRole; use omicron_common::api::external::Generation; use omicron_common::api::external::Name; use omicron_uuid_kinds::CollectionUuid; diff --git a/end-to-end-tests/src/helpers/ctx.rs b/end-to-end-tests/src/helpers/ctx.rs index 76b759608c..d9a2d7027a 100644 --- a/end-to-end-tests/src/helpers/ctx.rs +++ b/end-to-end-tests/src/helpers/ctx.rs @@ -224,7 +224,7 @@ impl ClientParams { let silo_name = config.recovery_silo.silo_name.as_str(); let login_url = format!("{}/v1/login/{}/local", base_url, silo_name); let username: oxide_client::types::UserId = - config.recovery_silo.user_name.as_str().parse().map_err(|s| { + config.recovery_silo.user_name.as_ref().parse().map_err(|s| { anyhow!("parsing configured recovery user name: {:?}", s) })?; // See the comment in the config file about this password. diff --git a/installinator/src/hardware.rs b/installinator/src/hardware.rs index cc71cea5ee..af2ecd2dff 100644 --- a/installinator/src/hardware.rs +++ b/installinator/src/hardware.rs @@ -6,7 +6,7 @@ use anyhow::anyhow; use anyhow::ensure; use anyhow::Context; use anyhow::Result; -use sled_hardware::DiskVariant; +use omicron_common::disk::DiskVariant; use sled_hardware::HardwareManager; use sled_hardware::SledMode; use sled_storage::config::MountConfig; diff --git a/nexus-sled-agent-shared/Cargo.toml b/nexus-sled-agent-shared/Cargo.toml new file mode 100644 index 0000000000..8e2358e902 --- /dev/null +++ b/nexus-sled-agent-shared/Cargo.toml @@ -0,0 +1,17 @@ +[package] +name = "nexus-sled-agent-shared" +version = "0.1.0" +edition = "2021" + +[lints] +workspace = true + +[dependencies] +omicron-common.workspace = true +omicron-passwords.workspace = true +omicron-uuid-kinds.workspace = true +omicron-workspace-hack.workspace = true +schemars.workspace = true +serde.workspace = true +sled-hardware-types.workspace = true +uuid.workspace = true diff --git a/nexus-sled-agent-shared/README.md b/nexus-sled-agent-shared/README.md new file mode 100644 index 0000000000..eeb3492eea --- /dev/null +++ b/nexus-sled-agent-shared/README.md @@ -0,0 +1,58 @@ +# nexus-sled-agent-shared + +Internal types shared between Nexus and sled-agent, with extra dependencies not +in omicron-common. + +**This crate should only be used for internal types and data structures.** + +## Why not omicron-common? + +omicron-common is used by several workspaces, so adding a dependency to it has +a large impact. This crate was motivated by the need to be a place to put types +that have dependencies not already in omicron-common. + +For example, with this crate, `omicron-common` avoids a dependency on +omicron-passwords, which pulls in `argon2`. + +An alternative would be to add a non-default feature to omicron-common to +include these types. But that would result in many extra, unnecessary rebuilds +-- both omicron-common and many of its dependents would need to be built twice, +once with the feature and once without. A separate crate avoids that. + +## Why not nexus-config? + +`nexus-config` is a similar crate that was split out of `omicron-common` for +dependency reasons. However, `nexus-config` depends on the rather heavyweight +tokio-postgres, a dependency that is not a necessary component of sled-agent. + +## Why not sled-agent-types or nexus-types? + +Types that are primarily used by sled-agent or nexus should continue to go in +those crates. However, types shared by both should go here. `sled-agent-types` +and `nexus-types` can thus avoid a dependency on each other: they're both "on +the same level" and neither dependency direction is clearly correct. + +## Why not Progenitor-generated types? + +Many of our types are actually generated by Progenitor within the +`sled-agent-client` and `nexus-client` crates. However, at least some of them +inconvenient to deal with, such as `OmicronZonesConfig` -- particularly the +fact that it stored IP addresses as strings and not as structured data. Before +making this change, there were a number of spots that had to deal with the +generated type and had to account for a string being invalid. + +Now, though, those types are replaced with the copy in this crate. + +Another issue this crate solves is circular dependencies. Thinking about the +organization in terms of layers, there's the `-types` layer and the `-client` +layer. Now, `sled-agent-client` uses `replace` to substitute the types in +`sled-agent-types` with its own. So logically, the `-client` layer is expected +to depend on the `-types` layer. But sled-agent makes API calls to Nexus, so +previously `sled-agent-types` depended on `nexus-client`, and similarly, +`nexus-types` depended on `sled-agent-client`. If this crate didn't exist, +then there would be a circular dependency: + +`sled-agent-client` -> `sled-agent-types` -> `nexus-client` -> `nexus-types` -> `sled-agent-client` + +This crate breaks that cycle by providing a place for shared types, and no +longer making either `-types` depend on either `-client`. diff --git a/nexus-sled-agent-shared/src/inventory.rs b/nexus-sled-agent-shared/src/inventory.rs new file mode 100644 index 0000000000..8a793f6150 --- /dev/null +++ b/nexus-sled-agent-shared/src/inventory.rs @@ -0,0 +1,464 @@ +// This Source Code Form is subject to the terms of the Mozilla Public +// License, v. 2.0. If a copy of the MPL was not distributed with this +// file, You can obtain one at https://mozilla.org/MPL/2.0/. + +//! Inventory types shared between Nexus and sled-agent. + +use std::net::{IpAddr, Ipv6Addr, SocketAddr, SocketAddrV6}; + +use omicron_common::{ + api::{ + external::{ByteCount, Generation}, + internal::shared::{NetworkInterface, SourceNatConfig}, + }, + disk::DiskVariant, + zpool_name::ZpoolName, +}; +use omicron_uuid_kinds::ZpoolUuid; +use schemars::JsonSchema; +use serde::{Deserialize, Serialize}; +// Export this type for convenience -- this way, dependents don't have to +// depend on sled-hardware-types. +pub use sled_hardware_types::Baseboard; +use uuid::Uuid; + +/// Identifies information about disks which may be attached to Sleds. +#[derive(Clone, Debug, Deserialize, JsonSchema, Serialize)] +pub struct InventoryDisk { + pub identity: omicron_common::disk::DiskIdentity, + pub variant: DiskVariant, + pub slot: i64, +} + +/// Identifies information about zpools managed by the control plane +#[derive(Clone, Debug, Deserialize, JsonSchema, Serialize)] +pub struct InventoryZpool { + pub id: ZpoolUuid, + pub total_size: ByteCount, +} + +/// Identity and basic status information about this sled agent +#[derive(Clone, Debug, Deserialize, JsonSchema, Serialize)] +pub struct Inventory { + pub sled_id: Uuid, + pub sled_agent_address: SocketAddrV6, + pub sled_role: SledRole, + pub baseboard: Baseboard, + pub usable_hardware_threads: u32, + pub usable_physical_ram: ByteCount, + pub reservoir_size: ByteCount, + pub disks: Vec, + pub zpools: Vec, +} + +/// Describes the role of the sled within the rack. +/// +/// Note that this may change if the sled is physically moved +/// within the rack. +#[derive( + Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize, JsonSchema, +)] +#[serde(rename_all = "snake_case")] +pub enum SledRole { + /// The sled is a general compute sled. + Gimlet, + /// The sled is attached to the network switch, and has additional + /// responsibilities. + Scrimlet, +} + +/// Describes the set of Omicron-managed zones running on a sled +#[derive( + Clone, Debug, Deserialize, Serialize, JsonSchema, PartialEq, Eq, Hash, +)] +pub struct OmicronZonesConfig { + /// generation number of this configuration + /// + /// This generation number is owned by the control plane (i.e., RSS or + /// Nexus, depending on whether RSS-to-Nexus handoff has happened). It + /// should not be bumped within Sled Agent. + /// + /// Sled Agent rejects attempts to set the configuration to a generation + /// older than the one it's currently running. + pub generation: Generation, + + /// list of running zones + pub zones: Vec, +} + +impl OmicronZonesConfig { + /// Generation 1 of `OmicronZonesConfig` is always the set of no zones. + pub const INITIAL_GENERATION: Generation = Generation::from_u32(1); +} + +/// Describes one Omicron-managed zone running on a sled +#[derive( + Clone, Debug, Deserialize, Serialize, JsonSchema, PartialEq, Eq, Hash, +)] +pub struct OmicronZoneConfig { + pub id: Uuid, + pub underlay_address: Ipv6Addr, + + /// The pool on which we'll place this zone's filesystem. + /// + /// Note that this is transient -- the sled agent is permitted to + /// destroy the zone's dataset on this pool each time the zone is + /// initialized. + pub filesystem_pool: Option, + pub zone_type: OmicronZoneType, +} + +/// Describes a persistent ZFS dataset associated with an Omicron zone +#[derive( + Clone, Debug, Deserialize, Serialize, JsonSchema, PartialEq, Eq, Hash, +)] +pub struct OmicronZoneDataset { + pub pool_name: ZpoolName, +} + +/// Describes what kind of zone this is (i.e., what component is running in it) +/// as well as any type-specific configuration +#[derive( + Clone, Debug, Deserialize, Serialize, JsonSchema, PartialEq, Eq, Hash, +)] +#[serde(tag = "type", rename_all = "snake_case")] +pub enum OmicronZoneType { + BoundaryNtp { + address: SocketAddrV6, + ntp_servers: Vec, + dns_servers: Vec, + domain: Option, + /// The service vNIC providing outbound connectivity using OPTE. + nic: NetworkInterface, + /// The SNAT configuration for outbound connections. + snat_cfg: SourceNatConfig, + }, + + Clickhouse { + address: SocketAddrV6, + dataset: OmicronZoneDataset, + }, + + ClickhouseKeeper { + address: SocketAddrV6, + dataset: OmicronZoneDataset, + }, + CockroachDb { + address: SocketAddrV6, + dataset: OmicronZoneDataset, + }, + + Crucible { + address: SocketAddrV6, + dataset: OmicronZoneDataset, + }, + CruciblePantry { + address: SocketAddrV6, + }, + ExternalDns { + dataset: OmicronZoneDataset, + /// The address at which the external DNS server API is reachable. + http_address: SocketAddrV6, + /// The address at which the external DNS server is reachable. + dns_address: SocketAddr, + /// The service vNIC providing external connectivity using OPTE. + nic: NetworkInterface, + }, + InternalDns { + dataset: OmicronZoneDataset, + http_address: SocketAddrV6, + dns_address: SocketAddrV6, + /// The addresses in the global zone which should be created + /// + /// For the DNS service, which exists outside the sleds's typical subnet + /// - adding an address in the GZ is necessary to allow inter-zone + /// traffic routing. + gz_address: Ipv6Addr, + + /// The address is also identified with an auxiliary bit of information + /// to ensure that the created global zone address can have a unique + /// name. + gz_address_index: u32, + }, + InternalNtp { + address: SocketAddrV6, + ntp_servers: Vec, + dns_servers: Vec, + domain: Option, + }, + Nexus { + /// The address at which the internal nexus server is reachable. + internal_address: SocketAddrV6, + /// The address at which the external nexus server is reachable. + external_ip: IpAddr, + /// The service vNIC providing external connectivity using OPTE. + nic: NetworkInterface, + /// Whether Nexus's external endpoint should use TLS + external_tls: bool, + /// External DNS servers Nexus can use to resolve external hosts. + external_dns_servers: Vec, + }, + Oximeter { + address: SocketAddrV6, + }, +} + +impl OmicronZoneType { + /// Returns the [`ZoneKind`] corresponding to this variant. + pub fn kind(&self) -> ZoneKind { + match self { + OmicronZoneType::BoundaryNtp { .. } => ZoneKind::BoundaryNtp, + OmicronZoneType::Clickhouse { .. } => ZoneKind::Clickhouse, + OmicronZoneType::ClickhouseKeeper { .. } => { + ZoneKind::ClickhouseKeeper + } + OmicronZoneType::CockroachDb { .. } => ZoneKind::CockroachDb, + OmicronZoneType::Crucible { .. } => ZoneKind::Crucible, + OmicronZoneType::CruciblePantry { .. } => ZoneKind::CruciblePantry, + OmicronZoneType::ExternalDns { .. } => ZoneKind::ExternalDns, + OmicronZoneType::InternalDns { .. } => ZoneKind::InternalDns, + OmicronZoneType::InternalNtp { .. } => ZoneKind::InternalNtp, + OmicronZoneType::Nexus { .. } => ZoneKind::Nexus, + OmicronZoneType::Oximeter { .. } => ZoneKind::Oximeter, + } + } + + /// Does this zone require time synchronization before it is initialized?" + /// + /// This function is somewhat conservative - the set of services + /// that can be launched before timesync has completed is intentionally kept + /// small, since it would be easy to add a service that expects time to be + /// reasonably synchronized. + pub fn requires_timesync(&self) -> bool { + match self { + // These zones can be initialized and started before time has been + // synchronized. For the NTP zones, this should be self-evident -- + // we need the NTP zone to actually perform time synchronization! + // + // The DNS zone is a bit of an exception here, since the NTP zone + // itself may rely on DNS lookups as a dependency. + OmicronZoneType::BoundaryNtp { .. } + | OmicronZoneType::InternalNtp { .. } + | OmicronZoneType::InternalDns { .. } => false, + _ => true, + } + } + + /// Identifies whether this is an NTP zone + pub fn is_ntp(&self) -> bool { + match self { + OmicronZoneType::BoundaryNtp { .. } + | OmicronZoneType::InternalNtp { .. } => true, + + OmicronZoneType::Clickhouse { .. } + | OmicronZoneType::ClickhouseKeeper { .. } + | OmicronZoneType::CockroachDb { .. } + | OmicronZoneType::Crucible { .. } + | OmicronZoneType::CruciblePantry { .. } + | OmicronZoneType::ExternalDns { .. } + | OmicronZoneType::InternalDns { .. } + | OmicronZoneType::Nexus { .. } + | OmicronZoneType::Oximeter { .. } => false, + } + } + + /// Identifies whether this is a Nexus zone + pub fn is_nexus(&self) -> bool { + match self { + OmicronZoneType::Nexus { .. } => true, + + OmicronZoneType::BoundaryNtp { .. } + | OmicronZoneType::InternalNtp { .. } + | OmicronZoneType::Clickhouse { .. } + | OmicronZoneType::ClickhouseKeeper { .. } + | OmicronZoneType::CockroachDb { .. } + | OmicronZoneType::Crucible { .. } + | OmicronZoneType::CruciblePantry { .. } + | OmicronZoneType::ExternalDns { .. } + | OmicronZoneType::InternalDns { .. } + | OmicronZoneType::Oximeter { .. } => false, + } + } + + /// Identifies whether this a Crucible (not Crucible pantry) zone + pub fn is_crucible(&self) -> bool { + match self { + OmicronZoneType::Crucible { .. } => true, + + OmicronZoneType::BoundaryNtp { .. } + | OmicronZoneType::InternalNtp { .. } + | OmicronZoneType::Clickhouse { .. } + | OmicronZoneType::ClickhouseKeeper { .. } + | OmicronZoneType::CockroachDb { .. } + | OmicronZoneType::CruciblePantry { .. } + | OmicronZoneType::ExternalDns { .. } + | OmicronZoneType::InternalDns { .. } + | OmicronZoneType::Nexus { .. } + | OmicronZoneType::Oximeter { .. } => false, + } + } + + /// This zone's external IP + pub fn external_ip(&self) -> Option { + match self { + OmicronZoneType::Nexus { external_ip, .. } => Some(*external_ip), + OmicronZoneType::ExternalDns { dns_address, .. } => { + Some(dns_address.ip()) + } + OmicronZoneType::BoundaryNtp { snat_cfg, .. } => Some(snat_cfg.ip), + + OmicronZoneType::InternalNtp { .. } + | OmicronZoneType::Clickhouse { .. } + | OmicronZoneType::ClickhouseKeeper { .. } + | OmicronZoneType::CockroachDb { .. } + | OmicronZoneType::Crucible { .. } + | OmicronZoneType::CruciblePantry { .. } + | OmicronZoneType::InternalDns { .. } + | OmicronZoneType::Oximeter { .. } => None, + } + } + + /// The service vNIC providing external connectivity to this zone + pub fn service_vnic(&self) -> Option<&NetworkInterface> { + match self { + OmicronZoneType::Nexus { nic, .. } + | OmicronZoneType::ExternalDns { nic, .. } + | OmicronZoneType::BoundaryNtp { nic, .. } => Some(nic), + + OmicronZoneType::InternalNtp { .. } + | OmicronZoneType::Clickhouse { .. } + | OmicronZoneType::ClickhouseKeeper { .. } + | OmicronZoneType::CockroachDb { .. } + | OmicronZoneType::Crucible { .. } + | OmicronZoneType::CruciblePantry { .. } + | OmicronZoneType::InternalDns { .. } + | OmicronZoneType::Oximeter { .. } => None, + } + } +} + +/// Like [`OmicronZoneType`], but without any associated data. +/// +/// This enum is meant to correspond exactly 1:1 with `OmicronZoneType`. +/// +/// # String representations of this type +/// +/// There are no fewer than four string representations for this type, all +/// slightly different from each other. +/// +/// 1. [`Self::zone_prefix`]: Used to construct zone names. +/// 2. [`Self::service_prefix`]: Used to construct SMF service names. +/// 3. [`Self::name_prefix`]: Used to construct `Name` instances. +/// 4. [`Self::report_str`]: Used for reporting and testing. +/// +/// There is no `Display` impl to ensure that users explicitly choose the +/// representation they want. (Please play close attention to this! The +/// functions are all similar but different, and we don't currently have great +/// type safety around the choice.) +/// +/// ## Adding new representations +/// +/// If you have a new use case for a string representation, please reuse one of +/// the four representations if at all possible. If you must add a new one, +/// please add it here rather than doing something ad-hoc in the calling code +/// so it's more legible. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, PartialOrd, Ord)] +pub enum ZoneKind { + BoundaryNtp, + Clickhouse, + ClickhouseKeeper, + CockroachDb, + Crucible, + CruciblePantry, + ExternalDns, + InternalDns, + InternalNtp, + Nexus, + Oximeter, +} + +impl ZoneKind { + /// The NTP prefix used for both BoundaryNtp and InternalNtp zones and + /// services. + pub const NTP_PREFIX: &'static str = "ntp"; + + /// Return a string that is used to construct **zone names**. This string + /// is guaranteed to be stable over time. + pub fn zone_prefix(self) -> &'static str { + match self { + // BoundaryNtp and InternalNtp both use "ntp". + ZoneKind::BoundaryNtp | ZoneKind::InternalNtp => Self::NTP_PREFIX, + ZoneKind::Clickhouse => "clickhouse", + ZoneKind::ClickhouseKeeper => "clickhouse_keeper", + // Note "cockroachdb" for historical reasons. + ZoneKind::CockroachDb => "cockroachdb", + ZoneKind::Crucible => "crucible", + ZoneKind::CruciblePantry => "crucible_pantry", + ZoneKind::ExternalDns => "external_dns", + ZoneKind::InternalDns => "internal_dns", + ZoneKind::Nexus => "nexus", + ZoneKind::Oximeter => "oximeter", + } + } + + /// Return a string that is used to construct **SMF service names**. This + /// string is guaranteed to be stable over time. + pub fn service_prefix(self) -> &'static str { + match self { + // BoundaryNtp and InternalNtp both use "ntp". + ZoneKind::BoundaryNtp | ZoneKind::InternalNtp => Self::NTP_PREFIX, + ZoneKind::Clickhouse => "clickhouse", + ZoneKind::ClickhouseKeeper => "clickhouse_keeper", + // Note "cockroachdb" for historical reasons. + ZoneKind::CockroachDb => "cockroachdb", + ZoneKind::Crucible => "crucible", + // Note "crucible/pantry" for historical reasons. + ZoneKind::CruciblePantry => "crucible/pantry", + ZoneKind::ExternalDns => "external_dns", + ZoneKind::InternalDns => "internal_dns", + ZoneKind::Nexus => "nexus", + ZoneKind::Oximeter => "oximeter", + } + } + + /// Return a string suitable for use **in `Name` instances**. This string + /// is guaranteed to be stable over time. + /// + /// This string uses dashes rather than underscores, as required by `Name`. + pub fn name_prefix(self) -> &'static str { + match self { + // BoundaryNtp and InternalNtp both use "ntp" here. + ZoneKind::BoundaryNtp | ZoneKind::InternalNtp => Self::NTP_PREFIX, + ZoneKind::Clickhouse => "clickhouse", + ZoneKind::ClickhouseKeeper => "clickhouse-keeper", + // Note "cockroach" for historical reasons. + ZoneKind::CockroachDb => "cockroach", + ZoneKind::Crucible => "crucible", + ZoneKind::CruciblePantry => "crucible-pantry", + ZoneKind::ExternalDns => "external-dns", + ZoneKind::InternalDns => "internal-dns", + ZoneKind::Nexus => "nexus", + ZoneKind::Oximeter => "oximeter", + } + } + + /// Return a string that is used for reporting and error messages. This is + /// **not guaranteed** to be stable. + /// + /// If you're displaying a user-friendly message, prefer this method. + pub fn report_str(self) -> &'static str { + match self { + ZoneKind::BoundaryNtp => "boundary_ntp", + ZoneKind::Clickhouse => "clickhouse", + ZoneKind::ClickhouseKeeper => "clickhouse_keeper", + ZoneKind::CockroachDb => "cockroach_db", + ZoneKind::Crucible => "crucible", + ZoneKind::CruciblePantry => "crucible_pantry", + ZoneKind::ExternalDns => "external_dns", + ZoneKind::InternalDns => "internal_dns", + ZoneKind::InternalNtp => "internal_ntp", + ZoneKind::Nexus => "nexus", + ZoneKind::Oximeter => "oximeter", + } + } +} diff --git a/nexus-sled-agent-shared/src/lib.rs b/nexus-sled-agent-shared/src/lib.rs new file mode 100644 index 0000000000..6781568d62 --- /dev/null +++ b/nexus-sled-agent-shared/src/lib.rs @@ -0,0 +1,11 @@ +// This Source Code Form is subject to the terms of the Mozilla Public +// License, v. 2.0. If a copy of the MPL was not distributed with this +// file, You can obtain one at https://mozilla.org/MPL/2.0/. + +//! Internal types shared between Nexus and sled-agent, with extra dependencies +//! not in omicron-common. +//! +//! For more information, see the crate [README](../README.md). + +pub mod inventory; +pub mod recovery_silo; diff --git a/nexus-sled-agent-shared/src/recovery_silo.rs b/nexus-sled-agent-shared/src/recovery_silo.rs new file mode 100644 index 0000000000..e9e70a1f65 --- /dev/null +++ b/nexus-sled-agent-shared/src/recovery_silo.rs @@ -0,0 +1,16 @@ +// This Source Code Form is subject to the terms of the Mozilla Public +// License, v. 2.0. If a copy of the MPL was not distributed with this +// file, You can obtain one at https://mozilla.org/MPL/2.0/. + +//! Configuration for the recovery silo. + +use omicron_common::api::external::{Name, UserId}; +use schemars::JsonSchema; +use serde::{Deserialize, Serialize}; + +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema)] +pub struct RecoverySiloConfig { + pub silo_name: Name, + pub user_name: UserId, + pub user_password_hash: omicron_passwords::NewPasswordHash, +} diff --git a/nexus/Cargo.toml b/nexus/Cargo.toml index 8d256aad5a..a949b31f0d 100644 --- a/nexus/Cargo.toml +++ b/nexus/Cargo.toml @@ -98,6 +98,7 @@ nexus-metrics-producer-gc.workspace = true nexus-reconfigurator-execution.workspace = true nexus-reconfigurator-planning.workspace = true nexus-reconfigurator-preparation.workspace = true +nexus-sled-agent-shared.workspace = true nexus-types.workspace = true omicron-common.workspace = true omicron-passwords.workspace = true diff --git a/nexus/db-model/Cargo.toml b/nexus/db-model/Cargo.toml index a7b6cd9de1..2e99d389a0 100644 --- a/nexus/db-model/Cargo.toml +++ b/nexus/db-model/Cargo.toml @@ -45,6 +45,7 @@ omicron-certificates.workspace = true omicron-common.workspace = true nexus-config.workspace = true nexus-defaults.workspace = true +nexus-sled-agent-shared.workspace = true nexus-types.workspace = true omicron-passwords.workspace = true sled-agent-client.workspace = true diff --git a/nexus/db-model/src/dataset_kind.rs b/nexus/db-model/src/dataset_kind.rs index 86495b9d61..395d01353e 100644 --- a/nexus/db-model/src/dataset_kind.rs +++ b/nexus/db-model/src/dataset_kind.rs @@ -3,7 +3,7 @@ // file, You can obtain one at https://mozilla.org/MPL/2.0/. use super::impl_enum_type; -use nexus_types::internal_api; +use omicron_common::api::internal; use serde::{Deserialize, Serialize}; impl_enum_type!( @@ -24,25 +24,21 @@ impl_enum_type!( InternalDns => b"internal_dns" ); -impl From for DatasetKind { - fn from(k: internal_api::params::DatasetKind) -> Self { +impl From for DatasetKind { + fn from(k: internal::shared::DatasetKind) -> Self { match k { - internal_api::params::DatasetKind::Crucible => { - DatasetKind::Crucible - } - internal_api::params::DatasetKind::Cockroach => { - DatasetKind::Cockroach - } - internal_api::params::DatasetKind::Clickhouse => { + internal::shared::DatasetKind::Crucible => DatasetKind::Crucible, + internal::shared::DatasetKind::Cockroach => DatasetKind::Cockroach, + internal::shared::DatasetKind::Clickhouse => { DatasetKind::Clickhouse } - internal_api::params::DatasetKind::ClickhouseKeeper => { + internal::shared::DatasetKind::ClickhouseKeeper => { DatasetKind::ClickhouseKeeper } - internal_api::params::DatasetKind::ExternalDns => { + internal::shared::DatasetKind::ExternalDns => { DatasetKind::ExternalDns } - internal_api::params::DatasetKind::InternalDns => { + internal::shared::DatasetKind::InternalDns => { DatasetKind::InternalDns } } diff --git a/nexus/db-model/src/deployment.rs b/nexus/db-model/src/deployment.rs index c398f6d11c..6bef893a5b 100644 --- a/nexus/db-model/src/deployment.rs +++ b/nexus/db-model/src/deployment.rs @@ -197,7 +197,7 @@ impl From for BlueprintPhysicalDiskConfig { } } -/// See [`nexus_types::deployment::OmicronZonesConfig`]. +/// See [`nexus_types::deployment::BlueprintZonesConfig`]. #[derive(Queryable, Clone, Debug, Selectable, Insertable)] #[diesel(table_name = bp_sled_omicron_zones)] pub struct BpSledOmicronZones { @@ -219,7 +219,7 @@ impl BpSledOmicronZones { } } } -/// See [`nexus_types::deployment::OmicronZoneConfig`]. +/// See [`nexus_types::deployment::BlueprintZoneConfig`]. #[derive(Queryable, Clone, Debug, Selectable, Insertable)] #[diesel(table_name = bp_omicron_zone)] pub struct BpOmicronZone { diff --git a/nexus/db-model/src/external_ip.rs b/nexus/db-model/src/external_ip.rs index 8226f8293e..bfa7c73d3d 100644 --- a/nexus/db-model/src/external_ip.rs +++ b/nexus/db-model/src/external_ip.rs @@ -17,6 +17,7 @@ use db_macros::Resource; use diesel::Queryable; use diesel::Selectable; use ipnetwork::IpNetwork; +use nexus_sled_agent_shared::inventory::ZoneKind; use nexus_types::deployment::OmicronZoneExternalFloatingIp; use nexus_types::deployment::OmicronZoneExternalIp; use nexus_types::deployment::OmicronZoneExternalSnatIp; @@ -36,7 +37,6 @@ use schemars::JsonSchema; use serde::Deserialize; use serde::Serialize; use sled_agent_client::types::InstanceExternalIpBody; -use sled_agent_client::ZoneKind; use slog_error_chain::SlogInlineError; use std::convert::TryFrom; use std::net::IpAddr; @@ -406,7 +406,7 @@ impl IncompleteExternalIp { IpKind::Floating, None, Some(name), - Some(zone_kind.to_string()), + Some(zone_kind.report_str().to_string()), state, ) } diff --git a/nexus/db-model/src/inventory.rs b/nexus/db-model/src/inventory.rs index 14c4684e1e..400acc68b3 100644 --- a/nexus/db-model/src/inventory.rs +++ b/nexus/db-model/src/inventory.rs @@ -28,6 +28,9 @@ use diesel::pg::Pg; use diesel::serialize::ToSql; use diesel::{serialize, sql_types}; use ipnetwork::IpNetwork; +use nexus_sled_agent_shared::inventory::{ + OmicronZoneConfig, OmicronZonesConfig, +}; use nexus_types::inventory::{ BaseboardId, Caboose, Collection, PowerState, RotPage, RotSlot, }; @@ -758,20 +761,28 @@ impl_enum_type!( Scrimlet => b"scrimlet" ); -impl From for SledRole { - fn from(value: nexus_types::inventory::SledRole) -> Self { +impl From for SledRole { + fn from(value: nexus_sled_agent_shared::inventory::SledRole) -> Self { match value { - nexus_types::inventory::SledRole::Gimlet => SledRole::Gimlet, - nexus_types::inventory::SledRole::Scrimlet => SledRole::Scrimlet, + nexus_sled_agent_shared::inventory::SledRole::Gimlet => { + SledRole::Gimlet + } + nexus_sled_agent_shared::inventory::SledRole::Scrimlet => { + SledRole::Scrimlet + } } } } -impl From for nexus_types::inventory::SledRole { +impl From for nexus_sled_agent_shared::inventory::SledRole { fn from(value: SledRole) -> Self { match value { - SledRole::Gimlet => nexus_types::inventory::SledRole::Gimlet, - SledRole::Scrimlet => nexus_types::inventory::SledRole::Scrimlet, + SledRole::Gimlet => { + nexus_sled_agent_shared::inventory::SledRole::Gimlet + } + SledRole::Scrimlet => { + nexus_sled_agent_shared::inventory::SledRole::Scrimlet + } } } } @@ -953,7 +964,7 @@ impl InvSledOmicronZones { time_collected: self.time_collected, source: self.source, sled_id: self.sled_id.into(), - zones: nexus_types::inventory::OmicronZonesConfig { + zones: OmicronZonesConfig { generation: *self.generation, zones: Vec::new(), }, @@ -1001,7 +1012,47 @@ impl From for ServiceKind { } } -/// See [`nexus_types::inventory::OmicronZoneConfig`]. +impl From for nexus_sled_agent_shared::inventory::ZoneKind { + fn from(zone_type: ZoneType) -> Self { + use nexus_sled_agent_shared::inventory::ZoneKind::*; + + match zone_type { + ZoneType::BoundaryNtp => BoundaryNtp, + ZoneType::Clickhouse => Clickhouse, + ZoneType::ClickhouseKeeper => ClickhouseKeeper, + ZoneType::CockroachDb => CockroachDb, + ZoneType::Crucible => Crucible, + ZoneType::CruciblePantry => CruciblePantry, + ZoneType::ExternalDns => ExternalDns, + ZoneType::InternalDns => InternalDns, + ZoneType::InternalNtp => InternalNtp, + ZoneType::Nexus => Nexus, + ZoneType::Oximeter => Oximeter, + } + } +} + +impl From for ZoneType { + fn from(zone_kind: nexus_sled_agent_shared::inventory::ZoneKind) -> Self { + use nexus_sled_agent_shared::inventory::ZoneKind::*; + + match zone_kind { + BoundaryNtp => ZoneType::BoundaryNtp, + Clickhouse => ZoneType::Clickhouse, + ClickhouseKeeper => ZoneType::ClickhouseKeeper, + CockroachDb => ZoneType::CockroachDb, + Crucible => ZoneType::Crucible, + CruciblePantry => ZoneType::CruciblePantry, + ExternalDns => ZoneType::ExternalDns, + InternalDns => ZoneType::InternalDns, + InternalNtp => ZoneType::InternalNtp, + Nexus => ZoneType::Nexus, + Oximeter => ZoneType::Oximeter, + } + } +} + +/// See [`nexus_sled_agent_shared::inventory::OmicronZoneConfig`]. #[derive(Queryable, Clone, Debug, Selectable, Insertable)] #[diesel(table_name = inv_omicron_zone)] pub struct InvOmicronZone { @@ -1033,7 +1084,7 @@ impl InvOmicronZone { pub fn new( inv_collection_id: CollectionUuid, sled_id: SledUuid, - zone: &nexus_types::inventory::OmicronZoneConfig, + zone: &OmicronZoneConfig, ) -> Result { // Inventory zones do not know the external IP ID. let external_ip_id = None; @@ -1074,7 +1125,7 @@ impl InvOmicronZone { pub fn into_omicron_zone_config( self, nic_row: Option, - ) -> Result { + ) -> Result { let zone = OmicronZone { sled_id: self.sled_id.into(), id: self.id, @@ -1137,7 +1188,7 @@ impl From for OmicronZoneNic { impl InvOmicronZoneNic { pub fn new( inv_collection_id: CollectionUuid, - zone: &nexus_types::inventory::OmicronZoneConfig, + zone: &OmicronZoneConfig, ) -> Result, anyhow::Error> { let Some(nic) = zone.zone_type.service_vnic() else { return Ok(None); diff --git a/nexus/db-model/src/network_interface.rs b/nexus/db-model/src/network_interface.rs index 79b16b5658..15404524c5 100644 --- a/nexus/db-model/src/network_interface.rs +++ b/nexus/db-model/src/network_interface.rs @@ -15,6 +15,7 @@ use db_macros::Resource; use diesel::AsChangeset; use ipnetwork::IpNetwork; use ipnetwork::NetworkSize; +use nexus_sled_agent_shared::inventory::ZoneKind; use nexus_types::external_api::params; use nexus_types::identity::Resource; use omicron_common::api::{external, internal}; @@ -22,7 +23,6 @@ use omicron_uuid_kinds::GenericUuid; use omicron_uuid_kinds::InstanceUuid; use omicron_uuid_kinds::OmicronZoneUuid; use omicron_uuid_kinds::VnicUuid; -use sled_agent_client::ZoneKind; use uuid::Uuid; /// The max number of interfaces that may be associated with a resource, @@ -161,26 +161,11 @@ pub struct ServiceNetworkInterface { impl ServiceNetworkInterface { /// Generate a suitable [`Name`] for the given Omicron zone ID and kind. pub fn name(zone_id: OmicronZoneUuid, zone_kind: ZoneKind) -> Name { - // Ideally we'd use `zone_kind.to_string()` here, but that uses - // underscores as separators which aren't allowed in `Name`s. We also - // preserve some existing naming behavior where NTP external networking - // is just called "ntp", not "boundary-ntp". - // - // Most of these zone kinds do not get external networking and therefore - // we don't need to be able to generate names for them, but it's simpler - // to give them valid descriptions than worry about error handling here. - let prefix = match zone_kind { - ZoneKind::BoundaryNtp | ZoneKind::InternalNtp => "ntp", - ZoneKind::Clickhouse => "clickhouse", - ZoneKind::ClickhouseKeeper => "clickhouse-keeper", - ZoneKind::CockroachDb => "cockroach", - ZoneKind::Crucible => "crucible", - ZoneKind::CruciblePantry => "crucible-pantry", - ZoneKind::ExternalDns => "external-dns", - ZoneKind::InternalDns => "internal-dns", - ZoneKind::Nexus => "nexus", - ZoneKind::Oximeter => "oximeter", - }; + // Most of these zone kinds do not get external networking and + // therefore we don't need to be able to generate names for them, but + // it's simpler to give them valid descriptions than worry about error + // handling here. + let prefix = zone_kind.name_prefix(); // Now that we have a valid prefix, we know this format string // always produces a valid `Name`, so we'll unwrap here. diff --git a/nexus/db-model/src/omicron_zone_config.rs b/nexus/db-model/src/omicron_zone_config.rs index bb3eac7046..9236fc9407 100644 --- a/nexus/db-model/src/omicron_zone_config.rs +++ b/nexus/db-model/src/omicron_zone_config.rs @@ -15,13 +15,15 @@ use crate::inventory::ZoneType; use crate::{ipv6, MacAddr, Name, SqlU16, SqlU32, SqlU8}; use anyhow::{anyhow, bail, ensure, Context}; use ipnetwork::IpNetwork; -use nexus_types::deployment::BlueprintZoneDisposition; -use nexus_types::deployment::BlueprintZoneType; +use nexus_sled_agent_shared::inventory::{ + OmicronZoneConfig, OmicronZoneDataset, OmicronZoneType, +}; use nexus_types::deployment::{ - blueprint_zone_type, OmicronZoneExternalFloatingAddr, - OmicronZoneExternalFloatingIp, OmicronZoneExternalSnatIp, + blueprint_zone_type, BlueprintZoneDisposition, BlueprintZoneType, + OmicronZoneExternalFloatingAddr, OmicronZoneExternalFloatingIp, + OmicronZoneExternalSnatIp, }; -use nexus_types::inventory::{NetworkInterface, OmicronZoneType}; +use nexus_types::inventory::NetworkInterface; use omicron_common::api::internal::shared::NetworkInterfaceKind; use omicron_common::zpool_name::ZpoolName; use omicron_uuid_kinds::{ @@ -63,7 +65,7 @@ impl OmicronZone { zone_id: Uuid, zone_underlay_address: Ipv6Addr, filesystem_pool: Option, - zone_type: &nexus_types::inventory::OmicronZoneType, + zone_type: &OmicronZoneType, external_ip_id: Option, ) -> anyhow::Result { let id = zone_id; @@ -82,8 +84,7 @@ impl OmicronZone { let mut second_service_ip = None; let mut second_service_port = None; - let (zone_type, primary_service_sockaddr_str, dataset) = match zone_type - { + let (zone_type, primary_service_sockaddr, dataset) = match zone_type { OmicronZoneType::BoundaryNtp { address, ntp_servers, @@ -124,16 +125,8 @@ impl OmicronZone { nic, } => { nic_id = Some(nic.id); - let sockaddr = dns_address - .parse::() - .with_context(|| { - format!( - "parsing address for external DNS server {:?}", - dns_address - ) - })?; - second_service_ip = Some(sockaddr.ip()); - second_service_port = Some(SqlU16::from(sockaddr.port())); + second_service_ip = Some(dns_address.ip()); + second_service_port = Some(SqlU16::from(dns_address.port())); (ZoneType::ExternalDns, http_address, Some(dataset)) } OmicronZoneType::InternalDns { @@ -145,16 +138,8 @@ impl OmicronZone { } => { dns_gz_address = Some(ipv6::Ipv6Addr::from(gz_address)); dns_gz_address_index = Some(SqlU32::from(*gz_address_index)); - let sockaddr = dns_address - .parse::() - .with_context(|| { - format!( - "parsing address for internal DNS server {:?}", - dns_address - ) - })?; - second_service_ip = Some(sockaddr.ip()); - second_service_port = Some(SqlU16::from(sockaddr.port())); + second_service_ip = Some(IpAddr::V6(*dns_address.ip())); + second_service_port = Some(SqlU16::from(dns_address.port())); (ZoneType::InternalDns, http_address, Some(dataset)) } OmicronZoneType::InternalNtp { @@ -187,14 +172,6 @@ impl OmicronZone { }; let dataset_zpool_name = dataset.map(|d| d.pool_name.to_string()); - let primary_service_sockaddr = primary_service_sockaddr_str - .parse::() - .with_context(|| { - format!( - "parsing socket address for primary IP {:?}", - primary_service_sockaddr_str - ) - })?; let (primary_service_ip, primary_service_port) = ( ipv6::Ipv6Addr::from(*primary_service_sockaddr.ip()), SqlU16::from(primary_service_sockaddr.port()), @@ -310,12 +287,7 @@ impl OmicronZone { ZoneType::InternalDns => BlueprintZoneType::InternalDns( blueprint_zone_type::InternalDns { dataset: common.dataset?, - dns_address: match common.dns_address? { - SocketAddr::V4(addr) => { - bail!("expected V6 address; got {addr}") - } - SocketAddr::V6(addr) => addr, - }, + dns_address: to_internal_dns_address(common.dns_address?)?, http_address: address, gz_address: *common.dns_gz_address.ok_or_else(|| { anyhow!("expected dns_gz_address, found none") @@ -379,9 +351,9 @@ impl OmicronZone { pub(crate) fn into_omicron_zone_config( self, nic_row: Option, - ) -> anyhow::Result { + ) -> anyhow::Result { let common = self.into_zone_config_common(nic_row)?; - let address = common.primary_service_address.to_string(); + let address = common.primary_service_address; let zone_type = match common.zone_type { ZoneType::BoundaryNtp => { @@ -432,13 +404,13 @@ impl OmicronZone { } ZoneType::ExternalDns => OmicronZoneType::ExternalDns { dataset: common.dataset?, - dns_address: common.dns_address?.to_string(), + dns_address: common.dns_address?, http_address: address, nic: common.nic?, }, ZoneType::InternalDns => OmicronZoneType::InternalDns { dataset: common.dataset?, - dns_address: common.dns_address?.to_string(), + dns_address: to_internal_dns_address(common.dns_address?)?, http_address: address, gz_address: *common.dns_gz_address.ok_or_else(|| { anyhow!("expected dns_gz_address, found none") @@ -472,7 +444,7 @@ impl OmicronZone { }, ZoneType::Oximeter => OmicronZoneType::Oximeter { address }, }; - Ok(nexus_types::inventory::OmicronZoneConfig { + Ok(OmicronZoneConfig { id: common.id, underlay_address: std::net::Ipv6Addr::from(common.underlay_address), filesystem_pool: common @@ -530,7 +502,7 @@ impl OmicronZone { let dataset = self .dataset_zpool_name .map(|zpool_name| -> Result<_, anyhow::Error> { - Ok(nexus_types::inventory::OmicronZoneDataset { + Ok(OmicronZoneDataset { pool_name: zpool_name.parse().map_err(|e| { anyhow!("parsing zpool name {:?}: {}", zpool_name, e) })?, @@ -608,13 +580,32 @@ struct ZoneConfigCommon { // These properties may or may not be needed, depending on the zone type. We // store results here that can be unpacked once we determine our zone type. nic: anyhow::Result, - dataset: anyhow::Result, + dataset: anyhow::Result, + // Note that external DNS is SocketAddr (also supports v4) while internal + // DNS is always v6. dns_address: anyhow::Result, ntp_dns_servers: anyhow::Result>, ntp_ntp_servers: anyhow::Result>, external_ip_id: anyhow::Result, } +// Ideally this would be a method on `ZoneConfigCommon`, but that's more +// annoying to deal with because often, at the time this function is called, +// part of `ZoneConfigCommon` has already been moved out. +fn to_internal_dns_address( + external_address: SocketAddr, +) -> anyhow::Result { + match external_address { + SocketAddr::V4(address) => { + bail!( + "expected internal DNS address to be v6, found v4: {:?}", + address + ) + } + SocketAddr::V6(v6) => Ok(v6), + } +} + #[derive(Debug)] pub(crate) struct OmicronZoneNic { pub(crate) id: Uuid, diff --git a/nexus/db-model/src/sled.rs b/nexus/db-model/src/sled.rs index b02a082f07..aa13951da7 100644 --- a/nexus/db-model/src/sled.rs +++ b/nexus/db-model/src/sled.rs @@ -10,6 +10,7 @@ use crate::sled::shared::Baseboard; use crate::sled_policy::DbSledPolicy; use chrono::{DateTime, Utc}; use db_macros::Asset; +use nexus_sled_agent_shared::inventory::SledRole; use nexus_types::{ external_api::{shared, views}, identity::Asset, @@ -141,9 +142,9 @@ impl From for views::Sled { impl From for params::SledAgentInfo { fn from(sled: Sled) -> Self { let role = if sled.is_scrimlet { - params::SledRole::Scrimlet + SledRole::Scrimlet } else { - params::SledRole::Gimlet + SledRole::Gimlet }; let decommissioned = match sled.state { SledState::Active => false, diff --git a/nexus/db-queries/Cargo.toml b/nexus/db-queries/Cargo.toml index cb7061f4ce..5192528944 100644 --- a/nexus/db-queries/Cargo.toml +++ b/nexus/db-queries/Cargo.toml @@ -53,6 +53,7 @@ nexus-auth.workspace = true nexus-config.workspace = true nexus-db-fixed-data.workspace = true nexus-db-model.workspace = true +nexus-sled-agent-shared.workspace = true nexus-types.workspace = true omicron-common.workspace = true omicron-passwords.workspace = true diff --git a/nexus/db-queries/src/db/datastore/external_ip.rs b/nexus/db-queries/src/db/datastore/external_ip.rs index 0614cd0d9f..9a3928dd58 100644 --- a/nexus/db-queries/src/db/datastore/external_ip.rs +++ b/nexus/db-queries/src/db/datastore/external_ip.rs @@ -39,6 +39,7 @@ use diesel::prelude::*; use nexus_db_model::FloatingIpUpdate; use nexus_db_model::Instance; use nexus_db_model::IpAttachState; +use nexus_sled_agent_shared::inventory::ZoneKind; use nexus_types::deployment::OmicronZoneExternalIp; use nexus_types::identity::Resource; use omicron_common::api::external::http_pagination::PaginatedBy; @@ -55,7 +56,6 @@ use omicron_uuid_kinds::GenericUuid; use omicron_uuid_kinds::InstanceUuid; use omicron_uuid_kinds::OmicronZoneUuid; use ref_cast::RefCast; -use sled_agent_client::ZoneKind; use std::net::IpAddr; use uuid::Uuid; diff --git a/nexus/db-queries/src/db/datastore/inventory.rs b/nexus/db-queries/src/db/datastore/inventory.rs index 289e443213..1774a25c48 100644 --- a/nexus/db-queries/src/db/datastore/inventory.rs +++ b/nexus/db-queries/src/db/datastore/inventory.rs @@ -1693,9 +1693,7 @@ impl DataStore { 0, 0, ), - sled_role: nexus_types::inventory::SledRole::from( - s.sled_role, - ), + sled_role: s.sled_role.into(), usable_hardware_threads: u32::from( s.usable_hardware_threads, ), diff --git a/nexus/db-queries/src/db/datastore/physical_disk.rs b/nexus/db-queries/src/db/datastore/physical_disk.rs index 5e3b51f228..dc26e093c0 100644 --- a/nexus/db-queries/src/db/datastore/physical_disk.rs +++ b/nexus/db-queries/src/db/datastore/physical_disk.rs @@ -329,13 +329,14 @@ mod test { use crate::db::model::{PhysicalDiskKind, Sled, SledUpdate}; use dropshot::PaginationOrder; use nexus_db_model::Generation; + use nexus_sled_agent_shared::inventory::{ + Baseboard, Inventory, InventoryDisk, SledRole, + }; use nexus_test_utils::db::test_setup_database; use nexus_types::identity::Asset; use omicron_common::api::external::ByteCount; - use omicron_common::disk::DiskIdentity; + use omicron_common::disk::{DiskIdentity, DiskVariant}; use omicron_test_utils::dev; - use sled_agent_client::types::DiskVariant; - use sled_agent_client::types::InventoryDisk; use std::net::{Ipv6Addr, SocketAddrV6}; use std::num::NonZeroU32; @@ -677,19 +678,19 @@ mod test { fn add_sled_to_inventory( builder: &mut nexus_inventory::CollectionBuilder, sled: &Sled, - disks: Vec, + disks: Vec, ) { builder .found_sled_inventory( "fake sled agent", - sled_agent_client::types::Inventory { - baseboard: sled_agent_client::types::Baseboard::Gimlet { + Inventory { + baseboard: Baseboard::Gimlet { identifier: sled.serial_number().to_string(), model: sled.part_number().to_string(), revision: 0, }, reservoir_size: ByteCount::from(1024), - sled_role: sled_agent_client::types::SledRole::Gimlet, + sled_role: SledRole::Gimlet, sled_agent_address: "[::1]:56792".parse().unwrap(), sled_id: sled.id(), usable_hardware_threads: 10, diff --git a/nexus/db-queries/src/db/datastore/rack.rs b/nexus/db-queries/src/db/datastore/rack.rs index dac1c2847d..c9fb61b15a 100644 --- a/nexus/db-queries/src/db/datastore/rack.rs +++ b/nexus/db-queries/src/db/datastore/rack.rs @@ -65,6 +65,7 @@ use omicron_common::api::external::ListResultVec; use omicron_common::api::external::LookupType; use omicron_common::api::external::ResourceType; use omicron_common::api::external::UpdateResult; +use omicron_common::api::external::UserId; use omicron_common::bail_unless; use omicron_uuid_kinds::GenericUuid; use omicron_uuid_kinds::SledUuid; @@ -86,7 +87,7 @@ pub struct RackInit { pub external_dns: InitialDnsGroup, pub recovery_silo: external_params::SiloCreate, pub recovery_silo_fq_dns_name: String, - pub recovery_user_id: external_params::UserId, + pub recovery_user_id: UserId, pub recovery_user_password_hash: omicron_passwords::PasswordHashString, pub dns_update: DnsVersionUpdateBuilder, pub allowed_source_ips: AllowedSourceIps, @@ -429,7 +430,7 @@ impl DataStore { log: &slog::Logger, recovery_silo: external_params::SiloCreate, recovery_silo_fq_dns_name: String, - recovery_user_id: external_params::UserId, + recovery_user_id: UserId, recovery_user_password_hash: omicron_passwords::PasswordHashString, dns_update: DnsVersionUpdateBuilder, ) -> Result<(), RackInitError> { @@ -520,6 +521,7 @@ impl DataStore { // For services with external connectivity, we record their // explicit IP allocation and create a service NIC as well. let zone_type = &zone_config.zone_type; + let zone_report_str = zone_type.kind().report_str(); let service_ip_nic = match zone_type { BlueprintZoneType::ExternalDns( blueprint_zone_type::ExternalDns { nic, dns_address, .. }, @@ -534,7 +536,7 @@ impl DataStore { name: nic.name.clone(), description: format!( "{} service vNIC", - zone_type.kind() + zone_report_str ), }, nic.ip, @@ -558,7 +560,7 @@ impl DataStore { name: nic.name.clone(), description: format!( "{} service vNIC", - zone_type.kind() + zone_report_str ), }, nic.ip, @@ -580,7 +582,7 @@ impl DataStore { name: nic.name.clone(), description: format!( "{} service vNIC", - zone_type.kind() + zone_report_str ), }, nic.ip, @@ -602,8 +604,7 @@ impl DataStore { let Some((external_ip, db_nic)) = service_ip_nic else { info!( log, - "No networking records needed for {} service", - zone_type.kind(), + "No networking records needed for {} service", zone_report_str, ); return Ok(()); }; @@ -619,7 +620,7 @@ impl DataStore { log, "Initializing Rack: Failed to allocate \ IP address for {}", - zone_type.kind(); + zone_report_str; "err" => %err, ); match err.retryable() { @@ -648,7 +649,7 @@ impl DataStore { info!( log, "Inserted networking records for {} service", - zone_type.kind(), + zone_type.kind().report_str(), ); Ok(()) @@ -1011,6 +1012,7 @@ mod test { use nexus_reconfigurator_planning::system::{ SledBuilder, SystemDescription, }; + use nexus_sled_agent_shared::inventory::OmicronZoneDataset; use nexus_test_utils::db::test_setup_database; use nexus_types::deployment::BlueprintZonesConfig; use nexus_types::deployment::CockroachDbPreserveDowngrade; @@ -1041,7 +1043,6 @@ mod test { use omicron_uuid_kinds::{GenericUuid, ZpoolUuid}; use omicron_uuid_kinds::{SledUuid, TypedUuid}; use oxnet::IpNet; - use sled_agent_client::types::OmicronZoneDataset; use std::collections::{BTreeMap, HashMap}; use std::net::{IpAddr, Ipv4Addr, Ipv6Addr, SocketAddr, SocketAddrV6}; use std::num::NonZeroU32; diff --git a/nexus/db-queries/src/db/queries/external_ip.rs b/nexus/db-queries/src/db/queries/external_ip.rs index b48196bde8..7ea44b33fb 100644 --- a/nexus/db-queries/src/db/queries/external_ip.rs +++ b/nexus/db-queries/src/db/queries/external_ip.rs @@ -881,6 +881,7 @@ mod tests { use nexus_db_model::InstanceCpuCount; use nexus_db_model::IpPoolResource; use nexus_db_model::IpPoolResourceType; + use nexus_sled_agent_shared::inventory::ZoneKind; use nexus_test_utils::db::test_setup_database; use nexus_types::deployment::OmicronZoneExternalFloatingIp; use nexus_types::deployment::OmicronZoneExternalIp; @@ -897,7 +898,6 @@ mod tests { use omicron_uuid_kinds::GenericUuid; use omicron_uuid_kinds::InstanceUuid; use omicron_uuid_kinds::OmicronZoneUuid; - use sled_agent_client::ZoneKind; use std::net::IpAddr; use std::net::Ipv4Addr; use std::sync::Arc; diff --git a/nexus/inventory/Cargo.toml b/nexus/inventory/Cargo.toml index e185808caa..3057617c67 100644 --- a/nexus/inventory/Cargo.toml +++ b/nexus/inventory/Cargo.toml @@ -14,6 +14,7 @@ chrono.workspace = true futures.workspace = true gateway-client.workspace = true gateway-messages.workspace = true +nexus-sled-agent-shared.workspace = true nexus-types.workspace = true omicron-common.workspace = true omicron-uuid-kinds.workspace = true diff --git a/nexus/inventory/src/builder.rs b/nexus/inventory/src/builder.rs index 65bdae63ce..3b6b4bc007 100644 --- a/nexus/inventory/src/builder.rs +++ b/nexus/inventory/src/builder.rs @@ -14,6 +14,9 @@ use chrono::Utc; use gateway_client::types::SpComponentCaboose; use gateway_client::types::SpState; use gateway_client::types::SpType; +use nexus_sled_agent_shared::inventory::Baseboard; +use nexus_sled_agent_shared::inventory::Inventory; +use nexus_sled_agent_shared::inventory::OmicronZonesConfig; use nexus_types::inventory::BaseboardId; use nexus_types::inventory::Caboose; use nexus_types::inventory::CabooseFound; @@ -469,12 +472,10 @@ impl CollectionBuilder { pub fn found_sled_inventory( &mut self, source: &str, - inventory: sled_agent_client::types::Inventory, + inventory: Inventory, ) -> Result<(), anyhow::Error> { let sled_id = SledUuid::from_untyped_uuid(inventory.sled_id); - // Normalize the baseboard id, if any. - use sled_agent_client::types::Baseboard; let baseboard_id = match inventory.baseboard { Baseboard::Pc { .. } => None, Baseboard::Gimlet { identifier, model, revision: _ } => { @@ -498,21 +499,10 @@ impl CollectionBuilder { // means they don't get validated when everything else does. This // error is an operational error in collecting the data, not a collector // bug. - let sled_agent_address = match inventory.sled_agent_address.parse() { - Ok(addr) => addr, - Err(error) => { - self.found_error(InventoryError::from(anyhow!( - "sled {sled_id}: bad sled agent address: {:?}: {:#}", - inventory.sled_agent_address, - error, - ))); - return Ok(()); - } - }; let time_collected = now_db_precision(); let sled = SledAgent { source: source.to_string(), - sled_agent_address, + sled_agent_address: inventory.sled_agent_address, sled_role: inventory.sled_role, baseboard_id, usable_hardware_threads: inventory.usable_hardware_threads, @@ -544,7 +534,7 @@ impl CollectionBuilder { &mut self, source: &str, sled_id: SledUuid, - zones: sled_agent_client::types::OmicronZonesConfig, + zones: OmicronZonesConfig, ) -> Result<(), anyhow::Error> { if let Some(previous) = self.omicron_zones.get(&sled_id) { Err(anyhow!( @@ -594,12 +584,12 @@ mod test { use gateway_client::types::SpComponentCaboose; use gateway_client::types::SpState; use gateway_client::types::SpType; + use nexus_sled_agent_shared::inventory::SledRole; use nexus_types::inventory::BaseboardId; use nexus_types::inventory::Caboose; use nexus_types::inventory::CabooseWhich; use nexus_types::inventory::RotPage; use nexus_types::inventory::RotPageWhich; - use nexus_types::inventory::SledRole; use omicron_common::api::external::ByteCount; // Verify the contents of an empty collection. diff --git a/nexus/inventory/src/collector.rs b/nexus/inventory/src/collector.rs index b65d87505f..25f8ffded6 100644 --- a/nexus/inventory/src/collector.rs +++ b/nexus/inventory/src/collector.rs @@ -377,6 +377,9 @@ mod test { use super::Collector; use crate::StaticSledAgentEnumerator; use gateway_messages::SpPort; + use nexus_sled_agent_shared::inventory::OmicronZoneConfig; + use nexus_sled_agent_shared::inventory::OmicronZoneType; + use nexus_sled_agent_shared::inventory::OmicronZonesConfig; use nexus_types::inventory::Collection; use omicron_common::api::external::Generation; use omicron_common::zpool_name::ZpoolName; @@ -496,7 +499,7 @@ mod test { &mut s, " zone {} type {}\n", zone.id, - zone.zone_type.kind(), + zone.zone_type.kind().report_str(), ) .unwrap(); } @@ -552,15 +555,14 @@ mod test { let filesystem_pool = ZpoolName::new_external(ZpoolUuid::new_v4()); let zone_address = SocketAddrV6::new(Ipv6Addr::LOCALHOST, 123, 0, 0); client - .omicron_zones_put(&sled_agent_client::types::OmicronZonesConfig { + .omicron_zones_put(&OmicronZonesConfig { generation: Generation::from(3), - zones: vec![sled_agent_client::types::OmicronZoneConfig { + zones: vec![OmicronZoneConfig { id: zone_id, underlay_address: *zone_address.ip(), - zone_type: - sled_agent_client::types::OmicronZoneType::Oximeter { - address: zone_address.to_string(), - }, + zone_type: OmicronZoneType::Oximeter { + address: zone_address, + }, filesystem_pool: Some(filesystem_pool), }], }) diff --git a/nexus/inventory/src/examples.rs b/nexus/inventory/src/examples.rs index c2e283a640..2dade3d34f 100644 --- a/nexus/inventory/src/examples.rs +++ b/nexus/inventory/src/examples.rs @@ -11,12 +11,18 @@ use gateway_client::types::RotState; use gateway_client::types::SpComponentCaboose; use gateway_client::types::SpState; use gateway_client::types::SpType; +use nexus_sled_agent_shared::inventory::Baseboard; +use nexus_sled_agent_shared::inventory::Inventory; +use nexus_sled_agent_shared::inventory::InventoryDisk; +use nexus_sled_agent_shared::inventory::InventoryZpool; +use nexus_sled_agent_shared::inventory::OmicronZonesConfig; +use nexus_sled_agent_shared::inventory::SledRole; use nexus_types::inventory::BaseboardId; use nexus_types::inventory::CabooseWhich; -use nexus_types::inventory::OmicronZonesConfig; use nexus_types::inventory::RotPage; use nexus_types::inventory::RotPageWhich; use omicron_common::api::external::ByteCount; +use omicron_common::disk::DiskVariant; use omicron_uuid_kinds::GenericUuid; use omicron_uuid_kinds::SledUuid; use std::sync::Arc; @@ -276,41 +282,41 @@ pub fn representative() -> Representative { // Add some disks to this first sled. let disks = vec![ // Let's say we have one manufacturer for our M.2... - sled_agent_client::types::InventoryDisk { + InventoryDisk { identity: omicron_common::disk::DiskIdentity { vendor: "macrohard".to_string(), model: "box".to_string(), serial: "XXIV".to_string(), }, - variant: sled_agent_client::types::DiskVariant::M2, + variant: DiskVariant::M2, slot: 0, }, // ... and a couple different vendors for our U.2s - sled_agent_client::types::InventoryDisk { + InventoryDisk { identity: omicron_common::disk::DiskIdentity { vendor: "memetendo".to_string(), model: "swatch".to_string(), serial: "0001".to_string(), }, - variant: sled_agent_client::types::DiskVariant::U2, + variant: DiskVariant::U2, slot: 1, }, - sled_agent_client::types::InventoryDisk { + InventoryDisk { identity: omicron_common::disk::DiskIdentity { vendor: "memetendo".to_string(), model: "swatch".to_string(), serial: "0002".to_string(), }, - variant: sled_agent_client::types::DiskVariant::U2, + variant: DiskVariant::U2, slot: 2, }, - sled_agent_client::types::InventoryDisk { + InventoryDisk { identity: omicron_common::disk::DiskIdentity { vendor: "tony".to_string(), model: "craystation".to_string(), serial: "5".to_string(), }, - variant: sled_agent_client::types::DiskVariant::U2, + variant: DiskVariant::U2, slot: 3, }, ]; @@ -321,12 +327,12 @@ pub fn representative() -> Representative { "fake sled agent 1", sled_agent( sled_agent_id_basic, - sled_agent_client::types::Baseboard::Gimlet { + Baseboard::Gimlet { identifier: String::from("s1"), model: String::from("model1"), revision: 0, }, - sled_agent_client::types::SledRole::Gimlet, + SledRole::Gimlet, disks, zpools, ), @@ -347,12 +353,12 @@ pub fn representative() -> Representative { "fake sled agent 4", sled_agent( sled_agent_id_extra, - sled_agent_client::types::Baseboard::Gimlet { + Baseboard::Gimlet { identifier: sled4_bb.serial_number.clone(), model: sled4_bb.part_number.clone(), revision: 0, }, - sled_agent_client::types::SledRole::Scrimlet, + SledRole::Scrimlet, vec![], vec![], ), @@ -369,11 +375,11 @@ pub fn representative() -> Representative { "fake sled agent 5", sled_agent( sled_agent_id_pc, - sled_agent_client::types::Baseboard::Pc { + Baseboard::Pc { identifier: String::from("fellofftruck1"), model: String::from("fellofftruck"), }, - sled_agent_client::types::SledRole::Gimlet, + SledRole::Gimlet, vec![], vec![], ), @@ -391,8 +397,8 @@ pub fn representative() -> Representative { "fake sled agent 6", sled_agent( sled_agent_id_unknown, - sled_agent_client::types::Baseboard::Unknown, - sled_agent_client::types::SledRole::Gimlet, + Baseboard::Unknown, + SledRole::Gimlet, vec![], vec![], ), @@ -501,12 +507,12 @@ pub fn rot_page(unique: &str) -> RotPage { pub fn sled_agent( sled_id: SledUuid, - baseboard: sled_agent_client::types::Baseboard, - sled_role: sled_agent_client::types::SledRole, - disks: Vec, - zpools: Vec, -) -> sled_agent_client::types::Inventory { - sled_agent_client::types::Inventory { + baseboard: Baseboard, + sled_role: SledRole, + disks: Vec, + zpools: Vec, +) -> Inventory { + Inventory { baseboard, reservoir_size: ByteCount::from(1024), sled_role, diff --git a/nexus/reconfigurator/execution/Cargo.toml b/nexus/reconfigurator/execution/Cargo.toml index 00103528bb..69f80209c3 100644 --- a/nexus/reconfigurator/execution/Cargo.toml +++ b/nexus/reconfigurator/execution/Cargo.toml @@ -20,6 +20,7 @@ nexus-config.workspace = true nexus-db-model.workspace = true nexus-db-queries.workspace = true nexus-networking.workspace = true +nexus-sled-agent-shared.workspace = true nexus-types.workspace = true omicron-common.workspace = true omicron-uuid-kinds.workspace = true diff --git a/nexus/reconfigurator/execution/src/datasets.rs b/nexus/reconfigurator/execution/src/datasets.rs index 6b7e30a738..6444934ba6 100644 --- a/nexus/reconfigurator/execution/src/datasets.rs +++ b/nexus/reconfigurator/execution/src/datasets.rs @@ -119,6 +119,7 @@ mod tests { use super::*; use nexus_db_model::Zpool; use nexus_reconfigurator_planning::example::example; + use nexus_sled_agent_shared::inventory::OmicronZoneDataset; use nexus_test_utils_macros::nexus_test; use nexus_types::deployment::blueprint_zone_type; use nexus_types::deployment::BlueprintZoneDisposition; @@ -127,7 +128,6 @@ mod tests { use omicron_common::zpool_name::ZpoolName; use omicron_uuid_kinds::GenericUuid; use omicron_uuid_kinds::ZpoolUuid; - use sled_agent_client::types::OmicronZoneDataset; use uuid::Uuid; type ControlPlaneTestContext = diff --git a/nexus/reconfigurator/execution/src/dns.rs b/nexus/reconfigurator/execution/src/dns.rs index bdcfc66bad..3504d41e4d 100644 --- a/nexus/reconfigurator/execution/src/dns.rs +++ b/nexus/reconfigurator/execution/src/dns.rs @@ -471,6 +471,7 @@ mod test { use nexus_reconfigurator_planning::blueprint_builder::EnsureMultiple; use nexus_reconfigurator_planning::example::example; use nexus_reconfigurator_preparation::PlanningInputFromDb; + use nexus_sled_agent_shared::inventory::ZoneKind; use nexus_test_utils::resource_helpers::create_silo; use nexus_test_utils::resource_helpers::DiskTestBuilder; use nexus_test_utils_macros::nexus_test; @@ -506,7 +507,6 @@ mod test { use omicron_uuid_kinds::ExternalIpUuid; use omicron_uuid_kinds::OmicronZoneUuid; use omicron_uuid_kinds::ZpoolUuid; - use sled_agent_client::ZoneKind; use std::collections::BTreeMap; use std::collections::BTreeSet; use std::collections::HashMap; diff --git a/nexus/reconfigurator/execution/src/external_networking.rs b/nexus/reconfigurator/execution/src/external_networking.rs index a451eeda0f..3e98aa4ff0 100644 --- a/nexus/reconfigurator/execution/src/external_networking.rs +++ b/nexus/reconfigurator/execution/src/external_networking.rs @@ -13,6 +13,7 @@ use nexus_db_queries::db::fixed_data::vpc_subnet::DNS_VPC_SUBNET; use nexus_db_queries::db::fixed_data::vpc_subnet::NEXUS_VPC_SUBNET; use nexus_db_queries::db::fixed_data::vpc_subnet::NTP_VPC_SUBNET; use nexus_db_queries::db::DataStore; +use nexus_sled_agent_shared::inventory::ZoneKind; use nexus_types::deployment::BlueprintZoneConfig; use nexus_types::deployment::OmicronZoneExternalIp; use omicron_common::api::external::IdentityMetadataCreateParams; @@ -20,7 +21,6 @@ use omicron_common::api::internal::shared::NetworkInterface; use omicron_common::api::internal::shared::NetworkInterfaceKind; use omicron_uuid_kinds::GenericUuid; use omicron_uuid_kinds::OmicronZoneUuid; -use sled_agent_client::ZoneKind; use slog::debug; use slog::error; use slog::info; @@ -40,7 +40,7 @@ pub(crate) async fn ensure_zone_external_networking_allocated( let log = opctx.log.new(slog::o!( "action" => "allocate-external-networking", - "zone_kind" => z.zone_type.kind().to_string(), + "zone_kind" => z.zone_type.kind().report_str(), "zone_id" => z.id.to_string(), "ip" => format!("{external_ip:?}"), "nic" => format!("{nic:?}"), @@ -75,7 +75,7 @@ pub(crate) async fn ensure_zone_external_networking_deallocated( let kind = z.zone_type.kind(); let log = opctx.log.new(slog::o!( "action" => "deallocate-external-networking", - "zone_kind" => z.zone_type.kind().to_string(), + "zone_kind" => kind.report_str(), "zone_id" => z.id.to_string(), "ip" => format!("{external_ip:?}"), "nic" => format!("{nic:?}"), @@ -87,7 +87,8 @@ pub(crate) async fn ensure_zone_external_networking_deallocated( .with_context(|| { format!( "failed to delete external IP {external_ip:?} \ - for {kind} zone {}", + for {} zone {}", + kind.report_str(), z.id ) })?; @@ -106,7 +107,8 @@ pub(crate) async fn ensure_zone_external_networking_deallocated( .await .with_context(|| { format!( - "failed to delete service VNIC {nic:?} for {kind} zone {}", + "failed to delete service VNIC {nic:?} for {} zone {}", + kind.report_str(), z.id ) })?; @@ -142,7 +144,10 @@ async fn is_external_ip_already_allocated( .external_ip_list_service(opctx, zone_id.into_untyped_uuid()) .await .with_context(|| { - format!("failed to look up external IPs for {zone_kind} {zone_id}") + format!( + "failed to look up external IPs for {} {zone_id}", + zone_kind.report_str() + ) })?; // We expect to find either 0 or exactly 1 IP for any given zone. If 0, @@ -212,7 +217,10 @@ async fn is_nic_already_allocated( .service_list_network_interfaces(opctx, zone_id.into_untyped_uuid()) .await .with_context(|| { - format!("failed to look up NICs for {zone_kind} {zone_id}") + format!( + "failed to look up NICs for {} {zone_id}", + zone_kind.report_str() + ) })?; if !allocated_nics.is_empty() { @@ -292,8 +300,8 @@ async fn ensure_external_service_ip( .await .with_context(|| { format!( - "failed to allocate IP to {zone_kind} {zone_id}: \ - {external_ip:?}" + "failed to allocate IP to {} {zone_id}: {external_ip:?}", + zone_kind.report_str() ) })?; @@ -336,7 +344,7 @@ async fn ensure_service_nic( | ZoneKind::InternalDns | ZoneKind::InternalNtp | ZoneKind::Oximeter => { - bail!("no VPC subnet available for {zone_kind} zone") + bail!("no VPC subnet available for {} zone", zone_kind.report_str()) } }; @@ -359,7 +367,7 @@ async fn ensure_service_nic( nic_subnet.clone(), IdentityMetadataCreateParams { name: nic.name.clone(), - description: format!("{zone_kind} service vNIC"), + description: format!("{} service vNIC", zone_kind.report_str()), }, nic.ip, nic.mac, @@ -376,8 +384,8 @@ async fn ensure_service_nic( .map_err(|err| err.into_external()) .with_context(|| { format!( - "failed to allocate NIC to {zone_kind} {service_id}: \ - {nic:?}" + "failed to allocate NIC to {} {service_id}: {nic:?}", + zone_kind.report_str() ) })?; @@ -408,7 +416,8 @@ async fn ensure_service_nic( // return a scary error here and expect to never see it. bail!( "database cleanup required: unexpected NIC ({created_nic:?}) \ - allocated for {zone_kind} {service_id}" + allocated for {} {service_id}", + zone_kind.report_str(), ); } @@ -426,12 +435,12 @@ mod tests { use nexus_config::NUM_INITIAL_RESERVED_IP_ADDRESSES; use nexus_db_model::SqlU16; use nexus_db_queries::db::queries::ALLOW_FULL_TABLE_SCAN_SQL; + use nexus_sled_agent_shared::inventory::OmicronZoneDataset; use nexus_test_utils_macros::nexus_test; use nexus_types::deployment::blueprint_zone_type; use nexus_types::deployment::BlueprintZoneConfig; use nexus_types::deployment::BlueprintZoneDisposition; use nexus_types::deployment::BlueprintZoneType; - use nexus_types::deployment::OmicronZoneDataset; use nexus_types::deployment::OmicronZoneExternalFloatingAddr; use nexus_types::deployment::OmicronZoneExternalFloatingIp; use nexus_types::deployment::OmicronZoneExternalSnatIp; diff --git a/nexus/reconfigurator/execution/src/omicron_zones.rs b/nexus/reconfigurator/execution/src/omicron_zones.rs index 404124ba25..acbb7a6b33 100644 --- a/nexus/reconfigurator/execution/src/omicron_zones.rs +++ b/nexus/reconfigurator/execution/src/omicron_zones.rs @@ -118,7 +118,7 @@ pub(crate) async fn clean_up_expunged_zones( let log = opctx.log.new(slog::o!( "sled_id" => sled_id.to_string(), "zone_id" => config.id.to_string(), - "zone_type" => config.zone_type.kind().to_string(), + "zone_type" => config.zone_type.kind().report_str(), )); let result = match &config.zone_type { @@ -309,12 +309,14 @@ mod test { use httptest::matchers::{all_of, json_decoded, request}; use httptest::responders::{json_encoded, status_code}; use httptest::Expectation; + use nexus_sled_agent_shared::inventory::{ + OmicronZoneDataset, OmicronZonesConfig, + }; use nexus_test_utils_macros::nexus_test; use nexus_types::deployment::{ blueprint_zone_type, Blueprint, BlueprintTarget, - CockroachDbPreserveDowngrade, OmicronZonesConfig, + CockroachDbPreserveDowngrade, }; - use nexus_types::inventory::OmicronZoneDataset; use omicron_common::api::external::Generation; use omicron_common::zpool_name::ZpoolName; use omicron_uuid_kinds::OmicronZoneUuid; diff --git a/nexus/reconfigurator/planning/Cargo.toml b/nexus/reconfigurator/planning/Cargo.toml index 989ad6aa32..a914f8a1e2 100644 --- a/nexus/reconfigurator/planning/Cargo.toml +++ b/nexus/reconfigurator/planning/Cargo.toml @@ -16,6 +16,7 @@ internal-dns.workspace = true ipnet.workspace = true nexus-config.workspace = true nexus-inventory.workspace = true +nexus-sled-agent-shared.workspace = true nexus-types.workspace = true omicron-common.workspace = true omicron-uuid-kinds.workspace = true diff --git a/nexus/reconfigurator/planning/src/blueprint_builder/builder.rs b/nexus/reconfigurator/planning/src/blueprint_builder/builder.rs index 93400a3708..09ae4132f3 100644 --- a/nexus/reconfigurator/planning/src/blueprint_builder/builder.rs +++ b/nexus/reconfigurator/planning/src/blueprint_builder/builder.rs @@ -12,6 +12,8 @@ use internal_dns::config::Host; use internal_dns::config::Zone; use ipnet::IpAdd; use nexus_inventory::now_db_precision; +use nexus_sled_agent_shared::inventory::OmicronZoneDataset; +use nexus_sled_agent_shared::inventory::ZoneKind; use nexus_types::deployment::blueprint_zone_type; use nexus_types::deployment::Blueprint; use nexus_types::deployment::BlueprintPhysicalDiskConfig; @@ -23,7 +25,6 @@ use nexus_types::deployment::BlueprintZoneType; use nexus_types::deployment::BlueprintZonesConfig; use nexus_types::deployment::CockroachDbPreserveDowngrade; use nexus_types::deployment::DiskFilter; -use nexus_types::deployment::OmicronZoneDataset; use nexus_types::deployment::OmicronZoneExternalFloatingIp; use nexus_types::deployment::PlanningInput; use nexus_types::deployment::SledDetails; @@ -51,7 +52,6 @@ use omicron_uuid_kinds::SledUuid; use omicron_uuid_kinds::ZpoolUuid; use rand::rngs::StdRng; use rand::SeedableRng; -use sled_agent_client::ZoneKind; use slog::debug; use slog::error; use slog::info; @@ -132,7 +132,7 @@ pub enum EnsureMultiple { /// "comment", identifying which operations have occurred on the blueprint. #[derive(Debug, Clone, Eq, PartialEq)] pub(crate) enum Operation { - AddZone { sled_id: SledUuid, kind: sled_agent_client::ZoneKind }, + AddZone { sled_id: SledUuid, kind: ZoneKind }, UpdateDisks { sled_id: SledUuid, added: usize, removed: usize }, ZoneExpunged { sled_id: SledUuid, reason: ZoneExpungeReason, count: usize }, } @@ -141,7 +141,7 @@ impl fmt::Display for Operation { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { match self { Self::AddZone { sled_id, kind } => { - write!(f, "sled {sled_id}: added zone: {kind}") + write!(f, "sled {sled_id}: added zone: {}", kind.report_str()) } Self::UpdateDisks { sled_id, added, removed } => { write!(f, "sled {sled_id}: added {added} disks, removed {removed} disks") diff --git a/nexus/reconfigurator/planning/src/planner.rs b/nexus/reconfigurator/planning/src/planner.rs index 08c25c20fd..509c6722cb 100644 --- a/nexus/reconfigurator/planning/src/planner.rs +++ b/nexus/reconfigurator/planning/src/planner.rs @@ -12,6 +12,7 @@ use crate::blueprint_builder::EnsureMultiple; use crate::blueprint_builder::Error; use crate::blueprint_builder::Operation; use crate::planner::omicron_zone_placement::PlacementError; +use nexus_sled_agent_shared::inventory::ZoneKind; use nexus_types::deployment::Blueprint; use nexus_types::deployment::BlueprintZoneConfig; use nexus_types::deployment::BlueprintZoneDisposition; @@ -26,7 +27,6 @@ use nexus_types::external_api::views::SledPolicy; use nexus_types::external_api::views::SledState; use nexus_types::inventory::Collection; use omicron_uuid_kinds::SledUuid; -use sled_agent_client::ZoneKind; use slog::error; use slog::{info, warn, Logger}; use std::collections::BTreeMap; @@ -714,6 +714,7 @@ mod test { use chrono::Utc; use expectorate::assert_contents; use nexus_inventory::now_db_precision; + use nexus_sled_agent_shared::inventory::ZoneKind; use nexus_types::deployment::blueprint_zone_type; use nexus_types::deployment::BlueprintDiff; use nexus_types::deployment::BlueprintZoneDisposition; @@ -737,7 +738,6 @@ mod test { use omicron_uuid_kinds::PhysicalDiskUuid; use omicron_uuid_kinds::SledUuid; use omicron_uuid_kinds::ZpoolUuid; - use sled_agent_client::ZoneKind; use std::collections::HashMap; use std::mem; use typed_rng::TypedUuidRng; diff --git a/nexus/reconfigurator/planning/src/planner/omicron_zone_placement.rs b/nexus/reconfigurator/planning/src/planner/omicron_zone_placement.rs index 08eccb0468..dcfb3b3150 100644 --- a/nexus/reconfigurator/planning/src/planner/omicron_zone_placement.rs +++ b/nexus/reconfigurator/planning/src/planner/omicron_zone_placement.rs @@ -4,9 +4,9 @@ //! Omicron zone placement decisions +use nexus_sled_agent_shared::inventory::ZoneKind; use nexus_types::deployment::BlueprintZoneType; use omicron_uuid_kinds::SledUuid; -use sled_agent_client::ZoneKind; use std::cmp::Ordering; use std::collections::BinaryHeap; use std::mem; @@ -56,7 +56,7 @@ impl From for ZoneKind { pub(super) enum PlacementError { #[error( "no sleds eligible for placement of new {} zone", - ZoneKind::from(*zone_kind) + ZoneKind::from(*zone_kind).report_str() )] NoSledsEligible { zone_kind: DiscretionaryOmicronZone }, } diff --git a/nexus/reconfigurator/planning/src/system.rs b/nexus/reconfigurator/planning/src/system.rs index 5f00ea8172..534d92bf08 100644 --- a/nexus/reconfigurator/planning/src/system.rs +++ b/nexus/reconfigurator/planning/src/system.rs @@ -10,6 +10,10 @@ use gateway_client::types::RotState; use gateway_client::types::SpState; use indexmap::IndexMap; use nexus_inventory::CollectionBuilder; +use nexus_sled_agent_shared::inventory::Baseboard; +use nexus_sled_agent_shared::inventory::Inventory; +use nexus_sled_agent_shared::inventory::InventoryDisk; +use nexus_sled_agent_shared::inventory::SledRole; use nexus_types::deployment::CockroachDbClusterVersion; use nexus_types::deployment::CockroachDbSettings; use nexus_types::deployment::PlanningInputBuilder; @@ -25,7 +29,6 @@ use nexus_types::external_api::views::SledState; use nexus_types::inventory::BaseboardId; use nexus_types::inventory::PowerState; use nexus_types::inventory::RotSlot; -use nexus_types::inventory::SledRole; use nexus_types::inventory::SpType; use omicron_common::address::get_sled_address; use omicron_common::address::IpRange; @@ -36,6 +39,7 @@ use omicron_common::address::SLED_PREFIX; use omicron_common::api::external::ByteCount; use omicron_common::api::external::Generation; use omicron_common::disk::DiskIdentity; +use omicron_common::disk::DiskVariant; use omicron_uuid_kinds::GenericUuid; use omicron_uuid_kinds::PhysicalDiskUuid; use omicron_uuid_kinds::SledUuid; @@ -443,7 +447,7 @@ struct Sled { sled_id: SledUuid, sled_subnet: Ipv6Subnet, inventory_sp: Option<(u16, SpState)>, - inventory_sled_agent: sled_agent_client::types::Inventory, + inventory_sled_agent: Inventory, zpools: BTreeMap, policy: SledPolicy, } @@ -517,23 +521,21 @@ impl Sled { let inventory_sled_agent = { let baseboard = match hardware { - SledHardware::Gimlet => { - sled_agent_client::types::Baseboard::Gimlet { - identifier: serial.clone(), - model: model.clone(), - revision, - } - } - SledHardware::Pc => sled_agent_client::types::Baseboard::Pc { + SledHardware::Gimlet => Baseboard::Gimlet { + identifier: serial.clone(), + model: model.clone(), + revision, + }, + SledHardware::Pc => Baseboard::Pc { identifier: serial.clone(), model: model.clone(), }, SledHardware::Unknown | SledHardware::Empty => { - sled_agent_client::types::Baseboard::Unknown + Baseboard::Unknown } }; - let sled_agent_address = get_sled_address(sled_subnet).to_string(); - sled_agent_client::types::Inventory { + let sled_agent_address = get_sled_address(sled_subnet); + Inventory { baseboard, reservoir_size: ByteCount::from(1024), sled_role, @@ -545,9 +547,9 @@ impl Sled { disks: zpools .values() .enumerate() - .map(|(i, d)| sled_agent_client::types::InventoryDisk { + .map(|(i, d)| InventoryDisk { identity: d.disk_identity.clone(), - variant: sled_agent_client::types::DiskVariant::U2, + variant: DiskVariant::U2, slot: i64::try_from(i).unwrap(), }) .collect(), @@ -588,12 +590,12 @@ impl Sled { // inventory types again. This is a little goofy. let baseboard = inventory_sp .as_ref() - .map(|sledhw| sled_agent_client::types::Baseboard::Gimlet { + .map(|sledhw| Baseboard::Gimlet { identifier: sledhw.baseboard_id.serial_number.clone(), model: sledhw.baseboard_id.part_number.clone(), revision: sledhw.sp.baseboard_revision, }) - .unwrap_or(sled_agent_client::types::Baseboard::Unknown); + .unwrap_or(Baseboard::Unknown); let inventory_sp = inventory_sp.map(|sledhw| { // RotStateV3 unconditionally sets all of these @@ -679,11 +681,11 @@ impl Sled { (sledhw.sp.sp_slot, sp_state) }); - let inventory_sled_agent = sled_agent_client::types::Inventory { + let inventory_sled_agent = Inventory { baseboard, reservoir_size: inv_sled_agent.reservoir_size, sled_role: inv_sled_agent.sled_role, - sled_agent_address: inv_sled_agent.sled_agent_address.to_string(), + sled_agent_address: inv_sled_agent.sled_agent_address, sled_id: sled_id.into_untyped_uuid(), usable_hardware_threads: inv_sled_agent.usable_hardware_threads, usable_physical_ram: inv_sled_agent.usable_physical_ram, @@ -705,7 +707,7 @@ impl Sled { self.inventory_sp.as_ref() } - fn sled_agent_inventory(&self) -> &sled_agent_client::types::Inventory { + fn sled_agent_inventory(&self) -> &Inventory { &self.inventory_sled_agent } } diff --git a/nexus/src/app/background/tasks/blueprint_execution.rs b/nexus/src/app/background/tasks/blueprint_execution.rs index f5d15eab3d..460d74360d 100644 --- a/nexus/src/app/background/tasks/blueprint_execution.rs +++ b/nexus/src/app/background/tasks/blueprint_execution.rs @@ -124,6 +124,7 @@ mod test { }; use nexus_db_queries::authn; use nexus_db_queries::context::OpContext; + use nexus_sled_agent_shared::inventory::OmicronZoneDataset; use nexus_test_utils_macros::nexus_test; use nexus_types::deployment::BlueprintZoneFilter; use nexus_types::deployment::{ @@ -132,7 +133,6 @@ mod test { BlueprintZoneType, BlueprintZonesConfig, CockroachDbPreserveDowngrade, }; use nexus_types::external_api::views::SledState; - use nexus_types::inventory::OmicronZoneDataset; use omicron_common::api::external::Generation; use omicron_common::zpool_name::ZpoolName; use omicron_uuid_kinds::GenericUuid; diff --git a/nexus/src/app/background/tasks/crdb_node_id_collector.rs b/nexus/src/app/background/tasks/crdb_node_id_collector.rs index d33dfe2634..e92b013ac2 100644 --- a/nexus/src/app/background/tasks/crdb_node_id_collector.rs +++ b/nexus/src/app/background/tasks/crdb_node_id_collector.rs @@ -238,6 +238,7 @@ mod tests { use httptest::Expectation; use nexus_db_queries::db::datastore::pub_test_utils::datastore_test; use nexus_reconfigurator_planning::blueprint_builder::BlueprintBuilder; + use nexus_sled_agent_shared::inventory::OmicronZoneDataset; use nexus_test_utils::db::test_setup_database; use nexus_types::deployment::BlueprintZoneConfig; use nexus_types::deployment::BlueprintZoneDisposition; @@ -277,7 +278,7 @@ mod tests { zone_type: BlueprintZoneType::CockroachDb( blueprint_zone_type::CockroachDb { address: addr, - dataset: nexus_types::inventory::OmicronZoneDataset { + dataset: OmicronZoneDataset { pool_name: format!("oxp_{}", zpool_id) .parse() .unwrap(), diff --git a/nexus/src/app/background/tasks/sync_service_zone_nat.rs b/nexus/src/app/background/tasks/sync_service_zone_nat.rs index 59cd6a6a79..eb9554cff7 100644 --- a/nexus/src/app/background/tasks/sync_service_zone_nat.rs +++ b/nexus/src/app/background/tasks/sync_service_zone_nat.rs @@ -18,10 +18,12 @@ use nexus_db_model::Ipv4NatValues; use nexus_db_queries::context::OpContext; use nexus_db_queries::db::lookup::LookupPath; use nexus_db_queries::db::DataStore; +use nexus_sled_agent_shared::inventory::{ + OmicronZoneConfig, OmicronZoneType, OmicronZonesConfig, +}; use omicron_common::address::{MAX_PORT, MIN_PORT}; use omicron_uuid_kinds::GenericUuid; use serde_json::json; -use sled_agent_client::types::OmicronZoneType; use std::net::{IpAddr, SocketAddr}; use std::sync::Arc; @@ -126,10 +128,8 @@ impl BackgroundTask for ServiceZoneNatTracker { let sled_address = oxnet::Ipv6Net::host_net(*sled.ip); - let zones_config: sled_agent_client::types::OmicronZonesConfig = - zones_found.zones; - let zones: Vec = - zones_config.zones; + let zones_config: OmicronZonesConfig = zones_found.zones; + let zones: Vec = zones_config.zones; for zone in zones { let zone_type: OmicronZoneType = zone.zone_type; @@ -201,19 +201,7 @@ impl BackgroundTask for ServiceZoneNatTracker { nexus_count += 1; }, OmicronZoneType::ExternalDns { nic, dns_address, .. } => { - let socket_addr: SocketAddr = match dns_address.parse() { - Ok(value) => value, - Err(e) => { - error!( - &log, - "failed to parse value into socketaddr"; - "value" => dns_address, - "error" => ?e, - ); - continue; - } - }; - let external_ip = match socket_addr { + let external_ip = match dns_address { SocketAddr::V4(v4) => { *v4.ip() }, diff --git a/nexus/src/app/sled.rs b/nexus/src/app/sled.rs index 6e21470368..261045670e 100644 --- a/nexus/src/app/sled.rs +++ b/nexus/src/app/sled.rs @@ -6,13 +6,14 @@ use crate::external_api::params; use crate::internal_api::params::{ - PhysicalDiskPutRequest, SledAgentInfo, SledRole, ZpoolPutRequest, + PhysicalDiskPutRequest, SledAgentInfo, ZpoolPutRequest, }; use nexus_db_queries::authz; use nexus_db_queries::context::OpContext; use nexus_db_queries::db; use nexus_db_queries::db::lookup; use nexus_db_queries::db::model::DatasetKind; +use nexus_sled_agent_shared::inventory::SledRole; use nexus_types::deployment::DiskFilter; use nexus_types::deployment::SledFilter; use nexus_types::external_api::views::PhysicalDiskPolicy; diff --git a/nexus/src/lib.rs b/nexus/src/lib.rs index 5d5e7d6eba..d5c853b15b 100644 --- a/nexus/src/lib.rs +++ b/nexus/src/lib.rs @@ -252,8 +252,8 @@ impl nexus_test_interface::NexusServer for Server { datasets: Vec, internal_dns_zone_config: nexus_types::internal_api::params::DnsConfigParams, external_dns_zone_name: &str, - recovery_silo: nexus_types::internal_api::params::RecoverySiloConfig, - certs: Vec, + recovery_silo: nexus_sled_agent_shared::recovery_silo::RecoverySiloConfig, + certs: Vec, disable_sled_id: Uuid, ) -> Self { // Perform the "handoff from RSS". diff --git a/nexus/test-interface/Cargo.toml b/nexus/test-interface/Cargo.toml index 03e38d3687..52298c1267 100644 --- a/nexus/test-interface/Cargo.toml +++ b/nexus/test-interface/Cargo.toml @@ -10,6 +10,7 @@ workspace = true [dependencies] async-trait.workspace = true nexus-config.workspace = true +nexus-sled-agent-shared.workspace = true nexus-types.workspace = true omicron-common.workspace = true slog.workspace = true diff --git a/nexus/test-interface/src/lib.rs b/nexus/test-interface/src/lib.rs index 06c5570b7b..c1e01cbc9e 100644 --- a/nexus/test-interface/src/lib.rs +++ b/nexus/test-interface/src/lib.rs @@ -62,8 +62,10 @@ pub trait NexusServer: Send + Sync + 'static { datasets: Vec, internal_dns_config: nexus_types::internal_api::params::DnsConfigParams, external_dns_zone_name: &str, - recovery_silo: nexus_types::internal_api::params::RecoverySiloConfig, - tls_certificates: Vec, + recovery_silo: nexus_sled_agent_shared::recovery_silo::RecoverySiloConfig, + tls_certificates: Vec< + omicron_common::api::internal::nexus::Certificate, + >, disable_sled_id: Uuid, ) -> Self; diff --git a/nexus/test-utils/Cargo.toml b/nexus/test-utils/Cargo.toml index 7732e00d70..a883bc83c5 100644 --- a/nexus/test-utils/Cargo.toml +++ b/nexus/test-utils/Cargo.toml @@ -27,6 +27,7 @@ illumos-utils.workspace = true internal-dns.workspace = true nexus-config.workspace = true nexus-db-queries.workspace = true +nexus-sled-agent-shared.workspace = true nexus-test-interface.workspace = true nexus-types.workspace = true omicron-common.workspace = true diff --git a/nexus/test-utils/src/lib.rs b/nexus/test-utils/src/lib.rs index 960ded50d5..7c190974a1 100644 --- a/nexus/test-utils/src/lib.rs +++ b/nexus/test-utils/src/lib.rs @@ -23,6 +23,9 @@ use nexus_config::InternalDns; use nexus_config::MgdConfig; use nexus_config::NexusConfig; use nexus_config::NUM_INITIAL_RESERVED_IP_ADDRESSES; +use nexus_sled_agent_shared::inventory::OmicronZoneDataset; +use nexus_sled_agent_shared::inventory::OmicronZonesConfig; +use nexus_sled_agent_shared::recovery_silo::RecoverySiloConfig; use nexus_test_interface::NexusServer; use nexus_types::deployment::blueprint_zone_type; use nexus_types::deployment::Blueprint; @@ -33,23 +36,20 @@ use nexus_types::deployment::BlueprintZonesConfig; use nexus_types::deployment::CockroachDbPreserveDowngrade; use nexus_types::deployment::OmicronZoneExternalFloatingAddr; use nexus_types::deployment::OmicronZoneExternalFloatingIp; -use nexus_types::external_api::params::UserId; use nexus_types::external_api::views::SledState; -use nexus_types::internal_api::params::Certificate; use nexus_types::internal_api::params::DatasetCreateRequest; -use nexus_types::internal_api::params::DatasetKind; use nexus_types::internal_api::params::DatasetPutRequest; -use nexus_types::internal_api::params::RecoverySiloConfig; -use nexus_types::inventory::OmicronZoneDataset; -use nexus_types::inventory::OmicronZonesConfig; use omicron_common::address::DNS_OPTE_IPV4_SUBNET; use omicron_common::address::NEXUS_OPTE_IPV4_SUBNET; use omicron_common::api::external::Generation; use omicron_common::api::external::MacAddr; +use omicron_common::api::external::UserId; use omicron_common::api::external::Vni; use omicron_common::api::external::{IdentityMetadata, Name}; +use omicron_common::api::internal::nexus::Certificate; use omicron_common::api::internal::nexus::ProducerEndpoint; use omicron_common::api::internal::nexus::ProducerKind; +use omicron_common::api::internal::shared::DatasetKind; use omicron_common::api::internal::shared::NetworkInterface; use omicron_common::api::internal::shared::NetworkInterfaceKind; use omicron_common::api::internal::shared::SwitchLocation; diff --git a/nexus/test-utils/src/resource_helpers.rs b/nexus/test-utils/src/resource_helpers.rs index 1a92a6ef8e..ac7188f232 100644 --- a/nexus/test-utils/src/resource_helpers.rs +++ b/nexus/test-utils/src/resource_helpers.rs @@ -15,7 +15,6 @@ use http::StatusCode; use nexus_db_queries::db::fixed_data::silo::DEFAULT_SILO; use nexus_test_interface::NexusServer; use nexus_types::external_api::params; -use nexus_types::external_api::params::UserId; use nexus_types::external_api::shared; use nexus_types::external_api::shared::Baseboard; use nexus_types::external_api::shared::IdentityType; @@ -40,6 +39,7 @@ use omicron_common::api::external::NameOrId; use omicron_common::api::external::RouteDestination; use omicron_common::api::external::RouteTarget; use omicron_common::api::external::RouterRoute; +use omicron_common::api::external::UserId; use omicron_common::disk::DiskIdentity; use omicron_sled_agent::sim::SledAgent; use omicron_test_utils::dev::poll::wait_for_condition; diff --git a/nexus/tests/integration_tests/certificates.rs b/nexus/tests/integration_tests/certificates.rs index 5a34caab49..ab9566d4ad 100644 --- a/nexus/tests/integration_tests/certificates.rs +++ b/nexus/tests/integration_tests/certificates.rs @@ -19,8 +19,8 @@ use nexus_test_utils_macros::nexus_test; use nexus_types::external_api::params; use nexus_types::external_api::shared; use nexus_types::external_api::views::Certificate; -use nexus_types::internal_api::params as internal_params; use omicron_common::api::external::IdentityMetadataCreateParams; +use omicron_common::api::internal::nexus::Certificate as InternalCertificate; use omicron_test_utils::certificates::CertificateChain; use omicron_test_utils::dev::poll::wait_for_condition; use omicron_test_utils::dev::poll::CondCheckError; @@ -645,7 +645,7 @@ struct SiloCert { silo_name: oxide_client::types::Name, dns_name: String, cert_name: oxide_client::types::Name, - cert: internal_params::Certificate, + cert: InternalCertificate, } impl SiloCert { @@ -659,7 +659,7 @@ impl SiloCert { dns_name.clone(), ])); let cert_name = format!("cert-{}", silo_name.as_str()).parse().unwrap(); - let cert = internal_params::Certificate { + let cert = InternalCertificate { cert: chain.cert_chain_as_pem(), key: chain.end_cert_private_key_as_pem(), }; diff --git a/nexus/tests/integration_tests/endpoints.rs b/nexus/tests/integration_tests/endpoints.rs index 52d9e14e35..6e4e59688a 100644 --- a/nexus/tests/integration_tests/endpoints.rs +++ b/nexus/tests/integration_tests/endpoints.rs @@ -33,6 +33,7 @@ use omicron_common::api::external::Name; use omicron_common::api::external::NameOrId; use omicron_common::api::external::RouteDestination; use omicron_common::api::external::RouteTarget; +use omicron_common::api::external::UserId; use omicron_common::api::external::VpcFirewallRuleUpdateParams; use omicron_test_utils::certificates::CertificateChain; use once_cell::sync::Lazy; @@ -877,7 +878,7 @@ pub static DEMO_TIMESERIES_QUERY: Lazy = // Users pub static DEMO_USER_CREATE: Lazy = Lazy::new(|| params::UserCreate { - external_id: params::UserId::from_str("dummy-user").unwrap(), + external_id: UserId::from_str("dummy-user").unwrap(), password: params::UserPassword::LoginDisallowed, }); diff --git a/nexus/tests/integration_tests/password_login.rs b/nexus/tests/integration_tests/password_login.rs index a7b0b627b9..23f86484c9 100644 --- a/nexus/tests/integration_tests/password_login.rs +++ b/nexus/tests/integration_tests/password_login.rs @@ -11,7 +11,7 @@ use nexus_test_utils_macros::nexus_test; use nexus_types::external_api::params; use nexus_types::external_api::shared::{self, SiloRole}; use nexus_types::external_api::views; -use omicron_common::api::external::Name; +use omicron_common::api::external::{Name, UserId}; use omicron_passwords::MIN_EXPECTED_PASSWORD_VERIFY_TIME; use std::str::FromStr; @@ -61,13 +61,13 @@ async fn test_local_user_basic(client: &ClientTestContext, silo: &views::Silo) { expect_login_failure( client, &silo_name, - params::UserId::from_str("bigfoot").unwrap(), + UserId::from_str("bigfoot").unwrap(), params::Password::from_str("ahh").unwrap(), ) .await; // Create a test user with a known password. - let test_user = params::UserId::from_str("abe-simpson").unwrap(); + let test_user = UserId::from_str("abe-simpson").unwrap(); let test_password = params::Password::from_str("let me in you idiot!").unwrap(); @@ -159,7 +159,7 @@ async fn test_local_user_basic(client: &ClientTestContext, silo: &views::Silo) { // Now, let's create an admin user and verify that they can change this // user's password. - let admin_user = params::UserId::from_str("comic-book-guy").unwrap(); + let admin_user = UserId::from_str("comic-book-guy").unwrap(); let admin_password = params::Password::from_str("toodle-ooh").unwrap(); let admin_user_obj = create_local_user( client, @@ -306,7 +306,7 @@ async fn test_local_user_with_no_initial_password( let silo_name = &silo.identity.name; // Create a user with no initial password. - let test_user = params::UserId::from_str("steven-falken").unwrap(); + let test_user = UserId::from_str("steven-falken").unwrap(); let created_user = create_local_user( client, silo, @@ -385,7 +385,7 @@ async fn expect_session_invalid( async fn expect_login_failure( client: &ClientTestContext, silo_name: &Name, - username: params::UserId, + username: UserId, password: params::Password, ) { let start = std::time::Instant::now(); @@ -425,7 +425,7 @@ async fn expect_login_failure( async fn expect_login_success( client: &ClientTestContext, silo_name: &Name, - username: params::UserId, + username: UserId, password: params::Password, ) -> String { let start = std::time::Instant::now(); diff --git a/nexus/tests/integration_tests/rack.rs b/nexus/tests/integration_tests/rack.rs index c72c59b6f7..2cc7dc8977 100644 --- a/nexus/tests/integration_tests/rack.rs +++ b/nexus/tests/integration_tests/rack.rs @@ -9,6 +9,7 @@ use nexus_client::types::SledId; use nexus_db_model::SledBaseboard; use nexus_db_model::SledSystemHardware; use nexus_db_model::SledUpdate; +use nexus_sled_agent_shared::inventory::SledRole; use nexus_test_utils::http_testing::AuthnMode; use nexus_test_utils::http_testing::NexusRequest; use nexus_test_utils::http_testing::RequestBuilder; @@ -18,7 +19,6 @@ use nexus_types::external_api::params; use nexus_types::external_api::shared::UninitializedSled; use nexus_types::external_api::views::Rack; use nexus_types::internal_api::params::SledAgentInfo; -use nexus_types::internal_api::params::SledRole; use omicron_common::api::external::ByteCount; use omicron_common::api::external::Generation; use omicron_uuid_kinds::GenericUuid; diff --git a/nexus/tests/integration_tests/silos.rs b/nexus/tests/integration_tests/silos.rs index 2e6c21bb79..2c861ff159 100644 --- a/nexus/tests/integration_tests/silos.rs +++ b/nexus/tests/integration_tests/silos.rs @@ -25,10 +25,10 @@ use nexus_types::external_api::views::{ }; use nexus_types::external_api::{params, shared}; use omicron_common::address::{IpRange, Ipv4Range}; -use omicron_common::api::external::ObjectIdentity; use omicron_common::api::external::{ IdentityMetadataCreateParams, LookupType, Name, }; +use omicron_common::api::external::{ObjectIdentity, UserId}; use omicron_test_utils::certificates::CertificateChain; use omicron_test_utils::dev::poll::{wait_for_condition, CondCheckError}; @@ -1776,7 +1776,7 @@ async fn test_jit_silo_constraints(cptestctx: &ControlPlaneTestContext) { Method::POST, "/v1/system/identity-providers/local/users?silo=jit", ¶ms::UserCreate { - external_id: params::UserId::from_str("dummy").unwrap(), + external_id: UserId::from_str("dummy").unwrap(), password: params::UserPassword::LoginDisallowed, }, ) @@ -1836,7 +1836,7 @@ async fn test_jit_silo_constraints(cptestctx: &ControlPlaneTestContext) { Method::POST, "/v1/login/jit/local", ¶ms::UsernamePasswordCredentials { - username: params::UserId::from_str(admin_username).unwrap(), + username: UserId::from_str(admin_username).unwrap(), password: password.clone(), }, )) @@ -1849,7 +1849,7 @@ async fn test_jit_silo_constraints(cptestctx: &ControlPlaneTestContext) { Method::POST, "/v1/login/jit/local", ¶ms::UsernamePasswordCredentials { - username: params::UserId::from_str("bogus").unwrap(), + username: UserId::from_str("bogus").unwrap(), password: password.clone(), }, )) @@ -2047,7 +2047,7 @@ async fn run_user_tests( client, &url_user_create, ¶ms::UserCreate { - external_id: params::UserId::from_str("a-test-user").unwrap(), + external_id: UserId::from_str("a-test-user").unwrap(), password: params::UserPassword::LoginDisallowed, }, ) diff --git a/nexus/types/Cargo.toml b/nexus/types/Cargo.toml index df976e2444..a4418d2a74 100644 --- a/nexus/types/Cargo.toml +++ b/nexus/types/Cargo.toml @@ -36,10 +36,13 @@ uuid.workspace = true api_identity.workspace = true dns-service-client.workspace = true gateway-client.workspace = true +nexus-sled-agent-shared.workspace = true omicron-common.workspace = true omicron-passwords.workspace = true omicron-workspace-hack.workspace = true -sled-agent-client.workspace = true +# Note: we're trying to avoid a dependency from nexus-types to sled-agent-types +# because the correct direction of dependency is unclear. If there are types +# common to both, put them in `omicron-common` or `nexus-sled-agent-shared`. [dev-dependencies] proptest.workspace = true diff --git a/nexus/types/src/deployment.rs b/nexus/types/src/deployment.rs index 6f6c10a9c2..4342adb02b 100644 --- a/nexus/types/src/deployment.rs +++ b/nexus/types/src/deployment.rs @@ -15,16 +15,17 @@ use crate::external_api::views::SledState; use crate::internal_api::params::DnsConfigParams; use crate::inventory::Collection; -pub use crate::inventory::OmicronZoneConfig; -pub use crate::inventory::OmicronZoneDataset; -pub use crate::inventory::OmicronZoneType; -pub use crate::inventory::OmicronZonesConfig; pub use crate::inventory::SourceNatConfig; pub use crate::inventory::ZpoolName; use derive_more::From; use newtype_uuid::GenericUuid; +use nexus_sled_agent_shared::inventory::OmicronZoneConfig; +use nexus_sled_agent_shared::inventory::OmicronZoneType; +use nexus_sled_agent_shared::inventory::OmicronZonesConfig; +use nexus_sled_agent_shared::inventory::ZoneKind; use omicron_common::api::external::Generation; use omicron_common::disk::DiskIdentity; +use omicron_common::disk::OmicronPhysicalDisksConfig; use omicron_uuid_kinds::CollectionUuid; use omicron_uuid_kinds::ExternalIpUuid; use omicron_uuid_kinds::OmicronZoneUuid; @@ -32,12 +33,10 @@ use omicron_uuid_kinds::SledUuid; use schemars::JsonSchema; use serde::Deserialize; use serde::Serialize; -use sled_agent_client::types::OmicronPhysicalDisksConfig; use slog_error_chain::SlogInlineError; use std::collections::BTreeMap; use std::collections::BTreeSet; use std::fmt; -use std::net::AddrParseError; use std::net::Ipv6Addr; use strum::EnumIter; use strum::IntoEnumIterator; @@ -74,7 +73,6 @@ pub use planning_input::SledDisk; pub use planning_input::SledFilter; pub use planning_input::SledResources; pub use planning_input::ZpoolFilter; -pub use sled_agent_client::ZoneKind; pub use zone_type::blueprint_zone_type; pub use zone_type::BlueprintZoneType; pub use zone_type::DurableDataset; @@ -341,7 +339,7 @@ impl BpSledSubtableData for BlueprintOrCollectionZonesConfig { BpSledSubtableRow::from_strings( state, vec![ - zone.kind().to_string(), + zone.kind().report_str().to_string(), zone.id().to_string(), zone.disposition().to_string(), zone.underlay_address().to_string(), @@ -597,20 +595,10 @@ fn zone_sort_key(z: &T) -> impl Ord { (z.kind(), z.id()) } -/// "Should never happen" errors from converting an [`OmicronZoneType`] into a -/// [`BlueprintZoneType`]. -// Removing this error type would be a side effect of fixing -// https://github.com/oxidecomputer/omicron/issues/4988. +/// Errors from converting an [`OmicronZoneType`] into a [`BlueprintZoneType`]. #[derive(Debug, Clone, Error, SlogInlineError)] pub enum InvalidOmicronZoneType { - #[error("invalid socket address for Omicron zone {kind} ({addr})")] - ParseSocketAddr { - kind: ZoneKind, - addr: String, - #[source] - err: AddrParseError, - }, - #[error("Omicron zone {kind} requires an external IP ID")] + #[error("Omicron zone {} requires an external IP ID", kind.report_str())] ExternalIpIdRequired { kind: ZoneKind }, } @@ -658,13 +646,6 @@ impl BlueprintZoneConfig { let external_ip_id = external_ip_id.ok_or( InvalidOmicronZoneType::ExternalIpIdRequired { kind }, )?; - let address = address.parse().map_err(|err| { - InvalidOmicronZoneType::ParseSocketAddr { - kind, - addr: address.clone(), - err, - } - })?; BlueprintZoneType::BoundaryNtp( blueprint_zone_type::BoundaryNtp { address, @@ -680,63 +661,28 @@ impl BlueprintZoneConfig { ) } OmicronZoneType::Clickhouse { address, dataset } => { - let address = address.parse().map_err(|err| { - InvalidOmicronZoneType::ParseSocketAddr { - kind, - addr: address.clone(), - err, - } - })?; BlueprintZoneType::Clickhouse(blueprint_zone_type::Clickhouse { address, dataset, }) } OmicronZoneType::ClickhouseKeeper { address, dataset } => { - let address = address.parse().map_err(|err| { - InvalidOmicronZoneType::ParseSocketAddr { - kind, - addr: address.clone(), - err, - } - })?; BlueprintZoneType::ClickhouseKeeper( blueprint_zone_type::ClickhouseKeeper { address, dataset }, ) } OmicronZoneType::CockroachDb { address, dataset } => { - let address = address.parse().map_err(|err| { - InvalidOmicronZoneType::ParseSocketAddr { - kind, - addr: address.clone(), - err, - } - })?; BlueprintZoneType::CockroachDb( blueprint_zone_type::CockroachDb { address, dataset }, ) } OmicronZoneType::Crucible { address, dataset } => { - let address = address.parse().map_err(|err| { - InvalidOmicronZoneType::ParseSocketAddr { - kind, - addr: address.clone(), - err, - } - })?; BlueprintZoneType::Crucible(blueprint_zone_type::Crucible { address, dataset, }) } OmicronZoneType::CruciblePantry { address } => { - let address = address.parse().map_err(|err| { - InvalidOmicronZoneType::ParseSocketAddr { - kind, - addr: address.clone(), - err, - } - })?; BlueprintZoneType::CruciblePantry( blueprint_zone_type::CruciblePantry { address }, ) @@ -750,20 +696,6 @@ impl BlueprintZoneConfig { let external_ip_id = external_ip_id.ok_or( InvalidOmicronZoneType::ExternalIpIdRequired { kind }, )?; - let dns_address = dns_address.parse().map_err(|err| { - InvalidOmicronZoneType::ParseSocketAddr { - kind, - addr: dns_address.clone(), - err, - } - })?; - let http_address = http_address.parse().map_err(|err| { - InvalidOmicronZoneType::ParseSocketAddr { - kind, - addr: http_address.clone(), - err, - } - })?; BlueprintZoneType::ExternalDns( blueprint_zone_type::ExternalDns { dataset, @@ -782,53 +714,28 @@ impl BlueprintZoneConfig { gz_address, gz_address_index, http_address, - } => { - let dns_address = dns_address.parse().map_err(|err| { - InvalidOmicronZoneType::ParseSocketAddr { - kind, - addr: dns_address.clone(), - err, - } - })?; - let http_address = http_address.parse().map_err(|err| { - InvalidOmicronZoneType::ParseSocketAddr { - kind, - addr: http_address.clone(), - err, - } - })?; - BlueprintZoneType::InternalDns( - blueprint_zone_type::InternalDns { - dataset, - http_address, - dns_address, - gz_address, - gz_address_index, - }, - ) - } + } => BlueprintZoneType::InternalDns( + blueprint_zone_type::InternalDns { + dataset, + http_address, + dns_address, + gz_address, + gz_address_index, + }, + ), OmicronZoneType::InternalNtp { address, dns_servers, domain, ntp_servers, - } => { - let address = address.parse().map_err(|err| { - InvalidOmicronZoneType::ParseSocketAddr { - kind, - addr: address.clone(), - err, - } - })?; - BlueprintZoneType::InternalNtp( - blueprint_zone_type::InternalNtp { - address, - ntp_servers, - dns_servers, - domain, - }, - ) - } + } => BlueprintZoneType::InternalNtp( + blueprint_zone_type::InternalNtp { + address, + ntp_servers, + dns_servers, + domain, + }, + ), OmicronZoneType::Nexus { external_dns_servers, external_ip, @@ -839,14 +746,6 @@ impl BlueprintZoneConfig { let external_ip_id = external_ip_id.ok_or( InvalidOmicronZoneType::ExternalIpIdRequired { kind }, )?; - let internal_address = - internal_address.parse().map_err(|err| { - InvalidOmicronZoneType::ParseSocketAddr { - kind, - addr: internal_address.clone(), - err, - } - })?; BlueprintZoneType::Nexus(blueprint_zone_type::Nexus { internal_address, external_ip: OmicronZoneExternalFloatingIp { @@ -859,13 +758,6 @@ impl BlueprintZoneConfig { }) } OmicronZoneType::Oximeter { address } => { - let address = address.parse().map_err(|err| { - InvalidOmicronZoneType::ParseSocketAddr { - kind, - addr: address.clone(), - err, - } - })?; BlueprintZoneType::Oximeter(blueprint_zone_type::Oximeter { address, }) @@ -1025,10 +917,10 @@ pub enum BlueprintZoneFilter { /// /// Part of [`Blueprint`]. pub type BlueprintPhysicalDisksConfig = - sled_agent_client::types::OmicronPhysicalDisksConfig; + omicron_common::disk::OmicronPhysicalDisksConfig; pub type BlueprintPhysicalDiskConfig = - sled_agent_client::types::OmicronPhysicalDiskConfig; + omicron_common::disk::OmicronPhysicalDiskConfig; /// Describe high-level metadata about a blueprint // These fields are a subset of [`Blueprint`], and include only the data we can diff --git a/nexus/types/src/deployment/blueprint_diff.rs b/nexus/types/src/deployment/blueprint_diff.rs index 4f2ee9ed3b..7cfeba360e 100644 --- a/nexus/types/src/deployment/blueprint_diff.rs +++ b/nexus/types/src/deployment/blueprint_diff.rs @@ -11,11 +11,11 @@ use super::blueprint_display::{ BpSledSubtableRow, KvListWithHeading, KvPair, }; use super::{zone_sort_key, CockroachDbPreserveDowngrade}; +use nexus_sled_agent_shared::inventory::ZoneKind; use omicron_common::api::external::Generation; use omicron_common::disk::DiskIdentity; use omicron_uuid_kinds::OmicronZoneUuid; use omicron_uuid_kinds::SledUuid; -use sled_agent_client::ZoneKind; use std::collections::{BTreeMap, BTreeSet}; use std::fmt; @@ -51,7 +51,7 @@ impl BpSledSubtableData for BpDiffZoneDetails { BpSledSubtableRow::from_strings( state, vec![ - zone.kind().to_string(), + zone.kind().report_str().to_string(), zone.id().to_string(), zone.disposition().to_string(), zone.underlay_address().to_string(), @@ -93,8 +93,8 @@ impl ModifiedZone { if before.kind() != after.kind() { let msg = format!( "mismatched zone kind: before: {}, after: {}\n", - before.kind(), - after.kind() + before.kind().report_str(), + after.kind().report_str(), ); reason.push_str(&msg); } @@ -152,7 +152,9 @@ impl BpSledSubtableData for BpDiffZonesModified { BpSledSubtableRow::new( state, vec![ - BpSledSubtableColumn::value(zone.zone.kind().to_string()), + BpSledSubtableColumn::value( + zone.zone.kind().report_str().to_string(), + ), BpSledSubtableColumn::value(zone.zone.id().to_string()), BpSledSubtableColumn::diff( zone.prior_disposition.to_string(), diff --git a/nexus/types/src/deployment/zone_type.rs b/nexus/types/src/deployment/zone_type.rs index eb0b2dc126..789a0215b7 100644 --- a/nexus/types/src/deployment/zone_type.rs +++ b/nexus/types/src/deployment/zone_type.rs @@ -9,14 +9,14 @@ //! that is not needed by sled-agent. use super::OmicronZoneExternalIp; -use crate::internal_api::params::DatasetKind; +use nexus_sled_agent_shared::inventory::OmicronZoneDataset; +use nexus_sled_agent_shared::inventory::OmicronZoneType; +use nexus_sled_agent_shared::inventory::ZoneKind; +use omicron_common::api::internal::shared::DatasetKind; use omicron_common::api::internal::shared::NetworkInterface; use schemars::JsonSchema; use serde::Deserialize; use serde::Serialize; -use sled_agent_client::types::OmicronZoneDataset; -use sled_agent_client::types::OmicronZoneType; -use sled_agent_client::ZoneKind; use std::net::SocketAddrV6; #[derive(Debug, Clone, Eq, PartialEq, JsonSchema, Deserialize, Serialize)] @@ -168,7 +168,7 @@ impl From for OmicronZoneType { fn from(zone_type: BlueprintZoneType) -> Self { match zone_type { BlueprintZoneType::BoundaryNtp(zone) => Self::BoundaryNtp { - address: zone.address.to_string(), + address: zone.address, ntp_servers: zone.ntp_servers, dns_servers: zone.dns_servers, domain: zone.domain, @@ -176,54 +176,53 @@ impl From for OmicronZoneType { snat_cfg: zone.external_ip.snat_cfg, }, BlueprintZoneType::Clickhouse(zone) => Self::Clickhouse { - address: zone.address.to_string(), + address: zone.address, dataset: zone.dataset, }, BlueprintZoneType::ClickhouseKeeper(zone) => { Self::ClickhouseKeeper { - address: zone.address.to_string(), + address: zone.address, dataset: zone.dataset, } } BlueprintZoneType::CockroachDb(zone) => Self::CockroachDb { - address: zone.address.to_string(), - dataset: zone.dataset, - }, - BlueprintZoneType::Crucible(zone) => Self::Crucible { - address: zone.address.to_string(), + address: zone.address, dataset: zone.dataset, }, + BlueprintZoneType::Crucible(zone) => { + Self::Crucible { address: zone.address, dataset: zone.dataset } + } BlueprintZoneType::CruciblePantry(zone) => { - Self::CruciblePantry { address: zone.address.to_string() } + Self::CruciblePantry { address: zone.address } } BlueprintZoneType::ExternalDns(zone) => Self::ExternalDns { dataset: zone.dataset, - http_address: zone.http_address.to_string(), - dns_address: zone.dns_address.addr.to_string(), + http_address: zone.http_address, + dns_address: zone.dns_address.addr, nic: zone.nic, }, BlueprintZoneType::InternalDns(zone) => Self::InternalDns { dataset: zone.dataset, - http_address: zone.http_address.to_string(), - dns_address: zone.dns_address.to_string(), + http_address: zone.http_address, + dns_address: zone.dns_address, gz_address: zone.gz_address, gz_address_index: zone.gz_address_index, }, BlueprintZoneType::InternalNtp(zone) => Self::InternalNtp { - address: zone.address.to_string(), + address: zone.address, ntp_servers: zone.ntp_servers, dns_servers: zone.dns_servers, domain: zone.domain, }, BlueprintZoneType::Nexus(zone) => Self::Nexus { - internal_address: zone.internal_address.to_string(), + internal_address: zone.internal_address, external_ip: zone.external_ip.ip, nic: zone.nic, external_tls: zone.external_tls, external_dns_servers: zone.external_dns_servers, }, BlueprintZoneType::Oximeter(zone) => { - Self::Oximeter { address: zone.address.to_string() } + Self::Oximeter { address: zone.address } } } } @@ -252,7 +251,7 @@ pub mod blueprint_zone_type { use crate::deployment::OmicronZoneExternalFloatingAddr; use crate::deployment::OmicronZoneExternalFloatingIp; use crate::deployment::OmicronZoneExternalSnatIp; - use crate::inventory::OmicronZoneDataset; + use nexus_sled_agent_shared::inventory::OmicronZoneDataset; use omicron_common::api::internal::shared::NetworkInterface; use schemars::JsonSchema; use serde::Deserialize; diff --git a/nexus/types/src/external_api/params.rs b/nexus/types/src/external_api/params.rs index 1776045ffd..8dcce913b3 100644 --- a/nexus/types/src/external_api/params.rs +++ b/nexus/types/src/external_api/params.rs @@ -12,8 +12,9 @@ use omicron_common::api::external::{ AddressLotKind, AllowedSourceIps, BfdMode, BgpPeer, ByteCount, Hostname, IdentityMetadataCreateParams, IdentityMetadataUpdateParams, InstanceCpuCount, LinkFec, LinkSpeed, Name, NameOrId, PaginationOrder, - RouteDestination, RouteTarget, SemverVersion, + RouteDestination, RouteTarget, SemverVersion, UserId, }; +use omicron_common::disk::DiskVariant; use oxnet::{IpNet, Ipv4Net, Ipv6Net}; use schemars::JsonSchema; use serde::{ @@ -392,48 +393,6 @@ pub struct UserCreate { pub password: UserPassword, } -/// A username for a local-only user -#[derive(Clone, Debug, Deserialize, Serialize)] -#[serde(try_from = "String")] -pub struct UserId(String); - -impl AsRef for UserId { - fn as_ref(&self) -> &str { - self.0.as_ref() - } -} - -impl FromStr for UserId { - type Err = String; - fn from_str(value: &str) -> Result { - UserId::try_from(String::from(value)) - } -} - -/// Used to impl `Deserialize` -impl TryFrom for UserId { - type Error = String; - fn try_from(value: String) -> Result { - // Mostly, this validation exists to cap the input size. The specific - // length is not critical here. For convenience and consistency, we use - // the same rules as `Name`. - let _ = Name::try_from(value.clone())?; - Ok(UserId(value)) - } -} - -impl JsonSchema for UserId { - fn schema_name() -> String { - "UserId".to_string() - } - - fn json_schema( - gen: &mut schemars::gen::SchemaGenerator, - ) -> schemars::schema::Schema { - Name::json_schema(gen) - } -} - /// A password used for authenticating a local-only user #[derive(Clone, Deserialize, Serialize)] #[serde(try_from = "String")] @@ -1350,12 +1309,11 @@ pub enum PhysicalDiskKind { U2, } -impl From for PhysicalDiskKind { - fn from(variant: sled_agent_client::types::DiskVariant) -> Self { - use sled_agent_client::types::DiskVariant; - match variant { - DiskVariant::U2 => Self::U2, - DiskVariant::M2 => Self::M2, +impl From for PhysicalDiskKind { + fn from(dv: DiskVariant) -> Self { + match dv { + DiskVariant::M2 => PhysicalDiskKind::M2, + DiskVariant::U2 => PhysicalDiskKind::U2, } } } diff --git a/nexus/types/src/internal_api/params.rs b/nexus/types/src/internal_api/params.rs index 143ca1be8b..3a26dde4ba 100644 --- a/nexus/types/src/internal_api/params.rs +++ b/nexus/types/src/internal_api/params.rs @@ -6,14 +6,17 @@ use crate::deployment::Blueprint; use crate::external_api::params::PhysicalDiskKind; -use crate::external_api::params::UserId; use crate::external_api::shared::Baseboard; use crate::external_api::shared::IpRange; +use nexus_sled_agent_shared::inventory::SledRole; +use nexus_sled_agent_shared::recovery_silo::RecoverySiloConfig; use omicron_common::api::external::ByteCount; use omicron_common::api::external::Generation; use omicron_common::api::external::MacAddr; use omicron_common::api::external::Name; +use omicron_common::api::internal::nexus::Certificate; use omicron_common::api::internal::shared::AllowedSourceIps; +use omicron_common::api::internal::shared::DatasetKind; use omicron_common::api::internal::shared::ExternalPortDiscovery; use omicron_common::api::internal::shared::RackNetworkConfig; use omicron_common::api::internal::shared::SourceNatConfig; @@ -25,20 +28,6 @@ use std::net::SocketAddr; use std::net::SocketAddrV6; use uuid::Uuid; -/// Describes the role of the sled within the rack. -/// -/// Note that this may change if the sled is physically moved -/// within the rack. -#[derive(Serialize, Deserialize, JsonSchema, Debug)] -#[serde(rename_all = "snake_case")] -pub enum SledRole { - /// The sled is a general compute sled. - Gimlet, - /// The sled is attached to the network switch, and has additional - /// responsibilities. - Scrimlet, -} - /// Sent by a sled agent to Nexus to inform about resources #[derive(Serialize, Deserialize, Debug, JsonSchema)] pub struct SledAgentInfo { @@ -102,35 +91,6 @@ pub struct ZpoolPutRequest { pub physical_disk_id: Uuid, } -/// Describes the purpose of the dataset. -#[derive( - Debug, Serialize, Deserialize, JsonSchema, Clone, Copy, PartialEq, Eq, -)] -#[serde(rename_all = "snake_case")] -pub enum DatasetKind { - Crucible, - Cockroach, - Clickhouse, - ClickhouseKeeper, - ExternalDns, - InternalDns, -} - -impl fmt::Display for DatasetKind { - fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { - use DatasetKind::*; - let s = match self { - Crucible => "crucible", - Cockroach => "cockroach", - Clickhouse => "clickhouse", - ClickhouseKeeper => "clickhouse_keeper", - ExternalDns => "external_dns", - InternalDns => "internal_dns", - }; - write!(f, "{}", s) - } -} - /// Describes a dataset within a pool. #[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)] pub struct DatasetPutRequest { @@ -201,21 +161,6 @@ pub struct DatasetCreateRequest { pub request: DatasetPutRequest, } -#[derive(Clone, Serialize, Deserialize, JsonSchema)] -pub struct Certificate { - pub cert: String, - pub key: String, -} - -impl std::fmt::Debug for Certificate { - fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - f.debug_struct("Certificate") - .field("cert", &self.cert) - .field("key", &"") - .finish() - } -} - #[derive(Debug, Clone, Deserialize, JsonSchema)] pub struct RackInitializationRequest { /// Blueprint describing services initialized by RSS. @@ -253,13 +198,6 @@ pub type DnsConfigZone = dns_service_client::types::DnsConfigZone; pub type DnsRecord = dns_service_client::types::DnsRecord; pub type Srv = dns_service_client::types::Srv; -#[derive(Debug, Clone, Deserialize, JsonSchema)] -pub struct RecoverySiloConfig { - pub silo_name: Name, - pub user_name: UserId, - pub user_password_hash: omicron_passwords::NewPasswordHash, -} - /// Message used to notify Nexus that this oximeter instance is up and running. #[derive(Debug, Clone, Copy, JsonSchema, Serialize, Deserialize)] pub struct OximeterInfo { diff --git a/nexus/types/src/inventory.rs b/nexus/types/src/inventory.rs index 661c4c088d..0ec2f6fbdb 100644 --- a/nexus/types/src/inventory.rs +++ b/nexus/types/src/inventory.rs @@ -11,13 +11,17 @@ use crate::external_api::params::PhysicalDiskKind; use crate::external_api::params::UninitializedSledId; -use crate::external_api::shared::Baseboard; use chrono::DateTime; use chrono::Utc; pub use gateway_client::types::PowerState; pub use gateway_client::types::RotImageError; pub use gateway_client::types::RotSlot; pub use gateway_client::types::SpType; +use nexus_sled_agent_shared::inventory::InventoryDisk; +use nexus_sled_agent_shared::inventory::InventoryZpool; +use nexus_sled_agent_shared::inventory::OmicronZoneConfig; +use nexus_sled_agent_shared::inventory::OmicronZonesConfig; +use nexus_sled_agent_shared::inventory::SledRole; use omicron_common::api::external::ByteCount; pub use omicron_common::api::internal::shared::NetworkInterface; pub use omicron_common::api::internal::shared::NetworkInterfaceKind; @@ -28,11 +32,6 @@ use omicron_uuid_kinds::SledUuid; use omicron_uuid_kinds::ZpoolUuid; use serde::{Deserialize, Serialize}; use serde_with::serde_as; -pub use sled_agent_client::types::OmicronZoneConfig; -pub use sled_agent_client::types::OmicronZoneDataset; -pub use sled_agent_client::types::OmicronZoneType; -pub use sled_agent_client::types::OmicronZonesConfig; -pub use sled_agent_client::types::SledRole; use std::collections::BTreeMap; use std::collections::BTreeSet; use std::net::SocketAddrV6; @@ -178,8 +177,8 @@ pub struct BaseboardId { pub serial_number: String, } -impl From for BaseboardId { - fn from(value: Baseboard) -> Self { +impl From for BaseboardId { + fn from(value: crate::external_api::shared::Baseboard) -> Self { BaseboardId { part_number: value.part, serial_number: value.serial } } } @@ -364,13 +363,17 @@ impl IntoRotPage for gateway_client::types::RotCfpa { /// the disk is being actively managed by the control plane. #[derive(Clone, Debug, PartialEq, Eq, Deserialize, Serialize)] pub struct PhysicalDisk { + // XXX: Should this just be InventoryDisk? Do we need a separation between + // InventoryDisk and PhysicalDisk? The types are structurally the same, but + // maybe the separation is useful to indicate that a `PhysicalDisk` doesn't + // always show up in the inventory. pub identity: omicron_common::disk::DiskIdentity, pub variant: PhysicalDiskKind, pub slot: i64, } -impl From for PhysicalDisk { - fn from(disk: sled_agent_client::types::InventoryDisk) -> PhysicalDisk { +impl From for PhysicalDisk { + fn from(disk: InventoryDisk) -> PhysicalDisk { PhysicalDisk { identity: disk.identity, variant: disk.variant.into(), @@ -388,10 +391,7 @@ pub struct Zpool { } impl Zpool { - pub fn new( - time_collected: DateTime, - pool: sled_agent_client::types::InventoryZpool, - ) -> Zpool { + pub fn new(time_collected: DateTime, pool: InventoryZpool) -> Zpool { Zpool { time_collected, id: pool.id, total_size: pool.total_size } } } diff --git a/openapi/bootstrap-agent.json b/openapi/bootstrap-agent.json index f268662ca9..7b4f257670 100644 --- a/openapi/bootstrap-agent.json +++ b/openapi/bootstrap-agent.json @@ -538,7 +538,6 @@ ] }, "Certificate": { - "description": "Certificate\n\n
JSON schema\n\n```json { \"type\": \"object\", \"required\": [ \"cert\", \"key\" ], \"properties\": { \"cert\": { \"type\": \"string\" }, \"key\": { \"type\": \"string\" } } } ```
", "type": "object", "properties": { "cert": { @@ -1155,7 +1154,6 @@ ] }, "RecoverySiloConfig": { - "description": "RecoverySiloConfig\n\n
JSON schema\n\n```json { \"type\": \"object\", \"required\": [ \"silo_name\", \"user_name\", \"user_password_hash\" ], \"properties\": { \"silo_name\": { \"$ref\": \"#/components/schemas/Name\" }, \"user_name\": { \"$ref\": \"#/components/schemas/UserId\" }, \"user_password_hash\": { \"$ref\": \"#/components/schemas/NewPasswordHash\" } } } ```
", "type": "object", "properties": { "silo_name": { @@ -1255,8 +1253,12 @@ ] }, "UserId": { - "description": "Names must begin with a lower case ASCII letter, be composed exclusively of lowercase ASCII, uppercase ASCII, numbers, and '-', and may not end with a '-'. Names cannot be a UUID, but they may contain a UUID. They can be at most 63 characters long.\n\n
JSON schema\n\n```json { \"title\": \"A name unique within the parent collection\", \"description\": \"Names must begin with a lower case ASCII letter, be composed exclusively of lowercase ASCII, uppercase ASCII, numbers, and '-', and may not end with a '-'. Names cannot be a UUID, but they may contain a UUID. They can be at most 63 characters long.\", \"type\": \"string\", \"maxLength\": 63, \"minLength\": 1, \"pattern\": \"^(?![0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$)^[a-z]([a-zA-Z0-9-]*[a-zA-Z0-9]+)?$\" } ```
", - "type": "string" + "title": "A username for a local-only user", + "description": "Usernames must begin with a lower case ASCII letter, be composed exclusively of lowercase ASCII, uppercase ASCII, numbers, and '-', and may not end with a '-'. Usernames cannot be a UUID, but they may contain a UUID. They can be at most 63 characters long.", + "type": "string", + "pattern": "^(?![0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$)^[a-z]([a-zA-Z0-9-]*[a-zA-Z0-9]+)?$", + "minLength": 1, + "maxLength": 63 } }, "responses": { diff --git a/openapi/nexus-internal.json b/openapi/nexus-internal.json index 257a505401..4e1a11aff1 100644 --- a/openapi/nexus-internal.json +++ b/openapi/nexus-internal.json @@ -3691,7 +3691,6 @@ "type": "string" }, "OmicronPhysicalDiskConfig": { - "description": "OmicronPhysicalDiskConfig\n\n
JSON schema\n\n```json { \"type\": \"object\", \"required\": [ \"id\", \"identity\", \"pool_id\" ], \"properties\": { \"id\": { \"type\": \"string\", \"format\": \"uuid\" }, \"identity\": { \"$ref\": \"#/components/schemas/DiskIdentity\" }, \"pool_id\": { \"$ref\": \"#/components/schemas/TypedUuidForZpoolKind\" } } } ```
", "type": "object", "properties": { "id": { @@ -3712,7 +3711,6 @@ ] }, "OmicronPhysicalDisksConfig": { - "description": "OmicronPhysicalDisksConfig\n\n
JSON schema\n\n```json { \"type\": \"object\", \"required\": [ \"disks\", \"generation\" ], \"properties\": { \"disks\": { \"type\": \"array\", \"items\": { \"$ref\": \"#/components/schemas/OmicronPhysicalDiskConfig\" } }, \"generation\": { \"description\": \"generation number of this configuration\\n\\nThis generation number is owned by the control plane (i.e., RSS or Nexus, depending on whether RSS-to-Nexus handoff has happened). It should not be bumped within Sled Agent.\\n\\nSled Agent rejects attempts to set the configuration to a generation older than the one it's currently running.\", \"allOf\": [ { \"$ref\": \"#/components/schemas/Generation\" } ] } } } ```
", "type": "object", "properties": { "disks": { @@ -3736,7 +3734,7 @@ ] }, "OmicronZoneDataset": { - "description": "Describes a persistent ZFS dataset associated with an Omicron zone\n\n
JSON schema\n\n```json { \"description\": \"Describes a persistent ZFS dataset associated with an Omicron zone\", \"type\": \"object\", \"required\": [ \"pool_name\" ], \"properties\": { \"pool_name\": { \"$ref\": \"#/components/schemas/ZpoolName\" } } } ```
", + "description": "Describes a persistent ZFS dataset associated with an Omicron zone", "type": "object", "properties": { "pool_name": { @@ -5087,8 +5085,8 @@ ] }, "UserId": { - "title": "A name unique within the parent collection", - "description": "Names must begin with a lower case ASCII letter, be composed exclusively of lowercase ASCII, uppercase ASCII, numbers, and '-', and may not end with a '-'. Names cannot be a UUID, but they may contain a UUID. They can be at most 63 characters long.", + "title": "A username for a local-only user", + "description": "Usernames must begin with a lower case ASCII letter, be composed exclusively of lowercase ASCII, uppercase ASCII, numbers, and '-', and may not end with a '-'. Usernames cannot be a UUID, but they may contain a UUID. They can be at most 63 characters long.", "type": "string", "pattern": "^(?![0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$)^[a-z]([a-zA-Z0-9-]*[a-zA-Z0-9]+)?$", "minLength": 1, diff --git a/openapi/nexus.json b/openapi/nexus.json index a4baa8124f..a6b45ec6c0 100644 --- a/openapi/nexus.json +++ b/openapi/nexus.json @@ -19928,8 +19928,8 @@ ] }, "UserId": { - "title": "A name unique within the parent collection", - "description": "Names must begin with a lower case ASCII letter, be composed exclusively of lowercase ASCII, uppercase ASCII, numbers, and '-', and may not end with a '-'. Names cannot be a UUID, but they may contain a UUID. They can be at most 63 characters long.", + "title": "A username for a local-only user", + "description": "Usernames must begin with a lower case ASCII letter, be composed exclusively of lowercase ASCII, uppercase ASCII, numbers, and '-', and may not end with a '-'. Usernames cannot be a UUID, but they may contain a UUID. They can be at most 63 characters long.", "type": "string", "pattern": "^(?![0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$)^[a-z]([a-zA-Z0-9-]*[a-zA-Z0-9]+)?$", "minLength": 1, diff --git a/openapi/sled-agent.json b/openapi/sled-agent.json index 1323769da2..42e40961ba 100644 --- a/openapi/sled-agent.json +++ b/openapi/sled-agent.json @@ -4656,7 +4656,7 @@ ] }, "SledRole": { - "description": "Describes the role of the sled within the rack.\n\nNote that this may change if the sled is physically moved within the rack.\n\n
JSON schema\n\n```json { \"description\": \"Describes the role of the sled within the rack.\\n\\nNote that this may change if the sled is physically moved within the rack.\", \"oneOf\": [ { \"description\": \"The sled is a general compute sled.\", \"type\": \"string\", \"enum\": [ \"gimlet\" ] }, { \"description\": \"The sled is attached to the network switch, and has additional responsibilities.\", \"type\": \"string\", \"enum\": [ \"scrimlet\" ] } ] } ```
", + "description": "Describes the role of the sled within the rack.\n\nNote that this may change if the sled is physically moved within the rack.", "oneOf": [ { "description": "The sled is a general compute sled.", diff --git a/schema/rss-sled-plan.json b/schema/rss-sled-plan.json index 1abba084ac..a3d3425870 100644 --- a/schema/rss-sled-plan.json +++ b/schema/rss-sled-plan.json @@ -428,7 +428,6 @@ ] }, "Certificate": { - "description": "Certificate\n\n
JSON schema\n\n```json { \"type\": \"object\", \"required\": [ \"cert\", \"key\" ], \"properties\": { \"cert\": { \"type\": \"string\" }, \"key\": { \"type\": \"string\" } } } ```
", "type": "object", "required": [ "cert", @@ -862,7 +861,6 @@ } }, "RecoverySiloConfig": { - "description": "RecoverySiloConfig\n\n
JSON schema\n\n```json { \"type\": \"object\", \"required\": [ \"silo_name\", \"user_name\", \"user_password_hash\" ], \"properties\": { \"silo_name\": { \"$ref\": \"#/components/schemas/Name\" }, \"user_name\": { \"$ref\": \"#/components/schemas/UserId\" }, \"user_password_hash\": { \"$ref\": \"#/components/schemas/NewPasswordHash\" } } } ```
", "type": "object", "required": [ "silo_name", @@ -1018,8 +1016,12 @@ } }, "UserId": { - "description": "Names must begin with a lower case ASCII letter, be composed exclusively of lowercase ASCII, uppercase ASCII, numbers, and '-', and may not end with a '-'. Names cannot be a UUID, but they may contain a UUID. They can be at most 63 characters long.\n\n
JSON schema\n\n```json { \"title\": \"A name unique within the parent collection\", \"description\": \"Names must begin with a lower case ASCII letter, be composed exclusively of lowercase ASCII, uppercase ASCII, numbers, and '-', and may not end with a '-'. Names cannot be a UUID, but they may contain a UUID. They can be at most 63 characters long.\", \"type\": \"string\", \"maxLength\": 63, \"minLength\": 1, \"pattern\": \"^(?![0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$)^[a-z]([a-zA-Z0-9-]*[a-zA-Z0-9]+)?$\" } ```
", - "type": "string" + "title": "A username for a local-only user", + "description": "Usernames must begin with a lower case ASCII letter, be composed exclusively of lowercase ASCII, uppercase ASCII, numbers, and '-', and may not end with a '-'. Usernames cannot be a UUID, but they may contain a UUID. They can be at most 63 characters long.", + "type": "string", + "maxLength": 63, + "minLength": 1, + "pattern": "^(?![0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$)^[a-z]([a-zA-Z0-9-]*[a-zA-Z0-9]+)?$" } } } \ No newline at end of file diff --git a/sled-agent/Cargo.toml b/sled-agent/Cargo.toml index 7747bb768e..52889d8fa2 100644 --- a/sled-agent/Cargo.toml +++ b/sled-agent/Cargo.toml @@ -49,6 +49,7 @@ macaddr.workspace = true mg-admin-client.workspace = true nexus-client.workspace = true nexus-config.workspace = true +nexus-sled-agent-shared.workspace = true nexus-types.workspace = true omicron-common.workspace = true omicron-ddm-admin-client.workspace = true diff --git a/sled-agent/src/bin/sled-agent-sim.rs b/sled-agent/src/bin/sled-agent-sim.rs index fcd7d02623..a8f240a4a1 100644 --- a/sled-agent/src/bin/sled-agent-sim.rs +++ b/sled-agent/src/bin/sled-agent-sim.rs @@ -12,7 +12,7 @@ use clap::Parser; use dropshot::ConfigDropshot; use dropshot::ConfigLogging; use dropshot::ConfigLoggingLevel; -use nexus_client::types as NexusTypes; +use omicron_common::api::internal::nexus::Certificate; use omicron_common::cmd::fatal; use omicron_common::cmd::CmdError; use omicron_sled_agent::sim::RssArgs; @@ -126,23 +126,22 @@ async fn do_run() -> Result<(), CmdError> { ) }; - let tls_certificate = match (args.rss_tls_cert, args.rss_tls_key) { - (None, None) => None, - (Some(cert_path), Some(key_path)) => { - let cert_bytes = std::fs::read_to_string(&cert_path) - .with_context(|| format!("read {:?}", &cert_path)) - .map_err(CmdError::Failure)?; - let key_bytes = std::fs::read_to_string(&key_path) - .with_context(|| format!("read {:?}", &key_path)) - .map_err(CmdError::Failure)?; - Some(NexusTypes::Certificate { cert: cert_bytes, key: key_bytes }) - } - _ => { - return Err(CmdError::Usage(String::from( + let tls_certificate = + match (args.rss_tls_cert, args.rss_tls_key) { + (None, None) => None, + (Some(cert_path), Some(key_path)) => { + let cert_bytes = std::fs::read_to_string(&cert_path) + .with_context(|| format!("read {:?}", &cert_path)) + .map_err(CmdError::Failure)?; + let key_bytes = std::fs::read_to_string(&key_path) + .with_context(|| format!("read {:?}", &key_path)) + .map_err(CmdError::Failure)?; + Some(Certificate { cert: cert_bytes, key: key_bytes }) + } + _ => return Err(CmdError::Usage(String::from( "--rss-tls-key and --rss-tls-cert must be specified together", - ))) - } - }; + ))), + }; let rss_args = RssArgs { nexus_external_addr: args.rss_nexus_external_addr, diff --git a/sled-agent/src/dump_setup.rs b/sled-agent/src/dump_setup.rs index 02d40195cf..b05667691e 100644 --- a/sled-agent/src/dump_setup.rs +++ b/sled-agent/src/dump_setup.rs @@ -90,7 +90,7 @@ use illumos_utils::dumpadm::{DumpAdm, DumpContentType}; use illumos_utils::zone::ZONE_PREFIX; use illumos_utils::zpool::{ZpoolHealth, ZpoolName}; use illumos_utils::ExecutionError; -use sled_hardware::DiskVariant; +use omicron_common::disk::DiskVariant; use sled_storage::config::MountConfig; use sled_storage::dataset::{CRASH_DATASET, DUMP_DATASET}; use sled_storage::disk::Disk; diff --git a/sled-agent/src/http_entrypoints.rs b/sled-agent/src/http_entrypoints.rs index 1ecda51657..23c897a643 100644 --- a/sled-agent/src/http_entrypoints.rs +++ b/sled-agent/src/http_entrypoints.rs @@ -9,8 +9,7 @@ use crate::bootstrap::params::AddSledRequest; use crate::params::{ BootstoreStatus, CleanupContextUpdate, DiskEnsureBody, InstanceEnsureBody, InstanceExternalIpBody, InstancePutMigrationIdsBody, InstancePutStateBody, - InstancePutStateResponse, InstanceUnregisterResponse, Inventory, - OmicronPhysicalDisksConfig, OmicronZonesConfig, SledRole, TimeSync, + InstancePutStateResponse, InstanceUnregisterResponse, TimeSync, VpcFirewallRulesEnsureBody, ZoneBundleId, ZoneBundleMetadata, Zpool, }; use crate::sled_agent::Error as SledAgentError; @@ -26,6 +25,9 @@ use dropshot::{ }; use illumos_utils::opte::params::VirtualNetworkInterfaceHost; use installinator_common::M2Slot; +use nexus_sled_agent_shared::inventory::{ + Inventory, OmicronZonesConfig, SledRole, +}; use omicron_common::api::external::Error; use omicron_common::api::internal::nexus::{ DiskRuntimeState, SledInstanceState, UpdateArtifactId, @@ -33,11 +35,11 @@ use omicron_common::api::internal::nexus::{ use omicron_common::api::internal::shared::{ ResolvedVpcRouteSet, ResolvedVpcRouteState, SledIdentifiers, SwitchPorts, }; +use omicron_common::disk::{DiskVariant, OmicronPhysicalDisksConfig}; use omicron_uuid_kinds::{GenericUuid, InstanceUuid}; use schemars::JsonSchema; use serde::{Deserialize, Serialize}; use sled_agent_types::early_networking::EarlyNetworkConfig; -use sled_hardware::DiskVariant; use sled_storage::resources::DisksManagementResult; use std::collections::BTreeMap; use uuid::Uuid; diff --git a/sled-agent/src/nexus.rs b/sled-agent/src/nexus.rs index 12fcc05ce3..5db25e0e52 100644 --- a/sled-agent/src/nexus.rs +++ b/sled-agent/src/nexus.rs @@ -4,6 +4,7 @@ pub use nexus_client::Client as NexusClient; use omicron_common::api::external::Generation; +use omicron_common::disk::DiskVariant; use crate::vmm_reservoir::VmmReservoirManagerHandle; use internal_dns::resolver::{ResolveError, Resolver}; @@ -127,15 +128,13 @@ pub(crate) trait ConvertInto: Sized { fn convert(self) -> T; } -impl ConvertInto - for sled_hardware::DiskVariant -{ +impl ConvertInto for DiskVariant { fn convert(self) -> nexus_client::types::PhysicalDiskKind { use nexus_client::types::PhysicalDiskKind; match self { - sled_hardware::DiskVariant::U2 => PhysicalDiskKind::U2, - sled_hardware::DiskVariant::M2 => PhysicalDiskKind::M2, + DiskVariant::U2 => PhysicalDiskKind::U2, + DiskVariant::M2 => PhysicalDiskKind::M2, } } } @@ -152,24 +151,6 @@ impl ConvertInto } } -impl ConvertInto - for sled_storage::dataset::DatasetKind -{ - fn convert(self) -> nexus_client::types::DatasetKind { - use nexus_client::types::DatasetKind; - use sled_storage::dataset::DatasetKind::*; - - match self { - CockroachDb => DatasetKind::Cockroach, - Crucible => DatasetKind::Crucible, - Clickhouse => DatasetKind::Clickhouse, - ClickhouseKeeper => DatasetKind::ClickhouseKeeper, - ExternalDns => DatasetKind::ExternalDns, - InternalDns => DatasetKind::InternalDns, - } - } -} - // Somewhat arbitrary bound size, large enough that we should never hit it. const QUEUE_SIZE: usize = 256; diff --git a/sled-agent/src/params.rs b/sled-agent/src/params.rs index 4bddbeaff8..f98d5c82e5 100644 --- a/sled-agent/src/params.rs +++ b/sled-agent/src/params.rs @@ -9,9 +9,7 @@ pub use crate::zone_bundle::ZoneBundleMetadata; pub use illumos_utils::opte::params::DhcpConfig; pub use illumos_utils::opte::params::VpcFirewallRule; pub use illumos_utils::opte::params::VpcFirewallRulesEnsureBody; -use illumos_utils::zpool::ZpoolName; -use omicron_common::api::external::ByteCount; -use omicron_common::api::external::Generation; +use nexus_sled_agent_shared::inventory::{OmicronZoneConfig, OmicronZoneType}; use omicron_common::api::internal::nexus::{ DiskRuntimeState, InstanceProperties, InstanceRuntimeState, SledInstanceState, VmmRuntimeState, @@ -19,18 +17,18 @@ use omicron_common::api::internal::nexus::{ use omicron_common::api::internal::shared::{ NetworkInterface, SourceNatConfig, }; +use omicron_common::disk::DiskVariant; use omicron_uuid_kinds::PropolisUuid; use omicron_uuid_kinds::ZpoolUuid; use schemars::JsonSchema; use serde::{Deserialize, Serialize}; pub use sled_hardware::DendriteAsic; use sled_hardware_types::Baseboard; -use sled_storage::dataset::DatasetKind; use sled_storage::dataset::DatasetName; +use sled_storage::dataset::DatasetType; use std::collections::BTreeSet; use std::fmt::{Debug, Display, Formatter, Result as FormatResult}; -use std::net::{IpAddr, Ipv6Addr, SocketAddr, SocketAddrV6}; -use std::str::FromStr; +use std::net::{IpAddr, SocketAddr, SocketAddrV6}; use std::time::Duration; use uuid::Uuid; @@ -241,12 +239,11 @@ pub enum DiskType { M2, } -impl From for DiskType { - fn from(v: sled_hardware::DiskVariant) -> Self { - use sled_hardware::DiskVariant::*; +impl From for DiskType { + fn from(v: DiskVariant) -> Self { match v { - U2 => Self::U2, - M2 => Self::M2, + DiskVariant::U2 => Self::U2, + DiskVariant::M2 => Self::M2, } } } @@ -257,300 +254,63 @@ pub struct Zpool { pub disk_type: DiskType, } -/// The type of zone that Sled Agent may run -#[derive( - Clone, Debug, Deserialize, Serialize, JsonSchema, PartialEq, Eq, Hash, -)] -#[serde(rename_all = "snake_case")] -pub enum ZoneType { - Clickhouse, - ClickhouseKeeper, - CockroachDb, - CruciblePantry, - Crucible, - ExternalDns, - InternalDns, - Nexus, - Ntp, - Oximeter, - Switch, -} - -impl std::fmt::Display for ZoneType { - fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - use ZoneType::*; - let name = match self { - Clickhouse => "clickhouse", - ClickhouseKeeper => "clickhouse_keeper", - CockroachDb => "cockroachdb", - Crucible => "crucible", - CruciblePantry => "crucible_pantry", - ExternalDns => "external_dns", - InternalDns => "internal_dns", - Nexus => "nexus", - Ntp => "ntp", - Oximeter => "oximeter", - Switch => "switch", - }; - write!(f, "{name}") - } -} - -pub type OmicronPhysicalDiskConfig = - sled_storage::disk::OmicronPhysicalDiskConfig; -pub type OmicronPhysicalDisksConfig = - sled_storage::disk::OmicronPhysicalDisksConfig; - -/// Describes the set of Omicron-managed zones running on a sled -#[derive( - Clone, Debug, Deserialize, Serialize, JsonSchema, PartialEq, Eq, Hash, -)] -pub struct OmicronZonesConfig { - /// generation number of this configuration - /// - /// This generation number is owned by the control plane (i.e., RSS or - /// Nexus, depending on whether RSS-to-Nexus handoff has happened). It - /// should not be bumped within Sled Agent. - /// - /// Sled Agent rejects attempts to set the configuration to a generation - /// older than the one it's currently running. - pub generation: Generation, - - /// list of running zones - pub zones: Vec, -} - -impl OmicronZonesConfig { - /// Generation 1 of `OmicronZonesConfig` is always the set of no zones. - pub const INITIAL_GENERATION: Generation = Generation::from_u32(1); -} - -impl From for sled_agent_client::types::OmicronZonesConfig { - fn from(local: OmicronZonesConfig) -> Self { - Self { - generation: local.generation, - zones: local.zones.into_iter().map(|s| s.into()).collect(), - } - } -} - -/// Describes one Omicron-managed zone running on a sled -#[derive( - Clone, Debug, Deserialize, Serialize, JsonSchema, PartialEq, Eq, Hash, -)] -pub struct OmicronZoneConfig { - pub id: Uuid, - pub underlay_address: Ipv6Addr, - - /// The pool on which we'll place this zone's filesystem. - /// - /// Note that this is transient -- the sled agent is permitted to - /// destroy the zone's dataset on this pool each time the zone is - /// initialized. - pub filesystem_pool: Option, - pub zone_type: OmicronZoneType, -} - -impl From for sled_agent_client::types::OmicronZoneConfig { - fn from(local: OmicronZoneConfig) -> Self { - Self { - id: local.id, - underlay_address: local.underlay_address, - filesystem_pool: local.filesystem_pool, - zone_type: local.zone_type.into(), - } - } +/// Extension trait for `OmicronZoneConfig`. +/// +/// This lives here because it is pretty specific to sled-agent, and also +/// requires extra dependencies that nexus-sled-agent-shared doesn't have. +pub(crate) trait OmicronZoneConfigExt { + fn zone_name(&self) -> String; } -impl OmicronZoneConfig { - /// If this kind of zone has an associated dataset, returns the dataset's - /// name. Othrwise, returns `None`. - pub fn dataset_name(&self) -> Option { - self.zone_type.dataset_name() - } - - /// If this kind of zone has an associated dataset, return the dataset's - /// name and the associated "service address". Otherwise, returns `None`. - pub fn dataset_name_and_address( - &self, - ) -> Option<(DatasetName, SocketAddrV6)> { - self.zone_type.dataset_name_and_address() - } - - /// Returns the name that is (or will be) used for the illumos zone - /// associated with this zone - pub fn zone_name(&self) -> String { +impl OmicronZoneConfigExt for OmicronZoneConfig { + fn zone_name(&self) -> String { illumos_utils::running_zone::InstalledZone::get_zone_name( - &self.zone_type.zone_type_str(), + self.zone_type.kind().zone_prefix(), Some(self.id), ) } } -/// Describes a persistent ZFS dataset associated with an Omicron zone -#[derive( - Clone, Debug, Deserialize, Serialize, JsonSchema, PartialEq, Eq, Hash, -)] -pub struct OmicronZoneDataset { - pub pool_name: ZpoolName, -} - -impl From for sled_agent_client::types::OmicronZoneDataset { - fn from(local: OmicronZoneDataset) -> Self { - Self { - pool_name: omicron_common::zpool_name::ZpoolName::from_str( - &local.pool_name.to_string(), - ) - .unwrap(), - } - } -} - -/// Describes what kind of zone this is (i.e., what component is running in it) -/// as well as any type-specific configuration -#[derive( - Clone, Debug, Deserialize, Serialize, JsonSchema, PartialEq, Eq, Hash, -)] -#[serde(tag = "type", rename_all = "snake_case")] -pub enum OmicronZoneType { - BoundaryNtp { - address: SocketAddrV6, - ntp_servers: Vec, - dns_servers: Vec, - domain: Option, - /// The service vNIC providing outbound connectivity using OPTE. - nic: NetworkInterface, - /// The SNAT configuration for outbound connections. - snat_cfg: SourceNatConfig, - }, - - Clickhouse { - address: SocketAddrV6, - dataset: OmicronZoneDataset, - }, - - ClickhouseKeeper { - address: SocketAddrV6, - dataset: OmicronZoneDataset, - }, - CockroachDb { - address: SocketAddrV6, - dataset: OmicronZoneDataset, - }, - - Crucible { - address: SocketAddrV6, - dataset: OmicronZoneDataset, - }, - CruciblePantry { - address: SocketAddrV6, - }, - ExternalDns { - dataset: OmicronZoneDataset, - /// The address at which the external DNS server API is reachable. - http_address: SocketAddrV6, - /// The address at which the external DNS server is reachable. - dns_address: SocketAddr, - /// The service vNIC providing external connectivity using OPTE. - nic: NetworkInterface, - }, - InternalDns { - dataset: OmicronZoneDataset, - http_address: SocketAddrV6, - dns_address: SocketAddrV6, - /// The addresses in the global zone which should be created - /// - /// For the DNS service, which exists outside the sleds's typical subnet - /// - adding an address in the GZ is necessary to allow inter-zone - /// traffic routing. - gz_address: Ipv6Addr, - - /// The address is also identified with an auxiliary bit of information - /// to ensure that the created global zone address can have a unique - /// name. - gz_address_index: u32, - }, - InternalNtp { - address: SocketAddrV6, - ntp_servers: Vec, - dns_servers: Vec, - domain: Option, - }, - Nexus { - /// The address at which the internal nexus server is reachable. - internal_address: SocketAddrV6, - /// The address at which the external nexus server is reachable. - external_ip: IpAddr, - /// The service vNIC providing external connectivity using OPTE. - nic: NetworkInterface, - /// Whether Nexus's external endpoint should use TLS - external_tls: bool, - /// External DNS servers Nexus can use to resolve external hosts. - external_dns_servers: Vec, - }, - Oximeter { - address: SocketAddrV6, - }, -} - -impl OmicronZoneType { - /// Returns a canonical string identifying the type of zone this is - /// - /// This is used to construct zone names, SMF service names, etc. - pub fn zone_type_str(&self) -> String { - match self { - OmicronZoneType::BoundaryNtp { .. } - | OmicronZoneType::InternalNtp { .. } => ZoneType::Ntp, - - OmicronZoneType::Clickhouse { .. } => ZoneType::Clickhouse, - OmicronZoneType::ClickhouseKeeper { .. } => { - ZoneType::ClickhouseKeeper - } - OmicronZoneType::CockroachDb { .. } => ZoneType::CockroachDb, - OmicronZoneType::Crucible { .. } => ZoneType::Crucible, - OmicronZoneType::CruciblePantry { .. } => ZoneType::CruciblePantry, - OmicronZoneType::ExternalDns { .. } => ZoneType::ExternalDns, - OmicronZoneType::InternalDns { .. } => ZoneType::InternalDns, - OmicronZoneType::Nexus { .. } => ZoneType::Nexus, - OmicronZoneType::Oximeter { .. } => ZoneType::Oximeter, - } - .to_string() - } - - /// If this kind of zone has an associated dataset, returns the dataset's - /// name. Othrwise, returns `None`. - pub fn dataset_name(&self) -> Option { - self.dataset_name_and_address().map(|d| d.0) +/// Extension trait for `OmicronZoneType` and `OmicronZoneConfig`. +/// +/// This lives here because it requires extra dependencies that +/// nexus-sled-agent-shared doesn't have. +pub(crate) trait OmicronZoneTypeExt { + fn as_omicron_zone_type(&self) -> &OmicronZoneType; + + /// If this kind of zone has an associated dataset, return the dataset's name. + /// Otherwise, return `None`. + fn dataset_name(&self) -> Option { + self.dataset_name_and_address().map(|(name, _)| name) } - /// If this kind of zone has an associated dataset, return the dataset's - /// name and the associated "service address". Otherwise, returns `None`. - pub fn dataset_name_and_address( - &self, - ) -> Option<(DatasetName, SocketAddrV6)> { - let (dataset, dataset_kind, address) = match self { + /// If this kind of zone has an associated dataset, return the dataset's name + /// and the associated "service address". Otherwise, return `None`. + fn dataset_name_and_address(&self) -> Option<(DatasetName, SocketAddrV6)> { + let (dataset, dataset_kind, address) = match self.as_omicron_zone_type() + { OmicronZoneType::BoundaryNtp { .. } | OmicronZoneType::InternalNtp { .. } | OmicronZoneType::Nexus { .. } | OmicronZoneType::Oximeter { .. } | OmicronZoneType::CruciblePantry { .. } => None, OmicronZoneType::Clickhouse { dataset, address, .. } => { - Some((dataset, DatasetKind::Clickhouse, address)) + Some((dataset, DatasetType::Clickhouse, address)) } OmicronZoneType::ClickhouseKeeper { dataset, address, .. } => { - Some((dataset, DatasetKind::ClickhouseKeeper, address)) + Some((dataset, DatasetType::ClickhouseKeeper, address)) } OmicronZoneType::CockroachDb { dataset, address, .. } => { - Some((dataset, DatasetKind::CockroachDb, address)) + Some((dataset, DatasetType::CockroachDb, address)) } OmicronZoneType::Crucible { dataset, address, .. } => { - Some((dataset, DatasetKind::Crucible, address)) + Some((dataset, DatasetType::Crucible, address)) } OmicronZoneType::ExternalDns { dataset, http_address, .. } => { - Some((dataset, DatasetKind::ExternalDns, http_address)) + Some((dataset, DatasetType::ExternalDns, http_address)) } OmicronZoneType::InternalDns { dataset, http_address, .. } => { - Some((dataset, DatasetKind::InternalDns, http_address)) + Some((dataset, DatasetType::InternalDns, http_address)) } }?; @@ -559,145 +319,29 @@ impl OmicronZoneType { *address, )) } +} - /// Does this zone require time synchronization before it is initialized?" - /// - /// This function is somewhat conservative - the set of services - /// that can be launched before timesync has completed is intentionally kept - /// small, since it would be easy to add a service that expects time to be - /// reasonably synchronized. - pub fn requires_timesync(&self) -> bool { - match self { - // These zones can be initialized and started before time has been - // synchronized. For the NTP zones, this should be self-evident -- - // we need the NTP zone to actually perform time synchronization! - // - // The DNS zone is a bit of an exception here, since the NTP zone - // itself may rely on DNS lookups as a dependency. - OmicronZoneType::BoundaryNtp { .. } - | OmicronZoneType::InternalNtp { .. } - | OmicronZoneType::InternalDns { .. } => false, - _ => true, - } +impl OmicronZoneTypeExt for OmicronZoneType { + fn as_omicron_zone_type(&self) -> &OmicronZoneType { + self + } +} + +impl OmicronZoneTypeExt for OmicronZoneConfig { + fn as_omicron_zone_type(&self) -> &OmicronZoneType { + &self.zone_type } } impl crate::smf_helper::Service for OmicronZoneType { fn service_name(&self) -> String { - // For historical reasons, crucible-pantry is the only zone type whose - // SMF service does not match the canonical name that we use for the - // zone. - match self { - OmicronZoneType::CruciblePantry { .. } => { - "crucible/pantry".to_owned() - } - _ => self.zone_type_str(), - } + self.kind().service_prefix().to_owned() } fn smf_name(&self) -> String { format!("svc:/oxide/{}", self.service_name()) } } -impl From for sled_agent_client::types::OmicronZoneType { - fn from(local: OmicronZoneType) -> Self { - use sled_agent_client::types::OmicronZoneType as Other; - match local { - OmicronZoneType::BoundaryNtp { - address, - ntp_servers, - dns_servers, - domain, - nic, - snat_cfg, - } => Other::BoundaryNtp { - address: address.to_string(), - dns_servers, - domain, - ntp_servers, - snat_cfg, - nic, - }, - OmicronZoneType::Clickhouse { address, dataset } => { - Other::Clickhouse { - address: address.to_string(), - dataset: dataset.into(), - } - } - OmicronZoneType::ClickhouseKeeper { address, dataset } => { - Other::ClickhouseKeeper { - address: address.to_string(), - dataset: dataset.into(), - } - } - OmicronZoneType::CockroachDb { address, dataset } => { - Other::CockroachDb { - address: address.to_string(), - dataset: dataset.into(), - } - } - OmicronZoneType::Crucible { address, dataset } => Other::Crucible { - address: address.to_string(), - dataset: dataset.into(), - }, - OmicronZoneType::CruciblePantry { address } => { - Other::CruciblePantry { address: address.to_string() } - } - OmicronZoneType::ExternalDns { - dataset, - http_address, - dns_address, - nic, - } => Other::ExternalDns { - dataset: dataset.into(), - http_address: http_address.to_string(), - dns_address: dns_address.to_string(), - nic, - }, - OmicronZoneType::InternalDns { - dataset, - http_address, - dns_address, - gz_address, - gz_address_index, - } => Other::InternalDns { - dataset: dataset.into(), - http_address: http_address.to_string(), - dns_address: dns_address.to_string(), - gz_address, - gz_address_index, - }, - OmicronZoneType::InternalNtp { - address, - ntp_servers, - dns_servers, - domain, - } => Other::InternalNtp { - address: address.to_string(), - ntp_servers, - dns_servers, - domain, - }, - OmicronZoneType::Nexus { - internal_address, - external_ip, - nic, - external_tls, - external_dns_servers, - } => Other::Nexus { - external_dns_servers, - external_ip, - external_tls, - internal_address: internal_address.to_string(), - nic, - }, - OmicronZoneType::Oximeter { address } => { - Other::Oximeter { address: address.to_string() } - } - } - } -} - #[derive(Clone, Debug, Deserialize, Serialize, JsonSchema, PartialEq)] pub struct TimeSync { /// The synchronization state of the sled, true when the system clock @@ -741,40 +385,6 @@ pub enum InstanceExternalIpBody { Floating(IpAddr), } -// Our SledRole and Baseboard types do not have to be identical to the Nexus -// ones, but they generally should be, and this avoids duplication. If it -// becomes easier to maintain a separate copy, we should do that. -pub type SledRole = nexus_client::types::SledRole; - -/// Identifies information about disks which may be attached to Sleds. -#[derive(Clone, Debug, Deserialize, JsonSchema, Serialize)] -pub struct InventoryDisk { - pub identity: omicron_common::disk::DiskIdentity, - pub variant: sled_hardware::DiskVariant, - pub slot: i64, -} - -/// Identifies information about zpools managed by the control plane -#[derive(Clone, Debug, Deserialize, JsonSchema, Serialize)] -pub struct InventoryZpool { - pub id: ZpoolUuid, - pub total_size: ByteCount, -} - -/// Identity and basic status information about this sled agent -#[derive(Clone, Debug, Deserialize, JsonSchema, Serialize)] -pub struct Inventory { - pub sled_id: Uuid, - pub sled_agent_address: SocketAddrV6, - pub sled_role: SledRole, - pub baseboard: Baseboard, - pub usable_hardware_threads: u32, - pub usable_physical_ram: ByteCount, - pub reservoir_size: ByteCount, - pub disks: Vec, - pub zpools: Vec, -} - #[derive(Clone, Debug, Deserialize, JsonSchema, Serialize)] pub struct EstablishedConnection { baseboard: Baseboard, diff --git a/sled-agent/src/rack_setup/plan/service.rs b/sled-agent/src/rack_setup/plan/service.rs index d23c6715c6..0062e02dd3 100644 --- a/sled-agent/src/rack_setup/plan/service.rs +++ b/sled-agent/src/rack_setup/plan/service.rs @@ -5,15 +5,14 @@ //! Plan generation for "where should services be initialized". use crate::bootstrap::params::StartSledAgentRequest; -use crate::params::{ - OmicronPhysicalDiskConfig, OmicronPhysicalDisksConfig, OmicronZoneConfig, - OmicronZoneDataset, OmicronZoneType, -}; use camino::Utf8PathBuf; use dns_service_client::types::DnsConfigParams; use illumos_utils::zpool::ZpoolName; use internal_dns::config::{Host, Zone}; use internal_dns::ServiceName; +use nexus_sled_agent_shared::inventory::{ + Inventory, OmicronZoneConfig, OmicronZoneDataset, OmicronZoneType, SledRole, +}; use omicron_common::address::{ get_sled_address, get_switch_zone_address, Ipv6Subnet, ReservedRackSubnet, COCKROACHDB_REDUNDANCY, DENDRITE_PORT, DNS_HTTP_PORT, DNS_PORT, @@ -28,6 +27,9 @@ use omicron_common::api::internal::shared::{ use omicron_common::backoff::{ retry_notify_ext, retry_policy_internal_service_aggressive, BackoffError, }; +use omicron_common::disk::{ + DiskVariant, OmicronPhysicalDiskConfig, OmicronPhysicalDisksConfig, +}; use omicron_common::ledger::{self, Ledger, Ledgerable}; use omicron_uuid_kinds::{GenericUuid, OmicronZoneUuid, SledUuid, ZpoolUuid}; use rand::prelude::SliceRandom; @@ -37,7 +39,7 @@ use sled_agent_client::{ types as SledAgentTypes, Client as SledAgentClient, Error as SledAgentError, }; use sled_agent_types::rack_init::RackInitializeRequest as Config; -use sled_storage::dataset::{DatasetKind, DatasetName, CONFIG_DATASET}; +use sled_storage::dataset::{DatasetName, DatasetType, CONFIG_DATASET}; use sled_storage::manager::StorageHandle; use slog::Logger; use std::collections::{BTreeMap, BTreeSet, HashMap, HashSet}; @@ -246,15 +248,15 @@ impl Plan { let role = client.sled_role_get().await?.into_inner(); match role { - SledAgentTypes::SledRole::Gimlet => Ok(false), - SledAgentTypes::SledRole::Scrimlet => Ok(true), + SledRole::Gimlet => Ok(false), + SledRole::Scrimlet => Ok(true), } } async fn get_inventory( log: &Logger, address: SocketAddrV6, - ) -> Result { + ) -> Result { let dur = std::time::Duration::from_secs(60); let client = reqwest::ClientBuilder::new() .connect_timeout(dur) @@ -279,9 +281,7 @@ impl Plan { if inventory .disks .iter() - .filter(|disk| { - matches!(disk.variant, SledAgentTypes::DiskVariant::U2) - }) + .filter(|disk| matches!(disk.variant, DiskVariant::U2)) .count() < MINIMUM_U2_COUNT { @@ -347,9 +347,7 @@ impl Plan { .inventory .disks .iter() - .filter(|disk| { - matches!(disk.variant, SledAgentTypes::DiskVariant::U2) - }) + .filter(|disk| matches!(disk.variant, DiskVariant::U2)) .map(|disk| OmicronPhysicalDiskConfig { identity: disk.identity.clone(), id: Uuid::new_v4(), @@ -405,7 +403,7 @@ impl Plan { ) .unwrap(); let dataset_name = - sled.alloc_dataset_from_u2s(DatasetKind::InternalDns)?; + sled.alloc_dataset_from_u2s(DatasetType::InternalDns)?; let filesystem_pool = Some(dataset_name.pool().clone()); sled.request.zones.push(OmicronZoneConfig { @@ -445,7 +443,7 @@ impl Plan { ) .unwrap(); let dataset_name = - sled.alloc_dataset_from_u2s(DatasetKind::CockroachDb)?; + sled.alloc_dataset_from_u2s(DatasetType::CockroachDb)?; let filesystem_pool = Some(dataset_name.pool().clone()); sled.request.zones.push(OmicronZoneConfig { // TODO-cleanup use TypedUuid everywhere @@ -489,7 +487,7 @@ impl Plan { .unwrap(); let dns_port = omicron_common::address::DNS_PORT; let dns_address = SocketAddr::new(external_ip, dns_port); - let dataset_kind = DatasetKind::ExternalDns; + let dataset_kind = DatasetType::ExternalDns; let dataset_name = sled.alloc_dataset_from_u2s(dataset_kind)?; let filesystem_pool = Some(dataset_name.pool().clone()); @@ -610,7 +608,7 @@ impl Plan { ) .unwrap(); let dataset_name = - sled.alloc_dataset_from_u2s(DatasetKind::Clickhouse)?; + sled.alloc_dataset_from_u2s(DatasetType::Clickhouse)?; let filesystem_pool = Some(dataset_name.pool().clone()); sled.request.zones.push(OmicronZoneConfig { // TODO-cleanup use TypedUuid everywhere @@ -649,7 +647,7 @@ impl Plan { ) .unwrap(); let dataset_name = - sled.alloc_dataset_from_u2s(DatasetKind::ClickhouseKeeper)?; + sled.alloc_dataset_from_u2s(DatasetType::ClickhouseKeeper)?; let filesystem_pool = Some(dataset_name.pool().clone()); sled.request.zones.push(OmicronZoneConfig { // TODO-cleanup use TypedUuid everywhere @@ -863,12 +861,12 @@ pub struct SledInfo { /// the address of the Sled Agent on the sled's subnet pub sled_address: SocketAddrV6, /// the inventory returned by the Sled - inventory: SledAgentTypes::Inventory, + inventory: Inventory, /// The Zpools available for usage by services u2_zpools: Vec, /// spreads components across a Sled's zpools u2_zpool_allocators: - HashMap + Send + Sync>>, + HashMap + Send + Sync>>, /// whether this Sled is a scrimlet is_scrimlet: bool, /// allocator for addresses in this Sled's subnet @@ -882,7 +880,7 @@ impl SledInfo { sled_id: SledUuid, subnet: Ipv6Subnet, sled_address: SocketAddrV6, - inventory: SledAgentTypes::Inventory, + inventory: Inventory, is_scrimlet: bool, ) -> SledInfo { SledInfo { @@ -909,7 +907,7 @@ impl SledInfo { /// this Sled fn alloc_dataset_from_u2s( &mut self, - kind: DatasetKind, + kind: DatasetType, ) -> Result { // We have two goals here: // diff --git a/sled-agent/src/rack_setup/service.rs b/sled-agent/src/rack_setup/service.rs index c8e56ae9f4..ac51912fe6 100644 --- a/sled-agent/src/rack_setup/service.rs +++ b/sled-agent/src/rack_setup/service.rs @@ -71,8 +71,8 @@ use crate::bootstrap::early_networking::{ }; use crate::bootstrap::params::StartSledAgentRequest; use crate::bootstrap::rss_handle::BootstrapAgentHandle; -use crate::nexus::{d2n_params, ConvertInto}; -use crate::params::{OmicronZoneType, OmicronZonesConfig, TimeSync}; +use crate::nexus::d2n_params; +use crate::params::{OmicronZoneTypeExt, TimeSync}; use crate::rack_setup::plan::service::{ Plan as ServicePlan, PlanError as ServicePlanError, }; @@ -88,6 +88,9 @@ use internal_dns::ServiceName; use nexus_client::{ types as NexusTypes, Client as NexusClient, Error as NexusError, }; +use nexus_sled_agent_shared::inventory::{ + OmicronZoneConfig, OmicronZoneType, OmicronZonesConfig, +}; use nexus_types::deployment::{ Blueprint, BlueprintPhysicalDisksConfig, BlueprintZoneConfig, BlueprintZoneDisposition, BlueprintZonesConfig, @@ -100,6 +103,9 @@ use omicron_common::api::internal::shared::ExternalPortDiscovery; use omicron_common::backoff::{ retry_notify, retry_policy_internal_service_aggressive, BackoffError, }; +use omicron_common::disk::{ + OmicronPhysicalDiskConfig, OmicronPhysicalDisksConfig, +}; use omicron_common::ledger::{self, Ledger, Ledgerable}; use omicron_ddm_admin_client::{Client as DdmAdminClient, DdmError}; use omicron_uuid_kinds::SledUuid; @@ -299,18 +305,16 @@ impl ServiceInner { plan.services.iter().map(|(sled_address, config)| async move { self.initialize_storage_on_sled( *sled_address, - SledAgentTypes::OmicronPhysicalDisksConfig { + OmicronPhysicalDisksConfig { generation: config.disks.generation, disks: config .disks .disks .iter() - .map(|disk| { - SledAgentTypes::OmicronPhysicalDiskConfig { - identity: disk.identity.clone(), - id: disk.id, - pool_id: disk.pool_id, - } + .map(|disk| OmicronPhysicalDiskConfig { + identity: disk.identity.clone(), + id: disk.id, + pool_id: disk.pool_id, }) .collect(), }, @@ -333,7 +337,7 @@ impl ServiceInner { async fn initialize_storage_on_sled( &self, sled_address: SocketAddrV6, - storage_config: SledAgentTypes::OmicronPhysicalDisksConfig, + storage_config: OmicronPhysicalDisksConfig, ) -> Result<(), SetupServiceError> { let dur = std::time::Duration::from_secs(60); let client = reqwest::ClientBuilder::new() @@ -435,8 +439,7 @@ impl ServiceInner { log, "attempting to set up sled's Omicron zones: {:?}", zones_config ); - let result = - client.omicron_zones_put(&zones_config.clone().into()).await; + let result = client.omicron_zones_put(zones_config).await; let Err(error) = result else { return Ok::< (), @@ -722,7 +725,7 @@ impl ServiceInner { dataset_id: zone.id, request: NexusTypes::DatasetPutRequest { address: dataset_address.to_string(), - kind: dataset_name.dataset().clone().convert(), + kind: dataset_name.dataset().kind(), }, }) } @@ -1370,11 +1373,11 @@ pub(crate) fn build_initial_blueprint_from_sled_configs( ) -> Result { // Helper to convert an `OmicronZoneConfig` into a `BlueprintZoneConfig`. // This is separate primarily so rustfmt doesn't lose its mind. - let to_bp_zone_config = |z: &crate::params::OmicronZoneConfig| { + let to_bp_zone_config = |z: &OmicronZoneConfig| { // All initial zones are in-service. let disposition = BlueprintZoneDisposition::InService; BlueprintZoneConfig::from_omicron_zone_config( - z.clone().into(), + z.clone(), disposition, // This is pretty weird: IP IDs don't exist yet, so it's fine for us // to make them up (Nexus will record them as a part of the @@ -1399,7 +1402,7 @@ pub(crate) fn build_initial_blueprint_from_sled_configs( .disks .disks .iter() - .map(|d| SledAgentTypes::OmicronPhysicalDiskConfig { + .map(|d| OmicronPhysicalDiskConfig { identity: d.identity.clone(), id: d.id, pool_id: d.pool_id, @@ -1549,17 +1552,16 @@ impl<'a> OmicronZonesConfigGenerator<'a> { #[cfg(test)] mod test { use super::{Config, OmicronZonesConfigGenerator}; - use crate::{ - params::OmicronZoneType, - rack_setup::plan::service::{Plan as ServicePlan, SledInfo}, + use crate::rack_setup::plan::service::{Plan as ServicePlan, SledInfo}; + use nexus_sled_agent_shared::inventory::{ + Baseboard, Inventory, InventoryDisk, OmicronZoneType, SledRole, }; use omicron_common::{ address::{get_sled_address, Ipv6Subnet, SLED_PREFIX}, api::external::{ByteCount, Generation}, - disk::DiskIdentity, + disk::{DiskIdentity, DiskVariant}, }; use omicron_uuid_kinds::{GenericUuid, SledUuid}; - use sled_agent_client::types as SledAgentTypes; fn make_sled_info( sled_id: SledUuid, @@ -1571,22 +1573,22 @@ mod test { sled_id, subnet, sled_agent_address, - SledAgentTypes::Inventory { + Inventory { sled_id: sled_id.into_untyped_uuid(), - sled_agent_address: sled_agent_address.to_string(), - sled_role: SledAgentTypes::SledRole::Scrimlet, - baseboard: SledAgentTypes::Baseboard::Unknown, + sled_agent_address, + sled_role: SledRole::Scrimlet, + baseboard: Baseboard::Unknown, usable_hardware_threads: 32, usable_physical_ram: ByteCount::from_gibibytes_u32(16), reservoir_size: ByteCount::from_gibibytes_u32(0), disks: (0..u2_count) - .map(|i| SledAgentTypes::InventoryDisk { + .map(|i| InventoryDisk { identity: DiskIdentity { vendor: "test-manufacturer".to_string(), serial: format!("test-{sled_id}-#{i}"), model: "v1".to_string(), }, - variant: SledAgentTypes::DiskVariant::U2, + variant: DiskVariant::U2, slot: i.try_into().unwrap(), }) .collect(), diff --git a/sled-agent/src/services.rs b/sled-agent/src/services.rs index 943ff44e06..2ce61b21ec 100644 --- a/sled-agent/src/services.rs +++ b/sled-agent/src/services.rs @@ -31,8 +31,8 @@ use crate::bootstrap::early_networking::{ use crate::bootstrap::BootstrapNetworking; use crate::config::SidecarRevision; use crate::params::{ - DendriteAsic, OmicronZoneConfig, OmicronZoneType, OmicronZonesConfig, - TimeSync, ZoneBundleCause, ZoneBundleMetadata, ZoneType, + DendriteAsic, OmicronZoneConfigExt, OmicronZoneTypeExt, TimeSync, + ZoneBundleCause, ZoneBundleMetadata, }; use crate::profile::*; use crate::smf_helper::SmfHelper; @@ -62,6 +62,9 @@ use illumos_utils::{execute, PFEXEC}; use internal_dns::resolver::Resolver; use itertools::Itertools; use nexus_config::{ConfigDropshotWithTls, DeploymentConfig}; +use nexus_sled_agent_shared::inventory::{ + OmicronZoneConfig, OmicronZoneType, OmicronZonesConfig, ZoneKind, +}; use omicron_common::address::CLICKHOUSE_KEEPER_PORT; use omicron_common::address::CLICKHOUSE_PORT; use omicron_common::address::COCKROACH_PORT; @@ -96,7 +99,7 @@ use sled_hardware::SledMode; use sled_hardware_types::Baseboard; use sled_storage::config::MountConfig; use sled_storage::dataset::{ - DatasetKind, DatasetName, CONFIG_DATASET, INSTALL_DATASET, ZONE_DATASET, + DatasetName, DatasetType, CONFIG_DATASET, INSTALL_DATASET, ZONE_DATASET, }; use sled_storage::manager::StorageHandle; use slog::Logger; @@ -194,9 +197,9 @@ pub enum Error { #[error("Failed to access underlay device: {0}")] Underlay(#[from] underlay::Error), - #[error("Failed to create OPTE port for service {service}: {err}")] + #[error("Failed to create OPTE port for service {}: {err}", service.report_str())] ServicePortCreation { - service: String, + service: ZoneKind, err: Box, }, @@ -1135,17 +1138,14 @@ impl ServiceManager { .collect(); let external_ip; - let (zone_type_str, nic, snat, floating_ips) = match &zone_args + let (zone_kind, nic, snat, floating_ips) = match &zone_args .omicron_type() { Some( zone_type @ OmicronZoneType::Nexus { external_ip, nic, .. }, - ) => ( - zone_type.zone_type_str(), - nic, - None, - std::slice::from_ref(external_ip), - ), + ) => { + (zone_type.kind(), nic, None, std::slice::from_ref(external_ip)) + } Some( zone_type @ OmicronZoneType::ExternalDns { dns_address, @@ -1155,7 +1155,7 @@ impl ServiceManager { ) => { external_ip = dns_address.ip(); ( - zone_type.zone_type_str(), + zone_type.kind(), nic, None, std::slice::from_ref(&external_ip), @@ -1165,7 +1165,7 @@ impl ServiceManager { zone_type @ OmicronZoneType::BoundaryNtp { nic, snat_cfg, .. }, - ) => (zone_type.zone_type_str(), nic, Some(*snat_cfg), &[][..]), + ) => (zone_type.kind(), nic, Some(*snat_cfg), &[][..]), _ => unreachable!("unexpected zone type"), }; @@ -1185,7 +1185,7 @@ impl ServiceManager { is_service: true, }) .map_err(|err| Error::ServicePortCreation { - service: zone_type_str.clone(), + service: zone_kind, err: Box::new(err), })?; @@ -1206,7 +1206,7 @@ impl ServiceManager { let nat_create = || async { info!( self.inner.log, "creating NAT entry for service"; - "zone_type" => &zone_type_str, + "zone_type" => zone_kind.report_str(), ); dpd_client @@ -1230,7 +1230,7 @@ impl ServiceManager { warn!( self.inner.log, "failed to create NAT entry for service"; "error" => ?error, - "zone_type" => &zone_type_str, + "zone_type" => zone_kind.report_str(), ); }; retry_notify( @@ -1464,9 +1464,9 @@ impl ServiceManager { let zone_type_str = match &request { ZoneArgs::Omicron(zone_config) => { - zone_config.zone.zone_type.zone_type_str() + zone_config.zone.zone_type.kind().zone_prefix() } - ZoneArgs::Switch(_) => "switch".to_string(), + ZoneArgs::Switch(_) => "switch", }; // We use the fake initialiser for testing @@ -1485,7 +1485,7 @@ impl ServiceManager { .with_underlay_vnic_allocator(&self.inner.underlay_vnic_allocator) .with_zone_root_path(request.root()) .with_zone_image_paths(zone_image_paths.as_slice()) - .with_zone_type(&zone_type_str) + .with_zone_type(zone_type_str) .with_datasets(datasets.as_slice()) .with_filesystems(&filesystems) .with_data_links(&data_links) @@ -1718,7 +1718,7 @@ impl ServiceManager { let dataset_name = DatasetName::new( dataset.pool_name.clone(), - DatasetKind::Crucible, + DatasetType::Crucible, ) .full_name(); let uuid = &Uuid::new_v4().to_string(); @@ -3049,7 +3049,7 @@ impl ServiceManager { fake_install_dir, ) .await - .map_err(|err| (zone.zone_name().to_string(), err)) + .map_err(|err| (zone.zone_name(), err)) }); let results = futures::future::join_all(futures).await; @@ -3475,7 +3475,7 @@ impl ServiceManager { // TODO: We could probably store the ZoneKind in the running zone to // make this "comparison to existing zones by name" mechanism a bit // safer. - if zone.name().contains(&ZoneType::CockroachDb.to_string()) { + if zone.name().contains(ZoneKind::CockroachDb.zone_prefix()) { let address = Zones::get_address( Some(zone.name()), &zone.runtime.control_interface(), @@ -3589,7 +3589,7 @@ impl ServiceManager { }; let ntp_zone_name = - InstalledZone::get_zone_name(&ZoneType::Ntp.to_string(), None); + InstalledZone::get_zone_name(ZoneKind::NTP_PREFIX, None); let ntp_zone = existing_zones .iter() @@ -4539,7 +4539,7 @@ mod test { zone_type: OmicronZoneType, tmp_dir: String, ) -> Result<(), Error> { - let zone_prefix = format!("oxz_{}", zone_type.zone_type_str()); + let zone_prefix = format!("oxz_{}", zone_type.kind().zone_prefix()); let _expectations = expect_new_service(&zone_prefix); let r = mgr .ensure_all_omicron_zones_persistent( diff --git a/sled-agent/src/sim/http_entrypoints.rs b/sled-agent/src/sim/http_entrypoints.rs index 399ec334f4..268e8a9cf1 100644 --- a/sled-agent/src/sim/http_entrypoints.rs +++ b/sled-agent/src/sim/http_entrypoints.rs @@ -8,8 +8,8 @@ use crate::bootstrap::params::AddSledRequest; use crate::params::{ DiskEnsureBody, InstanceEnsureBody, InstanceExternalIpBody, InstancePutMigrationIdsBody, InstancePutStateBody, - InstancePutStateResponse, InstanceUnregisterResponse, Inventory, - OmicronPhysicalDisksConfig, OmicronZonesConfig, VpcFirewallRulesEnsureBody, + InstancePutStateResponse, InstanceUnregisterResponse, + VpcFirewallRulesEnsureBody, }; use dropshot::ApiDescription; use dropshot::HttpError; @@ -20,12 +20,14 @@ use dropshot::RequestContext; use dropshot::TypedBody; use dropshot::{endpoint, ApiDescriptionRegisterError}; use illumos_utils::opte::params::VirtualNetworkInterfaceHost; +use nexus_sled_agent_shared::inventory::{Inventory, OmicronZonesConfig}; use omicron_common::api::internal::nexus::DiskRuntimeState; use omicron_common::api::internal::nexus::SledInstanceState; use omicron_common::api::internal::nexus::UpdateArtifactId; use omicron_common::api::internal::shared::{ ResolvedVpcRouteSet, ResolvedVpcRouteState, SwitchPorts, }; +use omicron_common::disk::OmicronPhysicalDisksConfig; use omicron_uuid_kinds::{GenericUuid, InstanceUuid}; use schemars::JsonSchema; use serde::{Deserialize, Serialize}; diff --git a/sled-agent/src/sim/server.rs b/sled-agent/src/sim/server.rs index 215cb7d5f4..189f775adb 100644 --- a/sled-agent/src/sim/server.rs +++ b/sled-agent/src/sim/server.rs @@ -10,9 +10,6 @@ use super::sled_agent::SledAgent; use super::storage::PantryServer; use crate::nexus::d2n_params; use crate::nexus::NexusClient; -use crate::params::OmicronZoneConfig; -use crate::params::OmicronZoneDataset; -use crate::params::OmicronZoneType; use crate::rack_setup::service::build_initial_blueprint_from_sled_configs; use crate::rack_setup::SledConfig; use anyhow::anyhow; @@ -22,12 +19,17 @@ use internal_dns::ServiceName; use nexus_client::types as NexusTypes; use nexus_client::types::{IpRange, Ipv4Range, Ipv6Range}; use nexus_config::NUM_INITIAL_RESERVED_IP_ADDRESSES; +use nexus_sled_agent_shared::inventory::OmicronZoneConfig; +use nexus_sled_agent_shared::inventory::OmicronZoneDataset; +use nexus_sled_agent_shared::inventory::OmicronZoneType; use nexus_types::inventory::NetworkInterfaceKind; use omicron_common::address::DNS_OPTE_IPV4_SUBNET; use omicron_common::address::NEXUS_OPTE_IPV4_SUBNET; use omicron_common::api::external::Generation; use omicron_common::api::external::MacAddr; use omicron_common::api::external::Vni; +use omicron_common::api::internal::nexus::Certificate; +use omicron_common::api::internal::shared::DatasetKind; use omicron_common::backoff::{ retry_notify, retry_policy_internal_service_aggressive, BackoffError, }; @@ -37,6 +39,7 @@ use omicron_uuid_kinds::GenericUuid; use omicron_uuid_kinds::SledUuid; use omicron_uuid_kinds::ZpoolUuid; use oxnet::Ipv6Net; +use sled_agent_types::rack_init::RecoverySiloConfig; use slog::{info, Drain, Logger}; use std::collections::BTreeMap; use std::collections::HashMap; @@ -196,7 +199,7 @@ impl Server { dataset_id, request: NexusTypes::DatasetPutRequest { address: address.to_string(), - kind: NexusTypes::DatasetKind::Crucible, + kind: DatasetKind::Crucible, }, }); @@ -283,7 +286,7 @@ pub struct RssArgs { pub internal_dns_dns_addr: Option, /// Specify a certificate and associated private key for the initial Silo's /// initial TLS certificates - pub tls_certificate: Option, + pub tls_certificate: Option, } /// Run an instance of the `Server` which is able to handoff to Nexus. @@ -475,7 +478,7 @@ pub async fn run_standalone_server( .push(IpRange::V6(Ipv6Range { first: ip, last: ip })); } - let recovery_silo = NexusTypes::RecoverySiloConfig { + let recovery_silo = RecoverySiloConfig { silo_name: "demo-silo".parse().unwrap(), user_name: "demo-privileged".parse().unwrap(), // The following is a hash for the password "oxide". This is @@ -505,7 +508,7 @@ pub async fn run_standalone_server( dataset_id, request: NexusTypes::DatasetPutRequest { address: address.to_string(), - kind: NexusTypes::DatasetKind::Crucible, + kind: DatasetKind::Crucible, }, }); } diff --git a/sled-agent/src/sim/sled_agent.rs b/sled-agent/src/sim/sled_agent.rs index f23b14c377..32d3d29c5d 100644 --- a/sled-agent/src/sim/sled_agent.rs +++ b/sled-agent/src/sim/sled_agent.rs @@ -14,8 +14,7 @@ use crate::nexus::NexusClient; use crate::params::{ DiskStateRequested, InstanceExternalIpBody, InstanceHardware, InstanceMetadata, InstanceMigrationSourceParams, InstancePutStateResponse, - InstanceStateRequested, InstanceUnregisterResponse, Inventory, - OmicronPhysicalDisksConfig, OmicronZonesConfig, SledRole, + InstanceStateRequested, InstanceUnregisterResponse, }; use crate::sim::simulatable::Simulatable; use crate::updates::UpdateManager; @@ -24,6 +23,9 @@ use anyhow::Context; use dropshot::{HttpError, HttpServer}; use futures::lock::Mutex; use illumos_utils::opte::params::VirtualNetworkInterfaceHost; +use nexus_sled_agent_shared::inventory::{ + Inventory, InventoryDisk, InventoryZpool, OmicronZonesConfig, SledRole, +}; use omicron_common::api::external::{ ByteCount, DiskState, Error, Generation, ResourceType, }; @@ -37,7 +39,9 @@ use omicron_common::api::internal::shared::{ RackNetworkConfig, ResolvedVpcRoute, ResolvedVpcRouteSet, ResolvedVpcRouteState, RouterId, RouterKind, RouterVersion, }; -use omicron_common::disk::DiskIdentity; +use omicron_common::disk::{ + DiskIdentity, DiskVariant, OmicronPhysicalDisksConfig, +}; use omicron_uuid_kinds::{GenericUuid, InstanceUuid, PropolisUuid, ZpoolUuid}; use oxnet::Ipv6Net; use propolis_client::{ @@ -593,7 +597,7 @@ impl SledAgent { id: Uuid, identity: DiskIdentity, ) { - let variant = sled_hardware::DiskVariant::U2; + let variant = DiskVariant::U2; self.storage .lock() .await @@ -850,7 +854,7 @@ impl SledAgent { disks: storage .physical_disks() .values() - .map(|info| crate::params::InventoryDisk { + .map(|info| InventoryDisk { identity: info.identity.clone(), variant: info.variant, slot: info.slot, @@ -860,7 +864,7 @@ impl SledAgent { .zpools() .iter() .map(|(id, zpool)| { - Ok(crate::params::InventoryZpool { + Ok(InventoryZpool { id: *id, total_size: ByteCount::try_from(zpool.total_size())?, }) diff --git a/sled-agent/src/sim/storage.rs b/sled-agent/src/sim/storage.rs index 0d534b9c4e..948ac96bcd 100644 --- a/sled-agent/src/sim/storage.rs +++ b/sled-agent/src/sim/storage.rs @@ -8,7 +8,6 @@ //! than the representation of "virtual disks" which would be presented //! through Nexus' external API. -use crate::params::OmicronPhysicalDisksConfig; use crate::sim::http_entrypoints_pantry::ExpectedDigest; use crate::sim::SledAgent; use anyhow::{self, bail, Result}; @@ -20,12 +19,13 @@ use dropshot::HandlerTaskMode; use dropshot::HttpError; use futures::lock::Mutex; use omicron_common::disk::DiskIdentity; +use omicron_common::disk::DiskVariant; +use omicron_common::disk::OmicronPhysicalDisksConfig; use omicron_uuid_kinds::GenericUuid; use omicron_uuid_kinds::InstanceUuid; use omicron_uuid_kinds::OmicronZoneUuid; use omicron_uuid_kinds::ZpoolUuid; use propolis_client::types::VolumeConstructionRequest; -use sled_hardware::DiskVariant; use sled_storage::resources::DiskManagementStatus; use sled_storage::resources::DisksManagementResult; use slog::Logger; diff --git a/sled-agent/src/sled_agent.rs b/sled-agent/src/sled_agent.rs index 4bf7117bc9..1a3750daf1 100644 --- a/sled-agent/src/sled_agent.rs +++ b/sled-agent/src/sled_agent.rs @@ -19,9 +19,8 @@ use crate::nexus::{ use crate::params::{ DiskStateRequested, InstanceExternalIpBody, InstanceHardware, InstanceMetadata, InstanceMigrationSourceParams, InstancePutStateResponse, - InstanceStateRequested, InstanceUnregisterResponse, Inventory, - OmicronPhysicalDisksConfig, OmicronZonesConfig, SledRole, TimeSync, - VpcFirewallRule, ZoneBundleMetadata, Zpool, + InstanceStateRequested, InstanceUnregisterResponse, OmicronZoneTypeExt, + TimeSync, VpcFirewallRule, ZoneBundleMetadata, Zpool, }; use crate::probe_manager::ProbeManager; use crate::services::{self, ServiceManager}; @@ -40,6 +39,9 @@ use illumos_utils::opte::params::VirtualNetworkInterfaceHost; use illumos_utils::opte::PortManager; use illumos_utils::zone::PROPOLIS_ZONE_PREFIX; use illumos_utils::zone::ZONE_PREFIX; +use nexus_sled_agent_shared::inventory::{ + Inventory, InventoryDisk, InventoryZpool, OmicronZonesConfig, SledRole, +}; use omicron_common::address::{ get_sled_address, get_switch_zone_address, Ipv6Subnet, SLED_PREFIX, }; @@ -58,6 +60,7 @@ use omicron_common::api::{ use omicron_common::backoff::{ retry_notify, retry_policy_internal_service_aggressive, BackoffError, }; +use omicron_common::disk::OmicronPhysicalDisksConfig; use omicron_ddm_admin_client::Client as DdmAdminClient; use omicron_uuid_kinds::{InstanceUuid, PropolisUuid}; use oximeter::types::ProducerRegistry; @@ -1221,17 +1224,14 @@ impl SledAgent { let usable_physical_ram = self.inner.hardware.usable_physical_ram_bytes(); let reservoir_size = self.inner.instances.reservoir_size(); - let sled_role = if is_scrimlet { - crate::params::SledRole::Scrimlet - } else { - crate::params::SledRole::Gimlet - }; + let sled_role = + if is_scrimlet { SledRole::Scrimlet } else { SledRole::Gimlet }; let mut disks = vec![]; let mut zpools = vec![]; let all_disks = self.storage().get_latest_disks().await; for (identity, variant, slot, _firmware) in all_disks.iter_all() { - disks.push(crate::params::InventoryDisk { + disks.push(InventoryDisk { identity: identity.clone(), variant, slot, @@ -1253,7 +1253,7 @@ impl SledAgent { } }; - zpools.push(crate::params::InventoryZpool { + zpools.push(InventoryZpool { id: zpool.id(), total_size: ByteCount::try_from(info.size())?, }); diff --git a/sled-agent/types/Cargo.toml b/sled-agent/types/Cargo.toml index 57881a37d1..a9ed8fcb22 100644 --- a/sled-agent/types/Cargo.toml +++ b/sled-agent/types/Cargo.toml @@ -11,7 +11,10 @@ workspace = true anyhow.workspace = true bootstore.workspace = true camino.workspace = true -nexus-client.workspace = true +nexus-sled-agent-shared.workspace = true +# Note: we're trying to avoid a dependency from sled-agent-types to nexus-types +# because the correct direction of dependency is unclear. If there are types +# common to both, put them in `omicron-common` or `nexus-sled-agent-shared`. omicron-common.workspace = true omicron-uuid-kinds.workspace = true omicron-workspace-hack.workspace = true @@ -20,9 +23,11 @@ schemars.workspace = true serde.workspace = true serde_json.workspace = true sled-hardware-types.workspace = true +sled-storage.workspace = true slog.workspace = true thiserror.workspace = true toml.workspace = true +uuid.workspace = true [dev-dependencies] camino-tempfile.workspace = true diff --git a/sled-agent/types/src/rack_init.rs b/sled-agent/types/src/rack_init.rs index 8fcf3c93fd..64ad63e9df 100644 --- a/sled-agent/types/src/rack_init.rs +++ b/sled-agent/types/src/rack_init.rs @@ -11,23 +11,26 @@ use std::{ use anyhow::{bail, Result}; use camino::{Utf8Path, Utf8PathBuf}; +pub use nexus_sled_agent_shared::recovery_silo::RecoverySiloConfig; use omicron_common::{ address::{ get_64_subnet, IpRange, Ipv6Subnet, AZ_PREFIX, RACK_PREFIX, SLED_PREFIX, }, - api::{external::AllowedSourceIps, internal::shared::RackNetworkConfig}, + api::{ + external::AllowedSourceIps, + internal::{nexus::Certificate, shared::RackNetworkConfig}, + }, }; use schemars::JsonSchema; use serde::{Deserialize, Serialize}; use sled_hardware_types::Baseboard; -pub type Certificate = nexus_client::types::Certificate; -pub type RecoverySiloConfig = nexus_client::types::RecoverySiloConfig; - /// Structures and routines used to maintain backwards compatibility. The /// contents of this module should only be used to convert older data into the /// current format, and not for any ongoing run-time operations. pub mod back_compat { + use omicron_common::api::internal::nexus::Certificate; + use crate::early_networking::back_compat::RackNetworkConfigV1; use super::*; diff --git a/sled-hardware/src/disk.rs b/sled-hardware/src/disk.rs index 5dfd9e2c23..f95efb985e 100644 --- a/sled-hardware/src/disk.rs +++ b/sled-hardware/src/disk.rs @@ -5,10 +5,9 @@ use camino::{Utf8Path, Utf8PathBuf}; use illumos_utils::fstyp::Fstyp; use illumos_utils::zpool::Zpool; -use omicron_common::disk::DiskIdentity; -use omicron_common::zpool_name::{ZpoolKind, ZpoolName}; +use omicron_common::disk::{DiskIdentity, DiskVariant}; +use omicron_common::zpool_name::ZpoolName; use omicron_uuid_kinds::ZpoolUuid; -use schemars::JsonSchema; use serde::{Deserialize, Serialize}; use slog::Logger; use slog::{info, warn}; @@ -423,33 +422,6 @@ pub fn ensure_zpool_failmode_is_continue( Ok(()) } -#[derive( - Debug, - Clone, - Copy, - PartialEq, - Eq, - Hash, - Serialize, - Deserialize, - JsonSchema, - Ord, - PartialOrd, -)] -pub enum DiskVariant { - U2, - M2, -} - -impl From for DiskVariant { - fn from(kind: ZpoolKind) -> DiskVariant { - match kind { - ZpoolKind::External => DiskVariant::U2, - ZpoolKind::Internal => DiskVariant::M2, - } - } -} - #[cfg(test)] mod test { use super::*; diff --git a/sled-hardware/src/illumos/mod.rs b/sled-hardware/src/illumos/mod.rs index 65f439eaeb..83ef685b67 100644 --- a/sled-hardware/src/illumos/mod.rs +++ b/sled-hardware/src/illumos/mod.rs @@ -3,14 +3,12 @@ // file, You can obtain one at https://mozilla.org/MPL/2.0/. use crate::DiskFirmware; -use crate::{ - DendriteAsic, DiskVariant, HardwareUpdate, SledMode, UnparsedDisk, -}; +use crate::{DendriteAsic, HardwareUpdate, SledMode, UnparsedDisk}; use camino::Utf8PathBuf; use gethostname::gethostname; use illumos_devinfo::{DevInfo, DevLinkType, DevLinks, Node, Property}; use libnvme::{controller::Controller, Nvme}; -use omicron_common::disk::DiskIdentity; +use omicron_common::disk::{DiskIdentity, DiskVariant}; use sled_hardware_types::Baseboard; use slog::debug; use slog::error; diff --git a/sled-hardware/src/illumos/partitions.rs b/sled-hardware/src/illumos/partitions.rs index 1386d07866..7b8f88a1f1 100644 --- a/sled-hardware/src/illumos/partitions.rs +++ b/sled-hardware/src/illumos/partitions.rs @@ -8,10 +8,10 @@ use std::collections::HashMap; use std::sync::OnceLock; use crate::illumos::gpt; -use crate::{DiskPaths, DiskVariant, Partition, PooledDiskError}; +use crate::{DiskPaths, Partition, PooledDiskError}; use camino::Utf8Path; use illumos_utils::zpool::ZpoolName; -use omicron_common::disk::DiskIdentity; +use omicron_common::disk::{DiskIdentity, DiskVariant}; use omicron_uuid_kinds::ZpoolUuid; use slog::info; use slog::Logger; diff --git a/sled-hardware/src/non_illumos/mod.rs b/sled-hardware/src/non_illumos/mod.rs index 955be9a35e..a61bbc495f 100644 --- a/sled-hardware/src/non_illumos/mod.rs +++ b/sled-hardware/src/non_illumos/mod.rs @@ -2,11 +2,9 @@ // License, v. 2.0. If a copy of the MPL was not distributed with this // file, You can obtain one at https://mozilla.org/MPL/2.0/. -use crate::disk::{ - DiskPaths, DiskVariant, Partition, PooledDiskError, UnparsedDisk, -}; +use crate::disk::{DiskPaths, Partition, PooledDiskError, UnparsedDisk}; use crate::SledMode; -use omicron_common::disk::DiskIdentity; +use omicron_common::disk::{DiskIdentity, DiskVariant}; use omicron_uuid_kinds::ZpoolUuid; use sled_hardware_types::Baseboard; use slog::Logger; diff --git a/sled-storage/src/dataset.rs b/sled-storage/src/dataset.rs index 7846826ee8..26b5085609 100644 --- a/sled-storage/src/dataset.rs +++ b/sled-storage/src/dataset.rs @@ -14,11 +14,11 @@ use illumos_utils::zfs::{ }; use illumos_utils::zpool::ZpoolName; use key_manager::StorageKeyRequester; -use omicron_common::disk::DiskIdentity; +use omicron_common::api::internal::shared::DatasetKind; +use omicron_common::disk::{DiskIdentity, DiskVariant}; use rand::distributions::{Alphanumeric, DistString}; use schemars::JsonSchema; use serde::{Deserialize, Serialize}; -use sled_hardware::DiskVariant; use slog::{debug, info, Logger}; use std::process::Stdio; use std::str::FromStr; @@ -126,13 +126,18 @@ impl ExpectedDataset { } } -/// The type of a dataset, and an auxiliary information necessary -/// to successfully launch a zone managing the associated data. +/// The type of a dataset, and an auxiliary information necessary to +/// successfully launch a zone managing the associated data. +/// +/// There is currently no auxiliary data here, but there's a separation from +/// omicron-common's `DatasetKind` in case there might be some in the future. #[derive( Clone, Debug, Deserialize, Serialize, JsonSchema, PartialEq, Eq, Hash, )] #[serde(tag = "type", rename_all = "snake_case")] -pub enum DatasetKind { +pub enum DatasetType { + // TODO: `DatasetKind` uses `Cockroach`, not `CockroachDb`, for historical + // reasons. It may be worth using the same name for both. CockroachDb, Crucible, Clickhouse, @@ -141,17 +146,28 @@ pub enum DatasetKind { InternalDns, } -impl DatasetKind { +impl DatasetType { pub fn dataset_should_be_encrypted(&self) -> bool { match self { // We encrypt all datasets except Crucible. // // Crucible already performs encryption internally, and we // avoid double-encryption. - DatasetKind::Crucible => false, + DatasetType::Crucible => false, _ => true, } } + + pub fn kind(&self) -> DatasetKind { + match self { + Self::Crucible => DatasetKind::Crucible, + Self::CockroachDb => DatasetKind::Cockroach, + Self::Clickhouse => DatasetKind::Clickhouse, + Self::ClickhouseKeeper => DatasetKind::ClickhouseKeeper, + Self::ExternalDns => DatasetKind::ExternalDns, + Self::InternalDns => DatasetKind::InternalDns, + } + } } #[derive(Debug, thiserror::Error)] @@ -160,11 +176,11 @@ pub enum DatasetKindParseError { UnknownDataset(String), } -impl FromStr for DatasetKind { +impl FromStr for DatasetType { type Err = DatasetKindParseError; fn from_str(s: &str) -> Result { - use DatasetKind::*; + use DatasetType::*; let kind = match s { "crucible" => Crucible, "cockroachdb" => CockroachDb, @@ -182,9 +198,9 @@ impl FromStr for DatasetKind { } } -impl std::fmt::Display for DatasetKind { +impl std::fmt::Display for DatasetType { fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - use DatasetKind::*; + use DatasetType::*; let s = match self { Crucible => "crucible", CockroachDb => "cockroachdb", @@ -204,11 +220,11 @@ pub struct DatasetName { // A unique identifier for the Zpool on which the dataset is stored. pool_name: ZpoolName, // A name for the dataset within the Zpool. - kind: DatasetKind, + kind: DatasetType, } impl DatasetName { - pub fn new(pool_name: ZpoolName, kind: DatasetKind) -> Self { + pub fn new(pool_name: ZpoolName, kind: DatasetType) -> Self { Self { pool_name, kind } } @@ -216,7 +232,7 @@ impl DatasetName { &self.pool_name } - pub fn dataset(&self) -> &DatasetKind { + pub fn dataset(&self) -> &DatasetType { &self.kind } @@ -558,7 +574,7 @@ async fn ensure_zpool_dataset_is_encrypted( zpool_name: &ZpoolName, unencrypted_dataset: &str, ) -> Result<(), DatasetEncryptionMigrationError> { - let Ok(kind) = DatasetKind::from_str(&unencrypted_dataset) else { + let Ok(kind) = DatasetType::from_str(&unencrypted_dataset) else { info!(log, "Unrecognized dataset kind"); return Ok(()); }; @@ -799,7 +815,7 @@ mod test { #[test] fn serialize_dataset_name() { let pool = ZpoolName::new_internal(ZpoolUuid::new_v4()); - let kind = DatasetKind::Crucible; + let kind = DatasetType::Crucible; let name = DatasetName::new(pool, kind); serde_json::to_string(&name).unwrap(); } diff --git a/sled-storage/src/disk.rs b/sled-storage/src/disk.rs index c67cce0dfc..51980bea3a 100644 --- a/sled-storage/src/disk.rs +++ b/sled-storage/src/disk.rs @@ -8,79 +8,17 @@ use anyhow::bail; use camino::{Utf8Path, Utf8PathBuf}; use derive_more::From; use key_manager::StorageKeyRequester; -use omicron_common::api::external::Generation; -use omicron_common::disk::DiskIdentity; -use omicron_common::ledger::Ledgerable; +use omicron_common::disk::{DiskIdentity, DiskVariant}; use omicron_common::zpool_name::{ZpoolKind, ZpoolName}; use omicron_uuid_kinds::ZpoolUuid; -use schemars::JsonSchema; -use serde::{Deserialize, Serialize}; use sled_hardware::{ - DiskFirmware, DiskVariant, Partition, PooledDisk, PooledDiskError, - UnparsedDisk, + DiskFirmware, Partition, PooledDisk, PooledDiskError, UnparsedDisk, }; use slog::{info, Logger}; -use uuid::Uuid; use crate::config::MountConfig; use crate::dataset; -#[derive( - Clone, - Debug, - Deserialize, - Serialize, - JsonSchema, - PartialEq, - Eq, - Hash, - PartialOrd, - Ord, -)] -pub struct OmicronPhysicalDiskConfig { - pub identity: DiskIdentity, - pub id: Uuid, - pub pool_id: ZpoolUuid, -} - -#[derive( - Clone, Debug, Deserialize, Serialize, JsonSchema, PartialEq, Eq, Hash, -)] -pub struct OmicronPhysicalDisksConfig { - /// generation number of this configuration - /// - /// This generation number is owned by the control plane (i.e., RSS or - /// Nexus, depending on whether RSS-to-Nexus handoff has happened). It - /// should not be bumped within Sled Agent. - /// - /// Sled Agent rejects attempts to set the configuration to a generation - /// older than the one it's currently running. - pub generation: Generation, - - pub disks: Vec, -} - -impl Default for OmicronPhysicalDisksConfig { - fn default() -> Self { - Self { generation: Generation::new(), disks: vec![] } - } -} - -impl Ledgerable for OmicronPhysicalDisksConfig { - fn is_newer_than(&self, other: &OmicronPhysicalDisksConfig) -> bool { - self.generation > other.generation - } - - // No need to do this, the generation number is provided externally. - fn generation_bump(&mut self) {} -} - -impl OmicronPhysicalDisksConfig { - pub fn new() -> Self { - Self { generation: Generation::new(), disks: vec![] } - } -} - #[derive(Debug, thiserror::Error)] pub enum DiskError { #[error(transparent)] diff --git a/sled-storage/src/manager.rs b/sled-storage/src/manager.rs index e081bc5034..8168f32cea 100644 --- a/sled-storage/src/manager.rs +++ b/sled-storage/src/manager.rs @@ -8,7 +8,7 @@ use std::collections::HashSet; use crate::config::MountConfig; use crate::dataset::{DatasetName, CONFIG_DATASET}; -use crate::disk::{OmicronPhysicalDisksConfig, RawDisk}; +use crate::disk::RawDisk; use crate::error::Error; use crate::resources::{AllDisks, DisksManagementResult, StorageResources}; use camino::Utf8PathBuf; @@ -17,9 +17,10 @@ use futures::future::FutureExt; use illumos_utils::zfs::{Mountpoint, Zfs}; use illumos_utils::zpool::ZpoolName; use key_manager::StorageKeyRequester; -use omicron_common::disk::DiskIdentity; +use omicron_common::disk::{ + DiskIdentity, DiskVariant, OmicronPhysicalDisksConfig, +}; use omicron_common::ledger::Ledger; -use sled_hardware::DiskVariant; use slog::{info, o, warn, Logger}; use std::future::Future; use tokio::sync::{mpsc, oneshot, watch}; @@ -822,7 +823,7 @@ impl StorageManager { /// systems. #[cfg(all(test, target_os = "illumos"))] mod tests { - use crate::dataset::DatasetKind; + use crate::dataset::DatasetType; use crate::disk::RawSyntheticDisk; use crate::manager_test_harness::StorageManagerTestHarness; use crate::resources::DiskManagementError; @@ -1297,7 +1298,7 @@ mod tests { let dataset_id = Uuid::new_v4(); let zpool_name = ZpoolName::new_external(config.disks[0].pool_id); let dataset_name = - DatasetName::new(zpool_name.clone(), DatasetKind::Crucible); + DatasetName::new(zpool_name.clone(), DatasetType::Crucible); harness .handle() .upsert_filesystem(dataset_id, dataset_name) diff --git a/sled-storage/src/manager_test_harness.rs b/sled-storage/src/manager_test_harness.rs index 4409275017..40c7c5fbed 100644 --- a/sled-storage/src/manager_test_harness.rs +++ b/sled-storage/src/manager_test_harness.rs @@ -5,10 +5,13 @@ //! Utilities for creating a StorageManager under test. use crate::config::MountConfig; -use crate::disk::{OmicronPhysicalDisksConfig, RawDisk}; +use crate::disk::RawDisk; use crate::manager::{StorageHandle, StorageManager}; use camino::Utf8PathBuf; use key_manager::StorageKeyRequester; +use omicron_common::disk::{ + OmicronPhysicalDiskConfig, OmicronPhysicalDisksConfig, +}; use omicron_uuid_kinds::ZpoolUuid; use slog::{info, Logger}; use std::sync::{ @@ -333,7 +336,7 @@ impl StorageManagerTestHarness { .map(|raw| { let identity = raw.identity(); - crate::disk::OmicronPhysicalDiskConfig { + OmicronPhysicalDiskConfig { identity: identity.clone(), id: Uuid::new_v4(), pool_id: ZpoolUuid::new_v4(), diff --git a/sled-storage/src/resources.rs b/sled-storage/src/resources.rs index f02f62e0a6..98d6398d8b 100644 --- a/sled-storage/src/resources.rs +++ b/sled-storage/src/resources.rs @@ -6,21 +6,21 @@ use crate::config::MountConfig; use crate::dataset::{DatasetError, M2_DEBUG_DATASET}; -use crate::disk::{ - Disk, DiskError, OmicronPhysicalDiskConfig, OmicronPhysicalDisksConfig, - RawDisk, -}; +use crate::disk::{Disk, DiskError, RawDisk}; use crate::error::Error; use camino::Utf8PathBuf; use cfg_if::cfg_if; use illumos_utils::zpool::{PathInPool, ZpoolName}; use key_manager::StorageKeyRequester; use omicron_common::api::external::Generation; -use omicron_common::disk::DiskIdentity; +use omicron_common::disk::{ + DiskIdentity, DiskVariant, OmicronPhysicalDiskConfig, + OmicronPhysicalDisksConfig, +}; use omicron_uuid_kinds::ZpoolUuid; use schemars::JsonSchema; use serde::{Deserialize, Serialize}; -use sled_hardware::{DiskFirmware, DiskVariant}; +use sled_hardware::DiskFirmware; use slog::{info, o, warn, Logger}; use std::collections::BTreeMap; use std::sync::Arc; diff --git a/wicketd/src/rss_config.rs b/wicketd/src/rss_config.rs index b7545af19d..c6f2dd5892 100644 --- a/wicketd/src/rss_config.rs +++ b/wicketd/src/rss_config.rs @@ -296,7 +296,7 @@ impl CurrentRssConfig { external_certificates: self.external_certificates.clone(), recovery_silo: RecoverySiloConfig { silo_name: Name::try_from(RECOVERY_SILO_NAME).unwrap(), - user_name: UserId(RECOVERY_SILO_USERNAME.into()), + user_name: UserId::try_from(RECOVERY_SILO_USERNAME).unwrap(), user_password_hash, }, rack_network_config,