Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[nexus] Support Bundle HTTP endpoint implementations #7187

Open
wants to merge 31 commits into
base: support-bundle-bg-task
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
03af36d
[nexus] Implement Support Bundle API
smklein Nov 28, 2024
9b2176c
Merge branch 'support-bundle-bg-task' into support-bundle-wire-up-htt…
smklein Nov 28, 2024
eb491da
Merge branch 'support-bundle-bg-task' into support-bundle-wire-up-htt…
smklein Dec 6, 2024
f983a9d
Merge branch 'support-bundle-bg-task' into support-bundle-wire-up-htt…
smklein Dec 6, 2024
65c3218
Merge branch 'support-bundle-bg-task' into support-bundle-wire-up-htt…
smklein Dec 10, 2024
b90aa2e
Merge branch 'support-bundle-bg-task' into support-bundle-wire-up-htt…
smklein Dec 10, 2024
c4cc0f0
Merge branch 'support-bundle-bg-task' into support-bundle-wire-up-htt…
smklein Dec 16, 2024
2f71eef
Merge branch 'support-bundle-bg-task' into support-bundle-wire-up-htt…
smklein Dec 17, 2024
660fa66
using new sled agent API
smklein Dec 17, 2024
04ed8d7
Merge branch 'support-bundle-bg-task' into support-bundle-wire-up-htt…
smklein Dec 17, 2024
c133782
Merge branch 'support-bundle-bg-task' into support-bundle-wire-up-htt…
smklein Dec 17, 2024
759a9f8
Fix merge (renamed method)
smklein Dec 17, 2024
053fddc
Starting to wire up authz for SupportBundle
smklein Dec 18, 2024
30bfe98
Merge branch 'support-bundle-bg-task' into support-bundle-wire-up-htt…
smklein Dec 19, 2024
bbeb75d
Working through authz tests
smklein Dec 20, 2024
6a2fa59
unauthorized tests working for subset of endpoints
smklein Dec 20, 2024
a0e5326
Update coverage of support bundle endpoints
smklein Dec 20, 2024
fae39ee
clippy
smklein Dec 20, 2024
d263770
Merge branch 'support-bundle-bg-task' into support-bundle-wire-up-htt…
smklein Jan 6, 2025
94c555d
Merge branch 'support-bundle-bg-task' into support-bundle-wire-up-htt…
smklein Jan 6, 2025
df4232d
Merge branch 'support-bundle-bg-task' into support-bundle-wire-up-htt…
smklein Jan 6, 2025
edb3422
Merge branch 'support-bundle-bg-task' into support-bundle-wire-up-htt…
smklein Jan 7, 2025
b73f3ee
fixing tests
smklein Jan 7, 2025
608f1b2
ordering matters
smklein Jan 7, 2025
7d7d7fd
cleanup
smklein Jan 7, 2025
3fec97e
Integration tests
smklein Jan 8, 2025
b8512d1
Merge branch 'support-bundle-bg-task' into support-bundle-wire-up-htt…
smklein Jan 8, 2025
b0a3aec
Merge branch 'support-bundle-bg-task' into support-bundle-wire-up-htt…
smklein Jan 9, 2025
e4c32e8
Merge branch 'support-bundle-bg-task' into support-bundle-wire-up-htt…
smklein Jan 9, 2025
fb5cae9
Fix merge
smklein Jan 9, 2025
f3144c3
omdb tests
smklein Jan 9, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions common/src/api/external/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -979,6 +979,7 @@ pub enum ResourceType {
Instance,
LoopbackAddress,
SwitchPortSettings,
SupportBundle,
IpPool,
IpPoolResource,
InstanceNetworkInterface,
Expand Down
8 changes: 4 additions & 4 deletions dev-tools/omdb/tests/successes.out
Original file line number Diff line number Diff line change
Expand Up @@ -698,11 +698,11 @@ task: "service_zone_nat_tracker"
last completion reported error: inventory collection is None

task: "support_bundle_collector"
configured period: every <REDACTED_DURATION>s
configured period: every <REDACTED_DURATION>days <REDACTED_DURATION>h <REDACTED_DURATION>m <REDACTED_DURATION>s
currently executing: no
last completed activation: <REDACTED ITERATIONS>, triggered by a periodic timer firing
started at <REDACTED_TIMESTAMP> (<REDACTED DURATION>s ago) and ran for <REDACTED DURATION>ms
last completion reported error: task disabled
warning: unknown background task: "support_bundle_collector" (don't know how to interpret details: Object {"cleanup_err": Null, "cleanup_report": Object {"db_destroying_bundles_removed": Number(0), "db_failing_bundles_updated": Number(0), "sled_bundles_delete_failed": Number(0), "sled_bundles_deleted_not_found": Number(0), "sled_bundles_deleted_ok": Number(0)}, "collection_err": Null, "collection_report": Null})

task: "switch_port_config_manager"
configured period: every <REDACTED_DURATION>s
Expand Down Expand Up @@ -1150,11 +1150,11 @@ task: "service_zone_nat_tracker"
last completion reported error: inventory collection is None

task: "support_bundle_collector"
configured period: every <REDACTED_DURATION>s
configured period: every <REDACTED_DURATION>days <REDACTED_DURATION>h <REDACTED_DURATION>m <REDACTED_DURATION>s
currently executing: no
last completed activation: <REDACTED ITERATIONS>, triggered by a periodic timer firing
started at <REDACTED_TIMESTAMP> (<REDACTED DURATION>s ago) and ran for <REDACTED DURATION>ms
last completion reported error: task disabled
warning: unknown background task: "support_bundle_collector" (don't know how to interpret details: Object {"cleanup_err": Null, "cleanup_report": Object {"db_destroying_bundles_removed": Number(0), "db_failing_bundles_updated": Number(0), "sled_bundles_delete_failed": Number(0), "sled_bundles_deleted_not_found": Number(0), "sled_bundles_deleted_ok": Number(0)}, "collection_err": Null, "collection_report": Null})

task: "switch_port_config_manager"
configured period: every <REDACTED_DURATION>s
Expand Down
2 changes: 1 addition & 1 deletion docs/adding-an-endpoint.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ this document should act as a jumping-off point.
=== **Testing**

* Authorization
** There exists a https://github.com/oxidecomputer/omicron/blob/main/nexus/src/authz/policy_test[policy test] which compares all Oso objects against an expected policy. New resources are usually added to https://github.com/oxidecomputer/omicron/blob/main/nexus/src/authz/policy_test/resources.rs[resources.rs] to get coverage.
** There exists a https://github.com/oxidecomputer/omicron/blob/main/nexus/db-queries/src/policy_test[policy test] which compares all Oso objects against an expected policy. New resources are usually added to https://github.com/oxidecomputer/omicron/blob/main/nexus/db-queries/src/policy_test/resources.rs[resources.rs] to get coverage.
* OpenAPI
** Once you've added or changed endpoint definitions in `nexus-external-api` or `nexus-internal-api`, you'll need to update the corresponding OpenAPI documents (the JSON files in `openapi/`).
** To update all OpenAPI documents, run `cargo xtask openapi generate`.
Expand Down
1 change: 1 addition & 0 deletions nexus/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@ progenitor-client.workspace = true
propolis-client.workspace = true
qorb.workspace = true
rand.workspace = true
range-requests.workspace = true
ref-cast.workspace = true
reqwest = { workspace = true, features = ["json"] }
ring.workspace = true
Expand Down
8 changes: 8 additions & 0 deletions nexus/auth/src/authz/api_resources.rs
Original file line number Diff line number Diff line change
Expand Up @@ -995,6 +995,14 @@ authz_resource! {
polar_snippet = FleetChild,
}

authz_resource! {
name = "SupportBundle",
parent = "Fleet",
primary_key = { uuid_kind = SupportBundleKind },
roles_allowed = false,
polar_snippet = FleetChild,
}

authz_resource! {
name = "PhysicalDisk",
parent = "Fleet",
Expand Down
1 change: 1 addition & 0 deletions nexus/auth/src/authz/oso_generic.rs
Original file line number Diff line number Diff line change
Expand Up @@ -154,6 +154,7 @@ pub fn make_omicron_oso(log: &slog::Logger) -> Result<OsoInit, anyhow::Error> {
Silo::init(),
SiloUser::init(),
SiloGroup::init(),
SupportBundle::init(),
IdentityProvider::init(),
SamlIdentityProvider::init(),
Sled::init(),
Expand Down
4 changes: 4 additions & 0 deletions nexus/db-model/src/support_bundle.rs
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,10 @@ impl SupportBundle {
assigned_nexus: Some(nexus_id.into()),
}
}

pub fn id(&self) -> SupportBundleUuid {
self.id.into()
}
}

impl From<SupportBundle> for SupportBundleView {
Expand Down
67 changes: 41 additions & 26 deletions nexus/db-queries/src/db/datastore/support_bundle.rs
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ use crate::context::OpContext;
use crate::db;
use crate::db::error::public_error_from_diesel;
use crate::db::error::ErrorHandler;
use crate::db::lookup::LookupPath;
use crate::db::model::Dataset;
use crate::db::model::DatasetKind;
use crate::db::model::SupportBundle;
Expand Down Expand Up @@ -164,16 +165,10 @@ impl DataStore {
opctx: &OpContext,
id: SupportBundleUuid,
) -> LookupResult<SupportBundle> {
opctx.authorize(authz::Action::Read, &authz::FLEET).await?;
use db::schema::support_bundle::dsl;
let (.., db_bundle) =
LookupPath::new(opctx, self).support_bundle(id).fetch().await?;

let conn = self.pool_connection_authorized(opctx).await?;
dsl::support_bundle
.filter(dsl::id.eq(id.into_untyped_uuid()))
.select(SupportBundle::as_select())
.first_async::<SupportBundle>(&*conn)
.await
.map_err(|e| public_error_from_diesel(e, ErrorHandler::Server))
Ok(db_bundle)
}

/// Lists one page of support bundles
Expand Down Expand Up @@ -420,18 +415,20 @@ impl DataStore {
pub async fn support_bundle_update(
&self,
opctx: &OpContext,
id: SupportBundleUuid,
authz_bundle: &authz::SupportBundle,
state: SupportBundleState,
) -> Result<(), Error> {
opctx.authorize(authz::Action::Modify, &authz::FLEET).await?;
opctx.authorize(authz::Action::Modify, authz_bundle).await?;

use db::schema::support_bundle::dsl;

let id = authz_bundle.id().into_untyped_uuid();
let conn = self.pool_connection_authorized(opctx).await?;
let result = diesel::update(dsl::support_bundle)
.filter(dsl::id.eq(id.into_untyped_uuid()))
.filter(dsl::id.eq(id))
.filter(dsl::state.eq_any(state.valid_old_states()))
.set(dsl::state.eq(state))
.check_if_exists::<SupportBundle>(id.into_untyped_uuid())
.check_if_exists::<SupportBundle>(id)
.execute_and_check(&conn)
.await
.map_err(|e| public_error_from_diesel(e, ErrorHandler::Server))?;
Expand All @@ -454,20 +451,21 @@ impl DataStore {
pub async fn support_bundle_delete(
&self,
opctx: &OpContext,
id: SupportBundleUuid,
authz_bundle: &authz::SupportBundle,
) -> Result<(), Error> {
opctx.authorize(authz::Action::Modify, &authz::FLEET).await?;
opctx.authorize(authz::Action::Delete, authz_bundle).await?;

use db::schema::support_bundle::dsl;

let id = authz_bundle.id().into_untyped_uuid();
let conn = self.pool_connection_authorized(opctx).await?;
diesel::delete(dsl::support_bundle)
.filter(
dsl::state
.eq(SupportBundleState::Destroying)
.or(dsl::state.eq(SupportBundleState::Failed)),
)
.filter(dsl::id.eq(id.into_untyped_uuid()))
.filter(dsl::id.eq(id))
.execute_async(&*conn)
.await
.map(|_rows_modified| ())
Expand Down Expand Up @@ -495,13 +493,24 @@ mod test {
use nexus_types::deployment::BlueprintZoneDisposition;
use nexus_types::deployment::BlueprintZoneFilter;
use nexus_types::deployment::BlueprintZoneType;
use omicron_common::api::external::LookupType;
use omicron_common::api::internal::shared::DatasetKind::Debug as DebugDatasetKind;
use omicron_test_utils::dev;
use omicron_uuid_kinds::DatasetUuid;
use omicron_uuid_kinds::PhysicalDiskUuid;
use omicron_uuid_kinds::SledUuid;
use rand::Rng;

fn authz_support_bundle_from_id(
id: SupportBundleUuid,
) -> authz::SupportBundle {
authz::SupportBundle::new(
authz::FLEET,
id,
LookupType::ById(id.into_untyped_uuid()),
)
}

// Pool/Dataset pairs, for debug datasets only.
struct TestPool {
pool: ZpoolUuid,
Expand Down Expand Up @@ -715,10 +724,11 @@ mod test {

// When we update the state of the bundles, the list results
// should also be filtered.
let authz_bundle = authz_support_bundle_from_id(bundle_a1.id.into());
datastore
.support_bundle_update(
&opctx,
bundle_a1.id.into(),
&authz_bundle,
SupportBundleState::Active,
)
.await
Expand Down Expand Up @@ -816,11 +826,11 @@ mod test {
// database.
//
// We should still expect to hit capacity limits.

let authz_bundle = authz_support_bundle_from_id(bundles[0].id.into());
datastore
.support_bundle_update(
&opctx,
bundles[0].id.into(),
&authz_bundle,
SupportBundleState::Destroying,
)
.await
Expand All @@ -835,8 +845,9 @@ mod test {
// If we delete a bundle, it should be gone. This means we can
// re-allocate from that dataset which was just freed up.

let authz_bundle = authz_support_bundle_from_id(bundles[0].id.into());
datastore
.support_bundle_delete(&opctx, bundles[0].id.into())
.support_bundle_delete(&opctx, &authz_bundle)
.await
.expect("Should be able to destroy this bundle");
datastore
Expand Down Expand Up @@ -888,11 +899,11 @@ mod test {
assert_eq!(bundle, observed_bundles[0]);

// Destroy the bundle, observe the new state

let authz_bundle = authz_support_bundle_from_id(bundle.id.into());
datastore
.support_bundle_update(
&opctx,
bundle.id.into(),
&authz_bundle,
SupportBundleState::Destroying,
)
.await
Expand All @@ -905,8 +916,9 @@ mod test {

// Delete the bundle, observe that it's gone

let authz_bundle = authz_support_bundle_from_id(bundle.id.into());
datastore
.support_bundle_delete(&opctx, bundle.id.into())
.support_bundle_delete(&opctx, &authz_bundle)
.await
.expect("Should be able to destroy our bundle");
let observed_bundles = datastore
Expand Down Expand Up @@ -1146,10 +1158,11 @@ mod test {
);

// Start the deletion of this bundle
let authz_bundle = authz_support_bundle_from_id(bundle.id.into());
datastore
.support_bundle_update(
&opctx,
bundle.id.into(),
&authz_bundle,
SupportBundleState::Destroying,
)
.await
Expand Down Expand Up @@ -1314,8 +1327,9 @@ mod test {
.unwrap()
.contains(FAILURE_REASON_NO_DATASET));

let authz_bundle = authz_support_bundle_from_id(bundle.id.into());
datastore
.support_bundle_delete(&opctx, bundle.id.into())
.support_bundle_delete(&opctx, &authz_bundle)
.await
.expect("Should have been able to delete support bundle");

Expand Down Expand Up @@ -1377,10 +1391,11 @@ mod test {
//
// This is what we would do when we finish collecting, and
// provisioned storage on a sled.
let authz_bundle = authz_support_bundle_from_id(bundle.id.into());
datastore
.support_bundle_update(
&opctx,
bundle.id.into(),
&authz_bundle,
SupportBundleState::Active,
)
.await
Expand Down
15 changes: 15 additions & 0 deletions nexus/db-queries/src/db/lookup.rs
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ use omicron_common::api::external::Error;
use omicron_common::api::external::InternalContext;
use omicron_common::api::external::{LookupResult, LookupType, ResourceType};
use omicron_uuid_kinds::PhysicalDiskUuid;
use omicron_uuid_kinds::SupportBundleUuid;
use omicron_uuid_kinds::TufRepoKind;
use omicron_uuid_kinds::TypedUuid;
use uuid::Uuid;
Expand Down Expand Up @@ -390,6 +391,11 @@ impl<'a> LookupPath<'a> {
PhysicalDisk::PrimaryKey(Root { lookup_root: self }, id)
}

/// Select a resource of type SupportBundle, identified by its id
pub fn support_bundle(self, id: SupportBundleUuid) -> SupportBundle<'a> {
SupportBundle::PrimaryKey(Root { lookup_root: self }, id)
}

pub fn silo_image_id(self, id: Uuid) -> SiloImage<'a> {
SiloImage::PrimaryKey(Root { lookup_root: self }, id)
}
Expand Down Expand Up @@ -878,6 +884,15 @@ lookup_resource! {
primary_key_columns = [ { column_name = "id", uuid_kind = PhysicalDiskKind } ]
}

lookup_resource! {
name = "SupportBundle",
ancestors = [],
children = [],
lookup_by_name = false,
soft_deletes = false,
primary_key_columns = [ { column_name = "id", uuid_kind = SupportBundleKind } ]
}

lookup_resource! {
name = "TufRepo",
ancestors = [],
Expand Down
1 change: 1 addition & 0 deletions nexus/db-queries/src/policy_test/resource_builder.rs
Original file line number Diff line number Diff line change
Expand Up @@ -271,6 +271,7 @@ impl_dyn_authorized_resource_for_resource!(authz::SiloUser);
impl_dyn_authorized_resource_for_resource!(authz::Sled);
impl_dyn_authorized_resource_for_resource!(authz::Snapshot);
impl_dyn_authorized_resource_for_resource!(authz::SshKey);
impl_dyn_authorized_resource_for_resource!(authz::SupportBundle);
impl_dyn_authorized_resource_for_resource!(authz::TufArtifact);
impl_dyn_authorized_resource_for_resource!(authz::TufRepo);
impl_dyn_authorized_resource_for_resource!(authz::Vpc);
Expand Down
9 changes: 9 additions & 0 deletions nexus/db-queries/src/policy_test/resources.rs
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ use nexus_db_model::SemverVersion;
use omicron_common::api::external::LookupType;
use omicron_uuid_kinds::GenericUuid;
use omicron_uuid_kinds::PhysicalDiskUuid;
use omicron_uuid_kinds::SupportBundleUuid;
use oso::PolarClass;
use std::collections::BTreeSet;
use uuid::Uuid;
Expand Down Expand Up @@ -111,6 +112,14 @@ pub async fn make_resources(
LookupType::ById(physical_disk_id.into_untyped_uuid()),
));

let support_bundle_id: SupportBundleUuid =
"d9f923f6-caf3-4c83-96f9-8ffe8c627dd2".parse().unwrap();
builder.new_resource(authz::SupportBundle::new(
authz::FLEET,
support_bundle_id,
LookupType::ById(support_bundle_id.into_untyped_uuid()),
));

let device_user_code = String::from("a-device-user-code");
builder.new_resource(authz::DeviceAuthRequest::new(
authz::FLEET,
Expand Down
14 changes: 14 additions & 0 deletions nexus/db-queries/tests/output/authz-roles.out
Original file line number Diff line number Diff line change
Expand Up @@ -1034,6 +1034,20 @@ resource: PhysicalDisk id "c9f923f6-caf3-4c83-96f9-8ffe8c627dd2"
silo1-proj1-viewer ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘
unauthenticated ! ! ! ! ! ! ! !

resource: SupportBundle id "d9f923f6-caf3-4c83-96f9-8ffe8c627dd2"

USER Q R LC RP M MP CC D
fleet-admin ✘ ✔ ✔ ✔ ✔ ✔ ✔ ✔
fleet-collaborator ✘ ✔ ✔ ✔ ✘ ✘ ✘ ✘
fleet-viewer ✘ ✔ ✔ ✔ ✘ ✘ ✘ ✘
silo1-admin ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘
silo1-collaborator ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘
silo1-viewer ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘
silo1-proj1-admin ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘
silo1-proj1-collaborator ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘
silo1-proj1-viewer ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘
unauthenticated ! ! ! ! ! ! ! !

resource: DeviceAuthRequest "a-device-user-code"

USER Q R LC RP M MP CC D
Expand Down
7 changes: 7 additions & 0 deletions nexus/src/app/background/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -140,6 +140,13 @@ pub use init::BackgroundTasksData;
pub use init::BackgroundTasksInitializer;
pub use tasks::saga_recovery::SagaRecoveryHelpers;

// Expose background task outputs to they can be deserialized and
// observed.
pub mod task_output {
pub use super::tasks::support_bundle_collector::CleanupReport;
pub use super::tasks::support_bundle_collector::CollectionReport;
}

use futures::future::BoxFuture;
use nexus_auth::context::OpContext;

Expand Down
Loading
Loading