Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add resource limits #4605

Merged
merged 53 commits into from
Dec 13, 2023
Merged
Show file tree
Hide file tree
Changes from 45 commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
b59092c
Outline basic CRUD actions for quotas
zephraph Nov 21, 2023
159644c
Only allow viewing and updating quotas
zephraph Nov 21, 2023
21ba99c
Start outlining quota database type
zephraph Nov 22, 2023
0127dd1
Simplify quota model to be flatter
zephraph Dec 2, 2023
5c50ce5
Rough API shape of resource quotas
zephraph Dec 4, 2023
3346a33
Create quotas during silo creation
zephraph Dec 4, 2023
501f5ac
Update comment for internal silo
zephraph Dec 4, 2023
1894eb3
Merge branch 'main' into add-resource-limits
zephraph Dec 5, 2023
9e51431
Drop inner silo quota view
zephraph Dec 5, 2023
caf371b
Convert storage/memory to bytecount instead of ints
zephraph Dec 5, 2023
96877b2
WIP quota check CTE
zephraph Dec 6, 2023
8025899
Add quota checks for cpu, memory, storage
zephraph Dec 6, 2023
a4002ff
Join do_update and quota_check into a single CTE step
zephraph Dec 6, 2023
76c722f
Try (and fail) to resolve CTE type issues
zephraph Dec 6, 2023
f8e2afc
Resolve CTE compliation issues... ???
zephraph Dec 6, 2023
3d4ebe2
Convert quota detlas to sql types; fix borrow issues
zephraph Dec 6, 2023
d317629
Merge branch 'main' into add-resource-limits
zephraph Dec 6, 2023
dcf974e
Remove a todo
zephraph Dec 6, 2023
83c9872
Wire up quota limit error handling
zephraph Dec 6, 2023
7e779a0
Fix missing quota specifier
zephraph Dec 6, 2023
9859e12
SELECT got you down? Try JOIN today!
smklein Dec 6, 2023
45eb0ef
Whoops, removing test cruft
smklein Dec 6, 2023
55ff0b2
Only perform quota checks if allocating resources
zephraph Dec 7, 2023
2ee6d36
Adjust some quotas in hopes of not failing all the tests
zephraph Dec 7, 2023
4c58f9e
Add WIP integration test setup for quotas
zephraph Dec 7, 2023
3b014c8
Show that quota checks are failing when they shouldn't
zephraph Dec 7, 2023
bd950bc
Ensure default silo gets constructed w/ a quota
zephraph Dec 8, 2023
4b0d236
Flesh out basic quota tests
zephraph Dec 8, 2023
0ad4340
Remove unnecessary transaction
zephraph Dec 8, 2023
4f6d49f
Remove quotas tag, rename system silo list
zephraph Dec 8, 2023
4111cc2
Drop custom quota configuration for standard half-rack config
zephraph Dec 8, 2023
501563b
Fixup auth tests
zephraph Dec 8, 2023
f15a451
Better docs
zephraph Dec 8, 2023
569b995
Delete quotas if the silo is deleted
zephraph Dec 8, 2023
ac8a2f3
Merge branch 'main' into add-resource-limits
zephraph Dec 8, 2023
43beca8
Add a migration for silo_quotas and backfill data
zephraph Dec 8, 2023
2c11210
Bump schema version
zephraph Dec 8, 2023
025cd84
Merge branch 'main' into add-resource-limits
zephraph Dec 11, 2023
8a7794b
Use insufficient capacity error
zephraph Dec 11, 2023
f8955a5
half_rack -> from_sled_count
zephraph Dec 11, 2023
77f6d29
Merge branch 'main' into add-resource-limits
zephraph Dec 11, 2023
760675d
Merge branch 'main' into add-resource-limits
zephraph Dec 12, 2023
371bc98
Fix db deadlock caused by checking authz before silo is created
zephraph Dec 12, 2023
de951a7
Add quotas for silo cert tests
zephraph Dec 12, 2023
410d3ae
Fix type failures
zephraph Dec 12, 2023
cd22388
Fix docs build
zephraph Dec 12, 2023
e86ec72
Update nexus/db-queries/src/db/queries/virtual_provisioning_collectio…
zephraph Dec 12, 2023
5974325
Use an arbitrarily high value over something that looks real but isn't
zephraph Dec 12, 2023
c602e29
Enhance tests to check errors; fix overflow
zephraph Dec 13, 2023
fe87ca8
Drop expects for unwraps when we don't expect them to fail
zephraph Dec 13, 2023
11ab326
Update comment
zephraph Dec 13, 2023
e369851
Fix clippy errors
zephraph Dec 13, 2023
635d1ba
ci: add quota setup to deploy test
rcgoodfellow Dec 13, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions common/src/api/external/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -702,6 +702,7 @@ pub enum ResourceType {
Silo,
SiloUser,
SiloGroup,
SiloQuotas,
IdentityProvider,
SamlIdentityProvider,
SshKey,
Expand Down
2 changes: 2 additions & 0 deletions nexus/db-model/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ mod system_update;
// for join-based marker trait generation.
mod ipv4_nat_entry;
pub mod queries;
mod quota;
mod rack;
mod region;
mod region_snapshot;
Expand Down Expand Up @@ -139,6 +140,7 @@ pub use physical_disk::*;
pub use physical_disk_kind::*;
pub use producer_endpoint::*;
pub use project::*;
pub use quota::*;
pub use rack::*;
pub use region::*;
pub use region_snapshot::*;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
//! for the construction of this query.

use crate::schema::silo;
use crate::schema::silo_quotas;
use crate::schema::virtual_provisioning_collection;

table! {
Expand All @@ -28,11 +29,32 @@ table! {
}
}

table! {
quotas (silo_id) {
silo_id -> Uuid,
cpus -> Int8,
memory -> Int8,
storage -> Int8,
}
}

table! {
silo_provisioned {
id -> Uuid,
virtual_disk_bytes_provisioned -> Int8,
cpus_provisioned -> Int8,
ram_provisioned -> Int8,
}
}

diesel::allow_tables_to_appear_in_same_query!(silo, parent_silo,);

diesel::allow_tables_to_appear_in_same_query!(
virtual_provisioning_collection,
silo_quotas,
parent_silo,
all_collections,
do_update,
quotas,
silo_provisioned
);
109 changes: 109 additions & 0 deletions nexus/db-model/src/quota.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
use super::ByteCount;
use crate::schema::silo_quotas;
use chrono::{DateTime, Utc};
use nexus_types::external_api::{params, views};
use serde::{Deserialize, Serialize};
use uuid::Uuid;

#[derive(
Queryable,
Insertable,
Debug,
Clone,
Selectable,
Serialize,
Deserialize,
AsChangeset,
)]
#[diesel(table_name = silo_quotas)]
pub struct SiloQuotas {
pub silo_id: Uuid,
pub time_created: DateTime<Utc>,
pub time_modified: DateTime<Utc>,

/// The number of CPUs that this silo is allowed to use
pub cpus: i64,

/// The amount of memory (in bytes) that this silo is allowed to use
#[diesel(column_name = memory_bytes)]
pub memory: ByteCount,

/// The amount of storage (in bytes) that this silo is allowed to use
#[diesel(column_name = storage_bytes)]
pub storage: ByteCount,
}

impl SiloQuotas {
pub fn new(
silo_id: Uuid,
cpus: i64,
memory: ByteCount,
storage: ByteCount,
) -> Self {
Self {
silo_id,
time_created: Utc::now(),
time_modified: Utc::now(),
cpus,
memory,
storage,
}
}

pub fn from_sled_count(silo_id: Uuid, num_sleds: u32) -> Self {
let count = params::SiloQuotasCreate::from_sled_count(num_sleds);
Self::new(
silo_id,
count.cpus,
count.memory.into(),
count.storage.into(),
)
}
}

impl From<SiloQuotas> for views::SiloQuotas {
fn from(silo_quotas: SiloQuotas) -> Self {
Self {
silo_id: silo_quotas.silo_id,
cpus: silo_quotas.cpus,
memory: silo_quotas.memory.into(),
storage: silo_quotas.storage.into(),
}
}
}

impl From<views::SiloQuotas> for SiloQuotas {
fn from(silo_quotas: views::SiloQuotas) -> Self {
Self {
silo_id: silo_quotas.silo_id,
time_created: Utc::now(),
time_modified: Utc::now(),
cpus: silo_quotas.cpus,
memory: silo_quotas.memory.into(),
storage: silo_quotas.storage.into(),
}
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why would you need to go from view to model? Is this used?


// Describes a set of updates for the [`SiloQuotas`] model.
#[derive(AsChangeset)]
#[diesel(table_name = silo_quotas)]
pub struct SiloQuotasUpdate {
pub cpus: Option<i64>,
#[diesel(column_name = memory_bytes)]
pub memory: Option<i64>,
#[diesel(column_name = storage_bytes)]
pub storage: Option<i64>,
pub time_modified: DateTime<Utc>,
}

impl From<params::SiloQuotasUpdate> for SiloQuotasUpdate {
fn from(params: params::SiloQuotasUpdate) -> Self {
Self {
cpus: params.cpus,
memory: params.memory.map(|f| f.into()),
storage: params.storage.map(|f| f.into()),
time_modified: Utc::now(),
}
}
}
13 changes: 12 additions & 1 deletion nexus/db-model/src/schema.rs
Original file line number Diff line number Diff line change
Expand Up @@ -409,6 +409,17 @@ table! {
}
}

table! {
silo_quotas(silo_id) {
silo_id -> Uuid,
time_created -> Timestamptz,
time_modified -> Timestamptz,
cpus -> Int8,
memory_bytes -> Int8,
storage_bytes -> Int8,
}
}

table! {
network_interface (id) {
id -> Uuid,
Expand Down Expand Up @@ -1322,7 +1333,7 @@ table! {
///
/// This should be updated whenever the schema is changed. For more details,
/// refer to: schema/crdb/README.adoc
pub const SCHEMA_VERSION: SemverVersion = SemverVersion::new(19, 0, 0);
pub const SCHEMA_VERSION: SemverVersion = SemverVersion::new(20, 0, 0);

allow_tables_to_appear_in_same_query!(
system_update,
Expand Down
1 change: 1 addition & 0 deletions nexus/db-queries/src/db/datastore/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,7 @@ mod network_interface;
mod oximeter;
mod physical_disk;
mod project;
mod quota;
mod rack;
mod region;
mod region_snapshot;
Expand Down
127 changes: 127 additions & 0 deletions nexus/db-queries/src/db/datastore/quota.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
use super::DataStore;
use crate::authz;
use crate::context::OpContext;
use crate::db;
use crate::db::error::public_error_from_diesel;
use crate::db::error::ErrorHandler;
use crate::db::pagination::paginated;
use crate::db::pool::DbConnection;
use async_bb8_diesel::AsyncRunQueryDsl;
use diesel::prelude::*;
use nexus_db_model::SiloQuotas;
use nexus_db_model::SiloQuotasUpdate;
use omicron_common::api::external::DataPageParams;
use omicron_common::api::external::DeleteResult;
use omicron_common::api::external::Error;
use omicron_common::api::external::ListResultVec;
use omicron_common::api::external::ResourceType;
use omicron_common::api::external::UpdateResult;
use uuid::Uuid;

impl DataStore {
/// Creates new quotas for a silo. This is grouped with silo creation
/// and shouldn't be called outside of that flow.
///
/// An authz check _cannot_ be performed here because the authz initialization
/// isn't complete and will lead to a db deadlock.
///
/// See https://github.com/oxidecomputer/omicron/blob/07eb7dafc20e35e44edf429fcbb759cbb33edd5f/nexus/db-queries/src/db/datastore/rack.rs#L407-L410
pub async fn silo_quotas_create(
&self,
conn: &async_bb8_diesel::Connection<DbConnection>,
authz_silo: &authz::Silo,
quotas: SiloQuotas,
) -> Result<(), Error> {
let silo_id = authz_silo.id();
use db::schema::silo_quotas::dsl;

diesel::insert_into(dsl::silo_quotas)
.values(quotas)
.execute_async(conn)
.await
.map_err(|e| {
public_error_from_diesel(
e,
ErrorHandler::Conflict(
ResourceType::SiloQuotas,
&silo_id.to_string(),
),
)
})
.map(|_| ())
zephraph marked this conversation as resolved.
Show resolved Hide resolved
}

pub async fn silo_quotas_delete(
&self,
opctx: &OpContext,
conn: &async_bb8_diesel::Connection<DbConnection>,
authz_silo: &authz::Silo,
) -> DeleteResult {
// Given that the quotas right now are somewhat of an extension of the
// Silo we just check for delete permission on the silo itself.
opctx.authorize(authz::Action::Delete, authz_silo).await?;

use db::schema::silo_quotas::dsl;
diesel::delete(dsl::silo_quotas)
.filter(dsl::silo_id.eq(authz_silo.id()))
.execute_async(conn)
.await
.map_err(|e| public_error_from_diesel(e, ErrorHandler::Server))?;

Ok(())
}

pub async fn silo_update_quota(
&self,
opctx: &OpContext,
authz_silo: &authz::Silo,
updates: SiloQuotasUpdate,
) -> UpdateResult<SiloQuotas> {
opctx.authorize(authz::Action::Modify, authz_silo).await?;
use db::schema::silo_quotas::dsl;
let silo_id = authz_silo.id();
diesel::update(dsl::silo_quotas)
.filter(dsl::silo_id.eq(silo_id))
.set(updates)
.returning(SiloQuotas::as_returning())
.get_result_async(&*self.pool_connection_authorized(opctx).await?)
.await
.map_err(|e| {
public_error_from_diesel(
e,
ErrorHandler::Conflict(
ResourceType::SiloQuotas,
&silo_id.to_string(),
),
)
})
}

pub async fn silo_quotas_view(
&self,
opctx: &OpContext,
authz_silo: &authz::Silo,
) -> Result<SiloQuotas, Error> {
opctx.authorize(authz::Action::Read, authz_silo).await?;
use db::schema::silo_quotas::dsl;
dsl::silo_quotas
.filter(dsl::silo_id.eq(authz_silo.id()))
.first_async(&*self.pool_connection_authorized(opctx).await?)
.await
.map_err(|e| public_error_from_diesel(e, ErrorHandler::Server))
}

pub async fn fleet_list_quotas(
&self,
opctx: &OpContext,
pagparams: &DataPageParams<'_, Uuid>,
) -> ListResultVec<SiloQuotas> {
opctx.authorize(authz::Action::ListChildren, &authz::FLEET).await?;
use db::schema::silo_quotas::dsl;
paginated(dsl::silo_quotas, dsl::silo_id, pagparams)
.select(SiloQuotas::as_select())
.load_async(&*self.pool_connection_authorized(opctx).await?)
.await
.map_err(|e| public_error_from_diesel(e, ErrorHandler::Server))
}
}
4 changes: 4 additions & 0 deletions nexus/db-queries/src/db/datastore/rack.rs
Original file line number Diff line number Diff line change
Expand Up @@ -912,6 +912,10 @@ mod test {
name: "test-silo".parse().unwrap(),
description: String::new(),
},
// Set a default quota of a half rack's worth of resources
quotas: external_params::SiloQuotasCreate::from_sled_count(
16,
),
discoverable: false,
identity_mode: SiloIdentityMode::LocalOnly,
admin_group_name: None,
Expand Down
Loading
Loading