Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

server: refactor instance initialization #715

Merged
merged 55 commits into from
Jul 18, 2024
Merged
Show file tree
Hide file tree
Changes from 50 commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
a69c52b
[WIP] refactor to make VM creation async
gjcolombo Jun 19, 2024
21869eb
[WIP] get rid of some request queue stuff we don't need
gjcolombo Jun 21, 2024
301ad33
[WIP] checkpoint: move some event handling logic
gjcolombo Jun 21, 2024
df4658d
[WIP] more vm lifecycle routines
gjcolombo Jun 21, 2024
356f84f
[WIP] do_reboot
gjcolombo Jun 21, 2024
a7af98f
[WIP] migration's not there but it builds at least
gjcolombo Jun 21, 2024
52d6eb8
[WIP] rip out block_on
gjcolombo Jun 22, 2024
e16af9b
[WIP] cut the server over to the new VM module (semi-disaster)
gjcolombo Jun 22, 2024
0c98b80
[WIP] refactor state driver to resolve disaster
gjcolombo Jun 24, 2024
c23065c
[WIP] figure out VM services
gjcolombo Jun 24, 2024
a2dbd78
[WIP] partially replace server's VmController
gjcolombo Jun 25, 2024
a5f1ec2
[WIP] improve ActiveVm cleanup (maybe)
gjcolombo Jun 25, 2024
3156df6
[WIP] fix VM start
gjcolombo Jun 25, 2024
e54ec7c
[WIP] resume fixing server build errors
gjcolombo Jun 25, 2024
eb33b42
[WIP] shunt Crucible mutation to worker thread
gjcolombo Jun 25, 2024
1a3430d
[WIP] clean up warnings
gjcolombo Jun 25, 2024
8cf4b9a
[WIP] oops I blocked the executor
gjcolombo Jun 25, 2024
c2fc6d2
[WIP] improve log inheritance
gjcolombo Jun 26, 2024
290cfbd
[WIP] hook up destination side of migration
gjcolombo Jun 26, 2024
2b4767f
[WIP] hook up source side of migration
gjcolombo Jun 26, 2024
96e867c
[WIP] fix migration bugs
gjcolombo Jun 26, 2024
96d24f0
[WIP] todo cleanup
gjcolombo Jun 26, 2024
10a2179
[WIP] fix standalone build
gjcolombo Jun 26, 2024
855950e
fix pedantic softnpu clippy warning
gjcolombo Jun 26, 2024
df479df
[WIP] display external status while initializing VMs
gjcolombo Jun 26, 2024
0dc6512
[WIP] set up correct external queue dispositions on migration in
gjcolombo Jun 27, 2024
a25e38f
[WIP] start clean instances in Creating; fix Crucible block_on
gjcolombo Jun 27, 2024
28d3e6a
remove old files
gjcolombo Jun 27, 2024
72439b3
request queue tests
gjcolombo Jun 27, 2024
d404d3e
[WIP] flatten out VM object hierarchy
gjcolombo Jun 27, 2024
52a9a5e
[WIP] allow WaitingForInit -> RundownComplete
gjcolombo Jun 28, 2024
6a4567e
[WIP] move to destroyed state on rundown complete
gjcolombo Jun 28, 2024
57711e3
[WIP] send ensure result before bulk of migration protocol
gjcolombo Jun 28, 2024
2eef0b4
drive state machine properly after migration in fails
gjcolombo Jun 28, 2024
3a7b484
all the doc comments
gjcolombo Jun 28, 2024
d933bc9
notify Crucible reconfiguration waiters when they're preempted by ins…
gjcolombo Jun 28, 2024
82376e5
TaskGroup mutex can still be std
gjcolombo Jul 2, 2024
2e70f58
use async_trait to reduce some paperwork
gjcolombo Jul 2, 2024
f9efa92
LifecycleMap -> DeviceMap
gjcolombo Jul 2, 2024
3b4870a
clean up interface to VmObjects
gjcolombo Jul 2, 2024
4e80e26
clean up unused async
gjcolombo Jul 2, 2024
746b34f
Run live migration directly on the state driver task (#720)
gjcolombo Jul 12, 2024
e0ed049
reimplement tokio task limits
gjcolombo Jul 16, 2024
8606927
lifecycle_components -> devices again
gjcolombo Jul 16, 2024
7b87f5b
don't undercut the state driver when completing rundown
gjcolombo Jul 16, 2024
45e76d3
use shutdown_background instead of moving to an OS thread
gjcolombo Jul 16, 2024
75d6a95
rm unused file mistakenly added back in merge
gjcolombo Jul 16, 2024
09d942c
clean up a few fully-qualified paths
gjcolombo Jul 16, 2024
96a1263
fmt
gjcolombo Jul 16, 2024
fc8a45e
rm one more unused file
gjcolombo Jul 16, 2024
749ee17
remove extra fn call now that block_on isn't needed
gjcolombo Jul 17, 2024
c9b1bf6
PR feedback: style fixes
gjcolombo Jul 17, 2024
f168f30
remove block_in_place from input queue
gjcolombo Jul 17, 2024
c4c80f2
rename async block_until_joined to join_all
gjcolombo Jul 18, 2024
e110a65
explain more clearly why it's safe not to use enable()
gjcolombo Jul 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

16 changes: 9 additions & 7 deletions bin/propolis-server/src/lib/initializer.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@ use std::sync::Arc;
use std::time::{SystemTime, UNIX_EPOCH};

use crate::serial::Serial;
use crate::server::{BlockBackendMap, CrucibleBackendMap, DeviceMap};
use crate::stats::virtual_machine::VirtualMachine;
use crate::vm::{BlockBackendMap, CrucibleBackendMap, DeviceMap};
use anyhow::{Context, Result};
use crucible_client_types::VolumeConstructionRequest;
pub use nexus_client::Client as NexusClient;
Expand Down Expand Up @@ -185,7 +185,7 @@ impl<'a> MachineInitializer<'a> {

pub fn initialize_chipset(
&mut self,
event_handler: &Arc<dyn super::vm::ChipsetEventHandler>,
event_handler: &Arc<dyn super::vm::guest_event::ChipsetEventHandler>,
) -> Result<RegisteredChipset, Error> {
let mut pci_builder = pci::topology::Builder::new();
for (name, bridge) in &self.spec.devices.pci_pci_bridges {
Expand Down Expand Up @@ -371,7 +371,7 @@ impl<'a> MachineInitializer<'a> {
Ok(())
}

fn create_storage_backend_from_spec(
async fn create_storage_backend_from_spec(
&self,
backend_spec: &instance_spec::v0::StorageBackendV0,
backend_name: &str,
Expand Down Expand Up @@ -409,9 +409,10 @@ impl<'a> MachineInitializer<'a> {
self.log.new(
slog::o!("component" => format!("crucible-{cru_id}")),
),
)?;
)
.await?;

let crucible = Some((be.get_uuid()?, be.clone()));
let crucible = Some((be.get_uuid().await?, be.clone()));
Ok(StorageBackendInstance { be, crucible })
}
instance_spec::v0::StorageBackendV0::File(spec) => {
Expand Down Expand Up @@ -480,7 +481,7 @@ impl<'a> MachineInitializer<'a> {
///
/// On success, returns a map from Crucible backend IDs to Crucible
/// backends.
pub fn initialize_storage_devices(
pub async fn initialize_storage_devices(
&mut self,
chipset: &RegisteredChipset,
nexus_client: Option<NexusClient>,
Expand Down Expand Up @@ -537,7 +538,8 @@ impl<'a> MachineInitializer<'a> {
backend_spec,
backend_name,
&nexus_client,
)?;
)
.await?;

self.block_backends.insert(backend_name.clone(), backend.clone());
match device_interface {
Expand Down
Loading
Loading