Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disallow disk snapshots for disks attached to stopped instances #1947

Closed
augustuswm opened this issue Feb 8, 2024 · 7 comments
Closed

Disallow disk snapshots for disks attached to stopped instances #1947

augustuswm opened this issue Feb 8, 2024 · 7 comments
Labels

Comments

@augustuswm
Copy link

Currently the web console allows for attempting to snapshot a disk that is attached to a stopped instance. This is currently not supported in nexus and either the instance needs to be running, or the disk needs to be detached for us to be able to snapshot it. We should not present the option to snapshot under these conditions.

@augustuswm augustuswm added the bug label Feb 8, 2024
@david-crespo
Copy link
Collaborator

david-crespo commented Feb 8, 2024

Found what looks like the relevant API-side check. It has been there since snapshots were introduced.

https://github.com/oxidecomputer/omicron/blob/a5430a6465f457175ca1cf0c0053ffc47f25bf62/sled-agent/src/instance.rs#L1183-L1203

Unfortunately the client-side check is written in a way that is hard to extend to include instance state. Not a big deal, but it won't be a one-line fix.

console/libs/api/util.ts

Lines 112 to 126 in cfda163

export const diskCan = mapValues(
{
// https://github.com/oxidecomputer/omicron/blob/4970c71e/nexus/db-queries/src/db/datastore/disk.rs#L578-L582.
delete: ['detached', 'creating', 'faulted'],
// TODO: link to API source
snapshot: ['attached', 'detached'],
},
(states: DiskState['state'][]) => {
// only have to Pick because we want this to work for both Disk and
// Json<Disk>, which we pass to it in the MSW handlers
const test = (d: Pick<Disk, 'state'>) => states.includes(d.state.state)
test.states = states
return test
}
)

@david-crespo
Copy link
Collaborator

Tested snapshotting a disk attached to a stopped instance on dogfood with #1948 deployed.

Screenshot 2024-02-20 at 2 56 54 PM

It's not great, but it would be better if the API error message was better.

{
  "request_id": "f14e26d3-f122-4f12-b284-daf99c2386c3",
  "error_code": "ServiceNotAvailable",
  "message": "Service Unavailable"
}

@david-crespo
Copy link
Collaborator

@augustuswm
Copy link
Author

augustuswm commented Feb 21, 2024

That is unfortunate. Maybe snapshot_create needs to check that invariant before issuing the saga given that we know it will fail. As it should really be using a 4xx status code like the other invariant failures as opposed to a 503, since no amount of calling the endpoint will succeed.

Eventually we will need to back it out once this functionality is working.

@david-crespo
Copy link
Collaborator

I think it might be a bug, as there is such a check before starting the saga. The logic might not be quite right.

oxidecomputer/omicron#5108

@augustuswm augustuswm changed the title Disallow disk snapshots under specific conditions Disallow disk snapshots for disks attached to stopped instances Feb 23, 2024
@askfongjojo
Copy link

We probably want to hold this ticket for a bit as oxidecomputer/omicron#3289 is being fixed by oxidecomputer/omicron#5221. Hopefully we no longer have to deal with the 503 error for much longer.

@askfongjojo
Copy link

I just confirmed on rack2 that we can now snapshot disks attached to stopped instance. The change requested here is no longer necessary.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants