Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement Qubes Management API methods #2622

Closed
56 of 62 tasks
woju opened this issue Feb 8, 2017 · 2 comments
Closed
56 of 62 tasks

Implement Qubes Management API methods #2622

woju opened this issue Feb 8, 2017 · 2 comments
Assignees
@woju
Labels
C: core C: mgmt
Milestone
Release 4.0 updates

Comments

@woju
Copy link
Member

woju commented Feb 8, 2017 •
edited by marmarek
Loading

  • mgmt.property.List
  • mgmt.property.Get
  • mgmt.property.Help
  • mgmt.property.Reset
  • mgmt.property.Set
  • mgmt.property.Set
  • mgmt.vmclass.List
  • mgmt.vm.List
  • mgmt.vm.Create.<class>
  • mgmt.vm.CreateInPool.<class>
  • mgmt.vm.property.List
  • mgmt.vm.property.Get
  • mgmt.vm.property.Help
  • mgmt.vm.property.Reset
  • mgmt.vm.feature.List
  • mgmt.vm.feature.Get
  • mgmt.vm.feature.CheckWithTemplate
  • mgmt.vm.feature.Remove
  • mgmt.vm.feature.Set
  • mgmt.vm.tag.List
  • mgmt.vm.tag.Get
  • mgmt.vm.tag.Remove
  • mgmt.vm.tag.Set
  • mgmt.vm.firewall.Get
  • mgmt.vm.firewall.Set
  • mgmt.vm.firewall.Reload
  • mgmt.vm.device.<class>.Attach
  • mgmt.vm.device.<class>.Detach
  • mgmt.vm.device.<class>.List
  • mgmt.vm.device.<class>.Available
  • mgmt.pool.ListDrivers
  • mgmt.pool.List
  • mgmt.pool.Info
  • mgmt.pool.Add
  • mgmt.pool.Remove
  • mgmt.pool.volume.List
  • mgmt.pool.volume.Info
  • mgmt.pool.volume.ListSnapshots
  • mgmt.pool.volume.Snapshot
  • mgmt.pool.volume.Revert
  • mgmt.pool.volume.Resize
  • mgmt.vm.volume.List
  • mgmt.vm.volume.Info
  • mgmt.vm.volume.ListSnapshots
  • mgmt.vm.volume.Snapshot
  • mgmt.vm.volume.Revert
  • mgmt.vm.volume.Resize
  • mgmt.vm.volume.Import
  • mgmt.vm.volume.CloneFrom
  • mgmt.vm.volume.CloneTo
  • mgmt.vm.Start
  • mgmt.vm.Shutdown
  • mgmt.vm.Pause
  • mgmt.vm.Unpause
  • mgmt.vm.Kill
  • mgmt.label.List
  • mgmt.label.Get
  • mgmt.label.Create
  • mgmt.label.Remove
  • mgmt.backup.Execute
  • mgmt.backup.Info
  • mgmt.Events
@woju woju added this to the Release 4.0 milestone Feb 8, 2017
@woju woju self-assigned this Feb 8, 2017
@woju
Copy link
Member Author

woju commented Feb 8, 2017

For documentation see https://www.qubes-os.org/doc/mgmt1/.

Cc: @marmarek

@woju woju added the C: mgmt label Feb 8, 2017
woju added a commit to woju/qubes-core-admin that referenced this issue Feb 10, 2017
@woju
qubes/mgmt: move mgmt api to separate module
c12fc74 
QubesOS/qubes-issues#2622
woju added a commit to woju/qubes-core-admin that referenced this issue Feb 10, 2017
@woju
qubes/mgmt: mgmt.vm.property.List
02639b8 
QubesOS/qubes-issues#2622
woju added a commit to woju/qubes-core-admin that referenced this issue Feb 10, 2017
@woju
qubesd: improve exception handling
323532f 
QubesOS/qubes-issues#2622
woju added a commit to woju/qubes-core-admin that referenced this issue Feb 10, 2017
@woju
qubesd: improve exception handling
e8a5bc9 
QubesOS/qubes-issues#2622
woju added a commit to woju/qubes-core-admin that referenced this issue Feb 15, 2017
@woju
qubes/mgmt: mgmt.vm.property.Help
ada0437 
QubesOS/qubes-issues#2622
woju added a commit to woju/qubes-core-admin that referenced this issue Feb 15, 2017
@woju
qubes/mgmt: mgmt.vm.property.Reset
8e678c0 
QubesOS/qubes-issues#2622
woju added a commit to woju/qubes-core-admin that referenced this issue Feb 21, 2017
@woju
qubes/tools/qubesd: add response header
fbd5de6 
QubesOS/qubes-issues#2622
woju added a commit to woju/qubes-core-admin that referenced this issue Feb 21, 2017
@woju
qubes/events: they accept only keyword arguments
ec803f7 
Positional arguments are hereby deprecated, with immediate effect.

QubesOS/qubes-issues#2622
woju added a commit to woju/qubes-core-admin that referenced this issue Feb 21, 2017
@woju
qubes/tools/qubesd: add response header
48f10a7 
QubesOS/qubes-issues#2622
woju added a commit to woju/qubes-core-admin that referenced this issue Feb 21, 2017
@woju
qubes/events: they accept only keyword arguments
be53db4 
Positional arguments are hereby deprecated, with immediate effect.

QubesOS/qubes-issues#2622
woju added a commit to woju/qubes-core-admin that referenced this issue Mar 1, 2017
@woju
qubesd+mgmt: convert mgmt functions to coroutines
f37895e 
QubesOS/qubes-issues#2622
woju added a commit to woju/qubes-core-admin that referenced this issue Mar 1, 2017
@woju
qubesd+mgmt: convert mgmt functions to coroutines
865ab10 
QubesOS/qubes-issues#2622
marmarek added a commit to marmarek/qubes-core-admin that referenced this issue Mar 9, 2017
@marmarek
mgmt: implement storage-related methods
5ce5e5f 
QubesOS/qubes-issues#2622
marmarek added a commit to marmarek/qubes-core-admin that referenced this issue Mar 15, 2017
@marmarek
mgmt: encode property type in property.Get
67eedef 
This also require having property.type public.

QubesOS/qubes-issues#2622
marmarek added a commit to marmarek/qubes-core-admin that referenced this issue Mar 15, 2017
@marmarek
mgmt: implement mgmt.vm.property.Set
8f748b8 
Sanitization of input value is tricky here, and also very important at
the same time. If property define value type (and it's something more
specific than 'str'), use that. Otherwise allow only printable ASCII
characters, and let appropriate event and setter handle value.
At this point I've reviewed all QubesVM properties in this category and
added appropriate setters where needed.

QubesOS/qubes-issues#2622
marmarek added a commit to marmarek/qubes-core-admin that referenced this issue Mar 15, 2017
@marmarek
mgmt: add label-related calls
1180c11 
QubesOS/qubes-issues#2622
marmarek added a commit to marmarek/qubes-core-admin that referenced this issue Mar 16, 2017
@marmarek
mgmt: implement storage-related methods
fb7bd68 
QubesOS/qubes-issues#2622
marmarek added a commit to marmarek/qubes-core-admin that referenced this issue Mar 16, 2017
@marmarek
mgmt: encode property type in property.Get
dbf2066 
This also require having property.type public.

QubesOS/qubes-issues#2622
marmarek added a commit to marmarek/qubes-core-admin that referenced this issue Mar 16, 2017
@marmarek
mgmt: implement mgmt.vm.property.Set
868dbea 
Sanitization of input value is tricky here, and also very important at
the same time. If property define value type (and it's something more
specific than 'str'), use that. Otherwise allow only printable ASCII
characters, and let appropriate event and setter handle value.
At this point I've reviewed all QubesVM properties in this category and
added appropriate setters where needed.

QubesOS/qubes-issues#2622
marmarek added a commit to marmarek/qubes-core-admin that referenced this issue Mar 16, 2017
@marmarek
mgmt: add label-related calls
010d40d 
QubesOS/qubes-issues#2622
marmarek added a commit to marmarek/qubes-core-admin that referenced this issue Mar 16, 2017
@marmarek
tests: storage and labels
8371ffc 
QubesOS/qubes-issues#2622
marmarek added a commit to marmarek/qubes-core-admin that referenced this issue Mar 16, 2017
@marmarek
mgmt: drop ProtocolRepr
f93674d 
Since we've added type= argument to property.Get format, it isn't
useful anymore.

QubesOS/qubes-issues#2622
marmarek added a commit to marmarek/qubes-core-admin that referenced this issue Mar 17, 2017
@marmarek
mgmt: allow functions to return None if no data is to be returned
2c115bb 
Don't force `return ''` everywhere.

QubesOS/qubes-issues#2622
marmarek added a commit to marmarek/qubes-core-admin that referenced this issue Mar 28, 2017
@marmarek
mgmt: minor adjustments
ad456a3 
per @woju review

QubesOS/qubes-issues#2622
marmarek added a commit to marmarek/qubes-core-admin that referenced this issue Jun 3, 2017
@marmarek
vm: change kernel=None to kernel=''
e4e1289 
vm.kernel property have type 'str'. Putting None there makes a lot of
troubles: it gets encoded as 'None' in qubes.xml and then loaded back as
'None' string, not None value. Also it isn't possible to assign None
value to str property throgh Admin API.

kernel='' is equally good to specify "no kernel from dom0".

QubesOS/qubes-issues#2622
marmarek added a commit to marmarek/qubes-core-admin that referenced this issue Jun 3, 2017
@marmarek
vm: change kernel=None to kernel=''
6cad3be 
vm.kernel property have type 'str'. Putting None there makes a lot of
troubles: it gets encoded as 'None' in qubes.xml and then loaded back as
'None' string, not None value. Also it isn't possible to assign None
value to str property throgh Admin API.

kernel='' is equally good to specify "no kernel from dom0".

QubesOS/qubes-issues#2622
marmarek added a commit to marmarek/qubes-core-admin that referenced this issue Jun 5, 2017
@marmarek
vm: change kernel=None to kernel=''
4c4f903 
vm.kernel property have type 'str'. Putting None there makes a lot of
troubles: it gets encoded as 'None' in qubes.xml and then loaded back as
'None' string, not None value. Also it isn't possible to assign None
value to str property throgh Admin API.

kernel='' is equally good to specify "no kernel from dom0".

QubesOS/qubes-issues#2622
marmarek added a commit to marmarek/qubes-core-admin-client that referenced this issue Jun 19, 2017
@marmarek
tags support
90a923c 
QubesOS/qubes-issues#2622
marmarek added a commit to marmarek/qubes-core-admin-client that referenced this issue Jun 19, 2017
@marmarek
storage: add volume clone method
3544c57 
Clone volume without retrieving all the data.

QubesOS/qubes-issues#2622
marmarek added a commit to marmarek/qubes-core-admin-client that referenced this issue Jun 19, 2017
@marmarek
Implement VM clone as create + copy data+metadata
91b7e24 
This way we don't need separate admin.vm.Clone call, which is tricky to
handler properly with policy.
A VM may not have access to all the properties and other metadata, so
add ignore_errors argument, for best-effort approach (copy what is
possible). In any case, failure of cloning VM data fails the whole
operation.
When operation fails, VM is removed.

While at it, allow to specify alternative VM class - this allows
morphing one VM into another (for example AppVM -> StandaloneVM).

Adjust qvm-clone tool and tests accordingly.

QubesOS/qubes-issues#2622
marmarek added a commit to marmarek/qubes-core-admin that referenced this issue Jun 19, 2017
@marmarek
admin: add admin.vm.volume.Clone
aadbe22 
QubesOS/qubes-issues#2622
marmarek added a commit to marmarek/qubes-core-admin that referenced this issue Jun 19, 2017
@marmarek
admin: implement admin.vm.tag.*
9242202 
QubesOS/qubes-issues#2622
marmarek added a commit to marmarek/qubes-core-admin-client that referenced this issue Jun 19, 2017
@marmarek
storage: add volume clone method
1dd49a7 
Clone volume without retrieving all the data.

QubesOS/qubes-issues#2622
marmarek added a commit to marmarek/qubes-core-admin-client that referenced this issue Jun 19, 2017
@marmarek
Implement VM clone as create + copy data+metadata
7cd233f 
This way we don't need separate admin.vm.Clone call, which is tricky to
handler properly with policy.
A VM may not have access to all the properties and other metadata, so
add ignore_errors argument, for best-effort approach (copy what is
possible). In any case, failure of cloning VM data fails the whole
operation.
When operation fails, VM is removed.

While at it, allow to specify alternative VM class - this allows
morphing one VM into another (for example AppVM -> StandaloneVM).

Adjust qvm-clone tool and tests accordingly.

QubesOS/qubes-issues#2622
marmarek added a commit to marmarek/qubes-core-admin-client that referenced this issue Jun 19, 2017
@marmarek
storage: add volume clone method
ab4be5f 
Clone volume without retrieving all the data.

QubesOS/qubes-issues#2622
marmarek added a commit to marmarek/qubes-core-admin-client that referenced this issue Jun 19, 2017
@marmarek
Implement VM clone as create + copy data+metadata
bfa8d0a 
This way we don't need separate admin.vm.Clone call, which is tricky to
handler properly with policy.
A VM may not have access to all the properties and other metadata, so
add ignore_errors argument, for best-effort approach (copy what is
possible). In any case, failure of cloning VM data fails the whole
operation.
When operation fails, VM is removed.

While at it, allow to specify alternative VM class - this allows
morphing one VM into another (for example AppVM -> StandaloneVM).

Adjust qvm-clone tool and tests accordingly.

QubesOS/qubes-issues#2622
marmarek added a commit to marmarek/qubes-core-admin-client that referenced this issue Jun 19, 2017
@marmarek
tags support
31988a9 
QubesOS/qubes-issues#2622
marmarek added a commit to marmarek/qubes-core-admin-client that referenced this issue Jun 19, 2017
@marmarek
storage: add volume clone method
998a427 
Clone volume without retrieving all the data.

QubesOS/qubes-issues#2622
marmarek added a commit to marmarek/qubes-core-admin-client that referenced this issue Jun 19, 2017
@marmarek
Implement VM clone as create + copy data+metadata
bcd026d 
This way we don't need separate admin.vm.Clone call, which is tricky to
handler properly with policy.
A VM may not have access to all the properties and other metadata, so
add ignore_errors argument, for best-effort approach (copy what is
possible). In any case, failure of cloning VM data fails the whole
operation.
When operation fails, VM is removed.

While at it, allow to specify alternative VM class - this allows
morphing one VM into another (for example AppVM -> StandaloneVM).

Adjust qvm-clone tool and tests accordingly.

QubesOS/qubes-issues#2622
marmarek added a commit to marmarek/qubes-core-admin that referenced this issue Jun 26, 2017
@marmarek
api/admin: skip firewall in vm.Clone
65d15e6 
This operation is going to be removed, so apply a quick fix for tests.

QubesOS/qubes-issues#2622
marmarek added a commit to marmarek/qubes-core-admin that referenced this issue Jun 26, 2017
@marmarek
qubes: validate if property value consists of ASCII only earlier
9198416 
Do this for all standard property types - even if other types do
additional validation, do not expose them to non-ASCII characters.

QubesOS/qubes-issues#2622
marmarek added a commit to marmarek/qubes-core-admin that referenced this issue Jun 26, 2017
@marmarek
api/admin: firewall-related methods
0200fda 
In the end firewall is implemented as .Get and .Set rules, with policy
statically set to 'drop'. This way allow atomic firewall updates.

Since we already have appropriate firewall format handling in
qubes.firewall module - reuse it from there, but adjust the code to be
prepared for potentially malicious input. And also mark such variables
with untrusted_ prefix.

There is also third method: .Reload - which cause firewall reload
without making any change.

QubesOS/qubes-issues#2622
Fixes QubesOS/qubes-issues#2869
marmarek added a commit to marmarek/qubes-core-admin that referenced this issue Jun 26, 2017
@marmarek
api/admin: remove admin.vm.Clone operation
3dcd29a 
The same can be achieved with Create+volume.Clone

QubesOS/qubes-issues#2622
marmarek added a commit to marmarek/qubes-core-admin that referenced this issue Jun 26, 2017
@marmarek
api/admin: split vm.volume.Clone to CloneFrom and CloneTo
ee29e24 
The first operation returns a token, which can be passed to the second
one to actually perform clone operation. This way the caller needs have
power over both source and destination VMs (or at least appropriate
volumes), so it's easier to enforce appropriate qrexec policy.

The pending tokens are stored on Qubes() instance (as QubesAdminAPI is
not persistent). It is design choice to keep them in RAM only - those
are one time use and this way restarting qubesd is a simple way to
invalidate all of them. Otherwise we'd need some additional calls like
CloneCancel or such.

QubesOS/qubes-issues#2622
marmarek added a commit to marmarek/qubes-core-admin that referenced this issue Jun 26, 2017
@marmarek
api/admin: split vm.volume.Clone to CloneFrom and CloneTo
26a9974 
The first operation returns a token, which can be passed to the second
one to actually perform clone operation. This way the caller needs have
power over both source and destination VMs (or at least appropriate
volumes), so it's easier to enforce appropriate qrexec policy.

The pending tokens are stored on Qubes() instance (as QubesAdminAPI is
not persistent). It is design choice to keep them in RAM only - those
are one time use and this way restarting qubesd is a simple way to
invalidate all of them. Otherwise we'd need some additional calls like
CloneCancel or such.

QubesOS/qubes-issues#2622
This was referenced Jul 4, 2017
Closed
Closed
@marmarek
Copy link
Member

marmarek commented Jul 7, 2017

I'm going to ignore admin.pool.volume.* for Qubes 4.0, since we don't need them now (there are admin.vm.volume.* equivalent). But keep it in specification and implement when we will have use case for them.

So, for now, only backup-related calls are missing.

@marmarek marmarek closed this as completed Oct 4, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@woju woju

Labels
C: core C: mgmt
Projects
None yet
Milestone
Release 4.0 updates
Development

No branches or pull requests
3 participants
@marmarek @woju @andrewdavidwong