Error when attempting to install Mirage template with disk size smaller than the default

[palainp]

How to file a helpful issue

The problem you're addressing (if any)

When trying to install in qubes 4.1 a TemplateVM created with qubes-builder-mirage (default TEMPLATE_ROOT_SIZE ?= 50MiB which suits well for unikernels) the root.img disk is smaller than default (10G) and installation fails with error "not enough data" and leads to an uninstalled template.

The installation script fails with the following logs and the template is not installed.

...
mirage-firewall: Importing data
Traceback (most recent call last):
  File "/bin/qvm-template-postprocess", line 5, in <module>
    sys.exit(main())
  File "/usr/lib/python3.8/site-packages/qubesadmin/tools/qvm_template_postprocess.py", line 431, in main
    loop.run_until_complete(post_install(args))
  File "/usr/lib64/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/lib/python3.8/site-packages/qubesadmin/tools/qvm_template_postprocess.py", line 297, in post_install
    import_root_img(vm, args.dir)
  File "/usr/lib/python3.8/site-packages/qubesadmin/tools/qvm_template_postprocess.py", line 107, in import_root_img
    vm.volumes['root'].import_data(stream=tar.stdout)
  File "/usr/lib/python3.8/site-packages/qubesadmin/storage.py", line 270, in import_data
    self._qubesd_call('Import', payload_stream=stream)
  File "/usr/lib/python3.8/site-packages/qubesadmin/storage.py", line 76, in _qubesd_call
    return self.app.qubesd_call(
  File "/usr/lib/python3.8/site-packages/qubesadmin/app.py", line 730, in qubesd_call
    return self._parse_qubesd_response(stdout)
  File "/usr/lib/python3.8/site-packages/qubesadmin/base.py", line 109, in _parse_qubesd_response
    raise exc_class(format_string, *args)
qubesadmin.exc.QubesException: Data import failed: not enough data (copied 52428800 bytes, expected 10737418240 bytes)
warning: %post(qubes-template-mirage-firewall-4.0.6-202204051348.noarch) scriptlet failed, exit status 1
...

The solution you'd like

I get over this error by modiying qvm_template_postprocess.py at lines https://github.com/QubesOS/qubes-core-admin-client/blob/400ea49e84bfba30f50b658ca0b23e4bb18c52f2/qubesadmin/tools/qvm_template_postprocess.py#L109 and https://github.com/QubesOS/qubes-core-admin-client/blob/400ea49e84bfba30f50b658ca0b23e4bb18c52f2/qubesadmin/tools/qvm_template_postprocess.py#L119:

-         vm.volumes['root'].import_data(stream=tar.stdout)
+         vm.volumes['root'].import_data_with_size(stream=tar.stdout, size=root_size)

Although it seems to be related on the surface, I cannot use the same solution for #7176, but both issues may have a common root?

Anyway an error message still remains:

mirage-firewall: Failed to resize root volume of mirage-firewall from 54525952 to 52428800 after import: For your own safety, shrinking of root is disabled (52428800 < 54525952). If you really know what you are doing, use `lvresize` on qubes_dom0/vm-mirage-firewall-root manually.

And my unikernels is still not booting, showing the following in the logs and then it hangs:

[2022-04-06 07:27:02] Logfile Opened
[2022-04-06 07:27:02] .[30m.[47mWelcome to GRUB!
[2022-04-06 07:27:02] 
[2022-04-06 07:27:02] .[37m.[40m.[37m.[40m.[37m.[40m.[3;35H      [ grub-xen.cfg  424B  100%  630.95B/s ].[3;1Herror: no such device: /boot/xen/pvboot-x86_64.elf.
[2022-04-06 07:27:02] Reading (xen/xvda/boot/grub/grub.cfg
[2022-04-06 07:27:02] .[H.[J.[1;1H.[1;41H      [ grub.cfg  81B  100%  85.44B/s ].[1;1H

The grub.cfg have the right size (81B): https://github.com/marmarek/qubes-builder-mirage/blob/master/template_scripts/grub.cfg

The value to a user, and who that user might be

It will be possible to install mirage unikernels templates into dom0.

[qubesos-bot]

Automated announcement from builder-github

The package core-admin-client has been pushed to the r4.1 testing repository for the CentOS centos-stream8 template.
To test this update, please install it with the following command:

sudo yum update --enablerepo=qubes-vm-r4.1-current-testing

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The component core-admin-client (including package python3-qubesadmin-4.1.22-1.fc32) has been pushed to the r4.1 testing repository for dom0.
To test this update, please install it with the following command:

sudo qubes-dom0-update --enablerepo=qubes-dom0-current-testing

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package qubes-core-admin-client_4.1.22-1 has been pushed to the r4.1 testing repository for the Debian template.
To test this update, first enable the testing repository in /etc/apt/sources.list.d/qubes-*.list by uncommenting the line containing buster-testing (or appropriate equivalent for your template version), then use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package core-admin-client has been pushed to the r4.1 stable repository for the CentOS centos-stream8 template.
To install this update, please use the standard update command:

sudo yum update

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The package qubes-core-admin-client_4.1.22-1+deb10u1 has been pushed to the r4.1 stable repository for the Debian template.
To install this update, please use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[qubesos-bot]

Automated announcement from builder-github

The component core-admin-client (including package python3-qubesadmin-4.1.22-1.fc32) has been pushed to the r4.1 stable repository for dom0.
To install this update, please use the standard update command:

sudo qubes-dom0-update

Or update dom0 via Qubes Manager.

Changes included in this update