Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reduce default SWIOTLB size when no PCI devs are used #514

Merged
merged 4 commits into from
Jan 19, 2023

Conversation

marmarek
Copy link
Member

Linux inside HVM will allocate 64MB for bouncing DMA (SWIOTLB) by default.
If no real PCI device is assigned, that's way too much, and wastes over 15%
of VM's initial memory. With real PCI devices, it's usually too much too,
but it's very device specific, so don't risk breaking it. In other cases,
reduce default to 4MB.

Note PVH domain will not allocate SWIOTLB anyway, as no PCI devices are
there at all. This difference contributes to the VM start time, so reducing
SWIOTLB should also improve that part.

QubesOS/qubes-issues#6174

Also, include few fixes from #506, to have them merged independently

Linux inside HVM will allocate 64MB for bouncing DMA (SWIOTLB) by
default. If no real PCI device is assigned, that's way too much, and
wastes over 15% of VM's initial memory.
With real PCI devices, it's usually too much too, but it's very device
specific, so don't risk breaking it. In other cases, reduce default to
4MB.

Note PVH domain will not allocate SWIOTLB anyway, as no PCI devices are
there at all. This difference contributes to the VM start time, so
reducing SWIOTLB should also improve that part.

QubesOS/qubes-issues#6174
After fixing cyclic import, qmemman is now imported in tests too and
expects "xid" property to be set on a running VM.
Set the permission only to the 'memory/meminfo' key (needs to be created
first), not the whole 'memory' dir. And do it only if memory balancing
is enabled.
This avoids VM potentially messing with other keys (like static-max), or
reporting meminfo when not expected to.
The old (multiline) meminfo format was used before R4.0. We can drop it
now.
@qubesos-bot
Copy link

OpenQA test summary

Complete test suite and dependencies: https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=4.2&build=2023011805-4.2&flavor=pull-requests

New failures, excluding unstable

Compared to: https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=4.2&build=2022121202-4.2&flavor=update

  • system_tests_basic_vm_qrexec_gui

  • system_tests_dispvm

    • TC_04_DispVM: test_003_cleanup_destroyed (error)
      raise exceptions.TimeoutError()... asyncio.exceptions.TimeoutError

    • TC_20_DispVM_whonix-ws-16: test_030_edit_file (failure + cleanup)
      AssertionError: Timeout while waiting for disp[0-9]* window to show

    • TC_20_DispVM_whonix-ws-16: test_100_open_in_dispvm (failure)
      AssertionError: libvirt event impl drain timeout

  • system_tests_whonix

    • whonix_torbrowser: unnamed test (unknown)

    • whonix_torbrowser: Failed (test died)
      # Test died: no candidate needle with tag(s) 'anon-whonix-tor-brows...

    • whonix_torbrowser: unnamed test (unknown)

  • system_tests_qwt_win7@hw1

    • windows_clipboard_and_filecopy: unnamed test (unknown)
    • windows_clipboard_and_filecopy: Failed (test died)
      # Test died: no candidate needle with tag(s) 'windows-Explorer, win...
  • system_tests_whonix@hw1

    • whonix_torbrowser: unnamed test (unknown)

    • whonix_torbrowser: Failed (test died)
      # Test died: no candidate needle with tag(s) 'anon-whonix-tor-brows...

    • whonix_torbrowser: unnamed test (unknown)

Failed tests

15 failures
  • system_tests_basic_vm_qrexec_gui

  • system_tests_splitgpg

  • system_tests_dispvm

    • TC_04_DispVM: test_003_cleanup_destroyed (error)
      raise exceptions.TimeoutError()... asyncio.exceptions.TimeoutError

    • TC_20_DispVM_fedora-36: test_100_open_in_dispvm (failure)
      self.assertEqual(test_txt_content.s... AssertionError: b'' != b'test1'

    • TC_20_DispVM_whonix-ws-16: test_030_edit_file (failure + cleanup)
      AssertionError: Timeout while waiting for disp[0-9]* window to show

    • TC_20_DispVM_whonix-ws-16: test_100_open_in_dispvm (failure)
      AssertionError: libvirt event impl drain timeout

  • system_tests_whonix

    • whonix_torbrowser: unnamed test (unknown)

    • whonix_torbrowser: Failed (test died)
      # Test died: no candidate needle with tag(s) 'anon-whonix-tor-brows...

    • whonix_torbrowser: unnamed test (unknown)

  • system_tests_qwt_win7@hw1

    • windows_clipboard_and_filecopy: unnamed test (unknown)
    • windows_clipboard_and_filecopy: Failed (test died)
      # Test died: no candidate needle with tag(s) 'windows-Explorer, win...
  • system_tests_whonix@hw1

    • whonix_torbrowser: unnamed test (unknown)

    • whonix_torbrowser: Failed (test died)
      # Test died: no candidate needle with tag(s) 'anon-whonix-tor-brows...

    • whonix_torbrowser: unnamed test (unknown)

Fixed failures

Compared to: https://openqa.qubes-os.org/tests/55523#dependencies

56 fixed
  • system_tests_basic_vm_qrexec_gui

  • system_tests_backupcompatibility

    • TC_00_BackupCompatibility: test_220_r2_encrypted (error + cleanup)
      lxml.etree.XMLSyntaxError: Document is empty, line 1, column 1

    • TC_01_BackupCompatibilityIntoLVM: test_220_r2_encrypted (error)
      lxml.etree.XMLSyntaxError: Document is empty, line 1, column 1

  • system_tests_splitgpg

  • system_tests_guivm_gui_interactive

    • update_templates: Failed (test died)
      # Test died: command 'qvm-template install --enablerepo=qubes-*temp...
  • system_tests_network_updates

    • TC_10_QvmTemplate_debian-11: test_000_template_list (failure)
      qvm-template: error: No matching templates to list

    • TC_10_QvmTemplate_debian-11: test_010_template_install (failure)
      qvm-template: error: Template 'debian-11-minimal' not found.

    • TC_10_QvmTemplate_fedora-36: test_000_template_list (failure)
      qvm-template: error: No matching templates to list

    • TC_10_QvmTemplate_fedora-36: test_010_template_install (failure)
      qvm-template: error: Template 'debian-11-minimal' not found.

    • TC_10_QvmTemplate_whonix-gw-16: test_000_template_list (failure)
      qvm-template: error: No matching templates to list

    • TC_10_QvmTemplate_whonix-gw-16: test_010_template_install (failure)
      qvm-template: error: Template 'debian-11-minimal' not found.

    • TC_11_QvmTemplateMgmtVM_debian-11: test_010_template_install (failure)
      AssertionError: qvm-template failed: Downloading 'qubes-template-de...

    • TC_11_QvmTemplateMgmtVM_fedora-36: test_010_template_install (failure)
      AssertionError: qvm-template failed: Downloading 'qubes-template-de...

    • TC_11_QvmTemplateMgmtVM_whonix-gw-16: test_010_template_install (failure)
      AssertionError: qvm-template failed: Downloading 'qubes-template-de...

  • system_tests_update@hw1

    • startup: unnamed test (unknown)
    • startup: Failed (test died)
      # Test died: no candidate needle with tag(s) 'luks-prompt, login-pr...
  • system_tests_basic_vm_qrexec_gui_btrfs

    • TC_00_AppVM_debian-11-pool: test_223_audio_play_hvm (error)
      libvirt.libvirtError: internal error: libxenlight failed to create ...

    • TC_00_AppVM_debian-11-pool: test_224_audio_rec_muted_hvm (error)
      libvirt.libvirtError: internal error: libxenlight failed to create ...

    • TC_00_AppVM_debian-11-pool: test_225_audio_rec_unmuted_hvm (error)
      libvirt.libvirtError: internal error: libxenlight failed to create ...

    • TC_00_AppVM_fedora-36-pool: test_223_audio_play_hvm (error)
      libvirt.libvirtError: internal error: libxenlight failed to create ...

    • TC_00_AppVM_fedora-36-pool: test_224_audio_rec_muted_hvm (error)
      libvirt.libvirtError: internal error: libxenlight failed to create ...

    • TC_00_AppVM_fedora-36-pool: test_225_audio_rec_unmuted_hvm (error)
      libvirt.libvirtError: internal error: libxenlight failed to create ...

    • TC_00_AppVM_whonix-ws-16-pool: test_223_audio_play_hvm (error)
      libvirt.libvirtError: internal error: libxenlight failed to create ...

    • TC_00_AppVM_whonix-ws-16-pool: test_224_audio_rec_muted_hvm (error)
      libvirt.libvirtError: internal error: libxenlight failed to create ...

    • TC_00_AppVM_whonix-ws-16-pool: test_225_audio_rec_unmuted_hvm (error)
      libvirt.libvirtError: internal error: libxenlight failed to create ...

  • system_tests_basic_vm_qrexec_gui_ext4

    • TC_00_AppVM_debian-11-pool: test_223_audio_play_hvm (error)
      libvirt.libvirtError: internal error: libxenlight failed to create ...

    • TC_00_AppVM_debian-11-pool: test_224_audio_rec_muted_hvm (error)
      libvirt.libvirtError: internal error: libxenlight failed to create ...

    • TC_00_AppVM_debian-11-pool: test_225_audio_rec_unmuted_hvm (error)
      libvirt.libvirtError: internal error: libxenlight failed to create ...

    • TC_00_AppVM_fedora-36-pool: test_223_audio_play_hvm (error)
      libvirt.libvirtError: internal error: libxenlight failed to create ...

    • TC_00_AppVM_fedora-36-pool: test_224_audio_rec_muted_hvm (error)
      libvirt.libvirtError: internal error: libxenlight failed to create ...

    • TC_00_AppVM_fedora-36-pool: test_225_audio_rec_unmuted_hvm (error)
      libvirt.libvirtError: internal error: libxenlight failed to create ...

    • TC_00_AppVM_whonix-ws-16-pool: test_223_audio_play_hvm (error)
      libvirt.libvirtError: internal error: libxenlight failed to create ...

    • TC_00_AppVM_whonix-ws-16-pool: test_224_audio_rec_muted_hvm (error)
      libvirt.libvirtError: internal error: libxenlight failed to create ...

    • TC_00_AppVM_whonix-ws-16-pool: test_225_audio_rec_unmuted_hvm (error)
      libvirt.libvirtError: internal error: libxenlight failed to create ...

  • system_tests_basic_vm_qrexec_gui_xfs

    • TC_00_AppVM_debian-11-pool: test_223_audio_play_hvm (error)
      libvirt.libvirtError: internal error: libxenlight failed to create ...

    • TC_00_AppVM_debian-11-pool: test_224_audio_rec_muted_hvm (error)
      libvirt.libvirtError: internal error: libxenlight failed to create ...

    • TC_00_AppVM_debian-11-pool: test_225_audio_rec_unmuted_hvm (error)
      libvirt.libvirtError: internal error: libxenlight failed to create ...

    • TC_00_AppVM_fedora-36-pool: test_223_audio_play_hvm (error)
      libvirt.libvirtError: internal error: libxenlight failed to create ...

    • TC_00_AppVM_fedora-36-pool: test_224_audio_rec_muted_hvm (error)
      libvirt.libvirtError: internal error: libxenlight failed to create ...

    • TC_00_AppVM_fedora-36-pool: test_225_audio_rec_unmuted_hvm (error)
      libvirt.libvirtError: internal error: libxenlight failed to create ...

    • TC_00_AppVM_whonix-ws-16-pool: test_223_audio_play_hvm (error)
      libvirt.libvirtError: internal error: libxenlight failed to create ...

    • TC_00_AppVM_whonix-ws-16-pool: test_224_audio_rec_muted_hvm (error)
      libvirt.libvirtError: internal error: libxenlight failed to create ...

    • TC_00_AppVM_whonix-ws-16-pool: test_225_audio_rec_unmuted_hvm (error)
      libvirt.libvirtError: internal error: libxenlight failed to create ...

  • system_tests_whonix

    • whonixcheck: fail (unknown)
      Whonixcheck for anon-whonix failed...

    • whonixcheck: unnamed test (unknown)

Unstable tests

  • system_tests_update

    update/Failed (1/4 times with errors)
    • job 55329 # Test died: command '(set -o pipefail; qubesctl --show-output stat...
  • system_tests_update@hw1

    update/Failed (1/4 times with errors)
    • job 55329 # Test died: command '(set -o pipefail; qubesctl --show-output stat...

@marmarek marmarek merged commit 6b4bd06 into QubesOS:main Jan 19, 2023
@marmarek marmarek deleted the smaller-swiotlb branch February 2, 2023 19:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants