Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[openSUSE] Add packaging for documentation in info format #65

Open
wants to merge 100 commits into
base: factory
Choose a base branch
from

Conversation

Thaodan
Copy link

@Thaodan Thaodan commented Apr 2, 2024

Add packaging for documentation in the info format.

The original source of the patch adding the info target is Guix and was proposed to upstream
in:
https://lists.gnu.org/archive/html/qemu-devel/2024-03/msg04257.html

@dfaggioli
Copy link
Member

Hello, and thanks for your contribution! Thanks also for providing the link to the upstream discussion.

It's an interesting topic and while I don't use info docs much myself, I see how it could be useful. My problem with this is that upstream does not seem keen on taking it for now, which means it might become a downstream only patch which we'll then have to carry ourselves. And however simple it looks, there's always the chance of clashes with new versions or fixes backported from upstream, which is something we really much would like to avoid.

I'll leave this open for now, (and I'll point it to my fellow QEMU package maintainers) to see what others think, but my inclination is to not deviate from upstream for something like this

@Thaodan
Copy link
Author

Thaodan commented Apr 16, 2024 via email

dfaggioli and others added 19 commits September 4, 2024 15:18
Stash the "packaging files" in the QEMU repository, in the rpm/
directory. During package build, they will be pulled out from there
and used as appropriate.

Signed-off-by: Dario Faggioli <[email protected]>
In an upstream tarball there are some special files, generated by a
script that is run when the archive is prepared. Let's make our
repository look a little more like that, so we can build it properly.

Signed-off-by: Dario Faggioli <[email protected]>
Create a rebuild (for pushes) and a pull request workflow.

Signed-off-by: Dario Faggioli <[email protected]>
The sgabios submodule is no longer there, so let's get rid of any
reference to it from our spec files.

Remove no longer supported './configure' options.

We're also not set yet for using the set_version service, so we need to
update the following manually:
- the Version: tags in the spec files
- the rpm/seabios_version and rpm/skiboot_version files (see qemu.spec
  for instructions on how to do that)
- the %{sbver} variable in rpm/common.inc

A better solution for handling this aspect is being worked on.

Signed-off-by: Dario Faggioli <[email protected]>
…ith date info (bsc#1011213)

Certain rom subpackages build from qemu git-submodules call the date
program to include date information in the packaged binaries. This
causes repeated builds of the package to be different, wkere the only
real difference is due to the fact that time build timestamp has
changed. To promote reproducible builds and avoid customers being
prompted to update packages needlessly, we'll use the timestamp of the
VERSION file as the packaging timestamp for all packages that build in a
timestamp for whatever reason.

References: bsc#1011213
Signed-off-by: Bruce Rogers <[email protected]>
…arch64

We add a --cross-file reference so that we can do cross compilation
of qboot from an aarch64 build.

Signed-off-by: Bruce Rogers <[email protected]>
Signed-off-by: Dario Faggioli <[email protected]>
Change QEMU_PATH from /usr/local/bin to /usr/bin prefix.

Signed-off-by: Andreas Färber <[email protected]>
Linux syscalls pass pointers or data length or other information of that sort
to the kernel. This is all stuff you don't want to have sign extended.
Otherwise a host 64bit variable parameter with a size parameter will extend
it to a negative number, breaking lseek for example.

Pass syscall arguments as ulong always.

Signed-off-by: Alexander Graf <[email protected]>
[JRZ: changes from linux-user/qemu.h wass moved to linux-user/user-internals.h]
Signed-off-by: Jose R Ziviani <[email protected]>
[DF: Forward port, i.e., use ulong for do_prctl too]
Signed-off-by: Dario Faggioli <[email protected]>
When doing lseek, SEEK_SET indicates that the offset is an unsigned variable.
Other seek types have parameters that can be negative.

When converting from 32bit to 64bit parameters, we need to take this into
account and enable SEEK_END and SEEK_CUR to be negative, while SEEK_SET stays
absolute positioned which we need to maintain as unsigned.

Signed-off-by: Alexander Graf <[email protected]>
When using hugetlbfs (which is required for HV mode KVM on 970), we
check for MMU notifiers that on 970 can not be implemented properly.

So disable the check for mmu notifiers on PowerPC guests, making
KVM guests work there, even if possibly racy in some odd circumstances.

Signed-off-by: Bruce Rogers <[email protected]>
Virtio-Console can only process one character at a time. Using it on S390
gave me strange "lags" where I got the character I pressed before when
pressing one. So I typed in "abc" and only received "a", then pressed "d"
but the guest received "b" and so on.

While the stdio driver calls a poll function that just processes on its
queue in case virtio-console can't take multiple characters at once, the
muxer does not have such callbacks, so it can't empty its queue.

To work around that limitation, I introduced a new timer that only gets
active when the guest can not receive any more characters. In that case
it polls again after a while to check if the guest is now receiving input.

This patch fixes input when using -nographic on s390 for me.

[AF: Rebased for v2.7.0-rc2]
[BR: minor edits to pass qemu's checkpatch script]
Signed-off-by: Bruce Rogers <[email protected]>
Change from using glib alloc and free routines to those
from libc. Also perform safety measure of dropping privs
to user if configured no-caps.

References: boo#988279
Signed-off-by: Bruce Rogers <[email protected]>
[AF: Rebased for v2.7.0-rc2]
Signed-off-by: Andreas Färber <[email protected]>
For SLES we want users to be able to use large memory configurations
with KVM without fiddling with ulimit -Sv.

Signed-off-by: Andreas Färber <[email protected]>
[BR: add include for sys/resource.h]
Signed-off-by: Bruce Rogers <[email protected]>
… to QEMU equiv (bsc#879425)

Add code to read the suse specific suse-diskcache-disable-flush flag out
of xenstore, and set the equivalent flag within QEMU.

Patch taken from Xen's patch queue, Olaf Hering being the original author.
[bsc#879425]

[BR: minor edits to pass qemu's checkpatch script]
[BR: With qdevification of xen-block, code has changed significantly]
Signed-off-by: Bruce Rogers <[email protected]>
Signed-off-by: Olaf Hering <[email protected]>
Provide monitor naming of xen disks, and plumb guest driver
notification through xenstore of resizing instigated via the
monitor.

[BR: minor edits to pass qemu's checkpatch script]
[BR: significant rework needed due to upstream xen disk qdevification]
[BR: At this point, monitor_add_blk call is all we need to add!]
Signed-off-by: Bruce Rogers <[email protected]>
…c#1079730, bsc#1101982, bsc#106399)

The final step of xl migrate|save for an HVM domU is saving the state of
qemu. This also involves releasing all block devices. While releasing
backends ought to be a separate step, such functionality is not
implemented.

Unfortunately, releasing the block devices depends on the optional
'live' option. This breaks offline migration with 'virsh migrate domU
dom0' because the sending side does not release the disks, as a result
the receiving side can not properly claim write access to the disks.

As a minimal fix, remove the dependency on the 'live' option. Upstream
may fix this in a different way, like removing the newly added 'live'
parameter entirely.

Fixes: 5d6c599 ("migration, xen: Fix block image lock issue on live migration")

Signed-off-by: Olaf Hering <[email protected]>
References: bsc#1079730, bsc#1101982, bsc#1063993
Signed-off-by: Bruce Rogers <[email protected]>
…EAD CAPACITY (SLE-20965)

While using SCSI passthrough, Following scenario makes qemu doesn't
realized the capacity change of remote scsi target:
1. online resize the scsi target.
2. issue 'rescan-scsi-bus.sh -s ...' in host.
3. issue 'rescan-scsi-bus.sh -s ...' in vm.

In above scenario I used to experienced errors while accessing the
additional disk space in vm. I think the reasonable operations should
be:
1. online resize the scsi target.
2. issue 'rescan-scsi-bus.sh -s ...' in host.
3. issue 'block_resize' via qmp to notify qemu.
4. issue 'rescan-scsi-bus.sh -s ...' in vm.

The errors disappear once I notify qemu by block_resize via qmp.

So this patch replaces the number of logical blocks of READ CAPACITY
response from scsi target by qemu's bs->total_sectors. If the user in
vm wants to access the additional disk space, The administrator of
host must notify qemu once resizeing the scsi target.

Bonus is that domblkinfo of libvirt can reflect the consistent capacity
information between host and vm in case of missing block_resize in qemu.
E.g:
...
    <disk type='block' device='lun'>
      <driver name='qemu' type='raw'/>
      <source dev='/dev/sdc' index='1'/>
      <backingStore/>
      <target dev='sda' bus='scsi'/>
      <alias name='scsi0-0-0-0'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
...

Before:
1. online resize the scsi target.
2. host:~  # rescan-scsi-bus.sh -s /dev/sdc
3. guest:~ # rescan-scsi-bus.sh -s /dev/sda
4  host:~  # virsh domblkinfo --domain $DOMAIN --human --device sda
Capacity:       4.000 GiB
Allocation:     0.000 B
Physical:       8.000 GiB

5. guest:~ # lsblk /dev/sda
NAME   MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda      8:0    0   8G  0 disk
└─sda1   8:1    0   2G  0 part

After:
1. online resize the scsi target.
2. host:~  # rescan-scsi-bus.sh -s /dev/sdc
3. guest:~ # rescan-scsi-bus.sh -s /dev/sda
4  host:~  # virsh domblkinfo --domain $DOMAIN --human --device sda
Capacity:       4.000 GiB
Allocation:     0.000 B
Physical:       8.000 GiB

5. guest:~ # lsblk /dev/sda
NAME   MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda      8:0    0   4G  0 disk
└─sda1   8:1    0   2G  0 part

References: [SUSE-JIRA] (SLE-20965)
Signed-off-by: Lin Ma <[email protected]>
…ancelled() (bsc#1180432, CVE-2020-35503)

Ensure that 'cmd->frame' is not NULL before accessing the 'header' field.
This check prevents a potential NULL pointer dereference issue.

RHBZ: https://bugzilla.redhat.com/show_bug.cgi?id=1910346
Signed-off-by: Mauro Matteo Cascella <[email protected]>
Reported-by: Cheolwoo Myung <[email protected]>
References: bsc#1180432, CVE-2020-35503
Signed-off-by: Dario Faggioli <[email protected]>
dfaggioli and others added 21 commits September 5, 2024 18:10
Update to latest upstream major release, 9.1.0:

  https://lore.kernel.org/qemu-devel/[email protected]/

Full changelog available here:

  https://wiki.qemu.org/ChangeLog/9.1

Some of the most notable features/fixes:
 * migration: compression offload support via Intel In-Memory Analytics
   Accelerator (IAA) or User Space Accelerator Development Kit (UADK),
   along with enhanced support for postcopy failure recovery
 * virtio: support for VIRTIO_F_NOTIFICATION_DATA, allowing guest
   drivers to provide additional data as part of sending device notifications
   for performance/debug purposes
 * guest-agent: support for guest-network-get-route command on linux,
   guest-ssh-* commands on Windows, and enhanced CLI support for
   configuring allowed/blocked commands
 * block: security fixes for QEMU NBD server and NBD TLS encryption
 * ARM: emulation support for FEAT_NMI, FEAT_CSV2_3, FEAT_ETS2,
   FEAT_Spec_FPACC, FEAT_WFxT, FEAT_Debugv8p8 architecture features
 * ARM: nested/two-stage page table support for emulated SMMUv3
 * ARM: xilinx_zynq board support for cache controller and multiple
   CPUs, and B-L475E-IOT01A board support for a DM163 display
 * LoongArch: support for directly booting an ELF kernel and for running
   up to 256 vCPUs via extioi virt extension
 * LoongArch: enhanced debug/GDB support
 * RISC-V: support for version 1.13 of privileged architecture specification
 * RISC-V: support for Zve32x, Zve64x, Zimop, Zcmop, Zama16b, Zabha,
   Zawrs, and Smcntrpmf extensions
 * RISC-V: enhanced debug/GDB support and general fixes
 * SPARC: emulation support for FMAF, IMA, VIS3, and VIS4 architecture
   features
 * x86: KVM support for running AMD SEV-SNP guests
 * x86: CPU emulation support for Icelake-Server-v7, SapphireRapids-v3,
   and SierraForest

Signed-off-by: Dario Faggioli <[email protected]>
The avx512f, live-block-migration and pvrdma options no longer exist
in upstream configure because those features were removed. Make the
corresponding changes in the spec files.

Signed-off-by: Fabiano Rosas <[email protected]>
Signed-off-by: Dario Faggioli <[email protected]>
Upstream commit 7c08eef (tests/data/acpi: Move x86 ACPI tables
under x86/${machine} path, 2024-06-25) has moved some files under
tests/data. Update the spec file to match.

Signed-off-by: Fabiano Rosas <[email protected]>
Signed-off-by: Dario Faggioli <[email protected]>
The nios2 emulation target has been removed upstream by commit
6c30148 (target/nios2: Remove the deprecated Nios II target,
2024-03-27).

Signed-off-by: Fabiano Rosas <[email protected]>
Signed-off-by: Dario Faggioli <[email protected]>
These files are not provided by the QEMU packages.

Signed-off-by: Fabiano Rosas <[email protected]>
Signed-off-by: Dario Faggioli <[email protected]>
There was a typo on the subshell invocation: s/%/$

Signed-off-by: Fabiano Rosas <[email protected]>
Signed-off-by: Dario Faggioli <[email protected]>
…in qapi.h (bsc#1211000)

The following patches will add co_wrapper annotations to functions
declared in qapi.h. Add that header to the set of files used by
block-coroutine-wrapper.py.

Link: https://lore.kernel.org/r/[email protected]
Reviewed-by: Hanna Czenczek <[email protected]>
References: bsc#1211000
Signed-off-by: Fabiano Rosas <[email protected]>
Signed-off-by: Dario Faggioli <[email protected]>
… mixed (bsc#1211000)

Some callers of this function are about to be converted to run in
coroutines, so allow it to be executed both inside and outside a
coroutine while we convert all the callers.

This will be reverted once all callers of bdrv_do_query_node_info run
in a coroutine.

Link: https://lore.kernel.org/r/[email protected]
Reviewed-by: Eric Blake <[email protected]>
Reviewed-by: Hanna Czenczek <[email protected]>
References: bsc#1211000
Signed-off-by: Fabiano Rosas <[email protected]>
Signed-off-by: Dario Faggioli <[email protected]>
…000)

This function has up until now always ran in the main loop, outside of
a coroutine. We're about to make it run inside a coroutine so start
actually taking the graph lock.

Link: https://lore.kernel.org/r/[email protected]
References: bsc#1211000
Signed-off-by: Fabiano Rosas <[email protected]>
Signed-off-by: Dario Faggioli <[email protected]>
…sc#1221812)

There is a small window at the end of block device migration when
devices are being re-activated. This includes a resetting of some
fields of BDRVQcow2State at qcow2_co_invalidate_cache(). A concurrent
QMP query-block command can call qcow2_get_specific_info() during this
window and see the cleared values, which leads to an assert:

  qcow2_get_specific_info: Assertion `false' failed

This is the same issue as Gitlab #1933, which has already been
resolved[1], but there the fix applied only to non-coroutine
commands. Once we move query-block to a coroutine the problem will
manifest again.

Add an operation blocker to the invalidation function to block the
query info path during this window.

Instead of failing query-block, which would be disruptive to users,
use the blocker to know when to reschedule the coroutine back into the
iohandler so it doesn't run while the BDRVQcow2State is inconsistent.

To avoid failing query-block when all block operations are blocked,
unblock the INFO operation at various places. This preserves the prior
situations where query-block used to work.

1 - https://gitlab.com/qemu-project/qemu/-/issues/1933

Link: https://lore.kernel.org/all/[email protected]/
Link: https://lore.kernel.org/r/[email protected]
References: bsc#1221812
Signed-off-by: Fabiano Rosas <[email protected]>
Signed-off-by: Dario Faggioli <[email protected]>
…1000)

Move this function into a coroutine so we can convert the whole
qmp_query_block command into a coroutine in the next patches.

Placing the entire command in a coroutine allow us to yield all the
way back to the main loop, releasing the BQL and unblocking the main
loop.

When the whole conversion is completed, we'll be able to avoid a
priority inversion that happens when a QMP command calls a slow
(buggy) system call and blocks the vcpu thread from doing mmio due to
contention on the BQL.

About coroutine safety:

Most callees have coroutine versions themselves and thus are safe to
call in a coroutine. The remaining ones:

- bdrv_refresh_filename, bdrv_get_full_backing_filename: String
  manipulation, nothing that would be unsafe for use in coroutines;

- bdrv_get_format_name: Just accesses a field;

- bdrv_get_specific_info, bdrv_query_snapshot_info_list: No locks or
  anything that would poll or block.

(using a mixed wrapper for now, but after all callers are converted,
this can become a coroutine exclusively)

Link: https://lore.kernel.org/r/[email protected]
References: bsc#1211000
Signed-off-by: Fabiano Rosas <[email protected]>
Signed-off-by: Dario Faggioli <[email protected]>
…sc#1211000)

We're converting callers of bdrv_co_get_allocated_file_size() to run
in coroutines because that function will be made asynchronous when
called (indirectly) from the QMP dispatcher.

This function is a candidate because it calls bdrv_do_query_node_info(),
which in turn calls bdrv_co_get_allocated_file_size().

All the functions called from bdrv_do_query_node_info() onwards are
coroutine-safe, either have a coroutine version themselves[1] or are
mostly simple code/string manipulation[2].

1) bdrv_co_getlength(), bdrv_co_get_allocated_file_size(),
   bdrv_co_get_info();

2) bdrv_refresh_filename(), bdrv_get_format_name(),
   bdrv_get_full_backing_filename(), bdrv_query_snapshot_info_list(),
   bdrv_get_specific_info();

Link: https://lore.kernel.org/r/[email protected]
Reviewed-by: Hanna Czenczek <[email protected]>
References: bsc#1211000
Signed-off-by: Fabiano Rosas <[email protected]>
Signed-off-by: Dario Faggioli <[email protected]>
…1000)

This function is a caller of bdrv_do_query_node_info(), which have
been converted to a coroutine. Convert this function as well so we're
closer from having the whole qmp_query_block as a single coroutine.

Also remove the wrapper for bdrv_co_do_query_node_info() now that all
its callers are converted.

Link: https://lore.kernel.org/r/[email protected]
References: bsc#1211000
Signed-off-by: Fabiano Rosas <[email protected]>
Signed-off-by: Dario Faggioli <[email protected]>
…#1211000)

We're converting callers of bdrv_co_get_allocated_file_size() to run
in coroutines because that function will be made asynchronous when
called (indirectly) from the QMP dispatcher.

This function is a candidate because it calls bdrv_query_image_info()
-> bdrv_co_do_query_node_info() -> bdrv_co_get_allocated_file_size().

It is safe to turn this is a coroutine because the code it calls is
made up of either simple accessors and string manipulation functions
[1] or it has already been determined to be safe [2].

1) bdrv_refresh_filename(), bdrv_is_read_only(),
   blk_enable_write_cache(), bdrv_cow_bs(), blk_get_public(),
   throttle_group_get_name(), bdrv_write_threshold_get(),
   bdrv_query_dirty_bitmaps(), throttle_group_get_config(),
   bdrv_filter_or_cow_bs(), bdrv_skip_implicit_filters()

2) bdrv_co_do_query_node_info() (see previous commits);

This was the only caller of bdrv_query_image_info(), so we can remove
the wrapper for that function now.

Link: https://lore.kernel.org/r/[email protected]
References: bsc#1211000
Signed-off-by: Fabiano Rosas <[email protected]>
Signed-off-by: Dario Faggioli <[email protected]>
…rt (bsc#1211000)

We're currently doing a full query-block just to enumerate the devices
for qmp_nbd_server_add and then discarding the BlockInfoList
afterwards. Alter hmp_nbd_server_start to instead iterate explicitly
over the block_backends list.

This allows the removal of the dependency on qmp_query_block from
hmp_nbd_server_start. This is desirable because we're about to move
qmp_query_block into a coroutine and don't need to change the NBD code
at the same time.

Add the GRAPH_RDLOCK_GUARD_MAINLOOP macro because
bdrv_skip_implicit_filters() needs the graph lock.

Link: https://lore.kernel.org/r/[email protected]
References: bsc#1211000
Signed-off-by: Fabiano Rosas <[email protected]>
Signed-off-by: Dario Faggioli <[email protected]>
…odes to coroutine (bsc#1211000)

Convert the remaining functions to make the QMP commands query-block
and query-named-block-nodes run in their entirety in a coroutine. With
this, any yield from those commands will return all the way back to
the main loop. This releases the BQL and the main loop and avoids
having the QMP command block another more important task from running.

Both commands need to be converted at once because hmp_info_block
calls both and it needs to be moved to a coroutine as well.

Now the wrapper for bdrv_co_get_allocated_file_size() can be made not
mixed and the wrapper for bdrv_co_block_device_info() can be removed.

Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Lin Ma <[email protected]>
References: bsc#1211000
Signed-off-by: Fabiano Rosas <[email protected]>
Signed-off-by: Dario Faggioli <[email protected]>
The fstat call can take a long time to finish when running over
NFS. Add a version of it that runs in the thread pool.

Adapt one of its users, raw_co_get_allocated_file size to use the new
version. That function is called via QMP under the qemu_global_mutex
so it has a large chance of blocking VCPU threads in case it takes too
long to finish.

Link: https://lore.kernel.org/r/[email protected]
Reviewed-by: Claudio Fontana <[email protected]>
Reviewed-by: Hanna Czenczek <[email protected]>
Signed-off-by: João Silva <[email protected]>
References: bsc#1211000
Signed-off-by: Fabiano Rosas <[email protected]>
Signed-off-by: Dario Faggioli <[email protected]>
Package qemu-vmsr-helper for letting VMs access the RAPL MSR.

I'll live in its own package and only makes sense on x86_64.

Signed-off-by: Dario Faggioli <[email protected]>
Upstream provides services for qemu-pr-helper. So far, we've not needed
them, so let's continue not to ship them for now.

However, in case at some point we want to start offering them, stash the
commented out runes for that in the spec file.

Signed-off-by: Dario Faggioli <[email protected]>
Convert conditional build of features to the %bcond_without, so they
can actually be disabled, e.g., at the project level.

Signed-off-by: Dario Faggioli <[email protected]>
… in a macro

When running configure, first of all we disable everything, and then we
enable only the feature that we know we want (and, of course, system
and user emulation use different sets of such features).

Consolidate the first part in a macro, that can be share between the two
spec files, making everything simpler and prettier.

Signed-off-by: Dario Faggioli <[email protected]>
dfaggioli and others added 3 commits September 11, 2024 18:47
Was disabled by mistake, when updating the QEMU version.
Reinstate it.

Signed-off-by: Dario Faggioli <[email protected]>
Take advantage of the Sphinx texinfo backend to generate a QEMU info
manual.  The Texinfo format allows for more structure and info readers
provide more advanced navigation capabilities compared to manpages
readers.

The original source of the patch is Guix and was proposed to upstream
in:
https://lists.gnu.org/archive/html/qemu-devel/2024-03/msg04257.html

* docs/meson.build (texi, info): New targets.

Signed-off-by: Maxim Cournoyer <[email protected]>
Signed-off-by: Björn Bidar <[email protected]>
@Thaodan Thaodan force-pushed the Thaodan/openSUSE/factory/info_package branch from 92f5efc to 271a3f2 Compare September 19, 2024 07:15
@dfaggioli dfaggioli force-pushed the factory branch 2 times, most recently from 036b767 to 080ae7a Compare October 15, 2024 16:53
@dfaggioli dfaggioli force-pushed the factory branch 2 times, most recently from 4e14b42 to b0b6a79 Compare October 29, 2024 10:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.