qed firmware missing from 40.20240519.3.0

[freedge]

Describe the bug

after zincati upgrade from 40.20240504.3.0 to 40.20240519.3.0 , my server does not have network anymore

logs

$ journalctl -b -1 -g qed
Jun 08 22:32:30 localhost kernel: QLogic FastLinQ 4xxxx Core Module qed
Jun 08 22:32:30 localhost kernel: qede init: QLogic FastLinQ 4xxxx Ethernet Driver qede
Jun 08 22:32:31 localhost kernel: qede 0000:5e:00.0: Direct firmware load for qed/qed_init_values_zipped-8.59.1.0.bin failed with error -2
Jun 08 22:32:31 localhost kernel: [qed_slowpath_start:1257()]Failed to find fw file - /lib/firmware/qed/qed_init_values_zipped-8.59.1.0.bin
Jun 08 22:32:31 localhost kernel: qede: probe of 0000:5e:00.0 failed with error -2
Jun 08 22:32:31 localhost kernel: qede 0000:5e:00.1: Direct firmware load for qed/qed_init_values_zipped-8.59.1.0.bin failed with error -2
Jun 08 22:32:31 localhost kernel: [qed_slowpath_start:1257()]Failed to find fw file - /lib/firmware/qed/qed_init_values_zipped-8.59.1.0.bin
Jun 08 22:32:31 localhost kernel: qede: probe of 0000:5e:00.1 failed with error -2

Reproduction steps

1. run a server with a QLogic nic (QLogic 10GE 2P QL41112HxCU-DE Adapter)
2. keep it up to date
3. try to access it through the network

Expected behavior

network up

Actual behavior

no network, Failed to find fw file - /lib/firmware/qed/qed_init_values_zipped-8.59.1.0.bin

System details

• bm

$ rpm-ostree status
State: idle
AutomaticUpdatesDriver: Zincati
  DriverState: active; periodically polling for updates (last checked Sun 2024-06-09 06:58:40 UTC)
Deployments:
  fedora:fedora/x86_64/coreos/stable
                  Version: 40.20240519.3.0 (2024-06-04T23:21:15Z)
                   Commit: 724ce262d4a27f6b7cb1508e8737e2244d69bb78509d2749cebd7972042bf814
             GPGSignature: Valid signature by 115DF9AEF857853EE8445D0A0727707EA15B79CC
                     Diff: 66 upgraded

● fedora:fedora/x86_64/coreos/stable
                  Version: 40.20240504.3.0 (2024-05-20T18:00:04Z)
                   Commit: 72cf2f80ba1496d478e110d03e1199d9d21382840e96ffeddf4303eb040fbb55
             GPGSignature: Valid signature by 115DF9AEF857853EE8445D0A0727707EA15B79CC

Butane or Ignition config

No response

Additional information

https://src.fedoraproject.org/rpms/linux-firmware/c/7418c6e7c3a4d8d976340837132488d43c3cdda6?branch=rawhide

the linux-firmware package has been split and the ramdisk needs to embed the qed-firmware package.
Reverting to previous version I read:

$ rpm -q --whatprovides /lib/firmware/qed/qed_init_values_zipped-8.59.1.0.bin.xz
linux-firmware-20240410-1.fc40.noarch

[freedge]

one workaround is to layer the qed-firmware package with sudo rpm-ostree install qed-firmware
then write a systemd service in this fashion

# https://unix.stackexchange.com/questions/229048/how-to-start-a-systemd-service-before-networking-starts
# https://github.com/coreos/fedora-coreos-tracker/issues/1746

[Unit]
Description=Reload qed firmware
DefaultDependencies=no

Before=network-pre.target
Wants=network-pre.target

Wants=systemd-modules-load.service local-fs.target
After=systemd-modules-load.service local-fs.target

Conflicts=shutdown.target
Before=shutdown.target

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStartPre=-/usr/sbin/modprobe -r qede
ExecStart=/usr/sbin/modprobe qede

[Install]
WantedBy=multi-user.target

[jlebon]

@freedge Sorry for missing this! We will add back the firmware for the time being.

One question: you mention the NIC no longer works without the firmware blob. Is there any e.g. "reduced capacity" mode which allows it to work, or is it entirely non-functional without the file?

[freedge]

haha you were working on another ticket I opened :D

without the qed-firmware the NIC is entirely non functional, "ip link" does not show the interface. In practice you need to access the system from the console and reboot on the previous ostree entry.

[marmijo]

The fix for this went into testing stream release 40.20240701.1.0. Please try out the new release and report issues.