containerbuilds failing in koji: Not Found ("lstat /usr/lib/sysimage/rpm: no such file or directory")

[juhp]

See https://pagure.io/releng/issue/11092 for more background.

As noted there, in Fedora 36+ /var/lib/rpm is now a symlink to /usr/lib/sysimage/rpm.

Whereas https://github.com/containerbuildsystem/atomic-reactor/blob/master/atomic_reactor/plugins/rpmqa.py#L20 still seem to hardcode RPMDB_PATH to /var/lib/rpm, which then causes the x86_64 container builds of fedora-toolbox to fail. (We are not sure yet why aarch64 is not affected.) It should be easy to test changing the path or rather adapting to the new location - though Kevin also wonders if not using rpmqa could be a temporary workaround?

[juhp]

This has been happening now for at least a couple of months and hence we don't have an up-to-date fedora-toolbox:37 image for GA yet unfortunately.

[MartinBasti]

This has been happening now for at least a couple of months

How we are supposed to know it, if nobody reported it till today? We don't maintain fedorainfra.

[MartinBasti]

Fedora chage https://fedoraproject.org/wiki/Changes/RelocateRPMToUsr

[MartinBasti]

Please let us know if you are going with workaround to disable rpmqa.

We will fix OSBS2 (or test if OSBS2 works fine with symlinks)

In general we prefer to fix only OSBS2 and migration to OSBS2

[juhp]

In general we prefer to fix only OSBS2 and migration to OSBS2

That was kind of my expectation too

How we are supposed to know it, if nobody reported it till today?

Well it is only last week that @cgwalters pointed out about rpmqa.py :)

[juhp]

Btw do you have any idea why this issue only affects x86_64 and not aarch64?

[MartinBasti]

I mentioned in that pagure issue, that it looks that you have disabled that plugin for aarch64, from logs it looks it didn't even run