Support for Rocky Linux

[DecayingSec]

Rocky Linux should be detected as RHEL/CentOS. See https://rockylinux.org/ for details. Similar to classic CentOS, Rocky Linux is a RHEL clone so Red Hat Security advisories apply to it. This is the same exact situation as AlmaLinux #1021

[apollo13]

Similar to classic CentOS, Rocky Linux is a RHEL clone so Red Hat Security advisories apply to it.

This is sadly no longer (easily) true for CentOS 8 and as such also not for Rocky Linux. This is due to how modules

As an example of running trivy against CentOS 8 yesterday:

+-----------------------------+------------------+          +-------------------------------------------+---------------------------------------------+-----------------------------------------+
| libslirp                    | CVE-2020-29652   |          | 4.3.1-1.module_el8.4.0+575+63b40ad7       | 4.3.1-1.module+el8.4.0+10607+f4da7515       | golang: crypto/ssh: crafted             |
|                             |                  |          |                                           |                                             | authentication request can              |
|                             |                  |          |                                           |                                             | lead to nil pointer dereference         |
|                             |                  |          |                                           |                                             | -->avd.aquasec.com/nvd/cve-2020-29652   |
+-----------------------------+                  +          +-------------------------------------------+---------------------------------------------+                                         +

You can see that the affected version is:

• 4.3.1-1.module_el8.4.0+575+63b40ad7
  and the fixed version is supposed to be:
• 4.3.1-1.module+el8.4.0+10607+f4da7515

Looking at the covering redhat advisory: https://access.redhat.com/errata/RHSA-2021:1796#packages we can see that redhat indeed fixed this in 4.3.1-1.module+el8.4.0+10607+f4da7515. Due to how the module system works in redhat/centos/forks the package names no longer have the exact same version. Basically everything after .module is information by the build-system -- but this information will not be the same over the distributions. 4.3.1-1.module_el8.4.0+575+63b40ad7 is actually the fixed version in centos since 4.3.1-1 is the only relevant part here.

[apollo13]

As another example, I just checked Rockylinux for libslirp (they seem to have rebuilt the package twice):

• 4.3.1-1.module+el8.4.0+537+38cf4e42
• 4.3.1-1.module+el8.4.0+538+900cb89c

[apollo13]

And just for fun, AlmaLinux:

• 4.3.1-1.module_el8.4.0+2331+dd38312c
• 4.3.1-1.module_el8.4.0+2332+f4da7515

That is now 4 distributions, all with different versions.

[DecayingSec]

Thank you for that information, I did't realize they did't synchronize version numbers.

Does that mean Trivy is currently broken for CentOS 8?

Does this mean Trivy support for will have to wait until Rockylinux and AlmaLinux both independently publish their own advisories with the correct revision numbers?

[apollo13]

Does that mean Trivy is currently broken for CentOS 8?

Define "broken". Trivy doesn't list CentOS 8 as supported https://aquasecurity.github.io/trivy/v0.17.2/vuln-detection/os/ -- so in that sense it is simply not supported and not technically broken.

Does this mean Trivy support for will have to wait until Rockylinux and AlmaLinux both independently publish their own advisories with the correct revision numbers?

No idea, if the version check were adjusted to strip everything after .module it should work. Because as far as RHEL is concerned very new version will increase the release number before .module. Everything after .module is just worthless chunk.

[DecayingSec]

Support for modular packages was added in #790.

[apollo13]

@knqyf263 Could you provide your insights here? Since CentOS is basically bug compatible with RHEL it could make sense to relax the module comparison, what do you think?

[carlwgeorge]

Because as far as RHEL is concerned very new version will increase the release number before .module. Everything after .module is just worthless chunk.

This isn't always true. For example, CVE-2020-25097 was fixed in RHSA-2021:1979 with squid-4.11-4.module+el8.4.0+10676+a969168e.2 (note the .2 after the modular information). This is because the modular build information is injected into the build via the %{dist} macro, and sometimes RHEL will append after the %{dist} instead of incrementing the number before the %{dist}. In this case that looked like Release: 4%{?dist}.2 in the spec file.

[apollo13]

@carlwgeorge Uff, that is unfortunate. So effectively RedHat did find a way to ensure that the security feed is only usable for RHEL no matter what :/ If that is really the case I think trivy should fully remove support for CentOS 8 and not pretend that it works by showing open issues…

[srbala]

Actually it make sence. CentOS, AlmaLiux and Rocky are RHEL family products. Any Security related issues related to sources are applicable all which might be 80%. Build, packageing and optimization might vary.

So the chalenge would be mater of findng the packaging pattern. May be related community members can help to improve the process

[carlwgeorge]

So effectively RedHat did find a way to ensure that the security feed is only usable for RHEL no matter what :/

There was no ill intent. Modularity was designed in Fedora, and that's where the decision to inject modular build information via the %{dist} macro happened. This is just an unfortunate side effect years later. If RHEL wanted to make life harder on rebuilds there are many easier ways for them to do it than inventing modularity.

The correct answer is for rebuilds to publish their own errata (like Alma does). Re-using RHEL errata as-is with RHEL rebuilds was always a hack that just happened to work, and it didn't even work consistently due to the rebuild gap. But if people insist on propping that hack up, it shouldn't be that difficult to write a regex to strip the modular info. Something like \.module[_+]el[0-9.]+\+[0-9]+\+[a-z0-9]+ would probably work.

[apollo13]

There was no ill intent.

Rereading my comment, I realize it sounded way harsher than intended. I am sorry if it created the impression that RHEL did this on purpose to cripple CentOS -- this was not my intention.

and it didn't even work consistently due to the rebuild gap.

Sure, but that was easy to understand and argue about :)

and sometimes RHEL will append after the %{dist} instead of incrementing the number before the %{dist}.

Do you happen to know the reason for this? The versioning guidelines seem to mention it as minor release bump (<minorbump>) which should be used when changes to an old branch need to be done without rebuilding the other branches ( https://docs.fedoraproject.org/en-US/packaging-guidelines/Versioning/#_you_need_to_change_an_old_branch_without_rebuilding_the_others ). Is this the case here?

What do you think would be the best resolution for distros like CentOS/Rocky/Alma as long as Trivy doesn't have it's own errata for them? Mark them as completely unsupported or relax the module comparison?

[carlwgeorge]

Yes, this is a case of a "minor release bump", exactly what you found documentation for. It may mean that the next RHEL branch (rhel-8.5.0 in this case) already has a build using the next whole release number, but it's necessary to apply a fix to the previous (rhel-8.4.0) branch.

The strictest interpretation of this would be that if a distro doesn't publish errata, it can't be supported. The CentOS project doesn't publish errata because even though it is built from RHEL sources, CentOS doesn't validate the fixes. I don't know if Alma validates fixes for it's errata, or if Rocky plans to validate fixes when they start publishing errata. Just saying "it's the same source so it must be the same" isn't always correct. Sometimes a release bump that fixes a CVE is a rebuild of a package after a dependent library was updated. Imagine a CVE fix in golang, where golang is updated in RHEL and podman must be rebuilt to get the fix. A distro built from RHEL sources could have an identical podman version and release, but if it was rebuilt in the wrong order it wouldn't resolve the CVE. Relaxing the module comparison puts modules on the same level as regular packages, but it doesn't resolve this fundamental flaw. Having the same NVR (name-version-release) only tells you that it was built from the same source, which is not a guarantee that it fixes the CVE in question.

[apollo13]

A distro built from RHEL sources could have an identical podman version and release, but if it was rebuilt in the wrong order it wouldn't resolve the CVE.

Oh -- my naive assumption was that in this case one would set a Requires to ensure that the fixes are part of the build. But if that isn't done I understand how one can end up in a world of pain with rebuild order.

[github-actions]

This issue is stale because it has been labeled with inactivity.

[srbala]

Link back for reference
aquasecurity/trivy-db#149
#1238

[knqyf263]

@carlwgeorge Thanks for the explanation! I'm so sorry for the too late reply. I was so busy on IaC scanning and other tasks.

The strictest interpretation of this would be that if a distro doesn't publish errata, it can't be supported.

Agree. Strictly speaking, we should not support CentOS due to missing errata. But there are many CentOS users and I guess they accept a certain amount of inconvenience because it is free. Even though it might be wrong in some cases, I think it is better to detect vulnerabilities relatively correctly rather than not supported at all. We probably should mention it in our documentation. We're ready to support CentOS errata once they publish it.

A distro built from RHEL sources could have an identical podman version and release, but if it was rebuilt in the wrong order it wouldn't resolve the CVE.

I completely understand what you mean, but as far as I know, security errata is usually published for the vulnerabilities affecting the product itself. I mean the errata of Podman doesn't have the vulnerabilities of Go. In the below example, it doesn't mention Podman.
https://access.redhat.com/security/cve/cve-2019-9741

In other words, when Podman built by newer Go in RHEL fixes the Go vulnerability and Podman built by older Go in CentOS doesn't fix the Go vulnerability, the transitive vulnerability cannot be detected. On the other hand, when podman-1.6.4-29.xxx in RHEL fixes CVE-2021-20188, we can assume podman-1.6.4-29.yyy in CentOS also fixes CVE-2021-20188, right? This is because the same NVR uses the same source of Podman. Am I missing something?

[carlwgeorge]

Again, matching the NVR only guarantees that it was built from the same source. It does not validate the build requirements it was built with, which breaks the assumption you're making when it comes to software that ships static binaries, such as golang or rust. If a projects like Alma and Rocky want to publish errata, they are taking responsibility for verifying that each errata fixes what they claim it does.

[knqyf263]

I mean even though Podman is built by vulnerable Go, those advisories are usually not provided anyway as far as I know. Please correct me if I'm wrong. The security advisories focus on vulnerabilities in the software itself, not build tools.

[apollo13]

@knqyf263 See https://access.redhat.com/errata/RHSA-2021:1796 as an example where (among others) Redhat explicitly rebuilt podman to fix a golang issue.

[knqyf263]

Oh, I didn't know Red Hat lists the CVE-ID of golang as well. Thanks! But I wonder if they will list all vulnerabilities of Podman dependencies...

[apollo13]

No idea. But the main take aways that I can see:

• Publishing errata without validating it like RHEL (ie simply copying from redhat) and applying it to other systems provides a false sense of security and should be avoided. I guess this applies to Rocky & Alma linux as well.

• Even if we were to use the "wrong" data, comparison for modules would need to get much more intelligent (ie strip out the automatically generated module parts -- can probably done with a regex under the assumption that the format doesn't change to often).

[GioMac]

Actually Rocky must be detected as "RHEL/Rocky".
But Centos is not the RHEL downstream anymore (well, definitely different after Y2022) and it should be detected as "Centos".

[knqyf263]

The outside contributor @MaineK00n added support for Rocky Linux 🎉 🥳

Note that their errata has some issues and Trivy cannot detect modular packages for now. See here for details.

[bendem]

I see the support was added in aquasecurity/trivy-db#154. Do we have to wait for a new trivy release before being able to use that new db version?

[knqyf263]

You need to wait for v0.23.0. It is supposed to be released on January 31st.