Integrate with the SBOM Ecosystem

[captn3m0]

Looking at the purl-spec, and think it might solve some of the challenges I've had with figuring out ways to join the software-supply-chain-ecosystem.

I was looking at how other projects in the ecosystem track EoL dates, and it's not very good:

• Trivy maintains the EoL dates in code: https://github.com/aquasecurity/trivy/blob/46051d5ec8c8c60425fa52c343e1df7151f0c3c2/pkg/detector/ospkg/alpine/alpine.go#L20-L45, https://github.com/aquasecurity/trivy/blob/46051d5ec8c8c60425fa52c343e1df7151f0c3c2/pkg/detector/ospkg/suse/suse.go#L21-L57
• Anchore/grype doesn't support EoL dates: Improve alpine matching for EOL distro versions anchore/grype#210

While our API is good for custom usecases where you define your own rules and build your own tooling, it doesn't work well with existing tools. @hugovk wrote a cli for the API, but it also suffers from the same problem: it exists for the EoL project, not for the people who're writing scanning tools.

I think I have one possible starting point: PURL. PURL is a specification to define a URL that links to a package (loosely defined) with an optional version. Here's some examples (for products we support):

• pkg:github/electron/electron
• pkg:github/electron/[email protected]
• pkg:maven/org.apache.logging.log4j/log4j-core
• pkg:maven/org.apache.logging.log4j/[email protected]
• pkg:deb/debian/[email protected]
• pkg:deb/ubuntu/[email protected]

Note that since the same package can be installed in multiple ways, there can be multiple URLs to the same "package". The spec is fairly readable.

Suggested Changes

1. Add a new purl key which lists out possible package names (without version numbers).
2. Generate a list of supported_versions in the API response which lists down complete PURLs for all possible combinations.

Not sure how to best expose this in the website.

Problems with this approach

• Not everything we track maps cleanly to (one or more) PURLs. Devices, Operating Systems, Client Side Applications (especially closed source ones), Managed Services (EKS,GKE) don't fit any of the available types.

• Most SBOM tooling looks for CVEs, so it's based on "no match=good". In our case, since we only provide "supported releases", these might need additional work to support the "match=good" usecase. We might have to generate "version ranges" as unsupported.

• Our version numbers might not match the version numbers actually reported by tools. As an example, apache-log4j package on debian is at version 2.17.1-1.

[captn3m0]

We could use CPEs for some of the hardware stuff.

[captn3m0]

So no new CPEs have been assigned for iPhones since the iPhone 5.

https://nvd.nist.gov/products/cpe/search/results?namingFormat=2.3&orderBy=CPEURI&keyword=cpe%3A2.3%3Ah%3Aapple%3A&status=FINAL%2CDEPRECATED&startIndex=60

Dropped a mail to [email protected] requesting the same.

[laeubi]

This PURL stuff looks very similar to what https://clearlydefined.io/ offers for licenses, mabe it makes semse to adopt those as well? as you can attach additional data to clearly-defined items an EOL might also be an option.

[captn3m0]

That does indeed make sense - will make sense to add a harvester for eol.date into clearlydefined project (with PURL actually being the linkage between the two as a bridge between ecosystems) - our data currently doesn't point out for eg that https://endoflife.date/api/laravel.json is actually a packagist package, and https://endoflife.date/api/rails.json is a ruby package.

With a PURL link in place, the above API URLs will link to canonical PURL representations of these packages, and that can be used to update the correct data in ClearlyDefined.

[witchcraze]

Please let me share my experience. Maybe something will help you.

In my organization, engineers install various of software with various method.
Someone will use OS vendor's package. Someone will use 3rd party package. Someone will use source code from upstream.
As you know, those software have different EOL (support period) and different version naming convention.
What is backporting, and how does it apply to RHEL and other Red Hat products?

Current my approach to apply EOL for those software dataset is ...

1. Get information from internal tool (about tens of thousands hosts)
   1. OS version
   2. Software version
   3. Package information
   4. License information
2. Classify package type
   1. OS vendor's package with OS EOL
   2. OS vendor's package with own EOL
      1. RHEL/RHEL Rebuild software collection
      2. RHEL/RHEL Rebuild application stream
      3. Supported packages by RHEL ELS
      4. Supported packages by Ubuntu ESM
      5. etcetc
   3. Non OS vendor's package
3. Formatting to match several datasets
4. Join EOL dataset

I think this site will be good source dataset for thos part for Non OS vendor's package.
If this site support OS vendor's package, I can use this site for all type.

And, current my approach to check vulnerability for those software dataset is ...

1. Get version information from internal tool
2. Classify package type
   1. OS vendor's package
   2. not OS vendor's package
3. Formatting to match several datasets
4. Check vulunerability
   1. Apply Vuls One Liner Scan for OS vendor's package (use OVAL dataset from OS vendor)
      One Liner Scan Mode(Scan with one-liner) · Vuls
   2. Apply Vuls CPE Scan for not OS vendor's package (use Vulnerability Database from NVD)
      CPE Scan · Vuls

To check vulnerability properly, Classify package type step is important.
Security members in our organization often point out vulunerability by mistake because their vulunerability check does not consider package type.
On the internet, there are many similar episodes by poor security tools.

Similarly I believe Classify package type step is also important to check EOL properly, .
Need to avoid pointing out EOL by mistake

[captn3m0]

Our current plan is for us to have a SBOM scanner that reports any EOL tooling no matter how it is installed (as long as it is tracked by your SBOM tool), irrespective of whether it is supported by your distro or not. If your distro decides to support a specific package really long-term, you can always mark it as a false-positive.

If this site support OS vendor's package, I can use this site for all type.

We do provide information about how long the OS itself is supported, but this ought to be treated with caution since many distros only provide this level of support for specific packages, or repositories (not covering community for eg). Others (such as Alpine) do not provide any security notices for unsupported releases. Even in cases where the package might be marked as supported, it can result in a false-sense of security, or in some cases weird security bugs due to backports.

As an example, Debian 10 (supported) is still shipping Nodejs 10 (went EOL April 2021) despite having known security concerns. The Jan-2022 Nodejs security fixes that notably include a certificate validation vulnerability are not present in the Debian release. As such, an end-user relying on just the distro EOL data here will be vulnerable despite thinking otherwise.

We've previously discussed tracking per-distro support for a given project (#365) and I believe current approach works for the best (track upstream).

[witchcraze]

Yes, I agreed your discussion (#365).
But most teams use OS package in our organization, and only applying upstream EOL with running software versions was not effective.

I felt this experience may be a littele helpful, but if you have no interest, please ignore my episode.

[captn3m0]

The current long-term plan is (going in reverse):

1. We want SBOM tooling that scans existing SBOMs.
2. Such tooling already exists, such as grype. Maybe we can provide a feed for Grype instead?
3. The feed for Grype uses package names, and version ranges. However, these can be generated from a known PURL (https://github.com/package-url/purl-spec/)
4. Our database doesn't have the relevant package names.
5. Lets have a mapping of each product to various packages it can be installed as.

The purl key solves for (5) here. We solve it in two manners:

1. Noting down any purls directly (via the purl.purl key)
2. Noting down linux distro package names (via the repology) key.

The plan is for (2) to be unrolled into (1) by using the repology API. So that leaves us with:

1. A list of valid PURLs for most products.
2. A list of all releases with release dates for many products (via release-data repo)
3. EOL dates for each of these PURL ranges.

As an example, for tomcat, we'll have:

1. A static PURL: pkg:maven/org.apache.tomcat/tomcat
2. A list of various package names for tomcat in various distros, via our repology key: https://repology.org/project/tomcat/versions.
3. List of all releases for tomcat: https://github.com/endoflife-date/release-data/blob/main/releases/tomcat.json
4. EOL dates for tomcat: https://endoflife.date/api/tomcat.json

With all of the above, we can generate a relevant pseudo-vulnerability report for tomcat:

Package	Version	Severity	Distro
tomcat	<8.0	HIGH	??
tomcat	<=8.0.53, >8.0	HIGH	??
tomcat	<9.0.69, >9.0	MEDIUM	??
tomcat	<10.0.27, >10.0	MEDIUM	??

There are two more problems:

1. How to decide severity? High/Medium/Low is quite vague, and it is probably a function of how far you've run away from the latest version. As an example, 9.0.1 released on 2017-09-27 v/s 9.0.63 released on 2022-05-11 should be treated differently when suggesting an upgrade to 9.0.69.
2. Some old distros might still be backporting security fixes. So Ubuntu Focal is at 9.0.31, but it is receiving fixes.

(1) can be solved by some customization - offer different feed variants. This might need some customization on the product level as well.
(2) can be solved partially, by using the repology data + our own EOL data to mark (supported distro + latest package) as green.

There's still a lot of open questions around the levers for both the questions - it often takes distros longer-than-expected to backport patches, and sometimes these are missed.

[marcwrobel]

From what I understand the final objective of this issue is to have an indicator on endoflife.date that tells if the version a user is currently using is "dangerous" with a High/Medium/Low severity. And this indicator could be exposed using the API or using a new CLI tool (that's the SBOM Ecosystem integration part). Did I understood correctly ?

If it is, I do not think Severity is the right term to use. When I think of severity, I immediately think of vulnerability. But from what I understand this will not be necessarily be the case? I think the term risk may be better. And in this case High/Medium/Low (and None?), despite being vague, gives a sense of the urgency to update.

[captn3m0]

We could use Risk, but existing tooling is hard wired for Severity, with a high/low/medium/info level already.

We can write our own tooling later, but for now we'll just need to document this the best we can.

[marcwrobel]

Yes, but did I understood correctly the final objective of this issue ? I am really not sure ;).

[noqcks]

Our current plan is for us to have a SBOM scanner that reports any EOL tooling no matter how it is installed (as long as it is tracked by your SBOM tool), irrespective of whether it is supported by your distro or not.

I've been working on some tooling for this. I ported the grype codebase and modified for this use case. https://github.com/noqcks/xeol (still have some more tests to add, but it should be useable).

I have a little lambda function running that pulls data from this repo and creates a sqlite database from static purls

You can see the DB structure by running

> xeol db update
> xeol db status 
# open the db file with sqlite3 at Location
> sqlite3 /Users/<user>/Library/Caches/xeol/db/1/xeol.db
# run sql commands to inspect the db

It only uses static purls listed in the endoflife.date repo, which I think is about 30 purls, so it probably won't match much if you try something other than mongo:x.x right now. My next step will be looking into ways I can help you all add more PURL usage across endoflife.date 😃

[captn3m0]

This is great work.

I have some WIP at endoflife-date/release-data#54 to integrate Repology Data to enhance the PURL coverage. Got slowed down due to some unexpected hurdles in what repology tracks.

Would be great if you could document the changes between grype->xeol. My original plan was to generate a feed with pseudo vulnerabilities (EOL-YYYY-XXXX) for a feed, but creating a new table might also work like you've done here.

It's still relying on Syft underneath, which isn't quite as comprehensive as I'd like (No PURL support for os for eg).

My next step will be looking into ways I can help you all add more PURL usage across endoflife.date smiley

Would appreciate help on this front. Feel free to file PRs, keeping repology identifiers in mind.

I'd also love to adopt xeol as our official scanner, but I'm wary of maintaining a fork, that might be too much work?

[adriens]

Exciting integrations with :

• xeol ("An end-of-life (EOL) package scanner for container images, systems, and SBOMs ")
• syft ("CLI tool and library for generating a Software Bill of Materials from container images and filesystems ")
• grype ("A vulnerability scanner for container images and filesystems")