Migrate Package managers

[keshav-space]

This PR migrates existing package managers code in VulnerableCode to FetchCode, also refactor and streamline the consumption using purl router.

[JonoYang]

@keshav-space I've left some suggestions to avoid accessing a dictionary multiple times for the same key-value pair.

[keshav-space]

Thanks @JonoYang, made changes as per your suggestions.

[pombredanne]

LGTM... we need to find a better name for this module, may be "package_versions.py" for now?

Beyond this we need to have a better design.
https://github.com/package-url/packageurl-python/blob/main/src/packageurl/contrib/purl2url.py#L70 design is not great but has its use.

Here are some thoughts (to track in new issue(s)):

1. We are going to have discrete functions each with a router that return a very specific URL or small piece of data like today. But here the functions that create URLs and fetch should be split in two so that we can have URL-only functions as explained in 2. that do not fetch anything.

2. Functions that only transform a PURL in a URL should be in one place (likely packageurl-python)

3. Then anything that fetches remote data should be of two kinds

• One may return raw data from a JSON or XML API
• One may return a ScanCode Package object converted from this raw data

4. Some basic function that only return versions may just return lists of PURLs alright

5. We need to account to for repository_url and download_url qualifiers

We also need to make the migration for VulnerableCode with this new code. Can you start a PR in parallel so we avoid duplicating code. In VCIO you could use a temp requirements in setup.cfg such as git+https://github.com/nexB/fetchcode@6faf26353b8cbc65b07ba6c3285a0e8c8a1c9f1b to collect a specific commit

[pombredanne]

Thanks. See fedback

[keshav-space]

LGTM... we need to find a better name for this module, may be "package_versions.py" for now?

package_managers.py is now renamed to package_versions.py.

Beyond this we need to have a better design. https://github.com/package-url/packageurl-python/blob/main/src/packageurl/contrib/purl2url.py#L70 design is not great but has its use.

Here are some thoughts (to track in new issue(s)):

1. We are going to have discrete functions each with a router that return a very specific URL or small piece of data like today. But here the functions that create URLs and fetch should be split in two so that we can have URL-only functions as explained in 2. that do not fetch anything.
2. Functions that only transform a PURL in a URL should be in one place (likely packageurl-python)
3. Then anything that fetches remote data should be of two kinds

• One may return raw data from a JSON or XML API
• One may return a ScanCode Package object converted from this raw data

4. Some basic function that only return versions may just return lists of PURLs alright
5. We need to account to for repository_url and download_url qualifiers

We're tracking this here: aboutcode-org/purldb#233

We also need to make the migration for VulnerableCode with this new code. Can you start a PR in parallel so we avoid duplicating code. In VCIO you could use a temp requirements in setup.cfg such as git+https://github.com/nexB/fetchcode@6faf26353b8cbc65b07ba6c3285a0e8c8a1c9f1b to collect a specific commit

Added PR for this here: aboutcode-org/vulnerablecode#1354

[JonoYang]

@keshav-space I left some comments regarding adding comments or docstring tests.

[JonoYang]

@keshav-space I think this looks good!