Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix data migration problems #818

Closed
pombredanne opened this issue Aug 2, 2022 · 5 comments
Closed

Fix data migration problems #818

pombredanne opened this issue Aug 2, 2022 · 5 comments
Milestone

Comments

@pombredanne
Copy link
Member

See #812
@TG1999 can you further elaborate on this?

@pombredanne pombredanne added this to the v30.0 milestone Aug 2, 2022
@TG1999
Copy link
Contributor

TG1999 commented Aug 5, 2022

Reference ID URL
cpe:2.3:a:ibm:tklm:1.0:::::::* https://nvd.nist.gov/vuln/search/results?adv_search=true&isCpeNameSearch=true&query=cpe:2.3:a:ibm:tklm:1.0:::::::*
-- --
cpe:2.3:a:ibm:tklm:1.0:::::::* -

Previous data migration fix up cpe urls, but rerun of the importers somehow recreated CPEs with or without the URL and we ended up with duplicate cpe entries as before #785

TG1999 added a commit to TG1999/vulnerablecode that referenced this issue Aug 5, 2022
TG1999 added a commit that referenced this issue Aug 5, 2022
Delete references to CPEs with empty URLs #818
@TG1999
Copy link
Contributor

TG1999 commented Aug 9, 2022

closed by #827

@TG1999 TG1999 closed this as completed Aug 9, 2022
@pombredanne
Copy link
Member Author

Based on a check the issue still exists:

{
    "reference_url": "",
    "reference_id": "cpe:2.3:a:lxml:lxml:*:*:*:*:*:*:*:*",
    "scores": [],
    "url": ""
},
{
    "reference_url": "https://nvd.nist.gov/vuln/search/results?adv_search=true&isCpeNameSearch=true&query=cpe:2.3:a:lxml:lxml:*:*:*:*:*:*:*:*",
    "reference_id": "cpe:2.3:a:lxml:lxml:*:*:*:*:*:*:*:*",
    "scores": [],
    "url": "https://nvd.nist.gov/vuln/search/results?adv_search=true&isCpeNameSearch=true&query=cpe:2.3:a:lxml:lxml:*:*:*:*:*:*:*:*"
},

@pombredanne pombredanne reopened this Sep 9, 2022
@pombredanne
Copy link
Member Author

I suggest two approaches:

  1. a code review to ensure this is fixed for good
  2. an improver to further fix this issue and similar issues

pombredanne added a commit that referenced this issue Sep 9, 2022
Also validate full_clean in the improve_runner to ensure we do not
have empty, invalid or blank URLs.

Refactor code to add new Manager to VulnerabilityReference and Package
Add convenience method accordingly to create Pckage from purls

Reference: #818
Co-authored-by: Tushar Goel <[email protected]>
Signed-off-by: Philippe Ombredanne <[email protected]>
@TG1999
Copy link
Contributor

TG1999 commented Sep 13, 2022

Fixed in 6d379d0

@TG1999 TG1999 closed this as completed Sep 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants