Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Openjdk importer branch #1589

Closed

Conversation

harmonicfunc
Copy link

for issue: #1496
Added a OpenJDK advisory importer that scrapes the vulnerability data from the link: https://openjdk.org/groups/vulnerability/advisories/ and then added the importer to importer registery

@harmonicfunc
Copy link
Author

@TG1999 @ambuj-1211 @ziadhany please review this pr

Copy link
Member

@keshav-space keshav-space left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @harmonicfunc, use the VulnerableCodeBaseImporterPipeline for new importers. See the detailed instructions below.

from vulnerabilities.importer import VulnerabilitySeverity


class OpenJDKImporter(Importer):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@harmonicfunc We now have a new Importer design. We use aboutcode.pipeline for importers/Improver,

Move this file to vulnerabilities/pipelines directory and rename this file to openjdk_importer.py :

And start with something like this:

class OpenJDKImporterPipeline(VulnerableCodeBaseImporterPipeline):
    """Collect advisories from OpenJDK."""

    root_url = "https://openjdk.org/groups/vulnerability/advisories/"
    license_url = "https://openjdk.org/legal/"
    spdx_license_expression = "CC-BY-4.0"
    importer_name = "OpenJDK Importer"

    @classmethod
    def steps(cls):
        return (
            cls.fetch_advisory,
            cls.collect_and_store_advisories,
            cls.import_new_advisories,
        )

    def fetch_advisory(self):
        self.log(f"Fetching {self.root_url}")
        self.advisory_data = requests.get(self.root_url).text

    def advisories_count(self) -> int:
       # Use self.advisory_data to return the estimated AdvisoryData to be yielded by ``collect_advisories()``. 
        pass
    
    def collect_advisories(self) -> Iterable[AdvisoryData]:
        # Yield AdvisoryData by processing the self.advisory_data (this is similar to `advisory_data()` in old importer)
        # Use self.log() to log info/errors

See this pypa_importer.py pipeline for example https://github.com/aboutcode-org/vulnerablecode/blob/main/vulnerabilities/pipelines/pypa_importer.py.

Also we're in process of migrating our existing importers/imporvers to the new pipeline architecture see #1509

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@keshav-space this would be a nice addition/update to the doc ;)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@harmonicfunc
Copy link
Author

ok, on it

Signed-off-by: Alok Kumar Singh <[email protected]>
@harmonicfunc
Copy link
Author

@keshav-space sorry but do i need to revamp or change logic parts of my code too??

@pombredanne
Copy link
Member

@harmonicfunc the core logic does not need change, just the integration in a pipeline!
I see you may have some issues with merging ... this is easy at first! You likely want to rebase or restart on the latest main branch and push a new PR, which I would guess is why you closed this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants