v30.0.0
Version v30.0.0
This is a major version that is not backward compatible.
-
We refactored the core processing with Importers that import data and Improvers that
transform imported data and convert that in Vulnerabilities and Packages. Improvers can
also improve and refine imported and existing data as well as enrich data using external
data sources. The migration to this new architecture is under way and not all importers
are available.Because of these extensive changes, it is not possible to migrate existing imported
data to the new schema. You will need instead to restart imports from an empty database
or access the new public.vulnerablecode.io live instance. We also provide a database dump. -
You can track the progress of this refactoring in this issue:
#597 -
We added new data sources including PYSEC, GitHub and GitLab.
-
We improved the documentation including adding development examples for importers and improvers.
-
We removed the ability to edit relationships from the UI. The UI is now read-only.
-
We replace the web UI with a brand new UI based on the same overall look and feel as ScanCode.io.
-
We added support for NixOS as a Linux deployment target.
-
The aliases of a vulnerabily are reported in the API vulnerabilities/ endpoint
-
There are breaking Changes at API level with changes in the data structure:
-
in the /api/vulnerabilities/ endpoint:
- Rename
resolved_packages
tofixed_packages
- Rename
unresolved_packages
toaffected_packages
- Rename
url
toreference_url
in the reference list - Add is_vulnerable property in fixed and affected_packages.
- Rename
-
in the /api/packages/ endpoint:
- Rename
unresolved_vulnerabilities
toaffected_by_vulnerabilities
- Rename
resolved_vulnerabilities
tofixing_vulnerabilities
- Rename
url
toreference_url
in the reference list - Add new attribute
is_resolved
- Add namespace filter
- Rename
-
-
We have provided backward compatibility for
url
andunresolved_vulnerabilities
for now.
These will be removed in the next major version and should be considered as deprecated. -
There is a new experimental
cpe/
API endpoint to lookup for vulnerabilities by CPE and
another aliases/ endpoint to lookup for vulnerabilities by aliases. These two endpoints will be
replaced by query parameters on the main vulnerabilities/ endpoint when stabilized. -
Added filters for vulnerabilities endpoint to get fixed packages in accordance
to the details given in filters: For example, when you call the endpoint this way
/api/vulnerabilities?type=pypi&namespace=foo&name=bar
, you will receive only
fixed versioned purls of the typepypi
, namespacefoo
and namebar
. -
Package endpoint will give fixed packages of only those that
matches type, name, namespace, subpath and qualifiers of the package queried. -
Paginated initial listings to display a small number of records
and provided page per size with a maximum limit of 100 records per page. -
Add fixed packages in vulnerabilities details in packages endpoint.
-
Add bulk search support for CPEs.
-
Add authentication for REST API endpoint.
The autentication is disabled by default and can be enabled using the
VULNERABLECODEIO_REQUIRE_AUTHENTICATION settings.
When enabled, users have to authenticate using
their API Key in the REST API.
Users can be created using the Django "createsuperuser" management command. -
The data license is now CC-BY-SA-4.0 as this is the highest common
denominator license among all the data sources we collect and aggregate.
Other:
- We dropped calver to use a plain semver.
- We adopted vers and the new univers library to handle version ranges.
What's Changed
- Improve error handling and other misc. updates by @pombredanne in #267
- Fixed the spelling mistakes and grammatical errors by @Abhigyankrsingh in #269
- Add Apache HTTPD advisory importer by @sbs2001 in #261
- Add kaybee statement importer by @sbs2001 in #263
- Use packageurl version 0.9.3 and Add nginx importer by @sbs2001 in #264
- Add postgresql importer by @sbs2001 in #265
- Use skeleton project structure by @sbs2001 in #274
- Added faq section by @tushar912 in #283
- Update docs by @sbs2001 in #271
- Adapt rust importer to new advisory format by @sbs2001 in #281
- Use GH action instead travis for CI. by @sbs2001 in #295
- Add SOURCES.rst to document data sources being used by @sbs2001 in #298
- Cleanup codebase and fix minor bugs and other improvements by @sbs2001 in #278
- Import apache tomcat by @sbs2001 in #292
- Improve GitHub importer by @sbs2001 in #291
- Stop debian importer from collecting temp vulnerabilities by @sbs2001 in #285
- Add tests for nginx and postgres importers by @sbs2001 in #301
- Elixir Security Importer by @tushar912 in #294
- Bump lxml from 4.3.3 to 4.6.2 by @dependabot in #306
- Verbose name plural for 'PackageRelatedVulnerability' by @Shivam-316 in #309
- Use drf-spectacular instead of drf-yasg for API docs by @sbs2001 in #310
- Add endpoints for bulk requesting vulnerabilities and packages by @sbs2001 in #303
- Don't allow null values for qualifiers by @sbs2001 in #313
- Add nix support by @rolfschr in #275
- Fix package result count in web ui by @sbs2001 in #329
- Collect references from github importer by @sbs2001 in #331
- Add django admin functionality for searching and filtering objects by @sbs2001 in #330
- Add message when no vulnerabilities are found for a vuln_id by @tushar912 in #337
- Change Alpine data source to use new source by @sbs2001 in #339
- Store severity scores by @sbs2001 in #290
- Improve UI by @sbs2001 in #335
- Fix regex in schema validator in alpine importer by @sbs2001 in #347
- Improve docs by @pombredanne in #316
- Collect kafka cves by @sbs2001 in #342
- Make trailing slash optional in apis by @sbs2001 in #350
- Update Nix deps to incorporate latest Python packages by @rolfschr in #352
- Disable schema validation for alpine linux to fix nix test by @sbs2001 in #353
- Collect suse scores by @sbs2001 in #354
- Collect archlinux severity scores by @sbs2001 in #355
- Handle vulnerabilities which don't have any vulnerability ids by @sbs2001 in #259
- Collect ghsa severity by @sbs2001 in #358
- Use case insensitive inexact lookups for search views by @sbs2001 in #360
- Make RedHat CVE import more robust by @pombredanne in #319
- Refactor codebase and tests to treat Advisory class mutable by @sbs2001 in #363
- Improve Ubuntu OVAL importer by @pombredanne in #322
- Bump aiohttp from 3.6.2 to 3.7.4 by @dependabot in #364
- Update nix deps by @rolfschr in #367
- UI compress vuln view by @sbs2001 in #368
- Update pypi deps db to 2021-03-06. by @rolfschr in #370
- Update README.rst by @InLaw in #371
- Send severity data along with vulnerability in bulk api by @sbs2001 in #369
- Use a more specific url for cvss qualitative severity system. by @tushar912 in #373
- Add istio importer and tests by @tushar912 in #336
- [Refactor] Rename
vuln_references
toreferences
by @imnitishng in #377 - Explicity provide lxml parser to beautifulsoup by @Hritik14 in #382
- Correct API docs path and fix pytest invocation by @Hritik14 in #379
- Sanity Checks for redhat import response by @savish28 in #387
- Make sure vulnerability id is_cve or is_vulcoid by @Hritik14 in #389
- Fix various importer errors by @sbs2001 in #378
- Bump pyyaml from 5.3.1 to 5.4 by @dependabot in #401
- Bump djangorestframework from 3.11.0 to 3.11.2 by @dependabot in #392
- Add me to AUTHORS by @Hritik14 in #405
- Bump pygments from 2.6.1 to 2.7.4 by @dependabot in #414
- Fix istio by @Hritik14 in #395
- Update nix deps. by @rolfschr in #406
- enable aiohttp client to trust environement for proxy by @tardyp in #411
- import: continue upon failure by @tardyp in #412
- Misc fixes for deploying vulnerablecode on a container platform by @tardyp in #413
- enable configuration of allowed host by @tardyp in #404
- Bump lxml from 4.6.2 to 4.6.3 by @dependabot in #402
- Fix redhat import failure by @Hritik14 in #418
- Add unspecified scoring system by @Hritik14 in #415
- Add tests to check upstream data by @sbs2001 in #365
- Resolves #399: Rectify invalid id in msr2019 by @Pushpit07 in #428
- Fixes #312 : Added solution for ModuleNotFoundError by @Pushpit07 in #427
- remove duplicate import in importers by @sify21 in #430
- Drop dephell specifier by @sbs2001 in #421
- Bump django from 3.0.13 to 3.0.14 by @dependabot in #435
- Fix ubuntu_usn importer for invalid CVE by @AmitGupta7580 in #432
- Fix nix Github workflow by @rolfschr in #444
- expose find_all_cve helper by @Hritik14 in #439
- Add patched package by @sbs2001 in #436
- add
blank=True
to fields of Vulnerability and Package by @sify21 in #433 - Switch data_source dependency to GitPython by @tardyp in #409
- Resolves #380: Added severity in the view at http://127.0.0.1:8000/vulnerabilities/<… by @Pushpit07 in #390
- Add a management command for creating cpe2purl mapping by @sbs2001 in #450
- Update Apache_httpd importer from XML to JSON Advisory by @AmitGupta7580 in #425
- Reduce queries in UI by @sbs2001 in #453
- Dump yaml in favor of saneyaml by @Hritik14 in #452
- Improve actions time by @Hritik14 in #459
- Add support for setting Postgres port via env var by @kipz in #471
- Improve import time by @Hritik14 in #478
- Add summary in bulk api by @sbs2001 in #479
- Remove old code and organize deps by @sbs2001 in #480
- helper: split_markdown_front_matter by @Hritik14 in #443
- Ignore legacy pypi package versions in github importer by @sbs2001 in #491
- Time travel to the date of advisory publish time when importing by @sbs2001 in #467
- Refactor package_managers by @Hritik14 in #495
- Speed up upstream tests by @Hritik14 in #490
- Importers bugfix [nginx, debian_oval, ubuntu] by @Hritik14 in #505
- Improve Docker configuration by @Hritik14 in #497
- Add docker docs by @Hritik14 in #521
- Use svn to collects tags in GitHubTagsAPI by @Hritik14 in #508
- Make upstream tests use makefile by @Hritik14 in #523
- Make docs RTD compatible by @AyanSinhaMahapatra in #527
- Gsoc report by @Hritik14 in #529
- Fix GitHub CI badge by @sbs2001 in #539
- Ensure Debian test are passing by @pombredanne in #538
- Fixed the broken Docker Installation Link by @Lawful2002 in #548
- Stop using drf-spectacular by @sbs2001 in #542
- Fix several problems with the nix setup. by @rolfschr in #546
- DebianVersionAPI: automatic support for proxy by @tardyp in #558
- use github api to find github releases by @tardyp in #555
- Docker: set STATIC_ROOT to /var/vulnerablecode/static/ by @vbisserie in #569
- Fix istio importer by skipping UI metadata files. by @sbs2001 in #570
- Fix deprecated Django API by @yilmi in #592
- Separate import and improve operations by @Hritik14 in #525
- Correct notes for cvssv3.1_qr by @keshav-space in #599
- Dump importer_yielder in favor of IMPORTER_REGISTRY and drop Etags by @Hritik14 in #600
- Collect Mozilla by @Hritik14 in #393
- Collect Mattermost by @Hritik14 in #397
- Collect xen by @Pushpit07 in #464
- Sort imports by @Hritik14 in #476
- validation for page_no and fix presentation url . by @ziadhany in #618
- Collect go vulnerabilities from github api by @sify21 in #578
- Initial Documentation by @Hritik14 in #624
- Fix nix setup by @rolfschr in #635
- Improve RTD documentation by @Hritik14 in #631
- Make sure fixed purl is optional in Inference by @Hritik14 in #639
- migrate current alpine importer to alpine importer-improver model by @TG1999 in #623
- Migrate github importer to importer improver model by @TG1999 in #642
- Work around PosgreSQL index issue by @TG1999 in #653
- Fix github improver by @TG1999 in #663
- Fix API crash due to model changes by @TG1999 in #669
- Add tests for checking the API by @TG1999 in #671
- Fix improver framework not accepting severities by @Hritik14 in #684
- Migrate nvd importer to importer-improver model by @TG1999 in #664
- Improve test run by @pombredanne in #687
- Enable deployment by @pombredanne in #677
- Migrate OpenSSL importer to importer-improver model by @keshav-space in #690
- Add example importer and improver by @Hritik14 in #672
- Fix typing error by @TG1999 in #696
- Add nginx tests and other related improvements by @pombredanne in #691
- Remove null from string based fields by @TG1999 in #699
- Allow default improver to improve without affected packages by @TG1999 in #693
- Update command-line-interface.rst by @aydinnyunus in #703
- Amend to_dict function for inferences by @TG1999 in #702
- Bump django from 4.0.3 to 4.0.4 by @dependabot in #709
- Lookup Vulnerabilities by CPE by @TG1999 in #667
- test openssl improve and import by @keshav-space in #710
- Fix UI by @TG1999 in #700
- Fix typos in CHANGELOG by @keshav-space in #724
- #725 by @markrouz in #726
- Migrate redhat importer by @TG1999 in #719
- Add PyPI OSV by @ziadhany in #632
- Migrate debian importer to importer-improver model by @TG1999 in #723
- Update debian NOTICE by @TG1999 in #734
- Fix pysec importer by @TG1999 in #739
- Fix severity model by @TG1999 in #731
- Add gitlab importer by @TG1999 in #662
- Add firstPatchedVersion in github API by @TG1999 in #748
- Put network calls in try/except block by @TG1999 in #757
- doc: Adviory -> Advisory by @armijnhemel in #762
- Search from alias in UI by @TG1999 in #765
- Add search in API via alias by @TG1999 in #766
- Add changelog by @TG1999 in #773
- Prepare release by @pombredanne in #776
- Add backward compatibility for url and unresolved_vulnerabilities by @TG1999 in #779
- Add URLs to CPEs by @TG1999 in #785
- Add fixed packages in packages endpoint by @TG1999 in #784
- Bump lxml from 4.8.0 to 4.9.1 by @dependabot in #794
- Bump django from 4.0.4 to 4.0.6 by @dependabot in #793
- Prepare Release 30.0.0rc2 by @pombredanne in #797
- Delete references to CPEs with empty URLs #818 by @TG1999 in #827
- Paginate initial listings to display a small number of or records by @TG1999 in #830
- Add fixed packages in vulnerabilities details in packages endpoint. by @TG1999 in #831
- Bump django from 4.0.6 to 4.0.7 by @dependabot in #840
- Add bulk search support for CPEs #808 by @TG1999 in #844
- Add authentication by @TG1999 in #848
- Fix typos in API by @TG1999 in #853
- Add is_vulnerable property in fixed and affected_packages by @TG1999 in #869
- Add namespace filter in packages api by @TG1999 in #893
- Improve web user interface #798 by @johnmhoran in #847
- Modernize UI #798 by @johnmhoran in #813
- Implement initial set of RTD updates #885 #886 #887 #888 by @johnmhoran in #890
- Allow case insensitive search for VCIDs #875 by @TG1999 in #898
- Make URLs mandatory for references #891 by @TG1999 in #899
- Improve UI by @pombredanne in #894
- Migrate from VULCOID to VCID #811 by @TG1999 in #896
- Prepare release 30 by @pombredanne in #909
- Prepare v30.rc6 by @pombredanne in #914
- Fix UI wrap issue in Reference table #916 by @johnmhoran in #919
- Bump package-url version #918 by @TG1999 in #921
- Add TOS and API key contact info by @johnmhoran in #933
New Contributors
- @pombredanne made their first contribution in #267
- @Abhigyankrsingh made their first contribution in #269
- @tushar912 made their first contribution in #283
- @Shivam-316 made their first contribution in #309
- @rolfschr made their first contribution in #275
- @InLaw made their first contribution in #371
- @imnitishng made their first contribution in #377
- @Hritik14 made their first contribution in #382
- @savish28 made their first contribution in #387
- @tardyp made their first contribution in #411
- @Pushpit07 made their first contribution in #428
- @sify21 made their first contribution in #430
- @AmitGupta7580 made their first contribution in #432
- @kipz made their first contribution in #471
- @AyanSinhaMahapatra made their first contribution in #527
- @Lawful2002 made their first contribution in #548
- @vbisserie made their first contribution in #569
- @yilmi made their first contribution in #592
- @TG1999 made their first contribution in #623
- @aydinnyunus made their first contribution in #703
- @markrouz made their first contribution in #726
- @armijnhemel made their first contribution in #762
- @johnmhoran made their first contribution in #847
Full Changelog: v20.10...v30.0.0