Improve npm package declared license detection

[pombredanne]

The goal of this ticket is to improve npm package license detection across the board. While scancode-toolkit's npm package detection basics are OK, there a few repeat cases where license information is not properly gathered from npm package metadata. Usually this is because a declared_license value contains things we did not expect (like a URL) or is improperly formed. There are also specific things we do not support such as SEE LICENSE (defined in https://docs.npmjs.com/cli/v7/configuring-npm/package-json#license and related ticket in #1364 )

Resolving this would likely require a mix of:

• adding new license detection rules to scancode,
• adding new and improved code to handle the specific patterns of license,
• creating new license mappings
• and possibly working with upstream maintainers to improve their license declarations.

The approach should be to start with a complete data set of all package manifests and find patterns of license issues and establish the baseline, possibly helped by heuristics, classifiers and ML if needed. The end results should be a significant improvement to the license detection quality for the npm packages.

See also these projects and pages of interest:

• https://github.com/jslicense/spdx-correct.js
• https://github.com/jslicense/npm-license-corrections.json
• https://github.com/nice-registry/
• https://github.com/nice-registry/npm-license-stats and https://github.com/nice-registry/npm-license-stats/blob/master/results.json in particular that may have all the npm licenses

There are also other related ticket for other package types such as:

• Improve license detection of declared RPM licenses #2412 for RPM that has some detailed examples
• Improve PyPI package declared license detection #2487 for PyPI