Bug fix on sitemap adds localFileScan, move report directory up one level

[angyonghaseyo]

• Allow sitemap to be able to recurse all the files (Previously it was just getting the last childSitemap files, instead of all of the files)

• Allow pdfScanFunc to run both filePath and url (it uses fs for file path and got for url)

• Prevent sitemap from recursing infinitely

• Do checking if its a file path or url else skip it, without stopping the scan

• Added crawlLocalFile (-c 5)

• Added results and log folder name based on the local file name given

• Added try catch to all the json parsing for sitemap

• Fix bug of able to scan files with dot separator within the file name

• Allow all types of files to be scanned (verapdf for pdf files, axescript for non pdf files)

• Added typing for crawLocalFile.js

• Move reports directory to parent directory

• I've kept this PR as small as possible (~500 lines) by splitting it into PRs with manageable chunks of code

• I've requested reviews from 1 reviewer

• I've tested existing features (website scan, sitemap, custom flow) in both node index and cli

• I've synced this fork with GovTechSG repo

• I've added/updated unit tests

• I've added/updated any necessary dependencies in package[-lock].json npm audit, portable installation on GitHub Actions

[younglim]

@angyonghaseyo

Local file scan:

Ambiguous help text

npm run cli -- -c 5 -p 100 -u "https://www.tech.gov.sg" -k "..." -a none -h no

Output

 Provided URL or filepath is not a sitemap.

Rename it to Provided URL file path is not a local file or sitemap.

Unhandled exception

npm run cli -- -c 5 -p 100 -u "/Users/young/purple-a11y/results/some_report.html" -k "..." -a none -h no

Output

{"timestamp":"2024-06-19 13:33:18","level":"error","message":"uncaughtException: Invalid URL\nTypeError: Invalid URL\n    at new URL (node:internal/url:797:36)\n    at combineRun (file:///Users/young/purple-a11y/dist/combine.js:89:13)\n    at async scanInit (file:///Users/young/purple-a11y/dist/cli.js:264:5)"}

[younglim]

See comments

[younglim]

Please set file:// as the finalUrl much earlier in the call stack and not check if local file scan superfluously. You need to think about code maintainability / readability