A malformed Python RECORD file stops Syft processing

[adinerman]

What happened:
while scanning an image (harbor-repo.vmware.com/coe_repo/priority_portal:3.0) the python cataloger failed with error:
unable to read python record file: record on line 27: wrong number of fields

What you expected to happen:
when it fails we should skip the line/pkg (with a warning) and not stop the run.

How to reproduce it (as minimally and precisely as possible):
syft harbor-repo.vmware.com/coe_repo/priority_portal:3.0

Anything else we need to know?:
when I went over the var file I found the problem lines:

app/venv/Lib/site-packages/Flask-1.0.2.dist-info/RECORD0100777000000000000000000000607213752730745020552 0ustar0000000000000000Flask-1.0.2.dist-info/LICENSE.txt,sha256=ziEXA3AIuaiUn1qe4cd1XxCESWTYrk4TjN7Qb06J3l8,1575
Flask-1.0.2.dist-info/METADATA,sha256=iA5tiNWzTtgCVe80aTZGNWsckj853fJyfvHs9U-WZRk,4182
Flask-1.0.2.dist-info/RECORD,,
Flask-1.0.2.dist-info/WHEEL,sha256=J3CsTk7Mf2JNUyhImI-mjX-fmI4oDjyiXgWT4qgZiCE,110
Flask-1.0.2.dist-info/entry_points.txt,sha256=gBLA1aKg0OYR8AhbAfg8lnburHtKcgJLDU52BBctN0k,42
Flask-1.0.2.dist-info/top_level.txt,sha256=dvi65F6AeGWVU0TBpYiC04yM60-FX1gJFkK31IKQr5c,6
flask/__init__.py,sha256=qq8lK6QQbxJALf1igz7qsvUwOTAoKvFGfdLm7jPNsso,1673
flask/__main__.py,sha256=pgIXrHhxM5MAMvgzAqWpw_t6AXZ1zG38us4JRgJKtxk,291
flask/_compat.py,sha256=UDFGhosh6mOdNB-4evKPuneHum1OpcAlwTNJCRm0irQ,2892
flask/app.py,sha256=ahpe3T8w98rQd_Er5d7uDxK57S1nnqGQx3V3hirBovU,94147
flask/blueprints.py,sha256=Cyhl_x99tgwqEZPtNDJUFneAfVJxWfEU4bQA7zWS6VU,18331
flask/cli.py,sha256=30QYAO10Do9LbZYCLgfI_xhKjASdLopL8wKKVUGS2oA,29442
flask/config.py,sha256=kznUhj4DLYxsTF_4kfDG8GEHto1oZG_kqblyrLFtpqQ,9951
flask/ctx.py,sha256=leFzS9fzmo0uaLCdxpHc5_iiJZ1H0X_Ig4yPCOvT--g,16224
flask/debughelpers.py,sha256=1ceC-UyqZTd4KsJkf0OObHPsVt5R3T6vnmYhiWBjV-w,6479
flask/globals.py,sha256=pGg72QW_-4xUfsI33I5L_y76c21AeqfSqXDcbd8wvXU,1649
flask/helpers.py,sha256=YCl8D1plTO1evEYP4KIgaY3H8Izww5j4EdgRJ89oHTw,40106
flask/logging.py,sha256=qV9h0vt7NIRkKM9OHDWndzO61E5CeBMlqPJyTt-W2Wc,2231
flask/sessions.py,sha256=2XHV4ASREhSEZ8bsPQW6pNVNuFtbR-04BzfKg0AfvHo,14452
flask/signals.py,sha256=BGQbVyCYXnzKK2DVCzppKFyWN1qmrtW1QMAYUs-1Nr8,2211
flask/templating.py,sha256=FDfWMbpgpC3qObW8GGXRAVrkHFF8K4CHOJymB1wvULI,4914
flask/testing.py,sha256=XD3gWNvLUV8dqVHwKd9tZzsj81fSHtjOphQ1wTNtlMs,9379
flask/views.py,sha256=Wy-_WkUVtCfE2zCXYeJehNgHuEtviE4v3HYfJ--MpbY,5733
flask/wrappers.py,sha256=1Z9hF5-hXQajn_58XITQFRY8efv3Vy3uZ0avBfZu6XI,7511
flask/json/__init__.py,sha256=Ns1Hj805XIxuBMh2z0dYnMVfb_KUgLzDmP3WoUYaPhw,10729
flask/json/tag.py,sha256=9ehzrmt5k7hxf7ZEK0NOs3swvQyU9fWNe-pnYe69N60,8223
<<<<<<< HEAD
../../Scripts/flask.exe,sha256=mPrbVeZCDX20himZ_bRai1nCs_tgr7jHIOGZlcgn-T4,93063
=======
../../Scripts/flask.exe,sha256=jvqh4N3qOqXLlq40i6ZOLCY9tAOwfwdzIpLDYhRjoqQ,89470
>>>>>>> 69c24e18ea630ec869a32ad7b0b0d9647714cd5d
Flask-1.0.2.dist-info/INSTALLER,sha256=zuuue4knoyJ-UwPPXg8fezS7VCrXJQrAP7zeNuwvFQg,4
flask/json/__pycache__/tag.cpython-36.pyc,,
flask/json/__pycache__/__init__.cpython-36.pyc,,
flask/__pycache__/app.cpython-36.pyc,,
flask/__pycache__/blueprints.cpython-36.pyc,,
flask/__pycache__/cli.cpython-36.pyc,,
flask/__pycache__/config.cpython-36.pyc,,
flask/__pycache__/ctx.cpython-36.pyc,,
flask/__pycache__/debughelpers.cpython-36.pyc,,
flask/__pycache__/globals.cpython-36.pyc,,
flask/__pycache__/helpers.cpython-36.pyc,,
flask/__pycache__/logging.cpython-36.pyc,,
flask/__pycache__/sessions.cpython-36.pyc,,
flask/__pycache__/signals.cpython-36.pyc,,
flask/__pycache__/templating.cpython-36.pyc,,
flask/__pycache__/testing.cpython-36.pyc,,
flask/__pycache__/views.cpython-36.pyc,,
flask/__pycache__/wrappers.cpython-36.pyc,,
flask/__pycache__/_compat.cpython-36.pyc,,
flask/__pycache__/__init__.cpython-36.pyc,,
flask/__pycache__/__main__.cpython-36.pyc,,
app/venv/Lib/site-packages/Flask-1.0.2.dist-info/WHEEL0100777000000000000000000000015613752730745020435 0ustar0000000000000000Wheel-Version: 1.0

Environment:

• Output of syft version:

Version:       0.14.0
BuildDate:     2021-03-20T16:47:03Z
GitCommit:     26a4dd36a8932c7867fcdd6296f50643be30f8a6
GitTreeState:  clean
Platform:      darwin/amd64
GoVersion:     go1.14.15
Compiler:      gc```

[luhring]

Confirmed this on my machine. This failure can be reproduced by using the above RECORD file as a fixture in the existing TestParseWheelEggRecord test.

[adinerman]

hey, is there any progress with the issue?

[yonatan-shorani]

Any update?

[tgerla]

Hi @yonatan-shorani and @adinerman, can you help me reproduce this? I'm trying to use the previously mentioned command:

syft harbor-repo.vmware.com/coe_repo/priority_portal:3.0

...but the hostname is no longer resolving and I'm not very familiar with Harbor. If you have a simplified way to reproduce this for us, we would appreciate it. Thanks.

[yonatan-shorani]

It should be a public image, also in grype you get the error

 grype harbor-repo.vmware.com/coe_repo/priority_portal:3.0
 ✔ Vulnerability DB        [updated]
 ✔ Pulled image
 ✔ Loaded image
 ✔ Parsed image
 ⠼ Cataloging packages     [packages 0]

[0249]  WARN cannot parse field from path: "/app/venv/Lib/site-packages/pycallgraph-1.0.1-py3.6.egg-info/PKG-INFO" from line: "\t\t       Version 2, June 1991" from-lib=syft
1 error occurred:

* failed to catalog: 1 error occurred:
* unable to catalog python package=/app/venv/Lib/site-packages/Flask-1.0.2.dist-info/METADATA: unable to read python record file: record on line 27: wrong number of fields

[tgerla]

That hostname doesn't resolve for me, it must be internal:

tgerla@Timothys-MacBook-Pro-2 ~ % syft harbor-repo.vmware.com/coe_repo/priority_portal:3.0
2022/10/19 09:50:07 error during command execution: 1 error occurred:
	* failed to construct source from user input "harbor-repo.vmware.com/coe_repo/priority_portal:3.0": could not fetch image "harbor-repo.vmware.com/coe_repo/priority_portal:3.0": unable to use OciRegistry source: failed to get image descriptor from registry: Get "https://harbor-repo.vmware.com/v2/": dial tcp: lookup harbor-repo.vmware.com on 192.168.1.1:53: no such host

...but I was able to reproduce it with a fixture based on @adinerman's original file. I think we have enough to move this into the backlog. Thank you for the help!