update superset 3.1.0/3.1.1 dependency "setuptools 58.1.0"

[nigzak]

Bug description

The docker inspector marks the image of superset 3.1.0 with a finding of setuptools 58.1.0

https://scout.docker.com/vulnerabilities/id/CVE-2022-40897?s=github&n=setuptools&t=pypi&vr=%3C65.5.1&utm_source=desktop&utm_medium=ExternalLink
CVSS = 7.5
fixed with 65.5.1

=> update to 65.5.1 (or newer) should be done

How to reproduce the bug

download superset 3.1.0 image
open docker scout

Screenshots/recordings

Superset version

3.1.0
3.1.1
master on 2024-03-05

Python version

3.9

Node version

16

Browser

Chrome

Additional context

V3.0.3 is also affected

Checklist

• I have searched Superset docs and Slack and didn't find a solution to my problem.
• I have searched the GitHub issue tracker and didn't find a similar bug report.
• I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.

[rusackas]

If I'm not mistaken, we're on >=65.5.1 now.

[rusackas]

CC @john-bodley to fact-check this for me... we can re-open if I'm mistaken.

[nigzak]

Hi again @rusackas

I am not sure that in the 3.1.0 docker image is the updated setuptools? image => apache/superset:3.1.0
If I download the blank image it still says there is setuptools 58.1.0 inside and not >=65.5.1 ...

I still think this here is something to update (?) - but at least I cannot tell it to you for sure

EDIT
I was able to start the container (had issues with the local secure key ...)
=> it has setuptools 58.1.0 inside

setuptools             58.1.0

Full list => 

pip list
Package                Version     Editable project location
---------------------- ----------- -------------------------
alembic                1.6.5
amqp                   5.1.0
apache-superset        3.1.0       /app
apispec                6.3.0
appnope                0.1.3
apsw                   3.42.0.1
astroid                2.15.8
asttokens              2.2.1
async-timeout          4.0.2
attrs                  23.1.0
Babel                  2.9.1
backcall               0.2.0
backoff                1.11.1
bcrypt                 4.0.1
billiard               3.6.4.0
boto3                  1.26.130
botocore               1.29.130
Bottleneck             1.3.7
Brotli                 1.0.9
cached-property        1.5.2
cachelib               0.9.0
cattrs                 23.2.1
celery                 5.2.2
certifi                2023.7.22
cffi                   1.15.1
chardet                5.1.0
charset-normalizer     3.2.0
click                  8.1.3
click-didyoumean       0.3.0
click-option-group     0.5.5
click-plugins          1.1.1
click-repl             0.2.0
colorama               0.4.6
convertdate            2.4.0
cron-descriptor        1.2.24
croniter               1.0.15
cryptography           41.0.2
decorator              5.1.1
Deprecated             1.2.13
deprecation            2.1.0
dill                   0.3.6
dnspython              2.1.0
email-validator        1.1.3
et-xmlfile             1.1.0
exceptiongroup         1.1.1
executing              1.2.0
Flask                  2.2.5
Flask-AppBuilder       4.3.10
Flask-Babel            1.0.0
Flask-Caching          2.1.0
Flask-Compress         1.13
Flask-Cors             3.0.10
Flask-JWT-Extended     4.3.1
Flask-Limiter          3.3.1
Flask-Login            0.6.0
Flask-Migrate          3.1.0
Flask-Session          0.5.0
Flask-SQLAlchemy       2.5.1
flask-talisman         1.0.0
Flask-WTF              1.1.1
func-timeout           4.3.5
future                 0.18.3
geographiclib          1.52
geopy                  2.2.0
greenlet               2.0.2
gunicorn               21.2.0
hashids                1.3.1
hijri-converter        2.3.1
holidays               0.23
humanize               3.11.0
idna                   3.2
ijson                  3.2.0.post0
importlib-metadata     6.6.0
importlib-resources    5.12.0
ipython                8.12.2
isodate                0.6.0
isort                  5.12.0
itsdangerous           2.1.2
jedi                   0.18.2
Jinja2                 3.1.2
jmespath               1.0.1
jsonlines              3.1.0
jsonschema             4.17.3
kombu                  5.2.4
korean-lunar-calendar  0.3.1
lazy-object-proxy      1.9.0
limits                 3.4.0
linear-tsv             1.1.0
llvmlite               0.40.1
Mako                   1.2.4
Markdown               3.3.4
markdown-it-py         2.2.0
MarkupSafe             2.1.1
marshmallow            3.19.0
marshmallow-sqlalchemy 0.23.1
matplotlib-inline      0.1.6
mccabe                 0.7.0
mdurl                  0.1.2
msgpack                1.0.2
mysqlclient            2.1.0
nh3                    0.2.11
numba                  0.57.1
numexpr                2.8.4
numpy                  1.23.5
openpyxl               3.1.2
ordered-set            4.1.0
packaging              23.1
pandas                 2.0.3
paramiko               2.11.0
parsedatetime          2.6
parso                  0.8.3
pexpect                4.8.0
pgsanity               0.2.9
pickleshare            0.7.5
Pillow                 9.5.0
pip                    23.0.1
platformdirs           3.8.1
polyline               2.0.0
prison                 0.2.1
progress               1.6
prompt-toolkit         3.0.38
psycopg2-binary        2.9.6
ptyprocess             0.7.0
pure-eval              0.2.2
pure-sasl              0.6.2
pyarrow                14.0.1
pyasn1                 0.5.0
pyasn1-modules         0.3.0
pycparser              2.20
pydruid                0.6.5
Pygments               2.15.0
PyHive                 0.7.0
pyinstrument           4.4.0
PyJWT                  2.4.0
pylint                 2.17.7
PyMeeus                0.5.12
PyNaCl                 1.5.0
pyparsing              3.0.6
pyrsistent             0.19.3
python-dateutil        2.8.2
python-dotenv          0.19.0
python-editor          1.0.4
python-geohash         0.8.5
python-ldap            3.4.3
pytz                   2021.3
PyYAML                 6.0.1
redis                  4.5.4
requests               2.31.0
requests-cache         1.1.1
rfc3986                2.0.0
rich                   13.3.4
s3transfer             0.6.1
selenium               3.141.0
setuptools             58.1.0
shillelagh             1.2.10
shortid                0.1.2
simplejson             3.17.3
six                    1.16.0
slack-sdk              3.21.3
SQLAlchemy             1.4.36
SQLAlchemy-Utils       0.38.3
sqloxide               0.1.33
sqlparse               0.4.4
sshtunnel              0.4.0
stack-data             0.6.2
tableschema            1.20.2
tabulate               0.8.9
tabulator              1.53.5
thrift                 0.16.0
thrift-sasl            0.4.3
tomli                  2.0.1
tomlkit                0.11.8
traitlets              5.9.0
typing_extensions      4.4.0
tzdata                 2023.3
unicodecsv             0.14.1
url-normalize          1.4.3
urllib3                1.26.6
vine                   5.0.0
wcwidth                0.2.5
Werkzeug               2.3.3
wheel                  0.42.0
wrapt                  1.15.0
WTForms                2.3.3
WTForms-JSON           0.3.5
xlrd                   2.0.1
XlsxWriter             3.0.7
zipp                   3.15.0

[nigzak]

Hi @rusackas

also superset 3.1.1 has setuptools 58.1.0 inside ... I think we should reopen this ticket or?

[rusackas]

Fair enough! Re-opening, pending further investigation/confirmation.

[nigzak]

I made some tests with blank python docker images, it seems somehow to be an issue with python 3.9 (?)

// use a default installation in python 3.9-slim

from python:3.9-slim
RUN pip install setuptools
RUN pip show setuptools

Setuptools 58.1.0 is inside

// use latest version in python 3.9-slim

from python:3.9-slim
RUN pip install setuptools==69.1.1
RUN pip show setuptools

Setuptools 69.1.1 is inside

// use latest version in python 3.10-slim

from python:3.10-slim
RUN pip install setuptools
RUN pip show setuptools

Setuptools 65.5.1 is inside

Because setuptool in default installation in 3.9-slim without version is python-version in superset ("buster") - might it be that there is some problem with a "default" package configuration depending the python version..?
(If you try to update to "latest" version bevore installing - and it picks a old version because of this - it might not be updated anywhere ... I don't know, it is the first time I see something like this?)

would also match with this: https://docs.python.org/3.9/whatsnew/changelog.html

I found here a installation (don't know if this is the relevant position)

here is no version set - so it MIGHT be installing the default old version inside the docker image?
File: RELEASING/Dockerfile.from_local_tarball

[rusackas]

Thanks for the additional diagnostics @nigzak! If this seems to center on the flavor of Python involved, I wonder if @john-bodley or @mistercrunch might have ideas of how to address it.

[mistercrunch]

We're still a bit stuck with #26944, let me try to revive it. I could also just bump that requirements/development.txtoutside of pip-compile-multi. Will do both

[mistercrunch]

Here -> #27405

[nigzak]

if I pull the master it now has setuptools V69.1.1 inside => should be fixed. Shall we close this ticket or let it be open until next stable release (3.1.2 or 4.x.x)? I don't know the procedure in this case ...

[rusackas]

I'll go ahead and close it. We typically close issues when there's no further developer action to take on master which seems to be the case here. It's still due to come out in a future release, but for now, there's really nothing more to be done.