Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update runtime's python packages #706

Merged
merged 1 commit into from
Sep 25, 2024
Merged

Update runtime's python packages #706

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 34 additions & 32 deletions runtimes/datascience/ubi9-python-3.11/Pipfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,41 +7,43 @@ name = "pypi"

[packages]
# Datascience and useful extensions
boto3 = "~=1.34.50"
kafka-python = "~=2.0.2"
matplotlib = "~=3.8.3"
numpy = "~=1.26.4"
pandas = "~=2.2.0"
plotly = "~=5.20.0"
scikit-learn = "~=1.4.0"
scipy = "~=1.12.0"
skl2onnx = "~=1.16.0"
codeflare-sdk = "~=0.19.1"
memray = "~=1.10.0" # any version above this would cause issue with rich package
boto3 = "~=1.35.14"
kafka-python-ng = "~=2.2.2"
matplotlib = "~=3.9.2"
numpy = "~=2.1.0"
pandas = "~=2.2.2"
plotly = "~=5.23.0"
scikit-learn = "~=1.5.1"
scipy = "~=1.14.1"
skl2onnx = "~=1.17.0"
onnxconverter-common = "~=1.13.0" # Required for skl2onnx, as upgraded version is not compatible with protobuf
codeflare-sdk = "~=0.18.0"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this version decrease intentional? 🤔 (same applies for this change in the rest of the files)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, because on jupyter ds notebook we have onnxconverter-common = "~=1.13.0" I kept the same on the runtime as well

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand the reason for the onnxconverter-common - but my question was for the codeflare-sdk.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I looked here - so I guess you tried to aligned these images... well, it's probably not a big deal for now as the regular codeflare-sdk update should fix it for all the images. But we should be sure we'll run it explicitly before the release then...


# DB connectors
pymongo = "~=4.6.2"
psycopg = "~=3.1.18"
pymongo = "~=4.8.0"
psycopg = "~=3.2.1"
pyodbc = "~=5.1.0"
mysql-connector-python = "~=8.3.0"
mysql-connector-python = "~=9.0.0"

# This is a comprehensive list of python dependencies that Elyra requires to execute Jupyter notebooks.
ipykernel = "==6.13.0"
ipython = "==8.10.0"
ipython-genutils = "==0.2.0"
jinja2 = "==3.1.4"
jupyter-client = "==7.3.1"
jupyter-core = "==4.11.2"
MarkupSafe = "==2.1.1"
minio = "==7.1.15"
nbclient = "==0.6.3"
nbconvert = "==7.1.0"
nbformat = "==5.4.0"
papermill = "==2.3.4"
pyzmq = "==24.0.1"
prompt-toolkit = "==3.0.30"
requests = "==2.31.0"
tornado = "==6.3.3"
traitlets = "==5.1.1"
urllib3 = "==1.26.18"
ipykernel = "~=6.29.5"
ipython = "~=8.27.0"
ipython-genutils = "~=0.2.0"
jinja2 = "~=3.1.4"
jupyter-client = ">=8.6.3"
jupyter-core = "~=5.7.2"
MarkupSafe = "~=2.1.5"
minio = "~=7.2.8"
nbclient = "~=0.10.0"
nbconvert = "~=7.16.4"
nbformat = "~=5.10.4"
papermill = "~=2.6.0"
pyzmq = "~=26.2.0"
prompt-toolkit = "~=3.0.47"
requests = "~=2.32.3"
tornado = "~=6.4.1"
traitlets = "~=5.14.3"
urllib3 = "~=1.26.20"

[requires]
python_version = "3.11"
2,685 changes: 1,433 additions & 1,252 deletions runtimes/datascience/ubi9-python-3.11/Pipfile.lock
View file
Open in desktop

Large diffs are not rendered by default.

32 changes: 16 additions & 16 deletions runtimes/datascience/ubi9-python-3.11/utils/requirements-elyra.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,24 +1,24 @@
# Copied from: https://github.com/elyra-ai/elyra/blob/main/etc/generic/requirements-elyra.txt

# This is a comprehensive list of python dependencies that Elyra requires to execute Jupyter notebooks.
ipykernel==6.13.0
ipython==8.10.0
ipykernel==6.29.5
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@harshad16 I've manually updated this file to the latest versions, and I've applied the same versions to the Pipfiles example as well.

The packages on the requirments.txt on elyra repo looks old: https://github.com/elyra-ai/elyra/blob/main/etc/generic/requirements-elyra.txt

Since I wasn't certain which versions to use, I opted to go with the latest updates.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personally I would remove any changes to these requirements-elyra.txt files from this PR. Or delete them completely to comply with: https://issues.redhat.com/browse/RHOAIENG-11068.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All the changes look good.
as we are not using the requirements-elyra.txt, we would anyway use the https://issues.redhat.com/browse/RHOAIENG-11068, to de-clutter this.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tnx for pointing this out, the changes have been incorporated under this commit 664ea37

ipython==8.27.0
ipython-genutils==0.2.0
jinja2==3.1.4
jupyter-client==7.3.1
jupyter-core==4.11.2
MarkupSafe==2.1.1
minio==7.1.15
nbclient==0.6.3
nbconvert==7.1.0
nbformat==5.4.0
papermill==2.3.4
pyzmq==24.0.1
prompt-toolkit==3.0.30
requests==2.31.0
tornado==6.3.3
traitlets==5.1.1
urllib3==1.26.9
jupyter-client==8.6.3
jupyter-core==5.7.2
MarkupSafe==2.1.5
minio==7.2.8
nbclient==0.10.0
nbconvert==7.16.4
nbformat==5.10.4
papermill==2.6.0
pyzmq==26.2.0
prompt-toolkit==3.0.47
requests==2.32.3
tornado==6.4.1
traitlets==5.14.3
urllib3==2.2.3
#
# These excluded are transitive dependencies of the included python packages.
#ansiwrap==0.8.4
Expand Down
36 changes: 18 additions & 18 deletions runtimes/minimal/ubi9-python-3.11/Pipfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,24 +7,24 @@ name = "pypi"

[packages]
# This is a comprehensive list of python dependencies that Elyra requires to execute Jupyter notebooks.
ipykernel = "==6.13.0"
ipython = "==8.10.0"
ipython-genutils = "==0.2.0"
jinja2 = "==3.1.4"
jupyter-client = "==7.3.1"
jupyter-core = "==4.11.2"
MarkupSafe = "==2.1.1"
minio = "==7.1.15"
nbclient = "==0.6.3"
nbconvert = "==7.1.0"
nbformat = "==5.4.0"
papermill = "==2.3.4"
pyzmq = "==24.0.1"
prompt-toolkit = "==3.0.30"
requests = "==2.31.0"
tornado = "==6.3.3"
traitlets = "==5.1.1"
urllib3 = "==1.26.18"
ipykernel = "~=6.29.5"
ipython = "~=8.27.0"
ipython-genutils = "~=0.2.0"
jinja2 = "~=3.1.4"
jupyter-client = "~=8.6.3"
jupyter-core = "~=5.7.2"
MarkupSafe = "~=2.1.5"
minio = "~=7.2.8"
nbclient = "~=0.10.0"
nbconvert = "~=7.16.4"
nbformat = "~=5.10.4"
papermill = "~=2.6.0"
pyzmq = "~=26.2.0"
prompt-toolkit = "~=3.0.47"
requests = "~=2.32.3"
tornado = "~=6.4.1"
traitlets = "~=5.14.3"
urllib3 = "~=2.2.3"

[requires]
python_version = "3.11"
1,030 changes: 623 additions & 407 deletions runtimes/minimal/ubi9-python-3.11/Pipfile.lock
View file
Open in desktop

Large diffs are not rendered by default.

32 changes: 16 additions & 16 deletions runtimes/minimal/ubi9-python-3.11/utils/requirements-elyra.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,24 +1,24 @@
# Copied from: https://github.com/elyra-ai/elyra/blob/main/etc/generic/requirements-elyra.txt

# This is a comprehensive list of python dependencies that Elyra requires to execute Jupyter notebooks.
ipykernel==6.13.0
ipython==8.10.0
ipykernel==6.29.5
ipython==8.27.0
ipython-genutils==0.2.0
jinja2==3.1.4
jupyter-client==7.3.1
jupyter-core==4.11.2
MarkupSafe==2.1.1
minio==7.1.15
nbclient==0.6.3
nbconvert==7.1.0
nbformat==5.4.0
papermill==2.3.4
pyzmq==24.0.1
prompt-toolkit==3.0.30
requests==2.31.0
tornado==6.3.3
traitlets==5.1.1
urllib3==1.26.9
jupyter-client==8.6.3
jupyter-core==5.7.2
MarkupSafe==2.1.5
minio==7.2.8
nbclient==0.10.0
nbconvert==7.16.4
nbformat==5.10.4
papermill==2.6.0
pyzmq==26.2.0
prompt-toolkit==3.0.47
requests==2.32.3
tornado==6.4.1
traitlets==5.14.3
urllib3==2.2.3
#
# These excluded are transitive dependencies of the included python packages.
#ansiwrap==0.8.4
Expand Down
73 changes: 38 additions & 35 deletions runtimes/pytorch/ubi9-python-3.11/Pipfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,45 +12,48 @@ name = "pytorch"

[packages]
# PyTorch packages
tensorboard = "~=2.16.2"
torch = {version = "~=2.2.1", index = "pytorch"}
torchvision = {version = "~=0.17.1", index = "pytorch"}
tensorboard = "~=2.17.1"
torch = {version = "~=2.4.0", index = "pytorch"}
torchvision = {version = "~=0.19.0", index = "pytorch"}

# Datascience and useful extensions
boto3 = "~=1.34.50"
kafka-python = "~=2.0.2"
matplotlib = "~=3.8.3"
numpy = "~=1.26.4"
pandas = "~=2.2.0"
plotly = "~=5.20.0"
scikit-learn = "~=1.4.0"
scipy = "~=1.12.0"
skl2onnx = "~=1.16.0"
codeflare-sdk = "~=0.19.1"
memray = "~=1.10.0" # any version above this would cause issue with rich package
boto3 = "~=1.35.14"
kafka-python-ng = "~=2.2.2"
matplotlib = "~=3.9.2"
numpy = "~=2.1.0"
pandas = "~=2.2.2"
plotly = "~=5.23.0"
scikit-learn = "~=1.5.1"
scipy = "~=1.14.1"
skl2onnx = "~=1.17.0"
onnxconverter-common = "~=1.13.0" # Required for skl2onnx, as upgraded version is not compatible with protobuf
codeflare-sdk = "~=0.18.0"

# DB connectors
pymongo = "~=4.6.2"
psycopg = "~=3.1.18"
pymongo = "~=4.8.0"
psycopg = "~=3.2.1"
pyodbc = "~=5.1.0"
mysql-connector-python = "~=8.3.0"
mysql-connector-python = "~=9.0.0"

# This is a comprehensive list of python dependencies that Elyra requires to execute Jupyter notebooks.
ipykernel = "==6.13.0"
ipython = "==8.10.0"
ipython-genutils = "==0.2.0"
jinja2 = "==3.1.4"
jupyter-client = "==7.3.1"
jupyter-core = "==4.11.2"
MarkupSafe = "==2.1.1"
minio = "==7.1.15"
nbclient = "==0.6.3"
nbconvert = "==7.1.0"
nbformat = "==5.4.0"
papermill = "==2.3.4"
pyzmq = "==24.0.1"
prompt-toolkit = "==3.0.30"
requests = "==2.31.0"
tornado = "==6.3.3"
traitlets = "==5.1.1"
urllib3 = "==1.26.18"
ipykernel = "~=6.29.5"
ipython = "~=8.27.0"
ipython-genutils = "~=0.2.0"
jinja2 = "~=3.1.4"
jupyter-client = "~=8.6.3"
jupyter-core = "~=5.7.2"
MarkupSafe = "~=2.1.5"
minio = "~=7.2.8"
nbclient = "~=0.10.0"
nbconvert = "~=7.16.4"
nbformat = "~=5.10.4"
papermill = "~=2.6.0"
pyzmq = "~=26.2.0"
prompt-toolkit = "~=3.0.47"
requests = "~=2.32.3"
tornado = "~=6.4.1"
traitlets = "~=5.14.3"
urllib3 = "~=1.26.20"

[requires]
python_version = "3.11"
Loading
Loading