Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update conda package publish for muliple python versions #1805

Merged
merged 13 commits into from
May 4, 2023

Conversation

oliverholworthy
Copy link
Member

@oliverholworthy oliverholworthy commented Apr 26, 2023

Updates conda package build so that we build and publish the NVTabular conda package for multiple python versions.

Currently we only publish a package for Python 3.8. This PR adds a package for 3.9, 3.10, in addition to 3.8.

And removes the upper bound on the cupy version in the dependencies

@github-actions
Copy link

Documentation preview

https://nvidia-merlin.github.io/NVTabular/review/pr-1805

runs-on: ${{ matrix.os }}
strategy:
matrix:
python-version: [3.8]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wondering if we could add 3.9 and 3.10 here so the builds run in separate actions?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I read some possibly old docs that you could pass in multiple values for the python parameter, which seems like it doesn't work. Updated to the matrix version again and checking if this works now

@karlhigley karlhigley added this to the Merlin 23.05 milestone May 4, 2023
echo "conda_package=$CONDA_PACKAGE" >> $GITHUB_OUTPUT
conda install -c conda-forge mamba
mamba install -c conda-forge conda-build boa conda-verify pybind11
conda mambabuild --python ${{ matrix.python-version }} . -c defaults -c conda-forge -c numba -c rapidsai -c nvidia --output-folder ./conda_packages
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using mambabuild instead of regular conda build cut the build time down from 3-5 hours to about 10 minutes.

@oliverholworthy
Copy link
Member Author

This PR has already been applied to the release branch, and sucessfully published versions of the conda package for Python 3.9 and 3.10 (in addition to 3.8). Still needs to be merged into main so that this will also apply to the next release

@karlhigley karlhigley merged commit 454b875 into main May 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants