Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deploy dash* wheels to PyPI on releases #1440

Closed
anders-kiaer opened this issue Oct 20, 2020 · 3 comments
Closed

Deploy dash* wheels to PyPI on releases #1440

anders-kiaer opened this issue Oct 20, 2020 · 3 comments

Comments

@anders-kiaer
Copy link
Contributor

anders-kiaer commented Oct 20, 2020

When wheels are available on PyPI, this typically makes pip install x significantly faster compared when only source distributions (.tar.gz) are available on PyPI. There are also other benefits to wheels compared with from source.

When you today do pip install dash you see output like

Building wheel for dash (setup.py): started
Building wheel for dash (setup.py): finished with status 'done'
...

as well as for dash-renderer, dash-core-components, dash-table, dash-html-components etc.

This is due to e.g. dash PyPI files only have source distribution (as compared to other packages like flask which have universal wheels - see https://packaging.python.org/guides/distributing-packages-using-setuptools/#universal-wheels for definition of universal wheels).

I could not find where in your CI (/CD) workflow deployments to PyPI are done on git tags/releases. Maybe deployments are currently done manually? If not, I can contribute with PRs in relevant repositories if location in CI/CD is pointed out.

Creating universal wheels, and not only source distribution, is typically as easy as changing from python setup.py sdist to
python setup.py sdist bdist_wheel. This should make pip install dash significantly faster.

@silverguo
Copy link

silverguo commented Nov 2, 2020

Definitely looking forward to have the wheels uploaded to pip for dash.

Happy to help if needed.

@anders-kiaer
Copy link
Contributor Author

Bumping this issue with some additional context/input, as I think this is an easy low hanging fruit to further improve dash user experience 🚀

In order to see how much faster installation is when a wheel is available compared to only sdist/tar.gz, I tested pip installing a wheel and sdist build of dash, respectively, using the current dev branch. On my machine I saw a 50% reduction in dash installation time (and I expect ~similar installation time improvements for the dependencies dash-core-components, dash-html-components, dash-table etc.)

Having a wheel available on PyPI also avoids some users getting an error during installation of the form (which pip recovers from by falling back to source installation, but still can confuse users):

pip install dash
...
Building wheels for collected packages: dash
  Building wheel for dash (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /home/[...]/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-jifoji898/setup.py'"'"'; __file__='"'"'/tmp/pip-req-build-jifoji898/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-89ud89a
       cwd: /tmp/pip-req-build-jifoji898/
  Complete output (6 lines):
  usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
     or: setup.py --help [cmd1 cmd2 ...]
     or: setup.py --help-commands
     or: setup.py cmd --help
  
  error: invalid command 'bdist_wheel'
  ----------------------------------------
  ERROR: Failed building wheel for dash
  Running setup.py clean for dash
Failed to build dash
Installing collected packages: dash
    Running setup.py install for dash ... done
Successfully installed dash-1.20.0
...

You also reduce the risk of different installation errors. An example is the one black has experienced lately (black is one of the few top 360 Python packages not currently having a wheel), where one of the pip maintainers give a general reminder that also Python projects (without e.g. C-code to compile) should have wheels available, which is pip's preferred/default format to download and install: psf/black#1847 (comment).


I don't know your release procedure (manual/automatic), but appending bdist_wheel --universal to the python setup.py sdist command (and potentially a pip install wheel prior to that command on the build computer/node) should be ~enough.

Are the artifacts after commands with sdist in the CI like here

cd dash-renderer && renderer build && python setup.py sdist && mv dist/* ../packages/ && cd ..
used in the release procedure? If so I can create a PR. Or is the release procedure defined outside the repo? (cc @harryturr - last release manager).

Thanks 🙂

@anders-kiaer
Copy link
Contributor Author

Starting with version 2.0, the PyPI release of dash now has wheels:
https://pypi.org/project/dash/2.0.0/#files

🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants