Feature request: provide a way to build a wheel based on a PyPI release

[astrofrog]

It would be nice to have a way to build wheels based on an existing source release on PyPI without having to manually download the tar file, expand it, etc. This could look like:

cibuildwheel --from-pypi=mypackage

It would be nice to be able to optionally specify a version (when building wheels for old versions of packages).

Another related feature that would be nice is the ability to also skip the building of wheels when a wheel with the same name exists on PyPI. This would then allow a workflow of having a dedicated repository with a bunch of cibuildwheel commands for different packages, and have it run e.g. as a daily cron job, building new wheels whenever necessary.

[joerick]

Interesting! I guess this is for running cibuildwheel on something other than Travis, Appveyor, something like Jenkins?

Just something that popped into my head - how does it work when the source tarball is uploaded before the wheels are? Do the installs in the meantime require source-builds, or could the version be marked as prerelease until the wheels are available?

[astrofrog]

@joerick - actually I still would like to use this on Travis and AppVeyor. I'd just rather keep the repo separately from the actual package repo - because we have 4-5 packages we need to build and it'd be nicer to just centralize the building.

When the source tarball is uploaded before the wheels (which is common for our workflow at the moment), the source version is used automatically by pip if wheels are not available.

[YannickJadoul]

@astrofrog @joerick This will make it probably rather easy to integrate in the existing workflow: IIRC pip wheel will by default try to get the wheel by first trying to download the correct wheel. If that doesn't exist, it will download the source distribution and build it. And if you give it a path (as we are doing) it will build the path.

So in principle, the solution might be as simple as optionally taking an argument/environment variable, and replace the abs_project_dir by the value of that option.

Happy to give it a shot, one of these days, if you guys don't beat me to it :)

[astrofrog]

@joerick @YannickJadoul - just for information, I've built a package called autowheel that is a wrapper around cibuildwheel, and makes it easy to automatically build any missing wheels from PyPI releases for one or more packages. I'm not sure if you are interested in having something like that be part of cibuildwheel or if it's best to keep it separate, but I just wanted to give you a heads-up in case you are interested. I haven't actually released it at this point and am just using it for personal projects, but I do plan to release it at some point (unless there is interest in making it part of cibuildwheel).

[joerick]

Hey @astrofrog ! Nice one, and thanks for letting us know. So I guess you keep your autowheel config in a different repo from the Python package itself?

[astrofrog]

@joerick - yes, in my case I'll have a repo that just has this config and runs on a cron job and builds wheels for a number of different packages.

[joerick]

Sweet. It's great that you just built it, wrapping cibuildwheel to use it the way you wanted! I like the YAML config too.

I'll add a link from the README. Also, let me know if there's something we can do to make it easier to wrap!