Compile with shared ITK/VTK to reduce wheel size #346
Conversation
|
I can't seem to attach a wheel (or a zip of a wheel) to this post, so here is a link: |
|
I defer to @stnava for pushing this along but thanks for this @aganders3 . |
|
hi - thx for this - I would like to see more evidence that this strategy leads to wheels that are truly portable. unfortunately, I don't have time right now to check that myself - hopefully next week. |
|
I've tested this working in our environment but unfortunately the large uncompressed size negates the benefit on our end (installed in a Docker image). I'm going to keep investing as time allows because the huge compression ratio does not seem right. |
|
Okay I spent another chunk of time on this today and I think the large inflated size comes from some issue in In any case here is the link to the version I am using internally; this wheel is now ~50 MB compressed and ~200 MB uncompressed: http://files.aga3.xyz/antspyx-0.3.2%2Bhri.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl Hopefully this turns out to be as portable as your current wheels and can benefit all the ANTsPy users! |
|
I know this is quite old, but is there any interest in moving this along? I'm no longer at the job this was relevant for, but I still think it's a good idea. The good news is that since I proposed this my other change has been released in auditwheel so it should be much smoother to implement and test. Otherwise I'm happy to close this and let it be a reference in case someone would like to pick it up in the future. |
|
Thanks for bringing this up @aganders3 , the wheels are much smaller now after @ncullen93's Herculean feat of porting ANTsPy to nanobind, but they haven't been updated on PyPI. I think we would need to delete the 0.3.8 release to make space, after which we could tag a new release. I may need some help if the automated push fails, but could have a go later in the week |
|
Once I am back from vacation, I will also take a closer look at the wheels - specifically for pypi. |
Hello - after some old packages were removed from PyPI I did a little digging to see if I could shrink the antspyx wheels. I was able to reduce the final wheel size from ~300 MB to ~51 MB. The key was configuring cmake to build shared libs for ITK (and VTK, though I don't think this mattered), which also required changing the configs for symbol visibility. I then built using
cibuildwheelwhich callsauditwheel repairto copy the resulting libs into a directory (antspyx.libs) in the wheel and modify RPATH in any libs as necessary. Surprisingly (to me) the unzipped size is about the same or even slightly bigger (extra symbols remaining?).To run the build yourself, check out this branch and run:
(this will produce manylinux wheels for cp36, 37, 38, and 39)
There is probably a bit more optimization possible here:
stripto remove debug symbols (my initial attempt broke the wheel, but being more selective may work)See below for test output (installed and run in the officeal python:3.7 Docker image) and a comparison of the final wheel contents.
Test output after install
Contents of wheel with shared ITK libs
Contents of wheel with static ITK libs