protobuf C++ implementation

[dmitriykovalev]

Package name: protobuf
Link to PyPI page: https://pypi.org/project/protobuf
Link to piwheels page: https://www.piwheels.org/project/protobuf/
Version: all

All protobuf releases on piwheels are pure python (platform tag is any). Native implementation is much faster and would be nice to have. Manual build using

python3 setup.py bdist_wheel --cpp_implementation

works fine on Raspberry Pi 3 and produces

protobuf-3.6.1-cp35-cp35m-linux_armv7l.whl

file.

For example, gRPC library already has armv6l and amrv7l wheels —https://www.piwheels.org/simple/grpcio/

[bennuttall]

Thanks for the info. Do you know if it's possible to pass the --cpp_implementation flag into the pip wheel command rather than setup.py? i.e pip3 wheel protobuf --cpp_implementation but that flag doesn't work there - so is there a way to pass it in?

I have a script for building wheels manually (outside of the automated piwheels system) of every version of a given package, and if I can modify it that will work well, otherwise I'll have to download the tarballs of each release and run setup.py.

[dmitriykovalev]

Thank you for looking into this!

I've just checked protobuf-3.6.1.tar.gz archive from https://pypi.org/project/protobuf/#files. It doesn't contain any native source code. Looks like the only way is to build it manually from GitHub sources using appropriate version tag.

There is a protobuf_optimized_pip.sh script which builds wheels for PyPI. It is required to use both --cpp_implementation and --compile_static_extension to build self-contained (statically linked .so) wheel. And protobuf library itself needs to be compiled manually before building the wheel:

./autogen.sh
CXXFLAGS="-fPIC -g -O2" ./configure
make -j8

This shell script also contains an URL template to download source code:

curl -SsL -O https://github.com/protocolbuffers/protobuf/archive/v${PROTOBUF_VERSION}.tar.gz

For example, https://github.com/protocolbuffers/protobuf/archive/v3.6.1.tar.gz seems to be working fine.

[dmitriykovalev]

@bennuttall Please let me know if you need more information or any other help! Would be really nice to install native protobuf from piwheels. This is a huge performance win on Pi Zero.

[bennuttall]

If the C++ implementation isn't what the maintainers provide on PyPI, I'm not sure we should on piwheels. If they/someone registered protobuf-cpp on PyPI and pushed an sdist targeting the C++ implementation, we could pick that up. I suggest you raise it with the maintainers, or pursue it yourself. What do you think?