Skip to content

Commit

Permalink
bindings: python: add a script to generate 32-bit ARM (armv7l) wheels
Browse files Browse the repository at this point in the history
Add a 'generate_armv7l_wheels.sh' shell script that makes armv7l (32-bit
ARM) Python wheels for combinations of libc (glibc, musl) and selected
CPython versions.

Currently, Python wheels are generated for the x86_64 and aarch64 CPU
architectures only, using the 'cibuildwheel' tool invoked by the existing
'generate_pypi_artifacts.sh' script. The 'cibuildwheel' tool, in turn,
relies on the PyPA manylinux and musllinux Docker images that, sadly, are
not available for 32-bit ARM architectures that are relatively common in
edge devices.

It was previously suggested that Python users could rely on the
piwheels.org repository for 32-bit ARM gpiod wheels. Alas, that repository
lags seriously behind the latest CPython releases. It currently provides
gpiod wheels for CPython versions 3.9 and 3.11 only, while CPython 3.12
was released 11 months ago and CPython 3.13 will be released in about 1
month's time.

It turns out that armv7l Python wheels can be relatively easily generated
without 'cibuildwheel', in about 150 lines of structured shell script
that introduce no new requirements (the script depends on Docker and
binfmt_misc only). Instead of manylinux and musllinux Docker images, the
script uses the official Python image in Docker Hub, which is available
for armv7l in Debian and Alpine flavours (for glibc and musl libc
respectively). This commit introduces such a shell script.

Signed-off-by: Paulo Ferreira de Castro <[email protected]>
  • Loading branch information
pdcastro committed Sep 2, 2024
1 parent c701785 commit 6db5bd1
Showing 1 changed file with 153 additions and 0 deletions.
153 changes: 153 additions & 0 deletions bindings/python/generate_armv7l_wheels.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,153 @@
#!/usr/bin/env bash
# SPDX-License-Identifier: GPL-2.0-or-later
# SPDX-FileCopyrightText: 2024 Paulo Ferreira de Castro <[email protected]>
#
# This script makes armv7l gpiod Python wheels for combinations of libc (glibc
# and musl) and CPython versions, complementing the wheels made by cibuildwheel
# (through the 'generate_pypi_artifacts.sh' script).
#
# Modify the TARGET_PYTHON_VERSIONS global variable below in order to change
# the targeted CPython versions.
#
# Dependencies: Docker and binfmt_misc for ARM emulation.
#
# Usage:
# ./generate_armv7l_wheels.sh
#
# The wheels will be placed in the ./dist directory.
#

TARGET_PYTHON_VERSIONS=(3.9 3.10 3.11 3.12)
CPU_ARCH='armv7l'
DOCKER_PLATFORM='linux/arm/v7'
ALPINE_DEPENDENCIES="
RUN apk add autoconf autoconf-archive automake bash binutils curl \
g++ git libtool linux-headers make patchelf pkgconfig
"
DEBIAN_DEPENDENCIES="
RUN apt-get update && apt-get install -y autoconf-archive
# auditwheel requires patchelf v0.14+, but Debian Bullseye comes with v0.12.
RUN curl -Ls 'https://github.com/NixOS/patchelf/releases/download/0.18.0/patchelf-0.18.0-${CPU_ARCH}.tar.gz' \
| tar -xzC /usr ./bin/patchelf
"
# Docker image used to run Python commands locally (host CPU architecture).
NATIVE_PYTHON_IMG="python:3.12-alpine"

quit() {
echo -e "\n${1}"
exit 1
}

print_gpiod_version() {
# Print the gpiod Python binding version from gpiod/version.py.
set -x
docker run --rm -iv "${PWD}/gpiod:/gpiod" "${NATIVE_PYTHON_IMG}" python <<-EOF
import sys
sys.path.insert(0, "/gpiod")
from version import __version__
print(__version__)
EOF
{ local status="$?"; set +x; } 2>/dev/null
return "${status}"
}

ensure_sdist() {
# Make the sdist if it does not already exist in the './dist' directory.
local gpiod_version
gpiod_version="$(print_gpiod_version)" || \
quit "Failed to determine the gpiod Python binding version"

SDIST_TARBALL="gpiod-${gpiod_version}.tar.gz"
if [ -f "dist/${SDIST_TARBALL}" ]; then
return
fi
if [ -z "${LIBGPIOD_VERSION}" ]; then
quit "Please set the LIBGPIOD_VERSION env var to enable making the sdist."
fi
set -x
docker run --rm -v "${PWD}:/py_bindings" -w /py_bindings -e LIBGPIOD_VERSION \
"${NATIVE_PYTHON_IMG}" python setup.py sdist
{ local status="$?"; set +x; } 2>/dev/null
return "${status}"
}

make_wheel_builder_image() {
# Make a "wheel builder" docker image used to build gpiod wheels.
# The base image is the official Python image on Docker Hub, in either the
# Debian variant (glibc) or the Alpine variant (musl). Currently using
# Debian 11 Bullseye because it supports building wheels targeting glibc
# v2.17. The newer Debian 12 Bookworm supports higher glibc versions only.

local libc="$1" # The string 'glibc' or 'musl'
local builder_img_tag="$2"
local python_version="$3"
local distro=
local distro_deps=
case "${libc}" in
glibc) distro="bullseye"; distro_deps="${DEBIAN_DEPENDENCIES}";;
musl) distro="alpine3.20"; distro_deps="${ALPINE_DEPENDENCIES}";;
esac
set -x
docker build --platform "${DOCKER_PLATFORM}" --pull --progress plain \
--tag "${builder_img_tag}" --file - . <<-EOF
FROM 'python:${python_version}-${distro}'
${distro_deps}
RUN pip install auditwheel
EOF
{ local status="$?"; set +x; } 2>/dev/null
return "${status}"
}

build_wheel() {
# Build a gpiod wheel in a container of the given builder image tag.
local libc="$1" # The string 'glibc' or 'musl'
local builder_img_tag="$2"
local libc_plat=
case "${libc}" in
glibc) libc_plat="manylinux_2_17_${CPU_ARCH}";;
musl) libc_plat="musllinux_1_2_${CPU_ARCH}";;
esac
set -x
docker run --rm --platform "${DOCKER_PLATFORM}" -iv "${PWD}/dist:/dist" \
-w /tmp "${builder_img_tag}" bash <<-EOF
set -ex
pip wheel --no-deps "/dist/${SDIST_TARBALL}"
auditwheel repair --plat "${libc_plat}" --wheel-dir /dist ./*.whl
EOF
{ local status="$?"; set +x; } 2>/dev/null
return "${status}"
}

build_combinations() {
# Build gpiod wheels for combinations of libc and Python versions.
local builder_img_tag
for libc in glibc musl; do
for ver in "${TARGET_PYTHON_VERSIONS[@]}"; do
builder_img_tag="gpiod-wheel-builder-cp${ver/./}-${libc}-${CPU_ARCH}"

make_wheel_builder_image "${libc}" "${builder_img_tag}" "${ver}" || \
quit "Failed to build Docker image '${builder_img_tag}'"

build_wheel "${libc}" "${builder_img_tag}" || \
quit "Failed to build gpiod wheel with image '${builder_img_tag}'"

set -x
docker image rm "${builder_img_tag}"
{ set +x; } 2>/dev/null
done
done
}

main() {
set -e
local script_dir # Directory where this script is located
script_dir="$(cd -- "$(dirname "$0")" >/dev/null 2>&1 || true; pwd -P)"
cd "${script_dir}" || quit "'cd ${script_dir}' failed"
ensure_sdist || quit "Failed to find or make the sdist tarball."
build_combinations
echo
ls -l dist/*"${CPU_ARCH}"*
echo -e "\nAll done! Hint: Use 'docker system prune' to free disk space."
}

main

0 comments on commit 6db5bd1

Please sign in to comment.