cross: wrong interpreter returned when no python available

[vfazio]

BPO	44062
Nosy	@moreati, @vfazio
PRs	bpo-44062: Fix PYTHON_FOR_BUILD interpreter lookup for cross-compile … #25951

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2021-05-06.18:23:43.293>
labels = ['build', '3.9', '3.10', '3.11']
title = 'cross: wrong interpreter returned when no python available'
updated_at = <Date 2021-05-06.19:06:58.698>
user = 'https://github.com/vfazio'

bugs.python.org fields:

activity = <Date 2021-05-06.19:06:58.698>
actor = 'vfazio'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Cross-Build']
creation = <Date 2021-05-06.18:23:43.293>
creator = 'vfazio'
dependencies = []
files = []
hgrepos = []
issue_num = 44062
keywords = ['patch']
message_count = 1.0
messages = ['393125']
nosy_count = 2.0
nosy_names = ['Alex.Willmer', 'vfazio']
pr_nums = ['25951']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'compile error'
url = 'https://bugs.python.org/issue44062'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

[vfazio]

When trying to cross compile python3.9, configure attempts to find a strict python 3.9 version match, however if it fails it still attempts to use python in PYTHON_FOR_BUILD instead of failing outright like the code implies it should

$/python3/targetbuild# which python3.9 python3 python
/usr/bin/python3

$/python3/targetbuild# python3 --version
Python 3.7.3

$/python3/targetbuild# PYTHON_FOR_REGEN=/python3/hostbuild/python \
ac_cv_file__dev_ptmx=yes \
ac_cv_file__dev_ptc=no \
ac_cv_buggy_getaddrinfo=no \
../configure --build=x86-linux-gnu \
--host=aarch64-linux-gnu \
--enable-loadable-sqlite-extensions \
--enable-option-checking=fatal \
--enable-shared \
--with-system-expat \
--with-system-ffi \
--without-ensurepip
checking build system type... x86-unknown-linux-gnu
checking host system type... aarch64-unknown-linux-gnu
checking for python3.9... /python3/hostbuild/python
checking for python interpreter for cross build... python
...

$/python3/targetbuild# grep PYTHON_FOR_BUILD config.log
PYTHON_FOR_BUILD='_PYTHON_PROJECT_BASE=$(abs_builddir) _PYTHON_HOST_PLATFORM=$(PYTHON_HOST_PLATFORM) PYTHONPATH=$(shell test -f pybuilddir.txt && echo $(abs_builddir)/cat pybuilddir.txt:)$(srcdir)/Lib PYTHON_SYSCONFIGDATA_NAME=sysconfigdata$(ABIFLAGS)$(MACHDEP)$(MULTIARCH) python'

in configure

if test "$cross_compiling" = yes; then
AC_MSG_CHECKING([for python interpreter for cross build])
if test -z "$PYTHON_FOR_BUILD"; then
for interp in python$PACKAGE_VERSION python3 python; do
which $interp >/dev/null 2>&1 || continue
if $interp -c "import sys;sys.exit(not '.'.join(str(n) for n in sys.version_info@<:@:2@:>@) == '$PACKAGE_VERSION')"; then
break
fi
interp=
done
if test x$interp = x; then
AC_MSG_ERROR([python$PACKAGE_VERSION interpreter not found])
fi
AC_MSG_RESULT($interp)
PYTHON_FOR_BUILD='_PYTHON_PROJECT_BASE=$(abs_builddir) _PYTHON_HOST_PLATFORM=$(PYTHON_HOST_PLATFORM) PYTHONPATH=$(shell test -f pybuilddir.txt && echo $(abs_builddir)/cat pybuilddir.txt:)$(srcdir)/Lib PYTHON_SYSCONFIGDATA_NAME=sysconfigdata$(ABIFLAGS)$(MACHDEP)$(MULTIARCH) '$interp
fi
elif test "$cross_compiling" = maybe; then
AC_MSG_ERROR([Cross compiling required --host=HOST-TUPLE and --build=ARCH])
else
PYTHON_FOR_BUILD='./$(BUILDPYTHON) -E'
fi
AC_SUBST(PYTHON_FOR_BUILD)

The issue is a failing edge case here:

    for interp in python$PACKAGE_VERSION python3 python; do
        which $interp >/dev/null 2>&1 || continue

where interp keeps it's last value doesn't trigger the empty check here:

    if test x$interp = x; then
        AC_MSG_ERROR([python$PACKAGE_VERSION interpreter not found])
    fi

Note that there's an explicit clearing of interp when the python version isn't a match:

            if $interp -c "import sys;sys.exit(not '.'.join(str(n) for n in sys.version_info@<:@:2@:>@) == '$PACKAGE_VERSION')"; then
                break
            fi
            interp=

The fix should be pretty straightforward:

    for interp in python$PACKAGE_VERSION python3 python ''; do

adding '' as the last possible interpreter means one hasn't been found up to that point and allows configure to properly fail

$/python3/targetbuild# PYTHON_FOR_REGEN=/python3/hostbuild/python ac_cv_file__dev_ptmx=yes ac_cv_file__dev_ptc=no ac_cv_buggy_getaddrinfo=no ../configure --build=x86-linux-gnu --host=aarch64-linux-gnu --enable-loadable-sqlite-extensions --enable-option-checking=fatal --enable-shared --with-system-expat --with-system-ffi --without-ensurepip
checking build system type... x86-unknown-linux-gnu
checking host system type... aarch64-unknown-linux-gnu
checking for python3.9... /python3/hostbuild/python
checking for python interpreter for cross build... configure: error: python3.9 interpreter not found

It will continue to work when a proper interpreter is found

$/python3/targetbuild# PATH=/python3/hostbuild:$PATH PYTHON_FOR_REGEN=/python3/hostbuild/python ac_cv_file__dev_ptmx=yes ac_cv_file__dev_ptc=no ac_cv_buggy_getaddrinfo=no ../configure --build=x86-linux-gnu --host=aarch64-linux-gnu --enable-loadable-sqlite-extensions --enable-option-checking=fatal --enable-shared --with-system-expat --with-system-ffi --without-ensurepip
checking build system type... x86-unknown-linux-gnu
checking host system type... aarch64-unknown-linux-gnu
checking for python3.9... /python3/hostbuild/python
checking for python interpreter for cross build... Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
python

This should help highlight any inconsistent environment between configuring and building.

[vfazio]

This should probably be closed in favor of #90039 which was merged in #29835.

It's still an issue for versions prior to 3.11