Skip to content

Commit

Permalink
Fixes hostpython build with macOS venv
Browse files Browse the repository at this point in the history
Running the freshly compiled host python binary from another Python
process (e.g. sh module or os.system()) may get wrongly detected as
if it was ran directly from the venv.
This happens when `pyvenv.cfg` is present (e.g. venv/pyvenv.cfg).
It makes `sysconfig.is_python_build()` returns `False` instead of `True`
since it's using the compiled interpreter (e.g. ./build-dir/python).
https://docs.python.org/3/library/sysconfig.html#sysconfig.is_python_build
https://github.com/python/cpython/blob/v3.8.2/Lib/sysconfig.py#L127
This is because the `sys._home` attribute is used during the detection.
The issue was first seen in macOS venv which generates the `pyvenv.cfg`.
To compare both behaviours, try the following within and without a venv:
```python
import os
os.system("./build-dir/python -E -c 'import sysconfig; print(sysconfig._sys_home)'")
```
One would return `/usr/local/bin` and the other `None`

Refs:
- kivy/kivy-ios#401
- #2063
  • Loading branch information
AndreMiras committed Apr 29, 2020
1 parent e2b9cfe commit b8f300f
Show file tree
Hide file tree
Showing 6 changed files with 31 additions and 9 deletions.
2 changes: 1 addition & 1 deletion .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ jobs:
# installs java 1.8, android's SDK/NDK and p4a
- make -f ci/makefiles/osx.mk
- export JAVA_HOME=/Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Home
script: make testapps-no-venv/armeabi-v7a
script: make testapps/armeabi-v7a
- <<: *testapps
name: Rebuild updated recipes
script: travis_wait 30 make docker/run/make/rebuild_updated_recipes
Expand Down
8 changes: 0 additions & 8 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -47,14 +47,6 @@ testapps/%: virtualenv
python setup.py apk --sdk-dir $(ANDROID_SDK_HOME) --ndk-dir $(ANDROID_NDK_HOME) \
--arch=$($@_APP_ARCH)

testapps-no-venv/%:
pip3 install Cython==0.28.6
pip3 install -e .
$(eval $@_APP_ARCH := $(shell basename $*))
cd testapps/on_device_unit_tests/ && \
python3 setup.py apk --sdk-dir $(ANDROID_SDK_HOME) --ndk-dir $(ANDROID_NDK_HOME) \
--arch=$($@_APP_ARCH)

clean:
find . -type d -name "__pycache__" -exec rm -r {} +
find . -type d -name "*.egg-info" -exec rm -r {} +
Expand Down
2 changes: 2 additions & 0 deletions pythonforandroid/recipes/hostpython3/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,8 @@ class Hostpython3Recipe(Recipe):
'''The default url to download our host python recipe. This url will
change depending on the python version set in attribute :attr:`version`.'''

patches = ('patches/pyconfig_detection.patch',)

@property
def _exe_name(self):
'''
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
diff -Nru Python-3.8.2/Lib/site.py Python-3.8.2-new/Lib/site.py
--- Python-3.8.2/Lib/site.py 2020-04-28 12:48:38.000000000 -0700
+++ Python-3.8.2-new/Lib/site.py 2020-04-28 12:52:46.000000000 -0700
@@ -487,7 +487,8 @@
if key == 'include-system-site-packages':
system_site = value.lower()
elif key == 'home':
- sys._home = value
+ # this is breaking pyconfig.h path detection with venv
+ print('Ignoring "sys._home = value" override')

sys.prefix = sys.exec_prefix = site_prefix

2 changes: 2 additions & 0 deletions pythonforandroid/recipes/python3/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,8 @@ class Python3Recipe(TargetPythonRecipe):
name = 'python3'

patches = [
('patches/pyconfig_detection.patch'),

# Python 3.7.1
('patches/py3.7.1_fix-ctypes-util-find-library.patch', version_starts_with("3.7")),
('patches/py3.7.1_fix-zlib-version.patch', version_starts_with("3.7")),
Expand Down
13 changes: 13 additions & 0 deletions pythonforandroid/recipes/python3/patches/pyconfig_detection.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
diff -Nru Python-3.8.2/Lib/site.py Python-3.8.2-new/Lib/site.py
--- Python-3.8.2/Lib/site.py 2020-04-28 12:48:38.000000000 -0700
+++ Python-3.8.2-new/Lib/site.py 2020-04-28 12:52:46.000000000 -0700
@@ -487,7 +487,8 @@
if key == 'include-system-site-packages':
system_site = value.lower()
elif key == 'home':
- sys._home = value
+ # this is breaking pyconfig.h path detection with venv
+ print('Ignoring "sys._home = value" override')

sys.prefix = sys.exec_prefix = site_prefix

0 comments on commit b8f300f

Please sign in to comment.