Python 3.6.3 or 3.6.4

[kuri65536]

2017/10/03 https://www.python.org/downloads/release/python-363/

[kuri65536]

some works remains before release r33.

• need to check setlocale problem.
• try to fix readline problem readline not work with Python 3 #144
• try to fix not be able to set PYTHONHOME in python3 binary. #88

[kuri65536]

Python 3.6.4 was released, Release Date: 2017-12-19

[kuri65536]

C extension update.

Python 3.6.4 can't load C extension modules with dlopen error.
Because C extension modules can't refer to libpython3.so symbols.

I tried several methods and not much success.

• can't resolve with sys.setdlopenflags(os.RTLD_LAZY or GLOBAL)
• can't resolve with including libpython3.6m.so into release.zip
• can't resolve with workaround distutils.sysconfig.EXT_SUFFIX in setup.py
• didnot try to resolve with old paching to _sysconfigdata's EXT_SUFFIX.
• didnot try to resolve with "-lpython3.6m" option to ldflags: kivy/python-for-android@ed1e1a8

[Wryhder]

Hello @kuri65536
Could you please take a look at my python.sh?
I've been trying to set up Python 3 in Terminal IDE, and have been successful to an extent.
I'm getting this:

@bash:~$ py
Could not find platform independent libraries <prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Python 3.6.4 (default, Jan  9 2018, 02:56:19)
[GCC 4.9 20140827 (prerelease)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

Contents of python3.sh:

@bash:~$ cat ./python3.sh
#! /system/bin/sh
st=`for i in /mnt/storage /mnt/sdcard /sdcard; do [ -d $i ] && echo $i && break; done`
if [ x$api != x ]; then :
elif ! type \cut > /dev/null; then api=14; else
    api=`grep build.version.sdk /system/build.prop | cut -d = -f 2`
fi

# android.Android() will launch SL4A server, if server down.

PW=`pwd`
ver=3.6
py4a=python3forandroid
binr=python3
export EXTERNAL_STORAGE=$st
export LANG=en
bin=/data/data/com.googlecode.$py4a/files/$binr
pfx=$st/com.googlecode.$py4a
ext=$pfx/extras/$binr
# normal (pure python modules only)
PYTHONUSERBASE=$pfx/local
# to use C extension modules (with shared lib)
# PYTHONUSERBASE=/data/local/tmp/local
PYTHONPATH=$ext
PYTHONPATH=${PYTHONPATH}:$PYTHONUSERBASE
PYTHONPATH=${PYTHONPATH}:$bin/lib/python$ver/lib-dynload
export PYTHONPATH
export PYTHONUSERBASE
export TEMP=$ext/tmp
export PYTHON_EGG_CACHE=$TEMP
# do not use this for Python3.4, see issue tracker.
# export PYTHONHOME=$ext:$bin
export LD_LIBRARY_PATH=$bin/lib
cd $PW
if [ $api -lt 14 ]; then
    run=/data/data/com.googlecode.android_scripting/files/run_pie
    if ! [ -x $run ]; then
        echo "need root permission to launch run_pie/python"
        ls -l "$run"
        # su -c "chmod 755 $run"
        su -c "$run $bin/bin/python3 $*"
    else
        $run $bin/bin/python3 $*
    fi
else
    $bin/bin/python3 $*
fi

@bash:~$

I installed the SL4A and Python for Android apps through the links provided in the README for this repo.
The interpreter was installed from inside the Py4a app.
This is what my /sdcard/com.googlecode.python3forandroid looks like:

Initially I was getting a Segmentation Fault (could not initialize sys) error until I found out in issue #138 that I wasn't supposed to set the PYTHONHOME variable.
I've made sure to add all the important paths in my .bashrc, in case I missed something in python.sh.

@bash:~$ cat .bashrc
...
export EXTERNAL_STORAGE=/mnt/sdcard/com.googlecode.python3forandroid
export PY34A=/data/data/com.googlecode.python3forandroid/files/python3
export PY34A_EXTRAS=$EXTERNAL_STORAGE/extras
PYTHONPATH=$PY34A_EXTRAS/python3
PYTHONPATH=$PY34A/lib/python3.6/lib-dynload:${PYTHONPATH}
# export PYTHONPATH
export TEMP2=$EXTERNAL_STORAGE/extras/python3/tmp
export PYTHON_EGG_CACHE=$TEMP
# export PYTHONHOME=$PY34A
export LD_LIBRARY_PATH=$PY34A/lib
export PYTHONPATH=${PYTHONPATH}:${LD_LIBRARY_PATH}
# export PYTHONHOME=/data/data/com.googlecode.python3forandroid/files/python3
# export PYTHONPATH=/sdcard/MyProjects/Python:${PYTHONHOME}/lib/python3.6/lib-dynload
# export LD_LIBRARY_PATH=${PYTHONHOME}/lib
...
@bash:~$

I'm using a Samsung Galaxy Note 10.1", running 4.1.2.

[kuri65536]

Thank you use python for android!
and you are right: PYTHONHOME is not acceptable by my python3.6 build now.

I confirmed your script and several difference from my python3.sh.

• mistake?) EXTERNAL_STORAGE=/mnt ... /mnt is not need in your environment (?)
• lack of PYTHONUSERBASE,
  need for pip, dynamic libraries can not locate in SDCARD by security reason,
  so you need to point the userbase in /data/data or system directory.

for long time, I did not touch the python and I forgot a lot of thing about it.
I wish this answer made your help.

[Wryhder]

Thanks a bunch for responding so quickly!
I tried removing /mnt from python3.sh and got this seg fault, so I promptly put it back:

@bash:~$ bash python3.sh
Could not find platform independent libraries <prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Fatal Python error: Py_Initialize: unable to load the file system codec
ModuleNotFoundError: No module named 'encodings'

Current thread 0x40023f30 (most recent call first):
Segmentation Fault

Could you explain the PYTHONUSERBASE a bit more? I actually just realised that the path it is set to in python3.sh does not exist.
Pip is in /sdcard/com.googlecode.python3forandroid/extras/ensurepip, yes? I'm to move it to a system directory? The whole extras folder?
I don't have root permissions, so I'm doing most of the work inside Terminal IDE's data directory.
I run python3.sh from there.
Should I just move the whole /sdcard/com.googlecode.python3forandroid into Terminal IDE's data directory?

Also, dynamic libraries would be the .../lib-dynload folder, right? That's already in a system directory and is exported as part of the PYTHONPATH variable.

Thanks very much for your help.

[kuri65536]

Hmmm, problem is not solved yet...

At now, please forget PYTHONUSERBASE, this is not needed now.
(this is for users packages, see PEP 370)

Sometime, I saw ModuleNotFoundError: No module named 'encodings' error as you, too.
encodings is pure python module and is located in extras directory.
but your PYTHONPATH point to extras/python3 already, this is strange...

You can get more information from

• python -v
• dump actual PYTHONPATH by echo $PYTHONPATH before python running.
• logcat at crash

[Wryhder]

Thanks!

echo $PYTHONPATH before running python gives this:

@bash:~$ echo $PYTHONPATH
/data/data/com.googlecode.python3forandroid/files/python3/lib/python3.6/lib-dynload:/sdcard/com.googlecode.python3forandroid/extras/python3:/data/data/com.googlecode.python3forandroid/files/python3/lib
@bash:~$

Output of ./python3.sh -v:

@bash:~$ ./python3.sh -v
Could not find platform independent libraries <prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
import _frozen_importlib # frozen
import _imp # builtin
import sys # builtin
import '_warnings' # <class '_frozen_importlib.BuiltinImporter'>
import '_thread' # <class '_frozen_importlib.BuiltinImporter'>
import '_weakref' # <class '_frozen_importlib.BuiltinImporter'>
import '_frozen_importlib_external' # <class '_frozen_importlib.FrozenImporter'>
import '_io' # <class '_frozen_importlib.BuiltinImporter'>
import 'marshal' # <class '_frozen_importlib.BuiltinImporter'>
import 'posix' # <class '_frozen_importlib.BuiltinImporter'>
import _thread # previously loaded ('_thread')
import '_thread' # <class '_frozen_importlib.BuiltinImporter'>
import _weakref # previously loaded ('_weakref')
import '_weakref' # <class '_frozen_importlib.BuiltinImporter'>
# installing zipimport hook
import 'zipimport' # <class '_frozen_importlib.BuiltinImporter'>
# installed zipimport hook
# code object from '/mnt/sdcard/com.googlecode.python3forandroid/extras/python3/encodings/__init__.pyc'
# code object from '/mnt/sdcard/com.googlecode.python3forandroid/extras/python3/codecs.pyc'
import '_codecs' # <class '_frozen_importlib.BuiltinImporter'>
import 'codecs' # <_frozen_importlib_external.SourcelessFileLoader object at 0x403e9a50>
# code object from '/mnt/sdcard/com.googlecode.python3forandroid/extras/python3/encodings/aliases.pyc'
import 'encodings.aliases' # <_frozen_importlib_external.SourcelessFileLoader object at 0x403f57b0>
import 'encodings' # <_frozen_importlib_external.SourcelessFileLoader object at 0x403e9770>
# code object from '/mnt/sdcard/com.googlecode.python3forandroid/extras/python3/encodings/utf_8.pyc'
import 'encodings.utf_8' # <_frozen_importlib_external.SourcelessFileLoader object at 0x404001b0>
import '_signal' # <class '_frozen_importlib.BuiltinImporter'>
# code object from '/mnt/sdcard/com.googlecode.python3forandroid/extras/python3/encodings/latin_1.pyc'
import 'encodings.latin_1' # <_frozen_importlib_external.SourcelessFileLoader object at 0x40400770>
# code object from '/mnt/sdcard/com.googlecode.python3forandroid/extras/python3/io.pyc'
# code object from '/mnt/sdcard/com.googlecode.python3forandroid/extras/python3/abc.pyc'
# code object from '/mnt/sdcard/com.googlecode.python3forandroid/extras/python3/_weakrefset.pyc'
import '_weakrefset' # <_frozen_importlib_external.SourcelessFileLoader object at 0x40400ed0>
import 'abc' # <_frozen_importlib_external.SourcelessFileLoader object at 0x404009d0>
import 'io' # <_frozen_importlib_external.SourcelessFileLoader object at 0x40400830>
# code object from '/mnt/sdcard/com.googlecode.python3forandroid/extras/python3/_bootlocale.pyc'
# destroy _bootlocale
# code object from '/mnt/sdcard/com.googlecode.python3forandroid/extras/python3/encodings/ascii.pyc'
import 'encodings.ascii' # <_frozen_importlib_external.SourcelessFileLoader object at 0x4040adf0>
# code object from '/mnt/sdcard/com.googlecode.python3forandroid/extras/python3/_bootlocale.pyc'
# destroy _bootlocale
# code object from '/mnt/sdcard/com.googlecode.python3forandroid/extras/python3/_bootlocale.pyc'
# destroy _bootlocale
# code object from '/mnt/sdcard/com.googlecode.python3forandroid/extras/python3/site.pyc'
# code object from '/mnt/sdcard/com.googlecode.python3forandroid/extras/python3/os.pyc'
import 'errno' # <class '_frozen_importlib.BuiltinImporter'>
# code object from '/mnt/sdcard/com.googlecode.python3forandroid/extras/python3/stat.pyc'
import '_stat' # <class '_frozen_importlib.BuiltinImporter'>
import 'stat' # <_frozen_importlib_external.SourcelessFileLoader object at 0x40420dd0>
# code object from '/mnt/sdcard/com.googlecode.python3forandroid/extras/python3/posixpath.pyc'
# code object from '/mnt/sdcard/com.googlecode.python3forandroid/extras/python3/genericpath.pyc'
import 'genericpath' # <_frozen_importlib_external.SourcelessFileLoader object at 0x40424cd0>
import 'posixpath' # <_frozen_importlib_external.SourcelessFileLoader object at 0x40420ef0>
# code object from '/mnt/sdcard/com.googlecode.python3forandroid/extras/python3/_collections_abc.pyc'
import '_collections_abc' # <_frozen_importlib_external.SourcelessFileLoader object at 0x4042d0f0>
import 'os' # <_frozen_importlib_external.SourcelessFileLoader object at 0x40418ad0>
# code object from '/mnt/sdcard/com.googlecode.python3forandroid/extras/python3/_sitebuiltins.pyc'
import '_sitebuiltins' # <_frozen_importlib_external.SourcelessFileLoader object at 0x40418d50>
# code object from '/mnt/sdcard/com.googlecode.python3forandroid/extras/python3/sysconfig.pyc'
import 'sysconfig' # <_frozen_importlib_external.SourcelessFileLoader object at 0x4044b770>
# code object from /data/data/com.googlecode.python3forandroid/files/python3/lib/python3.6/lib-dynload/_sysconfigdata_m_linux_arm-linux-androideabi.py
# could not create '/data/data/com.googlecode.python3forandroid/files/python3/lib/python3.6/lib-dynload/__pycache__': PermissionError(13, 'Permission denied')
# wrote '/data/data/com.googlecode.python3forandroid/files/python3/lib/python3.6/lib-dynload/__pycache__/_sysconfigdata_m_linux_arm-linux-androideabi.cpython-36.pyc'
import '_sysconfigdata_m_linux_arm-linux-androideabi' # <_frozen_importlib_external.SourceFileLoader object at 0x4045a6b0>
import 'site' # <_frozen_importlib_external.SourcelessFileLoader object at 0x4040af10>
Python 3.6.4 (default, Jan  9 2018, 02:56:19)
[GCC 4.9 20140827 (prerelease)] on linux
Type "help", "copyright", "credits" or "license" for more information.
import 'atexit' # <class '_frozen_importlib.BuiltinImporter'>
>>>

I'm not sure about logcat; so far python only crashes when I remove /mnt or set the PYTHONHOME variable.
I'll checkout how to use it later; I thought I'd look at the error output above first. Could you help me make sense of it?

[kuri65536]

logcat: you can use adb logcat output to get python crash information, this is useful for debug.

your .bashrc lack something,
I hope you will find the difference soon.