nrnpyenv.sh determines PYTHONHOME more robustly.

[nrnhines]

Uses sys.base_prefix whenever possible

Print the provenance of NRN_PYLIB.

On Linux, try to figure out the NRN_PYLIB using sysconfig LIBDIR but
search only in LIBDIR.
Additional method using h.libpython_path()

This only partially is an alternative to #560 and partially responds to #564.
Note that on my desktop, when using the system python3 I now (still) see:

$ nrnpyenv.sh /usr/bin/python3
...
#NRN_PYLIB provenance: search based on os.__file__, found one with version in name

# if launch nrniv, then likely need:
export PYTHONHOME="/usr"
export NRN_PYLIB="/usr/lib/python3.6/config-3.6m-x86_64-linux-gnu/libpython3.6m.so"

Selected interesting sysconfig variables are from:

from distutils import sysconfig as s
z = s.get_config_vars()
for i in z.items():
  if 'lib' in str(i[1]):
    print (i)

('BINLIBDEST', '/usr/lib/python3.6')
('DESTDIRS', '/usr /usr/lib /usr/lib/python3.6 /usr/lib/python3.6/lib-dynload')
('DESTLIB', '/usr/lib/python3.6')
('DESTSHARED', '/usr/lib/python3.6/lib-dynload')
('DYNLOADFILE', 'dynload_shlib.o')
('INSTSONAME', 'libpython3.6m.so.1.0')
('LDLIBRARY', 'libpython3.6m.so')
('LIBDEST', '/usr/lib/python3.6')
('LIBDIR', '/usr/lib')
('LIBPC', '/usr/lib/x86_64-linux-gnu/pkgconfig')
('LIBPL', '/usr/lib/python3.6/config-3.6m-x86_64-linux-gnu')
('LIBRARY', 'libpython3.6m.a')

So perhaps the folder search can be usefully elaborated beyond LIBDIR since in this situation we are falling back finally to a full 'find' starting from os.file/../.. which is LIBDIR

For now I have left in the h.libpython_path() stuff for linux and mac, though it has been giving disappointing results. (it seems that a lot of python installations have libpython statically linked to the executable).

[pramodk]

@nrnhines : MacOS travis build with following CMake like:

cmake /Users/travis/build/neuronsimulator/nrn \
-DCMAKE_INSTALL_PREFIX=/Users/travis/build/neuronsimulator/nrn/build/cmake_install \
-DPYTHON_EXECUTABLE=/Users/travis/build/neuronsimulator/nrn/nrn_build_venv37/bin/python \
-DCMAKE_BUILD_TYPE=Release \
-DNRN_ENABLE_INTERNAL_READLINE=ON \
-DNRN_ENABLE_CORENEURON=OFF \
-DNRN_ENABLE_INTERVIEWS=ON \
-DIV_ENABLE_X11_DYNAMIC=ON \
-DNRN_ENABLE_RX3D=OFF \
-DNRN_ENABLE_MPI=ON \
-DNRN_ENABLE_MPI_DYNAMIC=ON \
-DNRN_ENABLE_PYTHON_DYNAMIC=ON \
-DNRN_ENABLE_MODULE_INSTALL=OFF \
-DNRN_ENABLE_REL_RPATH=ON \
-DLINK_AGAINST_PYTHON=OFF \
-DNRN_MPI_DYNAMIC=/usr/local/opt/openmpi/include;/usr/local/opt/mpich/include

fails with:

+./x86_64/special -python -c 'import neuron; neuron.test(); neuron.test_rxd(); quit()'
4307NEURON -- VERSION 8.0.dev-116-g65f4c27e HEAD (65f4c27e) 2020-06-09
4308Duke, Yale, and the BlueBrain Project -- Copyright 1984-2019
4309See http://neuron.yale.edu/neuron/credits
4310
4311loading membrane mechanisms from /Users/travis/build/neuronsimulator/nrn/x86_64/.libs/libnrnmech.so
4312Additional mechanisms from files
4313 tmp_mod/gap.mod
4314Traceback (most recent call last):
4315  File "<stdin>", line 314, in <module>
4316TypeError: can only concatenate str (not "NoneType") to str
4317Python not available

build log here . Does this work locally on your Mac?

One Azure CI on MacOS while using testing wheel with brew installed python, we see here:

+ ./x86_64/special -python -c 'import neuron; neuron.test(); neuron.test_rxd(); quit()'
Warning: no DISPLAY environment variable.
--No graphics will be displayed.
NEURON -- VERSION 8.0.dev-116-g65f4c27e HEAD (65f4c27e) 2020-06-09
Duke, Yale, and the BlueBrain Project -- Copyright 1984-2019
See http://neuron.yale.edu/neuron/credits

Additional mechanisms from files
 tmp_mod/gap.mod
Traceback (most recent call last):
  File "<stdin>", line 314, in <module>
TypeError: can only concatenate str (not "NoneType") to str
Python not available

[pramodk]

@nrnhines : I build the wheel locally and tested it on neurosim.downstate.edu:

With Master:

(conda_env) kumbhar@ag:~$ pip install neuron
Collecting neuron
  Using cached https://files.pythonhosted.org/packages/34/e5/39cd59103d28a7f638b594ef8fd9767f19ba68b9785d5d4b2cf1eeee429e/NEURON-7.8.0.127-cp37-cp37m-manylinux1_x86_64.whl
Requirement already satisfied: numpy>=1.9.3 in ./conda_env/lib/python3.7/site-packages (from neuron) (1.18.4)
Installing collected packages: neuron
Successfully installed neuron-7.8.0.127
WARNING: You are using pip version 19.2.3, however version 20.1.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

(conda_env) kumbhar@ag:~$ time nrniv -python -c "print ('hello'); quit();"

Warning: no DISPLAY environment variable.
--No graphics will be displayed.
NEURON -- VERSION 7.8.0-127-g91f6e623+ real_master (91f6e623+) 2020-05-08
Duke, Yale, and the BlueBrain Project -- Copyright 1984-2019
See http://neuron.yale.edu/neuron/credits

hello

real	0m14.872s
user	0m0.884s
sys	0m1.969s

With This PR

(conda_env) kumbhar@ag:~$ pip install /u/kumbhar/wheelhouse/NEURON-8.0.dev115-cp37-cp37m-manylinux1_x86_64.whl
Processing ./wheelhouse/NEURON-8.0.dev115-cp37-cp37m-manylinux1_x86_64.whl
Requirement already satisfied: numpy>=1.9.3 in ./conda_env/lib/python3.7/site-packages (from NEURON==8.0.dev115) (1.18.4)
Installing collected packages: NEURON
Successfully installed NEURON-8.0.dev115
WARNING: You are using pip version 19.2.3, however version 20.1.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
(conda_env) kumbhar@ag:~$ time nrniv -python -c "print ('hello'); quit();"
Warning: no DISPLAY environment variable.
--No graphics will be displayed.
NEURON -- VERSION 8.0.dev-115-gc79a52c2+ nrnpyenv-slow (c79a52c2+) 2020-06-09
Duke, Yale, and the BlueBrain Project -- Copyright 1984-2019
See http://neuron.yale.edu/neuron/credits

hello

real	0m0.737s
user	0m0.399s
sys	0m0.066s

So the solution is working fine! We should just see what's going on with OSX with python dynamic.

@wwlytton : I have stored wheels in following directory that you can try to install and see if slow startup problem is resolved:

(conda_env) kumbhar@ag:~$ ls -l /u/kumbhar/wheelhouse/
total 34164
-rw-r--r-- 1 kumbhar lab 8734641 Jun  9 16:54 NEURON-8.0.dev115-cp35-cp35m-manylinux1_x86_64.whl
-rw-r--r-- 1 kumbhar lab 8790441 Jun  9 16:54 NEURON-8.0.dev115-cp36-cp36m-manylinux1_x86_64.whl
-rw-r--r-- 1 kumbhar lab 8793002 Jun  9 16:54 NEURON-8.0.dev115-cp37-cp37m-manylinux1_x86_64.whl
-rw-r--r-- 1 kumbhar lab 8657973 Jun  9 16:54 NEURON-8.0.dev115-cp38-cp38-manylinux1_x86_64.whl

[pramodk]

Good to merge!