Installation of neuron7.7 on PizDaint, Jureca and Galileo #525

clupascu · 2020-02-13T07:31:08Z

As Synaptic fitting usecases must be updated to Python3 I need to know if it would be possible to install neuron7.7 (that works with Python3) on PizDaint, Jureca and Galileo. Please let me know how I can use the new modules. Thank you.

neuron 7.7 working on PizDaint
neuron 7.7 working on Jureca
neuron 7.7 working on Galileo

Related task: #533

pramodk · 2020-02-18T14:57:17Z

Order of deployment following but make sure the existing module names remain intact (e.g. one which are already with Python2)

Jureca
Galileo
Piz-Daint

pramodk · 2020-02-25T06:17:33Z

@jorblancoa : this ticket requires python3 neuron module

jorblancoa · 2020-02-28T17:19:11Z

Hi @clupascu

We have installed the new modules in Jureca and Piz-Daint.
Could you please test them and let us know if you encounter any problem?

Jureca

For Booster:

module --force purge all
module use /usr/local/software/jurecabooster/OtherStages
module load Architecture/KNL
module load Stages/2019a
module load Intel ParaStationMPI/5.2.2-1-mt imkl
module load HDF5 Boost Python/3.6.8

module use /p/project/cvsk25/software-deployment/HBP/jureca-booster/26-02-2020/modules/tcl/linux-centos7-haswell

module load  neuron/7.8.0b-serial-python3

For Cluster:

module --force purge all
module use /usr/local/software/jureca/OtherStages
module load Architecture/Haswell
module load Stages/2019a
module load Intel ParaStationMPI/5.2.2-1-mt imkl
module load HDF5 Boost Python/3.6.8

module use /p/project/cvsk25/software-deployment/HBP/jureca-cluster/26-02-2020/modules/tcl/linux-centos7-haswell

module load  neuron/7.8.0b-serial-python3

Piz-Daint

export MODULEPATH=/apps/hbp/ich002/hbp-spack-deployments/modules:$MODULEPATH
module use /apps/hbp/ich002/hbp-spack-deployments/softwares/27-02-2020/install/modules/tcl/cray-cnl7-haswell
module swap PrgEnv-cray PrgEnv-intel
module load daint-mc
module load cray-python/3.6.5.7

module load neuron/7.8.0b/intel-serial-python3

Thanks!

alex4200 · 2020-03-09T10:48:11Z

@clupascu WIll test with python3

clupascu · 2020-03-13T08:38:19Z

@jorblancoa what about Galileo? On Galileo there is already an installation of neuron7.7?

jorblancoa · 2020-03-13T18:09:34Z

@clupascu
@pramodk is going to take care of Galileo. But before deploying everywhere, we want to be sure if the modules are working for you.
If you could test those in Daint or Jureca, that would be great!

Thanks!

clupascu · 2020-03-16T10:11:29Z

@jorblancoa I tried on Jureca, but it seems that the order of the parameters it is not working anymore.

I was using this before

srun ./x86_64/special 'config-exp1.txt' 'exp1.txt' 'ProbGABAAB_EMS_GEPH_g.mod' False True False 3 -mpi -python fitting.py

Any suggestion?

jorblancoa · 2020-03-16T10:22:59Z

Could you post the error message or log file so I can have a look at the issue?

clupascu · 2020-03-16T10:29:03Z

Traceback (most recent call last):
File "fitting.py", line 251, in
fitting(sys.argv[1],sys.argv[2],sys.argv[3],sys.argv[4],sys.argv[5],sys.argv[6],sys.argv[7])
File "fitting.py", line 53, in fitting
singletrace_number = int(singletrace_number)
ValueError: invalid literal for int() with base 10: 'True'

This is the error message. Like the arg 'True' is read instead of the arg 3.

clupascu · 2020-03-16T10:42:51Z

The folder I am working in Jureca is /p/home/jusers/lupascu1/jureca/testneuron+python3/.
You can have a look there.

pramodk · 2020-03-16T12:01:34Z

@clupascu : I am pretty sure NEURON installation is working with Python3:

[kumbhar1@jrl05 ~]$ nrniv -python
NEURON -- VERSION 7.8.0-2-g92a208b+ HEAD (92a208b+) 2019-10-29
Duke, Yale, and the BlueBrain Project -- Copyright 1984-2018
See http://neuron.yale.edu/neuron/credits

loading membrane mechanisms from x86_64/.libs/libnrnmech.so
Additional mechanisms from files

>>>
>>> print "a"
  File "stdin", line 1
    print "a"
            ^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print("a")?
>>> print ("A")
A
>>>

We are not familiar with the fitting.py but I was adding:

    singletrace_number = int(singletrace_number)
    print("----->", singletrace_number)

and I was testing as:

$ srun -p develbooster -A vsk25 -n 1 nrniv 'config-exp1.txt' 'exp1.txt' 'ProbGABAAB_EMS_GEPH_g.mod' False True False 3 -mpi -python fitting.py
srun: job 8109728 queued and waiting for resources
srun: job 8109728 has been allocated resources
NEURON -- VERSION 7.8.0-2-g92a208b+ HEAD (92a208b+) 2019-10-29
Duke, Yale, and the BlueBrain Project -- Copyright 1984-2018
See http://neuron.yale.edu/neuron/credits

Additional mechanisms from files
 netstims.mod ProbGABAAB_EMS_GEPH_g.mod
-----> 3

So I think there is something in your environment or script?

clupascu · 2020-03-16T12:53:26Z

I moved a little bit on. I replaced as suggested

srun ./x86_64/special 'config-exp1.txt' 'exp1.txt' 'ProbGABAAB_EMS_GEPH_g.mod' False True False 3 -mpi -python fitting.py

with

srun nrniv 'config-exp1.txt' 'exp1.txt' 'ProbGABAAB_EMS_GEPH_g.mod' False True False 3 -mpi -python fitting.py

and now I get in the output file this warning

Warning: detected user attempt to enable MPI, but MPI support was disabled at build time.

and in the error file

srun: error: jrc6369: tasks 24-31,33-47: Segmentation fault
srun: error: jrc6369: task 32: Terminated
srun: error: jrc6368: tasks 0-3,6-7,9-10,12,18,20-21,23: Terminated
srun: error: jrc6368: tasks 4-5,8,11,13-17,19,22: Segmentation fault
srun: Force Terminated job step 8109797.0

pramodk · 2020-03-16T13:04:59Z

@clupascu : I didn't mean to replace "srun ./x86_64/special" with "srun nrniv". I was just trying to run your example to see if NEURON works.

You still need to use "./x86_64/special" in order to get your local mod files included. nrniv just includes neuron's default mod files.

clupascu · 2020-03-16T13:11:30Z

With "srun nrniv" I don't have the ValueError: invalid literal for int() with base 10: 'True' error anymore.

clupascu · 2020-03-20T13:41:48Z

@pramodk have you got some time to look into this issue?

clupascu · 2020-03-25T13:08:52Z

I have the same error also on PizDaint.

pramodk · 2020-03-25T13:12:26Z

@clupascu : Haven't looked into this yet. Can you point out directory on Piz-Daint? (in scratch or non-home directory where I can access).

clupascu · 2020-03-25T13:33:38Z

@pramodk you can find the directory on Piz-Daint here /scratch/snx3000/bp000028/testneuron+python3

pramodk · 2020-03-25T21:57:15Z

@clupascu : I was looking at this with @jorblancoa and we couldn't find easily the issue. This is what we did:

Let's start with simple example of Python3 and neuron using pc.runworker:

from neuron import h
pc = h.ParallelContext()

def f(arg):
   id = int(pc.id())
   nhost = int(pc.nhost())
   print ('I am %d of %d'%(id, nhost))
   return arg*arg

pc.runworker()

s = 0
if pc.nhost() == 1:
   for i in range(1, 21):
      s += f(i)
else:
   for i in range(1, 21):
      pc.submit(f, i)
   while pc.working():
      s += pc.pyret()
print (s)

pc.done()
h.quit()

Running with NEURON and Python 3 just works fine:

module use /apps/hbp/ich002/hbp-spack-deployments/softwares/27-02-2020/install/modules/tcl/cray-cnl7-haswell
module load neuron/7.8.0b/intel-python3

bp000174@daint103:~/NRN-37> srun nrniv -python test_worker.py  -mpi
NEURON -- VERSION 7.8.0-2-g92a208b6+ HEAD (92a208b6+) 2019-10-29
Duke, Yale, and the BlueBrain Project -- Copyright 1984-2018
See http://neuron.yale.edu/neuron/credits

2870.0
I am 2 of 8
I am 2 of 8
I am 2 of 8
I am 2 of 8
I am 3 of 8
I am 3 of 8
I am 3 of 8
I am 3 of 8
I am 4 of 8
I am 4 of 8
I am 4 of 8
I am 5 of 8
I am 5 of 8
I am 6 of 8
I am 7 of 8
I am 1 of 8
I am 1 of 8
I am 1 of 8
I am 1 of 8
I am 1 of 8
numprocs=8

So we believe that there is no issue with neuron and python3 installation itself.

It's true that while using your example we were getting errors like below with Python3 but not with Python2:

srun: error: nid00008: tasks 0,7: Segmentation fault (core dumped)

I added import fitness at top of fitting.py file:

import random
import csv
import math
....
import subprocess
import fitness

Which runs program bit further but still see segfault error later during the execution. Could you check by adding import fitness near top?

I suspect the issue exist in the script somewhere (?) that is only becoming visible with Python3.

Do you have neuron installed on your desktop where you can test this? Otherwise, as I am not entirely familiar with the code, I think one needs to test part of the code and see which functions is causing the segfault error.

We can discuss this further tomorrow if required.

clupascu · 2020-03-27T09:19:31Z

@pramodk Can you please let me know what was the parallel version to use instead of module load neuron/7.8.0b-serial-python3 on Jureca? It seems that if I use this parameter order

srun nrniv -python 'configA1.txt' 'expA1.txt' 'ProbGABAAB_EMS_GEPH_g.mod' False True False 3 fitting.py -mpi

Neuron is loaded (but is loaded of course several times).

jorblancoa · 2020-03-27T13:44:34Z

Hi @clupascu
If you load the latest modules (25/03) you can find a parallel neuron.

module use /p/project/cvsk25/software-deployment/HBP/jureca-booster/25-03-2020/modules/tcl/linux-centos7-haswell
module load neuron/7.8.0b

Let me know if you have any problems.

clupascu · 2020-03-31T09:03:47Z

@pramodk and @jorblancoa I modified my code (the segmentation fault was due to one function not available anymore in python3) and my code works perfectly now with the new modules from issue #533 on Jureca booster and PizDaint. Any news on the installation of the same modules on Galileo?

jorblancoa · 2020-03-31T13:01:05Z

Hi @clupascu
Out of curiosity, what was the function not available in python3 causing the core dump?
Regarding the deployment in Galileo, we are finishing latest validations in Daint and Jureca, and once everything is properly tested, we will deploy in Galileo.

clupascu · 2020-03-31T13:26:47Z

@jorblancoa the function not available in python3 causing the core dump was file.

alex4200 · 2020-04-06T12:51:53Z

Hi, can you please tick the boxes in the top-most main comment to mark the cases that are done?

pramodk · 2020-04-07T00:41:40Z

As far as I know, all systems are up to date with neuron and python3. See #533.

clupascu · 2020-04-07T09:41:50Z

I tested neuron7.7 on PizDaint, Jureca and Galileo and everything works perfectly. I am closing this issue.

mmigliore · 2020-04-09T08:43:03Z

Loading the modules works from my account on ich002, but a user on the ich011 group (used by BSP use cases) sees: bp000338@daint105:~/Heart_3D/WholeCellSimulation_gpurk30000_StimUp> module load daint-mc cray-python/3.6.5.7 PyExtensions/3.6.5.7-CrayGNU-19.10 bp000338@daint105:~/Heart_3D/WholeCellSimulation_gpurk30000_StimUp> module use /apps/hbp/ich002/hbp-spack-deployments/softwares/25-03-2020/install/modules/tcl/cray-cnl7-haswell ModuleCmd_Use.c(240):ERROR:64: Directory '/apps/hbp/ich002/hbp-spack-deployments/softwares/25-03-2020/install/modules/tcl/cray-cnl7-haswell' not found bp000338@daint105:~/Heart_3D/WholeCellSimulation_gpurk30000_StimUp> Can you fix this?

…

On Tue, Apr 7, 2020 at 2:41 AM Pramod Kumbhar ***@***.***> wrote: As far as I know, all systems are up to date with neuron and python3. See #533 <#533>. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#525 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGZUP34UZ4W7UYNYB4BFLITRLJZFNANCNFSM4KUMGM6A> .

pramodk · 2020-04-09T15:41:55Z

@mmigliore : Has user bp000338 used the modules in the past? i.e. can he access to/apps/hbp/ich002?

$  ls -l /apps/hbp/ich002/

Note that the we can change permissions to /apps/hbp/ich002/hbp-spack-deployments but not top level /apps/hbp/ich002/.

mmigliore · 2020-04-10T06:47:17Z

@mmigliore <https://github.com/mmigliore> : Has user bp000338 used the modules in the past?

No i.e. can he access to/apps/hbp/ich002? No. This is going to be a general problem for many users not directly related to HBP. The simulation engines (the latest version of NEURON in this case) must be available for anybody. ~> ls -l /apps/hbp/ich002/

…

total 227834 drwxr-xr-x 3 bp000037 bp0 512 Oct 17 08:31 antonel -rwxr-xr-x. 1 bp000030 bp0 2266 Jan 30 06:56 BlueConfig.test Note that the we can change permissions to /apps/hbp/ich002/hbp-spack-deployments but not top level /apps/hbp/ich002/ . — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#525 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGZUP345CEGRM354IBASI7DRLXUFHANCNFSM4KUMGM6A> .

pramodk · 2020-04-11T11:14:27Z

i.e. can he access to/apps/hbp/ich002?

No. This is going to be a general problem for many users not directly
related to HBP.

Ok, this is something new then.

Which directory is accessible for all HBP as well as non-HBP users? This is something needs to be checked with CSCS?
Is only neuron, bluepyopt module needs to be available for non-HBP users? or, also neurodamus-hippocampus?
Is this the case also on other systems? (Juelich and Cineca)

clupascu added this to the M24: March 2020 milestone Feb 13, 2020

clupascu assigned pramodk Feb 13, 2020

jorblancoa mentioned this issue Feb 25, 2020

Fixes from Juelich jureca deployment BlueBrain/spack#545

Merged

pramodk mentioned this issue Mar 17, 2020

Installation of bluepyopt on Marconi Broadwell and Galileo #522

Closed

2 tasks

pramodk mentioned this issue Mar 26, 2020

Valgrinderr neuronsimulator/nrn#461

Merged

pramodk mentioned this issue Apr 6, 2020

Summarising new deployment on Jureca, Piz-Daint and Cineca systems #533

Closed

4 tasks

clupascu closed this as completed Apr 7, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Installation of neuron7.7 on PizDaint, Jureca and Galileo #525

Installation of neuron7.7 on PizDaint, Jureca and Galileo #525

clupascu commented Feb 13, 2020 •

edited by pramodk

Loading

pramodk commented Feb 18, 2020 •

edited

Loading

pramodk commented Feb 25, 2020

jorblancoa commented Feb 28, 2020

alex4200 commented Mar 9, 2020

clupascu commented Mar 13, 2020

jorblancoa commented Mar 13, 2020

clupascu commented Mar 16, 2020

jorblancoa commented Mar 16, 2020

clupascu commented Mar 16, 2020

clupascu commented Mar 16, 2020

pramodk commented Mar 16, 2020

clupascu commented Mar 16, 2020

pramodk commented Mar 16, 2020

clupascu commented Mar 16, 2020

clupascu commented Mar 20, 2020

clupascu commented Mar 25, 2020

pramodk commented Mar 25, 2020

clupascu commented Mar 25, 2020

pramodk commented Mar 25, 2020 •

edited

Loading

clupascu commented Mar 27, 2020 •

edited

Loading

jorblancoa commented Mar 27, 2020

clupascu commented Mar 31, 2020

jorblancoa commented Mar 31, 2020

clupascu commented Mar 31, 2020

alex4200 commented Apr 6, 2020

pramodk commented Apr 7, 2020

clupascu commented Apr 7, 2020

mmigliore commented Apr 9, 2020 via email

pramodk commented Apr 9, 2020 •

edited

Loading

mmigliore commented Apr 10, 2020 via email

pramodk commented Apr 11, 2020

Installation of neuron7.7 on PizDaint, Jureca and Galileo #525

Installation of neuron7.7 on PizDaint, Jureca and Galileo #525

Comments

clupascu commented Feb 13, 2020 • edited by pramodk Loading

pramodk commented Feb 18, 2020 • edited Loading

pramodk commented Feb 25, 2020

jorblancoa commented Feb 28, 2020

alex4200 commented Mar 9, 2020

clupascu commented Mar 13, 2020

jorblancoa commented Mar 13, 2020

clupascu commented Mar 16, 2020

jorblancoa commented Mar 16, 2020

clupascu commented Mar 16, 2020

clupascu commented Mar 16, 2020

pramodk commented Mar 16, 2020

clupascu commented Mar 16, 2020

pramodk commented Mar 16, 2020

clupascu commented Mar 16, 2020

clupascu commented Mar 20, 2020

clupascu commented Mar 25, 2020

pramodk commented Mar 25, 2020

clupascu commented Mar 25, 2020

pramodk commented Mar 25, 2020 • edited Loading

clupascu commented Mar 27, 2020 • edited Loading

jorblancoa commented Mar 27, 2020

clupascu commented Mar 31, 2020

jorblancoa commented Mar 31, 2020

clupascu commented Mar 31, 2020

alex4200 commented Apr 6, 2020

pramodk commented Apr 7, 2020

clupascu commented Apr 7, 2020

mmigliore commented Apr 9, 2020 via email

pramodk commented Apr 9, 2020 • edited Loading

mmigliore commented Apr 10, 2020 via email

pramodk commented Apr 11, 2020

clupascu commented Feb 13, 2020 •

edited by pramodk

Loading

pramodk commented Feb 18, 2020 •

edited

Loading

pramodk commented Mar 25, 2020 •

edited

Loading

clupascu commented Mar 27, 2020 •

edited

Loading

pramodk commented Apr 9, 2020 •

edited

Loading