www.neuron.yale.edu

Posted: **Mon Nov 14, 2022 1:26 pm**

Hi,

I am trying to execute a script using MPI

mpiexec -n 2 nrniv -nogui -mpi -python run_model_to_steady_state.py

However, the process terminates with a

Code: Select all

Python not available

error message.

When I try to run it directly from python

Code: Select all

mpiexec -n 2 python run_model_to_steady_state.py

it works fine but I would like to be able to call it from nrniv to take full advantage of the MPI.

I am using Red Hat Enterprise Linux Server release 7.9 (Maipo) with a Python 3.9.15 virtualenv with the following packages installed via pip:

Code: Select all

$ pip freeze
lazyarray==0.5.2
mpi4py==3.1.4
neo==0.11.1
NEURON==8.0.0
nrnutils==0.2.0
numpy==1.23.1
packaging==21.3
PyNN==0.10.0
pyparsing==3.0.9
quantities==0.13.0
scipy==1.9.0

and openmpi 3.1.4.

I can run
nrniv
nrniv -nopython
without any issues.

My NRN_PYTHONHOME is set to the value reported by nrnpyenv.sh and Python lib directory is added to LD_LIBRARY_PATH. PYTHONPATH and PYTHONHOME are unset.

Is there anything I can try without compiling NEURON from scratch?

EDIT:
Found the solution, thanks everyone!

Make sure your Python is built with a shared library
export NRN_PYTHONHOME=/path/to/python
export NRN_PYLIB=/path/to/your/shared/libpython3.so

Posted: **Mon Nov 14, 2022 2:56 pm**

Not sure why it doesn't like nrniv -python, but launching nrniv when you really want to run python complicates things unnecessarily.

As long as your script initializes MPI before setting up the model, either via mpi4py or via:

Code: Select all

h.nrnmpi_init()

it suffices to just:

Code: Select all

mpiexec -n 2 python run_model_to_steady_state.py

Posted: **Mon Nov 14, 2022 7:02 pm**

I was trying to run nrniv -python because running it directly through python didn't result in a speedup.

Turns out that mpi4py needs to be imported before pyNN.neuron.

Thanks for the hint about initialization!

Posted: **Tue Nov 15, 2022 2:48 am**

If you are happy with launching python instead of nrniv, then fine and we can consider the issue closed. If you do wish to resolve the

Code: Select all

nrniv -python

issue then I'm happy to help lead you through the diagnosis and hopefully a solution. To start, I'll need to know the complete output of the above command,
and also the complete output of

Code: Select all

nrnpyenv.sh

Posted: **Tue Nov 15, 2022 5:56 am**

I am still curious why it doesn't work, so I'd be grateful for your help with diagnosis.

The nrniv -python returns

Code: Select all

$ nrniv -python
Warning: no DISPLAY environment variable.
--No graphics will be displayed.
NEURON -- VERSION 8.0.0 HEAD (429d11ef) 2021-04-30
Duke, Yale, and the BlueBrain Project -- Copyright 1984-2021
See http://neuron.yale.edu/neuron/credits

Warning: no DISPLAY environment variable.
--No graphics will be displayed.
Python not available

The output of nrnpyenv.sh is as follows:

Code: Select all

$ nrnpyenv.sh
# PYTHON=/home/people/silvan/neuron_python/bin/python3
Warning: no DISPLAY environment variable.
--No graphics will be displayed.
# find /opt/software/python/3.9.15/lib/python3.9 -name libpython3.9\*.so
# find /opt/software/python/3.9.15/lib -name libpython3.9\*.so
# []
# items in sys.path = 5
# beginning with sys.prefix = 3
# site-3 same as sys.base_prefix
# in neither location ['.', '/home/people/silvan/neuron_python/lib/python3.9/site-packages']
# sys.base_prefix = /opt/software/python/3.9.15
# site-3 = /opt/software/python/3.9.15
#pythonhome=/opt/software/python/3.9.15

#NRN_PYLIB provenance: not found

# if launch nrniv, then likely need:
export NRN_PYTHONHOME="/opt/software/python/3.9.15"
export LD_LIBRARY_PATH="/opt/software/python/3.9.15/lib:$LD_LIBRARY_PATH"

Thanks!

Posted: **Tue Nov 15, 2022 7:39 am**

The problem is that nrnpyenv.sh cannot figure out the location of the shared python library. (I can't remember how the neuron wheel was configured but
assume it was with NRN_ENABLE_DYNAMIC_PYTHON=ON. Please check with

Code: Select all

nrniv
nrnversion(6)

)
If that really is the case, then a correct value for the NRN_PYLIB environment variable is necessary. What is the result of

Code: Select all

find /opt/software/python -name lib python\*.so\*

I see that there have been a few changes to nrnpyenv.sh between 8.0.0 and the current standard distribution 8.2.1.

Code: Select all

-  PYLIB=$($python_path -c 'from distutils import sysconfig; print(sysconfig.get_config_var("LIBDIR"))')
+  PYLIB=$($python_path -c 'import sysconfig; print(sysconfig.get_config_var("LIBDIR"))')

but that is for MAC DARWIN. But I am curious what you get for

Code: Select all

% python3 -c 'import sysconfig; print(sysconfig.get_config_var("LIBDIR"))'
/Library/Frameworks/Python.framework/Versions/3.11/lib

And then, if you get a path from that, move to that location and what is the result of

cd /Library/Frameworks/Python.framework/Versions/3.11/lib
ls -l libpython*

On my Mac I'm getting

Code: Select all

lrwxrwxr-x  1 root  admin  9 Nov  5 17:51 libpython3.11.dylib -> ../Python

That is why, on my machine, nrnpyenv.sh gives the substance

Code: Select all

# if launch nrniv, then likely need:
export NRN_PYTHONHOME="/Library/Frameworks/Python.framework/Versions/3.11"
export NRN_PYLIB="/Library/Frameworks/Python.framework/Versions/3.11/lib/libpython3.11.dylib"

Posted: **Tue Nov 15, 2022 12:37 pm**

Yes, the neuron wheel was configured with NRN_ENABLE_DYNAMIC_PYTHON=ON, as confirmed by nrnversion(6)

Code: Select all

oc>nrnversion(6)
cmake option default differences: 'NRN_ENABLE_MEMACS=OFF' 'NRN_ENABLE_RX3D=OFF' 'NRN_ENABLE_MODULE_INSTALL=OFF' 'NRN_ENABLE_PYTHON_DYNAMIC=ON' 'NRN_MPI_DYNAMIC=/nrnwheel/openmpi/include;/nrnwheel/mpich/include' 'CMAKE_BUILD_TYPE=Release' 'CMAKE_INSTALL_PREFIX=/root/nrn/build/cmake_install' 'CMAKE_C_COMPILER=/opt/rh/devtoolset-2/root/usr/bin/cc' 'PYTHON_EXECUTABLE=/root/nrn/nrn_build_venv39_-4550854/bin/python'

The LIBDIR is as follows:

Code: Select all

$ python3 -c 'import sysconfig; print(sysconfig.get_config_var("LIBDIR"))'
/opt/software/python/3.9.15/lib

and it contains only the static version of the library

Code: Select all

$ ls -l /opt/software/python/3.9.15/lib
total 33212
-rwxr-xr-x  1 root root 33996090 Oct 20 17:00 libpython3.9.a
drwxr-xr-x  2 root root       96 Oct 20 17:01 pkgconfig
drwxr-xr-x 36 root root     8192 Oct 20 17:01 python3.9

The dynamic version of libpython3.9 is nowhere to be found

Code: Select all

$ find /opt/software/python -name libpython\*.so\*
/opt/software/python/2.7.15-backup1/lib/libpython2.7.so.1.0
/opt/software/python/2.7.15-backup1/lib/libpython2.7.so
/opt/software/python/2.7.15/lib/libpython2.7.so.1.0
/opt/software/python/2.7.15/lib/libpython2.7.so

$ find /opt/software/python -name libpython3.9\*
/opt/software/python/3.9.15/lib/libpython3.9.a
/opt/software/python/3.9.15/lib/python3.9/config-3.9-x86_64-linux-gnu/libpython3.9.a

so I need to reach out to the system administrator to install it.

www.neuron.yale.edu

[SOLVED] Unable to run nrniv -python with NEURON 8.0.0 installed via pip

[SOLVED] Unable to run nrniv -python with NEURON 8.0.0 installed via pip

Re: Unable to run nrniv -python with NEURON 8.0.0 installed via pip

Re: Unable to run nrniv -python with NEURON 8.0.0 installed via pip

Re: Unable to run nrniv -python with NEURON 8.0.0 installed via pip

Re: Unable to run nrniv -python with NEURON 8.0.0 installed via pip

Re: Unable to run nrniv -python with NEURON 8.0.0 installed via pip

Re: Unable to run nrniv -python with NEURON 8.0.0 installed via pip