gui not accessible and graphs not updating while running simulations from Python

When Python is the interpreter, what is a good
design for the interface to the basic NEURON
concepts.

Moderator: hines

Post Reply
claudioez
Posts: 6
Joined: Thu Oct 23, 2014 10:19 am

gui not accessible and graphs not updating while running simulations from Python

Post by claudioez » Tue Apr 17, 2018 9:42 am

Hi,

I am migrating from hoc based control of Neuron, to a Python + hoc system. I am now able to run the simulations as I wanted and record the results. I am using Spyder for writing and to execute the commands. I initiate the model using:

from neuron import h, gui
import numpy
import matplotlib.pyplot as plt...etc.

I run the simulations using a for loop, exemplified below, that should update a graph after every simulation run. This actually does not happen, instead, the graph is updated only when exiting the loop. I try then to use graphs on neuron instead, but while running the simulation I can´t press anything in the Neuron gui, much in contrast to what happens if I use the HOC code. That applies to any menu in the Nueron gui.

Any suggestion to improve this behavior? Thanks, Claudio.


def RunSim():
global Savearray
V_vec=h.Vector(Pms['timestep']*Pms['expdur'])
t_vec=h.Vector(Pms['timestep']*Pms['expdur'])
V_vec.record(h.soma(0.5)._ref_v)
t_vec.record(h._ref_t)

for Run_n in range(0,Pms['batchRepetitions']): # the run loop
if Run_n!=0:
Pms[changewhatstring]=Pms[changewhatstring]+Pms['batchIncrement']
CClamp_stim() # changes the Iclamp properties in this example.
h.finitialize(-75)
h.fcurrent()
h.run()
if Run_n==0:
Pt_vec=t_vec.to_python()
Pt_vec=Pt_vec[:-1]
Savearray[Run_n]=Pt_vec
PV_vec=V_vec.to_python()
PV_vec=PV_vec[:-1]
Savearray[Run_n+1]=PV_vec
if Run_n!=0:
fig_temp=plt.plot(Savearray[0],Savearray[Run_n+1])
else:
fig_temp= plt.figure(figsize=(8,4)) # Default figsize is (8,6)
fig_temp=plt.plot(Savearray[0],Savearray[Run_n+1])
fig_temp=plt.xlabel('time')
fig_temp=plt.ylabel('mV')
plt.show()
Savearray=Savearray.T
numpy.savetxt("Results.csv", Savearray, delimiter=",")

ramcdougal
Posts: 138
Joined: Fri Nov 28, 2008 3:38 pm
Location: Yale School of Medicine

Re: gui not accessible and graphs not updating while running simulations from Python

Post by ramcdougal » Tue Apr 17, 2018 9:45 am

Is this problem Spyder-specific, or does it also occur if you run the script from the regular Python prompt?

What operating system and NEURON version are you using?

The "from neuron import gui" is supposed to be all you need.

claudioez
Posts: 6
Joined: Thu Oct 23, 2014 10:19 am

Re: gui not accessible and graphs not updating while running simulations from Python

Post by claudioez » Tue Apr 17, 2018 11:04 am

It is not Spyder-specific. From the python prompt the behavior is the same.

OS: Windows 10 Pro for workstations, 64 bits
Neuron: VERSION 7.5 master (6b4c19f) 2017-09-25

I was wondering if it´s maybe a normal python behavior (first time I program in python), and this can be solved by using different threads, but until now I didn´t figure out how to do this.

Thanks, Claudio.

ramcdougal
Posts: 138
Joined: Fri Nov 28, 2008 3:38 pm
Location: Yale School of Medicine

Re: gui not accessible and graphs not updating while running simulations from Python

Post by ramcdougal » Wed Apr 18, 2018 4:43 pm

Both of your difficulties are due to a Windows-specific bug that keeps NEURON from releasing the GIL during integration, which blocks both the NEURON GUI and Matplotlib threads from executing... thus the graphs never get updated. Thanks for bringing this to our attention.

A workaround for now (besides using Mac, Linux, or the Windows Subsystem for Linux) is to add the following code to your model at any point before initialization; it will explicitly release and reacquire the GIL at every time step.

Code: Select all

from neuron import h
import time

def abandon_gil():
    # necessary to allow other threads to update the GUI during integration
    time.sleep(0)

h.CVode().extra_scatter_gather(0, abandon_gil)
This will slow down execution slightly, but hopefully not too much, and it is safe to use on the other platforms that don't have this issue.

claudioez
Posts: 6
Joined: Thu Oct 23, 2014 10:19 am

Re: gui not accessible and graphs not updating while running simulations from Python

Post by claudioez » Thu Apr 19, 2018 7:12 am

Dear ramcdougal,

indeed that stops the problem of the neurongui being stuck, but at the same time creates a problem with memory management as it continuously stores data in the memory until 100% is used and eventually python crashes (I have 64 GB of RAM). Another symptom is that although the gui is accessible, "run control" does not show any progress.

Best, Claudio

ramcdougal
Posts: 138
Joined: Fri Nov 28, 2008 3:38 pm
Location: Yale School of Medicine

Re: gui not accessible and graphs not updating while running simulations from Python

Post by ramcdougal » Thu Apr 19, 2018 9:29 am

Point of clarification: are you saying it runs without memory problems if you don't use that workaround, but crashes when you do? How long does each simulation run (h.tstop) for, and how many simulations are you running?

If you would like, I can take a look at your code to try to track down the memory issue. You can email me privately at robert dot mcdougal at yale dot edu.

A few tips:
  • instead of using t_vec.to_python() it's probably more Pythonic to say list(t_vec).
  • Using to_python or list as above both create copies of the NEURON Vector; in most cases (e.g. plotting) you can use the Vector directly -- i.e. plt.plot(t_vec, V_vec) -- or use e.g. t_vec.as_numpy() which returns a numpy array that shares memory (so no copying) with the NEURON Vector.
  • The h.run() function automatically initializes to h.v_init, so assuming you have that set to -75, there is no need for the h.finitialize or h.fcurrent lines. (If you don't have h.v_init set to -75, then you're starting from some other voltage; the default is -65.)

claudioez
Posts: 6
Joined: Thu Oct 23, 2014 10:19 am

Re: gui not accessible and graphs not updating while running simulations from Python

Post by claudioez » Thu Apr 19, 2018 11:05 am

Hi,

indeed it runs without memory problems if I don't use thE workaround, but crashes when I do use it.

I am running just one simulation, CPU is only 3% used but memory fully used after some minutes.

I will send you the files, thanks! =)

ramcdougal
Posts: 138
Joined: Fri Nov 28, 2008 3:38 pm
Location: Yale School of Medicine

Re: gui not accessible and graphs not updating while running simulations from Python

Post by ramcdougal » Tue Apr 24, 2018 11:01 am

This windows-specific bug also seems to only occur if the simulation is run from the terminal or from a script. If you were to click the "Run" button on the GUI (e.g. in Tools - Run Control), the graphs should update etc during the simulation. That may or may not be useful to you.

ramcdougal
Posts: 138
Joined: Fri Nov 28, 2008 3:38 pm
Location: Yale School of Medicine

Re: gui not accessible and graphs not updating while running simulations from Python

Post by ramcdougal » Mon May 07, 2018 12:51 pm

This is still a bit of a hack, but it should keep the GUI responsive and avoid the memory issue.

Before you do your h.run() (or equivalent), do:

Code: Select all

h('''
proc advance() {
    fadvance()
    nrnpython("")
}
''')

Post Reply