to_python() causes big memory problems

When Python is the interpreter, what is a good
design for the interface to the basic NEURON
concepts.

Moderator: hines

Post Reply
arb
Posts: 19
Joined: Mon Jul 02, 2007 6:18 am
Location: Humboldt-University, Berlin

to_python() causes big memory problems

Post by arb » Wed Sep 17, 2008 5:43 pm

Since the new version (6.2.3) came out I am using the to_python function to copy neuron vectors to python arrays.. The function to_python seems to allocate new memory that it never gives free again..
I wrote a simple example to illustrate this problem. (Points are spaces below)

Example 1:
16GB Memory fills within 30s:!!!!!

import neuron
h = neuron.h

h("""
objref vec
vec = new Vector(1000)
""")

for i in range(100000):
......a = range(1000)
......h.vec.to_python(a)

######################
Example 2:
Everthing OK!

import neuron
h = neuron.h

h("""
objref vec
vec = new Vector(1000)
""")

a = range(1000)
for i in range(100000):
......h.vec.to_python(a)

################
Example 3:
Everthing OK:

import neuron
h = neuron.h

h("""
objref vec
vec = new Vector(1000)
""")


for i in range(100000):
......a = range(1000)

h.vec.to_python(a)


I need to run a loop and the vector I record I do not know the size of before my loop starts, so I need example 1 to be working....

Any ideas, any help?
Armin

hines
Site Admin
Posts: 1577
Joined: Wed May 18, 2005 3:32 pm

Re: to_python() causes big memory problems

Post by hines » Thu Sep 18, 2008 11:15 am

That leak has been fixed in the alpha version at
http://www.neuron.yale.edu/ftp/neuron/versions/alpha/
The to_python and from_python stil exist but are more or less obsolete since better idioms are now
v = h.Vector(numpy_array)
v = h.Vector(python_list)
python_list = list(v)
numpy_array = numpy.array(v)
Also it gives full speed between numpy and hoc without the requirement that
numpy be available during neuron build time. i.e. the --enable-numpy has been
removed since it is no longer needed.

henriklinden

Re: to_python() causes big memory problems

Post by henriklinden » Tue Sep 23, 2008 3:56 am

Is there in this new idiom a way to access a vector in python that has been created in a .hoc-script? This is a useful thing about the .to_python() since I have parts of my code in .hoc and parts of the code in .py-scripts.

mattions
Posts: 65
Joined: Tue Jul 15, 2008 11:21 am
Location: EMBL-EBI Cambridge UK

Re: to_python() causes big memory problems

Post by mattions » Fri Jan 14, 2011 6:27 am

If to_python() is deprecated, which inplace operation I can use to move from one HocVector to a numpy array?

ted
Site Admin
Posts: 5592
Joined: Wed May 18, 2005 4:50 pm
Location: Yale University School of Medicine
Contact:

Re: to_python() causes big memory problems

Post by ted » Fri Jan 14, 2011 10:35 am

mattions wrote:If to_python() is deprecated
Who said it was deprecated?

mattions
Posts: 65
Joined: Tue Jul 15, 2008 11:21 am
Location: EMBL-EBI Cambridge UK

Re: to_python() causes big memory problems

Post by mattions » Fri Jan 14, 2011 10:37 am

hines wrote: ...
The to_python and from_python stil exist but are more or less obsolete since better idioms are now
v = h.Vector(numpy_array)
v = h.Vector(python_list)
python_list = list(v)
numpy_array = numpy.array(v)
...
So it's ok to use it?

ted
Site Admin
Posts: 5592
Joined: Wed May 18, 2005 4:50 pm
Location: Yale University School of Medicine
Contact:

Re: to_python() causes big memory problems

Post by ted » Sat Jan 15, 2011 10:44 am

to_python and from_python are neither obsolete nor deprecated.

oren
Posts: 54
Joined: Fri Mar 22, 2013 1:03 am

Re: to_python() causes big memory problems

Post by oren » Thu Jun 19, 2014 4:15 am

I think the Leak has returned
NEURON -- VERSION 7.3 ansi (1078:2b0c984183df) 2014-04-04

ipython

Code: Select all

from neruon import h
Vec = h.Vector()
Vec.indgen(0,100,0.25)
while 1:
    Vec.to_python();
2.5GB in 5 Seconds.


Also when I run the original code that arb posted

Code: Select all

import neuron
h = neuron.h

h("""
objref vec
vec = new Vector(1000)
""")

for i in range(100000):
    a = range(1000) 
    h.vec.to_python(a)
3.1GB

hines
Site Admin
Posts: 1577
Joined: Wed May 18, 2005 3:32 pm

Re: to_python() causes big memory problems

Post by hines » Thu Jun 19, 2014 10:47 am

Thanks for pointing this out. The fix is
http://www.neuron.yale.edu/hg/neuron/nr ... d2c45fb1b4

Post Reply