Strange behavior Updating HOC Vectors inside Python+mpi deff

When Python is the interpreter, what is a good
design for the interface to the basic NEURON
concepts.

Moderator: hines

Post Reply
R_Jarvis

Strange behavior Updating HOC Vectors inside Python+mpi deff

Post by R_Jarvis » Thu Jan 22, 2015 8:31 pm

I have written an analysis to calculate transfer entropy on a large data set. The data set is so large I am using MPI such that I have access to more memory.

I am having trouble updating HOC Vectors inside Python definition, whenever I am using mpi4py as well, which is why I have made tautology statements like: 'h.py.j=j', the same is true for h.py.k=k, h.py.i=i. Strangely a statement like that will make update the python variable j, k, and i and if I did not make such a statement the values of j k i would not be updated. Which is catastrophic when they required to act as iterators in loops.

Sometimes k does get updated and sometimes it doesn't, it seems to be influenced by how many statements come before it. Is it possible that this is caused by blocking behavior of synchronous MPI calls like broadcast? Maybe the HOC object in Python is not designed to work with mpi4py.



Code: Select all

def t_e(SIZE, NCELL, RANK, trentm):
  
    h('storval = new Vector()')
    h('objref vec')
    h('vec=new Vector()')


    for k in xrange(0, SIZE):
        for j in xrange(0, int(NCELL)):
            if RANK == k:
                histogram = []
                test = 0
                h.py.j=j
                h('py.test=py.int(pc.gid_exists(py.int(py.j)))')
            
                if test != 0:
                    h('source=pc.gid2cell(py.int(py.j))')
                
                    h('vec=source.recvec1')

                    h('vec.x[0]=py.j')

                    histogram = h.source.recvec1.to_python()
                if test == 0:
                    histogram = None
            else:

                histogram = None
            histogram = COMM.bcast(histogram, root=k)  # ie root = rank
            h.py.k=k
            
            h('py.gidn=int(vec.x[0])')
            h('vec.x[0]=0')
            gidn = int(gidn)
            h('py.test2=int(vec.sum())')
            test2 = h.vec.sum()


            if int(test2) != 0:
                print 'cell number ', gidn
                for i in xrange(0, int(NCELL - 1)):
                    h.py.i=i
                    test3 = 0
                    h('py.test3=py.int(pc.gid_exists(py.int(py.i)))')
                    if test3 != 0:
                        h('target=pc.gid2cell(py.int(py.i))')
                   
                        h('storval=normte(target.recvec1,vec,20)')
                
                        
                        trentm[i][gidn] =float(h.storval.x[2])
    return trentm           

trentm=t_e(SIZE, NCELL, RANK, trentm,np)



Thanks for any help or suggestions. I think there will definitely be ways of simplify the code, I am open to any suggestions about how to do this. I think the whole thing should be rewritten as a more general python definition that performs a broadcast inside an iterator and returns a matrix.

aaronmil
Posts: 35
Joined: Fri Apr 25, 2014 10:54 am

Re: Strange behavior Updating HOC Vectors inside Python+mpi

Post by aaronmil » Fri Jan 23, 2015 1:54 pm

Why are your storval and vec objects created through the "last resort" hoc interpreter? Just declare them as Python objects like:

Code: Select all

storval = h.Vector()
vec = h.Vector()
Then interact with the objects purely in Python, rather than hoc.

hines
Site Admin
Posts: 1575
Joined: Wed May 18, 2005 3:32 pm

Re: Strange behavior Updating HOC Vectors inside Python+mpi

Post by hines » Sat Jan 24, 2015 7:49 am

To further emphasize aaronmil's point, I would also replace all the lines analogous to

Code: Select all

h('py.test3=py.int(pc.gid_exists(py.int(py.i)))')
with

Code: Select all

test3=int(pc.gid_exists(i))
ie. use pc directly in python with perhaps
pc = h.ParallelContext()
rank = int(pc.id())
nhost = int(pc.nhost())

However, you mentioned the problem:
I am having trouble updating HOC Vectors
h.py.i is in a different namespace than your local i used in your def. Circumstances where you need h('...') are very rare.

R_Jarvis

Re: Strange behavior Updating HOC Vectors inside Python+mpi

Post by R_Jarvis » Mon Jan 26, 2015 7:27 pm

Okay great. Thanks for the advice, I will try to write everything in a more pythonic way. I also ran into trouble when I tried that once. I will post what happened here later.

Post Reply