Nested for loops over SectionList objects

When Python is the interpreter, what is a good
design for the interface to the basic NEURON
concepts.

Moderator: hines

Post Reply
selfdestructo
Posts: 32
Joined: Wed Oct 14, 2009 11:12 am

Nested for loops over SectionList objects

Post by selfdestructo » Tue Nov 27, 2018 8:00 am

Dear NEURON devs;

The most recent version of NEURON v7.6.3 appears to have caused some problems in LFPy (github.com/LFPy/LFPy) over v7.6.2 as the SectionList class lost its allsec method (in this particular commit from the look of it: https://github.com/neuronsimulator/nrn/ ... 3ea28dc706).

The main problem is a nested for loop over sections put in a SectionList which is attached to a Cell object like demonstrated below. Most likely we were not doing things the right way in the first place, but, my question is: What is the correct way of doing a double for loop over the same SectionList object in NEURON 7.6.3?

Code: Select all

import neuron
neuron.h('forall delete_section()')

class Cell(object):
    def __init__(self):
        neuron.h.load_file(1, 'example_morphology.hoc') # https://raw.githubusercontent.com/LFPy/LFPy/master/examples/morphologies/example_morphology.hoc
        self.seclist = neuron.h.SectionList()
        for sec in neuron.h.allsec():
            seclist.append(sec=sec)
cell = Cell()

print('test nested loop 1 (ok, but we want to iterate over sections in the cell.seclist object):')
for sec_outer in neuron.h.allsec():
    for sec_inner in neuron.h.allsec():
        print(sec_outer.name(), sec_inner.name())
print('test nested loop 2 (not ok):')
for sec_outer in cell.seclist:
    for sec_inner in cell.seclist:
        print(sec_outer.name(), sec_inner.name())
print('test nested loop 3 (ok, up until v7.6.3):')
for sec_outer in cell.seclist.allsec():
    for sec_inner in cell.seclist.allsec():
        print(sec_outer.name(), sec_inner.name())
which results in:

Code: Select all

test nested loop 1 (ok, but we want to iterate over sections in the cell sectionlist):
('soma[0]', 'soma[0]')
('soma[0]', 'dend[0]')
('soma[0]', 'dend[1]')
('soma[0]', 'dend[2]')
('dend[0]', 'soma[0]')
('dend[0]', 'dend[0]')
('dend[0]', 'dend[1]')
('dend[0]', 'dend[2]')
('dend[1]', 'soma[0]')
('dend[1]', 'dend[0]')
('dend[1]', 'dend[1]')
('dend[1]', 'dend[2]')
('dend[2]', 'soma[0]')
('dend[2]', 'dend[0]')
('dend[2]', 'dend[1]')
('dend[2]', 'dend[2]')
test nested loop 2 (not ok):
test nested loop 3 (ok, up until v7.6.3):
('soma[0]', 'soma[0]')
('soma[0]', 'dend[0]')
('soma[0]', 'dend[1]')
('soma[0]', 'dend[2]')
('dend[0]', 'soma[0]')
('dend[0]', 'dend[0]')
('dend[0]', 'dend[1]')
('dend[0]', 'dend[2]')
('dend[1]', 'soma[0]')
('dend[1]', 'dend[0]')
('dend[1]', 'dend[1]')
('dend[1]', 'dend[2]')
('dend[2]', 'soma[0]')
('dend[2]', 'dend[0]')
('dend[2]', 'dend[1]')
('dend[2]', 'dend[2]')
Any help would be much appreciated - our new paper on LFPy 2.0 now got accepted (https://doi.org/10.3389/fninf.2018.00092) and we should obviously allow use of NEURON >= 7.6.3

Kind regards,
Espen

hines
Site Admin
Posts: 1528
Joined: Wed May 18, 2005 3:32 pm

Re: Nested for loops over SectionList objects

Post by hines » Tue Nov 27, 2018 9:03 am

The following is intended behavior

Code: Select all

>>> sl = h.SectionList()
>>> sl.allsec()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'hoc.HocObject' object has no attribute 'allsec'
that is, the allsec method is restricted to the <HocTopLevelInterpreter>. The changeset referred to fixed the issue that some top level methods
had leaked into the namespace of all Hoc objects. Very misleading. I think it is just an accident that your <7.6.3 code worked as it is likely that
you intended to iterate only over the sections that were contained in the sectionlist, not all the sections global to the process. (the latter is
properly h.allsec()).

So the bug, really is that one cannot nest loops involving the same sectionlist. This is something that should be fixable. It is undoubtedly due to my
lack of understanding of the subteties of python iterators in the early days of development of the HocObject. In the meantime,
nesting can be successful if one uses a python list or creates a copy of the SectionList. By the way, given the normal interpretation of the word
"Cell" as being a tree, a good idiom for generating a seciionlist for the entire tree
is to use the SectionList.wholetree(sec=section_in_the_tree)

hines
Site Admin
Posts: 1528
Joined: Wed May 18, 2005 3:32 pm

Re: Nested for loops over SectionList objects

Post by hines » Tue Nov 27, 2018 11:29 am

A fix for nested SectionList iteration in python has been pushed to the github repository. I use the test:

Code: Select all

$ cat slseq.py
from neuron import h
a = [h.Section(name='s%d'%i) for i in range(3)]
sl = h.SectionList()
for s in a:
  sl.append(sec=s)

for s in sl:
  print(s)
print("")

for x in sl:
  for y in sl:
    print (x, y)
print("")

h('''
objref z
z = new PythonObject()
z = z.sl
''')
for x in h.z:
  for y in h.z:
    print (x, y)
and that produces the output

Code: Select all

$ nrniv -python slseq.py
NEURON -- VERSION 7.6.3-1-g5174b0b master (5174b0b) 2018-11-27
Duke, Yale, and the BlueBrain Project -- Copyright 1984-2018
See http://neuron.yale.edu/neuron/credits

s0
s1
s2

(s0, s0)
(s0, s1)
(s0, s2)
(s1, s0)
(s1, s1)
(s1, s2)
(s2, s0)
(s2, s1)
(s2, s2)

(s0, s0)
(s0, s1)
(s0, s2)
(s1, s0)
(s1, s1)
(s1, s2)
(s2, s0)
(s2, s1)
(s2, s2)
>>> 
It is interesting that the iteration of h.z did not have the problem before this fix
because every execution creates a new Python HocObject instance so
the iterators inthe HocObject are not shared by the nested loops.

selfdestructo
Posts: 32
Joined: Wed Oct 14, 2009 11:12 am

Re: Nested for loops over SectionList objects

Post by selfdestructo » Wed Nov 28, 2018 3:12 am

Much obliged. I will compile NEURON on my end here and try it out.

hines
Site Admin
Posts: 1528
Joined: Wed May 18, 2005 3:32 pm

Re: Nested for loops over SectionList objects

Post by hines » Wed Nov 28, 2018 8:17 am

Please try the latest version. I inadvertently reffed the section list iterators twice which prevents deletion of the SectionList
See commit a3d1233b

selfdestructo
Posts: 32
Joined: Wed Oct 14, 2009 11:12 am

Re: Nested for loops over SectionList objects

Post by selfdestructo » Fri Nov 30, 2018 8:59 am

Hi, I can confirm that your fix pushed to the github repository works for me. Thanks a lot.

Post Reply