request for comments

When Python is the interpreter, what is a good
design for the interface to the basic NEURON
concepts.

Moderator: hines

Post Reply
hines
Site Admin
Posts: 1577
Joined: Wed May 18, 2005 3:32 pm

request for comments

Post by hines » Wed May 25, 2005 7:57 am

I have a question primarily directed to those who understand quite a bit about Python and a bit about NEURON's conceptual idea of the section.

Several months ago, Mike Neubig suggested that NEURON's usefulness would be greatly enhanced if Python could be used as an alternative interpreter. To this purpose I've started an experimental branch called nrnpython in the cvs repository http://www.neuron.yale.edu/cgi-bin/cvspage.cgi and have gotten to the point where I now believe it can feasibly be carried off with reasonable effort.

Where I am is that on linux, mswin, and mac os x I can run

Code: Select all

neurondemo -python
and the normal demo will start and operate normally (all the InterViews and hoc gui tools, so no problem with legacy models written in hoc) but the interactive interpreter is Python. (without the -python arg at launch, the interactive interpreter is hoc). At present the only communication I've implemented between hoc and Python is that in hoc one can say

Code: Select all

nrnpython("any python command")
and in Python one can say

Code: Select all

from nrn import execute
execute('load_file("nrngui.hoc")') #or any other hoc statement
# starts a usable NEURONMainMenu
Now it is time to make section information available to Python and I'd like advice about how it should be elegantly expressed. (after all, I am a python beginner).

For example, in hoc one can say

Code: Select all

    forall if (ismembrane("hh") for (x, 0){
        print secname(), x, v(x), gnabar_hh(x)
    }
Can one imagine saying in Python?

Code: Select all

    import nrn
    for sec in nrn.allsec():
        if hh in sec.mechanisms():
            for x in sec.locations():
                print sec.name, x, sec.v(x), sec.hh.gnabar(x)
The hh token is definitely wrong in the "if" statement and maybe it should be the string 'hh' but my assumption of variant fields in the sec object is kind of neat if it is possible (sec.hh exists only if it has been inserted into the section?) Perhaps it would be best to keep things functional as in

Code: Select all

    print sec.name(), x, sec.v(x), sec.mech("hh").gnabar(x)
How about

Code: Select all

    sec.v(x) = -65
?

I'd like to hear your ideas.

-Michael Hines

AG

neuron & python

Post by AG » Fri Jan 06, 2006 7:29 am

Dear M. Hines,

I would like to know if there was a way to join a neuron-python development group or if there was some recent development. I am very interested in participating to that kind of project.

AG.

hines
Site Admin
Posts: 1577
Joined: Wed May 18, 2005 3:32 pm

Post by hines » Fri Jan 06, 2006 12:54 pm

That would be great. I merged the nrnpython branch with the main trunk in the cvs sources last summer and so you can easily see where I left off. You need to use the --with-nrnpython configure option. If configure fails, take a look at the nrn/m4/nrnpython.m4 configuration code. Especially the comment about
Probably need to set PYLIBDIR to find libpython...
and PYINCDIR to find Python.h
This hasn't been worked on since last winter when I got it to the point of being able to launch nrniv with python as the replacement interpreter. The important design decisions of how to deal with sections, mechanisms, and range variables have not been made. And the tedious interface of neuron internal functions to python has not been made (Perhaps the latter would be done semiautomatically from the lists in nrn/src/oc/hoc_init.c and a few other files).

The thing I'd like to hear most from you at this point is what you consider to be a nice correspondence in python to the functionality of range variables.

eacheon
Posts: 97
Joined: Wed Jan 18, 2006 2:20 pm

Re: request for comments

Post by eacheon » Wed Jan 18, 2006 9:16 pm

hines wrote: Can one imagine saying in Python?

Code: Select all

    import nrn
    for sec in nrn.allsec():
        if hh in sec.mechanisms():
            for x in sec.locations():
                print sec.name, x, sec.v(x), sec.hh.gnabar(x)
If use list filtering, Python may be able to say:

Code: Select all

[ sec.name for sec in nrn.allsec() if sec.get("hh") ]
if each section "sec" is a dictionary-like object, and if nrn.allsec() returns a list of such dictionaries.
hines wrote: The hh token is definitely wrong in the "if" statement and maybe it should be the string 'hh' but my assumption of variant fields in the sec object is kind of neat if it is possible (sec.hh exists only if it has been inserted into the section?) Perhaps it would be best to keep things functional as in

Code: Select all

    print sec.name(), x, sec.v(x), sec.mech("hh").gnabar(x)
How about

Code: Select all

    sec.v(x) = -65
?

I'd like to hear your ideas.

-Michael Hines
The dictionary might be the right thing to go, though it may need some work to get a section class working exactly as it in hoc.

What's the purpose of making NEURON interprator python-based, can I ask?

AG

Post by AG » Thu Jan 19, 2006 4:50 am

Hi,

The purpose of making NEURON interpretor python based seems obvious if the development time is not too important of course. You must admit that HOC syntax and capabilities are poor, compared to Python or Java. Besides, Python has already many extensions (mathematical, graphical, 3D modelling ...) and the use of all these tools interacting directly with NEURON should contribute to improve its capabilities and use.

AG.

eacheon
Posts: 97
Joined: Wed Jan 18, 2006 2:20 pm

Post by eacheon » Wed Jan 25, 2006 10:44 pm

AG wrote:Hi,

The purpose of making NEURON interpretor python based seems obvious if the development time is not too important of course. You must admit that HOC syntax and capabilities are poor, compared to Python or Java. Besides, Python has already many extensions (mathematical, graphical, 3D modelling ...) and the use of all these tools interacting directly with NEURON should contribute to improve its capabilities and use.

AG.
Thanks for explanation, I am not sure how much work it would take. I just began with NEURON.

eacheon
Posts: 97
Joined: Wed Jan 18, 2006 2:20 pm

Post by eacheon » Wed Jan 25, 2006 10:57 pm

hines wrote: The thing I'd like to hear most from you at this point is what you consider to be a nice correspondence in python to the functionality of range variables.
If I ventured an idea I would suggest that one range variable would be implemented as a method for each "section" object. This class method accepts one argument which may be any float in [0,1], and return the value it found in the closest segment.

This would not be cumbersome to implemented, if the section object itself (which might be inherited from dictionary class) keeps data for every segment in this section. The data could be in form of a dictionary of list,
where each list can be referenced by the key (presumably the name of range variable in NEURON), storing all the values for this specific range variable.

I am not sure if my understanding of range variable is totally accurate.

apdavison
Posts: 14
Joined: Tue May 24, 2005 3:56 am
Location: CNRS, Gif sur Yvette, France
Contact:

Post by apdavison » Thu Mar 23, 2006 12:38 am

eacheon wrote: If I ventured an idea I would suggest that one range variable would be implemented as a method for each "section" object. This class method accepts one argument which may be any float in [0,1], and return the value it found in the closest segment.
Unfortunately, this works fine for
print soma.v(0.5)
but not for
soma.v(0.5) = -65

Python complains:
SyntaxError: can't assign to function call
no matter what the function call returns

I think the nearest one can come to the hoc syntax is to replace parentheses with square brackets, i.e.,
print soma.v[0.5]
soma.v[0.5] = -65
by redefining the __getitem__ and __setitem__ methods

Here's an example implementation:

Code: Select all

import numpy

allsec = []

def _checkLocation(loc):
    """Raise an Exception if loc < 0 or loc > 1."""
    if 0 <= loc < 1:
        return loc
    elif loc == 1:
        return 0.999999
    else:
        raise IndexError, "Range must be between 0 and 1"
        
class RangeVariable:
    """A range variable."""
    
    def __init__(self,parent,initval):
        self.parent = parent
        self.nseg = parent.nseg
        self.vals = numpy.array([initval]*parent.nseg)

    def __getitem__(self,loc):
        if self.nseg != self.parent.nseg:
            self.__resize()
        i = int(_checkLocation(loc) * self.parent.nseg)
        return self.vals[i]
    
    def __setitem__(self,loc,val):
        if self.nseg != self.parent.nseg:
            self.__resize()
        i = int(_checkLocation(loc) * self.parent.nseg)
        self.vals[i] = val

    def __resize(self):
        tmp = numpy.zeros(self.parent.nseg,float)
        for i in range(0,self.parent.nseg):
            j = int(i*self.nseg/self.parent.nseg)
            tmp[i] = self.vals[j]
        self.vals = tmp
        self.nseg = self.parent.nseg

class Section:
    """A cable section."""

    def __init__(self,name=None):
        self.nseg = 1
        self.v = RangeVariable(self,-65)
        allsec.append(self)
        if name:
            self.name = name
        else:
            self.name = "Section%d" % id(self)

    def insert(self,mechname):
        """Insert a mechanism into the section."""
        # should raise Exception if the mechanism does not exist        
        exec("self.%s = %s(self)" % (mechname,mechname))

    def locations(self):
        return [(i+0.5)/self.nseg for i in range(0,self.nseg)]

    def contains(self,mechname):
        if hasattr(self,mechname):
            return True
        else:
            return False

class hh:
    """Dummy mechanism."""
    
    def __init__(self,parent):
        self.parent = parent
        self.gnabar = RangeVariable(parent,0.12)
        self.gkbar  = RangeVariable(parent,0.036)
        self.gl     = RangeVariable(parent,0.0003)
        self.el     = RangeVariable(parent,-54.3)

        
if __name__ == "__main__":
    soma = Section("Soma")
    soma.nseg = 3
    soma.v[0.5] = -50
    soma.v[1] = -40
    soma.insert("hh")
   
    for sec in allsec:
        if sec.contains("hh"): 
            for x in sec.locations(): 
                print sec.name, x, sec.v[x], sec.hh.gnabar[x]
The only problem with this is that you have to use the brackets, i.e. you can't write
print soma.v
or
soma.v = -65
as in hoc. The latter part (assignment) can be done using descriptors, but I can't figure out a way to do the former.

hines
Site Admin
Posts: 1577
Joined: Wed May 18, 2005 3:32 pm

Post by hines » Fri Mar 31, 2006 7:56 am

Hi Andrew,
That is strange. I did not get notified by email that a post was added to that topic and the forum claims I
am watching it. I wonder if my spam filter is inadvertently throwing away those notifications.
Anyway, I am delighted for you to be involved. I am still working on the parallel computing and the
realtime dynamic clamp so it would be a while before I could devote a large amount of time to the
nrnpython branch.
The v[0.5] idea is certainly acceptable. I think in practice, the idea of section has turned out to be
a low order idea and the actual usage is at the level of either cells with subsets of sections (note that the
cell builder has introduced the idea of parameterized subsets to allow specification of inhomogeneous
channel densities) or else the single compartment. In the former case the basic idiom in hoc is
forsec seclist for (x, 0) { .... rangevar(x) .... }
and only in the single compartment case is it convenient to "access soma" once forever and then use
rangevar without the argument.
Tell you the truth, given the demonstrated
ability to run any (including GUI) legacy hoc code in combination with
Python, makes me fairly unconcerned about syntactic level closeness between HOC and Python.
It is only necessary to remember that the unique idea behind NEURON (not always carried out
successfully) is that a section is meant to represent a length of continuous cable. The Python syntax that
most closely suggests this and provides methods for specifying range variables as functions of position
is preferred. For a long time NEURON/HOC
had no notion of setting a value at rangevar(x) and one could only
use the rangevar(x1:x2) = e1:e2 syntax. From a high level perspective the
for (x,0) { rangevar(x) = ... } idiom is not high level enough since the x value along a section is not so
useful as the idea of position of that point in the context of the cell. I guess the upshot of these
ruminations is that since the rangevar(x) concept is midway between high neurobiology level and computer
level, conversion to rangevar[x] is probably the best idea on offer. By the way, I have been sadly undiligent
in my learning of Python, but I seem to remember that there is quite a bit of syntax available within the
[...] which may also be useful with respect to the idea of constant over an entire section (and what about
rangevar[] with nothing?) If so then I believe that confirms the appropriateness of the transformation
from (...) to [...]. Note; for range variable arrays I presume the syntax would be
rangevar[x]

jvtoups

Suggestions

Post by jvtoups » Thu Apr 06, 2006 10:26 pm

Dr Hines,

I would like to make the following suggestions about how to implement the section notation in a neuron/python environment.

I think sections should be represented as closely as possible to native Python classes, and that the internal code should use the information in this class representation to figure out what sorts of properties each section in the code has.

for example, you might write

Code: Select all


s = section();

to create a new section. Section() is the constructor for section objects, and it returns an object with properties s.L, s.diam, s.nseg etc.

To add a hh style channel to the section, you would write

Code: Select all


s.hh = HodgkinHuxley();

and this would add a sub class to the section with appropriate C book-keeping happening behind the scenes, but also with members
s.hh.gnabar, s.hh.gkbar, etc etc.

Either when a section is created or before each simulation, the undercode should read the properties for each section from the Python object which represents it. You could imagine extending this notation to include synapses as data members or subclasses with references to other sections. I think this would be the most expressive and natural way of representing the underlying simulation to the user. It doesn't seem like it would be too difficult to implement things this way either, although obviously I don't have a very good idea yet of how it would be done.

Anyway, those are my two bits.

hines
Site Admin
Posts: 1577
Joined: Wed May 18, 2005 3:32 pm

Post by hines » Fri Apr 07, 2006 6:49 am

That syntax is attractive in its clarity. But I don't immediately see how

Code: Select all

s.hh.gnabar = .12
when s.nseg = 50 ends up setting the
gnabar value in all 50 segments. I think andrew's idea (pace someone who actually knows python) is that would be done with

Code: Select all

s[].hh.gnabar = .12
Can a python person tell me what syntax could capture

Code: Select all

s[0<x<1].hh.gnabar = f(x)
One possiblity that seems exciting to me
is the generalization to parameterized subsets

Code: Select all

cell[i].distal[all x].hh.gnabar = f(x)
I guess I need to learn more python.

jvtoups

Post by jvtoups » Fri Apr 07, 2006 8:55 pm

Dr Hines, you are right. I had forgotten about the individual segments. It might be easiest, then, to have an array as part of the substructer which represents all sections. You could then do some python indexing tricks to intialize all the variables in whatever way you'd like. The following syntax might be about right:

Code: Select all


[s.sec[i].hh.gnabar = .2 for i in range(s.nsec)]

Have you heard of Pyrex? It is supposed to make writing glue code between c and python much easier. It may be useful to you. Let me find a link: Pyrex.

I am the student who would like to work with Lua, but if there is already an effort in the direction of using Python, I would be glad to lend a hand with the code. It is sort of difficult to dive head first into a stranger's code, though. Is there a quick "global view" of how Neuron's internals work programmatically and how they should sit with Python? I would find such a picture, even if it was cursory, very useful.

For example, are you extending or embedding the neuron things into Python? Most python users would find it easiest to use neuron by installing a module, starting up Python with some nice set of scientific packages like SciPy and then using import statements to get access to Neuron specific features.

Code: Select all


import neuron
neuron.\\dothings\\
...


eacheon
Posts: 97
Joined: Wed Jan 18, 2006 2:20 pm

Post by eacheon » Tue Jul 11, 2006 7:37 pm

hines wrote: Can a python person tell me what syntax could capture

Code: Select all

s[0<x<1].hh.gnabar = f(x)
One possiblity that seems exciting to me
is the generalization to parameterized subsets

Code: Select all

cell[i].distal[all x].hh.gnabar = f(x)
I guess I need to learn more python.
if cell returns a Section object then it is fine if you write the .distal method under the name __getslice__, and you need a __setslice__ too so that you can write:

Code: Select all

section[0:1].hh.gnabar = f(0:1)
assuming f has access to section.nseg.

Of course this would not work right now since section[0:1] needs to return a list or array-like object which has a reference to the RangeVariable object. So it seems this requires us to at least discuss problem in the level of segments: a housekeeping class to manage segments need to be there.

However, what suggests to me right now is that the following might be a better grammar:

Code: Select all

conform(section.hh, "gnabar", locations, f, *args)
or you want conform to be one method of any distibute mechanism:

Code: Select all

locations = section.locations([0,1])
section.hh.conform("gnabar", locations, f, *args)
f is a reference to a function, which accept "locations"(a list of x's) and some more *args.

The reasons are:
1. mechanisms are inserted to sections instead of part of sections or segments, so RangeVariable is part of a Section. In this case we are making constraints on a RangeVariable on a section, so come the section.hh, instead of section[0:1].hh.
2. if we absolutely need to discuss problems in terms of segments, as suggested beform

eacheon
Posts: 97
Joined: Wed Jan 18, 2006 2:20 pm

Post by eacheon » Wed Jul 12, 2006 12:19 am

read another thread and know Segment are already done.

Post Reply