Page 1 of 1

Network Parallelization

Posted: Mon May 14, 2018 9:44 am
by Krishna Chaitanya
Hi,

I have followed your example of ring network in the paper "Translating Network Models to Parallel Hardware". I see that for the NetStim connection to the first cell is made through NetCon and the rest of the cells are connected through gid_connect.

I am not sure why NetCon was used for making connection from NetStim to the first cell. If I have around 20-30 NetStims and wanted to connect to cells distributed over multiple processes. How do I achieve it? The NetStims as well as the cells are distributed over multiple processes using round robin distribution.

Can I use pc.gid_connect in this scenario? The reason I am asking is when I printed the netcons created using gid_connect, I see that some show NetCon statement with numbers and the rest of them show as None.

I don't understand if there is a mistake there.

Can you kindly let me know.

Thank you.

Re: Network Parallelization

Posted: Mon May 14, 2018 10:53 am
by ted
The NetStim is indeed a spike source, so in principle it could be assigned a gid. But before doing that, consider the three leading reasons why one would assign a gid to a spike source.
1. At least one of its targets exists on a different host.
2. Load balance.
3. Convenience of recording the events that it generates in the context of parallel simulation, i.e. if all spike sources have gids, then it is simpler to just iterate over all of them and execute nc.record(tvec, idvec, i) than to have to do something special for the spike sources that don't have gids.

Does any of these pertain to this NetStim?

It targets only one synaptic mechanism, so placing it on the same host as its target synapse avoids the need to broadcast its events to all hosts. Furthermore, a NetStim poses negligible computational overhead, so its effect on load balance is inconsequential--which means there is no need to place it on any host other than the one where its target exists. Finally, for this toy example, this NetStim's spike time is completely predetermined, so there is no need to record the time of the event that it generates.

Re: Network Parallelization

Posted: Mon May 14, 2018 4:42 pm
by Krishna Chaitanya
Hi Ted,

Thank you for the reply.

Is it possible to place source gids and Target gids on same host? I have a predetermined connection matrix. There is no problem in serial execution. But in parallel, I would have preferred a way where both source and Target gids exist on the same host. Thereby avoiding the need to search for target gid bcoz targets can only receive events from source gids on the same host. Isn't it?

Kindly let me know.

Thank you.

Re: Network Parallelization

Posted: Thu May 17, 2018 7:14 am
by Krishna Chaitanya
Hi,

I am sorry for asking this question. I didn't understand the use of pc.gid_connect properly for connecting source and target. Now I could use it to connect neurons.

When we run the network as 'python init.py' and as 'mpiexec -n 1 python -mpi init.py', should generate same number of spikes, right? In my case, the number of spikes generated by these two commands for the same networks is significantly different.

Is it correct that the spikes should be same for both the commands shown above?

Thank you.

Re: Network Parallelization

Posted: Thu May 17, 2018 8:26 am
by ted
Is it possible to place source gids and Target gids on same host?
Properly implemented parallel code should work regardless of the host on which any particular spike source or target is located. The purpose of distributing cells over multiple hosts is to achieve load balance, which is critically important for performance. A little bit of load imbalance makes all other hosts wait for the slowest one to catch up, and that can take much longer than the time required for spike exchange between hosts.

If a network combines biophysical model cells (implemented with sections, therefore need numerical integration) with event-based artificial spiking cells like NEURON's built-in IntFire classes (which do not have differential equations that require numerical integration), some network architectures might theoretically benefit by strategic placement of the artificial spiking cells. But somehow nobody seems to be implementing hybrid models for which the placement of artificial spiking cells matters much to performance.

What does matter is load balance, and that's why model setup code often includes clever strategies such as cell splitting (so that the equations that describe different parts of a biophysical model cell are solved on different processors). Another important factor is the programmer's own time and effort required to implement and debug model setup code. That's why the simplest cell distribution algorithm (so-called "round-robin"; think of dealing out cards to players) is the starting point for many network setup strategies.

Re: Network Parallelization

Posted: Thu May 17, 2018 8:40 am
by ted
When we run the network as 'python init.py' and as 'mpiexec -n 1 python -mpi init.py', should generate same number of spikes, right?
That is one indication of properly parallelized code. Another is that results produced by parallelized code should be identical to the results of a serial implementation of the same network, regardless of how many hosts are used. To help achieve this goal, it is useful to create analysis code that produces reports you can use to verify that connectivity (including weights and delays) is correct. A strategy for diagnosing and fixing particular cases is to identify the first spike that is incorrect, then verify the properties of the cell that generated that spike, and finally verify all of the projections to that cell's synaptic mechanisms (i.e. verify the identities of the spike sources that send events to the target cell, the thresholds of all of the NetCons that detect the presynaptic spikes, and the weights and delays of all of the NetCons that implement the connections).