Bulletin-Style Grid-Like Simulations: Indexing Getting Mixed Up
Posted: Sun May 12, 2019 1:23 pm
Hello,
I am doing a grid-like set of simulations on a single-neuron model (python/neuron) with increasing holding currents (i.e. using IClamp at the soma; all spike-generating current levels) and increasing synaptic input frequencies (i.e. using Exp2Syn processes at various dendritic locations). I've parallelized my code in a bulletin board style way as follows:
When I run it this way, I get some unexpected results that I would not otherwise obtain when running in serial. When looking at the baseline spike rates (i.e. without synaptic inputs - modfreqs[0]) they do not always increase gradually with increasing holding currents. When I plot out the baseline voltage traces (i.e. from the parallelized simulations), it is clear that they are from simulations that include synaptic inputs (or even from different current injections steps), indicating a mismatch between input parameters and output traces. When I print out the input parameters onto the filenames for the voltage plots that are generated, they are the correct input parameters (but of course this does not match the output that is plotted). Strangely, when printing out the lists of the input parameters (which I include as output parameters in my functions), they appear in the correct order, so the output indexing appears fine, suggesting that it has something more to do with the simulations themselves getting mixed up somehow during submission.
A few additional points:
-I run the parallel simulations on NSG, and I have received similar indexing mix-ups when running it on two different HPC resources (Stampede2 and Comet).
-The indexing mix-up is not consistent across repeated runs, and it seems like less indexing mix-ups occur when I have a closer 1:1 spread of simulations:processors (i.e. instead of multiple simulations on a smaller number of processors).
-From the troubleshooting I've done, it seems to me that the indexing mix-ups occur because of how I've parallelized my code, but I am having trouble pin-pointing where exactly I went wrong. Any input would be appreciated.
Thanks,
Alex GM
I am doing a grid-like set of simulations on a single-neuron model (python/neuron) with increasing holding currents (i.e. using IClamp at the soma; all spike-generating current levels) and increasing synaptic input frequencies (i.e. using Exp2Syn processes at various dendritic locations). I've parallelized my code in a bulletin board style way as follows:
Code: Select all
# Load the model and initiate functions
# "getMeasures" is a function written in python that calls to a hoc function to run the model and analyze the output traces from that simulation
numsteps = 50
modfreqs = numpy.array([0,0.5,1,2,3,4,5,8,9,10,12,15,16,20,25,30]) # Hz
currsteps = numpy.arange(rheobase,rheobase+0.0025*numsteps,0.0025) # nA
pc = h.ParallelContext()
pc.runworker()
if pc.nhost() == 1:
for y in range(0,numsteps):
for modfreq in range(0,len(modfreqs)):
results = getMeasures(y,modfreqs[modfreq],modfreq,currsteps[y])
# RECORD VARIABLES FROM RESULTS INTO LISTS USING "y" AND "modfreq" TO INDEX
else:
for y in range(0,numsteps):
for modfreq in range(0,len(modfreqs)):
pc.submit(getMeasures,y,modfreqs[modfreq],modfreq,currsteps[y])
while pc.working():
results = pc.pyret()
# RECORD VARIABLES FROM RESULTS INTO LISTS USING "y" AND "modfreq" TO INDEX
pc.done()
A few additional points:
-I run the parallel simulations on NSG, and I have received similar indexing mix-ups when running it on two different HPC resources (Stampede2 and Comet).
-The indexing mix-up is not consistent across repeated runs, and it seems like less indexing mix-ups occur when I have a closer 1:1 spread of simulations:processors (i.e. instead of multiple simulations on a smaller number of processors).
-From the troubleshooting I've done, it seems to me that the indexing mix-ups occur because of how I've parallelized my code, but I am having trouble pin-pointing where exactly I went wrong. Any input would be appreciated.
Thanks,
Alex GM