Native Mac OS X 15x slower than virtualized Windows

[bsuter]

Hi,
I've been using NEURON 7.1 in a virtualized Windows 7 environment (in VMWare Fusion, on a Macbook Pro running Mac OS X 10.6.3, with a 2.8 GHz Intel Core 2 Duo processor and 4GB of RAM). Initially, my reason for running in virtualized Windows was for compatibility with a colleague who runs NEURON under native Windows XP - I was learning from him, and wanted to avoid any OS-related differences. I'd prefer to run NEURON in my native OS, namely OS X 10.6, in part to avoid virtualization overhead, but mainly so I can take advantage of both cores (e.g. when running MultiRunFitter).

Paradoxically, running the same simulation under native OS X took about 15x longer to complete than under virtualized Windows. It took 24 sec virtualized, and 386 sec natively. I then closed two state-variable graphs and three mechanism parameter panels that were updating continuously, and reran the simulation (there was still one graph open). It took 11.5 sec virtualized, and 178 sec natively. The simulation is variable time-step using CVODE.

I'd greatly appreciate any help in figuring out how to improve the native Mac performance / figure out what's going on in my case. If there are any benchmarking scripts, I'd be happy to try them (also to supply my model files).

Thank you,
Ben.

[ted]

When a very unexpected result occurs, the first thing to do is to repeat the test--maybe at a different time of day, and even on a different day. It is possible that, during the "native OS" test, your Mac was busy with some other task(s) that chewed up cpu cycles, e.g. backup, downloading updates, prelinking. It might be informative to open a terminal and start top (see http://www.unixtop.org/top1.html) before running your tests, so you can discover what else is going on.

[bsuter]

Hi Ted,
Thank you for your quick response. I've only just seen it now, as I was expecting to receive an email notification of changes to the thread, but I did not. In my account, I had selected "yes" for the "Notify me upon replies by default" setting, and thought this was adequate. I've now also subscribed to the topic, so we'll see if that works.

I had experienced this same problem two weeks ago before submitting my post. I've run the same simulation again today, and it took 13 seconds virtualized, and 177 seconds natively, very close to the numbers I saw two days ago. I ran top as you suggested, and before starting the simulation, CPU was mostly idle. During the simulation, the nrniv process showed about 17% CPU and remained at or near the top of the list. Any particular fields I should watch in top? What expected pattern should I look for?

Thank you,
Ben.

[bsuter]

I reran the simulation after closing most user programs, and it took 189 seconds. Then I reran the simulation after closing all user programs, including this browser, and turning off WiFi, and again it took 178 seconds to complete natively. CPU usage reached 20% in top for the nrniv process. I'm launching the simulation by clicking on the .hoc file, in both the native and virtualized case.

[bsuter]

Progress! But still mysterious. In short: closing the RunControl window reduced the simulation time to 28 seconds (from about 180). Other changes to my hoc code reduced the simulation time to 11 seconds (with RunControl) or 6 seconds (without RunControl).

1a. For the previously reported simulations, I had already closed all NEURON windows except for the standard menu bar, RunControl, a graph, and the console. This took about three minutes natively. More details on the graph below.

1b. Relative to (1a), clicking "Init", then closing RunControl, starting the simulation by run() at console, then inspecting realtime at console: 28 seconds.

1c. Relative to (1a), changing the screen update interval (s) from 0.05 to 1 brought the time down to 121 seconds. Relative to (1b), changing the screen update interval had no effect (still 28 seconds).

2. I was curious how/why the RunControl window could have such a significant influence on total run time, and suspected it had something to do with updating the various GUI fields (such as Real Time, and t). This is where the story gets a bit complicated. The graph displayed data from an experiment in red, and soma.v(0.5) was redrawn over this with every run in black, for comparison. Experimental data was added via Graph.vector() and simulated potential via Graph.addvar(). The graph was added to a VBox, and it was NOT added to the standard run system via addplot(), but instead a custom advance() and init() procedure looked like this (where "f" is a Graph objvar):

Code: Select all

proc advance() {
  fadvance()
  f.plot(t)
  f.flush()
  doNotify()
}

proc init() {
  // omitting code that sets distributed mechanism parameters, 
  // which I'm trying to modify to fit the experiment

  finitialize(v)
  fcurrent()
  finitialize(v)
  cvode.re_init()
  f.begin()
}

This approach was something I inherited from someone introducing me to NEURON, and it worked so I kept it. I've been reading chapters 7 and 8 in the book to better understand the standard run system - I have a ways to go still. My sense is that this particular way of controlling the simulation and plotting the graph somehow interacted with RunControl to result in slower performance (e.g. 180 seconds). Again, simply closing RunControl resulted in a significant speed-up (e.g. 28 seconds).

3. I modified my .hoc code: The Graph is no longer in a VBox, I now do call addplot(f, 0), and the experimental data is added via Vector.plot(f, ...). I've also removed the advance() procedure entirely, and modified the init() procedure so it now reads:

Code: Select all

proc init() {
  // omitting code that sets distributed mechanism parameters, 
  // which I'm trying to modify to fit the experiment

  finitialize(v)
  fcurrent()
  finitialize(v)
  cvode.re_init()
}

Is this optimal, or can it be further improved? I've yet to complete chapters 7 and 8, so hope to come up with an answer myself in time. As it stands, the simulation takes 11 seconds to complete with RunControl open, and 5.7 seconds to complete with RunControl closed. Using top, I noticed that the nrniv process reaches 58% CPU with RunControl open, and 100% CPU with RunControl closed - as if having RunControl open prevents full CPU utilization.

So I'm still curious: Why does closing RunControl cut the time in half now (11 to 5.7)? Why did it cut the time by 6x with the previous code (178 to 28)? What about the previous code makes it slower than the updated code (28 to 5.7 seconds)?

Thank you!

[ted]

The purpose of the test with top was to see if something else was chewing up a lot of cpu time--if there were such a thing, it would have appeared near the top of the list of processes, with a big number in the %CPU column. So it appears that there is not.

The problem illustrates the perils of legacy code and the dangers of tinkering with the run time system. The slow code is probably trying to redraw Vectors and Graphs, and who knows what else, at every time step.

All you want to do is to show experimental data in the same graph as simulation results? Revise your program so that it waits until the simulation is complete, then uses the Vector class's plot() method to draw the experimental data on the graph that shows soma.v(0.5). Here's how.

1. Discard your custom proc init and advance.
2. Identify the name of the Graph object to which you will plot soma.v(0.5). In the following discussion, I'm going to assume that the objref for this Graph is g.
3. The code to set up g should look like this:

Code: Select all

objref g
g = new Graph()
soma g.addvar("v(0.5)")
g.addplot(0) // adds g to graphList[0]

g.addplot[0] ensures that, during a simulation, updated v will be plotted at integer multiples of dt. No need to tinker with the guts of the run time system.

If you want to specify exactly where g is drawn on the screen, and its axis scaling, read How to use hoc to plot a variable vs. time in the NEURON hacks area of the NEURON Forum. You'll need

OK, now you have a graph that shows v in the course of a simulation. How do you plot experimental data on that graph? Presumably you have a proc that reads the experimental data from a file into a Vector, or maybe a pair of Vectors if the sampling rate is different from the dt used in your simulation. Here I will assume the latter--that you have a tvec and a vvec, whose elements are the (t, v) pairs that you want to plot.
vvec.plot(g, tvec, 2, 1) // color 2 (red as I recall), brush 1 (linewidth 1 point)
will do it.

But when do you call vvec.plot? I'd wait until after the simulation stops*, then plot the data, by doing this:

1. Create a proc called myrun and a proc called plotdata()

Code: Select all

objref vvec, tvec

proc plotdata() {
  vvec = new Vector()
  tvec = new Vector()
  // . . . some code that reads the data from a file into vvec and tvec . . .
  vvec.plot(g, tvec, 2, 1) // color 2 (red), brush 1 (linewidth 1 point)
}

proc myrun() {
  run()
  plotdata()
}

2. If you want the RunControl panel's Init & Run button to make this happen, edit the ses file that recreates that panel and replace this line
xbutton("Init & Run","run()")
with
// xbutton("Init & Run","run()")
xbutton("Init & Run","myrun()")

If you want to generate a series of simulations, each with a different set of experimental data, you might want to insert this line at the beginning of proc myrun()
g.exec_menu("Erase") // erase experimental data, if any


*--just in case the time required for screen updates depends on the number of points that have been plotted.

[bsuter]

Hi Ted,
Thank you for your explanations and suggestions. My code no longer modifies any of the standard run procedures, I am plotting per your suggestion, and I've modified the mechanisms slightly to be thread-safe, allowing use of both processor cores under native Mac OS X. Things are pretty fast now.

However, and I mention this just because I find it curious (and to be honest, it seems like it could affect other users): a simulation run that takes >30 seconds with the RunControl window open, takes <13 seconds simply by closing the RunControl window and initiating the simulation from the console. So it still seems that the RunControl window's presence significantly slows the simulation.

Thanks,
Ben.

[hines]

I've also been disappointed with the Mac gui performance with Carbon. Carbon is no longer supported by Apple
and there is no 64bit version of it for Snow Leopard. Best performance is probably obtained by building with
x11. x11 has always been available for the os x but, at least for snowleopard, now seems to be installed
when DevloperTools is installed.
Anyway, when I test with the neurondemo, select pyramid, and change tstop to 1000, on my machine the gui
under carbon does not slow things significantly when I use a screen update interval of 1.