Parallelization and esfit

[arden13]

I've been experimenting with running simulations in parallel and have found (unsurprisingly) a significant increase in speed. If I'm doing "manual fitting" i.e. exploring a single parameter space in a for loop I will run the sims in parallel. i.e.

Code: Select all

%Generic Experimental Parameters
Exp.mwFreq=9.6;
Exp.Range=[200 400];
Exp.nPoints=1000;

%Build the Structure space to be explored, in this case g(3)
Sys=cell{100,1};
for i=1:100
     Sys{i}.S=1/2;
     Sys{i}.g=[2, 2, 2+0.01*i];
     Sys{i}.lw=0.5;
end

%Start the pool of workers and run the sims
parpool(4) %starts parallel pool with 4 workers
p=cell(100,1) %These will be the job handles. works with parfeval but not parfevalOnAll

for i=1:100
     p{i}=parfeval(gcp,@pepper,2,Sys{i},Exp);
end

%Fetch all of the data in Spec and plot
Spec=zeros(100,1000);
for i=1:100
     [B,Spec(i,:)]=fetchOutputs(p{i});
     hold on;
     plot(B,Spec(i,:)/max(Spec(i,:))+0.1*i);  %Scale to 1 and stack the spectra for easier visualization
end

This works really well for exploring whatever parameter space you want. I've tried using a parfor loop as well, but I like the ability to check to see when a single job is done. Additionally a parfor loop means you can't use the main command window of MATLAB while the loop is running, which you can do with parfeval (just limit the number of workers so your CPU isn't at 100%!).

With esfit I can't take as much advantage of the parallelization in MATLAB. I can run multiple fits at a time on different workers, but other than the ability to run two fits at once its not all that beneficial. I wouldn't think it would be too difficult to implement this kind of parallelization in esfit, as you could just run your pepper/garlic/etc in parallel then calculate the residuals as the outputs become available. As I imagine it, it would just be another option available in the FitOpt structure, i.e.

Code: Select all

%Assume that Spec, Sys, Vary, Exp and pepOpt are all accounted for already
FitOpt.Method = 'simplex';
FitOpt.Parallel = 4; %Number of Workers to be started in the pool

esfit('pepper',Spec,Sys,Vary,Exp,pepOpt,FitOpt);

I'm running MATLAB 2014a, but parfeval was first implemented in 2013b. The parfor command has been available since MATLAB 2006b.

Thoughts?

[Matt Krzyaniak]

Well technically you can run optimizations in parallel, you just can't use esfit. You would just need to find an appropriate minimization routine that runs in parallel and design a function that searches the parameter space you desire.

Though with respect to parallelization larger speed gains might be achieved by putting it in the simulation functions themselves. Plus, then you get the advantages all the time and not just when you're running esfit.

[Morgan Bye]

Parallelization in MatLab has been a tricky beast for many years and I've read countless discussion pages complaining how MatLab doesnt automatically distribute tasks amongst available resources (ie different threads of a single processor, or dual core Xeon rigs).

Sticking the esfit command (wo/GUI) into a parallel session would be trivial, and you've probably already done yourself.

For ES to implement it would require in theory, a switch based around getNumberOfComputationalThreads (-1 but >0) command with the number being used in the parallel loop call.

The problem I would foresee is trying to bring all of the data back in a meaningful way that the user could use without a coding nightmare.

[arden13]

Well technically you can run optimizations in parallel, you just can't use esfit. You would just need to find an appropriate minimization routine that runs in parallel and design a function that searches the parameter space you desire.

Though with respect to parallelization larger speed gains might be achieved by putting it in the simulation functions themselves. Plus, then you get the advantages all the time and not just when you're running esfit.

Putting parallelization in the sims itself would be pretty awesome, but that sounds like significantly more work than just running each individual pepper/garlic/saffron/etc. simulation and having esfit do the fitting algorithm for you. I was more suggesting it as an early stage of putting parallelization into EasySpin with the idea of later on moving towards parallel coding in pepper/garlic/etc.

Parallelization in MatLab has been a tricky beast for many years and I've read countless discussion pages complaining how MatLab doesnt automatically distribute tasks amongst available resources (ie different threads of a single processor, or dual core Xeon rigs).

Sticking the esfit command (wo/GUI) into a parallel session would be trivial, and you've probably already done yourself.

For ES to implement it would require in theory, a switch based around getNumberOfComputationalThreads (-1 but >0) command with the number being used in the parallel loop call.

The problem I would foresee is trying to bring all of the data back in a meaningful way that the user could use without a coding nightmare.

I've been putting all of my esfit commands into a parallel worker lately. That way I can run a few fits at once and still have my main MATLAB command window available for low level tasks like plotting.

[Stefan Stoll]

Putting some degree of parallelization into ES has been on the to-do list for quite a while. The downside would be that it will require an additional toolbox. I am trying to keep ES running with the (admittedly somewhat feature-anemic) Matlab core.

[Matt Krzyaniak]

There is also the potential for offloading some of the computation to the videocard(nvidia CUDA) and parallelization that you gain from doing that, again though, if you keep it all in Matlab that's more toolboxes.(unless the core functions are rewritten in Fortran or C++ and run in Matlab through MEX files but that adds complexity in terms of compilation)

[arden13]

Putting some degree of parallelization into ES has been on the to-do list for quite a while. The downside would be that it will require an additional toolbox. I am trying to keep ES running with the (admittedly somewhat feature-anemic) Matlab core.

I'm sure if I had access to the esfit code I could knock out a rough implementation of parallelization in esfit (unofficially of course). Maybe in due time pepper or some other function could be done too!

There is also the potential for offloading some of the computation to the videocard(nvidia CUDA) and parallelization that you gain from doing that, again though, if you keep it all in Matlab that's more toolboxes.(unless the core functions are rewritten in Fortran or C++ and run in Matlab through MEX files but that adds complexity in terms of compilation)

Doing CUDA parallelization would be awesome, but (in my case) I only have an intel integrated graphics card which would be significantly less powerful than a true dedicated video card. However for real computers or dedicated computation boxes that would be slick-as-spit.