HowTos / Compute

Using the cluster with the Matlab Parallel Toolbox

Cluster machines with matlab10, which has the parallel toolbox:

  • herbie
  • metheny
  • pat
  • lena
  • sarah
  • martino
  • oscar
  • ronnie
  • nina

From these machines, you can use the parallel toolbox to use the Distributed Computing Server (DCS), which allows you to use multiple cores or machines from the cluster for one job. If your own computer has the parallel toolbox, you can set it up to use the cluster as well, but you must be in the lab, i.e. behind the firewall, in order to use it.

Batch jobs can be submitted to the cluster using Moab, but Matlab configuration has to be setup for the cluster first:

Setting up matlabpool for DCS cluster:

  • if on cluster, from terminal start matlab10 --> duck@herbie:~$ matlab10 &
  • go to the Parallel menu and click on Manage Configurations
  • go to File > New > torque
  • The Properties Editor will open:
  • Configuration name and description are just for your use, so just use something convenient, defaults are fine
  • Root folder of MATLAB installation for workers: choose <Custom Value> and then edit to read /usr/local/matlab10
  • Number of workers available to scheduler: not necessary. Fyi, currently there are 107 workers, but only 16 can be used at one time by Matlab through Moab
  • Folder where job data is stored: path to the directory you run matlab from (typically /home/userid/matlab)
  • Resource list parameter (Resource Template): Optional. This is where you fill out any of the Moab parameters you want to use as defaults. Leave with Set to empty to use the Moab defaults.
  • Job data location is accessible from both client and cluster nodes: True.
  • Script that scheduler runs to start labs and Remote copy command for nonshared file systems should be left blank.
  • Remote shell command to call on UNIX when running parallel jobs: ssh
  • Now click on the jobs tab in the same window.
  • The files and paths can be left blank.
  • The maximum number of workers that can run job: Must be 16 or less
  • Minimum number of workers required to run job: 1 or more
  • Click OK

That should complete configuration. Now you can test:

  • click on Start Validation at the bottom right to make sure everything is working fine (won't work from a laptop unless you are in the lab, behind the firewall).

You now have 16 workers with 107 cores at your disposal.

The command matlabpool will start a worker pool using the default configuration with the pool size specified by that configuration. You can also specify the pool size using matlabpool open <poolsize>

Good introductions to DCS and parallel toolbox:

Files given to us in the presentation by Sarah and Angelo from Mathworks: http://www.shadlen.org/matlab/parallel_web_stuff

I ran the PhoneTowerScript.m from the parallelOptim folder from the above directory. I ran it first without matlabpool, and then ran matlabpool (using the DCS cluster as the pool) and ran it again. The process took 52 seconds on nina without the dcs pool, and 20 seconds with the pool.