How to Use Parallel Processing in Global Optimization Toolbox

Multicore Processors

If you have a multicore processor, you might see speedup using parallel processing. You can establish a parallel pool of several workers with a Parallel Computing Toolbox™ license. For a description of Parallel Computing Toolbox software, see Get Started with Parallel Computing Toolbox (Parallel Computing Toolbox).

Suppose you have a dual-core processor, and want to use parallel computing:

Enter
```
parpool
```
at the command line. MATLAB^® starts a pool of workers using the multicore processor. If you had previously set a nondefault cluster profile, you can enforce multicore (local) computing:
```
parpool('local')
```
Note
Depending on your preferences, MATLAB can start a parallel pool automatically. To enable this feature, check Automatically create a parallel pool in Home > Parallel > Parallel Preferences.

Set your solver to use parallel processing.

Solver	Command-Line Settings	Optimization App Settings
`ga`	`options = optimoptions('ga','UseParallel', true, 'UseVectorized', false);`	Options > User function evaluation > Evaluate fitness and constraint functions > in parallel
`gamultiobj`	`options = optimoptions('gamultiobj','UseParallel', true, 'UseVectorized', false);`	Options > User function evaluation > Evaluate fitness and constraint functions > in parallel
`MultiStart`	`ms = MultiStart('UseParallel', true);` or `ms.UseParallel = true`
`paretosearch`	`options = optimoptions('paretosearch','UseParallel',true);`
`particleswarm`	`options = optimoptions('particleswarm', 'UseParallel', true, 'UseVectorized', false);`
`patternsearch`	`options = optimoptions('patternsearch','UseParallel', true, 'UseCompletePoll', true, 'UseVectorized', false);`	Options > User function evaluation > Evaluate objective and constraint functions > in parallel Options > Complete poll > on
`surrogateopt`	`options = optimoptions('surrogateopt','UseParallel',true);`

Beginning in R2019a, when you set the 'UseParallel' option to true, patternsearch internally overrides the 'UseCompletePoll' setting to true so it polls in parallel.

When you run an applicable solver with options, applicable solvers automatically use parallel computing.

To stop computing optimizations in parallel, set UseParallel to false, or set the Optimization app not to compute in parallel. To halt all parallel computation, enter

delete(gcp)

Processor Network

If you have multiple processors on a network, use Parallel Computing Toolbox functions and MATLAB Parallel Server™ software to establish parallel computation. Here are the steps to take:

Make sure your system is configured properly for parallel computing. Check with your systems administrator, or refer to the Parallel Computing Toolbox documentation.
To perform a basic check:
1. At the command line, enter
  parpool(prof)
  where prof is your cluster profile.
2. Workers must be able to access your objective function file and, if applicable, your nonlinear constraint function file. There are two ways of ensuring access:
  1. Distribute the files to the workers using the parpool AttachedFiles argument. For example, if objfun.m is your objective function file, and constrfun.m is your nonlinear constraint function file, enter
    parpool('AttachedFiles',{'objfun.m','constrfun.m'});
    Workers access their own copies of the files.
  2. Give a network file path to your files. If network_file_path is the network path to your objective or constraint function files, enter
    pctRunOnAll('addpath network_file_path')
    Workers access the function files over the network.
3. Check whether a file is on the path of every worker by entering
  pctRunOnAll('which filename')
  If any worker does not have a path to the file, it reports
  filename not found.

Set your solver to use parallel processing.

Solver	Command-Line Settings	Optimization App Settings
`ga`	`options = optimoptions('ga','UseParallel', true, 'UseVectorized', false);`	Options > User function evaluation > Evaluate fitness and constraint functions > in parallel
`gamultiobj`	`options = optimoptions('gamultiobj','UseParallel', true, 'UseVectorized', false);`	Options > User function evaluation > Evaluate fitness and constraint functions > in parallel
`MultiStart`	`ms = MultiStart('UseParallel', true);` or `ms.UseParallel = true`
`paretosearch`	`options = optimoptions('paretosearch','UseParallel',true);`
`particleswarm`	`options = optimoptions('particleswarm', 'UseParallel', true, 'UseVectorized', false);`
`patternsearch`	`options = optimoptions('patternsearch','UseParallel', true, 'UseCompletePoll', true, 'UseVectorized', false);`	Options > User function evaluation > Evaluate objective and constraint functions > in parallel Options > Complete poll > on
`surrogateopt`	`options = optimoptions('surrogateopt','UseParallel',true);`

Beginning in R2019a, when you set the 'UseParallel' option to true, patternsearch internally overrides the 'UseCompletePoll' setting to true so it polls in parallel.

After you establish your parallel computing environment, applicable solvers automatically use parallel computing whenever you call them with options.

To stop computing optimizations in parallel, set UseParallel to false, or set the Optimization app not to compute in parallel. To halt all parallel computation, enter

delete(gcp)

Parallel Search Functions or Hybrid Functions

To have a patternsearch search function run in parallel, or a hybrid function for ga or simulannealbnd run in parallel, do the following.

Set up parallel processing as described in Multicore Processors or Processor Network.
Ensure that your search function or hybrid function has the conditions outlined in these sections:

patternsearch Search Function
Parallel Hybrid Functions

patternsearch Search Function

patternsearch uses a parallel search function under the following conditions:

UseCompleteSearch is true.
The search method is not @searchneldermead or custom.
If the search method is a patternsearch poll method or Latin hypercube search, UseParallel is true. Set at the command line with optimoptions:
```
options = optimoptions('patternsearch','UseParallel',true,...
    'UseCompleteSearch',true,'SearchFcn',@GPSPositiveBasis2N);
```
Or you can use the Optimization app.

If the search method is ga, the search method option has UseParallel set to true. Set at the command line with optimoptions:

iterlim = 1; % iteration limit, specifies # ga runs
gaopt = optimoptions('ga','UseParallel',true);
options = optimoptions('patternsearch','SearchFcn',...
    {@searchga,iterlim,gaopt});

In the Optimization app, first create gaopt as above, and then use these settings:

For more information about search functions, see Using a Search Method.

Parallel Hybrid Functions

ga, particleswarm, and simulannealbnd can have other solvers run after or interspersed with their iterations. These other solvers are called hybrid functions. For information on using a hybrid function with gamultiobj, see Parallel Computing with gamultiobj. Both patternsearch and fmincon can be hybrid functions. You can set options so that patternsearch runs in parallel, or fmincon estimates gradients in parallel.

Set the options for the hybrid function as described in Hybrid Function Options for ga, Hybrid Function for particleswarm, or Hybrid Function Options for simulannealbnd. To summarize:

If your hybrid function is patternsearch

Create patternsearch options:

hybridopts = optimoptions('patternsearch','UseParallel',true,...
    'UseCompletePoll',true);

Set the ga or simulannealbnd options to use patternsearch as a hybrid function:

options = optimoptions('ga','UseParallel',true); % for ga
options = optimoptions('ga',options,...
    'HybridFcn',{@patternsearch,hybridopts});
% or, for simulannealbnd:
options = optimoptions(@simulannealbnd,'HybridFcn',{@patternsearch,hybridopts});

Or use the Optimization app.

For more information on parallel patternsearch, see Pattern Search.

If your hybrid function is fmincon:

Create fmincon options:

hybridopts = optimoptions(@fmincon,'UseParallel',true,...
    'Algorithm','interior-point');
% You can use any Algorithm except trust-region-reflective

Set the ga or simulannealbnd options to use fmincon as a hybrid function:

options = optimoptions('ga','UseParallel',true);
options = optimoptions('ga',options,'HybridFcn',{@fmincon,hybridopts});
% or, for simulannealbnd:
options = optimoptions(@simulannealbnd,'HybridFcn',{@fmincon,hybridopts});

Or use the Optimization app.

For more information on parallel fmincon, see Parallel Computing (Optimization Toolbox).

Testing Parallel Optimization

To test see if a problem runs correctly in parallel,

Try your problem without parallel computation to ensure that it runs properly serially. Make sure this is successful (gives correct results) before going to the next test.
Set UseParallel to true, and ensure that there is no parallel pool using delete(gcp). Uncheck Automatically create a parallel pool in Home > Parallel > Parallel Preferences so MATLAB does not create a parallel pool . Your problem runs parfor serially, with loop iterations in reverse order from a for loop. Make sure this is successful (gives correct results) before going to the next test.
Set UseParallel to true, and create a parallel pool using parpool. Unless you have a multicore processor or a network set up, you won't see any speedup. This testing is simply to verify the correctness of the computations.

Remember to call your solver using an options argument to test or use parallel functionality.

Documentation