How to Use Parallel Processing in Global Optimization Toolbox

Multicore Processors

If you have a multicore processor, you can increase processing speed by using parallel processing. You can establish a parallel pool of several workers with a Parallel Computing Toolbox™ license. For a description of Parallel Computing Toolbox software, see Get Started with Parallel Computing Toolbox (Parallel Computing Toolbox).

Suppose you have a dual-core processor, and want to use parallel computing. Enter this code at the command line.

parpool

MATLAB^® starts a pool of workers using the multicore processor. If you previously set a nondefault cluster profile, you can enforce multicore (local) computing by entering this code.

parpool('local')

Note

Depending on your preferences, MATLAB can start a parallel pool automatically. To enable this feature, select Parallel > Parallel Preferences in the Environment group on the Home tab, and then select Automatically create a parallel pool.

Set your solver to use parallel processing.

Solver	Command-Line Settings
`ga`	`options = optimoptions('ga','UseParallel', true, 'UseVectorized', false);`
`gamultiobj`	`options = optimoptions('gamultiobj','UseParallel', true, 'UseVectorized', false);`
`MultiStart`	`ms = MultiStart('UseParallel', true);` or `ms.UseParallel = true`
`paretosearch`	`options = optimoptions('paretosearch','UseParallel',true);`
`particleswarm`	`options = optimoptions('particleswarm', 'UseParallel', true, 'UseVectorized', false);`
`patternsearch`	`options = optimoptions('patternsearch','UseParallel', true, 'UseCompletePoll', true, 'UseVectorized', false);`
`surrogateopt`	`options = optimoptions('surrogateopt','UseParallel',true);`

Beginning in R2019a, when you set the UseParallel option to true, patternsearch internally overrides the UseCompletePoll setting to true so it polls in parallel.

When you run an applicable solver with options, applicable solvers automatically use parallel computing.

To stop computing optimizations in parallel, set UseParallel to false. To halt all parallel computation, enter this code.

delete(gcp)

Note

The documentation recommends not to use parfor or parfeval when calling Simulink^®; see Using sim function within parfor (Simulink). Therefore, you might encounter issues when optimizing a Simulink simulation in parallel using a solver's built-in parallel functionality.

Processor Network

If you have multiple processors on a network, use Parallel Computing Toolbox functions and MATLAB Parallel Server™ software to establish parallel computation.

Make sure your system is configured properly for parallel computing. Check with your systems administrator, or refer to the Parallel Computing Toolbox documentation.

Perform a basic check by entering this code, where prof is your cluster profile.
```
parpool(prof)
```
Workers must be able to access your objective function file and, if applicable, your nonlinear constraint function file. Complete one of these steps to ensure access:
- Distribute the files to the workers using the parpool (Parallel Computing Toolbox) AttachedFiles argument. In this example, objfun.m is your objective function file, and constrfun.m is your nonlinear constraint function file.
  parpool('AttachedFiles',{'objfun.m','constrfun.m'});
  Workers access their own copies of the files.
- Give a network file path to your objective or constraint function files.
  pctRunOnAll('addpath network_file_path')
  Workers access the function files over the network.
Check whether a file is on the path of every worker.
```
pctRunOnAll('which filename')
```
If any worker does not have a path to the file, it reports
```
filename not found.
```

Set your solver to use parallel processing.

Solver	Command-Line Settings
`ga`	`options = optimoptions('ga','UseParallel', true, 'UseVectorized', false);`
`gamultiobj`	`options = optimoptions('gamultiobj','UseParallel', true, 'UseVectorized', false);`
`MultiStart`	`ms = MultiStart('UseParallel', true);` or `ms.UseParallel = true`
`paretosearch`	`options = optimoptions('paretosearch','UseParallel',true);`
`particleswarm`	`options = optimoptions('particleswarm', 'UseParallel', true, 'UseVectorized', false);`
`patternsearch`	`options = optimoptions('patternsearch','UseParallel', true, 'UseCompletePoll', true, 'UseVectorized', false);`
`surrogateopt`	`options = optimoptions('surrogateopt','UseParallel',true);`

Beginning in R2019a, when you set the UseParallel option to true, patternsearch internally overrides the UseCompletePoll setting to true so it polls in parallel.

After you establish your parallel computing environment, applicable solvers automatically use parallel computing whenever you call them with options.

To stop computing optimizations in parallel, set UseParallel to false. To halt all parallel computation, enter this code.

delete(gcp)

Note

The documentation recommends not to use parfor or parfeval when calling Simulink; see Using sim function within parfor (Simulink). Therefore, you might encounter issues when optimizing a Simulink simulation in parallel using a solver's built-in parallel functionality.

Parallel Search Functions or Hybrid Functions

To have a patternsearch search function run in parallel, or a hybrid function for ga or simulannealbnd run in parallel, do the following.

Set up parallel processing as described in Multicore Processors or Processor Network.
Ensure that your search function or hybrid function has the conditions outlined in these sections:

patternsearch Search Function
Parallel Hybrid Functions

patternsearch Search Function

patternsearch uses a parallel search function under the following conditions:

UseCompleteSearch is true.
The search method is not @searchneldermead or custom.
If the search method is a patternsearch poll method or Latin hypercube search, UseParallel is true. Set at the command line with optimoptions:
```
options = optimoptions('patternsearch','UseParallel',true,...
    'UseCompleteSearch',true,'SearchFcn',@GPSPositiveBasis2N);
```

If the search method is ga, the search method option has UseParallel set to true. Set at the command line with optimoptions:

iterlim = 1; % iteration limit, specifies # ga runs
gaopt = optimoptions('ga','UseParallel',true);
options = optimoptions('patternsearch','SearchFcn',...
    {@searchga,iterlim,gaopt});

For more information about search options, see Search Options. For an example, see Search and Poll.

Parallel Hybrid Functions

ga, particleswarm, and simulannealbnd can have other solvers run after or interspersed with their iterations. These other solvers are called hybrid functions. For information on using a hybrid function with gamultiobj, see Parallel Computing with gamultiobj. Both patternsearch and fmincon can be hybrid functions. You can set options so that patternsearch runs in parallel, or fmincon estimates gradients in parallel.

Set the options for the hybrid function as described in Hybrid Function Options for ga, Hybrid Function for particleswarm, or Hybrid Function Options for simulannealbnd. To summarize:

If your hybrid function is patternsearch

Create patternsearch options:

hybridopts = optimoptions('patternsearch','UseParallel',true,...
    'UseCompletePoll',true);

Set the ga or simulannealbnd options to use patternsearch as a hybrid function:

options = optimoptions('ga','UseParallel',true); % for ga
options = optimoptions('ga',options,...
    'HybridFcn',{@patternsearch,hybridopts});
% or, for simulannealbnd:
options = optimoptions(@simulannealbnd,'HybridFcn',{@patternsearch,hybridopts});

For more information on parallel patternsearch, see Pattern Search.

If your hybrid function is fmincon:

Create fmincon options:

hybridopts = optimoptions(@fmincon,'UseParallel',true,...
    'Algorithm','interior-point');
% You can use any Algorithm except trust-region-reflective

Set the ga or simulannealbnd options to use fmincon as a hybrid function:

options = optimoptions('ga','UseParallel',true);
options = optimoptions('ga',options,'HybridFcn',{@fmincon,hybridopts});
% or, for simulannealbnd:
options = optimoptions(@simulannealbnd,'HybridFcn',{@fmincon,hybridopts});

For more information on parallel fmincon, see Parallel Computing.

Testing Parallel Optimization

Follow these steps to test whether your problem runs correctly in parallel.

Try your problem without parallel computation to ensure that it runs serially. Make sure this test is successful (gives correct results) before going to the next test.
Set UseParallel to true, and ensure that no parallel pool exists by entering delete(gcp). To make sure that MATLAB does not create a parallel pool, select Parallel > Parallel Preferences in the Environment group on the Home tab, and then clear Automatically create a parallel pool. Your problem runs parfor serially, with loop iterations in reverse order from a for loop. Make sure this test is successful (gives correct results) before going to the next test.
Set UseParallel to true, and create a parallel pool using parpool. Unless you have a multicore processor or a network set up, this test does not increase processing speed. This testing is simply to verify the correctness of the computations.

Remember to call your solver using an options argument to test or use parallel functionality.

Documentation