%% Use Multiple GPUs in a Parallel Pool
% If you have access to several GPUs, you can perform your calculations on multiple GPUs in parallel using a parallel pool.
% Start a parallel pool with as many workers as GPUs. To determine the number of GPUs available, use the gpuDeviceCount function. By default, MATLAB assigns a different GPU to each worker for best performance.

parpool('local',gpuDeviceCount);

% To identify which GPU each worker is using, call gpuDevice inside an spmd block. The spmd block runs gpuDevice on every worker.

spmd
    gpuDevice
end

% Use parallel language features, such as parfor or parfeval, to distribute your computations to workers in the parallel pool. If you use gpuArray enabled functions in your computations, these functions run on the GPU of the worker. For more information, see Run MATLAB Functions on a GPU. For an example, see Run MATLAB Functions on Multiple GPUs.
% When you are done with your computations, shut down the parallel pool. You can use the gcp function to obtain the current parallel pool.

delete(gcp('nocreate'));

% If you want to use a different choice of GPUs, then can use gpuDevice to select a particular GPU on each worker. Define an array, for example gpuIndices, that contains the indices of the GPUs to activate on each worker. Then, start a parallel pool with as many workers as GPUs to select, and use an spmd block to run gpuDevice on each worker. The labindex function identifies each worker. Use this function to associate a worker with a GPU index.

gpuIndices = [1 3];
parpool(numel(gpuIndices));
spmd
    gpuDevice(gpuIndices(labindex));
end

% As a best practice, and for best performance, assign a different GPU to each worker.
%
% Copyright 2020 The MathWorks, Inc.