parallel.pool.PollableDataQueue

Class that enables sending and polling for data between client and workers

collapse all in page

Description

PollableDataQueue enables sending and polling for data or messages between workers and client in a parallel pool while a computation is being carried out. You can get intermediate values and progress of the computation.

To send data from a parallel pool worker back to the client, first construct a PollableDataQueue in the client. Pass this PollableDataQueue into a parfor-loop or other parallel language construct, such as parfeval. From the workers, call send to send data back to the client. At the client, use poll to retrieve the result of a message or data sent from a worker.

  • You can call send from the process that calls the constructor, if required.

  • You can construct the queue on the workers and send it back to the client to enable communication in the reverse direction. However, you cannot send a queue from one worker to another. Use spmd, labSend, or labReceive instead.

  • Unlike all other handle objects, PollableDataQueue instances do remain connected when they are sent to workers.

Construction

p = parallel.pool.PollableDataQueue

The constructor for a PollableDataQueue takes no arguments and returns an object that can be used to send and poll for messages (or data) from different workers. You call the constructor only in the process where you want to receive the data. In the usual workflow, the workers should not be calling the constructor, but should be handed an existing PollableDataQueue instance instead.

Properties

collapse all

Read-only property that indicates how many items of data are waiting to be removed from the queue. The value is 0 or a positive integer on the process that created the DataQueue instance. The value is 0 on all other processes.

Methods

A parallel.pool.PollableDataQueue object has the following methods.

poll Retrieve data sent from a worker
sendSend data from worker to client using a data queue

Copy Semantics

Handle. To learn how handle classes affect copy operations, see Copying Objects.

Examples

collapse all

Open Live Script

When you send a message to a PollableDataQueue object, the message waits in the queue. Each message adds 1 to the queue length. When you use poll, one message is collected from the queue. In this example, you use the QueueLength property to find the length of a PollableDataQueue object.

When a MATLAB process creates a PollableDataQueue object, any messages that are sent to the queue are held in the memory of that process. Therefore, the QueueLength property on all other processes is 0. In this example, you create a PollableDataQueue object on the client, and send data from a worker.

First, create a parallel pool with one worker.

parpool(1);
Starting parallel pool (parpool) using the 'local' profile ...
Connected to the parallel pool (number of workers: 1).

Create a PollableDataQueue.

pdq = parallel.pool.PollableDataQueue
pdq = 
  PollableDataQueue with properties:

    QueueLength: 0

A newly created PollableDataQueue has an empty queue. You can use parfor to find pdq.QueueLength on the worker. Find the queue length on the client, and the queue length on the worker.

fprintf('On the client: %i\n', pdq.QueueLength)
On the client: 0

parfor i = 1
    fprintf('On the worker: %i\n', pdq.QueueLength)
end
On the worker: 0

As the queue is empty, the QueueLength is 0 for both the client and the worker. Next, send a message to the queue. Then, use the QueueLength property to find the length of the queue.

% Send a message first
parfor i = 1
    send(pdq, 'A message');
end

% Find the length
fprintf('On the client: %i\n', pdq.QueueLength)
On the client: 1

parfor i = 1
    fprintf('On the worker: %i\n', pdq.QueueLength)
end
On the worker: 0

The QueueLength property is 1 on the client, and 0 on the worker. Use poll to retrieve the message from the queue.

msg = poll(pdq);
disp(msg)
A message

Use the QueueLength property to find the length of the queue.

fprintf('On the client: %i\n', pdq.QueueLength)
On the client: 0

QueueLength is 0 because the queue processing is complete.

Construct a PollableDataQueue.

p = parallel.pool.PollableDataQueue;

Start a parfor-loop, and send a message, such as data with the value 1.

parfor i = 1
    send(p, i); 
end

Poll for the result.

poll(p)
1

For more details on polling for data using a PollableDataQueue, see poll.

See Also

| | | | | | | |

Topics

Introduced in R2017a

Parallel Computing Toolbox Documentation

Support