Number of datastore partitions
n = numpartitions(ds)
n = numpartitions(ds,pool)
example
n = numpartitions(ds) returns the default number of partitions for datastore ds.
ds
n = numpartitions(ds,pool) returns a number of partitions to parallelize datastore access over the parallel pool specified by pool. To parallelize datastore access, you must have Parallel Computing Toolbox™ installed.
pool
collapse all
Create a datastore from the sample file, mapredout.mat, which is the output file of the mapreduce function.
mapredout.mat
mapreduce
ds = datastore('mapredout.mat');
Get the default number of partitions.
n = 1
By default, there is only one partition in ds because it contains only one small file.
Partition the datastore and return the datastore corresponding to the first part.
subds = partition(ds,n,1);
Read the data in subds.
subds
while hasdata(subds) data = read(subds); end
Get a number of partitions to parallelize datastore access over the current parallel pool. You must have Parallel Computing Toolbox installed.
Get a number of partitions to parallelize datastore access over the current parallel pool.
n = numpartitions(ds, gcp);
Partition the datastore and read the data in each part.
parfor ii=1:n subds = partition(ds,n,ii); while hasdata(subds) data = read(subds); end end
Input datastore. You can use the datastore function to create a datastore object from your data.
datastore
Parallel pool object.
Example: gcp
gcp
datastore | partition
partition
You have a modified version of this example. Do you want to open this example with your edits?