getDefaultReducePartitions

Class: matlab.compiler.mlspark.RDD
Package: matlab.compiler.mlspark

Get the number of default reduce partitions in an RDD

expand all in page

Syntax

numPartitions = getDefaultReducePartitions(obj)

Description

numPartitions = getDefaultReducePartitions(obj) gets the number of default reduce partitions in obj.

Input Arguments

expand all

An input RDD, specified as an RDD object

Output Arguments

expand all

The number of default reduce partitions in the input RDD, returned as a scalar value.

Examples

expand all

Get the number of default reduce partitions in an RDD.

%% Connect to Spark
sparkProp = containers.Map({'spark.executor.cores'}, {'1'});
conf = matlab.compiler.mlspark.SparkConf('AppName','myApp', ...
                        'Master','local[1]','SparkProperties',sparkProp);
sc = matlab.compiler.mlspark.SparkContext(conf);

%% getDefaultReducePartitions
x = sc.parallelize({1,2,3});
y = x.map(@(x)({x,1}));
z1 = y.reduceByKey(@(a,b)(a+b));
z2 = y.reduceByKey(@(a,b)(a+b), 3);

z1.getDefaultReducePartitions() % ans is 1
z2.getDefaultReducePartitions() % ans is 3, as the 2nd argument to reduceByKey is the number of reduce partitions

See Also

| |

Introduced in R2016b

MATLAB Compiler Documentation

Support