getNumPartitions

Class: matlab.compiler.mlspark.RDD
Package: matlab.compiler.mlspark

Return the number of partitions in an RDD

expand all in page

Syntax

numPartitions = getNumPartitions(obj)

Description

numPartitions = getNumPartitions(obj) returns the number of partitions in obj.

Input Arguments

expand all

An input RDD, specified as an RDD object.

Output Arguments

expand all

Number of partitions in the input RDD, returned as a scalar value.

Examples

expand all

Use the getNumPartitions method to return the number of partitions in an RDD.

%% Connect to Spark
sparkProp = containers.Map({'spark.executor.cores'}, {'1'});
conf = matlab.compiler.mlspark.SparkConf('AppName','myApp', ...
                        'Master','local[1]','SparkProperties',sparkProp);
sc = matlab.compiler.mlspark.SparkContext(conf);

%% getNumPartitions
inputRDD = sc.parallelize({'A','B','C','A','B'},2);
redRDD= inputRDD.map(@(x)({x,1})).reduceByKey(@(x,y)(x+y),3);
coaRDD = redRDD.coalesce(2); % {{{'B',2}},{{'C',1},{'A',2}}}*
disp(['Number of Partitions: ' num2str(coaRDD.getNumPartitions())]);

See Also

| | |

Introduced in R2016b

MATLAB Compiler Documentation

Support