checkpoint

Class: matlab.compiler.mlspark.RDD
Package: matlab.compiler.mlspark

Mark an RDD for checkpointing

expand all in page

Syntax

checkpoint(obj)

Description

checkpoint(obj) marks an RDD for checkpointing.

Input Arguments

expand all

An input RDD that is to be marked for checkpointing, specified as an RDD object.

Examples

expand all

Use the checkpoint method to save an RDD to a file inside the checkpoint directory.

%% Connect to Spark
sparkProp = containers.Map({'spark.executor.cores'}, {'1'});
conf = matlab.compiler.mlspark.SparkConf('AppName','myApp', ...
                        'Master','local[1]','SparkProperties',sparkProp);
sc = matlab.compiler.mlspark.SparkContext(conf);

%% checkpoint
sc.setCheckpointDir('myDir')
myFile = sc.parallelize({1,2,3});
mapRDD = myFile.map(@(x)({x,1}));
mapRDD.checkpoint();

See Also

| | |

Introduced in R2016b

MATLAB Compiler Documentation
Support