Build Effective Algorithms with MapReduce

The mapreduce example files that ship with MATLAB® illustrate different programming techniques. You can use these examples as a starting point to quickly prototype similar mapreduce calculations.

Note

The associated files for these examples are all in the toolbox/matlab/demos/ folder.

Example LinkPrimary FileDescriptionNotable Programming Techniques
Find Maximum Value with MapReduceMaxMapReduceExample.mFind maximum arrival delay

One intermediate key and minimal computation.

Compute Mean Value with MapReduceMeanMapReduceExample.mFind mean arrival delay

One intermediate key with intermediate state (accumulating intermediate sum and count).

Create Histograms Using MapReduceVisualizationMapReduceExample.mVisualize data using histograms

Low-volume summaries of data, sufficient to generate a graphic and gain preliminary insights.

Compute Mean by Group Using MapReduceMeanByGroupMapReduceExample.mCompute mean arrival delay for each day of the week

Perform simple computations on subgroups of input data using several intermediate keys.

Compute Maximum Average HSV of Images with MapReduceHueSaturationValueExample.mDetermine average maximum hue, saturation, and brightness in an image collection

Analyzes an image datastore using three intermediate keys. The outputs are filenames, which can be used to view the images.

Simple Data Subsetting Using MapReduceSubsettingMapReduceExample.mCreate single table from subset of large data set

Extraction of subset of large data set to look for patterns. The procedure is generalized using a parameterized map function to pass in the subsetting criteria.

Using MapReduce to Compute Covariance and Related QuantitiesCovarianceMapReduceExample.mCompute covariance and related quantities

Calculate several intermediate values and store them with the same key. Use covariance to obtain a correlation matrix and regression coefficients, and to perform principal components analysis.

Compute Summary Statistics by Group Using MapReduceStatisticsByGroupMapReduceExample.mCompute summary statistics organized by group

Use an anonymous function to pass an extra grouping parameter to a parameterized map function. This parameterization allows you to quickly recalculate statistics using different grouping variables.

Using MapReduce to Fit a Logistic Regression ModelLogitMapReduceExample.mFit simple logistic regression model

Chain multiple mapreduce calls to carry out an iterative regression algorithm. An anonymous function passes information from one iteration to the next to supply information directly to the map function.

Tall Skinny QR (TSQR) Matrix Factorization Using MapReduceTSQRMapReduceExample.mTall skinny QR decomposition

Chain multiple mapreduce calls to perform multiple iterations of factorizations. Also use the info input argument of the map function to compute intermediate numeric keys.