ClassificationECOC

Multiclass model for support vector machines (SVMs) and other classifiers

Description

ClassificationECOC is an error-correcting output codes (ECOC) classifier for multiclass learning, where the classifier consists of multiple binary learners such as support vector machines (SVMs). Trained ClassificationECOC classifiers store training data, parameter values, prior probabilities, and coding matrices. Use these classifiers to perform tasks such as predicting labels or posterior probabilities for new data (see predict).

Creation

Create a ClassificationECOC object by using fitcecoc.

If you specify linear or kernel binary learners without specifying cross-validation options, then fitcecoc returns a CompactClassificationECOC object instead.

Properties

expand all

After you create a ClassificationECOC model object, you can use dot notation to access its properties. For an example, see Train Multiclass Model Using SVM Learners.

ECOC Properties

`BinaryLearners` — Trained binary learners
cell vector of model objects

Trained binary learners, specified as a cell vector of model objects. The number of binary learners depends on the number of classes in Y and the coding design.

The software trains BinaryLearner{j} according to the binary problem specified by CodingMatrix(:,j). For example, for multiclass learning using SVM learners, each element of BinaryLearners is a CompactClassificationSVM classifier.

Data Types: cell

`BinaryLoss` — Binary learner loss function
`'binodeviance'` | `'exponential'` | `'hamming'` | `'hinge'` | `'linear'` | `'logit'` | `'quadratic'`

Binary learner loss function, specified as a character vector representing the loss function name.

If you train using binary learners that use different loss functions, then the software sets BinaryLoss to 'hamming'. To potentially increase accuracy, specify a binary loss function other than the default during a prediction or loss computation by using the 'BinaryLoss' name-value pair argument of predict or loss.

Data Types: char

`BinaryY` — Binary learner class labels
numeric matrix

Binary learner class labels, specified as a numeric matrix. BinaryY is a NumObservations-by-L matrix, where L is the number of binary learners (length(Mdl.BinaryLearners)).

Elements of BinaryY are –1, 0, or 1, and the value corresponds to a dichotomous class assignment. This table describes how learner j assigns observation k to a dichotomous class corresponding to the value of BinaryY(k,j).

Value	Dichotomous Class Assignment
`–1`	Learner `j` assigns observation `k` to a negative class.
`0`	Before training, learner `j` removes observation `k` from the data set.
`1`	Learner `j` assigns observation `k` to a positive class.

Data Types: double

`BinEdges` — Bin edges for numeric predictors
cell array of numeric vectors | `[]`

Bin edges for numeric predictors, specified as a cell array of p numeric vectors, where p is the number of predictors. Each vector includes the bin edges for a numeric predictor. The element in the cell array for a categorical predictor is empty because the software does not bin categorical predictors.

The software bins numeric predictors only if you specify the 'NumBins' name-value pair argument as a positive integer scalar when training a model with tree learners. The BinEdges property is empty if the 'NumBins' value is empty (default).

You can reproduce the binned predictor data Xbinned by using the BinEdges property of the trained model mdl.

X = mdl.X; % Predictor data
Xbinned = zeros(size(X));
edges = mdl.BinEdges;
% Find indices of binned predictors.
idxNumeric = find(~cellfun(@isempty,edges));
if iscolumn(idxNumeric)
    idxNumeric = idxNumeric';
end
for j = idxNumeric 
    x = X(:,j);
    % Convert x to array if x is a table.
    if istable(x) 
        x = table2array(x);
    end
    % Group x into bins by using the discretize function.
    xbinned = discretize(x,[-inf; edges{j}; inf]); 
    Xbinned(:,j) = xbinned;
end

Xbinned contains the bin indices, ranging from 1 to the number of bins, for numeric predictors. Xbinned values are 0 for categorical predictors. If X contains NaNs, then the corresponding Xbinned values are NaNs.

Data Types: cell

`CodingMatrix` — Class assignment codes
numeric matrix

Class assignment codes for the binary learners, specified as a numeric matrix. CodingMatrix is a K-by-L matrix, where K is the number of classes and L is the number of binary learners.

The elements of CodingMatrix are –1, 0, or 1, and the values correspond to dichotomous class assignments. This table describes how learner j assigns observations in class i to a dichotomous class corresponding to the value of CodingMatrix(i,j).

Value	Dichotomous Class Assignment
`–1`	Learner `j` assigns observations in class `i` to a negative class.
`0`	Before training, learner `j` removes observations in class `i` from the data set.
`1`	Learner `j` assigns observations in class `i` to a positive class.

Data Types: double | single | int8 | int16 | int32 | int64

`CodingName` — Coding design name
character vector

Coding design name, specified as a character vector. For more details, see Coding Design.

Data Types: char

`LearnerWeights` — Binary learner weights
numeric row vector

Binary learner weights, specified as a numeric row vector. The length of LeanerWeights is equal to the number of binary learners (length(Mdl.BinaryLearners)).

LearnerWeights(j) is the sum of the observation weights that binary learner j uses to train its classifier.

The software uses LearnerWeights to fit posterior probabilities by minimizing the Kullback-Leibler divergence. The software ignores LearnerWeights when it uses the quadratic programming method of estimating posterior probabilities.

Data Types: double | single

Other Classification Properties

`CategoricalPredictors` — Categorical predictor indices
vector of positive integers | `[]`

Categorical predictor indices, specified as a vector of positive integers. CategoricalPredictors contains index values corresponding to the columns of the predictor data that contain categorical predictors. If none of the predictors are categorical, then this property is empty ([]).

Data Types: single | double

`ClassNames` — Unique class labels
categorical array | character array | logical vector | numeric vector | cell array of character vectors

Unique class labels used in training, specified as a categorical or character array, logical or numeric vector, or cell array of character vectors. ClassNames has the same data type as the class labels Y. (The software treats string arrays as cell arrays of character vectors.) ClassNames also determines the class order.

`Cost` — Misclassification costs
square numeric matrix

This property is read-only.

Misclassification costs, specified as a square numeric matrix. Cost has K rows and columns, where K is the number of classes.

Cost(i,j) is the cost of classifying a point into class j if its true class is i. The order of the rows and columns of Cost corresponds to the order of the classes in ClassNames.

fitcecoc incorporates misclassification costs differently among different types of binary learners.

Data Types: double

`ExpandedPredictorNames` — Expanded predictor names
cell array of character vectors

Expanded predictor names, specified as a cell array of character vectors.

If the model uses encoding for categorical variables, then ExpandedPredictorNames includes the names that describe the expanded variables. Otherwise, ExpandedPredictorNames is the same as PredictorNames.

Data Types: cell

`ModelParameters` — Parameter values
object

Parameter values, such as the name-value pair argument values, used to train the ECOC classifier, specified as an object. ModelParameters does not contain estimated parameters.

Access properties of ModelParameters using dot notation. For example, list the templates containing parameters of the binary learners by using Mdl.ModelParameters.BinaryLearner.

`NumObservations` — Number of observations
positive numeric scalar

Number of observations in the training data, specified as a positive numeric scalar.

Data Types: double

`PredictorNames` — Predictor names
cell array of character vectors

Predictor names in order of their appearance in the predictor data X, specified as a cell array of character vectors. The length of PredictorNames is equal to the number of columns in X.

Data Types: cell

`Prior` — Prior class probabilities
numeric vector

This property is read-only.

Prior class probabilities, specified as a numeric vector. Prior has as many elements as the number of classes in ClassNames, and the order of the elements corresponds to the order of the classes in ClassNames.

fitcecoc incorporates misclassification costs differently among different types of binary learners.

Data Types: double

`ResponseName` — Response variable name
character vector

Response variable name, specified as a character vector.

Data Types: char

`RowsUsed` — Rows used in fitting
`[]` (default) | logical vector

Rows of the original data X used in fitting the ClassificationECOC model, specified as a logical vector. This property is empty if all rows are used.

Data Types: logical

`ScoreTransform` — Score transformation function to apply to predicted scores
`'doublelogit'` | `'invlogit'` | `'ismax'` | `'logit'` | `'none'` | function handle | ...

Score transformation function to apply to predicted scores, specified as a function name or function handle.

To change the score transformation function to function, for example, use dot notation.

For a built-in function, enter this code and replace function with a value in the table.

Mdl.ScoreTransform = 'function';

Value	Description
`'doublelogit'`	1/(1 + e^–2x)
`'invlogit'`	log(x / (1 – x))
`'ismax'`	Sets the score for the class with the largest score to 1, and sets the scores for all other classes to 0
`'logit'`	1/(1 + e^–x)
`'none'` or `'identity'`	x (no transformation)
`'sign'`	–1 for x < 0 0 for x = 0 1 for x > 0
`'symmetric'`	2x – 1
`'symmetricismax'`	Sets the score for the class with the largest score to 1, and sets the scores for all other classes to –1
`'symmetriclogit'`	2/(1 + e^–x) – 1

For a MATLAB^® function or a function that you define, enter its function handle.
```
Mdl.ScoreTransform = @function;
```
function must accept a matrix (the original scores) and return a matrix of the same size (the transformed scores).

Data Types: char | function_handle

`W` — Observation weights
numeric vector

Observation weights used to train the ECOC classifier, specified as a numeric vector. W has NumObservations elements.

The software normalizes the weights used for training so that nansum(W) is 1.

Data Types: single | double

`X` — Unstandardized predictor data
numeric matrix | table

Unstandardized predictor data used to train the ECOC classifier, specified as a numeric matrix or table.

Each row of X corresponds to one observation, and each column corresponds to one variable.

Data Types: single | double | table

`Y` — Observed class labels
categorical array | character array | logical vector | numeric vector | cell array of character vectors

Observed class labels used to train the ECOC classifier, specified as a categorical or character array, logical or numeric vector, or cell array of character vectors. Y has NumObservations elements and has the same data type as the input argument Y of fitcecoc. (The software treats string arrays as cell arrays of character vectors.)

Each row of Y represents the observed classification of the corresponding row of X.

Hyperparameter Optimization Properties

`HyperparameterOptimizationResults` — Description of cross-validation optimization of hyperparameters
`BayesianOptimization` object | table

Description of the cross-validation optimization of hyperparameters, specified as a BayesianOptimization object or a table of hyperparameters and associated values. This property is nonempty if the 'OptimizeHyperparameters' name-value pair argument is nonempty when you create the model. The value of HyperparameterOptimizationResults depends on the setting of the Optimizer field in the HyperparameterOptimizationOptions structure when you create the model, as described in this table.

Value of `Optimizer` Field	Value of `HyperparameterOptimizationResults`
`'bayesopt'` (default)	Object of class `BayesianOptimization`
`'gridsearch'` or `'randomsearch'`	Table of hyperparameters used, observed objective function values (cross-validation loss), and rank of observations from lowest (best) to highest (worst)

Object Functions

`compareHoldout`	Compare accuracies of two classification models using new data
`compact`	Reduce size of multiclass error-correcting output codes (ECOC) model
`crossval`	Cross-validate multiclass error-correcting output codes (ECOC) model
`discardSupportVectors`	Discard support vectors of linear SVM binary learners in ECOC model
`edge`	Classification edge for multiclass error-correcting output codes (ECOC) model
`loss`	Classification loss for multiclass error-correcting output codes (ECOC) model
`margin`	Classification margins for multiclass error-correcting output codes (ECOC) model
`predict`	Classify observations using multiclass error-correcting output codes (ECOC) model
`resubEdge`	Resubstitution classification edge for multiclass error-correcting output codes (ECOC) model
`resubLoss`	Resubstitution classification loss for multiclass error-correcting output codes (ECOC) model
`resubMargin`	Resubstitution classification margins for multiclass error-correcting output codes (ECOC) model
`resubPredict`	Classify observations in multiclass error-correcting output codes (ECOC) model

Examples

collapse all

Train Multiclass Model Using SVM Learners

Open Live Script

Train a multiclass error-correcting output codes (ECOC) model using support vector machine (SVM) binary learners.

Load Fisher's iris data set. Specify the predictor data X and the response data Y.

load fisheriris
X = meas;
Y = species;

Train a multiclass ECOC model using the default options.

Mdl = fitcecoc(X,Y)

Mdl = 
  ClassificationECOC
             ResponseName: 'Y'
    CategoricalPredictors: []
               ClassNames: {'setosa'  'versicolor'  'virginica'}
           ScoreTransform: 'none'
           BinaryLearners: {3x1 cell}
               CodingName: 'onevsone'


  Properties, Methods

Mdl is a ClassificationECOC model. By default, fitcecoc uses SVM binary learners and a one-versus-one coding design. You can access Mdl properties using dot notation.

Display the class names and the coding design matrix.

Mdl.ClassNames

ans = 3x1 cell
    {'setosa'    }
    {'versicolor'}
    {'virginica' }

CodingMat = Mdl.CodingMatrix

CodingMat = 3×3

     1     1     0
    -1     0     1
     0    -1    -1

A one-versus-one coding design for three classes yields three binary learners. The columns of CodingMat correspond to the learners, and the rows correspond to the classes. The class order is the same as the order in Mdl.ClassNames. For example, CodingMat(:,1) is [1; –1; 0] and indicates that the software trains the first SVM binary learner using all observations classified as 'setosa' and 'versicolor'. Because 'setosa' corresponds to 1, it is the positive class; 'versicolor' corresponds to –1, so it is the negative class.

You can access each binary learner using cell indexing and dot notation.

Mdl.BinaryLearners{1}   % The first binary learner

ans = 
  classreg.learning.classif.CompactClassificationSVM
             ResponseName: 'Y'
    CategoricalPredictors: []
               ClassNames: [-1 1]
           ScoreTransform: 'none'
                     Beta: [4x1 double]
                     Bias: 1.4505
         KernelParameters: [1x1 struct]


  Properties, Methods

Compute the resubstitution classification error.

error = resubLoss(Mdl)

error = 0.0067

The classification error on the training data is small, but the classifier might be an overfitted model. You can cross-validate the classifier using crossval and compute the cross-validation classification error instead.

Inspect Binary Learner Properties of ECOC Classifier

Open Live Script

Train an ECOC classifier using SVM binary learners. Then, access properties of the binary learners, such as estimated parameters, by using dot notation.

Load Fisher's iris data set. Specify the petal dimensions as the predictors and the species names as the response.

load fisheriris
X = meas(:,3:4);
Y = species;

Train an ECOC classifier using SVM binary learners and the default coding design (one-versus-one). Standardize the predictors and save the support vectors.

t = templateSVM('Standardize',true,'SaveSupportVectors',true);
predictorNames = {'petalLength','petalWidth'};
responseName = 'irisSpecies';
classNames = {'setosa','versicolor','virginica'}; % Specify class order
Mdl = fitcecoc(X,Y,'Learners',t,'ResponseName',responseName,...
    'PredictorNames',predictorNames,'ClassNames',classNames)

Mdl = 
  ClassificationECOC
           PredictorNames: {'petalLength'  'petalWidth'}
             ResponseName: 'irisSpecies'
    CategoricalPredictors: []
               ClassNames: {'setosa'  'versicolor'  'virginica'}
           ScoreTransform: 'none'
           BinaryLearners: {3x1 cell}
               CodingName: 'onevsone'


  Properties, Methods

t is a template object that contains options for SVM classification. The function fitcecoc uses default values for the empty ([]) properties. Mdl is a ClassificationECOC classifier. You can access properties of Mdl using dot notation.

Display the class names and the coding design matrix.

Mdl.ClassNames

ans = 3x1 cell
    {'setosa'    }
    {'versicolor'}
    {'virginica' }

Mdl.CodingMatrix

ans = 3×3

     1     1     0
    -1     0     1
     0    -1    -1

The columns correspond to SVM binary learners, and the rows correspond to the distinct classes. The row order is the same as the order in the ClassNames property of Mdl. For each column:

1 indicates that fitcecoc trains the SVM using observations in the corresponding class as members of the positive group.
–1 indicates that fitcecoc trains the SVM using observations in the corresponding class as members of the negative group.
0 indicates that the SVM does not use observations in the corresponding class.

In the first SVM, for example, fitcecoc assigns all observations to 'setosa' or 'versicolor', but not 'virginica'.

Access properties of the SVMs using cell subscripting and dot notation. Store the standardized support vectors of each SVM. Unstandardize the support vectors.

L = size(Mdl.CodingMatrix,2); % Number of SVMs
sv = cell(L,1); % Preallocate for support vector indices
for j = 1:L
    SVM = Mdl.BinaryLearners{j};
    sv{j} = SVM.SupportVectors;
    sv{j} = sv{j}.*SVM.Sigma + SVM.Mu;
end

sv is a cell array of matrices containing the unstandardized support vectors for the SVMs.

Plot the data, and identify the support vectors.

figure
gscatter(X(:,1),X(:,2),Y);
hold on
markers = {'ko','ro','bo'}; % Should be of length L
for j = 1:L
    svs = sv{j};
    plot(svs(:,1),svs(:,2),markers{j},...
        'MarkerSize',10 + (j - 1)*3);
end
title('Fisher''s Iris -- ECOC Support Vectors')
xlabel(predictorNames{1})
ylabel(predictorNames{2})
legend([classNames,{'Support vectors - SVM 1',...
    'Support vectors - SVM 2','Support vectors - SVM 3'}],...
    'Location','Best')
hold off

You can pass Mdl to these functions:

predict, to classify new observations
resubLoss, to estimate the classification error on the training data
crossval, to perform 10-fold cross-validation

Cross-Validate ECOC Classifier

Open Live Script

Cross-validate an ECOC classifier with SVM binary learners, and estimate the generalized classification error.

Load Fisher's iris data set. Specify the predictor data X and the response data Y.

load fisheriris
X = meas;
Y = species;
rng(1); % For reproducibility

Create an SVM template, and standardize the predictors.

t = templateSVM('Standardize',true)

t = 
Fit template for classification SVM.

                     Alpha: [0x1 double]
             BoxConstraint: []
                 CacheSize: []
             CachingMethod: ''
                ClipAlphas: []
    DeltaGradientTolerance: []
                   Epsilon: []
              GapTolerance: []
              KKTTolerance: []
            IterationLimit: []
            KernelFunction: ''
               KernelScale: []
              KernelOffset: []
     KernelPolynomialOrder: []
                  NumPrint: []
                        Nu: []
           OutlierFraction: []
          RemoveDuplicates: []
           ShrinkagePeriod: []
                    Solver: ''
           StandardizeData: 1
        SaveSupportVectors: []
            VerbosityLevel: []
                   Version: 2
                    Method: 'SVM'
                      Type: 'classification'

t is an SVM template. Most of the template object properties are empty. When training the ECOC classifier, the software sets the applicable properties to their default values.

Train the ECOC classifier, and specify the class order.

Mdl = fitcecoc(X,Y,'Learners',t,...
    'ClassNames',{'setosa','versicolor','virginica'});

Mdl is a ClassificationECOC classifier. You can access its properties using dot notation.

Cross-validate Mdl using 10-fold cross-validation.

CVMdl = crossval(Mdl);

CVMdl is a ClassificationPartitionedECOC cross-validated ECOC classifier.

Estimate the generalized classification error.

genError = kfoldLoss(CVMdl)

genError = 0.0400

The generalized classification error is 4%, which indicates that the ECOC classifier generalizes fairly well.

More About

expand all

Error-Correcting Output Codes Model

An error-correcting output codes (ECOC) model reduces the problem of classification with three or more classes to a set of binary classification problems.

ECOC classification requires a coding design, which determines the classes that the binary learners train on, and a decoding scheme, which determines how the results (predictions) of the binary classifiers are aggregated.

Assume the following:

The classification problem has three classes.
The coding design is one-versus-one. For three classes, this coding design is

$\begin{matrix} Learner 1 & Learner 2 & Learner 3 \\ Class 1 & 1 & 1 & 0 \\ Class 2 & - 1 & 0 & 1 \\ Class 3 & 0 & - 1 & - 1 \end{matrix}$
The decoding scheme uses loss g.
The learners are SVMs.

To build this classification model, the ECOC algorithm follows these steps.

Learner 1 trains on observations in Class 1 or Class 2, and treats Class 1 as the positive class and Class 2 as the negative class. The other learners are trained similarly.
Let M be the coding design matrix with elements m_kl, and s_l be the predicted classification score for the positive class of learner l. The algorithm assigns a new observation to the class ( $\hat{k}$ ) that minimizes the aggregation of the losses for the L binary learners.

$\hat{k} = \underset{k}{argmin} \frac{\sum_{l = 1}^{L} | m_{k l} | g (m_{k l}, s_{l})}{\sum_{l = 1}^{L} | m_{k l} |} .$

ECOC models can improve classification accuracy, compared to other multiclass models [2].

Coding Design

A coding design is a matrix where elements direct which classes are trained by each binary learner, that is, how the multiclass problem is reduced to a series of binary problems.

Each row of the coding design corresponds to a distinct class, and each column corresponds to a binary learner. In a ternary coding design, for a particular column (or binary learner):

A row containing 1 directs the binary learner to group all observations in the corresponding class into a positive class.
A row containing –1 directs the binary learner to group all observations in the corresponding class into a negative class.
A row containing 0 directs the binary learner to ignore all observations in the corresponding class.

Coding design matrices with large, minimal, pairwise row distances based on the Hamming measure are optimal. For details on the pairwise row distance, see Random Coding Design Matrices and [4].

This table describes popular coding designs.

Coding Design	Description	Number of Learners	Minimal Pairwise Row Distance
one-versus-all (OVA)	For each binary learner, one class is positive and the rest are negative. This design exhausts all combinations of positive class assignments.	K	2
one-versus-one (OVO)	For each binary learner, one class is positive, another is negative, and the rest are ignored. This design exhausts all combinations of class pair assignments.	K(K – 1)/2	1
binary complete	This design partitions the classes into all binary combinations, and does not ignore any classes. That is, all class assignments are `–1` and `1` with at least one positive class and one negative class in the assignment for each binary learner.	2^{K – 1} – 1	2^{K – 2}
ternary complete	This design partitions the classes into all ternary combinations. That is, all class assignments are `0`, `–1`, and `1` with at least one positive class and one negative class in the assignment for each binary learner.	(3^K – 2^{K + 1} + 1)/2	3^{K – 2}
ordinal	For the first binary learner, the first class is negative and the rest are positive. For the second binary learner, the first two classes are negative and the rest are positive, and so on.	K – 1	1
dense random	For each binary learner, the software randomly assigns classes into positive or negative classes, with at least one of each type. For more details, see Random Coding Design Matrices.	Random, but approximately 10 log₂K	Variable
sparse random	For each binary learner, the software randomly assigns classes as positive or negative with probability 0.25 for each, and ignores classes with probability 0.5. For more details, see Random Coding Design Matrices.	Random, but approximately 15 log₂K	Variable

This plot compares the number of binary learners for the coding designs with increasing K.

Algorithms

expand all

Random Coding Design Matrices

For a given number of classes K, the software generates random coding design matrices as follows.

The software generates one of these matrices:
1. Dense random — The software assigns 1 or –1 with equal probability to each element of the K-by-L_d coding design matrix, where $L_{d} \approx ⌈ 10 \log_{2} K ⌉$ .
2. Sparse random — The software assigns 1 to each element of the K-by-L_s coding design matrix with probability 0.25, –1 with probability 0.25, and 0 with probability 0.5, where $L_{s} \approx ⌈ 15 \log_{2} K ⌉$ .
If a column does not contain at least one 1 and at least one –1, then the software removes that column.
For distinct columns u and v, if u = v or u = –v, then the software removes v from the coding design matrix.

The software randomly generates 10,000 matrices by default, and retains the matrix with the largest, minimal, pairwise row distance based on the Hamming measure ([4]) given by

$Δ (k_{1}, k_{2}) = 0.5 \sum_{l = 1}^{L} | m_{k_{1} l} | | m_{k_{2} l} | | m_{k_{1} l} - m_{k_{2} l} |,$

where m_{k_jl} is an element of coding design matrix j.

Support Vector Storage

By default and for efficiency, fitcecoc empties the Alpha, SupportVectorLabels, and SupportVectors properties for all linear SVM binary learners. fitcecoc lists Beta, rather than Alpha, in the model display.

To store Alpha, SupportVectorLabels, and SupportVectors, pass a linear SVM template that specifies storing support vectors to fitcecoc. For example, enter:

t = templateSVM('SaveSupportVectors',true)
Mdl = fitcecoc(X,Y,'Learners',t);

You can remove the support vectors and related values by passing the resulting ClassificationECOC model to discardSupportVectors.

Alternative Functionality

You can use these alternative algorithms to train a multiclass model:

Classification ensembles—see fitcensemble and ClassificationEnsemble
Classification trees—see fitctree and ClassificationTree
Discriminant analysis classifiers—see fitcdiscr and ClassificationDiscriminant
k-nearest neighbor classifiers—see fitcknn and ClassificationKNN
Naive Bayes classifiers—see fitcnb and ClassificationNaiveBayes

References

[1] Fürnkranz, Johannes. “Round Robin Classification.” Journal of Machine Learning Research, Vol. 2, 2002, pp. 721–747.

[2] Escalera, S., O. Pujol, and P. Radeva. “Separability of ternary codes for sparse designs of error-correcting output codes.” Pattern Recognition Letters, Vol. 30, Issue 3, 2009, pp. 285–297.

Extended Capabilities

C/C++ Code Generation
Generate C and C++ code using MATLAB® Coder™.

Usage notes and limitations:

The predict and update functions support code generation.
When you train an ECOC model by using fitcecoc, the following restrictions apply.
- You cannot fit posterior probabilities by using the 'FitPosterior' name-value pair argument.
- All binary learners must be either SVM classifiers or linear classification models. For the 'Learners' name-value pair argument, you can specify:
  - 'svm' or 'linear'
  - An SVM template object or a cell array of such objects (see templateSVM)
  - A linear classification model template object or a cell array of such objects (see templateLinear)
- When you generate code using a coder configurer for predict and update, the following additional restrictions apply for binary learners.
  - If you use a cell array of SVM template objects, the value of 'Standardize' for SVM learners must be consistent. For example, if you specify 'Standardize',true for one SVM learner, you must specify the same value for all SVM learners.
  - If you use a cell array of SVM template objects, and you use one SVM learner with a linear kernel ('KernelFunction','linear') and another with a different type of kernel function, then you must specify 'SaveSupportVectors',true for the learner with a linear kernel.
  For details, see ClassificationECOCCoderConfigurer. For information on name-value pair arguments that you cannot modify when you retrain a model, see Tips.
- Code generation limitations for SVM classifiers and linear classification models also apply to ECOC classifiers, depending on the choice of binary learners. For more details, see Code Generation of the CompactClassificationSVM class and Code Generation of the ClassificationLinear class.

For more information, see Introduction to Code Generation.

Documentation

ClassificationECOC

Description

Creation

Properties

ECOC Properties

BinaryLearners — Trained binary learners cell vector of model objects

BinaryLoss — Binary learner loss function 'binodeviance' | 'exponential' | 'hamming' | 'hinge' | 'linear' | 'logit' | 'quadratic'

BinaryY — Binary learner class labels numeric matrix

BinEdges — Bin edges for numeric predictors cell array of numeric vectors | []

CodingMatrix — Class assignment codes numeric matrix

CodingName — Coding design name character vector

LearnerWeights — Binary learner weights numeric row vector

Other Classification Properties

CategoricalPredictors — Categorical predictor indices vector of positive integers | []

ClassNames — Unique class labels categorical array | character array | logical vector | numeric vector | cell array of character vectors

Cost — Misclassification costs square numeric matrix

ExpandedPredictorNames — Expanded predictor names cell array of character vectors

ModelParameters — Parameter values object

NumObservations — Number of observations positive numeric scalar

PredictorNames — Predictor names cell array of character vectors

Prior — Prior class probabilities numeric vector

ResponseName — Response variable name character vector

RowsUsed — Rows used in fitting [] (default) | logical vector

ScoreTransform — Score transformation function to apply to predicted scores 'doublelogit' | 'invlogit' | 'ismax' | 'logit' | 'none' | function handle | ...

W — Observation weights numeric vector

X — Unstandardized predictor data numeric matrix | table

Y — Observed class labels categorical array | character array | logical vector | numeric vector | cell array of character vectors

Hyperparameter Optimization Properties

HyperparameterOptimizationResults — Description of cross-validation optimization of hyperparameters BayesianOptimization object | table

Object Functions

Examples

Train Multiclass Model Using SVM Learners

Inspect Binary Learner Properties of ECOC Classifier

Cross-Validate ECOC Classifier

More About

Error-Correcting Output Codes Model

Coding Design

Algorithms

Random Coding Design Matrices

Support Vector Storage

Alternative Functionality

References

Extended Capabilities

C/C++ Code Generation Generate C and C++ code using MATLAB® Coder™.

See Also

Introduced in R2014b

Statistics and Machine Learning Toolbox Documentation

Support

`BinaryLearners` — Trained binary learners
cell vector of model objects

`BinaryLoss` — Binary learner loss function
`'binodeviance'` | `'exponential'` | `'hamming'` | `'hinge'` | `'linear'` | `'logit'` | `'quadratic'`

`BinaryY` — Binary learner class labels
numeric matrix

`BinEdges` — Bin edges for numeric predictors
cell array of numeric vectors | `[]`

`CodingMatrix` — Class assignment codes
numeric matrix

`CodingName` — Coding design name
character vector

`LearnerWeights` — Binary learner weights
numeric row vector

`CategoricalPredictors` — Categorical predictor indices
vector of positive integers | `[]`

`ClassNames` — Unique class labels
categorical array | character array | logical vector | numeric vector | cell array of character vectors

`Cost` — Misclassification costs
square numeric matrix

`ExpandedPredictorNames` — Expanded predictor names
cell array of character vectors

`ModelParameters` — Parameter values
object

`NumObservations` — Number of observations
positive numeric scalar

`PredictorNames` — Predictor names
cell array of character vectors

`Prior` — Prior class probabilities
numeric vector

`ResponseName` — Response variable name
character vector

`RowsUsed` — Rows used in fitting
`[]` (default) | logical vector

`ScoreTransform` — Score transformation function to apply to predicted scores
`'doublelogit'` | `'invlogit'` | `'ismax'` | `'logit'` | `'none'` | function handle | ...

`W` — Observation weights
numeric vector

`X` — Unstandardized predictor data
numeric matrix | table

`Y` — Observed class labels
categorical array | character array | logical vector | numeric vector | cell array of character vectors

`HyperparameterOptimizationResults` — Description of cross-validation optimization of hyperparameters
`BayesianOptimization` object | table

C/C++ Code Generation
Generate C and C++ code using MATLAB® Coder™.