resubEdge

Classification edge for naive Bayes classifiers by resubstitution

Syntax

e = resubEdge(Mdl)

Description

e = resubEdge(Mdl) returns the resubstitution classification edge (e) for the naive Bayes classifier Mdl using the training data stored in Mdl.X and corresponding class labels stored in Mdl.Y.

Input Arguments

expand all

`Mdl` — Fully trained naive Bayes classifier
`ClassificationNaiveBayes` model

A fully trained naive Bayes classifier, specified as a ClassificationNaiveBayes model trained by fitcnb.

Output Arguments

expand all

`e` — Classification edge
scalar

Classification edge, returned as a scalar. If you passed in weights when training the classifier, then e is the weighted classification edge. The software normalizes the weights so that they sum to the prior probability of their respective class.

Examples

expand all

Estimate the Resubstitution Edge of Naive Bayes Classifiers

Open Live Script

Load Fisher's iris data set.

load fisheriris
X = meas;    % Predictors
Y = species; % Response
rng(1);

Train a naive Bayes classifier. It is good practice to specify the class order. Assume that each predictor is conditionally, normally distributed given its label.

Mdl = fitcnb(X,Y,'ClassNames',{'setosa','versicolor','virginica'});

Mdl is a trained ClassificationNaiveBayes classifier.

Estimate the resubstitution edge.

e = resubEdge(Mdl)

e = 0.8944

The mean of the training sample margins is approximately 0.9, which indicates that the classifier classifies in-sample observations with high confidence.

Select Naive Bayes Classifier Features by Comparing In-Sample Edges

Open Live Script

The classifier edge measures the average of the classifier margins. One way to perform feature selection is to compare training sample edges from multiple models. Based solely on this criterion, the classifier with the highest edge is the best classifier.

Load Fisher's iris data set. Define two data sets:

fullX contains all predictors.
partX contains the last two predictors.

load fisheriris
X = meas;    % Predictors
Y = species; % Response
fullX = X;
partX = X(:,3:4);

Train naive Bayes classifiers for each predictor set.

FullMdl = fitcnb(fullX,Y);
PartMdl = fitcnb(partX,Y);

Estimate the training sample edge for each classifier.

fullEdge = resubEdge(FullMdl)

fullEdge = 0.8944

partEdge = resubEdge(PartMdl)

partEdge = 0.9169

The edge for the classifier trained on predictors 3 and 4 is greater, suggesting that the classifier trained using only those predictors has a better in-sample fit.

More About

expand all

Classification Edge

The classification edge is the weighted mean of the classification margins.

If you supply weights, then the software normalizes them to sum to the prior probability of their respective class. The software uses the normalized weights to compute the weighted mean.

One way to choose among multiple classifiers, e.g., to perform feature selection, is to choose the classifier that yields the highest edge.

Classification Margins

The classification margins are, for each observation, the difference between the score for the true class and maximal score for the false classes. Provided that they are on the same scale, margins serve as a classification confidence measure, i.e., among multiple classifiers, those that yield larger margins are better.

Posterior Probability

The posterior probability is the probability that an observation belongs in a particular class, given the data.

For naive Bayes, the posterior probability that a classification is k for a given observation (x₁,...,x_P) is

$\hat{P} (Y = k | x_{1}, .., x_{P}) = \frac{P (X_{1}, ..., X_{P} | y = k) π (Y = k)}{P (X_{1}, ..., X_{P})},$

where:

$P (X_{1}, ..., X_{P} | y = k)$ is the conditional joint density of the predictors given they are in class k. Mdl.DistributionNames stores the distribution names of the predictors.
π(Y = k) is the class prior probability distribution. Mdl.Prior stores the prior distribution.
$P (X_{1}, .., X_{P})$ is the joint density of the predictors. The classes are discrete, so $P (X_{1}, ..., X_{P}) = \sum_{k = 1}^{K} P (X_{1}, ..., X_{P} | y = k) π (Y = k) .$

Prior Probability

The prior probability of a class is the believed relative frequency with which observations from that class occur in a population.

Score

The naive Bayes score is the class posterior probability given the observation.

Documentation

resubEdge

Syntax

Description

Input Arguments

`Mdl` — Fully trained naive Bayes classifier
`ClassificationNaiveBayes` model

Output Arguments

`e` — Classification edge
scalar

Examples

Estimate the Resubstitution Edge of Naive Bayes Classifiers

Select Naive Bayes Classifier Features by Comparing In-Sample Edges

More About

Classification Edge

Classification Margins

Posterior Probability

Prior Probability

Score

See Also

Topics

Statistics and Machine Learning Toolbox Documentation

Support

Documentation

resubEdge

Syntax

Description

Input Arguments

Mdl — Fully trained naive Bayes classifier ClassificationNaiveBayes model

Output Arguments

e — Classification edge scalar

Examples

Estimate the Resubstitution Edge of Naive Bayes Classifiers

Select Naive Bayes Classifier Features by Comparing In-Sample Edges

More About

Classification Edge

Classification Margins

Posterior Probability

Prior Probability

Score

See Also

Topics

Statistics and Machine Learning Toolbox Documentation

Support

`Mdl` — Fully trained naive Bayes classifier
`ClassificationNaiveBayes` model

`e` — Classification edge
scalar