kfoldLoss

Cross-validation loss of partitioned regression model

Syntax

L = kfoldLoss(cvmodel) L = kfoldLoss(cvmodel,Name,Value)

Description

L = kfoldLoss(cvmodel) returns the cross-validation loss of cvmodel.

L = kfoldLoss(cvmodel,Name,Value) returns cross-validation loss with additional options specified by one or more Name,Value pair arguments. You can specify several name-value pair arguments in any order as Name1,Value1,…,NameN,ValueN.

Input Arguments

cvmodel

Object of class RegressionPartitionedModel. Create obj with fitrtree along with one of the cross-validation options: 'CrossVal', 'KFold', 'Holdout', 'Leaveout', or 'CVPartition'. Alternatively, create obj from a regression tree with crossval.

Name-Value Pair Arguments

Specify optional comma-separated pairs of Name,Value arguments. Name is the argument name and Value is the corresponding value. Name must appear inside quotes. You can specify several name and value pair arguments in any order as Name1,Value1,...,NameN,ValueN.

'folds'

Indices of folds ranging from 1 to obj.KFold. Use only these folds for predictions.

Default: 1:obj.KFold

'lossfun'

Function handle for loss function or 'mse', meaning mean squared error. If you pass a function handle fun, kfoldLoss calls it as

fun(Y,Yfit,W)

where Y, Yfit, and W are numeric vectors of the same length.

Y is the observed response.
Yfit is the predicted response.
W is the observation weights.

The returned value fun(Y,Yfit,W) should be a scalar.

Default: 'mse'

'mode'

One of the following:

'average' — L is the average loss over all folds.
'individual' — L is a vector of the individual losses of in-fold observations trained on out-of-fold data.

Default: 'average'

Output Arguments

`L`	The loss (mean squared error) between the observations in a fold when compared against predictions made with a tree trained on the out-of-fold data. If `mode` is `'individual'`, `L` is a vector of the losses. If `mode` is `'average'`, `L` is the average loss.

Examples

Construct a partitioned regression model, and examine the cross-validation losses for the folds:

load carsmall
XX = [Cylinders Displacement Horsepower Weight];
YY = MPG;
cvmodel = fitrtree(XX,YY,'crossval','on');
L = kfoldLoss(cvmodel,'mode','individual')

L =
   44.9635
   11.8525
   18.2046
    9.2965
   29.4329
   54.8659
   24.6446
    8.2085
   19.7593
   16.7394

Alternatives

You can avoid constructing a cross-validated tree model by calling cvloss. The cross-validated tree can save time if you are going to examine it more than once.

Documentation