Superclasses: CompactRegressionSVM
Support vector machine regression model
RegressionSVM
is a support vector machine (SVM) regression model.
Train a RegressionSVM
model using fitrsvm
and the sample data.
RegressionSVM
models store data, parameter values, support vectors,
and algorithmic implementation information. You can use these models to:
Estimate resubstitution predictions. For details, see resubPredict
.
Predict values for new data. For details, see predict
.
Compute resubstitution loss. For details, see resubLoss
.
Compute the mean square error or epsilon-insensitive loss. For details, see
loss
.
Create a RegressionSVM
object by using fitrsvm
.
Alpha
— Dual problem coefficientsDual problem coefficients, specified as a vector of numeric values. Alpha
contains m elements, where m is the number of
support vectors in the trained SVM regression model. The dual problem introduces two
Lagrange multipliers for each support vector. The values of Alpha
are
the differences between the two estimated Lagrange multipliers for the support vectors.
For more details, see Understanding Support Vector Machine Regression.
If you specified to remove duplicates using RemoveDuplicates
,
then, for a particular set of duplicate observations that are support vectors,
Alpha
contains one coefficient corresponding to the entire set.
That is, MATLAB® attributes a nonzero coefficient to one observation from the set of
duplicates and a coefficient of 0
to all other duplicate observations
in the set.
Data Types: single
| double
Beta
— Primal linear problem coefficients'[]'
Primal linear problem coefficients, stored as a numeric vector of length p, where p is the number of predictors in the SVM regression model.
The values in Beta
are the linear coefficients for the
primal optimization problem.
If the model is obtained using a kernel function other than
'linear'
, this property is empty
('[]'
).
The predict
method computes predicted
response values for the model as YFIT = (X/S)×Beta +
Bias
, where S
is the value of the kernel
scale stored in the KernelParameters.Scale
property.
Data Types: single
| double
Bias
— Bias termBias term in the SVM regression model, stored as a scalar value.
Data Types: single
| double
BoxConstraints
— Box constraints for dual problem coefficientsBox constraints for dual problem alpha coefficients, stored as a numeric
vector containing n elements, where n
is the number of observations in X
(Mdl.NumObservations
).
The absolute value of the dual coefficient Alpha
for
observation i cannot exceed
BoxConstraints(i)
.
If you specify removing duplicates using 'RemoveDuplicates'
, then for a given set of duplicate
observations, MATLAB sums the box constraints, and then attributes the sum to one
observation and box constraints of 0
to all other
observations in the set.
Data Types: single
| double
CacheInfo
— Caching informationCaching information, stored as a structure with the following fields.
Field | Description |
---|---|
'Size' | Positive scalar value indicating the cache size (in MB)
that the software reserves to store entries of the Gram
matrix. Set the cache size by using the
'CacheSize' name-value pair argument
in fitrsvm . |
'Algorithm' | Character vector containing the name of the algorithm
used to remove entries from the cache when its capacity is
exceeded. Currently, the only available caching algorithm is
'Queue' . You cannot set the caching
algorithm. |
Data Types: struct
CategoricalPredictors
— Indices of categorical predictorsCategorical predictor
indices, specified as a vector of positive integers. CategoricalPredictors
contains index values corresponding to the columns of the predictor data that contain
categorical predictors. If none of the predictors are categorical, then this property is empty
([]
).
Data Types: single
| double
ConvergenceInfo
— Convergence informationConvergence information, stored as a structure with the following fields.
Field | Description |
---|---|
Converged | Logical flag indicating whether the algorithm converged.
A value of 1 indicates
convergence. |
ReasonForConvergence | Character vector indicating the criterion the software used to detect convergence. |
Gap | Scalar feasibility gap between the dual and primal objective functions. |
GapTolerance | Scalar tolerance for the feasibility gap. You can set
this tolerance using the 'GapTolerance'
name-value pair argument in fitrsvm . |
DeltaGradient | Scalar gradient difference between upper and lower violators. |
DeltaGradientTolerance | Scalar tolerance for the gradient difference. You can set
this tolerance using the
DeltaGradientTolerance name-value
pair argument in fitrsvm . |
LargestKKTViolation | Maximal scalar Karush-Kuhn-Tucker (KKT) violation value. |
KKTTolerance | Scalar tolerance for the largest KKT violation. You can
set this tolerance using the
'KKTTolerance' name-value pair
argument in fitrsvm . |
History | Structure containing convergence information recorded at periodic intervals during the model training process. This structure contains the following fields:
|
Objective | Numeric value of the dual objective. |
Data Types: struct
Epsilon
— Half the width of the epsilon-insensitive bandHalf the width of the epsilon-insensitive band, stored as a nonnegative scalar value.
Data Types: single
| double
ExpandedPredictorNames
— Expanded predictor namesExpanded predictor names, stored as a cell array of character vectors.
If the model uses encoding for categorical variables, then
ExpandedPredictorNames
includes the names that
describe the expanded variables. Otherwise,
ExpandedPredictorNames
is the same as
PredictorNames
.
Data Types: cell
Gradient
— Gradient values in training dataGradient values in training data, stored as a numeric vector containing 2n elements, where n is the number of observations in the training data.
Element i of Gradient
contains the
gradient value for the Alpha
coefficient that corresponds
to the upper boundary of the epsilon-insensitive band at observation
i at the end of the optimization.
Element i + NumObservations of
Gradient
contains the gradient value for the
Alpha
coefficient that corresponds to the lower
boundary of the epsilon-insensitive band at observation i
at the end of the optimization.
Data Types: single
| double
HyperparameterOptimizationResults
— Cross-validation optimization of hyperparametersBayesianOptimization
object | tableThis property is read-only.
Cross-validation optimization of hyperparameters, specified as a BayesianOptimization
object or a table of hyperparameters and associated
values. This property is nonempty if the 'OptimizeHyperparameters'
name-value pair argument is nonempty when you create the model. The value of
HyperparameterOptimizationResults
depends on the setting of the
Optimizer
field in the
HyperparameterOptimizationOptions
structure when you create the
model.
Value of Optimizer Field | Value of HyperparameterOptimizationResults |
---|---|
'bayesopt' (default) | Object of class BayesianOptimization |
'gridsearch' or 'randomsearch' | Table of hyperparameters used, observed objective function values (cross-validation loss), and rank of observations from lowest (best) to highest (worst) |
IsSupportVector
— Flag indicating whether observation is support vectorFlag indicating whether an observation is a support vector, stored as an
n-by-1 logical vector. n is the
number of observations in X
(see
NumObservations
). A value of 1
indicates that the corresponding observation in the training data is a
support vector.
If you specify removing duplicates using RemoveDuplicates
, then for a given set of duplicate
observations that are support vectors, IsSupportVector
flags only one as a support vector.
Data Types: logical
KernelParameters
— Kernel function parametersKernel function parameters, stored as a structure with the following fields.
Field | Description |
---|---|
Function
| Kernel function name (a character vector). |
Scale | Numeric scale factor used to divide predictor values. |
You can specify values for KernelParameters.Function
and KernelParameters.Scale
by using the
KernelFunction
and KernelScale
name-value pair arguments in fitrsvm
, respectively.
Data Types: struct
ModelParameters
— Parameter valuesParameter values used to train the SVM regression model, stored as a model
parameter object. Access the properties of
ModelParameters
using dot notation. For example,
access the value of Epsilon
used to train the model as
Mdl.ModelParameters.Epsilon
.
Mu
— Predictor means'[]'
Predictor means, stored as a vector of numeric values.
If the predictors are standardized, then Mu
is a
numeric vector of length p, where p is
the number of predictors used to train the model. In this case, the predict
method centers predictor
matrix X
by subtracting the corresponding element of
Mu
from each column.
If the predictors are not standardized, then Mu
is
empty ('[]'
).
If the data contains categorical predictors, then Mu
includes elements for the dummy variables for those predictors. The
corresponding entries in Mu
are 0 because dummy variables
are not centered or scaled.
Data Types: single
| double
NumIterations
— Number of iterations required for convergenceNumber of iterations required for the optimization routine to reach convergence, stored as a positive integer value.
To set a limit on the number of iterations, use the
'IterationLimit'
name-value pair argument of
fitrsvm
.
Data Types: single
| double
NumObservations
— Number of observationsNumber of observations in the training data, stored as a positive integer value.
Data Types: single
| double
PredictorNames
— Predictor namesPredictor names, stored as a cell array of character vectors containing
the name of each predictor in the order they appear in X
.
PredictorNames
has a length equal to the number of
columns in X
.
Data Types: cell
OutlierFraction
— Expected fraction of outliersExpected fraction of outliers in the training set, stored as a scalar
value in the range [0,1]. You can specify the expected fraction of outliers
using the 'OutlierFraction'
name-value pair argument in
fitrsvm
.
Data Types: double
ResponseName
— Response variable nameResponse variable name, stored as a character vector.
Data Types: char
ResponseTransform
— Response transformation function'none'
| function handleResponse transformation function, specified as 'none'
or a function handle.
ResponseTransform
describes how the software transforms raw
response values.
For a MATLAB function, or a function that you define, enter
its function handle. For example, you can enter Mdl.ResponseTransform
= @function
, where function
accepts
a numeric vector of the original responses and returns a numeric vector
of the same size containing the transformed responses.
Data Types: char
| function_handle
ShrinkagePeriod
— Number of iterations between reductions of active setNumber of iterations between reductions of the active set during optimization, stored as a nonnegative integer value.
You can set the shrinkage period by using the
'ShrinkagePeriod'
name-value pair argument in
fitrsvm
.
Data Types: single
| double
Sigma
— Predictor standard deviations'[]'
Predictor standard deviations, stored as a vector of numeric values.
If the predictors are standardized, then Sigma
is a
numeric vector of length p, where p is
the number of predictors used to train the model. In this case, the predict
method scales the
predictor matrix X
by dividing each column by the
corresponding element of Sigma
, after centering each
element using Mu
.
If the predictors are not standardized, then Sigma
is
empty ('[]'
).
If the data contains categorical predictors, Sigma
includes elements for the dummy variables for those predictors. The
corresponding entries in Sigma
are 1, because dummy
variables are not centered or scaled.
Data Types: single
| double
Solver
— Name of solver algorithmName of the solver algorithm used to solve the optimization problem, stored as a value in this table.
Value | Description |
---|---|
'SMO' | Sequential Minimal Optimization |
'ISDA' | Iterative Single Data Algorithm |
'L1QP' | L1 soft-margin minimization by quadratic programming (requires an Optimization Toolbox™ license). |
SupportVectors
— Support vectorsSupport vectors, stored as an m-by-p matrix
of numeric values. m is the number of support vectors
(sum(Mdl.IsSupportVector)
), and p is
the number of predictors in X
.
If you specified to remove duplicates using RemoveDuplicates
,
then for a given set of duplicate observations that are support vectors,
SupportVectors
contains one unique support vector.
Data Types: single
| double
W
— Observation weightsObservation weights used to train the model, stored as a numeric vector
containing NumObservation
number of elements.
fitrsvm
normalizes the weights used for training so
that they sum to 1.
Data Types: single
| double
X
— Predictor valuesPredictor values used to train the model, stored as a matrix of numeric
values if the model is trained on a matrix, or a table if the model is
trained on a table. X
has size
n-by-p, where n
is the number of rows and p is the number of predictor
variables or columns in the training data.
Data Types: single
| double
| table
Y
— Observed response valuesObserved response values, stored as a numeric vector containing
NumObservations
number of elements.
Data Types: single
| double
compact | Compact support vector machine regression model |
crossval | Cross-validated support vector machine regression model |
discardSupportVectors | Discard support vectors |
incrementalLearner | Convert support vector machine (SVM) regression model to incremental learner |
loss | Regression error for support vector machine regression model |
partialDependence | Compute partial dependence |
plotPartialDependence | Create partial dependence plot (PDP) and individual conditional expectation (ICE) plots |
predict | Predict responses using support vector machine regression model |
resubLoss | Resubstitution loss for support vector machine regression model |
resubPredict | Predict resubstitution response of support vector machine regression model |
resume | Resume training support vector machine regression model |
Value. To learn how value classes affect copy operations, see Copying Objects.
This example shows how to train a linear support vector machine (SVM) regression model using sample data stored in matrices.
Load the carsmall
data set.
load carsmall rng default % for reproducibility
Specify Horsepower
and Weight
as the predictor variables (X
) and MPG
as the response variable (Y
).
X = [Horsepower,Weight]; Y = MPG;
Train a linear SVM regression model.
Mdl = fitrsvm(X,Y)
Mdl = RegressionSVM ResponseName: 'Y' CategoricalPredictors: [] ResponseTransform: 'none' Alpha: [75x1 double] Bias: 57.3800 KernelParameters: [1x1 struct] NumObservations: 93 BoxConstraints: [93x1 double] ConvergenceInfo: [1x1 struct] IsSupportVector: [93x1 logical] Solver: 'SMO' Properties, Methods
The Command Window shows that Mdl
is a trained RegressionSVM
model and a list of its properties.
Check the model for convergence.
Mdl.ConvergenceInfo.Converged
ans = logical
0
0
indicates that the model did not converge.
MdlStd = fitrsvm(X,Y,'Standardize',true)
MdlStd = RegressionSVM ResponseName: 'Y' CategoricalPredictors: [] ResponseTransform: 'none' Alpha: [77x1 double] Bias: 22.9131 KernelParameters: [1x1 struct] Mu: [109.3441 2.9625e+03] Sigma: [45.3545 805.9668] NumObservations: 93 BoxConstraints: [93x1 double] ConvergenceInfo: [1x1 struct] IsSupportVector: [93x1 logical] Solver: 'SMO' Properties, Methods
Check the model for convergence.
MdlStd.ConvergenceInfo.Converged
ans = logical
1
1
indicates that the model did converge.
Compute the resubstitution mean squared error for the new model.
lStd = resubLoss(MdlStd)
lStd = 17.0256
Train a support vector machine regression model using the abalone data from the UCI Machine Learning Repository.
Download the data and save it in your current folder with the name 'abalone.csv'
.
url = 'https://archive.ics.uci.edu/ml/machine-learning-databases/abalone/abalone.data'; websave('abalone.csv',url);
Read the data into a table. Specify the variable names.
varnames = {'Sex'; 'Length'; 'Diameter'; 'Height'; 'Whole_weight';... 'Shucked_weight'; 'Viscera_weight'; 'Shell_weight'; 'Rings'}; Tbl = readtable('abalone.csv','Filetype','text','ReadVariableNames',false); Tbl.Properties.VariableNames = varnames;
The sample data contains 4177 observations. All the predictor variables are continuous except for Sex
, which is a categorical variable with possible values 'M'
(for males), 'F'
(for females), and 'I'
(for infants). The goal is to predict the number of rings (stored in Rings
) on the abalone and determine its age using physical measurements.
Train an SVM regression model, using a Gaussian kernel function with an automatic kernel scale. Standardize the data.
rng default % For reproducibility Mdl = fitrsvm(Tbl,'Rings','KernelFunction','gaussian','KernelScale','auto',... 'Standardize',true)
Mdl = RegressionSVM PredictorNames: {1×8 cell} ResponseName: 'Rings' CategoricalPredictors: 1 ResponseTransform: 'none' Alpha: [3635×1 double] Bias: 10.8144 KernelParameters: [1×1 struct] Mu: [1×10 double] Sigma: [1×10 double] NumObservations: 4177 BoxConstraints: [4177×1 double] ConvergenceInfo: [1×1 struct] IsSupportVector: [4177×1 logical] Solver: 'SMO'
The Command Window shows that Mdl
is a trained RegressionSVM
model and displays a property list.
Display the properties of Mdl
using dot notation. For example, check to confirm whether the model converged and how many iterations it completed.
conv = Mdl.ConvergenceInfo.Converged iter = Mdl.NumIterations
conv = logical 1 iter = 2759
The returned results indicate that the model converged after 2759 iterations.
[1] Nash, W.J., T. L. Sellers, S. R. Talbot, A. J. Cawthorn, and W. B. Ford. The Population Biology of Abalone (Haliotis species) in Tasmania. I. Blacklip Abalone (H. rubra) from the North Coast and Islands of Bass Strait, Sea Fisheries Division, Technical Report No. 48, 1994.
[2] Waugh, S. Extending and benchmarking Cascade-Correlation, Ph.D. thesis, Computer Science Department, University of Tasmania, 1995.
[3] Clark, D., Z. Schreter, A. Adams. A Quantitative Comparison of Dystal and Backpropagation, submitted to the Australian Conference on Neural Networks, 1996.
[4] Lichman, M. UCI Machine Learning Repository, [http://archive.ics.uci.edu/ml]. Irvine, CA: University of California, School of Information and Computer Science.
Usage notes and limitations:
To integrate the prediction of an SVM regression model into Simulink®, you can use the RegressionSVM Predict block in the Statistics and Machine Learning Toolbox™ library or a MATLAB Function block with the predict
function.
When you train an SVM regression model by using fitrsvm
, the following restrictions apply.
Code generation does
not support categorical predictors (logical
, categorical
,
char
, string
, or cell
). If you
supply training data in a table, the predictors must be numeric (double
or
single
). Also, you cannot use the
'CategoricalPredictors'
name-value pair argument.To include categorical predictors in a model, preprocess the
categorical predictors by using dummyvar
before fitting the model.
The value of the 'ResponseTransform'
name-value pair argument must be 'none'
(default).
For fixed-point code generation, the value of the 'KernelFunction'
name-value pair argument must be 'gaussian'
, 'linear'
, or 'polynomial'
.
For more information, see Introduction to Code Generation.
You have a modified version of this example. Do you want to open this example with your edits?