RegressionLinear class

Linear regression model for high-dimensional data

Description

RegressionLinear is a trained linear model object for regression; the linear model is a support vector machine regression (SVM) or linear regression model. fitrlinear fits a RegressionLinear model by minimizing the objective function using techniques that reduce computation time for high-dimensional data sets (e.g., stochastic gradient descent). The regression loss plus the regularization term compose the objective function.

Unlike other regression models, and for economical memory usage, RegressionLinear model objects do not store the training data. However, they do store, for example, the estimated linear model coefficients, estimated coefficients, and the regularization strength.

You can use trained RegressionLinear models to predict responses for new data. For details, see predict.

Construction

Create a RegressionLinear object by using fitrlinear.

Properties

expand all

Linear Regression Properties

`Epsilon` — Half of width of epsilon-insensitive band
nonnegative scalar

Half of the width of the epsilon-insensitive-band, specified as a nonnegative scalar.

If Learner is not 'svm', then Epsilon is an empty array ([]).

Data Types: single | double

`Lambda` — Regularization term strength
nonnegative scalar | vector of nonnegative values

Regularization term strength, specified as a nonnegative scalar or vector of nonnegative values.

Data Types: double | single

`Learner` — Linear regression model type
`'leastsquares'` | `'svm'`

Linear regression model type, specified as 'leastsquares' or 'svm'.

In this table, $f (x) = x β + b .$

β is a vector of p coefficients.
x is an observation from p predictor variables.
b is the scalar bias.

Value	Algorithm	Loss function	`FittedLoss` Value
`'leastsquares'`	Linear regression through ordinary least squares	Mean squared error (MSE): $ℓ [y, f (x)] = \frac{1}{2} {[y - f (x)]}^{2}$	`'mse'`
`'svm'`	Support vector machine regression	Epsilon insensitive: $ℓ [y, f (x)] = \max [0, \| y - f (x) \| - ε]$	`'epsiloninsensitive'`

`Beta` — Linear coefficient estimates
numeric vector

Linear coefficient estimates, specified as a numeric vector with length equal to the number of predictors.

Data Types: double

`Bias` — Estimated bias term
numeric scalar

Estimated bias term or model intercept, specified as a numeric scalar.

Data Types: double

`FittedLoss` — Loss function used to fit the linear model
`'epsiloninsensitive'` | `'mse'`

Loss function used to fit the model, specified as 'epsiloninsensitive' or 'mse'.

Value	Algorithm	Loss function	`Learner` Value
`'epsiloninsensitive'`	Support vector machine regression	Epsilon insensitive: $ℓ [y, f (x)] = \max [0, \| y - f (x) \| - ε]$	`'svm'`
`'mse'`	Linear regression through ordinary least squares	Mean squared error (MSE): $ℓ [y, f (x)] = \frac{1}{2} {[y - f (x)]}^{2}$	`'leastsquares'`

`Regularization` — Complexity penalty type
`'lasso (L1)'` | `'ridge (L2)'`

Complexity penalty type, specified as 'lasso (L1)' or 'ridge (L2)'.

The software composes the objective function for minimization from the sum of the average loss function (see FittedLoss) and a regularization value from this table.

Value	Description
`'lasso (L1)'`	Lasso (L₁) penalty: $λ \sum_{j = 1}^{p} \| β_{j} \|$
`'ridge (L2)'`	Ridge (L₂) penalty: $\frac{λ}{2} \sum_{j = 1}^{p} β_{j}^{2}$

λ specifies the regularization term strength (see Lambda).

The software excludes the bias term (β₀) from the regularization penalty.

Other Regression Properties

`CategoricalPredictors` — Categorical predictor indices
vector of positive integers | `[]`

Categorical predictor indices, specified as a vector of positive integers. Assuming that the predictor data contains observations in rows, CategoricalPredictors contains index values corresponding to the columns of the predictor data that contain categorical predictors. If none of the predictors are categorical, then this property is empty ([]).

Data Types: single | double

`ModelParameters` — Parameters used for training model
structure

Parameters used for training the RegressionLinear model, specified as a structure.

Access fields of ModelParameters using dot notation. For example, access the relative tolerance on the linear coefficients and the bias term by using Mdl.ModelParameters.BetaTolerance.

Data Types: struct

`PredictorNames` — Predictor names
cell array of character vectors

Predictor names in order of their appearance in the predictor data, specified as a cell array of character vectors. The length of PredictorNames is equal to the number of variables in the training data X or Tbl used as predictor variables.

Data Types: cell

`ExpandedPredictorNames` — Expanded predictor names
cell array of character vectors

Expanded predictor names, specified as a cell array of character vectors.

If the model uses encoding for categorical variables, then ExpandedPredictorNames includes the names that describe the expanded variables. Otherwise, ExpandedPredictorNames is the same as PredictorNames.

Data Types: cell

`ResponseName` — Response variable name
character vector

Response variable name, specified as a character vector.

Data Types: char

`ResponseTransform` — Response transformation function
`'none'` | function handle

Response transformation function, specified as 'none' or a function handle. ResponseTransform describes how the software transforms raw response values.

For a MATLAB^® function, or a function that you define, enter its function handle. For example, you can enter Mdl.ResponseTransform = @function, where function accepts a numeric vector of the original responses and returns a numeric vector of the same size containing the transformed responses.

Data Types: char | function_handle

Object Functions

`incrementalLearner`	Convert linear regression model to incremental learner
`loss`	Regression loss for linear regression models
`partialDependence`	Compute partial dependence
`plotPartialDependence`	Create partial dependence plot (PDP) and individual conditional expectation (ICE) plots
`predict`	Predict response of linear regression model
`selectModels`	Select fitted regularized linear regression models
`update`	Update model parameters for code generation

Copy Semantics

Value. To learn how value classes affect copy operations, see Copying Objects.

Examples

collapse all

Train Linear Regression Model

Open Live Script

Train a linear regression model using SVM, dual SGD, and ridge regularization.

Simulate 10000 observations from this model

$y = x_{100} + 2 x_{200} + e .$

$X = x_{1}, . . ., x_{1000}$ is a 10000-by-1000 sparse matrix with 10% nonzero standard normal elements.
e is random normal error with mean 0 and standard deviation 0.3.

rng(1) % For reproducibility
n = 1e4;
d = 1e3;
nz = 0.1;
X = sprandn(n,d,nz);
Y = X(:,100) + 2*X(:,200) + 0.3*randn(n,1);

Train a linear regression model. By default, fitrlinear uses support vector machines with a ridge penalty, and optimizes using dual SGD for SVM. Determine how well the optimization algorithm fit the model to the data by extracting a fit summary.

[Mdl,FitInfo] = fitrlinear(X,Y)

Mdl = 
  RegressionLinear
         ResponseName: 'Y'
    ResponseTransform: 'none'
                 Beta: [1000x1 double]
                 Bias: -0.0056
               Lambda: 1.0000e-04
              Learner: 'svm'


  Properties, Methods

FitInfo = struct with fields:
                    Lambda: 1.0000e-04
                 Objective: 0.2725
                 PassLimit: 10
                 NumPasses: 10
                BatchLimit: []
             NumIterations: 100000
              GradientNorm: NaN
         GradientTolerance: 0
      RelativeChangeInBeta: 0.4907
             BetaTolerance: 1.0000e-04
             DeltaGradient: 1.5816
    DeltaGradientTolerance: 0.1000
           TerminationCode: 0
         TerminationStatus: {'Iteration limit exceeded.'}
                     Alpha: [10000x1 double]
                   History: []
                   FitTime: 0.0458
                    Solver: {'dual'}

Mdl is a RegressionLinear model. You can pass Mdl and the training or new data to loss to inspect the in-sample mean-squared error. Or, you can pass Mdl and new predictor data to predict to predict responses for new observations.

FitInfo is a structure array containing, among other things, the termination status (TerminationStatus) and how long the solver took to fit the model to the data (FitTime). It is good practice to use FitInfo to determine whether optimization-termination measurements are satisfactory. In this case, fitrlinear reached the maximum number of iterations. Because training time is fast, you can retrain the model, but increase the number of passes through the data. Or, try another solver, such as LBFGS.

Predict Responses Using Linear Regression Model

Open Live Script

Simulate 10000 observations from this model

$y = x_{100} + 2 x_{200} + e .$

$X = {x_{1}, . . ., x_{1000}}$ is a 10000-by-1000 sparse matrix with 10% nonzero standard normal elements.
e is random normal error with mean 0 and standard deviation 0.3.

rng(1) % For reproducibility
n = 1e4;
d = 1e3;
nz = 0.1;
X = sprandn(n,d,nz);
Y = X(:,100) + 2*X(:,200) + 0.3*randn(n,1);

Hold out 5% of the data.

rng(1); % For reproducibility
cvp = cvpartition(n,'Holdout',0.05)

cvp = 
Hold-out cross validation partition
   NumObservations: 10000
       NumTestSets: 1
         TrainSize: 9500
          TestSize: 500

cvp is a CVPartition object that defines the random partition of n data into training and test sets.

Train a linear regression model using the training set. For faster training time, orient the predictor data matrix so that the observations are in columns.

idxTrain = training(cvp); % Extract training set indices
X = X';
Mdl = fitrlinear(X(:,idxTrain),Y(idxTrain),'ObservationsIn','columns');

Predict observations and the mean squared error (MSE) for the hold out sample.

idxTest = test(cvp); % Extract test set indices
yHat = predict(Mdl,X(:,idxTest),'ObservationsIn','columns');
L = loss(Mdl,X(:,idxTest),Y(idxTest),'ObservationsIn','columns')

L = 0.1851

The hold-out sample MSE is 0.1852.

Extended Capabilities

C/C++ Code Generation
Generate C and C++ code using MATLAB® Coder™.

Usage notes and limitations:

The predict and update functions support code generation.
When you train a linear regression model by using fitrlinear, the following restrictions apply.
- Code generation does not support categorical predictors (logical, categorical, char, string, or cell). If you supply training data in a table, the predictors must be numeric (double or single). Also, you cannot use the 'CategoricalPredictors' name-value pair argument.To include categorical predictors in a model, preprocess the categorical predictors by using dummyvar before fitting the model.
- If the predictor data input argument value is a matrix, it must be a full, numeric matrix. Code generation does not support sparse data.
- You can specify only one regularization strength, either 'auto' or a nonnegative scalar for the 'Lambda' name-value pair argument.
- The value of the 'ResponseTransform' name-value pair argument cannot be an anonymous function.

For more information, see Introduction to Code Generation.

Documentation

RegressionLinear class

Description

Construction

Properties

`Epsilon` — Half of width of epsilon-insensitive band
nonnegative scalar

`Lambda` — Regularization term strength
nonnegative scalar | vector of nonnegative values

`Learner` — Linear regression model type
`'leastsquares'` | `'svm'`

`Beta` — Linear coefficient estimates
numeric vector

`Bias` — Estimated bias term
numeric scalar

`FittedLoss` — Loss function used to fit the linear model
`'epsiloninsensitive'` | `'mse'`

`Regularization` — Complexity penalty type
`'lasso (L1)'` | `'ridge (L2)'`

`CategoricalPredictors` — Categorical predictor indices
vector of positive integers | `[]`

`ModelParameters` — Parameters used for training model
structure

`PredictorNames` — Predictor names
cell array of character vectors

`ExpandedPredictorNames` — Expanded predictor names
cell array of character vectors

`ResponseName` — Response variable name
character vector

`ResponseTransform` — Response transformation function
`'none'` | function handle

Object Functions

Copy Semantics

Examples

Train Linear Regression Model

Predict Responses Using Linear Regression Model

Extended Capabilities

C/C++ Code Generation
Generate C and C++ code using MATLAB® Coder™.

See Also

Statistics and Machine Learning Toolbox Documentation

Support

Documentation

RegressionLinear class

Description

Construction

Properties

Epsilon — Half of width of epsilon-insensitive band nonnegative scalar

Lambda — Regularization term strength nonnegative scalar | vector of nonnegative values

Learner — Linear regression model type 'leastsquares' | 'svm'

Beta — Linear coefficient estimates numeric vector

Bias — Estimated bias term numeric scalar

FittedLoss — Loss function used to fit the linear model 'epsiloninsensitive' | 'mse'

Regularization — Complexity penalty type 'lasso (L1)' | 'ridge (L2)'

CategoricalPredictors — Categorical predictor indices vector of positive integers | []

ModelParameters — Parameters used for training model structure

PredictorNames — Predictor names cell array of character vectors

ExpandedPredictorNames — Expanded predictor names cell array of character vectors

ResponseName — Response variable name character vector

ResponseTransform — Response transformation function 'none' | function handle

Object Functions

Copy Semantics

Examples

Train Linear Regression Model

Predict Responses Using Linear Regression Model

Extended Capabilities

C/C++ Code Generation Generate C and C++ code using MATLAB® Coder™.

See Also

Statistics and Machine Learning Toolbox Documentation

Support

`Epsilon` — Half of width of epsilon-insensitive band
nonnegative scalar

`Lambda` — Regularization term strength
nonnegative scalar | vector of nonnegative values

`Learner` — Linear regression model type
`'leastsquares'` | `'svm'`

`Beta` — Linear coefficient estimates
numeric vector

`Bias` — Estimated bias term
numeric scalar

`FittedLoss` — Loss function used to fit the linear model
`'epsiloninsensitive'` | `'mse'`

`Regularization` — Complexity penalty type
`'lasso (L1)'` | `'ridge (L2)'`

`CategoricalPredictors` — Categorical predictor indices
vector of positive integers | `[]`

`ModelParameters` — Parameters used for training model
structure

`PredictorNames` — Predictor names
cell array of character vectors

`ExpandedPredictorNames` — Expanded predictor names
cell array of character vectors

`ResponseName` — Response variable name
character vector

`ResponseTransform` — Response transformation function
`'none'` | function handle

C/C++ Code Generation
Generate C and C++ code using MATLAB® Coder™.