Hessian Output

Returned Hessian

The fmincon and fminunc solvers return an approximate Hessian as an optional output:

[x,fval,exitflag,output,grad,hessian] = fminunc(fun,x0)
% or
[x,fval,exitflag,output,lambda,grad,hessian] = fmincon(fun,x0,A,b,Aeq,beq,lb,ub,nonlcon)

This section describes the meaning of the returned Hessian, and the accuracy you can expect.

You can also specify the type of Hessian that the solvers use as input Hessian arguments. For fmincon, see Hessian as an Input. For fminunc, see Including Gradients and Hessians.

fminunc Hessian

The Hessian for an unconstrained problem is the matrix of second derivatives of the objective function f:

Hessian Hij=∂2f∂xi∂xj.

  • Quasi-Newton Algorithm — fminunc returns an estimated Hessian matrix at the solution. fminunc computes the estimate by finite differences, so the estimate is generally accurate.

  • Trust-Region Algorithm — fminunc returns a Hessian matrix at the next-to-last iterate.

    • If you supply a Hessian in the objective function and set the HessianFcn option to 'objective', fminunc returns this Hessian.

    • If you supply a HessianMultiplyFcn function, fminunc returns the Hinfo matrix from the HessianMultiplyFcn function. For more information, see HessianMultiplyFcn in the trust-region section of the fminunc options table.

    • Otherwise, fminunc returns an approximation from a sparse finite difference algorithm on the gradients.

    This Hessian is accurate for the next-to-last iterate. However, the next-to-last iterate might not be close to the final point.

    The reason the trust-region algorithm returns the Hessian at the next-to-last point is for efficiency. fminunc uses the Hessian internally to compute its next step. When fminunc reaches a stopping condition, it does not need to compute the next step, so does not compute the Hessian.

fmincon Hessian

The Hessian for a constrained problem is the Hessian of the Lagrangian. For an objective function f, nonlinear inequality constraint vector c, and nonlinear equality constraint vector ceq, the Lagrangian is

L=f+∑iλici+∑jλjceqj.

The λi are Lagrange multipliers; see First-Order Optimality Measure and Lagrange Multiplier Structures. The Hessian of the Lagrangian is

H=∇2L=∇2f+∑iλi∇2ci+∑jλj∇2ceqj.

fmincon has four algorithms, with several options for Hessians, as described in fmincon Trust Region Reflective Algorithm, fmincon Active Set Algorithm, and fmincon Interior Point Algorithm. fmincon returns the following for the Hessian:

  • active-set or sqp Algorithm — fmincon returns the Hessian approximation it computes at the next-to-last iterate. fmincon computes a quasi-Newton approximation of the Hessian matrix at the solution in the course of its iterations. This approximation does not, in general, match the true Hessian in every component, but only in certain subspaces. Therefore the Hessian that fmincon returns can be inaccurate. For more details of the active-set calculation, see SQP Implementation.

  • trust-region-reflective Algorithm — fmincon returns the Hessian it computes at the next-to-last iterate.

    • If you supply a Hessian in the objective function and set the HessianFcn option to 'objective', fmincon returns this Hessian.

    • If you supply a HessianMultiplyFcn function, fmincon returns the Hinfo matrix from the HessianMultiplyFcn function. For more information, see Trust-Region-Reflective Algorithm in fmincon options.

    • Otherwise, fmincon returns an approximation from a sparse finite difference algorithm on the gradients.

    This Hessian is accurate for the next-to-last iterate. However, the next-to-last iterate might not be close to the final point.

    The reason the trust-region-reflective algorithm returns the Hessian at the next-to-last point is for efficiency. fmincon uses the Hessian internally to compute its next step. When fmincon reaches a stopping condition, it does not need to compute the next step, so does not compute the Hessian.

  • interior-point Algorithm

    • If the HessianApproximation option is 'lbfgs' or 'finite-difference', or if you supply a Hessian multiply function (HessianMultiplyFcn), fmincon returns [] for the Hessian.

    • If the HessianApproximation option is 'bfgs' (the default), fmincon returns a quasi-Newton approximation to the Hessian at the final point. This Hessian can be inaccurate, as in the active-set or sqp algorithm Hessian.

    • If the HessianFcn option is a function handle, fmincon returns this function as the Hessian at the final point.

Related Topics

Optimization Toolbox Documentation
Support