Variables and Starting Conditions

All the functions (RefLink[FindMinimum, paclet:ref/FindMinimum]), (RefLink[FindMaximum, paclet:ref/FindMaximum]), and (RefLink[FindRoot, paclet:ref/FindRoot]) take variable specifications of the same form. The function (RefLink[FindFit, paclet:ref/FindFit]) uses the same form for its parameter specifications.

Variables and parameters in the "Find" functions.

The list vars (pars for (RefLink[FindFit, paclet:ref/FindFit])) should be a list of individual variable specifications. Each variable specification should be of the following form.

Individual variable specifications in the "Find" functions.

The specifications in vars all need to have the same number of starting values. When region bounds are not specified, they are taken to be unbounded, that is, rl=-∞, ru=∞.

Vector- and Matrix-Valued Variables

The most common use of variables is to represent numbers. However, the variable input syntax supports variables that are treated as vectors, matrices, or higher-rank tensors. In general, the "Find" commands, with the exception of (RefLink[FindFit, paclet:ref/FindFit]), which currently only works with scalar variables, will consider a variable to take on values with the same rectangular structure as the starting conditions given for it.

Here is a matrix:

A = (({{0, 1, 2}, {3, 4, 5}, {6, 7, 8}}));

This uses (RefLink[FindRoot, paclet:ref/FindRoot]) to find an eigenvalue and corresponding normalized eigenvector for A:

FindRoot[{A.x == λ x, x.x == 1},{{λ, 1},{x, {1,2,3}}}]

Of course, this is not the best way to compute the eigenvalue, but it does show how the variable dimensions are picked up from the starting values. Since λ has a starting value of 1, it is taken to be a scalar. On the other hand, x is given a starting value, which is a vector of length 3, so it is always taken to be a vector of length 3.

If you use multiple starting values for variables, it is necessary that the values have consistent dimensions and that each component of the starting values is distinct.

This finds a different eigenvalue using two starting conditions for each variable:

FindRoot[{A.x == λ x, x.x == 1},{{λ, -2,-1},{x, {-1,0,0},{0,1,1}}}]

One advantage of variables that can take on vector and matrix values is that they allow you to write functions, which can be very efficient for larger problems and/or handle problems of different sizes automatically.

This defines a function that gives an objective function equivalent to the ExtendedRosenbrock problem in the (RefLinkPlain[UnconstrainedProblems, paclet:tutorial/UnconstrainedOptimizationTestProblems]) package. The function expects a value of x that is a matrix with two rows:

ExtendedRosenbrockObjective[x_ /; ((Length[x] == 2) && MatrixQ[x])] := Module[{x1, x2},
{x1, x2} = x;
x2 -= x1^2;
x1 -= 1;
x1.x1 + 100 x2.x2]

Note that since the value of the function would be meaningless unless x had the correct structure, the definition is restricted to arguments with that structure. For example, if you defined the function for any pattern x_, then evaluating with an undefined symbol x (which is what (RefLink[FindMinimum, paclet:ref/FindMinimum]) does) gives meaningless unintended results. It is often the case that when working with functions for vector-valued variables, you will have to restrict the definitions. Note that the definition above does not rule out symbolic values with the right structure. For example, ExtendedRosenbrockObjective[{{(Subscript[x, 11]),(Subscript[x, 12])},{(Subscript[x, 21]),(Subscript[x, 22])}}] gives a symbolic representation of the function for scalar  (Subscript[x, 11]), ….

This uses (RefLink[FindMinimum, paclet:ref/FindMinimum]) to solve the problem given a generic value for the problem size. You can change the value of n without changing anything else to solve problems of different size:

n = 10;
start = {Table[-1.2,{n}], Table[1., {n}]};
FindMinimum[ExtendedRosenbrockObjective[x], {x, start}]

The solution did not achieve the default tolerances due to the fact that the Wolfram Language was not able to get symbolic derivatives for the function, so it had to fall back on finite differences that are not as accurate.

A disadvantage of using vector- and matrix-valued variables is that the Wolfram Language cannot currently compute symbolic derivatives for them. Sometimes it is not difficult to develop a function that gives the correct derivative. (Failing that, if you really need greater accuracy, you can use higher-order finite differences.)

This defines a function that returns the gradient for the ExtendedRosenbrockObjective function. Note that the gradient is a vector obtained by flattening the matrix corresponding to the variable positions:

ExtendedRosenbrockGradient[x_ /; ((Length[x] == 2) && MatrixQ[x])] := Module[{x1, x2},
{x1, x2} = x;
x2 -= x1^2;
Flatten[{2 (x1 - 1) - 400 x1 x2, 200 x2}]]

This solves the problem using the symbolic value of the gradient:

n = 10;
start = {Table[-1.2,{n}], Table[1., {n}]};
FindMinimum[ExtendedRosenbrockObjective[x], {x, start}, Gradient->ExtendedRosenbrockGradient[x]]

Jacobian and Hessian derivatives are often sparse. You can also specify the structural sparsity of these derivatives when appropriate, which can reduce overall solution complexity by quite a bit.