Test Problems

All the test problems presented in [MGH81] have been coded into the Wolfram Language in the Optimization`UnconstrainedProblems` package. A data structure is used so that the problems can be processed for solution and testing with (RefLink[FindMinimum, paclet:ref/FindMinimum]) and (RefLink[FindRoot, paclet:ref/FindRoot]) in a seamless way. The lists of problems for (RefLink[FindMinimum, paclet:ref/FindMinimum]) and (RefLink[FindRoot, paclet:ref/FindRoot]) are in $FindMinimumProblems and $FindRootProblems, respectively, and a problem can be accessed using GetFindMinimumProblem and GetFindRootProblem.

Accessing (RefLink[FindMinimum, paclet:ref/FindMinimum]) problems.

Accessing (RefLink[FindRoot, paclet:ref/FindRoot]) problems.

GetFindMinimumProblem and GetFindRootProblem are both pass options to be used by other commands. They also accept the option (RefLink[Variables, paclet:ref/Variables])->vars which is used to specify what variables to use for the problems.

Specifying variable names.

This loads the package:

<<Optimization`UnconstrainedProblems`

This gets the Beale problem in a FindMinimumProblem data structure:

beale = GetFindMinimumProblem[Beale]

This gets the Powell singular function problem in a FindRootProblem data structure:

ps = GetFindRootProblem[PowellSingular, Variables->{x,y,z,w}]

Once you have a FindMinimumProblem or FindRootProblem object, in addition to simply solving the problem, there are various tests that you can run.

Operations with FindMinimumProblem and FindRootProblem data objects.

Any of the previous commands shown can take options that are passed on directly to (RefLink[FindMinimum, paclet:ref/FindMinimum]) or (RefLink[FindRoot, paclet:ref/FindRoot]) and override any options for these functions which may have been specified when the problem was set up.

This uses (RefLink[FindRoot, paclet:ref/FindRoot]) to solve the Powell singular function problem and gives the root:

ProblemSolve[ps]

This does the same as the previous example, but includes statistics on steps and evaluations required:

ProblemStatistics[ps]

This uses (RefLink[FindMinimum, paclet:ref/FindMinimum]) to solve the Beale problem and averages the timing over several trials to get the average time it takes to solve the problem:

ProblemTime[beale]

This uses (RefLink[FindMinimum, paclet:ref/FindMinimum]) to solve the Beale problem, compares the result with a reference solution, and gives a list of rules indicating the results of the test:

ProblemTest[beale]

ProblemTest gives a way to easily compare two different methods for the same problem.

This uses (RefLink[FindMinimum, paclet:ref/FindMinimum]) to solve the Beale problem using Newton's method, compares the result with a reference solution, and gives a list of rules indicating the results of the test:

ProblemTest[beale, Method->"Newton"]

Most of the rules returned by these functions are self-explanatory, but a few require some description. Here is a table clarifying those rules.

A very useful comparison is to see how a list of methods affect a particular problem. This is easy to do by setting up a FindMinimumProblem object and mapping a problem test over a list of methods.

This gets the Chebyquad problem. The output has been abbreviated to save space:

Short[cq = GetFindMinimumProblem[Chebyquad], 5]

Here is a list of possible methods:

methods = {Automatic, "QuasiNewton", {"QuasiNewton", "StepMemory"->10}, "Newton", {"Newton", "StepControl"->"TrustRegion"}, "ConjugateGradient"};

This makes a table comparing the different methods in terms of accuracy and computation time:

TableForm[Map[Join[{#}, {"Time", "FunctionAccuracy", "SpatialAccuracy"} /. ProblemTest[cq, Method->#]]&, methods]]

It is possible to generate tables of how a particular method affects a variety of problems by mapping over the names in $FindMinimumProblems or $FindRootProblems.

This sets up a function that tests a problem with (RefLink[FindMinimum, paclet:ref/FindMinimum]) using its default settings except with a large setting for (RefLink[MaxIterations, paclet:ref/MaxIterations]) so that the default (Levenberg–Marquardt) method can run to convergence:

TestDefault[problem_] := Join[{"Name"->problem}, ProblemTest[GetFindMinimumProblem[problem, MaxIterations->1000]]]

This makes a table showing some of the results from testing all the problems in $FindMinimumProblems. It may take several minutes to run:

TableForm[Map[({"Name", "Time", "Residual", "Jacobian", "FunctionAccuracy", "SpatialAccuracy"} /. TestDefault[#])&, $FindMinimumProblems]]

The two cases where the spatial accuracy is shown as ERROR are for linear problems, which do not have an isolated minimizer. The one case, which has a spatial accuracy that is quite poor, has multiple minimizers, and the method goes to a different minimum than the reference one. Many of these functions have multiple local minima, so be aware that the error may be reported as large only because a method went to a different minimum than the reference one.