The Uncertainties of Numerical Mathematics

The Wolfram System does operations like numerical integration very differently from the way it does their symbolic counterparts.

When you do a symbolic integral, the Wolfram System takes the functional form of the integrand you have given, and applies a sequence of exact symbolic transformation rules to it, to try and evaluate the integral.

However, when the Wolfram System does a numerical integral, after some initial symbolic preprocessing, the only information it has about your integrand is a sequence of numerical values for it. To get a definite result for the integral, the Wolfram System then effectively has to make certain assumptions about the smoothness and other properties of your integrand. If you give a sufficiently pathological integrand, these assumptions may not be valid, and as a result, the Wolfram System may simply give you the wrong answer for the integral.

This problem may occur, for example, if you try to integrate numerically a function which has a very thin spike at a particular position. The Wolfram System samples your function at a number of points, and then assumes that the function varies smoothly between these points. As a result, if none of the sample points come close to the spike, then the spike will go undetected, and its contribution to the numerical integral will not be correctly included.

Here is a plot of the function exp(-(x^2)):

Plot[Exp[-x^2],{x,-10,10},PlotRange->All]

(RefLink[NIntegrate, paclet:ref/NIntegrate]) gives the correct answer for the numerical integral of this function from -10 to +10:

NIntegrate[Exp[-x^2],{x,-10,10}]

If, however, you ask for the integral from -10000 to 10000, with its default settings (RefLink[NIntegrate, paclet:ref/NIntegrate]) will miss the peak near x==0, and give the wrong answer:

NIntegrate[Exp[-x^2],{x,-10000,10000}]

(RefLink[NIntegrate, paclet:ref/NIntegrate]) tries to make the best possible use of the information that it can get about the numerical values of the integrand. Thus, for example, by default, if (RefLink[NIntegrate, paclet:ref/NIntegrate]) notices that the estimated error in the integral in a particular region is large, it will take more samples in that region. In this way, (RefLink[NIntegrate, paclet:ref/NIntegrate]) tries to "adapt" its operation to the particular integrand you have given.

The kind of adaptive procedure that (RefLink[NIntegrate, paclet:ref/NIntegrate]) uses is similar, at least in spirit, to what (RefLink[Plot, paclet:ref/Plot]) does in trying to draw smooth curves for functions. In both cases, the Wolfram System tries to go on taking more samples in a particular region until it has effectively found a smooth approximation to the function in that region.

The kinds of problems that can appear in numerical integration can also arise in doing other numerical operations on functions.

For example, if you ask for a numerical approximation to the sum of an infinite series, the Wolfram System samples a certain number of terms in the series, and then does an extrapolation to estimate the contributions of other terms. If you insert large terms far out in the series, they may not be detected when the extrapolation is done, and the result you get for the sum may be incorrect.

A similar problem arises when you try to find a numerical approximation to the minimum of a function. The Wolfram System samples only a finite number of values, then effectively assumes that the actual function interpolates smoothly between these values. If in fact the function has a sharp dip in a particular region, then the Wolfram System may miss this dip, and you may get the wrong answer for the minimum.

If you work only with numerical values of functions, there is simply no way to avoid the kinds of problems discussed here. Exact symbolic computation, of course, allows you to get around these problems.

In many calculations, it is therefore worthwhile to go as far as you can symbolically, and then resort to numerical methods only at the very end. This gives you the best chance of avoiding the problems that can arise in purely numerical computations.