Random Number Generation

Introduction

The ability to generate pseudorandom numbers is important for simulating events, estimating probabilities and other quantities, making randomized assignments or selections, and numerically testing symbolic results. Such applications may require uniformly distributed numbers, nonuniformly distributed numbers, elements sampled with replacement, or elements sampled without replacement.

The functions (RefLink[RandomReal, paclet:ref/RandomReal]), (RefLink[RandomInteger, paclet:ref/RandomInteger]), and (RefLink[RandomComplex, paclet:ref/RandomComplex]) generate uniformly distributed random numbers. (RefLink[RandomVariate, paclet:ref/RandomVariate]) generates numbers for built-in distributions. (RefLink[RandomPrime, paclet:ref/RandomPrime]) generates primes within a range. The functions (RefLink[RandomChoice, paclet:ref/RandomChoice]) and (RefLink[RandomSample, paclet:ref/RandomSample]) sample from a list of values with or without replacement. The elements may have equal or unequal weights. A framework is also included for defining additional methods and distributions for random number generation.

A sequence of nonrecurring events can be simulated via (RefLink[RandomSample, paclet:ref/RandomSample]). For instance, the probability of randomly sampling the integers 1 through n in order might be simulated.

This estimates the probability of getting n elements in order for n from 2 to 8:

Block[{trials = 10^5, count, rn},
Table[
count = 0;
rn = Range[n];
Do[If[RandomSample[rn] == rn, count++], {trials}]; 
{n, N[count/trials]},
{n, 2, 8}]]

The results can be compared with the theoretical probabilities:

Table[{i,N[1/i!]},{i,2,8}]

Random number generation is at the heart of Monte Carlo estimates. An estimate of an expected value of a function f can be obtained by generating values from the desired distribution and finding the mean of f applied to those values.

This estimates the sixth raw moment for a normal distribution:

Mean[RandomVariate[NormalDistribution[0,2],10^6]^6]

In this case, the estimate can be compared with an exact result:

Expectation[x^6,x\[Distributed]NormalDistribution[0,2]]

Random processes can be simulated by generating a series of numbers with the desired properties. A random walk can be created by recursively summing pseudorandom numbers.

Here a random walk starting at 0 is created:

ListLinePlot[Join[{0.},Accumulate[RandomReal[{-1,1},{100}]]]]

Substitution of random numbers can be used to test the equivalence of symbolic expressions. For instance, the absolute difference between two expressions could be evaluated at randomly generated points to test for inequality of the expressions.

This provides no evidence that (Sqrt[(x^2)]) and |x| are different for real values:

Max[Abs[Sqrt[x^2]-Abs[x]/.x->RandomReal[{-10,10},10000]]]

This provides evidence that (Sqrt[(x^2)]) and |x| differ for at least some complex values:

Max[Abs[Sqrt[x^2]-Abs[x]/.x->RandomComplex[{-10+10I,10+10I},10000]]]

(RefLink[RandomPrime, paclet:ref/RandomPrime]) chooses prime numbers with equal probability, which can be useful—for instance, to generate large primes for RSA encryption. The prime numbers are uniformly distributed on the primes in the range but are not uniformly distributed on the entire range because primes are in general not uniformly distributed over ranges of positive integers.

Primes in a given range are generated with equal probability:

ListPlot[Tally[RandomPrime[50,10^5]],Filling->Axis,PlotRange->{0,Automatic}]

Random Generation Functions

The main functions are (RefLink[RandomReal, paclet:ref/RandomReal]), (RefLink[RandomInteger, paclet:ref/RandomInteger]), (RefLink[RandomComplex, paclet:ref/RandomComplex]), (RefLink[RandomVariate, paclet:ref/RandomVariate]), (RefLink[RandomChoice, paclet:ref/RandomChoice]), and (RefLink[RandomSample, paclet:ref/RandomSample]). (RefLink[RandomReal, paclet:ref/RandomReal]), (RefLink[RandomInteger, paclet:ref/RandomInteger]), and (RefLink[RandomComplex, paclet:ref/RandomComplex]) generate numbers given some range of numeric values. (RefLink[RandomVariate, paclet:ref/RandomVariate]) generates numbers from a statistical distribution. (RefLink[RandomChoice, paclet:ref/RandomChoice]) and (RefLink[RandomSample, paclet:ref/RandomSample]) generate elements from finite sets that may include non-numeric values.

Random Numbers

(RefLink[RandomReal, paclet:ref/RandomReal]) generates pseudorandom real numbers over a specified range of real values. (RefLink[RandomInteger, paclet:ref/RandomInteger]) generates pseudorandom integer numbers over a specified range of integer values. (RefLink[RandomComplex, paclet:ref/RandomComplex]) generates pseudorandom complex numbers over a specified rectangular region in the complex plane. (RefLink[RandomVariate, paclet:ref/RandomVariate]) generates pseudorandom numbers from a specified statistical distribution. (RefLink[RandomPrime, paclet:ref/RandomPrime]) generates prime numbers with equal probability within a range.

Generation of random reals.

Generation of random integers.

Generation of random complex numbers.

Generation of random values from a distribution.

Generation of random primes.

When the domain is specified in terms of (Subscript[x, min]) and (Subscript[x, max]), (RefLink[RandomReal, paclet:ref/RandomReal]) and (RefLink[RandomInteger, paclet:ref/RandomInteger]) generate uniformly distributed numbers over the specified range. (RefLink[RandomVariate, paclet:ref/RandomVariate]) uses rules defined for the specified distribution. Additionally, mechanisms are included for defining new methods and distributions.

The two-argument interface provides a convenient way to obtain multiple random numbers at once. Even more importantly, there is a significant efficiency advantage to generating a large number of pseudorandom numbers at once.

Generating (10^7) numbers between 0 and 1 takes a fraction of a second:

Timing[RandomReal[1,10^7];]

Generating (10^7) numbers one at a time takes roughly five times as long:

Timing[Table[RandomReal[1],{10^7}];]

For multidimensional arrays with dimensions (Subscript[n, 1]) through (Subscript[n, k]), the total number of required pseudorandom numbers (Subscript[n, total])==(∏_(i==1)^k)(Subscript[n, i]) is generated and then partitioned. This makes the multidimensional array generation as efficient as possible because the total number of random values is generated as efficiently as possible and the time required for partitioning is negligible.

The time required for a 100×100×100×10 array is about the same as for a vector of (10^7) numbers:

Timing[RandomInteger[100,{100,100,100,10}];]

Timing[RandomInteger[100,10^7];]

An array of the same dimensions generated 10 numbers at a time takes several times as long:

Timing[Table[RandomInteger[100,{10}],{100},{100},{100}];]

For statistical distributions, the speed advantage of generating many numbers at once can be even greater. In addition to the efficiency benefit inherited from the uniform number generators used, many statistical distributions also benefit from vectorized evaluation of elementary and special functions. For instance, (RefLink[WeibullDistribution, paclet:ref/WeibullDistribution]) benefits from vector evaluations of the elementary functions (RefLink[Power, paclet:ref/Power]), (RefLink[Times, paclet:ref/Times]), and (RefLink[Log, paclet:ref/Log]).

Generation of (10^5) Weibull numbers takes virtually no time:

Timing[RandomVariate[WeibullDistribution[2,1],10^5]//Length]

Several seconds are required when (10^5) Weibulls are generated one at a time:

Timing[Table[RandomVariate[WeibullDistribution[2,1]],{10^5}]//Length]

Random number generation can be useful in exploratory investigations. For instance, you might look for occurrences of a random sequence of digits in a longer sequence of digits.

This converts a list of 5 random decimal digits to a string:

digits = Apply[StringJoin, Map[ToString, RandomInteger[9, 5]]]

The following converts the first million digits of π to a string of integers:

pistring = StringJoin[Map[ToString,RealDigits[N[Pi, 10^6]][[1]]]];

This gives the positions where the string of five digits appears in the first million digits of π:

StringPosition[pistring,digits]

Random number generation is also highly useful in estimating distributions for which closed-form results are not known or known to be computationally difficult. Properties of random matrices provide one example.

This estimates the probability that a 5×5 matrix of uniform reals will have real eigenvalues:

Block[{count = 0,ev}, 
Do[ev = Eigenvalues[RandomReal[{0,1}, {5,5}]]; 
If[Re[ev] == ev,count++],{10^5}];
N[count/10^5]]

The following does the same for a matrix of standard normal numbers:

Block[{count = 0,ev}, 
Do[ev = Eigenvalues[RandomVariate[NormalDistribution[0,1], {5,5}]]; 
If[Re[ev] == ev,count++],{10^5}];
N[count/10^5]]

An example of simulating a multivariate distribution is the Gibbs sampler used in Bayesian statistics [1]. The Gibbs sampler provides a means by which to simulate values from multivariate distributions provided the distributions of each coordinate conditional on the other coordinates are known. Under some restrictions, the distribution of random vectors constructed by iteratively sampling from the conditional distributions will converge to the true multivariate distribution.

The following example will construct a Gibbs sampler for an example given by Casella and George [2]. The distribution of interest is bivariate. The conditional distribution of x given y is a binomial, and the conditional distribution of y given x is a beta. As Casella and George mention, various strategies for detecting convergence and sampling using the Gibbs sampler have been suggested. For simplicity, assume that convergence will occur within 1000 iterations. A sample of size n from the distribution will be taken as the n values following the 1000(^th) iteration. It should be noted that these n values will, however, be dependent.

This defines the sampler with a binomial and a beta conditional distribution:

sampler[len_]:=Block[{y0,dist1,dist2,x0},
y0=.5;
dist1[y_]:=RandomVariate[BinomialDistribution[16,y]];dist2[x_]:=RandomVariate[BetaDistribution[x+2,16-x+4]];
Do[{x0=dist1[y0],y0=dist2[x0]},{1000}];
Table[{x0=dist1[y0],y0=dist2[x0]},{len}]]

A Gibbs sampler could also be defined as a distribution object within the distribution framework for random number generation. An example of this particular Gibbs sampler as a distribution object is provided in "Defining Distributional Generators".

data is a sample of length (10^4):

data=sampler[10^4];

The following bar chart shows the marginal distribution of the first dimension:

frqs=Sort[Tally[data[[All,1]]]];
BarChart[frqs[[All,2]]/10^4,ChartLabels->frqs[[All,1]]]

The marginal distribution of the second coordinate can be visualized with a histogram:

Histogram[data[[All,-1]],Automatic,"ProbabilityDensity"]

Conditional distributions should closely match the assumed binomial and beta distributions provided there is enough data for the conditional distribution. The greatest amount of data occurs when the densities of the marginal distributions are highest, so those values can be used for comparisons. The following graphics compare the empirical and assumed conditional distributions, using bins of width .05 for estimating probabilities of continuous values.

This compares the empirical and theoretical distributions of x for 0.3`≤y<0.35`:

Block[{y=.3,bcounts,probs,cdata},
cdata=Select[data,y<=#[[2]]<y+.05&][[All,1]];
bcounts=BinCounts[cdata,{0,17}]/Length[cdata];
probs=Table[{x-.025,PDF[BinomialDistribution[16,y],x]},{x,0,16}];
ListPlot[{Transpose[{Range[0,16],bcounts}],probs},Filling->Axis]]

This compares the empirical and theoretical distributions of y for x==1:

Block[{x=1,bcounts,probs,cdata},
cdata=Select[data,#[[1]]==x&][[All,-1]];
bcounts=BinCounts[cdata,{0,1,.05}]/Length[cdata];
probs=Table[{y-.025,CDF[BetaDistribution[x+2,16-x+4],y]-CDF[BetaDistribution[x+2,16-x+4],y-.05]},{y,0,1,.05}];
ListPlot[{Transpose[{Range[.025,.975,.05],bcounts}],probs},Filling->Axis,PlotRange->All]]

Arbitrary-Precision Reals and Complexes

By default, (RefLink[RandomReal, paclet:ref/RandomReal]) and (RefLink[RandomComplex, paclet:ref/RandomComplex]) generate machine-precision numbers. (RefLink[RandomVariate, paclet:ref/RandomVariate]) generates machine numbers for continuous distributions by default. Arbitrary-precision numbers can be obtained by setting the (RefLink[WorkingPrecision, paclet:ref/WorkingPrecision]) option.

Option for (RefLink[RandomReal, paclet:ref/RandomReal]), (RefLink[RandomComplex, paclet:ref/RandomComplex]), and (RefLink[RandomVariate, paclet:ref/RandomVariate]).

The option is valid for uniformly distributed reals, complexes, and reals from built-in distributions. (RefLink[WorkingPrecision, paclet:ref/WorkingPrecision]) can also be incorporated into user-defined distributions.

Here is a precision-25 real number between 5 and 50:

RandomReal[{5,50},WorkingPrecision->25]

This gives a precision-50 t-distributed number:

RandomVariate[StudentTDistribution[10],WorkingPrecision->50]

Increased (RefLink[WorkingPrecision, paclet:ref/WorkingPrecision]) can be useful in simulations where loss of precision can be expected and highly accurate results are necessary. Increased precision can also be used to estimate the precision loss in computations.

This estimates the worst precision loss in computing (Subscript[J, 1]) on the interval [0,1000]:

100- Precision[BesselJ[1, RandomReal[{0,1000}, 1000, WorkingPrecision->100]]]

If the precision of the input is less than the specified (RefLink[WorkingPrecision, paclet:ref/WorkingPrecision]), the function will warn of the problem. The precision of the input will then be artificially increased to generate a pseudorandom number of the desired precision.

A warning is generated because the machine number 7.5 has precision less than 50:

RandomComplex[7.5+I,WorkingPrecision->50]

(RefLink[WorkingPrecision, paclet:ref/WorkingPrecision]) is not an option for (RefLink[RandomInteger, paclet:ref/RandomInteger]). Integers have infinite precision, so the precision is completely specified by the function name.

(RefLink[WorkingPrecision, paclet:ref/WorkingPrecision]) is not meaningful for pseudorandom integers:

RandomInteger[10,WorkingPrecision->50]

Random Elements

(RefLink[RandomChoice, paclet:ref/RandomChoice]) and (RefLink[RandomSample, paclet:ref/RandomSample]) generate pseudorandom selections from a list of possible elements. The elements can be numeric or non-numeric.

Random choice from a list.

Random sample from a list.

The main difference between (RefLink[RandomChoice, paclet:ref/RandomChoice]) and (RefLink[RandomSample, paclet:ref/RandomSample]) is that (RefLink[RandomChoice, paclet:ref/RandomChoice]) selects from the (Subscript[e, i]) with replacement, while (RefLink[RandomSample, paclet:ref/RandomSample]) samples without replacement. The number of elements chosen by (RefLink[RandomChoice, paclet:ref/RandomChoice]) is not limited by the number of elements in elist, and an element (Subscript[e, i]) may be chosen more than once. The size of a sample returned by (RefLink[RandomSample, paclet:ref/RandomSample]) is limited by the number of elements in elist, and the number of occurrences of a distinct element in that sample is limited by the number of occurrences of that element in elist.

If the first argument to (RefLink[RandomChoice, paclet:ref/RandomChoice]) or (RefLink[RandomSample, paclet:ref/RandomSample]) is a list, elements are selected with equal probability. The weight specification defines a distribution on the set of the (Subscript[e, i]). The weights must be positive, but need not sum to 1. For weights {(Subscript[w, 1]),…,(Subscript[w, n])} the probability of (Subscript[e, i]) in the initial distribution is (Subscript[w, i])/(∑_(j==1)^n)(Subscript[w, j]). Since (RefLink[RandomSample, paclet:ref/RandomSample]) samples without replacement, weights are updated internally based on the total remaining weight after each selection.

(RefLink[RandomChoice, paclet:ref/RandomChoice]) can be used for simulation of independent identically distributed events with a finite list of possible outcomes.

This gives 15 simulated fair coin tosses:

RandomChoice[{"heads","tails"},15]

This gives 20 rolls of a die loaded toward 5s:

RandomChoice[{.15,.1,.15,.15,.3,.15}->Range[6],20]

(RefLink[RandomChoice, paclet:ref/RandomChoice]) can be used to generate observations from any discrete distribution with finite support.

The following generates a random observation from a discrete analog of a (RefLink[TriangularDistribution, paclet:ref/TriangularDistribution]):

RandomChoice[{1,3,5,3,1}->Range[4,8]]

Here is the empirical PDF for 1000 simulated points:

frqs=Tally[RandomChoice[{1,3,5,3,1}->Range[4,8],1000]];
BarChart[frqs[[All,2]]/100,ChartLabels->frqs[[All,1]]]

(RefLink[RandomSample, paclet:ref/RandomSample]) can be used to simulate observations from a finite set of outcomes in which each element in the list of outcomes can only be observed once. There may be more than one occurrence of distinct values in the list.

This simulates 7 draws from a container of 80 blue and 45 red objects:

RandomSample[Join[Table["blue",{80}],Table["red",{45}]],7]

Randomly sampling all elements in the list results in a random permutation.

The following is a random permutation of the integers from 1 to 10:

RandomSample[Range[10]]

Assigning weights to the elements results in a random permutation in which values with greater weight tend to appear earlier in the permutation than values with lesser weight.

Here is a random permutation weighted by the squares of the data values:

RandomSample[Range[10]^2->Range[10]]

For the same list of weighted or unweighted elements, (RefLink[RandomSample, paclet:ref/RandomSample])[#,1]& is distributionally equivalent to (RefLink[RandomChoice, paclet:ref/RandomChoice]).

This gives an empirical PDF for (10^5) random samples of size 1:

data=Range[10];

tallies=Tally[Table[RandomSample[data^2->data,1],{10^5}]]

Sort[tallies][[All,2]]/10^5.

Here is an empirical distribution for a distributionally equivalent (RefLink[RandomChoice, paclet:ref/RandomChoice]):

Sort[Tally[RandomChoice[data^2->data,10^5]]][[All,2]]/10^5.

The probabilities for the two examples are very close to each other and to the theoretical values.

These are the theoretical probabilities:

N[data^2/Total[data^2]]

(RefLink[RandomSample, paclet:ref/RandomSample]) can also be used for random assignments to groups, such as in clinical trials. The following uses integers, but other identifying values such as name or identification number could be used instead.

The following randomly places 20 elements into four groups of equal size:

Partition[RandomSample[Range[20]],5]

(RefLink[RandomChoice, paclet:ref/RandomChoice]) and (RefLink[RandomSample, paclet:ref/RandomSample]) can be affected by changes to the (RefLink[Method, paclet:ref/Method]) option to (RefLink[SeedRandom, paclet:ref/SeedRandom]). Built-in methods are described in "Methods". Additionally, mechanisms for defining new methods are described in "Defining Your Own Generator".

Seeding and Localization

Pseudorandom number generators algorithmically create numbers that have some apparent level of randomness. Methods for pseudorandom number generation typically use a recurrence relation to generate a number from the current state and to establish a new state from which the next number will be generated. The state can be set by seeding the generator with an integer that will be used to initialize the recurrence relation in the algorithm.

Given an initial starting point, called a seed, pseudorandom number generators are completely deterministic. In many cases it is desirable to locally or globally set the seed for a random number generator to obtain a constant sequence of "random" values. If set globally, the seed will affect future pseudorandom numbers unless a new seed is explicitly set. If set locally, the seed will only affect random number and element generation within the localized code.

Localization and seeding functions.

The (RefLink[SeedRandom, paclet:ref/SeedRandom]) function provides a means by which to seed the random generator. Used on its own, (RefLink[SeedRandom, paclet:ref/SeedRandom]) will globally set the seed for random generators. The (RefLink[BlockRandom, paclet:ref/BlockRandom]) function provides a means by which to locally set or change the seed for random generators without affecting the global state.

The following seeds the random generator globally:

{SeedRandom[1];RandomReal[],SeedRandom[1];RandomReal[]}

The following gives two different numbers because the first (RefLink[RandomReal, paclet:ref/RandomReal]) is generated within (RefLink[BlockRandom, paclet:ref/BlockRandom]), while the second is generated outside of (RefLink[BlockRandom, paclet:ref/BlockRandom]).

The second (RefLink[RandomReal, paclet:ref/RandomReal]) is not generated using the seed 1:

{BlockRandom[SeedRandom[1];RandomReal[]],BlockRandom[SeedRandom[1]];RandomReal[]}

(RefLink[SeedRandom, paclet:ref/SeedRandom]) also provides the mechanism for switching the random generator.

Option for (RefLink[SeedRandom, paclet:ref/SeedRandom]).

An individual generator can be seeded directly by specifying that generator via the (RefLink[Method, paclet:ref/Method]) option. All generators can be seeded by setting (RefLink[Method, paclet:ref/Method])->(RefLink[All, paclet:ref/All]).

Here the default generator is seeded with 1, but the "Rule30CA" generator is not:

BlockRandom[SeedRandom[1];
SeedRandom[Method->"Rule30CA"];
RandomReal[]]

Seeding the "Rule30CA" generator with 1 gives a different random number:

BlockRandom[SeedRandom[1,Method->"Rule30CA"];
RandomReal[]]

SeedRandom and BlockRandom in Parallel Computations

There are some subtleties to using the commands (RefLink[SeedRandom, paclet:ref/SeedRandom]) and (RefLink[BlockRandom, paclet:ref/BlockRandom]) in parallel computations. Within a parallel computation, these commands only affect the generator that is used in the current thread. Typically you will want to use these before or enclosing an entire parallel computation.

For parallel computations it is very advantageous to have a generator on each thread that produces random numbers independent from the generators on other threads. In the Wolfram System each thread used in a parallel computation will be given a unique index starting from zero (and typically going sequentially through (RefLink[$ProcessorCount, paclet:ref/$ProcessorCount])) that will be used to give different seeds and generators on each thread.

The table below describes some of the differences between using these in serial and parallel.

(RefLink[SeedRandom, paclet:ref/SeedRandom]) and (RefLink[BlockRandom, paclet:ref/BlockRandom]) in serial and parallel computations.

This defines a (RefLink[CompiledFunction, paclet:ref/CompiledFunction]) that approximates the area of a quarter circle using n samples that will run in parallel when given a list:

cf = Compile[{{n, _Integer}}, Total[Map[If[Norm[#]>1.,0.,1.]&,RandomReal[1,{n,2}]]]/n, RuntimeAttributes->Listable, Parallelization->True];

This runs the (RefLink[CompiledFunction, paclet:ref/CompiledFunction]) in parallel after seeding all generators:

SeedRandom[1234]; t1 = cf[ConstantArray[10^4,20]]

This runs it again, but with the parallel computation done inside (RefLink[BlockRandom, paclet:ref/BlockRandom]):

SeedRandom[1234]; t2 =BlockRandom[cf[ConstantArray[10^4,20]]]

The results are different in spite of having the same seed. Most of the difference is in the ordering, since the parallel scheduler may run one thread before another when a computation is repeated.

This compares the results:

Intersection[t1, t2]

Many, but not all of the same results are found in both computations. This is because there is no guarantee that a given thread will be used exactly the same number of times when a computation is repeated.

Since the previous parallel computation was done inside (RefLink[BlockRandom, paclet:ref/BlockRandom]), the parallel generators have been restored to the state they were in before, so running again will effectively be a repeat:

t3 =BlockRandom[cf[ConstantArray[10^4,20]]];
Intersection[t1,t2,t3]

Use of (RefLink[SeedRandom, paclet:ref/SeedRandom]) and (RefLink[BlockRandom, paclet:ref/BlockRandom]) inside a parallel computation should be done with care, since different parts done with the same thread may wind up with identical results.

This defines the same (RefLink[CompiledFunction, paclet:ref/CompiledFunction]) as before, but with a (RefLink[SeedRandom, paclet:ref/SeedRandom]) done before using (RefLink[RandomReal, paclet:ref/RandomReal]):

cfws = Compile[{{n, _Integer}}, SeedRandom[1234];Total[Map[If[Norm[#]>1.,0.,1.]&,RandomReal[1,{n,2}]]]/n, RuntimeAttributes->Listable, Parallelization->True];

This runs the (RefLink[CompiledFunction, paclet:ref/CompiledFunction]) in parallel:

t4 = cfws[ConstantArray[10^4,20]]

You may notice that some of the results appear the same. This can be checked using (RefLink[Union, paclet:ref/Union]).

This gets the distinct sums from the result:

Union[t4]

So in this case, there are only eight distinct sums out of 20. If you run this, the length of the union will typically be equal to the number of processors you have on your machine. This is because the generator for each processor is reseeded before each use, and since the use of (RefLink[RandomReal, paclet:ref/RandomReal]) in each case is the same, the results are identical.

This defines the same (RefLink[CompiledFunction, paclet:ref/CompiledFunction]) as before, but with a (RefLink[RandomReal, paclet:ref/RandomReal]) used inside (RefLink[BlockRandom, paclet:ref/BlockRandom]):

cfwb = Compile[{{n, _Integer}}, BlockRandom[Total[Map[If[Norm[#]>1.,0.,1.]&,RandomReal[1,{n,2}]]]/n], RuntimeAttributes->Listable, Parallelization->True];

This runs the (RefLink[CompiledFunction, paclet:ref/CompiledFunction]) in parallel:

t5 = cfwb[ConstantArray[10^4,20]]

This gets the distinct sums from the result:

Union[t5]

One thing you can do with (RefLink[SeedRandom, paclet:ref/SeedRandom]) inside a parallel computation is to set the generator. Suppose that you want to set the generator on each thread to be the default "ExtendedCA" generator with different seeds.

This defines a compiled function that changes the random generator to the "ExtendedCA" method and seeds it with seed s:

cfg = Compile[{{s, _Integer}}, SeedRandom[s, Method->"ExtendedCA"], RuntimeAttributes->Listable, Parallelization->True];

This gives a randomly chosen seed for each generator:

seeds = RandomInteger[{1, 2^16}, $ProcessorCount];

This runs the (RefLink[CompiledFunction, paclet:ref/CompiledFunction]) in parallel. Only the parallel random generators are affected by this:

cfg[seeds]

Running the area approximation function in parallel will use these generators:

t6=BlockRandom[cf[ConstantArray[10^4,$ProcessorCount]]]

You can verify that these generators were used by comparing to a serial computation where the generator is set the same way.

Compute in serial, locally setting the generator the same way the parallel ones were set:

t7 = Map[BlockRandom[SeedRandom[#, Method->"ExtendedCA"]; cf[10^4]]&, seeds]

The parallel result is just a permutation of this.

Verify that the parallel result is a permutation of the serial result:

p = Range[$ProcessorCount];
p[[Ordering[t6]]] = p;
p = t7[[p]]

t6 === t7[[p]]

Setting up generators in this way is not advisable since just changing the seed with the same generator does not give any guarantee that the generated numbers are not correlated in some way.

An easier and more reliable way of setting up parallel generators is provided with the "ParallelGenerator" method described in "Methods".

Methods

Five pseudorandom generator methods are available on all systems. Of those five, the Mersenne Twister method is provided in both a serial and parallel version. A sixth platform-dependent method is available on Intel-based systems. A method name is used for handling generators for parallel computations. A framework for defining new methods, described in "Defining Your Own Generator", is also included.

Built-in methods.

This gives pseudorandom integers from each method with seed 2020:

Map[BlockRandom[SeedRandom[2020,Method->#];RandomInteger[10^20]]&,{"Congruential","ExtendedCA","Legacy","MersenneTwister","MKL","Rule30CA"}]

This gives pseudorandom reals from the same seed:

Map[BlockRandom[SeedRandom[2020,Method->#];RandomReal[]]&,{"Congruential","ExtendedCA","Legacy","MersenneTwister","MKL","Rule30CA"}]

Congruential

"Congruential" uses a linear congruential generator. This is one of the simplest types of pseudorandom number generators, with pseudorandom numbers between 0 and 1 obtained from (Subscript[x, i])/m, where (Subscript[x, i]) is given by the modular recurrence relation

for some fixed integers b, c, and m called the multiplier, increment, and modulus, respectively. If the increment is 0, the generator is a multiplicative congruential generator. The values of b, c, and m can be set via options to the "Congruential" method.

Options for (RefLink[Method, paclet:ref/Method]) "Congruential".

Linear congruential generators are periodic and tend to give a lower quality of randomness, especially when a large number of random values is needed. If reals are generated directly from the congruence relation, the period is less than or equal to m.

The default option values are chosen to have a large period and for 64-bit efficiency. With the default options, the "Congruential" generator passes many standard tests of randomness despite the inherent issues with congruential number generators.

This generates 40 numbers from a multiplicative congruential generator:

lcdata=BlockRandom[SeedRandom[1,Method->{"Congruential","Multiplier"->11,"Increment"->0,"Modulus"->63}];RandomReal[1,40]];

The period of a multiplicative congruential generator is bounded above by the number of positive integers less than or equal to the modulus that are relatively prime to the modulus. This upper bound is Euler's totient function of the modulus.

With a modulus of 63, the period of the cycle is at most 36:

EulerPhi[63]

The actual period can be determined by finding the smallest integer i such that i≡(b^i)  mod m.

The period with multiplier 11 and modulus 63 is 6:

First[Select[Range[36],(Mod[11^#,63]===1&)]]

Partitioning the data into sets of 6 elements shows the recursion:

Partition[lcdata,6]

The distinct numbers can also be seen graphically by plotting a sequence of generated numbers.

Here is a plot of 1000 values from the congruential generator:

ListPlot[BlockRandom[SeedRandom[1,Method->{"Congruential","Multiplier"->11,"Increment"->0,"Modulus"->63}];RandomReal[1,1000]]]

If "ConvertToRealsDirectly" is set to (RefLink[False, paclet:ref/False]), reals are generated by taking eight bits at a time from elements of the sequence to construct a 52-bit machine-precision number. Congruential numbers generated in this fashion will still cycle, but cycling will depend on repetition in the bit pattern rather than in the initial congruence relation.

The "Bits" option can be (RefLink[Automatic, paclet:ref/Automatic]), a nonzero integer, or a list of two nonzero integers specifying the range of bits in the modulus m used for constructing numbers from bits. (RefLink[Automatic, paclet:ref/Automatic]) uses {2,-1} unless m is a power of 2, in which case {1,-1} is used.

ExtendedCA

The default "ExtendedCA" method makes use of cellular automata to generate high-quality pseudorandom numbers. This generator uses a particular five-neighbor rule, so each new cell depends on five nonadjacent cells from the previous step.

Cellular-automata-based random number generators evolve a state vector of 0s and 1s according to a deterministic rule. For a given cellular automaton, an element (or cell) at a given position in the new state vector is determined by certain neighboring cells of that cell in the old state vector. A subset of cells in the state vectors is then output as random bits from which the pseudorandom numbers are generated.

The cellular automaton used by "ExtendedCA" produces an extremely high level of randomness. It is so high that even using every single cell in output will give a stream of bits that passes many randomness tests, in spite of the obvious correlation between one cell and five previous ones.

Options are included for modifying the size of the state vector, the cells skipped, and the starting cell. The defaults are chosen for quality and speed and there is typically no need to modify these options.

Options for (RefLink[Method, paclet:ref/Method]) "ExtendedCA".

The length of the state vectors used is by default set to 80×64==5120… cells. The multiple of 64 can be controlled by the "Size" option. Once a state vector is computed by evolving the cellular automaton using the five-neighbor rule, bits are selected for random numbers from bits {start,start+skip,…}.

In practice, using every fourth cell in each state vector proves to be sufficient to pass very stringent randomness tests. This is the default used for the "Skip" option. For even faster random number generation, a "Skip" setting of 2 or even 1 could be used, but the quality of the random numbers will then decline.

The "Start" option tied with a larger "Size" and "Skip" is useful for setting up a family of independent generators that can be used in parallel computations.

"ExtendedCA" is the default number generator:

BlockRandom[SeedRandom[1];RandomReal[1,5]]

BlockRandom[SeedRandom[1,Method->"ExtendedCA"];RandomReal[1,5]]

Legacy

The "Legacy" method uses the generator called by (RefLink[Random, paclet:ref/Random]) in versions of Mathematica prior to Version 6.0. A Marsaglia–Zaman subtract-with-borrow generator is used for reals. The integer generator is based on a Wolfram rule 30 cellular automaton generator. The rule 30 generator is used directly for small integers and used to generate certain bits for large integers.

Here are (RefLink[RandomReal, paclet:ref/RandomReal]) and (RefLink[RandomInteger, paclet:ref/RandomInteger]) values obtained via the "Legacy" method:

BlockRandom[SeedRandom[31,Method->"Legacy"];{RandomReal[],RandomInteger[50]}]

The same values are given by equivalent (RefLink[Random, paclet:ref/Random]) calls:

BlockRandom[SeedRandom[31];{Random[],Random[Integer,{0,50}]}]

To guarantee consistency with sequences generated prior to Version 6.0, seeds set for the (RefLink[Automatic, paclet:ref/Automatic]) method are also applied to the "Legacy" method.

The "Legacy" method has no options.

MersenneTwister

"MersenneTwister" uses the Mersenne Twister generator due to Matsumoto and Nishimura [3][4]. The Mersenne Twister is a generalized feedback shift register generator with period (2^19937)-1.

This gives 5 random numbers from a Mersenne Twister generator:

BlockRandom[SeedRandom[1,Method->"MersenneTwister"];RandomReal[1,5]]

The "MersenneTwister" method has no options.

MKL

The "MKL" method uses the random number generators provided in Intel's MKL libraries.

Option for (RefLink[Method, paclet:ref/Method]) "MKL".

"MKL" methods.

The first six methods are uniform generators. "Niederreiter" and "Sobol" generate Niederreiter and Sobol sequences. These sequences are nonuniform and have underlying structure which is sometimes useful in numerical methods. For instance, these sequences typically provide faster convergence in multidimensional Monte Carlo integration.

The following shows the structure of a Niederreiter sequence in dimension 2:

ListPlot[BlockRandom[
SeedRandom[Method->{"MKL", Method->{"Niederreiter", "Dimension"->2}}];RandomReal[1,{1000,2}]], 
AspectRatio->1]

This shows the structure of a Sobol sequence in dimension 2:

ListPlot[BlockRandom[
SeedRandom[Method->{"MKL", Method->{"Sobol", "Dimension"->2}}];RandomReal[1,{1000,2}]], 
AspectRatio->1]

Rule30CA

The "Rule30CA" method uses a Wolfram rule 30 cellular automaton generator. Bits are obtained by evolving a state vector of 0s and 1s using the relation

where f(i,t) is the value of cell i at time t.

Option for (RefLink[Method, paclet:ref/Method]) "Rule30CA".

The length of the state vectors used is by default set to 9×29==261 cells. The multiplier for 29 can be controlled by the "Size" option.

This gives a 2×3×4 tensor of random integers using "Rule30CA":

BlockRandom[SeedRandom[1,Method->"Rule30CA"];RandomInteger[10^5,{2,3,4}]]

The "Rule30CA" method uses only the first bit from each state vector, making it slower than the "ExtendedCA" method, which uses multiple bits from each state vector.

ParallelMersenneTwister

"ParallelMersenneTwister" uses a set of Mersenne Twister generators due to Matsumoto and Nishimura [3][4] with parameters chosen using their "Dynamic Creator" program dcmt [19]. The program computes parameters for the Mersenne Twister generator that are relatively prime and so should produce independent results. The parameters were computed to produce Mersenne Twister generalized feedback shift register generators with period (2^4423)-1.

An option is included to choose which of the set of generators to use.

Option for (RefLink[Method, paclet:ref/Method]) "ParallelMersenneTwister".

This gives two sets of 2500 random numbers from different parallel Mersenne Twister generators and makes a plot of the pairs as points:

data = Table[BlockRandom[SeedRandom[1,Method->{"ParallelMersenneTwister", "Index"->i}]; RandomReal[1, 2500]], {i,0,1}];
ListPlot[Transpose[data], AspectRatio->1]

There are no apparent correlations between the numbers produced by the two generators. Because of the lack of correlation and the speed, this set of generators is used as the default generators for parallel computations.

ParallelGenerator

"ParallelGenerator" is a controller method that allows you to seed and change the generators used for parallel computations.

An option is included to choose which of the set of generators to use.

Option for (RefLink[Method, paclet:ref/Method]) "ParallelGenerator".

The value of the (RefLink[Method, paclet:ref/Method]) option given to the "ParallelGenerator" method can be a string specifying a built-in parametrized method or a function that will give a random generator specification for non-negative integers. Each thread used in a parallel computation will be given a unique index starting from zero (and typically going sequentially through (RefLink[$ProcessorCount, paclet:ref/$ProcessorCount])) that will be used to give different seeds and generators on each thread.

Methods for parallel generators.

The string shortcuts are provided as convenient ways to get to two high-quality independent sets of generators.

Using "ParallelMersenneTwister" is equivalent to using the function f=(RefLink[Function, paclet:ref/Function])[{i},{"ParallelMersenneTwister","Index"->i}]. This is the default for parallel computations since the generators are fast and produce good quality random numbers.

Using "ExtendedCA" is typically equivalent to using the function f defined below with the number of processors on your machine.

"Default" resets the method to the default "ParallelMersenneTwister" method.

This defines the default function for the "ParallelGenerator" option (RefLink[Method, paclet:ref/Method])->"ExtendedCA":

f = With[{n = $ProcessorCount}, Function[{"ExtendedCA", "Size"->80 n, "Skip"->4 n, "Start"->4 #}]]

The parameters are chosen so that if you use all of the (RefLink[$ProcessorCount, paclet:ref/$ProcessorCount]) processors on your machine you will still get random numbers as good as the default serial "ExtendedCA" random generator.

The "ParallelGenerator" method also does generator seeding in a slightly different way. Instead of just using the same seed on each processor, (RefLink[SeedRandom, paclet:ref/SeedRandom])[seed,(RefLink[Method, paclet:ref/Method])->"ParallelGenerator"] uses seed+i on each thread where i is the unique index for that thread. This allows you to get different numbers from different threads even if you set the generator on each thread to be the same (e.g. (RefLink[Method, paclet:ref/Method])->(RefLink[Function, paclet:ref/Function])[{i},"ExtendedCA"]), though that is not advisable since even with different seeds the numbers could have unexpected correlations.

In general, the function f to give generator methods for different threads can return anything that is a legitimate random generator method.

Here is a (RefLink[CompiledFunction, paclet:ref/CompiledFunction]) that will run in parallel when given a list:

cf = Compile[{{n, _Integer}}, Total[RandomInteger[1, n]], RuntimeAttributes->Listable, Parallelization->True];

This seeds the parallel generators:

SeedRandom[1234, Method->"ParallelGenerator"]

This runs the (RefLink[CompiledFunction, paclet:ref/CompiledFunction]) in parallel:

cf[ConstantArray[10^4, $ProcessorCount]]

This defines a function that gives a different generator method for indices between 0 and 7:

gfun = Function[{index}, Switch[Mod[index, 8],
0, "ExtendedCA",
1, "Rule50025CA",
2, "Rule30CA",
3, "MersenneTwister", 
4, "Congruential",
5, {"MKL", Method->"MCG31"},
6, {"MKL", Method->"MCG59"},
7, {"MKL", Method->"MRG32K3A"}]];

This changes the parallel generators to be the ones given by the function and seeds them:

SeedRandom[1234, Method->{"ParallelGenerator", Method->gfun}]

This runs the compiled function in parallel using the selected generators:

t1 = cf[ConstantArray[10^4, $ProcessorCount]]

This does the computation serially, setting the generator locally to the one given by the function:

t2 = Table[BlockRandom[SeedRandom[1234+index, Method->gfun[index]]; cf[10^4]], {index, 0, 7}]

The results are the same up to order:

Intersection[t1, t2]

To restore the parallel generators to their default method, you need to explicitly give a method option, otherwise, it just changes the seed.

This restores the parallel generators to the default method:

SeedRandom[Method->{"ParallelGenerator", Method->"Default"}]

Defining Your Own Generator

Methods can be plugged into the random framework as long as they follow the correct template. A generator object is of the form gsym[data] where gsym is the symbol that identifies the generator and to which rules are attached. data is effectively private to the top-level evaluations associated with the generator definitions.

Generator initialization is handled by a call to Random`InitializeGenerator.

Generator initialization function.

Random`InitializeGenerator is expected to return a generator object gobj of the form gsym[data].

Generators can support generation of random bit streams, random integers, and random reals. If the generator supports bit streams, reals and integers can be generated by conversion of the bit stream. At method setup time, properties are queried to determine what is supported and how.

Generator properties.

If bit streams are supported, then gobj["GenerateBits"[nbits]] is expected to return an integer comprised of n random bits or a list of length nbits with entries that are 0 or 1.

If random integers are supported, then gobj["GenerateIntegers"[n,{a,b}]] is expected to return a list of n random integers in the range [a,b]. A warning message will be issued when results are out of range.

If random reals are supported, then gobj["GenerateReals"[n,{a,b},prec]] is expected to return a list of n random reals with precision prec in the range [a,b]. A warning message will be issued when results are out of range or of the wrong precision.

For any of the generation functions, the return can be {res,gobj}, where res is the result of the correct type and gobj is a new generator object (reflecting any state change).

Seeding is done by gobj["SeedGenerator"[seed]] for an integer seed. gobj["SeedGenerator"[seed]] is expected to return a new generator object.

Example: Multiplicative Congruential Generator

In the following example a multiplicative congruential generator will be defined. A multiplicative congruential generator follows the recurrence relation

The generator, as defined below, will allow only for generation of real numbers.

This sets default options for the generator MultiplicativeCongruential:

Options[MultiplicativeCongruential] = {"Multiplier"->123456789, "Modulus"->2^35-1};

Initialization of the generator will extract the values of the multiplier and modulus. Initialization will fail if either of these values is not a positive integer.

The following initializes the generator:

MultiplicativeCongruential /: Random`InitializeGenerator[MultiplicativeCongruential, opts___] := Module[{mult, mod, flops = Flatten[{opts, Options[MultiplicativeCongruential]}]},
mult = "Multiplier" /. flops;
If[!(IntegerQ[mult]&&Positive[mult]),
Throw[$Failed]
];
mod = "Modulus" /. flops;
If[!(IntegerQ[mod]&&Positive[mult]),
Throw[$Failed]
];
MultiplicativeCongruential[mult, mod,1]];

Calls from the kernel to Random`IntializeGenerator are effectively wrapped in (RefLink[Catch, paclet:ref/Catch]). (RefLink[Throw, paclet:ref/Throw]) can be used in the initialization code to easily exit in case of problems.

This establishes that MultiplicativeCongruential generates reals:

MultiplicativeCongruential[___]["GeneratesRealsQ"] := True;

The following seeds the generator using the recurrence relation:

MultiplicativeCongruential[mult_,mod_, ___]["SeedGenerator"[seed_]] := 
MultiplicativeCongruential[mult,mod, Mod[(mult*seed),mod]];

The real number generator will return the desired number of reals and a new MultiplicativeCongruential generator. The seed for the new generator is updated based on the recurrence relation.

This defines the real number generator:

MultiplicativeCongruential[mult_,mod_, s_]["GenerateReals"[n_,{a_,b_},prec_]] := 
Module[{x=s},
{a+(b-a)Table[x=mult*x;Mod[x,mod],{n}]/mod,MultiplicativeCongruential[mult,mod, x]}
]

This generates 10 reals using the MultiplicativeCongruential generator:

BlockRandom[
SeedRandom[Method->MultiplicativeCongruential];
RandomReal[{5,50},10]]

The generator is not defined for integers:

BlockRandom[
SeedRandom[Method->MultiplicativeCongruential];
RandomInteger[{5,50}]]

Example: Blum–Blum–Shub Generator

The Blum–Blum–Shub generator is a quadratic congruential method for generating pseudorandom bits for cryptographic purposes [5]. The congruence is mod p×q for specified primes p and q.

This sets default options for the generator BlumBlumShub:

Options[BlumBlumShub] = {"BlumPrimes"->{1267650600228229401496703981519, 1267650600228229401496704318359}, "BitWidth"->Automatic};

The following define an auxiliary function and error messages for the generator:

SpecialBlumPrimeQ[x_] := (Positive[x] && (Mod[x, 4] == 3) && PrimeQ[x])

BlumBlumShub::bprime = "`1` is not a list of two distinct special Blum primes.";

BlumBlumShub::bw = "Warning: the value of the option BitWidth->`1` exceeds the number `2` that has been proved cyptographically secure.";

BlumBlumShub::bw1 = "The value of the option BitWidth->`1` should be a positive machine sized integer or Automatic.";

The generator initialization will extract option values and issue error messages if necessary before calling the actual generator.

The following initializes the generator:

BlumBlumShub /: Random`InitializeGenerator[BlumBlumShub, opts___] := Module[{n, abw, bw, flops = Flatten[{opts, Options[BlumBlumShub]}]},
n = "BlumPrimes" /. flops;
If[!And[VectorQ[n, SpecialBlumPrimeQ], Length[n] == 2, Not[Apply[Equal, n]]],
Message[BlumBlumShub::"bprime", n];
Throw[$Failed]
];
n = Apply[Times, n];
abw = Max[1,Floor[Log[2., Log[2.,n]]]];
bw = "BitWidth" /. flops;
If[bw === Automatic,
bw = abw,
If[!(IntegerQ[bw] && Positive[bw]),
Message[BlumBlumShub::bwi, bw];
Throw[$Failed]];
If[bw > abw,
Message[BlumBlumShub::bw, bw, abw]];
];
BlumBlumShub[n, bw, 2^bw - 1, 2]];

This establishes that BlumBlumShub is a bit generator and determines the bit width:

BlumBlumShub[___]["GeneratesBitsQ"] := True;

BlumBlumShub[n_, bw_, __]["BitWidth"] := bw;

The following seeds the generator:

BlumBlumShub[n_,bw_, mask_,  ___]["SeedGenerator"[seed_]] := 
Module[{x , i = 0, state = {}},
While[Length[Union[state]] < 10,
x = seed + i;
state = NestList[PowerMod[#, 2, n]&, x, 9];
];
BlumBlumShub[n, bw, mask,  Last[state]]]

This defines the bit generator:

BlumBlumShub[n_, bw_, mask_, s_]["GenerateBits"[bits_]] := 
Module[{x = PowerMod[s, 2, n]},
{BitAnd[x, mask], BlumBlumShub[n,bw, mask, x]}]

This generates 5 integers and 5 reals using the BlumBlumShub generator:

BlockRandom[
SeedRandom[Method->BlumBlumShub];
{RandomInteger[{0,10},5],RandomReal[4,5]}]

Statistical Distributions

The general idea behind generating random variates from a nonuniform statistical distribution is to generate a random uniform variate between 0 and 1 and then compute the inverse CDF of that random value in the desired distribution. In practice, however, following this recipe directly can be very computationally intensive if a large number of random variates is desired, particularly when the inverse CDF is complicated or cannot be expressed in a closed form.

In such cases, table lookups, direct construction based on distributional relationships, or acceptance-rejection methods are often more efficient alternatives to direct inversion of the CDF. On some level, these methodologies will all still rely on uniformly distributed (RefLink[RandomReal, paclet:ref/RandomReal]) values, uniformly distributed (RefLink[RandomInteger, paclet:ref/RandomInteger]) values, observations from a weighted (RefLink[RandomChoice, paclet:ref/RandomChoice]), or a combination of these values. As a result, methods set via (RefLink[SeedRandom, paclet:ref/SeedRandom]) will have an effect on random observations from statistical distributions.

Random observations from all built-in statistical distributions can be generated using (RefLink[RandomVariate, paclet:ref/RandomVariate]). The methods used by (RefLink[RandomVariate, paclet:ref/RandomVariate]) for many of the distributions in the Wolfram Language follow methods suggested or described in Gentle [6] or other literature.

Generation of random values from statistical distributions.

Observations from statistical distributions are obtained via (RefLink[RandomVariate, paclet:ref/RandomVariate]). This includes all built-in distributions and constructors including univariate and multivariate distributions, continuous and discrete distributions, parametric and derived distributions, and distributions defined by data.

This generates a number for a continuous distribution and a discrete distribution:

{RandomVariate[ChiSquareDistribution[8]],RandomVariate[PoissonDistribution[100]]}

(RefLink[WorkingPrecision, paclet:ref/WorkingPrecision]) can be used to get higher-precision values for continuous distributions just as it is for uniform numbers over ranges.

Here is a precision-30 beta-distributed variate:

RandomVariate[BetaDistribution[3,7],WorkingPrecision->30]

Random values from multivariate distributions can be generated in the same way.

Here is a random vector from a bivariate normal distribution:

RandomVariate[MultinormalDistribution[{1,2},{{2,.5},{.5,3}}]]

This is a random vector from a multinomial distribution:

RandomVariate[MultinomialDistribution[20,{1/3,1/2,1/6}]]

Here a random value is generated from a distribution defined by its PDF:

dist=ProbabilityDistribution[E^-x^3/Gamma[4/3],{x,0,Infinity}];
RandomVariate[dist]

In the following sections, methodologies for generating random variates are discussed with some specific examples of where such methods are employed in the Wolfram Language.

Continuous Distributions

For univariate distributions whose inverse CDFs contain only elementary functions, direct computation of the inverse CDF for a random uniform is generally used. This can be seen as a direct construction from a uniformly distributed random variable. Some continuous distributions falling in this category include (RefLink[CauchyDistribution, paclet:ref/CauchyDistribution]), (RefLink[ExponentialDistribution, paclet:ref/ExponentialDistribution]), (RefLink[ExtremeValueDistribution, paclet:ref/ExtremeValueDistribution]), (RefLink[GumbelDistribution, paclet:ref/GumbelDistribution]), (RefLink[LaplaceDistribution, paclet:ref/LaplaceDistribution]), (RefLink[LogisticDistribution, paclet:ref/LogisticDistribution]), (RefLink[ParetoDistribution, paclet:ref/ParetoDistribution]), (RefLink[RayleighDistribution, paclet:ref/RayleighDistribution]), (RefLink[TriangularDistribution, paclet:ref/TriangularDistribution]), and (RefLink[WeibullDistribution, paclet:ref/WeibullDistribution]).

Direct construction of a single random variate from multiple uniform variates, or from variates other than the uniform distribution are also employed. Normal variates are generated in pairs from pairs of random uniforms using the Box–Müller method. (RefLink[HalfNormalDistribution, paclet:ref/HalfNormalDistribution]) and (RefLink[LogNormalDistribution, paclet:ref/LogNormalDistribution]), and (PackageLink[MultinormalDistribution, paclet:MultivariateStatistics/ref/MultinormalDistribution, MultivariateStatistics Package Symbol]) variates, for instance, are obtained by direct transformation of normal variates.

(RefLink[InverseGaussianDistribution, paclet:ref/InverseGaussianDistribution]) uses an acceptance-complement method involving normal and uniform variates. The method is due to Michael, Schucany, and Haas and described in Gentle [6]. (RefLink[MaxwellDistribution, paclet:ref/MaxwellDistribution]) variates are constructed from (RefLink[ChiDistribution, paclet:ref/ChiDistribution]) variates. The chi variates themselves are obtained from (RefLink[ChiSquareDistribution, paclet:ref/ChiSquareDistribution]) variates, which are special cases of (RefLink[GammaDistribution, paclet:ref/GammaDistribution]) variates.

In most cases (RefLink[FRatioDistribution, paclet:ref/FRatioDistribution]) constructs each random value from a single random beta variate. For small degrees of freedom, (RefLink[FRatioDistribution, paclet:ref/FRatioDistribution]) variates are instead generated from pairs of gamma variates to avoid possible divisions by 0 that may arise in the beta construction.

(RefLink[NoncentralChiSquareDistribution, paclet:ref/NoncentralChiSquareDistribution])[ν,λ], (Subscript[χ, ν]^2)(λ), variate generation uses additive properties of (χ^2) distributions to avoid expensive inverse CDF computations for nonintegral ν. The additive properties are given in, for instance, Johnson, Kotz, and Balakrishnan [7]. For ν==1 a noncentral (χ^2) variate can be generated as the square of a normal variate with mean (Sqrt[λ]) and variance 1. For ν≠1 noncentral (χ^2) variates are obtained as the sum of a central and a noncentral (χ^2) random variable. For ν>1, X+Y is distributed (Subscript[χ, ν]^2)(λ) if X~(Subscript[χ, 1]^2)(λ) and Y~(Subscript[χ, ν-1]^2). This relationship cannot be used for ν<1. In that case the construction is X+Y with X~(Subscript[χ, 0]^2)(λ) and Y~(Subscript[χ, ν-1]^2), where (Subscript[χ, 0]^2)(λ) is the limiting noncentral (χ^2) distribution as ν goes to 0. The limiting distribution (Subscript[χ, 0]^2)(λ) is a mixture of Poisson and (χ^2) variables, which has a nonzero probability mass at 0 and a continuous density for positive values. (RefLink[NoncentralFRatioDistribution, paclet:ref/NoncentralFRatioDistribution]) variates are obtained from one central and one noncentral (χ^2) variate.

For the (PackageLink[WishartDistribution, paclet:MultivariateStatistics/ref/WishartDistribution, MultivariateStatistics Package Symbol]) from the Multivariate Statistics Package, matrices are generated via Smith and Hocking's method [8]. This method constructs Wishart matrices from matrices with chi-distributed diagonal entries and normally distributed off-diagonal entries.

(RefLink[NoncentralStudentTDistribution, paclet:ref/NoncentralStudentTDistribution]), (PackageLink[HotellingTSquareDistribution, paclet:MultivariateStatistics/ref/HotellingTSquareDistribution, MultivariateStatistics Package Symbol]), and (PackageLink[MultivariateTDistribution, paclet:MultivariateStatistics/ref/MultivariateTDistribution, MultivariateStatistics Package Symbol]) each use direct construction from univariate random variates.

(RefLink[GammaDistribution, paclet:ref/GammaDistribution]), (RefLink[BetaDistribution, paclet:ref/BetaDistribution]), and (RefLink[StudentTDistribution, paclet:ref/StudentTDistribution]) use acceptance-rejection methods to some extent.

For (RefLink[GammaDistribution, paclet:ref/GammaDistribution])[α,β] exponential variates are generated when α==1. Otherwise, methods due to Cheng and Feast [9] and Ahrens and Dieter [10] are used.

Beta variates are constructed by switching between multiple methods, depending on the values of the beta parameters α and β. If both parameters are 1, uniform random variates will be generated. If one of the beta parameters is 1, then a closed-form inverse CDF evaluation is used. Otherwise, (RefLink[RandomVariate, paclet:ref/RandomVariate]) switches between acceptance-rejection methods due to Jöhnk [11], Cheng [12], and Atkinson [13]. An example of the advantage of using an acceptance-rejection method over construction from two gammas can be seen in the following. The direct acceptance-rejection method is nearly twice as fast as the gamma-pair construction.

This shows a comparison of direct construction and acceptance-rejection methods for beta variates:

Timing[With[{x1=RandomVariate[GammaDistribution[7,1],10^6]},x1/(x1+RandomVariate[GammaDistribution[3,1],10^6])];]

Timing[RandomVariate[BetaDistribution[7,3],10^6];]

For (RefLink[StudentTDistribution, paclet:ref/StudentTDistribution]) the method used by (RefLink[RandomVariate, paclet:ref/RandomVariate]) is a polar rejection method due to Bailey [14]. This method is more efficient than direct construction from normal and (χ^2) variates, as can be seen in the following. The direct construction takes roughly 1.5 times as long as the polar method for a million Student t variates.

This shows a comparison of direct construction and Bailey's polar rejection method for Student t:

Timing[RandomVariate[NormalDistribution[0,1],10^6]/Sqrt[RandomVariate[ChiSquareDistribution[6],10^6]/6];]

Timing[RandomVariate[StudentTDistribution[6],10^6];]

Discrete Distributions

(RefLink[GeometricDistribution, paclet:ref/GeometricDistribution]), (RefLink[BetaBinomialDistribution, paclet:ref/BetaBinomialDistribution]), and (RefLink[BetaNegativeBinomialDistribution, paclet:ref/BetaNegativeBinomialDistribution]) use direct construction. (RefLink[GeometricDistribution, paclet:ref/GeometricDistribution]) variates are generated as ⌊((log(U))/(log(1-p)))⌋ where U follows (RefLink[UniformDistribution, paclet:ref/UniformDistribution])[0,1]. (RefLink[BetaBinomialDistribution, paclet:ref/BetaBinomialDistribution]) and (RefLink[BetaNegativeBinomialDistribution, paclet:ref/BetaNegativeBinomialDistribution]) are constructed from (RefLink[BinomialDistribution, paclet:ref/BinomialDistribution]) and (RefLink[NegativeBinomialDistribution, paclet:ref/NegativeBinomialDistribution]) variates with probability parameters taken as random (RefLink[BetaDistribution, paclet:ref/BetaDistribution]) variates.

When used, table lookups for random integer generation are implemented via (RefLink[RandomChoice, paclet:ref/RandomChoice]) using the distribution's probability mass function for the weights. Most discrete distributions switch to other methods whenever construction of the list of weights is expected to be expensive given the desired sampled size. For example, as p approaches 1 (RefLink[LogSeriesDistribution, paclet:ref/LogSeriesDistribution])[p] switches to the direct construction ⌊1+((log(V))/(log(1-(((1-p))^U))))⌋, where U and V are uniformly distributed on the interval [0,1] [15]. Depending on parameters and sample size, (RefLink[NegativeBinomialDistribution, paclet:ref/NegativeBinomialDistribution])[n,p] may switch to construction as a Poisson-gamma mixture, which is a Poisson variable with mean following a gamma distribution [6].

(RefLink[BinomialDistribution, paclet:ref/BinomialDistribution]), (RefLink[HypergeometricDistribution, paclet:ref/HypergeometricDistribution]), and (RefLink[PoissonDistribution, paclet:ref/PoissonDistribution]) rely on direct sampling from the density function if the computational overhead of computing the PDF values is small relative to the number of desired random values. Otherwise they switch to acceptance-rejection methods. The acceptance-rejection methods also allow for generation of variates when overflows or underflows would occur in directly computing the PDF values, thus extending the range of parameter values for which random numbers can be generated.

The binomial and hypergeometric distributions switch to acceptance-rejection methods due to Kachitvichyanukul and Schmeiser, with small modifications. The binomial method, based on the acceptance-rejection portion of their BTPE (Binomial, Triangle, Parallelogram, Exponential) algorithm [16], effectively uses a piecewise majorizing function with three regions and a triangular minorizing function for a quick acceptance test. The majorizing and minorizing functions create a two-parallelogram envelope around the center of the rescaled binomial density, and the tails of the majorizing function form exponential envelopes on the tails of the scaled binomial distribution. One case where it is clearly better to use BTPE rather than to construct a lookup table is when few observations are desired and the lookup table would be large.

The hypergeometric method, based on the acceptance-rejection portion of Kachitvichyanukul and Schmeiser's H2PE algorithm [17], uses a majorizing function with three regions around a scaled hypergeometric density. The middle portion of the density is enveloped by a rectangular region and the tails of the distribution are bounded by exponentials.

The acceptance-rejection method used by (RefLink[PoissonDistribution, paclet:ref/PoissonDistribution]) is due to Ahrens and Dieter [18]. The acceptance and rejection is carried out using discrete normal variates, taking advantage of the tendency of (RefLink[PoissonDistribution, paclet:ref/PoissonDistribution])[μ] toward (RefLink[NormalDistribution, paclet:ref/NormalDistribution])[μ,(Sqrt[μ])]as μ increases.

Random values from the (RefLink[ZipfDistribution, paclet:ref/ZipfDistribution]) are generated via an acceptance-rejection method described by Devroye [15]. The method uses pairs of uniform variates and a test involving only a (RefLink[Floor, paclet:ref/Floor]) and noninteger powers, aside from basic arithmetic, to efficiently obtain Zipf-distributed values.

Defining Distributional Generators

A number of distribution constructors are included in the Wolfram Language which make it possible to define new distribution objects which can be treated like any other distribution. This includes random number generation. Suppose, however, that you are only interested in generating random values from a distribution and have an algorithm for doing so. In such cases it can be beneficial to just define the random number generator. Definitions for such distributional generators are supported through rules for Random`DistributionVector.

Function for defining random generation from distributions.

DistributionVector is expected to return a vector of the given length with numbers of the given precision. Because the expression expr is not a completely defined distribution object, the numbers will be generated via (RefLink[RandomReal, paclet:ref/RandomReal]) or (RefLink[RandomInteger, paclet:ref/RandomInteger]) instead of (RefLink[RandomVariate, paclet:ref/RandomVariate]). If the precision is (RefLink[Infinity, paclet:ref/Infinity]), the values will be generated via (RefLink[RandomInteger, paclet:ref/RandomInteger]). Otherwise, values will be generated via (RefLink[RandomReal, paclet:ref/RandomReal]).

Rules for generating random values from distributions are generally defined via a (RefLink[TagSet, paclet:ref/TagSet]) on the head of the distribution. The distribution itself may contain parameters. As a simple example, the following defines rules for NegativeOfUniform[a,b], which represents a uniform distribution on the interval [-b,-a].

NegativeOfUniform/:
Random`DistributionVector[NegativeOfUniform[a_,b_],n_Integer,prec_?Positive]:=-RandomReal[{a,b},n,WorkingPrecision->prec]/;
VectorQ[{a,b},NumericQ]&&Element[{a,b},Reals]

Random numbers from NegativeOfUniform can now be generated via (RefLink[RandomReal, paclet:ref/RandomReal]).

The following gives a machine-precision number and a precision-20 number from NegativeOfUniform:

{RandomReal[NegativeOfUniform[1,3]],RandomReal[NegativeOfUniform[1,3],WorkingPrecision->20]}

Matrices and higher-dimensional tensors can also be generated directly via (RefLink[RandomReal, paclet:ref/RandomReal]). (RefLink[RandomReal, paclet:ref/RandomReal]) uses the definition given to Random`DistributionVector to generate the total number of random values desired, and partitions that total number into the specified dimensions.

Here is a 3×4 array of NegativeOfUniform numbers:

RandomReal[NegativeOfUniform[7,21],{3,4}]

Discrete distributional generators can be defined in a similar way. The main difference is that the precision argument to Random`DistributionVector will now be (RefLink[Infinity, paclet:ref/Infinity]). The discrete version of NegativeOfUniform provides a simple example.

NegativeOfDiscreteUniform/:
Random`DistributionVector[NegativeOfDiscreteUniform[a_Integer,b_Integer],n_Integer,Infinity]:=-RandomInteger[{a,b},n]

Random values from NegativeOfDiscreteUniform can now be obtained from (RefLink[RandomInteger, paclet:ref/RandomInteger]).

Here are 10 NegativeOfDiscreteUniform numbers:

RandomInteger[NegativeOfDiscreteUniform[1,3],10]

While the previous examples show the basic framework for defining distributional generators, the distributions themselves are not particularly interesting. In fact, it would have been easier in these two cases to just generate values from (RefLink[RandomVariate, paclet:ref/RandomVariate]) and multiply the end result by -1 instead of attaching definitions to a new symbol. The following examples will demonstrate slightly more complicated distributions, in which case attaching definitions to a symbol will be more useful.

Example: Normal Distribution by Inversion

The textbook definition for generating random values from a generic univariate statistical distribution involves two steps:

To demonstrate the process, use the normal distribution. To generate random normal variates using this method, start with the (RefLink[Quantile, paclet:ref/Quantile]) for the normal distribution at a point q and replace q with a random uniform between 0 and 1.

Here is the (RefLink[Quantile, paclet:ref/Quantile]) function for a normal with mean μ and standard deviation σ:

Quantile[NormalDistribution[μ,σ],q]

A new distribution object can now be used to define a normal random number generator that uses inversion of the cumulative distribution function.

This defines generation of normals by inversion:

NormalByInversion/:
Random`DistributionVector[
NormalByInversion[mu_?(NumericQ[#]&&Im[#]===0&),sigma_?Positive],n_Integer,prec_?Positive]:=mu+(Sqrt[2]) sigma InverseErf[-1+2 RandomReal[1,n,WorkingPrecision->prec]]

Here are 10 random normals generated by inversion:

RandomReal[NormalByInversion[1,2],10]

Here is a sample of (10^4) random normals along with the sample mean and standard deviation:

(ninv=RandomReal[NormalByInversion[1,2],10^4]);//Timing

{Mean[ninv],StandardDeviation[ninv]}

The normal distribution is one example where methods other than direct inversion are generally preferred. While inversion of the CDF is a perfectly valid method for generating pseudorandom normal numbers, it is not particularly efficient. Numeric evaluation of (RefLink[InverseErf, paclet:ref/InverseErf]) is computationally much more expensive than the sinusoid and logarithmic evaluations required by the Box–Müller method used by (RefLink[NormalDistribution, paclet:ref/NormalDistribution]).

The built-in method takes almost no time for the same number of values:

(ndist=RandomVariate[NormalDistribution[1,2],10^4]);//Timing

{Mean[ndist],StandardDeviation[ndist]}

Clear[ninv,ndist]

Example: Uniform Distribution on a Disk

Random`DistributionVector can also be used to define generators for multidimensional distributions. For instance, suppose a random point from a uniform distribution on the unit disk, the set of real points {x,y} with (Sqrt[(x^2)+(y^2)])≤1, is desired. Such a random point can be constructed as follows:

The returned ordered pair can be multiplied by r to generate points uniformly distributed on a disk of radius r.

The following defines a generator for a uniform disk of radius r:

UniformDisk/:
Random`DistributionVector[UniformDisk[r_?Positive],n_Integer,prec_?Positive]:=
r*Sqrt[RandomReal[1,n,WorkingPrecision->prec]]*
Transpose[{Cos[#],Sin[#]}&[RandomReal[2Pi,n,WorkingPrecision->prec]]]

Here is one random ordered pair from a disk of radius 2:

RandomReal[UniformDisk[2]]

The following visualizes the distribution of (10^4) generated points on this disk:

ListPlot[RandomReal[UniformDisk[2],1000],AspectRatio->1]

Example: Gibbs Sampler

Gibbs samplers can also be defined as distributional generators. As an example consider a Gibbs sampler that mixes beta and binomial distributions. A specific case of this sampler was explored in a previous example. Here, the distribution will be defined with two parameters m and α.

This defines a Gibbs sampler BinomialBetaSampler:

BinomialBetaSampler/:Random`DistributionVector[
BinomialBetaSampler[m_Integer,α_?Positive],n_Integer,prec_?Positive]:=Block[{y0,dist1,dist2,x0},
y0=.5;
dist1[y_]:=RandomVariate[BinomialDistribution[m,y]];dist2[x_]:=RandomVariate[BetaDistribution[x+α,m-x+4],WorkingPrecision->prec];
Do[{x0=dist1[y0],y0=dist2[x0]},{1000}];
Table[{x0=dist1[y0],y0=dist2[x0]},{n}]]

For the specific Gibbs sampler constructed earlier, m was 16 and α was 2.

Here are 5 vectors from the sampler with m==16 and α==2:

RandomReal[BinomialBetaSampler[16,2],5]

References

[] Geman, S. and D. Geman. "Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images." IEEE Transactions on Pattern Analysis and Machine Intelligence 6, no. 6 (1984): 721–741.

[] Casella, G. and E. I. George. "Explaining the Gibbs Sampler." The American Statistician 46, no. 3 (1992): 167–174.

[] Matsumoto, M. and T. Nishimura. "Mersenne Twister: A 623-Dimensionally Equidistributed Uniform Pseudorandom Number Generator." ACM Transactions on Modeling and Computer Simulation 8, no. 1 (1998): 3–30.

[] Nishimura, T. "Tables of 64-Bit Mersenne Twisters." ACM Transactions on Modeling and Computer Simulation 10, no. 4 (2000): 348–357.

[] Junod, P. "Cryptographic Secure Pseudo-Random Bits Generation: The Blum–Blum–Shub Generator." August 1999. http://crypto.junod.info/bbs.pdf

[] Gentle, J. E. Random Number Generation and Monte Carlo Methods, 2nd ed. Springer-Verlag, 2003.

[] Johnson, N. L., S. Kotz, and N. Balakrishnan. Continuous Univariate Distributions, Volume 2, 2nd ed. John Wiley & Sons, 1995.

[] Smith, W. B. and R. R. Hocking. "Algorithm AS 53: Wishart Variate Generator." Applied Statistics 21, no. 3 (1972): 341–345.

[] Cheng, R. C. H. and G. M. Feast. "Some Simple Gamma Variate Generators." Applied Statistics 28, no. 3 (1979): 290–295.

[] Johnson, M. E. Multivariate Statistical Simulation. John Wiley & Sons, 1987.

[] Jöhnk, M. D. "Erzeugung von Betaverteilten und Gammaverteilten Zufallszahlen." Metrika 8 (1964): 5–15.

[] Cheng, R. C. H. "Generating Beta Variables with Nonintegral Shape Parameters." Communications of the ACM 21, no. 4 (1978): 317–322.

[] Atkinson, A. C. "A Family of Switching Algorithms for the Computer Generation of Beta Random Variables." Biometrika 66, no. 1 (1979): 141–145.

[] Bailey, R. W. "Polar Generation of Random Variates with the t-Distribution." Mathematics of Computation 62, no. 206 (1994): 779–781.

[] Devroye, L. Non-Uniform Random Variate Generation. Springer-Verlag, 1986.

[] Kachitvichyanukul, V. and B. W. Schmeiser. "Binomial Random Variate Generation." Communications of the ACM 31, no. 2 (1988): 216–223.

[] Kachitvichyanukul, V. and B. W. Schmeiser. "Computer Generation of Hypergeometric Random Variates." Journal of Statistical Computation and Simulation 22, no. 2 (1985): 127–145.

[] Ahrens, J. H. and U. Dieter. "Computer Generation of Poisson Deviates from Modified Normal Distributions." ACM Transactions on Mathematical Software 8, no. 2 (1982): 163–179.

[] Matsumoto, M. and T. Nishimura. "Dynamic Creation of Pseudorandom Number Generators." In Proceedings of the Third International Conference on Monte Carlo and Quasi-Monte Carlo Methods in Scientific Computing: Monte Carlo and Quasi-Monte Carlo Methods 1998, 56–69, 2000.