({{, (RefLink[LogLikelihood, paclet:ref/LogLikelihood])[dist,{(Subscript[x, 1]),(Subscript[x, 2]),…}] 
gives the log-likelihood function for observations (Subscript[x, 1]), (Subscript[x, 2]), … from the distribution dist.}, {, (RefLink[LogLikelihood, paclet:ref/LogLikelihood])[proc,{{(Subscript[t, 1]),(Subscript[x, 1])},{(Subscript[t, 2]),(Subscript[x, 2])},…]
gives the log-likelihood function for the observations (Subscript[x, i]) at time (Subscript[t, i]) from the process proc.}, {, (RefLink[LogLikelihood, paclet:ref/LogLikelihood])[proc,{(Subscript[path, 1]),(Subscript[path, 2]),…}]
gives the log-likelihood function for the observations from (Subscript[path, 1]), (Subscript[path, 2]), … from the process proc.}})

OpenersNotesSection(Spacer1)Details

The log-likelihood function (RefLink[LogLikelihood, paclet:ref/LogLikelihood])[dist,{(Subscript[x, 1]),(Subscript[x, 2]),…}] is given by (∑_i)log(f((Subscript[x, i]))), where f((Subscript[x, i])) is the probability density function at (Subscript[x, i]), (RefLink[PDF, paclet:ref/PDF])[dist,(Subscript[x, i])].

For a scalar-valued process proc, the log-likelihood function (RefLink[LogLikelihood, paclet:ref/LogLikelihood])[proc,{{(Subscript[t, 1]),(Subscript[x, 1])},{(Subscript[t, 2]),(Subscript[x, 2])},…}] is given by (RefLink[LogLikelihood, paclet:ref/LogLikelihood])[(RefLink[SliceDistribution, paclet:ref/SliceDistribution])[proc,{(Subscript[t, 1]),(Subscript[t, 2]),…}],{{(Subscript[x, 1]),(Subscript[x, 2]),…}}].

For a vector-valued process proc, the log-likelihood function (RefLink[LogLikelihood, paclet:ref/LogLikelihood])[proc,{{(Subscript[t, 1]),{(Subscript[x, 1]),…,(Subscript[z, 1])},{(Subscript[t, 2]),{(Subscript[x, 2]),…,(Subscript[z, 2])}},…}] is given by (RefLink[LogLikelihood, paclet:ref/LogLikelihood])[(RefLink[SliceDistribution, paclet:ref/SliceDistribution])[proc,{(Subscript[t, 1]),(Subscript[t, 2]),…}],{{(Subscript[x, 1]),…,(Subscript[z, 1]),(Subscript[x, 2]),…,(Subscript[z, 2]),…}}].

The log-likelihood function for a collection of paths (RefLink[LogLikelihood, paclet:ref/LogLikelihood])[proc,{(Subscript[path, 1]),(Subscript[path, 2]),…}] is given by (∑_i)LogLikelihood[proc,(Subscript[path, i])].

OpenersPrimaryExamplesSection(Spacer1)Examples  (28)

OpenersExampleSection0(Spacer1)Basic Examples  (4)

Get the log-likelihood function for a normal distribution:

LogLikelihood[NormalDistribution[μ,σ],{(Subscript[x, 1]),(Subscript[x, 2]),(Subscript[x, 3]),(Subscript[x, 4]),(Subscript[x, 5])}]

Compute a log-likelihood for numeric data:

data=RandomVariate[GammaDistribution[2,3],100];

LogLikelihood[GammaDistribution[2,3],data]

Plot log-likelihood contours as a function of α and β:

ContourPlot[LogLikelihood[GammaDistribution[α,β],data],{α,1,3},{β,1,5}]

Compute the log-likelihood for multivariate data:

data=RandomVariate[BinormalDistribution[.8],100];

LogLikelihood[BinormalDistribution[ρ],data]

Compute the log-likelihood for a process:

proc=WienerProcess[];
data=RandomFunction[proc,{1,2,.1}];

LogLikelihood[proc,data]

OpenersExampleSection1(Spacer1)Scope  (12)

OpenersExampleSubsection0(Spacer1)Univariate Parametric Distributions  (2)

Compute the log-likelihood for a continuous distribution:

data=RandomVariate[GumbelDistribution[3,6],100];

LogLikelihood[GumbelDistribution[3,6],data]

Compute the log-likelihood for a discrete distribution:

LogLikelihood[BinomialDistribution[20,1/4],Range[5,10]]

Plot the log-likelihood, assuming p is unknown:

Plot[LogLikelihood[BinomialDistribution[20,p],Range[5,10]],{p,0,1}]

OpenersExampleSubsection1(Spacer1)Multivariate Parametric Distributions  (2)

Obtain the log-likelihood for a continuous multivariate distribution with unknown parameters:

data=RandomVariate[DirichletDistribution[{2,5,10}],100];

ll=LogLikelihood[DirichletDistribution[{(Subscript[γ, 1]),(Subscript[γ, 2]),(Subscript[γ, 3])}],data]//PowerExpand

Visualize the log-likelihood surface, assuming (Subscript[γ, 3])=10:

Plot3D[ll/.(Subscript[γ, 3])->10,{(Subscript[γ, 1]),1,3},{(Subscript[γ, 2]),3,7}]

For a multivariate discrete distribution with known parameters:

𝒟=MultinomialDistribution[100,{.3,.2,.5}];
data=RandomVariate[𝒟,100];

LogLikelihood[𝒟,data]

OpenersExampleSubsection2(Spacer1)Derived Distributions  (5)

Compute the log-likelihood for a truncated standard normal:

𝒟=TruncatedDistribution[{-1,3/2},NormalDistribution[]];
data=RandomVariate[𝒟,100];

LogLikelihood[𝒟,data]

Plot the log-likelihood contours as a function of the truncation points:

ContourPlot[LogLikelihood[TruncatedDistribution[{min,max},NormalDistribution[]],data],{min,-2,Min[data]},{max,Max[data],3}]

Compute the log-likelihood for a constructed distribution:

dist=ProbabilityDistribution[λ Exp[λ(1-x)],{x,1,Infinity},Assumptions->λ>0]

LogLikelihood[dist,{1.4,5.1,1.7,1.6,1.1,3.9,2.2,1.3,2.,1.5}]//PowerExpand

Compute the log-likelihood for a product distribution:

𝒟=ProductDistribution[BetaDistribution[2,3],LaplaceDistribution[10,1]];

data=RandomVariate[𝒟,100];

LogLikelihood[𝒟,data]

Obtain the result as a sum of the independent component-wise log-likelihoods:

LogLikelihood[BetaDistribution[2,3],data[[All,1]]]+LogLikelihood[LaplaceDistribution[10,1],data[[All,2]]]

Compute the log-likelihood for a copula distribution:

𝒟=CopulaDistribution[{"Frank",α},{GammaDistribution[2,3],ChiSquareDistribution[10]}];

data=RandomVariate[𝒟/.α->1/2,100];

Plot the log-likelihood as a function of the kernel parameter:

Plot[LogLikelihood[𝒟,data],{α,1/10,9/10}]

Compute the log-likelihood for a component mixture:

𝒟=MixtureDistribution[{1/3,2/3},{GammaDistribution[2,3],NormalDistribution[1,1/4]}];

data=RandomVariate[𝒟,100];

LogLikelihood[𝒟,data]

OpenersExampleSubsection3(Spacer1)Random Processes  (3)

Compute the log-likelihood of a continuous parametric process:

proc=GeometricBrownianMotionProcess[1,2,1];
data=RandomFunction[proc,{1,2,.1}];

LogLikelihood[proc,data]

Compute the log-likelihood of a scalar-valued discrete parametric process:

proc=BinomialProcess[.3];
data=RandomFunction[proc,{10}];

LogLikelihood[proc,data]

Plot the log-likelihood as a function of the process parameter:

Plot[LogLikelihood[BinomialProcess[p],data],{p,0,1}]

Compute the log-likelihood of a scalar-valued time series process:

proc=ARMAProcess[{.3,.2},{-.8},1];
data=RandomFunction[proc,{100}];

LogLikelihood[proc,data]

Compute the log-likelihood of a vector-valued time series process:

phi={{{.2,.1},{.3,.1}}};
sigma={{1,.3},{.3,.5}};
proc=ARProcess[phi,sigma];
data=RandomFunction[proc,{100}];

LogLikelihood[proc,data]

OpenersExampleSection2(Spacer1)Applications  (4)

Visualize the log-likelihood surface for a distribution of two parameters:

data=RandomVariate[WeibullDistribution[4,5],100];

Plot3D[LogLikelihood[WeibullDistribution[α,β],data],{α,3,5},{β,4,6}]

Visualize as contours of equal log-likelihood:

ContourPlot[LogLikelihood[WeibullDistribution[α,β],data],{α,3,5},{β,4,6}]

Show log-likelihood functions with mixed continuous and discrete parameters:

𝒟=BinomialDistribution[n,p];
data=RandomVariate[𝒟/.{n->10,p->0.1},100];

Plot[Table[LogLikelihood[𝒟,data],{n,r={5,10,15}}]//Evaluate,{p,0,1},PlotLegends->(Row[{"n = ", #}]&/@r)]

DiscretePlot[Table[LogLikelihood[𝒟,data],{p,r={0.05,0.1,0.15}}]//Evaluate,{n,1,15},ExtentSize->1/2,AxesOrigin->{0,0},PlotLegends->(Row[{"p = ", #}]&/@r)]

Solve for the Poisson maximum log-likelihood estimate in closed form:

loglike=LogLikelihood[PoissonDistribution[μ],{x,y,z}]

Solve[Refine[D[loglike,μ]==0,x>0&&y>0&&z>0],μ]

Compute a maximum log-likelihood estimate directly:

data=RandomVariate[GeometricDistribution[1/2],500];

loglfun[p_]=LogLikelihood[GeometricDistribution[p],data];

Maximize:

{maxVal,maxPar}=FindMaximum[{loglfun[p],0<p<1},p]

Label the optimal point on a plot of the log-likelihood function:

Plot[loglfun[p],{p,.01,.99},Epilog->{Directive[Red,PointSize[.03]],Point[{p/.maxPar,maxVal}]}]

Estimate the variance of the MLE estimator as the reciprocal of the expectation of second derivative of the log-likelihood function with respect to parameters:

((Expectation[-D[Refine[LogLikelihood[GeometricDistribution[p],{x}],x≥0],{p,2}]/.maxPar,x\[Distributed]data])^(-1))

OpenersExampleSection3(Spacer1)Properties & Relations  (5)

(RefLink[LogLikelihood, paclet:ref/LogLikelihood]) is the sum of logs of (RefLink[PDF, paclet:ref/PDF]) values for data:

𝒟=ChiSquareDistribution[10];
data=RandomVariate[𝒟,100];

LogLikelihood[𝒟,data]

Total[Log[PDF[𝒟,data]]]

(RefLink[LogLikelihood, paclet:ref/LogLikelihood]) is the log of (RefLink[Likelihood, paclet:ref/Likelihood]):

𝒟=BetaDistribution[2,6];
data=RandomVariate[𝒟,100];

llhd=Likelihood[𝒟,data]

Log[llhd]

LogLikelihood[𝒟,data]

(RefLink[EstimatedDistribution, paclet:ref/EstimatedDistribution]) estimates parameters by maximizing the log-likelihood:

𝒟=PoissonDistribution[μ];
data=RandomVariate[𝒟/.μ->10,100];

EstimatedDistribution[data,𝒟]

(RefLink[FindDistributionParameters, paclet:ref/FindDistributionParameters]) gives the parameter estimates as rules:

muhat=FindDistributionParameters[data,𝒟]

Visualize the log-likelihood function near the optimal value:

Show[Plot[LogLikelihood[𝒟,data],{μ,8,12}],Graphics[{PointSize[Medium],Red,Point[{μ,LogLikelihood[𝒟,data]}/.muhat]}]]

Log-likelihood of a process can be computed using its slice distribution:

proc=ARMAProcess[{.3,-.2},{.8},.5];
data=RandomFunction[proc,{100}];

LogLikelihood[proc,data]

Use the slice distribution:

dist=proc[data["Times"]];

LogLikelihood[dist,data["ValueList"]]

For a vector-valued process:

phi={{{.2,.1},{.3,.1}}};
sigma={{1,.3},{.3,.5}};
proc=ARProcess[phi,sigma];
data=RandomFunction[proc,{100},2];

LogLikelihood[proc,data]

Use the slice distribution:

dist=proc[data["Times"]];

Vectorize the path values for use in the (RefLink[LogLikelihood, paclet:ref/LogLikelihood]) of the time slice distribution:

LogLikelihood[dist,Flatten/@data["ValueList"]]

Given a sample from a distribution 𝒟, the difference between values of (RefLink[LogLikelihood, paclet:ref/LogLikelihood]) for 𝒟 and the MLE estimated distribution doubled is random and follows (RefLink[ChiSquareDistribution, paclet:ref/ChiSquareDistribution]) with degrees of freedom equal to the number of distribution parameters:

ℓ[data_,s𝒟_,𝒟_]:=2(LogLikelihood[EstimatedDistribution[data,s𝒟],data]-LogLikelihood[𝒟,data])

Draw samples from a 5-parameter binormal distribution in 1000 batches:

𝒟=BinormalDistribution[{1,1},{2,2},2/3];
batches=RandomVariate[𝒟,{1000,1000}];

Compute the difference of log-likelihoods for each batch:

gaps=ℓ[#,BinormalDistribution[{m1,m2},{s1,s2},rho],𝒟]&/@batches;

Check that the log-likelihood differences are consistent with (Subscript[χ, 5]^2)-distribution:

QuantilePlot[gaps,ChiSquareDistribution[5]]

DistributionFitTest[gaps,ChiSquareDistribution[5],"TestConclusion"]

OpenersExampleSection4(Spacer1)Possible Issues  (1)

Log-likelihood of a continuous parametric process may be undefined:

proc=GeometricBrownianMotionProcess[1,3,1/4];
data=RandomFunction[proc,{0,0.2,.1}];

LogLikelihood[proc,data]

This is due to degenerate slice distribution at time 0:

Plot[Table[PDF[proc[t],x],{t,r={(10^(-4)),(10^(-3)),(10^(-2)),(10^(-1))}}]//Evaluate,{x,0,0.6},PlotRange->All,PlotLegends->(Row[{"t = ", #}]&/@N@r)]

Start at positive time:

data=RandomFunction[proc,{0.1,1,.1}];
LogLikelihood[proc,data]

OpenersExampleSection5(Spacer1)Neat Examples  (2)

Visualize isosurfaces for an exponential power log-likelihood:

data={2.03,2.92,3.32,1.51,2.28,2.13,2.09,1.12,0.73,1.06};

ContourPlot3D[LogLikelihood[ExponentialPowerDistribution[k,μ,σ],data]//Evaluate,{k,0.5,2},{μ,2,3.5},{σ,1.5,3}]

Visualize isosurfaces for a bivariate normal log-likelihood:

data={{2.03,2.92},{3.32,1.51},{2.28,2.13},{2.09,1.12},{0.73,1.06}};

ContourPlot3D[Evaluate@LogLikelihood[BinormalDistribution[{μ,μ},{σ,σ},ρ],data],{μ,1.5,3.5},{σ,2.5,4.5},{ρ,-.75,.75}]

({{DocumentationFooterIconsRelatedFunction.png, ({{See Also}, {(RefLink[Likelihood, paclet:ref/Likelihood])  ▪  (RefLink[PDF, paclet:ref/PDF])  ▪  (RefLink[EstimatedDistribution, paclet:ref/EstimatedDistribution])}})}})

({{DocumentationFooterIconsRelatedGuide.png, ({{Related Guides}, {▪(RefLinkPlain[Statistical Distribution Functions, paclet:guide/StatisticalDistributionFunctions])}})}})