NAG Library Routine Document
G07BBF
1 Purpose
G07BBF computes maximum likelihood estimates and their standard errors for parameters of the Normal distribution from grouped and/or censored data.
2 Specification
| SUBROUTINE G07BBF ( | METHOD, N, X, XC, IC, XMU, XSIG, TOL, MAXIT, SEXMU, SEXSIG, CORR, DEV, NOBS, NIT, WK, IFAIL) |
| INTEGER | N, IC(N), MAXIT, NOBS(4), NIT, IFAIL |
| double precision | X(N), XC(N), XMU, XSIG, TOL, SEXMU, SEXSIG, CORR, DEV, WK(2*N) |
| CHARACTER*1 | METHOD |
3 Description
A sample of size
n is taken from a Normal distribution with mean
μ and variance
σ2 and consists of grouped and/or censored data. Each of the
n observations is known by a pair of values
Li,Ui such that:
The data is represented as particular cases of this form:
- exactly specified observations occur when Li=Ui=xi,
- right-censored observations, known only by a lower bound, occur when Ui→∞,
- left-censored observations, known only by a upper bound, occur when Li→-∞,
- and interval-censored observations when Li<xi<Ui.
Let the set
A identify the exactly specified observations, sets
B and
C identify the observations censored on the right and left respectively, and set
D identify the observations confined between two finite limits. Also let there be
r exactly specified observations, i.e., the number in
A. The probability density function for the standard Normal distribution is
and the cumulative distribution function is
The log-likelihood of the sample can be written as:
where
pi=Pui-Pli and
ui=Ui-μ/σ, li=Li-μ/σ.
Let
and
then the first derivatives of the log-likelihood can be written as:
and
The maximum likelihood estimates,
μ^ and
σ^, are the solution to the equations:
and
and if the second derivatives
∂2
L
∂2
μ
,
∂2
L
∂μ
∂σ
and
∂2L
∂2σ
are denoted by
L11,
L12 and
L22 respectively, then estimates of the standard errors of
μ^ and
σ^ are given by:
and an estimate of the correlation of
μ^ and
σ^ is given by:
To obtain the maximum likelihood estimates the equations
(1) and
(2) can be solved using either the Newton–Raphson method or the Expectation-maximization
EM algorithm of
Dempster et al. (1977).
Newton–Raphson Method
This consists of using approximate estimates
μ~ and
σ~ to obtain improved estimates
μ~+δμ~ and
σ~+δσ~ by solving
for the corrections
δμ~ and
δσ~.
EM Algorithm
The expectation step consists of constructing the variable
wi as follows:
the maximization step consists of substituting
(3),
(4),
(5) and
(6) into
(1) and
(2) giving:
and
where
and where
w^i,
l^i and
u^i are
wi,
li and
ui evaluated at
μ^ and
σ^. Equations
(3) to
(8) are the basis of the
EM iterative procedure for finding
μ^ and
σ^2. The procedure consists of alternately estimating
μ^ and
σ^2 using
(7) and
(8) and estimating
w^i using
(3) to
(6).
In choosing between the two methods a general rule is that the Newton–Raphson method converges more quickly but requires good initial estimates whereas the EM algorithm converges slowly but is robust to the initial values. In the case of the censored Normal distribution, if only a small proportion of the observations are censored then estimates based on the exact observations should give good enough initial estimates for the Newton–Raphson method to be used. If there are a high proportion of censored observations then the EM algorithm should be used and if high accuracy is required the subsequent use of the Newton–Raphson method to refine the estimates obtained from the EM algorithm should be considered.
4 References
Dempster A P, Laird N M and Rubin D B (1977) Maximum likelihood from incomplete data via the
EM algorithm (with discussion)
J. Roy. Statist. Soc. Ser. B 39 1–38
Swan A V (1969) Algorithm AS 16. Maximum likelihood estimation from grouped and censored normal data
Appl. Statist. 18 110–114
Wolynetz M S (1979) Maximum likelihood estimation from confined and censored normal data
Appl. Statist. 28 185–195
5 Parameters
- 1: METHOD – CHARACTER*1Input
On entry: indicates whether the Newton–Raphson or
EM algorithm should be used.
- If METHOD='N', then the Newton–Raphson algorithm is used.
- If METHOD='E', then the EM algorithm is used.
Constraint:
METHOD='N' or 'E'.
- 2: N – INTEGERInput
-
On entry:
n, the number of observations.
Constraint:
N≥2.
- 3: X(N) – double precision arrayInput
On entry: the observations
xi,
Li or
Ui, for
i=1,2,…,n.
If the observation is exactly specified – the exact value, xi.
If the observation is right-censored – the lower value, Li.
If the observation is left-censored – the upper value, Ui.
If the observation is interval-censored – the lower or upper value,
Li or
Ui, (see
XC).
- 4: XC(N) – double precision arrayInput
On entry: if the jth observation, for j=1,2,…,n is an interval-censored observation then XCj should contain the complementary value to Xj, that is, if Xj<XCj, then XCj contains upper value, Ui, and if Xj>XCj, then XCj contains lower value, Li. Otherwise if the jth observation is exact or right- or left-censored XCj need not be set.
Note: if Xj=XCj then the observation is ignored.
- 5: IC(N) – INTEGER arrayInput
On entry:
ICi contains the censoring codes for the
ith observation, for
i=1,2,…,n.
If ICi=0, the observation is exactly specified.
If ICi=1, the observation is right-censored.
If ICi=2, the observation is left-censored.
If ICi=3, the observation is interval-censored.
Constraint:
ICi=0, 1, 2 or 3, for i=1,2,…,n.
- 6: XMU – double precisionInput/Output
On entry: if
XSIG>0.0 the initial estimate of the mean,
μ; otherwise
XMU need not be set.
On exit: the maximum likelihood estimate, μ^, of μ.
- 7: XSIG – double precisionInput/Output
On entry: specifies whether an initial estimate of
μ and
σ are to be supplied.
- XSIG>0.0
- XSIG is the initial estimate of σ and XMU must contain an initial estimate of μ.
- XSIG≤0.0
- Onitial estimates of XMU and XSIG are calculated internally from:
| (a) |
the exact observations, if the number of exactly specified observations is ≥2; or |
| (b) |
the interval-censored observations; if the number of interval-censored observations is ≥1; or |
| (c) |
they are set to 0.0 and 1.0 respectively. |
On exit: the maximum likelihood estimate, σ^, of σ.
- 8: TOL – double precisionInput
On entry: the relative precision required for the final estimates of
μ and
σ. Convergence is assumed when the absolute relative changes in the estimates of both
μ and
σ are less than
TOL.
If TOL=0.0, then a relative precision of 0.000005 is used.
Constraint:
machine precision<TOL≤1.0 or TOL=0.0.
- 9: MAXIT – INTEGERInput
On entry: the maximum number of iterations.
If MAXIT≤0, then a value of 25 is used.
- 10: SEXMU – double precisionOutput
On exit: the estimate of the standard error of μ^.
- 11: SEXSIG – double precisionOutput
On exit: the estimate of the standard error of σ^.
- 12: CORR – double precisionOutput
On exit: the estimate of the correlation between μ^ and σ^.
- 13: DEV – double precisionOutput
On exit: the maximized log-likelihood, Lμ^,σ^.
- 14: NOBS(4) – INTEGER arrayOutput
On exit: the number of the different types of each observation;
NOBS1 contains number of right-censored observations.
NOBS2 contains number of left-censored observations.
NOBS3 contains number of interval-censored observations.
NOBS4 contains number of exactly specified observations.
- 15: NIT – INTEGEROutput
-
On exit: the number of iterations performed.
- 16: WK(2×N) – double precision arrayWorkspace
- 17: IFAIL – INTEGERInput/Output
-
On entry:
IFAIL must be set to
0,
-1 or 1. If you are unfamiliar with this parameter you should refer to
Section 3.3 in the Essential Introduction for details.
On exit:
IFAIL=0 unless the routine detects an error (see
Section 6).
For environments where it might be inappropriate to halt program execution when an error is detected, the value
-1 or 1 is recommended. If the output of error messages is undesirable, then the value
1 is recommended. Otherwise, if you are not familiar with this parameter, the recommended value is
0.
When the value -1 or 1 is used it is essential to test the value of IFAIL on exit.
6 Error Indicators and Warnings
If on entry
IFAIL=0 or
-1, explanatory error messages are output on the current error message unit (as defined by
X04AAF).
Errors or warnings detected by the routine:
- IFAIL=1
-
| On entry, | METHOD≠'N' or 'E', |
| or | N<2, |
| or | ICi≠0, 1, 2 or 3, for some i, |
| or | TOL<0.0, |
| or | 0.0<TOL<machine precision, |
| or | TOL>1.0. |
- IFAIL=2
-
The chosen method failed to converge in
MAXIT iterations. You should either increase
TOL or
MAXIT or, if using the
EM algorithm try using the Newton–Raphson method with initial values those returned by the current call to G07BBF. All returned values will be reasonable approximations to the correct results if
MAXIT is not very small.
- IFAIL=3
-
The chosen method is diverging. This will be due to poor initial values. You should try different initial values.
- IFAIL=4
-
G07BBF was unable to calculate the standard errors. This can be caused by the method starting to diverge when the maximum number of iterations was reached.
7 Accuracy
The accuracy is controlled by the parameter
TOL.
If high precision is requested with the
EM algorithm then there is a possibility that, due to the slow convergence, before the correct solution has been reached the increments of
μ^ and
σ^ may be smaller than
TOL and the process will prematurely assume convergence.
8 Further Comments
The process is deemed divergent if three successive increments of μ or σ increase.
9 Example
A sample of 18 observations and their censoring codes are read in and the Newton–Raphson method used to compute the estimates.
9.1 Program Text
Program Text (g07bbfe.f)
9.2 Program Data
Program Data (g07bbfe.d)
9.3 Program Results
Program Results (g07bbfe.r)