G02HDF (PDF version)
G02 Chapter Contents
G02 Chapter Introduction
NAG Library Manual

NAG Library Routine Document

G02HDF

Note:  before using this routine, please read the Users' Note for your implementation to check the interpretation of bold italicised terms and other implementation-dependent details.

+ Contents

    1  Purpose
    7  Accuracy

1  Purpose

G02HDF performs bounded influence regression (M-estimates) using an iterative weighted least squares algorithm.

2  Specification

SUBROUTINE G02HDF ( CHI, PSI, PSIP0, BETA, INDW, ISIGMA, N, M, X, LDX, Y, WGT, THETA, K, SIGMA, RS, TOL, EPS, MAXIT, NITMON, NIT, WK, IFAIL)
INTEGER  INDW, ISIGMA, N, M, LDX, K, MAXIT, NITMON, NIT, IFAIL
REAL (KIND=nag_wp)  CHI, PSI, PSIP0, BETA, X(LDX,M), Y(N), WGT(N), THETA(M), SIGMA, RS(N), TOL, EPS, WK((M+4)*N)
EXTERNAL  CHI, PSI

3  Description

For the linear regression model
y=Xθ+ε,
where y is a vector of length n of the dependent variable,
X is a n by m matrix of independent variables of column rank k,
θ is a vector of length m of unknown parameters,
and ε is a vector of length n of unknown errors with var εi=σ2,
G02HDF calculates the M-estimates given by the solution, θ^, to the equation
i=1nψri/σwiwixij=0,  j=1,2,,m, (1)
where ri is the ith residual, i.e., the ith element of the vector r=y-Xθ^,
ψ is a suitable weight function,
wi are suitable weights such as those that can be calculated by using output from G02HBF,
and σ may be estimated at each iteration by the median absolute deviation of the residuals σ^=mediri/β1
or as the solution to
i=1nχri/σ^wiwi2=n-kβ2
for a suitable weight function χ, where β1 and β2 are constants, chosen so that the estimator of σ is asymptotically unbiased if the errors, εi, have a Normal distribution. Alternatively σ may be held at a constant value.
The above describes the Schweppe type regression. If the wi are assumed to equal 1 for all i, then Huber type regression is obtained. A third type, due to Mallows, replaces (1) by
i=1nψri/σwixij=0,  j=1,2,,m.
This may be obtained by use of the transformations
wi* wi yi* yiwi xij* xijwi,   j= 1,2,,m
(see Marazzi (1987)).
The calculation of the estimates of θ can be formulated as an iterative weighted least squares problem with a diagonal weight matrix G given by
Gii= ψri/σwi ri/σwi , ri0 ψ0, ri=0. .
The value of θ at each iteration is given by the weighted least squares regression of y on X. This is carried out by first transforming the y and X by
y~i =yiGii x~ij =xijGii,  j=1,2,,m
and then using F04JGF . If X is of full column rank then an orthogonal-triangular (QR) decomposition is used; if not, a singular value decomposition is used.
Observations with zero or negative weights are not included in the solution.
Note:  there is no explicit provision in the routine for a constant term in the regression model. However, the addition of a dummy variable whose value is 1.0 for all observations will produce a value of θ^ corresponding to the usual constant term.
G02HDF is based on routines in ROBETH, see Marazzi (1987).

4  References

Hampel F R, Ronchetti E M, Rousseeuw P J and Stahel W A (1986) Robust Statistics. The Approach Based on Influence Functions Wiley
Huber P J (1981) Robust Statistics Wiley
Marazzi A (1987) Subroutines for robust and bounded influence regression in ROBETH Cah. Rech. Doc. IUMSP, No. 3 ROB 2 Institut Universitaire de Médecine Sociale et Préventive, Lausanne

5  Parameters

1:     CHI – REAL (KIND=nag_wp) FUNCTION, supplied by the user.External Procedure
If ISIGMA>0, CHI must return the value of the weight function χ for a given value of its argument. The value of χ must be non-negative.
The specification of CHI is:
FUNCTION CHI ( T)
REAL (KIND=nag_wp) CHI
REAL (KIND=nag_wp)  T
1:     T – REAL (KIND=nag_wp)Input
On entry: the argument for which CHI must be evaluated.
CHI must either be a module subprogram USEd by, or declared as EXTERNAL in, the (sub)program from which G02HDF is called. Parameters denoted as Input must not be changed by this procedure.
If ISIGMA0, the actual parameter CHI may be the dummy routine G02HDZ. (G02HDZ is included in the NAG Library.)
2:     PSI – REAL (KIND=nag_wp) FUNCTION, supplied by the user.External Procedure
PSI must return the value of the weight function ψ for a given value of its argument.
The specification of PSI is:
FUNCTION PSI ( T)
REAL (KIND=nag_wp) PSI
REAL (KIND=nag_wp)  T
1:     T – REAL (KIND=nag_wp)Input
On entry: the argument for which PSI must be evaluated.
PSI must either be a module subprogram USEd by, or declared as EXTERNAL in, the (sub)program from which G02HDF is called. Parameters denoted as Input must not be changed by this procedure.
3:     PSIP0 – REAL (KIND=nag_wp)Input
On entry: the value of ψ0.
4:     BETA – REAL (KIND=nag_wp)Input
On entry: if ISIGMA<0, BETA must specify the value of β1.
For Huber and Schweppe type regressions, β1 is the 75th percentile of the standard Normal distribution (see G01FAF). For Mallows type regression β1 is the solution to
1ni=1nΦβ1/wi=0.75,
where Φ is the standard Normal cumulative distribution function (see S15ABF).
If ISIGMA>0, BETA must specify the value of β2.
β2= -χzϕzdz, in the Huber case; β2= 1ni=1nwi-χzϕzdz, in the Mallows case; β2= 1ni=1nwi2-χz/wiϕzdz, in the Schweppe case;
where ϕ is the standard normal density, i.e., 12πexp-12x2 .
If ISIGMA=0, BETA is not referenced.
Constraint: if ISIGMA0, BETA>0.0.
5:     INDW – INTEGERInput
On entry: determines the type of regression to be performed.
INDW=0
Huber type regression.
INDW<0
Mallows type regression.
INDW>0
Schweppe type regression.
6:     ISIGMA – INTEGERInput
On entry: determines how σ is to be estimated.
ISIGMA=0
σ is held constant at its initial value.
ISIGMA<0
σ is estimated by median absolute deviation of residuals.
ISIGMA>0
σ is estimated using the χ function.
7:     N – INTEGERInput
On entry: n, the number of observations.
Constraint: N>1.
8:     M – INTEGERInput
On entry: m, the number of independent variables.
Constraint: 1M<N.
9:     X(LDX,M) – REAL (KIND=nag_wp) arrayInput/Output
On entry: the values of the X matrix, i.e., the independent variables. Xij must contain the ijth element of X, for i=1,2,,n and j=1,2,,m.
If INDW<0, during calculations the elements of X will be transformed as described in Section 3. Before exit the inverse transformation will be applied. As a result there may be slight differences between the input X and the output X.
On exit: unchanged, except as described above.
10:   LDX – INTEGERInput
On entry: the first dimension of the array X as declared in the (sub)program from which G02HDF is called.
Constraint: LDXN.
11:   Y(N) – REAL (KIND=nag_wp) arrayInput/Output
On entry: the data values of the dependent variable.
Yi must contain the value of y for the ith observation, for i=1,2,,n.
If INDW<0, during calculations the elements of Y will be transformed as described in Section 3. Before exit the inverse transformation will be applied. As a result there may be slight differences between the input Y and the output Y.
On exit: unchanged, except as described above.
12:   WGT(N) – REAL (KIND=nag_wp) arrayInput/Output
On entry: the weight for the ith observation, for i=1,2,,n.
If INDW<0, during calculations elements of WGT will be transformed as described in Section 3. Before exit the inverse transformation will be applied. As a result there may be slight differences between the input WGT and the output WGT.
If WGTi0, the ith observation is not included in the analysis.
If INDW=0, WGT is not referenced.
On exit: unchanged, except as described above.
13:   THETA(M) – REAL (KIND=nag_wp) arrayInput/Output
On entry: starting values of the parameter vector θ. These may be obtained from least squares regression. Alternatively if ISIGMA<0 and SIGMA=1 or if ISIGMA>0 and SIGMA approximately equals the standard deviation of the dependent variable, y, then THETAi=0.0, for i=1,2,,m may provide reasonable starting values.
On exit: the M-estimate of θi, for i=1,2,,m.
14:   K – INTEGEROutput
On exit: the column rank of the matrix X.
15:   SIGMA – REAL (KIND=nag_wp)Input/Output
On entry: a starting value for the estimation of σ. SIGMA should be approximately the standard deviation of the residuals from the model evaluated at the value of θ given by THETA on entry.
Constraint: SIGMA>0.0.
On exit: the final estimate of σ if ISIGMA0 or the value assigned on entry if ISIGMA=0.
16:   RS(N) – REAL (KIND=nag_wp) arrayOutput
On exit: the residuals from the model evaluated at final value of THETA, i.e., RS contains the vector y-Xθ^.
17:   TOL – REAL (KIND=nag_wp)Input
On entry: the relative precision for the final estimates. Convergence is assumed when both the relative change in the value of SIGMA and the relative change in the value of each element of THETA are less than TOL.
It is advisable for TOL to be greater than 100×machine precision.
Constraint: TOL>0.0.
18:   EPS – REAL (KIND=nag_wp)Input
On entry: a relative tolerance to be used to determine the rank of X. See F04JGF for further details.
If EPS<machine precision or EPS>1.0 then machine precision will be used in place of TOL.
A reasonable value for EPS is 5.0×10-6 where this value is possible.
19:   MAXIT – INTEGERInput
On entry: the maximum number of iterations that should be used during the estimation.
A value of MAXIT=50 should be adequate for most uses.
Constraint: MAXIT>0.
20:   NITMON – INTEGERInput
On entry: determines the amount of information that is printed on each iteration.
NITMON0
No information is printed.
NITMON>0
On the first and every NITMON iterations the values of SIGMA, THETA and the change in THETA during the iteration are printed.
When printing occurs the output is directed to the current advisory message unit (see X04ABF).
21:   NIT – INTEGEROutput
On exit: the number of iterations that were used during the estimation.
22:   WK(M+4×N) – REAL (KIND=nag_wp) arrayWorkspace
23:   IFAIL – INTEGERInput/Output
On entry: IFAIL must be set to 0, -1​ or ​1. If you are unfamiliar with this parameter you should refer to Section 3.3 in the Essential Introduction for details.
For environments where it might be inappropriate to halt program execution when an error is detected, the value -1​ or ​1 is recommended. If the output of error messages is undesirable, then the value 1 is recommended. Otherwise, because for this routine the values of the output parameters may be useful even if IFAIL0 on exit, the recommended value is -1. When the value -1​ or ​1 is used it is essential to test the value of IFAIL on exit.
On exit: IFAIL=0 unless the routine detects an error or a warning has been flagged (see Section 6).

6  Error Indicators and Warnings

If on entry IFAIL=0 or -1, explanatory error messages are output on the current error message unit (as defined by X04AAF).
Note: G02HDF may return useful information for one or more of the following detected errors or warnings.
Errors or warnings detected by the routine:
IFAIL=1
On entry,N1,
orM<1,
orNM,
orLDX<N.
IFAIL=2
On entry,BETA0.0, and ISIGMA0,
orSIGMA0.0.
IFAIL=3
On entry,TOL0.0,
orMAXIT0.
IFAIL=4
A value returned by the CHI function is negative.
IFAIL=5
During iterations a value of SIGMA0.0 was encountered.
IFAIL=6
A failure occurred in F04JGF . This is an extremely unlikely error. If it occurs, please contact NAG.
IFAIL=7
The weighted least squares equations are not of full rank. This may be due to the X matrix not being of full rank, in which case the results will be valid. It may also occur if some of the Gii values become very small or zero, see Section 8. The rank of the equations is given by K. If the matrix just fails the test for nonsingularity then the result IFAIL=7 and K=M is possible (see F04JGF).
IFAIL=8
The routine has failed to converge in MAXIT iterations.
IFAIL=9
Having removed cases with zero weight, the value of N-K0, i.e., no degree of freedom for error. This error will only occur if ISIGMA>0.

7  Accuracy

The accuracy of the results is controlled by TOL. For the accuracy of the weighted least squares see F04JGF.

8  Further Comments

In cases when ISIGMA0 it is important for the value of SIGMA to be of a reasonable magnitude. Too small a value may cause too many of the winsorized residuals, i.e., ψri/σ, to be zero, which will lead to convergence problems and may trigger the IFAIL=7 error.
By suitable choice of the functions CHI and PSI this routine may be used for other applications of iterative weighted least squares.
For the variance-covariance matrix of θ see G02HFF.

9  Example

Having input X, Y and the weights, a Schweppe type regression is performed using Huber's ψ function. The subroutine BETCAL calculates the appropriate value of β2.

9.1  Program Text

Program Text (g02hdfe.f90)

9.2  Program Data

Program Data (g02hdfe.d)

9.3  Program Results

Program Results (g02hdfe.r)


G02HDF (PDF version)
G02 Chapter Contents
G02 Chapter Introduction
NAG Library Manual

© The Numerical Algorithms Group Ltd, Oxford, UK. 2012