NAG Library Routine Document

g07gaf  (outlier_peirce_1var)

 Contents

    1  Purpose
    7  Accuracy

1
Purpose

g07gaf identifies outlying values using Peirce's criterion.

2
Specification

Fortran Interface
Subroutine g07gaf ( n, p, y, mean, var, iout, niout, ldiff, diff, llamb, ifail)
Integer, Intent (In):: n, p, ldiff
Integer, Intent (Inout):: ifail
Integer, Intent (Out):: iout(n), niout
Real (Kind=nag_wp), Intent (In):: y(n), mean, var
Real (Kind=nag_wp), Intent (Out):: diff(ldiff), llamb(ldiff)
C Header Interface
#include nagmk26.h
void  g07gaf_ ( const Integer *n, const Integer *p, const double y[], const double *mean, const double *var, Integer iout[], Integer *niout, const Integer *ldiff, double diff[], double llamb[], Integer *ifail)

3
Description

g07gaf flags outlying values in data using Peirce's criterion. Let
Peirce's method flags yi as a potential outlier if yi-μx, where x=σ2z and z is obtained from the solution of
Rm = λ m-n mm n-m n-m nn (1)
where
R = 2 exp z2 - 1 2 1- Φz (2)
and Φ is the cumulative distribution function for the standard Normal distribution.
As σ~2 is unknown an assumption is made that the relationship between σ~2 and σ2, hence λ, depends only on the sum of squares of the rejected observations and the ratio estimated as
λ2 = n-p-m z2 n-p-m  
which gives
z2 = 1+ n-p-m m 1-λ2 (3)
A value for the cutoff x is calculated iteratively. An initial value of R=0.2 is used and a value of λ is estimated using equation (1). Equation (3) is then used to obtain an estimate of z and then equation (2) is used to get a new estimate for R. This process is then repeated until the relative change in z between consecutive iterations is ε, where ε is machine precision.
By construction, the cutoff for testing for m+1 potential outliers is less than the cutoff for testing for m potential outliers. Therefore Peirce's criterion is used in sequence with the existence of a single potential outlier being investigated first. If one is found, the existence of two potential outliers is investigated etc.
If one of a duplicate series of observations is flagged as an outlier, then all of them are flagged as outliers.

4
References

Gould B A (1855) On Peirce's criterion for the rejection of doubtful observations, with tables for facilitating its application The Astronomical Journal 45
Peirce B (1852) Criterion for the rejection of doubtful observations The Astronomical Journal 45

5
Arguments

1:     n – IntegerInput
On entry: n, the number of observations.
Constraint: n3.
2:     p – IntegerInput
On entry: p, the number of parameters in the model used in obtaining the y. If y is an observed set of values, as opposed to the residuals from fitting a model with p parameters, then p should be set to 1, i.e., as if a model just containing the mean had been used.
Constraint: 1pn-2.
3:     yn – Real (Kind=nag_wp) arrayInput
On entry: y, the data being tested.
4:     mean – Real (Kind=nag_wp)Input
On entry: if var>0.0, mean must contain μ, the mean of y, otherwise mean is not referenced and the mean is calculated from the data supplied in y.
5:     var – Real (Kind=nag_wp)Input
On entry: if var>0.0, var must contain σ2, the variance of y, otherwise the variance is calculated from the data supplied in y.
6:     ioutn – Integer arrayOutput
On exit: the indices of the values in y sorted in descending order of the absolute difference from the mean, therefore y iouti-1 - μ y iouti - μ , for i=2,3,,n.
7:     niout – IntegerOutput
On exit: the number of potential outliers. The indices for these potential outliers are held in the first niout elements of iout. By construction there can be at most n-p-1 values flagged as outliers.
8:     ldiff – IntegerInput
On entry: the maximum number of values to be returned in arrays diff and llamb.
If ldiff0, arrays diff and llamb are not referenced.
9:     diffldiff – Real (Kind=nag_wp) arrayOutput
On exit: diffi holds y-μ-σ2z for observation yiouti, for i=1,2,,minldiff,niout+1,n-p-1.
10:   llambldiff – Real (Kind=nag_wp) arrayOutput
On exit: llambi holds logλ2 for observation yiouti, for i=1,2,,minldiff,niout+1,n-p-1.
11:   ifail – IntegerInput/Output
On entry: ifail must be set to 0, -1​ or ​1. If you are unfamiliar with this argument you should refer to Section 3.4 in How to Use the NAG Library and its Documentation for details.
For environments where it might be inappropriate to halt program execution when an error is detected, the value -1​ or ​1 is recommended. If the output of error messages is undesirable, then the value 1 is recommended. Otherwise, if you are not familiar with this argument, the recommended value is 0. When the value -1​ or ​1 is used it is essential to test the value of ifail on exit.
On exit: ifail=0 unless the routine detects an error or a warning has been flagged (see Section 6).

6
Error Indicators and Warnings

If on entry ifail=0 or -1, explanatory error messages are output on the current error message unit (as defined by x04aaf).
Errors or warnings detected by the routine:
ifail=1
On entry, n=value.
Constraint: n3.
ifail=2
On entry, p=value and n=value.
Constraint: 1pn-2.
ifail=-99
An unexpected error has been triggered by this routine. Please contact NAG.
See Section 3.9 in How to Use the NAG Library and its Documentation for further information.
ifail=-399
Your licence key may have expired or may not have been installed correctly.
See Section 3.8 in How to Use the NAG Library and its Documentation for further information.
ifail=-999
Dynamic memory allocation failed.
See Section 3.7 in How to Use the NAG Library and its Documentation for further information.

7
Accuracy

Not applicable.

8
Parallelism and Performance

g07gaf is not threaded in any implementation.

9
Further Comments

One problem with Peirce's algorithm as implemented in g07gaf is the assumed relationship between σ2, the variance using the full dataset, and σ~2, the variance with the potential outliers removed. In some cases, for example if the data y were the residuals from a linear regression, this assumption may not hold as the regression line may change significantly when outlying values have been dropped resulting in a radically different set of residuals. In such cases g07gbf should be used instead.

10
Example

This example reads in a series of data and flags any potential outliers.
The dataset used is from Peirce's original paper and consists of fifteen observations on the vertical semidiameter of Venus.

10.1
Program Text

Program Text (g07gafe.f90)

10.2
Program Data

Program Data (g07gafe.d)

10.3
Program Results

Program Results (g07gafe.r)

© The Numerical Algorithms Group Ltd, Oxford, UK. 2017