Standard Practice for Dealing With Outlying Observations

ABSTRACT
This practice covers outlying observations in samples and how to test the statistical significance of them. An outlying observation, or outlier, is one that appears to deviate markedly from other members of the sample in which it occurs. In this connection, the following two alternatives are of interest: (i) an outlying observation may be merely an extreme manifestation of the random variability inherent in the data. If this is true, the value should be retained and processed in the same manner as the other observations in the sample. (ii) An outlying observation may be the result of gross deviation from prescribed experimental procedure or an error in calculating or recording the numerical value. In such cases, it may be desirable to institute an investigation to ascertain the reason for the aberrant value. The observation may even actually be rejected as a result of the investigation, though not necessarily so. At any rate, in subsequent data analysis, the outlier or outliers will be recognized as probably being from a different population than that of the other sample values. Recommended criteria and illustrations for single samples including the Dixon criteria which are based entirely on ratios of differences between the observations, criterion using independent standard deviation, and criterion for known standard deviation are presented.
SCOPE
1.1 This practice covers outlying observations in samples and how to test the statistical significance of them. An outlying observation, or "outlier," is one that appears to deviate markedly from other members of the sample in which it occurs. In this connection, the following two alternatives are of interest:  
1.1.1 An outlying observation may be merely an extreme manifestation of the random variability inherent in the data. If this is true, the value should be retained and processed in the same manner as the other observations in the sample.  
1.1.2 On the other hand, an outlying observation may be the result of gross deviation from prescribed experimental procedure or an error in calculating or recording the numerical value. In such cases, it may be desirable to institute an investigation to ascertain the reason for the aberrant value. The observation may even actually be rejected as a result of the investigation, though not necessarily so. At any rate, in subsequent data analysis the outlier or outliers will be recognized as probably being from a different population than that of the other sample values.  
1.2 It is our purpose here to provide statistical rules that will lead the experimenter almost unerringly to look for causes of outliers when they really exist, and hence to decide whether alternative 1.1.1 above, is not the more plausible hypothesis to accept, as compared to alternative 1.1.2, in order that the most appropriate action in further data analysis may be taken. The procedures covered herein apply primarily to the simplest kind of experimental data, that is, replicate measurements of some property of a given material, or observations in a supposedly single random sample. Nevertheless, the tests suggested do cover a wide enough range of cases in practice to have broad utility.

General Information

Status
Historical
Publication Date
09-May-2002
Technical Committee
Current Stage
Ref Project

Relations

Buy Standard

Standard
ASTM E178-02 - Standard Practice for Dealing With Outlying Observations
English language
18 pages
sale 15% off
Preview
sale 15% off
Preview

Standards Content (Sample)

NOTICE: This standard has either been superseded and replaced by a new version or withdrawn.
Contact ASTM International (www.astm.org) for the latest information
An American National Standard
Designation: E 178 – 02
Standard Practice for
1
Dealing With Outlying Observations
This standard is issued under the fixed designation E178; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision.Anumber in parentheses indicates the year of last reapproval.A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
1. Scope 3. Terminology
1.1 This practice covers outlying observations in samples 3.1 Definitions: The terminology defined in Terminology
andhowtotestthestatisticalsignificanceofthem.Anoutlying E456 applies to this standard unless modified herein.
observation, or “outlier,” is one that appears to deviate mark- 3.1.1 outlier—see outlying observation.
edly from other members of the sample in which it occurs. In 3.1.2 outlying observation, n—an observation that appears
this connection, the following two alternatives are of interest: todeviatemarkedlyinvaluefromothermembersofthesample
1.1.1 An outlying observation may be merely an extreme in which it appears.
manifestation of the random variability inherent in the data. If
4. Significance and Use
this is true, the value should be retained and processed in the
same manner as the other observations in the sample. 4.1 When the experimenter is clearly aware that a gross
deviation from prescribed experimental procedure has taken
1.1.2 Ontheotherhand,anoutlyingobservationmaybethe
result of gross deviation from prescribed experimental proce- place,theresultantobservationshouldbediscarded,whetheror
not it agrees with the rest of the data and without recourse to
dureoranerrorincalculatingorrecordingthenumericalvalue.
In such cases, it may be desirable to institute an investigation statistical tests for outliers. If a reliable correction procedure,
for example, for temperature, is available, the observation may
to ascertain the reason for the aberrant value. The observation
may even actually be rejected as a result of the investigation, sometimes be corrected and retained.
4.2 In many cases evidence for deviation from prescribed
though not necessarily so. At any rate, in subsequent data
analysis the outlier or outliers will be recognized as probably procedure will consist primarily of the discordant value itself.
Insuchcasesitisadvisabletoadoptacautiousattitude.Useof
being from a different population than that of the other sample
values. one of the criteria discussed below will sometimes permit a
clear-cut decision to be made. In doubtful cases the experi-
1.2 Itisourpurposeheretoprovidestatisticalrulesthatwill
menter’s judgment will have considerable influence.When the
lead the experimenter almost unerringly to look for causes of
outliers when they really exist, and hence to decide whether experimentercannotidentifyabnormalconditions,heshouldat
least report the discordant values and indicate to what extent
alternative 1.1.1 above, is not the more plausible hypothesis to
accept, as compared to alternative 1.1.2, in order that the most they have been used in the analysis of the data.
4.3 Thus, for purposes of orientation relative to the over-all
appropriate action in further data analysis may be taken. The
procedures covered herein apply primarily to the simplest kind problem of experimentation, our position on the matter of
screening samples for outlying observations is precisely the
of experimental data, that is, replicate measurements of some
property of a given material, or observations in a supposedly following:
4.3.1 Physical Reason Known or Discovered for Outlier(s):
single random sample. Nevertheless, the tests suggested do
cover a wide enough range of cases in practice to have broad 4.3.1.1 Reject observation(s).
4.3.1.2 Correct observation(s) on physical grounds.
utility.
4.3.1.3 Reject it (them) and possibly take additional obser-
2. Referenced Documents
vation(s).
2.1 ASTM Standards: 4.3.2 Physical Reason Unknown—Use Statistical Test:
2
E456 Terminology Relating to Quality and Statistics 4.3.2.1 Reject observation(s).
4.3.2.2 Correct observation(s) statistically.
4.3.2.3 Reject it (them) and possibly take additional obser-
1
ThispracticeisunderthejurisdictionofASTMCommitteeE11onQualityand
vation(s).
Statistics and is the direct responsibility of Subcommittee E11.10 on Sampling.
4.3.2.4 Employtruncated-sampletheoryforcensoredobser-
Current edition approved May 10, 2002. Published July 2002. Originally
published as E178–61 T. Last previous edition E178–00.
vations.
2
Annual Book of ASTM Standards, Vol 14.02.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959, United States.
1

---------------------- Page: 1 ----------------------
E178–02
4.4 The statistical test may always be used to support a T
...

Questions, Comments and Discussion

Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.