ASTM C1215-92(2006)

NOTICE: This standard has either been superseded and replaced by a new version or withdrawn.
Please contact ASTM International (www.astm.org) for the latest information.
Designation:C1215–92 (Reapproved 2006)
Standard Guide for
Preparing and Interpreting Precision and Bias Statements in
Test Method Standards Used in the Nuclear Industry
This standard is issued under the fixed designation C1215; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision.Anumber in parentheses indicates the year of last reapproval.A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
INTRODUCTION
Test method standards are required to contain precision and bias statements. This guide contains a
glossary that explains various terms that often appear in these statements as well as an example
illustrating such statements for a specific set of data. Precision and bias statements are shown to vary
according to the conditions under which the data were collected.This guide emphasizes that the error
model (an algebraic expression that describes how the various sources of variation affect the
measurement) is an important consideration in the formation of precision and bias statements.
1. Scope Nuclear Materials Management
1.1 Thisguidecoversterminologyusefulforthepreparation
3. Terminology for Precision and Bias Statements
and interpretation of precision and bias statements.
3.1 Definitions:
1.2 In formulating precision and bias statements, it is
3.1.1 accuracy (see bias)—(1) bias. (2) the closeness of a
importanttounderstandthestatisticalconceptsinvolvedandto
measured value to the true value. (3) the closeness of a
identify the major sources of variation that affect results.
measured value to an accepted reference or standard value.
Appendix X1 provides a brief summary of these concepts.
3.1.1.1 Discussion—For many investigators, accuracy is
1.3 To illustrate the statistical concepts and to demonstrate
attained only if a procedure is both precise and unbiased (see
some sources of variation, a hypothetical data set has been
bias). Because this blending of precision into accuracy can
analyzed in Appendix X2. Reference to this example is made
resultoccasionallyinincorrectanalysesandunclearstatements
throughout this guide.
of results, ASTM requires statement on bias instead of accu-
1.4 It is difficult and at times impossible to ship nuclear
racy.
materialsforinterlaboratorytesting.Thus,precisionstatements
3.1.2 analysis of variance (ANOVA)—thebodyofstatistical
for test methods relating to nuclear materials will ordinarily
theory,methods,andpracticesinwhichthevariationinasetof
reflect only within-laboratory variation.
data is partitioned into identifiable sources of variation.
2. Referenced Documents Sources of variation may include analysts, instruments,
samples, and laboratories. To use the analysis of variance, the
2.1 ASTM Standards:
data collection method must be carefully designed based on a
E177 Practice for Use of the Terms Precision and Bias in
modelthatincludesallthesourcesofvariationofinterest.(See
ASTM Test Methods
Example, X2.1.1)
E691 Practice for Conducting an Interlaboratory Study to
3.1.3 bias (see accuracy)—a constant positive or negative
Determine the Precision of a Test Method
deviation of the method average from the correct value or
2.2 ANSI Standard:
accepted reference value.
ANSI N15.5 Statistical Terminology and Notation for
3.1.3.1 Discussion—Bias represents a constant error as
opposed to a random error.
(a) A method bias can be estimated by the difference (or
This guide is under the jurisdiction ofASTM Committee C26 on Nuclear Fuel
relative difference) between a measured average and an ac-
Cycle and is the direct responsibility of Subcommittee C26.08 on Quality Assur-
cepted standard or reference value. The data from which the
ance, Statistical Applications, and Reference Materials.
Current edition approved Jan. 1, 2006. Published February 2006. Originally estimateisobtainedshouldbestatisticallyanalyzedtoestablish
approvedin1992.Lastpreviouseditionapprovedin1997asC1215–92(1997).DOI:
10.1520/C1215-92R06.
For referenced ASTM standards, visit the ASTM website, www.astm.org, or
contact ASTM Customer Service at service@astm.org. For Annual Book of ASTM Available fromAmerican National Standards Institute (ANSI), 25 W. 43rd St.,
Standards volume information, refer to the standard’s Document Summary page on 4th Floor, New York, NY 10036, http://www.ansi.org.
the ASTM website. Refer to Form and Style for ASTM Standards, 8th Ed., 1989, ASTM.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959, United States.
NOTICE: This standard has either been superseded and replaced by a new version or withdrawn.
Please contact ASTM International (www.astm.org) for the latest information.
C1215–92 (2006)
error models. In the additive model, the errors are independent of the
bias in the presence of random error. A thorough bias investi-
value of the item being measured. Thus, for example, for repeated
gation of a measurement procedure requires a statistically
measurements under identical conditions, the additive error model
designed experiment to repeatedly measure, under essentially
might be
thesameconditions,asetofstandardsorreferencematerialsof
X 5µ 1 b 1´ (1)
known value that cover the range of application. Bias often i i
varies with the range of application and should be reported
where:
accordingly. th
X = the result of the i measurement,
i
(b) In statistical terminology, an estimator is said to be
µ = the true value of the item,
unbiased if its expected value is equal to the true value of the
b = a bias, and
parameter being estimated. (See Appendix X1.)
´ = a random error usually assumed to have a normal
i
(c)Thebiasofatestmethodisalsocommonlyindicatedby
distribution with mean zero and variance s .
analytical chemists as percent recovery. A number of repeti-
In the multiplicative model, the error is proportional to the true
tions of the test method on a reference material are performed,
value. A multiplicative error model for percent recovery (see bias)
and an average percent recovery is calculated. This average
might be:
provides an estimate of the test method bias, which is multi-
X 5µb´ (2)
i i
plicative in nature, not additive. (See Appendix X2.)
andamultiplicativemodelforaneutroncountermeasurementmight
(d) Use of a single test result to estimate bias is strongly
be:
discouraged because, even if there were no bias, random error
X 5 µ 1 µb 1 µ· ´
i i
alone would produce a nonzero bias estimate.
3.1.4 coeffıcient of variation—see relative standard devia-
5 µ 1 1 b1´ (3)
~ !
i
tion.
( b) Clearly, there are many ways in which errors may affect a final
3.1.5 confidence interval—an interval used to bound the
measurement. The additive model is frequently assumed and is the
value of a population parameter with a specified degree of basis for many common statistical procedures. The form of the model
influences how the error components will be estimated and is very
confidence (this is an interval that has different values for
important, for example, in the determination of measurement uncer-
different random samples).
tainties. Further discussion of models is given in the Example of
3.1.5.1 Discussion—When providing a confidence interval,
Appendix X2 and in Appendix X4.
analysts should give the number of observations on which the
3.1.8 precision—a generic concept used to describe the
interval is based.The specified degree of confidence is usually
dispersion of a set of measured values.
90, 95, or 99%. The form of a confidence interval depends on
3.1.8.1 Discussion—It is important that some quantitative
underlying assumptions and intentions. Usually, confidence
measure be used to specify precision. A statement such as,
intervals are taken to be symmetric, but that is not necessarily
“The precision is 1.54 g” is useless. Measures frequently used
so, as in the case of confidence intervals for variances.
to express precision are standard deviation, relative standard
Construction of a symmetric confidence interval for a popula-
deviation, variance, repeatability, reproducibility, confidence
tion mean is discussed in Appendix X3.
interval, and range. In addition to specifying the measure and
It is important to realize that a given confidence-interval estimate
the precision, it is important that the number of repeated
eitherdoesordoesnotcontainthepopulationparameter.Thedegreeof
measurementsuponwhichtheprecisionestimatedisbasedalso
confidence is actually in the procedure. For example, if the interval (9,
be given. (See Example, Appendix X2.)
13)isa90%confidenceintervalforthemean,weareconfidentthatthe
procedure (take a sample, construct an interval) by which the interval
(a) It is strongly recommended that a statement on precision of a
(9, 13) was constructed will 90% of the time produce an interval that
measurement procedure include the following:
does indeed contain the mean. Likewise, we are confident that 10% of
the time the interval estimate obtained will not contain the mean. Note
(1) Adescription of the procedure used to obtain the data,
thattheabsenceofsamplesizeinformationdetractsfromtheusefulness
(2) The number of repetitions, n, of the measurement
of the confidence interval. If the interval were based on five observa-
procedure,
tions, a second set of five might produce a very different interval. This
(3) The sample mean and standard deviation of the
would not be the case if 50 observations were taken.
measurements,
3.1.6 confidencelevel—theprobability,usuallyexpressedas
(4) The measure of precision being reported,
a percent, that a confidence interval will contain the parameter
(5) The computed value of that measure, and
of interest. (See discussion of confidence interval inAppendix
(6) The applicable range or concentration.
X3.)
The importance of items (3) and (4) lies in the fact that with these a
3.1.7 error model—an algebraic expression that describes
readermaycalculateaconfidenceintervalorrelativestandarddeviation
how a measurement is affected by error and other sources of
as desired.
variation. The model may or may not include a sampling error
(b) Precision is sometimes measured by repeatability and reproduc-
term.
ibility (see Practice E177, and Mandel and Laskof (3)). TheANSI and
3.1.7.1 Discussion—Ameasurement error is an error attrib- ASTM documents differ slightly in their usages of these terms. The
following is quoted from Kendall and Buckland (2):
utable to the measurement process. The error may affect the
“In some situations, especially interlaboratory comparisons, preci-
measurement in many ways and it is important to correctly
sionisdefinedbyemployingtwoadditionalconcepts:repeatabilityand
model the effect of the error on the measurement.
reproducibility. The general situation giving rise to these distinctions
(a) Two common models are the additive and the multiplicative comes from the interest in assessing the variability within several
NOTICE: This standard has either been superseded and replaced by a new version or withdrawn.
Please contact ASTM International (www.astm.org) for the latest information.
C1215–92 (2006)
groups of measurements and between those groups of measurements.
when taking measurements. Any value in an interval is
Repeatability, then, refers to the within-group dispersion of the
considered possible and thus the population is conceptually
measurements, while reproducibility refers to the between-group dis-
infinite. The definition given in 3.1.11 is then the appropriate
persion. In interlaboratory comparison studies, for example, the inves-
definition. (See representative sample and Appendix X5.)
tigation seeks to determine how well each laboratory can repeat its
3.1.12 range—the largest minus the smallest of a set of
measurements (repeatability) and how well the laboratories agree with
each other (reproducibility). Similar discussions can apply to the numbers.
comparison of laboratory technicians’ skills, the study of competing
3.1.13 relative standard deviation (percent)— the sample
types of equipment, and the use of particular procedures within a
standard deviation expressed as a percent of the sample mean.
laboratory. An essential feature usually required, however, is that
The %RSD is calculated using the following equation:
repeatability and reproducibility be measured as variances (or standard
deviationsincertaininstances),sothatbothwithin-andbetween-group s
%RSD 5100 (4)
dispersionsaremodeledasarandomvariable.Thestatisticaltooluseful
| x¯ |
for the analysis of such comparisons is the analysis of variance.”
where:
( c) In Practice E177 it is recommended that the term repeatability
s = sample standard deviation and
be reserved for the intrinsic variation due solely to the measurement
x¯ = sample mean.
procedure, excluding all variation from factors such as analyst, time
and laboratory and reserving reproducibility for the variation due to all
3.1.13.1 Discussion—Theuseofthe%RSD(orRSD(%))to
factors including laboratory. Repeatability can be measured by the
describe precision implies that the uncertainty is a function of
standard deviation, s ,of n consecutive measurements by the same
r
the measurement values. An appropriate error model might
operator on the same instrument. Reproducibility can be measured by
then be X =µ(1+ b+ ´). (See Example, Appendix X2.)
i i
thestandarddeviation, s ,ofmmeasurements,oneobtainedfromeach
R
Some authors use RSD for the ratio,s/|x|, while others call
of m independent laboratories. When interlaboratory testing is not
this the coeffıcient of variation. At times authors use RSD to
practical, the reproducibility conditions should be described.
mean%RSD.Thus,itisimportanttodeterminewhichmeaning
(d) Two additional terms are recommended in Practice E177.These
are repeatability limit and reproducibility limit. These are intended to is intended when RSD without the percent sign is used. The
give estimates of how different two measurements can be. The
recommended practice is %RSD=100 (s/| x¯ |) and RSD= s/|
repeatability limit is defined as 1.96 2 s , and the reproducibility
=
r
x¯ |.
limit is defined as 1.96=2 s , where s is the estimated standard
R r
3.1.14 repeatability—see Discussion in 3.1.8.
deviationassociatedwithrepeatability,ands istheestimatedstandard
R
3.1.15 representative sample—a generic term indicating
deviation associated with reproducibility. Thus, if normality can be
assumed, these limits represent 95% limits for the difference between thatthesampleistypicalofthepopulationwithrespecttosome
two measurements taken under the respective conditions. In the
specified characteristic(s).
reproducibilitycase,thismeansthat“approximately95%ofallpairsof
3.1.15.1 Discussion—Taken literally, a repres
...