ASTM E691-99

NOTICE: This standard has either been superseded and replaced by a new version or withdrawn.
Contact ASTM International (www.astm.org) for the latest information
An American National Standard
Designation: E 691 – 99
Standard Practice for
Conducting an Interlaboratory Study to Determine the
Precision of a Test Method
This standard is issued under the fixed designation E691; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision.Anumber in parentheses indicates the year of last reapproval.A
superscript epsilon (e) indicates an editorial change since the last revision or reapproval.
INTRODUCTION
Testsperformedonpresumablyidenticalmaterialsinpresumablyidenticalcircumstancesdonot,in
general, yield identical results. This is attributed to unavoidable random errors inherent in every test
procedure; the factors that may influence the outcome of a test cannot all be completely controlled. In
the practical interpretation of test data, this inherent variability has to be taken into account. For
instance, the difference between a test result and some specified value may be within that which can
beexpectedduetounavoidablerandomerrors,inwhichcasearealdeviationfromthespecifiedvalue
has not been demonstrated. Similarly, the difference between test results from two batches of material
will not indicate a fundamental quality difference if the difference is no more than can be attributed
to inherent variability in the test procedure. Many different factors (apart from random variations
between supposedly identical specimens) may contribute to the variability in application of a test
method, including: a the operator, b equipment used, c calibration of the equipment, and d
environment (temperature, humidity, air pollution, etc.). It is considered that changing laboratories
changes each of the above factors.The variability between test results obtained by different operators
or with different equipment will usually be greater than between test results obtained by a single
operator using the same equipment. The variability between test results taken over a long period of
time even by the same operator will usually be greater than that obtained over a short period of time
because of the greater possibility of changes in each of the above factors, especially the environment.
The general term for expressing the closeness of test results to the“ true” value or the accepted
referencevalueisaccuracy.Tobeofpracticalvalue,standardproceduresarerequiredfordetermining
the accuracy of a test method, both in terms of its bias and in terms of its precision. This practice
provides a standard procedure for determining the precision of a test method. Precision, when
evaluating test methods, is expressed in terms of two measurement concepts, repeatability and
reproducibility. Under repeatability conditions the factors listed above are kept or remain reasonably
constantandusuallycontributeonlyminimallytothevariability.Underreproducibilityconditionsthe
factors are generally different (that is, they change from laboratory to laboratory) and usually
contribute appreciably to the variability of test results.Thus, repeatability and reproducibility are two
practical extremes of precision.
The repeatability measure, by excluding the factors a through d as contributing variables, is not
intended as a mechanism for verifying the ability of a laboratory to maintain“ in-control” conditions
for routine operational factors such as operator-to-operator and equipment differences or any effects
of longer time intervals between test results. Such a control study is a separate issue for each
laboratory to consider for itself, and is not a recommended part of an interlaboratory study.
The reproducibility measure (including the factors a through d as sources of variability) reflects
whatprecisionmightbeexpectedwhenrandomportionsofahomogeneoussamplearesenttorandom
“in-control” laboratories.
To obtain reasonable estimates of repeatability and reproducibility precision, it is necessary in an
interlaboratory study to guard against excessively sanitized data in the sense that only the uniquely
best operators are involved or that a laboratory takes unusual steps to get“ good” results. It is also
importanttorecognizeandconsiderhowtotreat“poor”resultsthatmayhaveunacceptableassignable
causes (for example, departures from the prescribed procedure). The inclusion of such results in the
final precision estimates might be questioned.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959, United States.
E691–99
Anessentialaspectofcollectingusefulconsistentdataiscarefulplanningandconductofthestudy.
Questionsconcerningthenumberoflaboratoriesrequiredforasuccessfulstudyaswellasthenumber
of test results per laboratory affect the confidence in the precision statements resulting from the study.
Other issues involve the number, range, and types of materials to be selected for the study, and the
need for a well-written test method and careful instructions to the participating laboratories.
To evaluate the consistency of the data obtained in an interlaboratory study, two statistics may be
used: the “k-value”, used to examine the consistency of the within-laboratory precision from
laboratory to laboratory, and the “h-value”, used to examine the consistency of the test results from
laboratory to laboratory. Graphical as well as tabular diagnostic tools help in these examinations.
1. Scope
Test Method 8
Laboratories 9
1.1 This practice describes the techniques for planning,
Section
conducting, analyzing, and treating the results of an interlabo-
Materials 10
Number of Test Results per Material 11
ratory study (ILS) of a test method. The statistical techniques
Protocol 12
described in this practice provide adequate information for
Conducting the Testing Phase of the ILS
formulating the precision statement of a test method.
Pilot Run 13
Full Scale Run 14
1.1.1 A computer software package for performing the
Section
calculations and producing the tables and graphs associated
Calculation and Display of Statistics
with Practice E691. This software can be run on PC compat- Calculation of the Statistics 15
Tabular and Graphical Display of Statistics 16
iblecomputers,andhardcopytablesandgraphscanbeprinted
Data Consistency
on dot-matrix printers.
Flagging Inconsistent Results 17
1.2 This practice does not concern itself with the develop- Investigation 18
Task Group Actions 19
ment of test methods but rather with gathering the information
Examples of Interlaboratory Studies 20
needed for a test method precision statement after the devel-
Precision Statement Information
opment stage has been successfully completed. The data Repeatability and Reproducibility 21
Appendixes
obtained in the interlaboratory study may indicate, however,
Theoretical Considerations A1
that further effort is needed to improve the test method.
Index to Selected Terms A2
1.3 Since the primary purpose of this practice is the devel- References
Tables and Figures
opmentoftheinformationneededforaprecisionstatement,the
Tables Table
experimental design in this practice may not be optimum for
Glucose in Serum Example 1–4 & 6–7
evaluating materials, apparatus, or individual laboratories. Pentosans in Pulp Example 8–11
Critical Values of Consistency Statistics, h and k 5
1.4 Field of Application—This practice is concerned exclu-
Figures Fig.
sively with test methods which yield a single numerical figure
Glucose in Serum Example 1–5
Pentosans in Pulp Example 6–10
asthetestresult,althoughthesinglefiguremaybetheoutcome
of a calculation from a set of measurements.
1.6 This standard may involve hazardous materials, opera-
1.4.1 This practice does not cover methods in which the
tions, and equipment. This standard does not purport to
measurement is a categorization, such as a go-no-go allocation
address all of the safety problems associated with its use. It is
(two categories) or a sorting scheme into two or more
the responsibility of the user of this standard to establish
categories. For practical purposes, the discontinuous nature of
appropriate safety and health practices and determine the
measurementsofthesetypesmaybeignoredwhenatestresult
applicability of regulatory limitations prior to use.
is defined as an average of several individual measurements.
Then, this practice may be applicable, but caution is required
2. Referenced Documents
and a statistician should be consulted.
2.1 ASTM Standards:
1.5 The information in this practice is arranged as follows:
E177 Practice for Use of the Terms Precision and Bias in
Section 2
ASTM Test Methods
Scope 1
E456 Terminology Related to Quality and Statistics
Referenced Documents 2
Terminology 3
E1169 Guide for Conducting Ruggedness Tests
Summary of Practice 4
2.2 ASTM Adjuncts:
Significance and Use 5
E691ConductinganInterlaboratoryStudytoDeterminethe
Planning the Interlaboratory Study (ILS)
Precision of a Test Method
ILS Membership 6
Basic Design 7
3. Terminology
3.1 Definitions—For formal definitions of statistical terms,
see Terminology E456.
This practice is under the jurisdiction ofASTM Committee E-11 on Statistical
Methods and is the direct responsibility of Subcommittee E11.20 on Test Method
Evaluation and Quality Control.
Current edition approved May 10, 1999. Published August 1999. Originally Annual Book of ASTM Standards, Vol 14.02.
published as E 691–79. Last previous edition E 691–92. An adjunct is available from ASTM Headquarters. Request ADJE0691.
E691–99
3.2 Definitions of Terms Specific to This Standard: the bias of the test method. For a discussion of bias estimation
andtherelationshipsbetweenprecision,bias,andaccuracy,see
3.2.1 Test Method and Protocol—In this practice, the term
Practice E177.
“test method” is used both for the actual measurement process
3.2.5 Repeatability and Reproducibility—These terms deal
and for the written description of the process, while the term
with the variability of test results obtained under specified
“protocol” is used for the directions given to the laboratories
laboratory conditions. Repeatability concerns the variability
for conducting the ILS.
between independent test results obtained within a single
3.2.2 Observations, Test Determinations and Test Results:
laboratory in the shortest practical period of time by a single
3.2.2.1 A test method often has three distinct stages, the
operator with a specific set of test apparatus using test
direct observation of dimensions or properties, the arithmetic
specimens(ortestunits)takenatrandomfromasinglequantity
combination of the observed values to obtain a test determina-
of homogeneous material obtained or prepared for the ILS.
tion, and the arithmetic combination of a number of test
Reproducibility deals with the variability between single test
determinations to obtain the test result of the test method. In
results obtained in different laboratories, each of which has
the simplest of test methods a single direct observation is both
applied the test method to test specimens (or test units) taken
the test determination and the test result. For example, the test
at random from a single quantity of homogeneous material
method may require the measurement of the mass of a test
obtained or prepared for the ILS.
specimen prepared in a prescribed way. Another test method
3.2.5.1 Repeatability Conditions—The within-laboratory
may require the measurement of the area of the test specimen conditions specified above for repeatability. The single-
aswellasthemass,andthendirectthatthemassbedividedby
operator, single-set-of-apparatus requirement means that for a
the area to obtain the mass per unit area of the specimen. The particular step in the measurement process the same combina-
whole process of measuring the mass and the area and tion of operator and apparatus is used for every test result and
on every material. Thus, one operator may prepare the test
calculating the mass per unit area is a test determination. If the
test method specifies that only one test determination is to be specimens, a second measure the dimensions and a third
measurethebreakingforce.“Shortestpracticalperiodoftime”
made, then the test determination value is the test result of the
means that the test results, at least for one material, are
test method. Some test methods require that several determi-
obtained in a time not less than in normal testing and not so
nations be made and the values obtained be averaged or
long as to permit significant changes in test material, equip-
otherwisecombinedtoobtainthetestresultofthetestmethod.
ment or environment.
Averagingofseveraldeterminationsisoftenusedtoreducethe
3.3 For further discussion of the terms discussed above, see
effect of local variations of the property within the material.
Practice E177, and the formal definitions in Practice E456.
3.2.2.2 Inthispractice,theterm“testdetermination”isused
both for the process and for the value obtained by the process,
4. Summary of Practice
except when “test determination value” is needed for clarity.
4.1 The procedure presented in this practice consists of
3.2.2.3 Thenumberoftestdeterminationsrequiredforatest
three basic steps: planning the interlaboratory study, guiding
result should be specified in each individual test method. The
thetestingphaseofthestudy,andanalyzingthetestresultdata.
number of test results required for an interlaboratory study of
The analysis utilizes tabular, graphical, and statistical diagnos-
a test method is specified in the protocol of that study.
tic tools for evaluating the consistency of the data so that
3.2.3 Test Specimens and Test Units—In this practice a test
unusual values may be detected and investigated, and also
unit is the total quantity of material needed for obtaining a test
includesthecalculationofthenumericalmeasuresofprecision
result as specified by the test method. The portion of the test
of the test method pertaining to both within-laboratory repeat-
unitneededforobtainingasingletestdeterminationiscalleda
ability and between-laboratory reproducibility.
test specimen. Usually a separate test specimen is required for
5. Significance and Use
each test determination.
5.1 ASTM regulations require precision statements in all
3.2.4 Precision, Bias, and Accuracy of a Test Method:
test methods in terms of repeatability and reproducibility. This
3.2.4.1 When a test method is applied to a large number of
practice may be used in obtaining the needed information as
portions of a material, that are as nearly alike as possible, the
simply as possible. This information may then be used to
test results obtained nevertheless will not all have the same
prepare a precision statement in accordance with Practice
value.Ameasure of the degree of agreement among these test
E177.
results describes the p
...