ASTM E1601-19

This international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for the
Development of International Standards, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
Designation: E1601 − 19
Standard Practice for
Conducting an Interlaboratory Study to Evaluate the
Performance of an Analytical Method
This standard is issued under the fixed designation E1601; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision.Anumber in parentheses indicates the year of last reapproval.A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
1. Scope Interlaboratory Testing of Chemical Analysis Methods
(Withdrawn 2015)
1.1 This practice covers procedures and statistics for an
E2972GuideforProduction,Testing,andValueAssignment
interlaboratory study (ILS) of the performance of an analytical
of In-House Reference Materials for Metals, Ores, and
method. The study provides statistical values which are useful
Other Related Materials
in determining if a method is satisfactory for the purposes for
which it was developed. These statistical values may be
3. Terminology
incorporated in the method’s precision and bias section. This
3.1 Definitions—For definitions of terms used in this
practice discusses the meaning of the statistics and what users
practice, refer to Terminology E135.
of analytical methods may learn from them.
3.2 Definitions of Terms Specific to This Standard:
1.2 This standard does not purport to address all of the
3.2.1 interlaboratory test—measures the variability of re-
safety concerns, if any, associated with its use. It is the
sults when a test method is applied many times in a number of
responsibility of the user of this standard to establish appro-
laboratories.
priate safety, health, and environmental practices and deter-
3.2.2 replicate results—results obtained by applying a test
mine the applicability of regulatory limitations prior to use.
method a specified number of times to a material.
1.3 This international standard was developed in accor-
3.2.3 test protocol—gives instructions to each participating
dance with internationally recognized principles on standard-
laboratory, detailing the way it is to conduct its part of the
ization established in the Decision on Principles for the
interlaboratory test program.
Development of International Standards, Guides and Recom-
mendations issued by the World Trade Organization Technical
4. Summary of Practice
Barriers to Trade (TBT) Committee.
4.1 Instructions are provided for planning and conducting a
cooperative evaluation of a proposed analytical method.
2. Referenced Documents
4.2 The following list describes the organization of this
2.1 ASTM Standards:
practice:
E135Terminology Relating to Analytical Chemistry for
4.2.1 Sections1–5 define the scope, significance and use,
Metals, Ores, and Related Materials
referenced documents, and terms used in this practice.
E177Practice for Use of the Terms Precision and Bias in
4.2.2 Section6helpsusersofanalyticalmethodsunderstand
ASTM Test Methods
andusethestatisticsfoundinthePrecisionandBiassectionof
E691Practice for Conducting an Interlaboratory Study to
methods.
Determine the Precision of a Test Method
4.2.3 Sections 7 and 8 instruct the ILS coordinator and
E1169Practice for Conducting Ruggedness Tests
members of the task group on how to plan and conduct the
E1763Guide for Interpretation and Use of Results from
experimental phase of the study.
4.2.4 Section 9 discusses the procedures for collecting,
evaluating,anddisseminatingthedatafromtheinterlaboratory
This practice is under the jurisdiction ofASTM Committee E01 on Analytical test.
ChemistryforMetals,Ores,andRelatedMaterialsandisthedirectresponsibilityof
4.2.5 Section 10 presents the statistical calculations.
Subcommittee E01.22 on Laboratory Quality.
4.2.6 Sections 11 and 12 discuss the use of statistics to
Current edition approved Nov. 1, 2019. Published January 2020. Originally
evaluate a test method and the means of incorporating the ILS
approved in 1994. Last previous edition approved in 2012 as E1601–12. DOI:
10.1520/E1601-19.
statistics into Precision and Bias statements.
For referenced ASTM standards, visit the ASTM website, www.astm.org, or
contactASTM Customer Service at service@astm.org. ForAnnual Book ofASTM
Standards volume information, refer to the standard’s Document Summary page on The last approved version of this historical standard is referenced on
theASTM website. www.astm.org.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
E1601 − 19
4.2.7 The AnnexA1 gives the rationale for the calculations good condition; and (4) the method must be performed exactly
in Section 10. as written (for formal definitions, refer to Terminology E135).
6.2.1 Reproducibility Index, R—This statistic estimates the
5. Significance and Use
expected range of differences in results reported from two
laboratories, a range that is not exceeded in more than 5% of
5.1 Ideally, interlaboratory testing of a method is conducted
such comparisons. Use R to predict how well your results
by a randomly chosen group of laboratories that typifies the
should agree with those from another laboratory: First, obtain
kind of laboratory that is likely to use the method. In actuality,
a result under the conditions stated in 6.2, then add R to, and
this ideal is only approximated by the laboratories that are
subtract R from, this result to form a confidence interval. Such
available and willing to undertake the test work. The coordi-
an interval has a 95% probability of including a result
nator of the program must ensure that every participating
obtainablebythemethodshouldanotherlaboratoryanalyzethe
laboratory has appropriate facilities and personnel and per-
same sample. For example, a result of 46.57% was obtained.
forms the method exactly as written. If this goal is achieved,
IfRforthemethodatabout45%is0.543,the95%confidence
the statistics developed during the ILS will be adequate for
intervalfortheresult(thatis,oneexpectedtoincludetheresult
determining if the method is capable of producing satisfactory
obtainedinanotherlaboratory19timesoutof20)extendsfrom
precision in actual use. If the program includes certified
46.03% to 47.11%.
reference materials, the test data also provide information
concerning the bias of the method. The statistics provide a
NOTE 1—For those not conversant with statistical concepts, it is
general guide to the expected performance of the method.
importanttorealizethatinmostsuchcomparisons,thedifferenceswillbe
much smaller than the confidence interval implies. The 50% confidence
interval is only about one third (34.6%) as wide. Thus, the “average”
6. Statistical Guide for the Users of Analytical Methods
interval for the above result (one expected to include the result obtained
Evaluated as Directed in This Practice
by another laboratory half the time) extends from 46.4% to 46.8%. The
6.1 Standard Deviations (for formal definitions, refer to obviousimplicationisthat,althoughhalfthedifferenceswillbemorethan
0.2%, half will be less than 0.2%.
Terminology E135):
6.1.1 Minimum Standard Deviation of Method, s —This
6.2.2 Repeatability Index, r—This statistic is given in the
M
statistic measures the precision of test results under conditions
methodonlyiftheinterlaboratorytestwasdesignedtomeasure
ofminimumvariability.Becauseitisimprobablethatamethod
s. It estimates the expected range of results reported in the
r
in ordinary use will exhibit precision this good, no predictive
same laboratory on different days, a range that is not exceeded
indexiscalculatedfors .Usersadeptinstatisticsmaywishto
in more than 5% of such comparisons.
M
compare s and the short-term standard deviation of the
M
method measured in their laboratory. For most methods,
7. Interlaboratory Test Planning
short-term variability refers to results obtained within several
7.1 Analytical test methods start from a perceived need to
minutes by the same operator using the same equipment.
support one or more material specifications.
(Warning—The standard deviation of results obtained on
7.1.1 Develop a performance requirement for a method
different occasions, even in the same laboratory, probably will
from the material specification(s). Include the following fac-
exceed s .)
M
tors:expectedrangesofchemicalcompositionsofthematerials
6.1.2 Between-Laboratory Standard Deviation, s —This
R
tobecovered(method’sgeneralscope);specifiedelementsand
statistic is a measure of the precision expected for results
their amounts (scope ranges); and the precision required.
obtained in different laboratories. It reflects all sources of
7.1.2 Prepare a table of the elements and scope ranges to
variabilitythatoperateduringtheinterlaboratorytest,exceptin
cover the critical values in the material specifications. Use this
a test designed to eliminate the effect of test material inhomo-
information together with knowledge of the characteristics of
geneity.Itisusedtocalculatethereproducibilityindex, R.Use
the candidate analytical method to select test materials for the
s for evaluating the precision of methods. It represents the
R
interlaboratory program.
expected variability of results when a method is used in
different laboratories. 7.2 Draft Method—The process of developing methods and
testing them in a preliminary way is beyond the scope of this
6.1.3 Within-Laboratory Standard Deviation, s —This sta-
r
tistic cannot be calculated in a normal interlaboratory test. It is practice. All analytical skill and experience available to the
taskgroupmustbeexertedtoensurethatthemethodwillmeet
determinedonlyintestsdesignedtomeasurevariabilitywithin
laboratories.Whenthisstatisticisgiveninamethod,itreflects the project requirements in 7.1 and that it is free of technical
faults.Apreliminary,informaltestofamethodmustbecarried
all variability that may occur from day-to-day within a labo-
ratory (for example, from calibration, standardization, drift out in several laboratories before the final draft is prepared.
Individuals responsible for selecting the method may find
correction, or environmental changes). It is used to calculate
the repeatability index, r. The user is cautioned that additional helpful information in Practice E691 and Practice E1169. The
formal interlaboratory test must not start until the task group
sources of variation may affect results obtained in other
laboratories. reaches consensus on a clearly written, explicitly stated, and
unambiguously worded draft of the method in ASTM format,
6.2 Predictive Indexes—For the following indexes to apply,
which has completed editorial review.
these conditions must be met: (1) the test materials must be
sufficiently homogeneous; (2) analysts must be competent and 7.3 Test Materials—Appropriate test materials are essential
diligent; (3) analytical instruments and equipment must be in for a successful ILS. The larger the number of test materials
E1601 − 19
included in the test program, the better the statistical informa- study task group should consist of typical users’ laboratories.
tion generated. Conversely, the burden of running a very large Thereiswideagreementthatestimatesofprecisionbasedupon
number of materials may reduce the number of laboratories fewer than six laboratories become increasingly unreliable as
willing to participate. A method must cover a scope range the number decreases.Atest program involving fewer than six
extending both above and below the specified value(s). If laboratories does not comply with the requirements of this
possible, provide test materials near each limit. Ranges cover- practice. An effort should be made to enlist at least seven
ing several orders of magnitude should be tested with three or qualifiedlaboratoriesbeforebeginningatestprogram,toallow
more materials. for attrition. To be qualified to participate, a laboratory must
have proper equipment and personnel with sufficient training
7.3.1 Material composition and form must be within the
general scope of the method. If possible, include all material and experience to enable them to perform the method exactly
as it is written.
types the scope is expected to cover. Often, only limited
numbers of certified reference materials are available. Use 7.4.1 If all reasonable efforts fail to recruit at least six
those that best meet the criteria for the test. If important cooperatinglaboratories,uptotwooftherecruitedlaboratories
compositions are not covered by available reference materials, mayeachvolunteertosubmittwoindependentsetsoftestdata
as an expedient to provide a total of at least six sets of data.
find or prepare in-house reference materials for the missing
compositions (refer to Guide E2972. Minimum requirements for independence are that two typical
analysts,whodonotconsultwitheachotheraboutthemethod,
7.3.2 The quantity of the material must be sufficient to
perform the test protocol on different days. They should use
distribute to all laboratories participating in the test with about
separate equipment if possible and must not share calibration
50% held in reserve to cover unforeseen eventualities.
solutions or calibration curves.
7.3.3 Materials should be homogeneous on the scale of the
test portion consumed in each determination as well as among
8. Conducting the Interlaboratory Study (ILS)
the portions sent to different laboratories. Usually certified
reference materials have been tested for homogeneity, but test
8.1 Program Coordinator—One individual (presumably the
materials from other sources may have had only a minimal
task group chairman) will coordinate the entire ILS, if practi-
examination. The use of laboratory-scale melting and casting
cal. A prospective ILS program coordinator will find helpful
to produce test materials can sometimes lead to segregation of
informationonconductingtheprograminPracticeE691.Steps
one or more components in an alloy. Unless specially gathered
to organize the work to provide control while moving steadily
or prepared materials have been shown to be sufficiently
to its conclusion are:
homogeneous, they require the use of Test Plan B. It statisti-
8.1.1 Prepare a draft of the method to be tested.
cally removes the effect of moderate test material inhomoge-
8.1.2 Recruit a task group of participating laboratories.
neity from the estimates of the ILS statistics.
8.1.3 Select a set of test materials and assemble them into
7.3.4 Test material sent to each laboratory must be perma-
kits, one for each laboratory.
nently marked with its identity in such a manner that the
8.1.4 Write the test protocol to instruct each laboratory how
identification is not likely to be lost or obliterated.
to run the test.
7.3.5 If the test program is to evaluate the bias of the
8.1.5 Prepare a report form.
method, at least one test material must be certified for the
8.1.6 Establish a realistic time schedule for each part of the
amount of each element in the scope of the method. More
test program.
certified materials provide more complete information on bias.
8.1.7 Assemble and deliver to each participating laboratory
7.3.6 Prepare a list of the test materials, their identifying
everything needed to run the test: the draft method; the test
numbers, a brief description of material type (for example,
materials and a document which describes them; the test
low-carbonsteel),andapproximateamountsoftheelementsto
protocol; the report forms; a cover letter which includes the
be determined. This table becomes part of the documentation
deadlineforreturnofresults;andthename,address,telephone
sent to participating laboratories and provides information
and fax numbers, and email address of the person who will
needed for the research report and the precision and bias
handle problems and receive the completed report forms. The
statement.
program coordinator is strongly encouraged to request that all
7.3.7 Test Plan B is effective only when duplicate results
information be returned in electronic format, as most support
can be taken on a relatively homogeneous test portion. Ideal
documentation must be provided toASTM headquarters in the
methods for this approach are those in which replicate test
research report. Refer to the ASTM website for specific
portions can be put into solution and duplicate results obtained
requirements regarding the support information that must be
on each solution. If determinations are made directly on solid
provided in the research report. The program coordinator is
specimens, Test Plan B should be attempted only if each
strongly encouraged to become familiar with the format
laboratory can be provided with at least three portions of the
required for data entry into the program being used for
test material and there is reason to expect that duplicate results
statistical calculations and to request that cooperating labora-
on each portion will show less variability than results obtained
tories report data in a format amenable to the tool selected for
from different portions.
these calculations. For instance, CommitteeE01 maintains an
7.4 Number of Cooperating Laboratories—Conventional Excel spreadsheet for calculation of PracticeE1601 statistics
wisdom holds that the more laboratories participating in an on the E01 website. The spreadsheet requires that the data for
ILS, the better. Further, the laboratory types included in the each laboratory be compiled and entered into a single column.
E1601 − 19
Requiring ILS cooperators to report data in a similar format amaterialofprovenhomogeneity,specifyTestPlanA:threeor
greatly simplifies use of this statistical tool. If the ASTM more sequential replicate results on one portion of the material
Headquarters Statistics support group is used, then they may (Note 2). Direct each laboratory to analyze test materials in
have specific requirements for data submission. random order, but to complete measurements for the replicate
8.1.8 Expedite the laboratory testing. Follow up to ensure results (number specified in the protocol) on one test material
that the laboratories receive the test materials and understand before proceeding to another. For a test material of unknown
what is expected of them. Encourage laboratories to complete homogeneity, specify Test Plan B: sequential duplicate results
the work. onatleastthreeportionsofthematerial.Directeachlaboratory
8.1.9 Inspect results on each report form as it is received. to obtain the measurements for duplicate results on one test
Resolve omissions and apparent clerical errors at once. Obtain portion, followed by the specified number of other portions of
missing values. If obvious erroneous data are submitted, the same material before proceeding to another material. Give
determine the cause, if possible, and help the laboratory explicit instructions to the analyst for each test material,
eliminate the problem. Encourage the laboratory to submit a especially if the study uses Test PlanAfor some materials and
replacement set of data, if circumstances permit. (The final Test Plan B for others.
decision about replacing data will be made by the task group
NOTE 2—In some methods, the test portion is completely consumed in
after the testing is complete.)
obtaining one result. In these cases, select the sequential test portions to
8.1.10 Perform a preliminary statistical analysis. Summa-
minimize variation in composition, if possible. Any variation that does
occur will increase the method’s minimum standard deviation.
rize the comments from laboratories to explain questionable
results. Present this information to the task group.
8.3.2 A third test pattern may be used if the task group
8.1.11 As approved by the task group, prepare the final
wishes to measure the within-laboratory standard deviation, s,
r
statistical evaluation and the research report. Obtain the task
and calculate the repeatability index, r. Obtain sequential
group’s approval for the completed study.
duplicate results on a test material of proven homogeneity on
8.1.12 Modify the scope of the method, if necessary, and
each of at least three days. Direct each laboratory to obtain
preparetheprecisionandbiasstatement.Submitthecompleted
duplicate results on one test portion of a material on the
method to the technical subcommittee chairman for editorial
specified number of (not necessarily sequential) days. Several
review, followed by subcommittee ballot.
conditions must be explicitly spelled out in the protocol, as
follows:
8.2 Task Group—The task group usually consists of one
8.3.2.1 For methods in which samples are dissolved, pre-
representative from each participating laboratory. The labora-
pare a single test solution each day. For solid specimens,
tory representative’s name, address, telephone and fax
prepare them each day in the manner specified by the method.
numbers, and email address should be given to the task group
8.3.2.2 Each day the method must be performed in its
chairman when a laboratory agrees to participate.
entirety, including instrument setup, preparation of the calibra-
8.2.1 The laboratory representative shall be fully cognizant
tion solutions and calibration (for methods in which samples
of the laboratory’s capabilities and be in a position to ensure
are dissolved), and other steps necessary for each day’s work
the following:
as directed in the method. If the method includes
8.2.1.1 The laboratory is capable of performing the method
standardization, it must be performed before each day’s work
properly,
whether or not need for it is indicated.
8.2.1.2 Appropriate personnel are assigned to perform the
8.3.2.3 Determine the duplicate results on a single test
work and the method is followed exactly as written,
solution. For solid samples, determine the duplicate results
8.2.1.3 Test materials are handled properly,
with as little disturbance of the specimen as the method
8.2.1.4 The test protocol is complied with in all details,
permits.
8.2.1.5 The results are recorded accurately on the report
8.3.3 Thefollowingareinstructionsthatshouldbeusedina
form, and
test protocol (this list is not exhaustive):
8.2.1.6 The laboratory adheres to the program time sched-
8.3.3.1 Specify the number of significant digits with which
ule.
resultsaretoberecorded(thisshouldbeatleastonemoredigit
8.2.2 As a member of the task group, the laboratory repre-
than is expected from the test method in its final form to allow
sentative must be familiar enough with the analytical tech-
for greater flexibility in statistical review);
niques used in the method to be able to understand the
8.3.3.2 Explain how to complete the report forms;
significance of the test statistics and render considered judg-
mentonhowwellthemethod’sperformancemeetstheoriginal 8.3.3.3 Emphasize the importance of keeping written obser-
vations that might reveal the cause of unexpected results;
analytical requirements.
8.3.3.4 Emphasize the necessity for immediate communica-
8.3 Test Protocol—Preparation of the test protocol is the
tion with the coordinator when a problem is encountered; and
responsibility of the coordinator. The protocol gives instruc-
8.3.3.5 Ask for information that might prove useful in the
tions to the participating laboratories such as the following:
taskgroup’sevaluationofthetestdata,suchasadescriptionof
8.3.1 Test Pattern—Practice E691 requires estimates of the
test equipment, which is required for the research report.
performance of a method under two extreme conditions of
variability, minimum variability, and variability among differ- 8.4 Report Forms—Provide official report forms to each
ent laboratories. Minimum variability requires that replicate laboratory. Data forms should be convenient to complete and
results be obtained with as little elapsed time as possible. For simpletousewhentranscribingthedataforstatisticalanalysis.
E1601 − 19
Provide spaces for the laboratory to identify itself and the date after testing is completed, must be examined carefully to
the test was performed. It is strongly suggested that these ensure that it does not make or imply a change in the technical
report forms be in electronic format (see comments in 8.1.7). substanceofthemethodnorthatsuchachangecanbeinferred
from the edited wording.
9. Evaluating Data
9.4 The coordinator performs a final statistical analysis
9.1 The task group must ensure that data are handled
usingthedataauthorizedbythetaskgroupinthepreviousstep
properly both in the laboratory and during statistical analysis.
and prepares the research report and the precision and bias
Laboratoryrepresentativesshouldbecautionedagainstsubmit-
section of the method. If the method meets the original project
ting “selected” data. For example, a laboratory might be
requirements, the task group authorizes its chairman to submit
temptedtotakeextrareadingsandsubmitonlythosethatagree
the method to the technical subcommittee chairman for final
well with each other. Such practices or other deviations from
editorial review and subcommittee ballot. If the task group
thetestprotocolmustnotbetoleratedbecausetheydestroythe
decides that the method does not meet the requirements, it
integrity of the test design and make correct interpretation of
should examine the test data (with the help of someone who is
the test results impossible. No result may be rejected just
both adept at using statistics and experienced in analytical
because it does not look good or exceeds a statistical rejection
chemistry) in order to change the method to improve its
limit. Results may be rejected only when an assignable cause
performance.Proposedchangestothemethodshouldbetested
has been documented. Assignable cause is evidence that the
by a small group of laboratories before attempting a full-scale
method was not performed as written or that standard labora-
retest. Because such changes affect the technical substance of
tory practice was not followed. This may involve human error
the method, the revised method must undergo another ILS.
orequipmentmalfunction,orboth.Inthisevent,thelaboratory
shouldcorrecttheproblemand,ifpossible,rerunthetestorthe
10. Calculation
portionofthetestaffectedbyit.However,laboratorypersonnel
must not make changes in the method. Problems that are 10.1 The ILS test program measures the variability of the
perceived as stemming from the method must be discussed test method in typical laboratories. The between-laboratory
with the coordinator. Any unauthorized deviation from the standard deviation, s , and reproducibility index, R, are calcu-
R
latedforthispurpose.Ifthecalculatedvaluesofthesestatistics
written method, no matter how trivial it may seem to the
analyst, may render the laboratory’s results unusable. are to reflect the expected future performance of the method,
the test data should not contain extraneous results.The h and k
9.1.1 Intheeventthatalaboratoryisunwillingtorespondto
the task group’s request for additional information on how statistics are provided to aid the task group in its search for
extraneous data, but the task group is cautioned that statistics
questionable data was obtained, the task group may elect to
discard all results from that laboratory. If the task group takes alone cannot provide sufficient cause for excluding data. For
therelativelysmalldatasetproducedinatypicalILSusingthis
thisapproach,thereasonsmustbeclearlystatedintheresearch
report. practice, a result is truly extraneous only if it is caused by
errors in chemical manipulations, improper operation of
9.2 When test data are received from a laboratory, the
equipment, or failure to follow generally accepted procedures
coordinator immediately reviews it for consistency and adher-
orspecificinstructionsofthemethod.Thetaskgroupmustuse
ence to the test protocol.
principles of chemistry and physics as well as its analytical
9.2.1 The coordinator discusses questionable values with
experience to show that flagged data are inconsistent with
the laboratory representative and clarifies the reasons for rerun
reasonable interpretation and execution of the instructions
data (if any). He transfers the original data to test material
provided in the method and test protocol. Failing that, the task
tables,markinganyvaluesthatwerequestionedorwarranteda
group must retain the data.
rerun and recording substitute values (if any) as footnotes.The
reasons for proposed deletions or substitutions are
10.2 The equations are arranged for manual calculation of
documented, observations on the method reported by the
the statistics, but the coordinator is encouraged to use a
laboratories are summarized, and a preliminary statistical
computer version to save time and avoid errors. A separate
evaluation to flag inconsistent data by the h and k statistics is
statistical analysis is performed for each test material.
performed.The coordinator questions laboratories that submit-
10.3 The data for an ILS run according to Test Plan A are
ted flagged data to see if assignable causes can be found.
shown in Table 1. Each column represents a test material with
9.3 When all data have been received and the tables and
each laboratory’s replicate results in rows.
comments have been assembled, the coordinator presents this
10.4 Test Plan A Calculations—The results of the statistical
information to the task group. The task group must decide
calculationsonthedatainTable1aredisplayedinTable2.(In
whether or not the evidence supplied by the contributing
these equations, x represents the replicate results reported by a
laboratory supports rejecting questionable data. When rerun
laboratory, n equals the number of replicate results per
data are presented, it should also consider whether or not the
laboratory, and p equals the number of laboratories which
integrity of the test is jeopardized by substitution of the rerun
provided the data used for this material.)
data for the rejected data. If a misunderstanding of the method
contributed to a problem, the task group may wish to edit the 10.4.1 For each laboratory, calculate the mean (x-bar),
language of the method to ensure that it will not continue to standarddeviation (s),andthesquareofthestandarddeviation
troublefutureusers.Aneditorialchangetoamethod,proposed (s ):
E1601 − 19
TABLE 1 Nickel ILS Data (% Nickel)
10.4.8 Calculate the reproducibility index and percent rela-
Test Materials tive reproducibility index:
Laboratory
Number
AB C D E
R 52.8 s ; and R 5100R/x%
~ !
R rel
1 0.0053 0.053 0.122 0.217 1.08
NOTE 3—The factor S is equivalent to factor S from Practice E691
M r
0.0053 0.052 0.120 0.215 1.07
because the data in both methods are obtained under repeatability
0.0054 0.053 0.120 0.215 1.07
conditions. This equivalency applies to test plan A only.
2 0.0057 0.052 0.124 0.207 1.07
NOTE 4—The factor of 2.8 (2*sqrt 2) used to calculate R in 10.4.8 and
0.0077 0.054 0.124 0.204 1.06
r in 10.6.12 conforms to the calculations for R and r found in Practice
0.0059 0.053 0.119 0.195 1.05
E691, 21.1, and originates in Practice E177.
3 0.0060 0.053 0.120 0.221 1.08
0.0057 0.055 0.113 0.213 1.05
10.4.9 For each laboratory, calculate its between-laboratory
0.0060 0.053 0.119 0.220 1.07
consistency statistic:
4 0.0058 0.057 0.121 0.219 1.06
0.0053 0.056 0.123 0.225 1.08
h 5 d/s
x¯
0.0065 0.058 0.130 0.230 1.14
5 0.0058 0.054 0.125 0.220 1.06
10.4.10 For each laboratory, calculate its within-laboratory
0.0050 0.054 0.123 0.220 1.06
0.0057 0.053 0.126 0.219 1.08 consistency statistic:
6 0.0060 0.054 0.120 0.215 1.05
k 5 s/s
0.0059 0.054 0.115 0.215 1.05 M
0.0060 0.054 0.120 0.210 1.05
10.5 Test Plan B Calculations—Data for a single material
7 0.0055 0.056 0.120 0.221 1.05
obtained as directed in Test Plan B are shown in Table 3.Itis
0.0060 0.057 0.125 0.221 1.07
0.0050 0.057 0.125 0.215 1.05
arranged like Table 1, except that space is provided for
8 0.0069 0.058 0.118 0.218 1.07
duplicate results on each replicate portion analyzed by a
0.0069 0.058 0.121 0.216 1.06
laboratory. Other test materials in the iron method test are not
0.0063 0.057 0.118 0.217 1.08
9 0.0066 0.056 0.117 0.213 1.10
shown.Theresultsofthestatisticalcalculationsstartinthelast
0.0060 0.057 0.130 0.220 1.05
two columns of Table 3 and continue in Table 4. For a test
0.0062 0.054 0.123 0.225 1.05
including data for day-to-day within-laboratory variability
10 0.0058 0.055 0.122 0.221 1.08
0.0056 0.053 0.124 0.223 1.06
(replicates analyzed in duplicate on different days in the same
0.0055 0.055 0.120 0.220 1.08
laboratory), proceed as directed in 10.6. For a test including
11 0.0049 0.055 0.127 0.220 1.03
data for material variability (replicates are separate portions
0.0043 0.057 0.132 0.216 1.06
0.0053 0.054 0.125 0.214 1.05
analyzed on the one day), proceed as directed in 10.7.
NOTE 5—In the following equations, x and x represent the duplicate
1 2
results from one replicate in one laboratory, X represents their mean, n
equalsthenumberofreplicatesperlaboratory,andpequalsthenumberof
laboratories providing data used in the calculations for one material.
xH5 s X/nd;
o
10.6 Test Plan B—Day-to-Day Variability (see Note 5)—
Thereplicatesareportionsofthetestmaterialthatareanalyzed
s5 sX2 xHd /sn2 1d;
œo
in duplicate on each of several days in each laboratory (see
8.3.2).
and s
10.6.1 For each test portion, calculate the mean of the
=
10.4.2 Calculate the overall mean result (x)for the material: duplicate results, their difference, and the square of the
difference:
x% 5 ~ x¯!/p
(
X 5 ~x 1x !/2
1 2
10.4.3 For each laboratory, calculate its laboratory differ-
D 5 x 2 x ; and D
1 2
ence (d) and the square of the difference (d ):
10.6.2 Calculatethemethod’sminimumstandarddeviation:
d 5 x¯ 2 x%; and d
=
s 5 D /2pn
M (
10.4.4 Calculate the standard deviation of laboratory differ-
ences:
10.6.3 For each laboratory, calculate the laboratory mean,
thestandarddeviationofthereplicatemeans,andthesquareof
s 5 = d / p 21
~ ! ~ !
x¯ (
the standard deviation:
10.4.5 Calculatethemethod’sminimumstandarddeviation:
xH5 s X/nd;
o
s 5 = s /p
~ !
M
(
s5 X2 xH / n2 1 ;
s d s d
œo
10.4.6 Calculate a trial value for the reproducibility stan-
dard deviation:
and s
2 2
s 5 s 1 s n 21 /n
= @~ ! ~ ! # 10.6.4 Calculate the overall mean result for the material:
t ~ x¯! M
10.4.7 Select the final value for the reproducibility standard x% 5 x¯/p
(
deviation:
10.6.5 For each laboratory, calculate its laboratory differ-
s 5thelargerofs ors ence and the square of the difference:
R t M
E1601 − 19
TABLE 2 Statistical Calculations for Nickel Material E (NBS 82a, 1.07 % Nickel)
Test Results, x
Laboratory
2 2
x-bar s d s d hk
Number
12 3
1 1.08 1.07 1.07 1.0733 0.0058 0.0076 0.00003329 0.00005746 0.59 0.32
2 1.07 1.06 1.05 1.0600 0.0100 −0.0058 0.00010000 0.00003318 −0.45 0.55
3 1.08 1.05 1.07 1.0667 0.0153 0.0009 0.00023348 0.00000083 0.07 0.84
4 1.06 1.08 1.14 1.0933 0.0416 −0.0276 0.00173306 0.00076066 2.16 2.28
5 1.06 1.06 1.08 1.0667 0.0116 0.0009 0.00013340 0.00000083 0.07 0.63
6 1.05 1.05 1.05 1.0500 0.0000 −0.0158 0.00000000 0.00024838 −1.24 0.00
7 1.05 1.07 1.05 1.0567 0.0116 −0.0091 0.00013340 0.00008263 −0.71 0.63
8 1.07 1.06 1.08 1.0700 0.0100 0.0042 0.00010000 0.00001798 0.33 0.55
9 1.10 1.05 1.05 1.0667 0.0289 0.0009 0.00083348 0.00000083 0.07 1.58
10 1.08 1.06 1.08 1.0733 0.0116 0.0076 0.00013340 0.00005625 0.59 0.63
11 1.03 1.06 1.05 1.0467 0.0153 −0.0191 0.00023348 0.00036443 −1.50 0.84
^(s ) = 0.00366699
X51.0658
n =3, p =11 ^(d ) = 0.00162346
s 5 0.00162346/1050.01274; s 5 0.00366699/1150.01826;
x¯ œ M œ
s5 0.0001623461s0.000333363ds2/3d50.01961; s 50.01961;
t œ R
R = (2.8)(0.01961) = 0.0594; R = (100)(0.0594) ⁄1.0658 = 5.15 %.
rel
ILS Statistics Summary:
Material Mean Test Result: = 1.066
Minimum Standard Deviation of the Method: s = 0.0183
M
Reproducibility Standard Deviation: s = 0.0196
R
Reproducibility Index: R = 0.0549; R =5.15%
rel
TABLE 3 Iron Material 1A Data, µg/g Iron
2 2
s 5 s 1 s
Œ
Test Results t1 X M
Laboratory Replicate
A 2 2
Replicate D D
Number Mean, X
x x
1 2
10.6.9 Select the final value for the repeatability standard
11 348 345 346.5 3 9
2 343 339 341.0 4 16 deviation:
3 332 327 329.5 5 25
s 5thelargerofs ors
21 347 356 351.5 9 81
r t1 M
2 333 340 336.5 7 49
NOTE 6—The factor S of test plan B and S of Practice E691 are not
r r
3 363 357 360.0 6 36
equivalent factors because data obtained using test plan B are not
31 325 317 321.0 8 64
determined under repeatability conditions.
2 313 310 311.5 3 9
3 330 320 325.0 10 100
10.6.10 Calculate the reproducibility standard deviation:
41 326 322 324.0 4 16
2 322 329 325.5 7 49
n 21 1
2 2 2
3 325 337 331.0 12 144
s 5Œs 1 s 1 s
S D
t2 x¯ x M
n 2
51 338 336 337.0 2 4
2 335 331 333.0 4 16
10.6.11 Select the final value for the reproducibility stan-
3 325 343 334.0 18 324
61 339 335 337.0 4 16
dard deviation:
2 333 335 334.0 2 4
s 5thelargerofs ors
3 338 340 339.0 2 4
R t2 r
71 356 346 351.0 10 100
2 336 331 333.5 5 25 10.6.12 Calculatetherepeatabilityindex,thereproducibility
3 343 346 344.5 3 9
index and percent relative reproducibility index:
n =3, p =7 ^(D ) = 1100
s 5 1100/ 2 3 7 55.118
s ds ds d r 52.8 s ; R 52.8 s ; and R 5100R/x%
M œ ~ ! ~ !
r R rel
A
10.6.13 For each laboratory, calculate its between-
A The difference between duplicate test results is D.
laboratory consistency statistic:
h 5 d/s
x¯
2 10.6.14 For each laboratory, calculate its within-laboratory
d 5 x¯ 2 x% ; and d
consistency statistic:
10.6.6 Calculate the pooled standard deviation of the repli-
k 5 s/s
X
cate means and its square:
10.7 Test Plan B—Material Variability (see Note 5)—
2 2
s 5 = s /p; ands
Separate replicate portions of a test material are analyzed in
x ( x
duplicate on one day in each laboratory (see 8.3.1)
10.6.7 Calculate the standard deviation of the laboratory
10.7.1 For each replicate, calculate the mean of the dupli-
means and its square:
cate results, their difference, and the square of the difference:
2 2
=
s 5 d / p 21 ; and s
~ !
x¯ ( x¯
X 5 x 1x /2
~ !
1 2
10.6.8 Calculate the repeatability standard deviation: D 5 x 2 x ; and D
1 2
E1601 − 19
TABLE 4 Statistical Calculations for Iron Material 1A
Replicate Means, X
Laboratory Laboratory
2 2
sd s d hk
Number mean, x¯
12 3
1 346.5 341.0 329.5 339.00 8.675 3.476 75.255625 12.082576 0.35 1.20
2 351.5 336.5 360.0 349.33 11.899 13.810 141.586201 190.716100 1.38 1.64
3 321.0 311.5 325.0 319.17 6.934 −16.357 48.080356 267.551449 −1.63 0.96
4 324.0 325.5 331.0 326.83 3.686 −8.690 13.586596 75.516100 −0.87 0.51
5 337.0 333.0 334.0 334.67 2.082 −0.857 4.334724 0.734449 −0.09 0.29
6 337.0 334.0 339.0 336.67 2.517 1.143 6.335289 1.306449 0.11 0.35
7 351.0 333.5 344.5 343.00 8.846 7.476 78.251716 55.890576 0.75 1.22
2 2
n =3, p =7 ^(s ) = 367.430507 ^(d ) = 603.797699
X5335.5238
2 2 2
s = 26.190476 (from Table 3); s = 367.430507 ⁄7 = 52.490072; s = 603.797699 ⁄6 = 100.632950;
M X x¯
s = 5.118; Proceed to either (1)or(2) (but not both), depending on the provisions of the test protocol:
M
(1) Statistics for Day-to-Day ILS:
2 2 2
s 5s 1 s 552.490072126.190476/2565.58531
r X M
s = 8.098
r
n21 1
2 2 2 2
s 5s 1 s 1 s
R x X M
n 2
2 1
5100.6329501 52.4900721 26.190476
3 2
= 148.721569
S = 12.195
R
r = 2.8 × 8.098 = 22.67; R = 2.8 × 12.195 = 34.15
R = 100 × 34.15 ⁄335.52 = 10.18 %
rel
(2) Statistics for ILS to Eliminate Material Variability Effect:
1 1
2 2 2
s 5s 2 s 552.4900712 26.190476539.394834
H X M
2 2
1 1
2 2 2 2
s 5s 2 s 1 s
t x¯ X M
n 2
1 1
5100.6329502 52.4900721 26.190476596.231497
3 2
s 5 s 59.810; R52.839.810527.47
R œ t
R = 100 × 27.47 ⁄335.52 = 8.19 %
rel
2 2
s 12s
M H
F 5 5s26.19047612339.394834d/26.190476
H 2
s
M
= 4.01, with
f =2×7=14 and f =3×7=21 degrees of freedom
1 2
10.7.2 Calculatethemethod’sminimumstandarddeviation: 10.7.7 Calculate the standard deviation of the laboratory
differences and its square:
s 5 = D /2np
M (
d
(
10.7.3 For each laboratory, calculate the laboratory mean, Œ
s ; and s
x¯ x¯
p 21
thestandarddeviationofthereplicatemeans,andthesquareof
the standard deviation:
10.7.8 Calculate the variance of the material homogeneity
effect:
xH5 s X/nd;
o
2 2 2
s 5 s 2 s
H X M
s5 X2 xH / n2 1 ;
s d s d
œo
2 2
if s 2 s isnegativeorzero,
S D
X M
and s
sets 50
H
10.7.4 Calculate the overall mean result for the material:
10.7.9 Calculate the reproducibility standard deviation:
x% 5 x¯/p
(
2 2 2
s 5 s 2 s 1s
Œ
10.7.5 For each laboratory, calculate its laboratory differ- t3 x¯ X M
n
ence and the square of the difference:
10.7.10 Select the final value for the reproducibility stan-
d 5 x¯ 2 x%; and d
dard deviation:
10.7.6 Calculate the pooled standard deviation of the repli-
s 5thelargerof s ors
R t3 M
cate means and its square:
10.7.11 Calculate the reproducibility index and percent
2 2
= relative reproducibility index:
s 5 s /p; and s
x ( x
E1601 − 19
A A
TABLE 5 Nickel—h Statistic TABLE 6 Nickel—k Statistic
(Between-laboratory consistency statistic.) (Within-laboratory consistency statistic.)
Test Material Test Material
Laboratory Laboratory
Number Number
AB C D E AB C D E
1 −0.90 −1.31 −0.47 −0.22 0.59 1 0.12 0.59 0.34 0.30 0.32
A,B AB
2 1.17 −1.11 0.06 x−2.58x −0.45 2 x2.29x 1.02 0.85 1.64 0.55
3 0.17 −0.72 −1.53 0.18 0.07 3 0.36 1.17 1.11 1.15 0.84
A A,B
4 0.10 1.25 0.80 1.33 2.16 4 1.25 1.02 1.39 1.45 x2.28x
5 −0.59 −0.72 0.80 0.47 0.07 5 0.91 0.59 0.45 0.15 0.63
6 0.29 −0.52 −1.21 −0.63 −1.24 6 0.12 0 0.85 0.76 0
7 −0.59 1.05 0.37 0.35 −0.71 7 1.04 0.59 0.85 0.91 0.63
8 1.67 1.64 −1.00 0.01 0.33 8 0.72 0.59 0.51 0.26 0.55
9 0.85 0.46 0.37 0.41 0.07 9 0.64 1.55 1.91 1.58 1.58
10 −0.34 −0.32 −0.05 0.75 0.59 10 0.32 1.17 0.59 0.40 0.63
11 −1.84 0.27 1.85 −0.05 −1.50 11 1.05 1.55 1.06 0.80 0.84
CV ±2.34 ±2.34 ±2.34 ±2.34 ±2.34 CV 2.13 2.13 2.13 2.13 2.13
A A
Values exceed approximately 87 % of CV. Values exceed approximately 87 % of CV.
B B
Values flagged with x___x exceed CV. Values flagged with x___x exceed CV.
TABLE 7 Critical Values of h and k at the
R 52.8~s !; R 5100R/x%5
R rel
0.5 % Significance Level
10.7.12 For each laboratory, calculate its between-
Critical Values of k
Critical
A
laboratory consistency statistic: Number of Replicates, n
Value p
of h
2 3 4 567 8 9 10
h 5 d/s
x¯
1.15 3 1.72 1.67 1.61 1.56 1.52 1.49 1.47 1.44 1.42
1.49 4 1.95 1.82 1.73 1.66 1.60 1.56 1.53 1.50 1.47
10.7.13 For each laboratory, calculate its within-laboratory
1.74 5 2.11 1.92 1.79 1.71 1.65 1.60 1.56 1.53 1.50
consistency statistic:
1.92 6 2.22 1.98 1.84 1.75 1.68 1.63 1.59 1.55 1.52
2.05 7 2.30 2.03 1.87 1.77 1.70 1.65 1.60 1.57 1.54
k 5 s/s
X
2.15 8 2.36 2.06 1.90 1.79 1.72 1.66 1.62 1.58 1.55
2.23 9 2.41 2.09 1.92 1.81 1.73 1.67 1.62 1.59 1.56
10.7.14 Optional (see Note 7)—Calculate the material ho-
2.29 10 2.45 2.11 1.93 1.82 1.74 1.68 1.63 1.59 1.56
mogeneity F-statistic and its numerator (f ) and denominator
2.34 11 2.49 2.13 1.94 1.83 1.75 1.69 1.64 1.60 1.57
(f ) degrees of freedom:
2.38 12 2.51 2.14 1.96 1.84 1.76 1.69 1.64 1.60 1.57
2.41 13 2.54 2.15 1.96 1.84 1.76 1.70 1.65 1.61 1.58
2 2 2
F 5 s 12s /s
~ !
H M H M 2.44 14 2.56 2.16 1.97 1.85 1.77 1.70 1.65 1.61 1.58
f 5 p n 21 2.47 15 2.57 2.17 1.98 1.86 1.77 1.71 1.66 1.62 1.58
~ !
2.49 16 2.59 2.18 1.98 1.86 1.77 1.71 1.66 1.62 1.58
f 5 pn
2.51 17 2.60 2.19 1.99 1.86 1.78 1.71 1.66 1.62 1.59
NOTE 7—Those adept at statistics may wish to calculate the homoge-
2.53 18 2.61 2.20 1.99 1.87 1.78 1.72 1.66 1.62 1.59
neity F-statistic to test the hypothesis that the test material is sufficiently
2.54 19 2.62 2.20 2.00 1.87 1.78 1.72 1.67 1.62 1.59
homogeneous.
2.56 20 2.63 2.21 2.00 1.87 1.79 1.72 1.67 1.63 1.59
2.57 21 2.64 2.21 2.00 1.88 1.79 1.72 1.67 1.63 1.59
11. Using Statistics in Task Group Decisions
2.58 22 2.65 2.21 2.01 1.88 1.79 1.72 1.67 1.63 1.59
2.59 23 2.66 2.22 2.01 1.88 1.79 1.72 1.67 1.63 1.59
11.1 Preliminary Screening of Test Data for Consistency—
2.60 24 2.66 2.22 2.01 1.88 1.79 1.73 1.67 1.63 1.60
Most outright mistakes (of the types where equipment fails 2.61 25 2.67 2.23 2.01 1.88 1.79 1.73 1.67 1.63 1.60
2.62 26 2.67 2.23 2.02 1.89 1.80 1.73 1.68 1.63 1.60
during the test, a wrong reagent is used, or a test solution is
2.62 27 2.68 2.23 2.02 1.89 1.80 1.73 1.68 1.63 1.60
spilled) are caught immediately in the laboratory and are
2.63 28 2.68 2.23 2.02 1.89 1.80 1.73 1.68 1.63 1.60
corrected before the test data are submitted. In the same 2.64 29 2.6
...