ASTM D7778-15(2022)e1

This international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for the
Development of International Standards, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
´1
Designation: D7778 − 15 (2022) An American National Standard
Standard Guide for
Conducting an Interlaboratory Study to Determine the
Precision of a Test Method
This standard is issued under the fixed designation D7778; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
ε NOTE—Editorially corrected 6.2.2 in May 2022.
1. Scope 1.6 This guide represents the suggested practices for a
typical precision study. Individual subcommittees may modify
1.1 This guide describes the procedures for planning and
this approach as their expertise directs.
conductinganinterlaboratorystudy(ILS)ofatestmethodused
1.7 This guide is voluntary, and thus, is not required for an
in Petroleum Products and Lubricants Committee D02 of
approved precision study.
ASTM for the purpose of estimating repeatability and repro-
ducibility of the test method in accordance with ASTM Form
1.8 This international standard was developed in accor-
and Style requirements.
dance with internationally recognized principles on standard-
ization established in the Decision on Principles for the
1.2 ThisguideisconcernedwiththemanagementoftheILS
Development of International Standards, Guides and Recom-
and intended to provide guidance for the planning stage and
mendations issued by the World Trade Organization Technical
ensuretheprocess,logistics,andtoolsareidentifiedandagreed
Barriers to Trade (TBT) Committee.
upon in advance.
1.2.1 Selection of the samples and the impact of sample
2. Referenced Documents
selection on the final scope of the test method—both the range
2.1 ASTM Standards:
ofmaterialscoveredinthescopeandprecisionsectionsandthe
measurement range covered in the precision section—are D130 Test Method for Corrosiveness to Copper from Petro-
leum Products by Copper Strip Test
important, and careful consideration needs to be given to these
aspects before an ILS is launched. D4306 Practice for Aviation Fuel Sample Containers for
Tests Affected by Trace Contamination
1.3 This guide does not concern itself with the development
D5854 Practice for Mixing and Handling of Liquid Samples
of test methods but rather with gathering the information
of Petroleum and Petroleum Products
needed for a test method precision statement.
D6300 Practice for Determination of Precision and Bias
1.4 This guide is concerned with test methods which yield a
Data for Use in Test Methods for Petroleum Products,
single numerical figure as the test result, although the single
Liquid Fuels, and Lubricants
figure may be the result of a calculation from a set of
D6708 Practice for StatisticalAssessment and Improvement
measurements.
of Expected Agreement Between Two Test Methods that
Purport to Measure the Same Property of a Material
1.5 This guide is designed for tests of properties that are
D6792 Practice for Quality Management Systems in Petro-
stable, such that bulk samples can be homogenized and
leum Products, Liquid Fuels, and Lubricants Testing
sub-samples can be prepared that will be identical and stable
Laboratories
for the duration of the ILS testing period. This guide is not
E177 Practice for Use of the Terms Precision and Bias in
designed for unstable properties or for properties that make it
ASTM Test Methods
difficult to obtain stable, representative samples for an ILS
E178 Practice for Dealing With Outlying Observations
conducted at multiple laboratory sites.
E456 Terminology Relating to Quality and Statistics
E691 Practice for Conducting an Interlaboratory Study to
Determine the Precision of a Test Method
This test method is under the jurisdiction of ASTM Committee D02 on
Petroleum Products, Liquid Fuels, and Lubricants and is the direct responsibility of
Subcommittee D02.94 on Coordinating Subcommittee on Quality Assurance and
Statistics. For referenced ASTM standards, visit the ASTM website, www.astm.org, or
Current edition approved May 1, 2022. Published June 2022. Originally contact ASTM Customer Service at service@astm.org. For Annual Book of ASTM
approved in 2012. Last previous edition approved in 2015 as D7778 – 15. DOI: Standards volume information, refer to the standard’s Document Summary page on
10.1520/D7778-15R22E01. the ASTM website.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
´1
D7778 − 15 (2022)
E1169 Practice for Conducting Ruggedness Tests 4.1.1 This guide does not address the data analysis tech-
2.2 ISO/IEC Standards: niques. For details of such analysis techniques refer to Practice
ISO/IEC Guide 30 Terms and Definitions Used in Connec- D6300 and E691, or other technically equivalent documents.
tion with Reference Materials 4.1.2 The relevant subcommittee is the sole arbiter of
ISO/IEC Guide 17025 General Requirements for the Com- technical equivalence.
petence of Testing and Calibration Laboratories
5. Significance and Use
3. Terminology
5.1 AllASTM standard test methods are required to include
3.1 Definitions: statements on precision and bias. To obtain such data it is
3.1.1 accuracy, n—closeness of agreement between an in-
necessarytoconductaninterlaboratorystudy.Thisdocumentis
dividual test result and an accepted reference value. E456 designed to provide a brief overview of the steps necessary in
an ILS and to suggest an appropriate sequence in carrying out
3.1.2 bias, n—difference between the population mean of
these steps.
the test results and an accepted reference value. E456
5.1.1 Qualitative tests such as Test Method D130 copper
3.1.3 outlier, n—observation that appears to deviate mark-
corrosion may not require an ILS.
edly from the other observations of the sample (also referred to
as extreme result, outlying or doubtful observation, or aberrant
6. Planning an Interlaboratory Study
value. E178
6.1 Suggested steps in an ILS are given in Table 1.
3.1.4 precision, n—closeness of agreement among test re-
6.2 Support for conducting the ILS is available fromASTM
sults obtained under prescribed conditions. E456
through its Interlaboratory Studies Office (ILS@astm.org).
3.1.5 reference material, n—material or substance, one or
Individuals involved in conducting an ILS are listed below.
more of whose property values are sufficiently homogenous
6.2.1 “Support” does not include financial support for new
and well established to be used for the calibration of an
or alternate equipment or procedure beyond sampling and
apparatus, the assessment of a measurement method, or for
shipping materials to the participants.
assigning values to other materials. E177
6.2.2 A Task Force should be formed consisting at a
3.1.6 repeatability, n—quantitative expression of the ran-
minimum of the method developers, users, and statistician.
dom error associated with a single operator in a given
Based on the objectives of the ILS, this task force should have
laboratoryobtainingrepetitiveresultsbyapplyingthesametest
the overall responsibility for the design, disclosure, and execu-
method with the same apparatus under constant operating
tion of the ILS. The Task Force is charged with the develop-
conditions on identical test material within a short interval of
ment of an ILS plan that includes scope and purpose, funding,
time on the same day. It is defined as the difference between
staffing, participants (equipment structure), sample types, tests
two such results at the 95% confidence level. D6792, E177
required, special instructions, reporting form, and sample
distribution, participant protocol, and a questionnaire and
3.1.7 reproducibility, n—quantitative expression of the ran-
dom error associated with different operators from different statistical tools to be used, and criteria for acceptance of
laboratories, using different apparatus, each obtaining a single results.
resultbyapplyingthesamemethodonanidenticaltestsample. 6.2.3 An ILS Coordinator should be appointed as an overall
It is defined as the 95% confidence limit for the difference person responsible for the distribution of materials and proto-
cols to the laboratories, and receive the test results from the
betweentwosuchsingleandindependentresults. D6792, E177
laboratories. Eventually this person should be responsible for
3.1.8 trueness, n—closeness of agreement between the
all aspects of conducting the ILS, and writing the research
population mean of the measurements or the test results and an
report.
accepted reference value. E456
4. Summary of Guide
TABLE 1 Sequence of an Interlaboratory Study
4.1 The procedure presented in this guide consists of five
Sequence Procedure Section Number in this
basic steps: Guide
(1) Planning an interlaboratory study (including objectives 1 Select ILS membership 6.2.2
2 Prepare Basic Design 6.3
and expected outcome).
3 Specify Test Method(s) 6.4
(2) Disclosure of plan details (participants, equipment,
4 List Participating 6.5
Laboratories
samples, logistics).
5 List Materials 6.6
(3) Approval by membership.
6 Number of Test Results 6.7
(4) Guiding the testing phase of the study and collecting
per Test Material per
test data results. Lab
7 Provide Protocol 6.8
(5) PostvalidationthattheILSwasexecutedinaccordance
8 Ruggedness Study 7 and 8
to the agreed plan.
(Pilot Run)
9 Full Scale Run 9
10 Data Handling 10
Available from Available from Aerospace Industries Association of America,
11 Data Presentation 11
Inc. (AIA), 1000 Wilson Blvd., Suite 1700, Arlington, VA 22209-3928, http:// 12 Prepare Study Report 12
www.aia-aerospace.org.
´1
D7778 − 15 (2022)
6.2.3.1 A running activity record may be useful to be 6.4 Test Method—To establish initial precision statements
maintained by the ILS Coordinator to overview the progress of foranewtestmethod,adraftofthetestmethodisrequiredthat
the ILS activity. has been developed in one or more competent laboratories and
6.2.4 Statistician—The Task Force should obtain the ser- studied in a pilot ILS (see Section 8).This draft method should
vices of a statistician who is familiar with the protocols used in describe the test procedure in terms that the steps can be
ASTM for precision calculations during the planning stage for unequivocally followed in a typical laboratory by competent
input towards the design of the ILS, material selection, personnel with the knowledge of the materials and the property
establishment of the statistical tools to be used, acceptance to be tested. The test conditions that affect the test results
criteria, and testing protocol. This person may be a member of appreciably should be identified and the proper degree of
theTask Force.This person should also assist the task group in control of sample handling and the test conditions be specified
interpreting the results of the data analysis. in the description of the test procedure.The test method should
6.2.4.1 The statistical software available from D02 Com- specify calibration procedure and its frequency and the format
mittee can help in these calculations, but it is highly recom- for reporting the test results.
mendedthatthisworkbedoneby,orundertheoversightofthe 6.4.1 Reevaluation of Existing Precision Statements—To
statistician engaged in 6.2.4. reevaluate existing precision statements for test methods, all
aspects of the existing test method that are not intended to be
6.3 Basic Design—The design should be kept as simple as
revised shall be followed. Intended revisions to the test method
possible in order to obtain estimates of within- and between-
shall be highlighted in the test method and explicitly stated to
laboratory variability that are free of secondary effects. The
the participants.
basic design is represented by a two-way classification table in
6.4.1.1 ILSExecutionProtocol—Theproceduresandspecial
which the rows represent the laboratories, the columns repre-
instructions provided in the ILS protocol for participation shall
sent the materials, and each cell (that is, the intersection of a
not introduce restrictions or controls in the execution of the
row with a column) contains the test results obtained by a
method that are not in the normal execution of the method
particular laboratory on a particular material. See Table 2 for
being evaluated.
illustration where x and y are the test results obtained on the
6.4.1.2 Variables—All variables in the test method that are
same materials by the same operator from each laboratory.
intended to be variables shall be left as variables in the ILS.To
6.3.1 Samples to be tested shall be presented as a single
remove, control, or set limits on factors that are not intended to
series of test samples, requiring only a single analysis per
be controlled in the normal usage of the test method in the
sample. Both the material identification and the corresponding
execution of the ILS will result in the precision being overly
duplicateshallbeblindedsuchthatthetesterdoesnotknowthe
optimistic,andassuchprecisionstatementsthusgeneratedwill
material identification of the test sample, nor its corresponding
likely be unattainable by users in the normal usage of the
duplicate run. An example of a randomized series for the ILS
method.
design in Table 2 is presented in Table 3. If this single
randomized series approach is not possible, then blind repeats 6.5 Laboratories—An ILS should ideally include enough
of the same material(s) should be embedded among the laboratories to produce a reproducibility estimate with at least
samples provided. The blind samples should represent the 30 degrees of freedom. See Practice D6300 for further guid-
materials used in the scope of the test method and concentra- ance. It is important that a reasonable cross section of the
tion range for which the method is applicable.At the least, the population of qualified laboratories be represented in the ILS.
blind duplicates should include a range of easy and difficult Special attention is required to ensure that at least 75 % of the
samples and include the full sample matrix in the sample set. participants represent product manufacturer laboratories or
By including only easy samples in blind testing or the full ILS third party commercial independent testing laboratories, or
for that matter, the resulting precision may demonstrate better both.Take care to ensure that the population is random and not
repeatability than otherwise would be found in practice. biased towards special interest participants such as equipment
6.3.2 The randomness of testing samples is so important vendors. Vendor participation shall be managed carefully so as
that this has to be a requirement, not just a good practice. The not to bias the data population.
actual duplicates run for each test material shall be randomized 6.5.1 The final statement of precision of a test method shall
within the series. not be based on fewer than six laboratories with acceptable test
TABLE 2 Basic Design of an Interlaboratory Study
Materials
Laboratory
AB C D E F G H I
x x x x x xxxx
A
y y y y y yyyy
x x x x x xxxx
B
y y y y y yyyy
x x x x x xxxx
C
y y y y y yyyy
x x x x x xxxx
D
y y y y y yyyy
´1
D7778 − 15 (2022)
TABLE 3 Randomized Test Series
Test 12345678910 11 12 13 14 15 16 17 18
Series
MaterialC F V D BEBAF H A G I EG D H I
results for each material. More than 7 or 8 laboratories should be needed than if it is merely known that the precision is
participate, preferably a minimum of 10 laboratories in case different at different levels. In such cases ruggedness test (see
some data is unacceptable and data from one or more labora- 7.1) or the preliminary pilot program (see Section 8) can help
tories has to be excluded from the data evaluation. settle some of these questions, and may often result in the
6.5.2 Any laboratory considered qualified to run the test saving of considerable time and expense in the full ILS.
routinely should be encouraged to participate in the ILS. In 6.6.3 An ILS of a test method should include at least three
order to obtain an adequate number of participating
materials representing different test levels, and for develop-
laboratories, the ILS study should be publicized through trade ment of a broadly applicable precision statement six or more
magazines (if feasible), meetings, broadcast through pertinent
materials should be included in the study. The number of
ASTM committees, etc. different materials used in the ILS will be dictated by the scope
6.5.3 “Qualified” above implies proper laboratory facilities
of the method.
and testing equipment, competent operators, familiar with the
6.6.3.1 The range of materials in the ILS shall cover the
test method, a reputation for reliable testing work, and suffi-
range of materials identified in the scope of the Test Method.
cient time and interest to perform the task.
Thebroadertherangeofmaterialsidentifiedinthescopeofthe
6.5.3.1 A good gauge of a laboratories competence can be
Test Method, the larger number of samples will be needed in
estimated by its compliance with ISO 17025 or PracticeD6792
the ILS.
laboratory quality management standards.
6.6.3.2 The range in concentration or level of the parameter
6.5.4 If a laboratory is not familiar with the test method, it
being tested should start at the lowest level obtainable or
should do several runs on available materials so all the
reasonably expected in commerce, ranging to a level that
difficulties are worked out before participating in the actual
exceeds the expected specification limit. For example, if the
ILS.
testisforsulfur,andtheexpectedrangecouldbefrom1 mg⁄kg
6.5.4.1 Analysts and laboratories new to the Test Method
uptoalimitof15 mg⁄kg,theILSshouldincludesamplesfrom
should be encouraged to document any aspects of the Test
as close to zero as possible, ranging up to, perhaps, 25 mg⁄kg
Method that are unclear or ambiguous so the wording of the
or 30 mg⁄kg purposely exceeding the limit of 15 mg⁄kg.
Test Method, especially the procedure, can be clarified to
6.6.4 The materials involved in any one ILS should differ
remove ambiguities.
primarily only in the level of property measured by the test
6.5.5 The ILS should not be restricted to a group of
method.When it is known or suspected that different classes of
laboratories judged to be exceptionally qualified and equipped
materials will exhibit different levels of precision when tested
for the ILS. This can lead to extremely precise data which
by the proposed test method, consideration should be given to
cannot be replicated later on when the test method is used in
conducting separate interlaboratory studies for each class of
general laboratories. Precision estimates in a standard test
material. However, in real-world this may be difficult to
method should reflect real-world situation existing at the time
arrange; hence it should be considered to arrange an ILS using
of the ILS.
both similar materials and a collection of different materials.
When an ILS is conducted only on one type of samples, this
6.6 Materials—If possible, different materials having the
should be noted in the precision section that the data was
same property being investigated should be included in the
collected in such a manner and mixing of sample types may
ILS, preferably at different levels of concentrations. Different
affect the individual laboratory’s precision.
dilutions of the same material or compound to be assayed can
6.6.5 Each material in an ILS should be made to be or
be considered as “different materials” for the purposes of this
selected to be as homogenous as possible prior to its subdivi-
guide.
sion into test units or test specimens. Distribution of the test
6.6.1 Thenumberandtypesofmaterialstobeincludedinan
specimen should be done among the laboratories by random-
ILS will depend on a number of considerations including the
ization. Homogeneity must be tested/confirmed on random
range of the levels in the class of materials to be tested and
sub-samples prior to sample shipment by methods that can
likely relation of precision to level over the range, the number
detect non-homogeneity, for example, density or other simple
of different types of materials to which the test method is to be
tests.
applied, the difficulty and expense involved in obtaining,
6.6.6 It may be convenient to use established standard
processing, and distributing samples, the difficulty of, length of
time required for, and expense of performing the test, the referencematerialsintheILSsincetheirhomogeneityhasbeen
demonstrated. It will also help in establishing the bias of the
commercial or legal need for obtaining a reliable and compre-
hensive estimate of precision and the uncertainty on any one of test method after the ILS is completed.
these points. 6.6.7 The materials used in the ILS shall be homogenous
6.6.2 For example, if it is already known that the precision and stable, and not volatilize, degrade, gel, evaporate, form
is either constant or proportional to the average level over the sediment, or otherwise change between dispatch and receipt in
range of values of interest, a smaller number of materials will Practices D4306 and D5854.
´1
D7778 − 15 (2022)
6.6.8 The containers used for shipping shall not react with members. This is then recorded in the subcommittee meeting
or get degraded by the samples. Appropriate containers for minutes.An example of a protocol for an ILS is shown in Fig.
different types of materials are suggested in Practice D5854.
X1.2.
6.7 Number of Test Results per Test Material Samples: 6.8.2 The ILS Task Force Coordinator shall send a letter to
6.7.1 In the design of an ILS a sufficient total number of test
all participating laboratory contacts. (An example of such a
results on each sample shall be specified to obtain a good
letter is given in Fig. X1.1.) This letter should include the
estimate of the measure of repeatability standard deviation. In
name, address, telephone number, and e-mail address of the
many cases, the standard deviation will be a function of the
Coordinator. Laboratories should be encouraged to contact the
property level being measured. Although it is suggested in
Coordinator regarding any questions about the ILS. The letter
Practice E691 to limit the replicates to three for chemical tests
should also include the ILS plan outlining the complete details
and three or four for a physical or optical test, Committee
of the ILS.
D02’s standard protocol for statistical calculations of precision
6.8.3 The ILS Task Force shall determine sample analysis
permits analysis of duplicate samples.
protocol, including sample amount, time and measurement
6.7.2 Generally, the time and effort invested in an ILS is
time.
better spent on examining more samples across more labora-
6.8.4 The specific version of the test method being studied
tories than on recording a large number of test results per
should be clearly identified. If the test method allows several
sample within a few laboratories. The aim is to get a broad
options in apparatus or procedure, the protocol should specify
consensus of the laboratory community rather just a narrow
which option or options have been selected for the ILS. Test
view of a small segment of the industry.
units and test data sheets must be provided for each option.An
6.8 Protocol of an ILS:
example of such a data return form is given in Fig. X1.3.
6.8.1 The ILS protocol begins with the development of a
6.8.5 When special calibration procedures are required be-
plan (see Table 4 for requirements of the protocol plan)
fore every determination or every test result, they should be
developed by the ILS Task Force. The plan requires approval
from the Task Force and notification to the subcommittee described specifically in the test method.
TABLE 4 ILS Protocol Plan
Subcommittee:
ILS Title:
Activity Description Status
A. Design
1. Purpose Describe the purpose of the ILS (precision, analyzer
evaluation, etc.)
2. Funding Describe how the ILS will be funded.
3. Staffing Identify the ILS leader and the team.
4. Participants Identify the participant laboratories in the ILS. Define
participation criteria and balance of participation.
5. Equipment Identify the models and make of analyzer(s)
involved in the ILS.
6. Samples Types Define the product(s) to be analyzed.
7. Tests Define test method(s) for analysis and special
requirements, if any.
8. Special Instructions Define any special instructions for sample handling,
multiple analysis, etc.
9. Reporting Form Develop the reporting form for participants for data
and unusual observations.
10. Statistical Tools Define what statistical tools will be applied to the
data.
11. Criteria for Data Acceptance Define what criteria will be used for rejection or
acceptance of the data.
12. Participants’ Questionnaire All information (
...