ASTM D6299-00

NOTICE: This standard has either been superseded and replaced by a new version or discontinued.
Contact ASTM International (www.astm.org) for the latest information.
Designation: D 6299 – 00 An American National Standard
Standard Practice for
Applying Statistical Quality Assurance Techniques to
Evaluate Analytical Measurement System Performance
This standard is issued under the fixed designation D 6299; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (e) indicates an editorial change since the last revision or reapproval.
1. Scope D 5191 Test Method for Vapor Pressure of Petroleum Prod-
ucts (Mini Method)
1.1 This practice provides information for the design and
E 177 Practice for Use of the Terms Precision and Bias in
operation of a program to monitor and control on-going
ASTM Test Methods
stability and precision and bias performance of selected ana-
E 178 Practice for Dealing with Outlying Observations
lytical measurement systems using a collection of generally
E 456 Terminology Relating to Quality and Statistics
accepted statistical quality control (SQC) procedures and tools.
E 691 Practice for Conducting an Interlaboratory Study to
NOTE 1—A complete list of criteria for selecting measurement systems 4
Determine the Precision of a Test Method
to which this practice should be applied and for determining the frequency
at which it should be applied is beyond the scope of this practice.
3. Terminology
However, some factors to be considered include (1) frequency of use of
3.1 Definitions:
the analytical measurement system, (2) criticality of the parameter being
3.1.1 accepted reference value, n—a value that serves as an
measured, (3) system stability and precision performance based on
historical data, (4) business economics, and (5) regulatory, contractual, or agreed-upon reference for comparison and that is derived as (1)
test method requirements.
a theoretical or established value, based on scientific principles,
(2) an assigned value, based on experimental work of some
1.2 This practice is applicable to stable analytical measure-
national or international organization, such as the U.S. Na-
ment systems that produce results on a continuous numerical
tional Institute of Standards and Technology (NIST), or (3)a
scale.
consensus value, based on collaborative experimental work
1.3 This practice is applicable to laboratory test methods.
under the auspices of a scientific or engineering group.
1.4 This practice is applicable to validated process stream
(E 456/E 177)
analyzers.
3.1.2 accuracy, n—the closeness of agreement between an
NOTE 2—For validation of univariate process stream analyzers, see also
observed value and an accepted reference value. (E 456,
Practice D 3764.
E 177)
1.5 This practice assumes that the normal (Gaussian) model
3.1.3 assignable cause, n—a factor that contributes to
is adequate for the description and prediction of measurement
variation and that is feasible to detect and identify. (E 456)
system behavior when it is in a state of statistical control.
3.1.4 bias, n—a systematic error that contributes to the
difference between a population mean of the measurements or
NOTE 3—For non-Gaussian processes, transformations of test results
may permit proper application of these tools. Consult a statistician for test results and an accepted reference or true value. (E 456,
further guidance and information.
E 177)
3.1.5 control limits, n—limits on a control chart that are
1.6 This practice does not address statistical techniques for
used as criteria for signaling the need for action or for judging
comparing two or more analytical measurement systems ap-
whether a set of data does or does not indicate a state of
plying different analytical techniques or equipment compo-
statistical control. (E 456)
nents that purport to measure the same property(s).
3.1.6 lot, n—a definite quantity of a product or material
2. Referenced Documents
accumulated under conditions that are considered uniform for
sampling purposes. (E 456)
2.1 ASTM Standards:
3.1.7 precision, n—the closeness of agreement between test
D 3764 Practice for Validation of Process Stream Analyz-
results obtained under prescribed conditions. (E 456)
ers
3.1.8 repeatability conditions, n—conditions where mutu-
ally independent test results are obtained with the same test
This practice is under the jurisdiction of ASTM Committee D02 on Petroleum
method in the same laboratory by the same operator with the
Products and Lubricants and is the direct responsibility of Subcommittee CS-94 on
Quality Assurance and Statistics .
Current edition approved April 10, 2000. Published July 2000. Originally
published as D 6299 – 98. Last previous edition D 6299 – 99. Annual Book of ASTM Standards, Vol 05.03.
2 4
Annual Book of ASTM Standards, Vol 05.02. Annual Book of ASTM Standards, Vol 14.02.
Copyright © ASTM, 100 Barr Harbor Drive, West Conshohocken, PA 19428-2959, United States.
D 6299
same equipment within short intervals of time, using test material having physical or chemical properties, or both,
specimens taken at random from a single sample of material. similar to those of typical samples tested by the analytical
(E 456, E 177) measurement system. The material is properly stored to ensure
sample integrity, and is available in sufficient quantity for
3.1.9 reproducibility conditions, n—conditions under which
repeated, long term testing.
test results are obtained in different laboratories with the same
3.2.10 site precision conditions, n—conditions under which
test method, using test specimens taken at random from the
same sample of material. (E 456, E 177) test results are obtained by one or more operators in a single
site location practicing the same test method on a single
3.2 Definitions of Terms Specific to This Standard:
measurement system using test specimens taken at random
3.2.1 analytical measurement system, n—a collection of one
from the same sample of material, over an extended period of
or more components or subsystems, such as samplers, test
time spanning at least a 15 day interval.
equipment, instrumentation, display devices, data handlers,
3.2.10.1 Discussion—An analytical measurement system
printouts or output transmitters, that is used to determine a
may comprise multiple instruments being used for the same
quantitative value of a specific property for an unknown
test method.
sample.
3.2.10.2 Discussion—Site precision conditions should in-
3.2.1.1 Discussion—An analytical measurement system
clude all sources of variation that are typically encountered
may comprise multiple instruments being used for the same
during normal, long term operation of the measurement sys-
test method.
tem. Thus, all operators who are involved in the routine use of
3.2.2 blind submission, n—submission of a check standard
the measurement system should contribute results to the site
or quality control (QC) sample for analysis without revealing
precision determination. If multiple results are obtained within
the expected value to the person performing the analysis.
a 24–h period, then it is recommended that the number of
3.2.3 check standard, n—in QC testing, a material having
results used in site precision calculations be increased to
an accepted reference value used to determine the accuracy of
capture the longer term variation in the system.
a measurement system.
3.2.11 site precision standard deviation, n—the standard
3.2.3.1 Discussion—A check standard is preferably a mate-
deviation of results obtained under site precision conditions.
rial that is either a certified reference material with traceability
3.2.12 validation audit sample, n—a QC sample or check
to a nationally recognized body or a material that has an
standard used to verify precision and bias estimated from
accepted reference value established through interlaboratory
routine quality assurance testing.
testing. For some measurement systems, a pure, single com-
3.3 Symbols:
ponent material having known value or a simple gravimetric or
3.3.1 ARV—accepted reference value.
volumetric mixture of pure components having calculable
3.3.2 EWMA—exponentially weighted moving average.
value may serve as a check standard. Users should be aware
3.3.3 I—individual observation (as in I-chart).
that for measurement systems that show matrix dependencies,
3.3.4 MR—moving range.
accuracy determined from pure compounds or simple mixtures
3.3.5 MR —average of moving range.
may not be representative of that achieved on actual samples.
3.3.6 QC—quality control.
3.2.4 common (chance, random) cause, n—for quality as-
3.3.7 R8—site precision.
surance programs, one of generally numerous factors, individu-
3.3.8 s —site precision standard deviation.
ally of relatively small importance, that contributes to varia-
R8
3.3.9 VA—validation audit.
tion, and that is not feasible to detect and identify.
3.3.10 x —chi squared.
3.2.5 double blind submission, n—submission of a check
standard or QC sample for analysis without revealing the check 3.3.11 l—lambda.
standard or QC sample status and expected value to the person
4. Summary of Practice
performing the analysis.
3.2.6 expected value, n—for a QC sample analyzed using an
4.1 QC samples and check standards are regularly analyzed
in-statistical control measurement system, the estimate of the
by the measurement system. Control charts and other statistical
theoretical limiting value to which the average of results tends
techniques are presented to screen, plot, and interpret test
when the number of results approaches infinity.
results in accordance with industry-accepted practices to as-
3.2.7 in-statistical-control, adj—a process, analytical mea-
certain the in-statistical-control status of the measurement
surement system, or function that exhibits variations that can
system.
only be attributable to common cause.
4.2 Statistical estimates of the measurement system preci-
3.2.8 proficiency testing, n—determination of a laboratory’s
sion and bias are calculated and periodically updated using
testing capability by participation in an interlaboratory cross-
accrued data.
check program.
4.3 In addition, as part of a separate validation audit
3.2.8.1 Discussion—ASTM Committee D02 conducts pro-
procedure, QC samples and check standards may be submitted
ficiency testing among hundreds of laboratories, using a wide blind or double-blind and randomly to the measurement system
variety of petroleum products and lubricants.
for routine testing to verify that the calculated precision and
3.2.9 quality control (QC) sample, n—for use in quality bias are representative of routine measurement system perfor-
assurance programs to determine and monitor the precision and mance when there is no priori knowledge of the expected value
stability of a measurement system, a stable and homogeneous or sample status.
D 6299
5. Significance and Use a material that is analyzed under reproducibility conditions by
multiple measurement systems. The accepted reference value
5.1 This practice can be used to continuously demonstrate
(ARV) for this check standard shall be the average after
the proficiency of analytical measurement systems that are
statistical examination and outlier treatment has been applied.
used for establishing and ensuring the quality of petroleum and
6.2.2.1 Exchange samples circulated as part of an interlabo-
petroleum products.
ratory exchange program, or round robin, may be used as check
5.2 Data accrued, using the techniques included in this
standards. For an exchange sample to be usable as a check
practice, provide the ability to monitor analytical measurement
standard, the standard deviation of the interlaboratory ex-
system precision and bias.
change program shall not be statistically greater than the
5.3 These data are useful for updating test methods as well
reproducibility standard deviation for the test method. An
as for indicating areas of potential measurement system im-
F-test should be applied to test acceptability.
provement.
NOTE 6—The uncertainty in the ARV is inversely proportional to the
6. Reference Materials
square root of the number of values in the average. This practice
6.1 QC samples are used to establish and monitor the
recommends that a minimum of 16 non-outlier results be used in
precision of the analytical measurement system.
calculating the ARV to reduce the uncertainty of the ARV by a factor of
6.1.1 Select a stable and homogeneous material having
4 relative to the measurement system single value precision. The bias tests
described in this practice assume that the uncertainty in the ARV is
physical or chemical properties, or both, similar to those of
negligible relative to the measurement system precision. If less than 16
typical samples tested by the analytical measurement system.
values are used in calculating the average, this assumption may not be
NOTE 4—When the QC sample is to be utilized for monitoring a
valid.
process stream analyzer performance, it is often helpful to supplement the NOTE 7—Examples of exchanges that may be acceptable are ASTM
process analyzer system with a subsystem to automate the extraction, D02.CS92 ILCP program; ASTM D02.01 N.E.G.; ASTM D02.01.A
mixing, storage, and delivery functions associated with the QC sample. Regional Exchanges; International Quality Assurance Exchange Program,
administered by Alberta Research Council.
6.1.2 Estimate the quantity of the material needed for each
6.2.3 For some measurement systems, single, pure compo-
specific lot of QC sample to (1) accommodate the number of
nent materials with known value, or simple gravimetric or
analytical measurement systems for which it is to be used
volumetric mixtures of pure components having calculable
(laboratory test apparatuses as well as process stream analyzer
value may serve as a check standard. For example, pure
systems) and (2) provide determination of QC statistics for a
solvents, such as 2,2-dimethylbutane, are used as check stan-
useful and desirable period of time.
dards for the measurement of Reid vapor pressure by Test
6.1.3 Collect the material into a single container and isolate
Method D 5191. Users should be aware that for measurement
it.
systems that show matrix dependencies, accuracy determined
6.1.4 Thoroughly mix the material to ensure homogeneity.
from pure compounds or simple mixtures may not be repre-
6.1.5 Conduct any testing necessary to ensure that the QC
sample meets the characteristics for its intended use. sentative of that achieved on actual samples.
6.3 Validation audit (VA) samples are QC samples and
6.1.6 Package or store QC samples, or both, as appropriate
for the specific analytical measurement system to ensure that check standards, which may, at
...