ASTM D3244-21a

This document is not an ASTM standard and is intended only to provide the user of an ASTM standard an indication of what changes have been made to the previous version. Because
it may not be technically possible to adequately depict all changes accurately, ASTM recommends that users consult prior editions as appropriate. In all cases only the current version
of the standard as published by ASTM is to be considered the official document.
Designation: D3244 − 21 D3244 − 21a An American National Standard
Standard Practice for
Utilization of Test Data to Determine Conformance with
Specifications
This standard is issued under the fixed designation D3244; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
INTRODUCTION
The properties of commercial petroleum products are measured by standardized laboratory test
methods to assess their conformance to specifications. Two or more measurement results obtained by
performing the same test method for the same property of a specific sample usually will not be
numerically identical. Therefore, the test methods generally include a paragraph on the precision of
results. This precision (or, a more appropriate term is imprecision) is an expression of the degree of
agreement that can be expected between the aforementioned measurements.
Many difficulties that arise in assessing conformance to specifications are due to test imprecision.
Because of this, a true value of a property can never be determined exactly; and it is necessary to infer
from measured values the range within which the “true value” is likely to lie. The main purpose of this
practice is to indicate how test imprecision should be interpreted relative to specification limit values.
1. Scope*
1.1 This practice covers guidelines and statistical methodologies with which two parties (see Note 1) can compare and combine
independently obtained test results to obtain an Assigned Test Value (ATV) for the purpose of resolving a dispute over product
property conformance with specification.
NOTE 1—Application of this practice is usually, but not limited to, between supplier and receiver of a product.
1.2 This practice defines a technique for establishing an Acceptance Limit (AL) and Assigned Test Value (ATV) to resolve the
dispute over a property conformance with specification by comparing the ATV to the AL.
1.3 This practice applies only to those test methods which specifically state that the repeatability and reproducibility values
conform to the definitions herein.
1.4 The statistical principles and methodology outlined in this practice can also be used to obtain an ATV for specification
conformance decision when multiple results are obtained for the same batch of product within a single laboratory. For this
application, site precision (R’) as defined in Practice D6299 shall be used in lieu of test method published reproducibility (R).
1.5 This international standard was developed in accordance with internationally recognized principles on standardization
established in the Decision on Principles for the Development of International Standards, Guides and Recommendations issued
by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
This practice is under the jurisdiction of ASTM Committee D02 on Petroleum Products, Liquid Fuels, and Lubricants and is the direct responsibility of D02.94 on
Coordinating Subcommittee on Quality Assurance and Statistics.
Current edition approved July 1, 2021Oct. 1, 2021. Published July 2021October 2021. Originally published as an appendix to the 1968 Annual Book of ASTM Standards,
Part 18. Originally approved as a standard in 1974. Last previous edition approved in 20202021 as D3244 – 20.D3244 – 21. DOI: 10.1520/D3244-21.10.1520/D3244-21A.
*A Summary of Changes section appears at the end of this standard
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
D3244 − 21a
2. Referenced Documents
2.1 ASTM Standards:
D1319 Test Method for Hydrocarbon Types in Liquid Petroleum Products by Fluorescent Indicator Adsorption
D4057 Practice for Manual Sampling of Petroleum and Petroleum Products
D4177 Practice for Automatic Sampling of Petroleum and Petroleum Products
D6299 Practice for Applying Statistical Quality Assurance and Control Charting Techniques to Evaluate Analytical Measure-
ment System Performance
D6300 Practice for Determination of Precision and Bias Data for Use in Test Methods for Petroleum Products, Liquid Fuels, and
Lubricants
D6792 Practice for Quality Management Systems in Petroleum Products, Liquid Fuels, and Lubricants Testing Laboratories
D7372 Guide for Analysis and Interpretation of Proficiency Test Program Results
E29 Practice for Using Significant Digits in Test Data to Determine Conformance with Specifications
2.2 ISO Standard:
ISO 4259 Determination and Application of Precision Data in Relation to Methods of Test
3. Terminology
3.1 Definitions:
3.1.1 acceptance limit (AL), n—a numerical value that defines the point between making the property conformance or
non-conformance to a specification decision.
3.1.1.1 Discussion—
The AL is not necessarily the specification limit. It is a value that takes into account the specification limit, the test method
precision, and the desired probability of making the conformance to specification decision if the true value (see 3.1.17) of the
property is at the specification limit.
3.1.2 assigned test value (ATV), n—the average of all results obtained in the several laboratories which are considered acceptable
based on the reproducibility of the test method.
3.1.3 determination, n—the process of carrying out the series of operations specified in the test method whereby a single value
is obtained.
3.1.4 dispute, n—when there is a question as to product property conformance to specification because a test value obtained falls
outside the specification limit(s).
3.1.5 dispute adjudication sample, n—a mutually agreed sample between the parties in dispute to be used for the purpose of
arriving at the ATV for the property that is in dispute with regards to its specification conformance status.
3.1.6 operator, n—a person who normally and regularly carries out a particular test.
3.1.7 precision, n—the degree of agreement between two or more test results on the same property obtained using the same test
method on identical test material. In this practice, precision statements are framed in terms of the published repeatability and
reproducibility of the test method.
3.1.8 receiver, n—any individual or organization who receives or accepts the product delivered by the supplier.
3.1.9 receiver’s risk, n—the probability of accepting a product that fails to meet the specification.
3.1.10 repeatability (a.k.a. Repeatability Limit) (r), n—the quantitative expression for the random error associated with the
difference between two independent results obtained under repeatability conditions that would be exceeded with an approximate
probability of 5 % (one case in 20 in the long run) in the normal and correct operation of the test method. D6300
For referenced ASTM standards, visit the ASTM website, www.astm.org, or contact ASTM Customer Service at service@astm.org. For Annual Book of ASTM Standards
volume information, refer to the standard’s Document Summary page on the ASTM website.
Available from American National Standards Institute (ANSI), 25 W. 43rd St., 4th Floor, New York, NY 10036, http://www.ansi.org.
D3244 − 21a
3.1.11 repeatability conditions, n—conditions where independent test results are obtained with the same method on identical test
items in the same laboratory by the same operator using the same equipment within short intervals of time. D6300
3.1.12 reproducibility (a.k.a. Reproducibility Limit) (R), n—a quantitative expression for the random error associated with the
difference between two independent results obtained under reproducibility conditions that would be exceeded with an approximate
probability of 5 % (one case in 20 in the long run) in the normal and correct operation of the test method. D6300
3.1.13 reproducibility conditions, n—conditions where independent test results are obtained with the same method on identical test
items in different laboratories with different operators using different equipment. D6300
3.1.14 result, n—the value obtained by following the complete set of instructions of a test method. It may be obtained from a single
determination or several determinations, depending on the instruction of the test method.
3.1.15 supplier, n—any individual or organization responsible for the quality of a product just before it is taken over by the
receiver.
3.1.16 supplier’s risk, n—the probability of rejecting a product that meets the specification.
3.1.17 true value (μ), n—for practical purposes, the value towards which the average of single results obtained by N laboratories
using the same standard test method tends, when N becomes very large. Consequently, this definition of true value is associated
with the particular test method employed.
4. Significance and Use
4.1 This practice provides a means whereby the parties can resolve disputes over specification conformance for those product
properties which can be tested and expressed numerically.
4.1.1 This practice can be used to ensure that such properties are correctly stated on labels or in other descriptions of the product.
4.1.2 This practice can be implemented in those cases where a supplier uses an in-house or a commercial testing laboratory to
sample and test a product prior to releasing the product to a shipper (intermediate receiver) and the ultimate receiver also uses an
in-house or commercial testing laboratory to sample and test the product upon arrival at the destination. The ATV would still be
determined according to 8.3.
4.2 This practice can be applied in the determination of tolerances from specification limits based on a mutually agreed probability
between parties for making the conformance to specification decision if the true value of a property is sufficiently close to the
specification limit. Such tolerances are bounded by an acceptance limit (AL). If the ATV value determined by applying this practice
falls on the AL or on the acceptable side of the AL, the product property can be considered to have met the specification; otherwise
it shall be considered to have failed to meet the specification.
4.3 Application of this practice requires the AL be determined prior to actual commencement of testing. Therefore, the degree of
criticality of the specification, as determined by the Probability of Acceptance (P value) that is required to calculate the AL, shall
have been mutually agreed upon between both parties prior to execution of actual product testing.
4.3.1 This agreement should include a decision as to whether the ATV is to be determined by the absolute or rounding-off method
of Practice E29, as therein defined.
4.3.1.1 If the rounding-off method is to be used, the number of significant digits to be retained must also be agreed upon.
4.3.1.2 These decisions must also be made in the case where only one party is involved, as in the case of a label.
4.3.1.3 In the absence of such an agreement, this practice recommends the ATV be rounded in accordance with the rounding-off
method in Practice E29 to the number of significant digits that are specified in the governing specification.
D3244 − 21a
4.4 This practice is designed to be suitable for reference in contracts governing the transfer of petroleum products and lubricants
from a supplier to a receiver.
4.5 As a prerequisite for acceptance for lab test results to be used in this practice, the following conditions shall be satisfied:
4.5.1 Site precision (R') as defined in Practice D6299 for the appropriate test method(s) from each lab, as substantiated by control
charts meeting the requirement of D6299 from in-house quality control programs, for property typical of the product in dispute,
should have a TPI > 1.2 for methods with Precision Ratio <4 and TPI > 2.4 for methods with Precision Ratio ≥4 (see Practice
D6792 for TPI explanation).
4.5.2 Each lab shall be able to demonstrate, by way of results from interlaboratory exchange programs, a lack of a systemic bias
relative to exchange averages for the appropriate test method(s) as per methodology outlined in Guide D7372.
4.5.3 In the event that the site precision of laboratories from two parties are statistically different as confirmed by the F-test (see
Annex A4), then, for the purpose of establishing the ATV, each laboratory’s test result shall be inversely weighted in accordance
with laboratory’s demonstrated variance.
4.6 It is recommended that this practice be conducted under the guidance of a qualified statistician.
5. Sampling
5.1 The disputing parties shall agree on the sampling procedure to obtain the dispute adjudication sample. Obtain enough sample
to allow for all required determinations to be made by at least two, and a possible third party.
6. Applying Test Method Precision Data to Accept or Reject Test Results
6.1 This section describes procedures in which the precision limits of test methods can be used as a decision criterion to accept
or reject test results.
6.2 Significance of Repeatability (r):
6.2.1 Acceptance of Results—When only two results are obtained under conditions of repeatability and the difference is equal to
or less than the repeatability of the method, the operator may report the average of the two results as being applicable to the sample
tested.
6.2.2 Rejection of Results—When two results are obtained that differ by more than the repeatability of the method, both should
be rejected. Obtain two additional results immediately under conditions of repeatability. If the difference between these two results
is equal to or less than the repeatability of the method, the operator should report the average of the two as being applicable to
the sample tested. If, however, the difference so obtained again exceeds the repeatability, reject the results and investigate the
application of the method.
6.3 Significance of Reproducibility (R):
6.3.1 Acceptance of Results—When two results are obtained and comprise one result from each laboratory (Note 2), if the
difference is equal to or less than the reproducibility of the method, then both results should be considered acceptable.
NOTE 2—When a comparison for reproducibility is made between results from two laboratories, it is a common practice that single results from each will
be compared. If each of the laboratories has produced more than a single result, see 6.4.
6.3.2 Rejection of Results—When the results from two laboratories differ by more than the reproducibility of the method, reject
both results and each laboratory should repeat the test on the retained sample. If the difference is now equal to or less than the
reproducibility, both results should be considered acceptable. If, however, the difference between these results is still greater than
the reproducibility, reject the results and investigate the application of the method at each laboratory, sampling, sample preparation
and storage and all other factors which can contribute to the variance.
D3244 − 21a
6.4 Significance of Reduced Reproducibility (R_reduced) from Multiple Testing—If the number of results obtained in either one
or both laboratories is more than one, then the allowable difference between the averages from the two laboratories is given as
follows:
1 1
2 2
Difference, R_reduced 5ŒR 2 r 12 2 (1)
S D
2n 2n
1 2
where:
R = reproducibility of the method,
r = repeatability of the method,
n = number of results of the first laboratory, and
n = number of results of the second laboratory.
6.5 Referee Laboratory—In the event a third or referee laboratory is invited to perform the test using a portion of one of the
samples described in 6.3.2, multiply the reproducibility, R, by 1.2 (to convert a range for two to a range for three) and compare
this value with the difference between the two extreme results for acceptance. If acceptance is indicated, the ATV for the sample
should be the average of the three results.
7. Determination of Acceptance Limits by Applying Test Method Precision Data and Specification Criticality
Considerations to Specification Limits
7.1 Specifications—A specification fixes a limit to the true value of a given property. In practice, however, this true value can never
be established exactly. The property is measured in the laboratory by applying a standard test method, the results of which may
show some random scattering within tolerances as defined by the test method repeatability and reproducibility limits. Therefore,
there is always some uncertainty as to the true value of the tested property.
7.2 Although the true value is never known exactly, the probability of obtaining any specific test result, relative to a hypothesized
true value, can be calculated if the probability distribution function for the test method is known (for example, the Normal or
Gaussian distribution).
7.2.1 Some specifications, because of the product characteristic or the end use of the product, or both, require that the receiver
have a high degree of assurance that the true value of the product property actually meets or exceeds the quality level indicated
by the specification limit value. For the purpose of this practice, such specifications are called critical specifications.
7.2.2 Specifications that require assurance only that the product property is not substantially poorer than is indicated by the
specification limit are called noncritical specifications for the purposes of this practice.
7.3 Specification Conformance Decision Guidelines:
7.3.1 Whenever a product is tested for conformity to a specification for a specific property, a decision must ultimately be made
as to whether the property conforms to specification.
7.3.2 The numerical value that delineates the regions of property conformance or nonconformance to specification is the AL. The
AL may or may not coincide with the specification limit value (S) used to define the requirements for the product.
7.3.3 The AL value, calculated as described in this practice, shall be agreed upon between the disputing parties prior to
commencement of testing.
7.3.4 The probability (P) of making the decision that the property conforms to specification when the true value of the property
exactly equals the specification limit value is shown in Fig. 1 and Fig. 2 as a function of D = (AL − S )/(0.255R), where D is a direct
measure of the difference between AL and S. This relationship is based (1) on the assumption of normally (Gaussian) distributed
testing errors, which is adequate for most test procedures, and (2) on using an ATV for making the specification conformance
decision that is the average of precision-acceptable results from two laboratories.
7.3.4.1 For values of P greater than 0.5 (Noncritical Spec Region in Fig. 1), the AL decision is primarily driven by supplier’s risk
considerations (that is, probability of rejecting a product which actually meets the specification).
D3244 − 21a
NOTE 1—Based on N = 2 = number of different laboratories’ results used to obtain ATV. See text for use of this table.
FIG. 1 Deviation of AL from Specification for Product Acceptance at a Given Probability
7.3.4.2 For values of P less than 0.5 (Critical Spec Region in Fig. 1), the AL decision is primarily driven by receiver’s risk
considerations (that is, probability of accepting a product which fails to meet the specification).
7.3.4.3 When P = 0.5, the AL coincides with the specification limit (see limit; 7.3.8); the conformance to specification decision
is based on equal sharing of test method imprecision related risks between the disputing parties for making the incorrect decision.
7.3.5 The AL associated with probability P of accepting the product when the true value exactly equals the specification limit value
S is then given by:
AL 5 S10.255·R·D (2)
=
7.3.5.1 The factor 0.255 in Eq 2 is for N (no. of labs) = 2. For N greater than 2, the 0.255 factor should be multiplied by 2/N.
7.3.6 In the absence of an agreement to the contrary, this practice recommends that for noncritical specifications, the AL is set such
that there is 95 % probability that the product will be accepted if the true value of the property is exactly at the specification limit
value. Thus, the AL will be set by using a confidence level P = 0.95 as shown in 7.3.5. It should be noted that for P = 0.95, the
AL will actually be numerically outside the specification limit values.
7.3.7 In the absence of an agreement to the contrary, this practice recommends that for critical specifications, the AL is set such
that there is 5 % probability that the product will be accepted if the true value of the property is exactly at the specification limit
value. Thus, the AL will be set by using a confidence level P = 0.05 as shown in 7.3.5. It should be noted that for P = 0.05, the
AL will actually be numerically inside the specification limit values.
7.3.6 WhenFor Dspecifications having = 0, the both minimum and maximum limits, the procedure in 7.3.5 must be applied twice
to give both upper and lower AL coincides exactly with the specification limit. Thes. There must be some allowable region
remaining between the lower and upper PAL value for s.D = 0 is 0.5, which means that there is a 50 % probability of making the
D3244 − 21a
FIG. 2 Probability of Acceptance vs Deviation of ALfrom True Value = S
conformance to specification decision if the true value of the property is exactly at the specification limit. This is also the
delineation point between critical and noncritical specification as chosen by this practice.
7.3.8.1 For specifications having both minimum and maximum limits, the procedure in 7.3.5 must be applied twice to give both
upper and lower ALs. There must be some allowable region remaining between the lower and upper ALs.
7.3.7 When only a single test result is or will be available, the relationships given should be used with N = 1 (7.3.5.1). Obviously,
no check on reproducibility precision can be made with a single test result, and the single value becomes the ATV for the sample.
7.3.8 The relationships between the ALs for critical and noncritical specifications are shown in Fig. 3 for a minimum specification.
7.3.9 For the risk of making the decision that the property conforms to specification when the true value fails to meet specification
by various off-spec amounts expressed in units of R, see Table
...