ASTM E2617-08

NOTICE: This standard has either been superseded and replaced by a new version or withdrawn.
Contact ASTM International (www.astm.org) for the latest information
Designation:E2617–08
Standard Practice for
Validation of Empirically Derived Multivariate Calibrations
This standard is issued under the fixed designation E 2617; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
1. Scope 3. Terminology
1.1 This practice covers requirements for the validation of 3.1 For terminology related to molecular spectroscopic
empirically derived calibrations (Note 1) such as calibrations methods, refer to Terminology E 131. For terminology related
derived by Multiple Linear Regression (MLR), Principal Com- to multivariate quantitative modeling refer to Practices E 1655.
ponent Regression (PCR), Partial Least Squares (PLS), Artifi- WhilePracticesE 1655iswritteninthecontextofmultivariate
cial Neural Networks (ANN), or any other empirical calibra- spectroscopic methods, the terminology is also applicable to
tion technique whereby a relationship is postulated between a other multivariate technologies.
setofvariablesmeasuredforagivensampleundertestandone 3.2 Definitions of Terms Specific to This Standard:
or more physical, chemical, quality, or membership properties 3.2.1 accuracy—the closeness of agreement between a test
applicable to that sample. result and an accepted reference value.
3.2.2 bias—the arithmetic average difference between the
NOTE 1—Empirically derived calibrations are sometimes referred to as
reference values and the values produced by the analytical
“models”or“calibrations.”Inthefollowingtext,forconciseness,theterm
method under test, for a set of samples.
“calibration” may be used instead of the full name of the procedure.
3.2.3 detection limit—the lowest level of a property in a
1.2 This practice does not cover procedures for establishing
sample that can be detected, but not necessarily quantified, by
said postulated relationship.
the measurement system.
1.3 This practice serves as an overview of techniques used
3.2.4 estimate—the constituent concentration, identifica-
to verify the applicability of an empirically derived multivari-
tion, or other property of a sample as determined by the
ate calibration to the measurement of a sample under test and
analytical method being validated.
to verify equivalence between the properties calculated from
3.2.5 initial validation—validation that is performed when
the empirically derived multivariate calibration and the results
an analyzer system is initially installed or after major mainte-
of an accepted reference method of measurement to within
nance.
control limits established for the prespecified statistical confi-
3.2.6 Negative Fraction Identified—the fraction of samples
dence level.
not having a particular characteristic that is identified as not
1.4 This standard does not purport to address all of the
having that characteristic.
safety concerns, if any, associated with its use. It is the
3.2.6.1 Discussion—Negative Fraction Identified assumes
responsibility of the user of this standard to establish appro-
that the characteristic that the test measures either is or is not
priate safety and health practices and determine the applica-
present. It is not applicable to tests with multiple possible
bility of regulatory limitations prior to use.
outcomes.
3.2.7 ongoing periodic revalidation—the quality assurance
2. Referenced Documents
process by which, in the case of quantitative calibrations, the
2.1 ASTM Standards:
bias and precision or, in the case of qualitative calibrations, the
E 131 Terminology Relating to Molecular Spectroscopy
Positive Fraction Identified and Negative Fraction Identified
E 1655 Practices for Infrared Multivariate Quantitative
performance determined during initial validation are shown to
Analysis
be sustained.
E 1790 Practice for Near Infrared Qualitative Analysis
3.2.8 Positive Fraction Identified—the fraction of samples
having a particular characteristic that is identified as having
This practice is under the jurisdiction of ASTM Committee E13 on Molecular
that characteristic.
Spectroscopy and Separation Science and is the direct responsibility of Subcom-
3.2.8.1 Discussion—Positive Fraction Identified assumes
mittee E13.11 on Multivariate Analysis.
that the characteristic that the test measures either is or is not
Current edition approved May 15, 2008. Published June 2008.
present. It is not applicable to tests with multiple possible
For referenced ASTM standards, visit the ASTM website, www.astm.org, or
contact ASTM Customer Service at service@astm.org. For Annual Book of ASTM
outcomes.
Standards volume information, refer to the standard’s Document Summary page on
the ASTM website.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959, United States.
E2617–08
3.2.9 precision—the closeness of agreement between inde- property levels and the compositional characteristics of the
pendent test results obtained under stipulated conditions. validation samples such that they are suitable to the applica-
3.2.9.1 Discussion—Precision may be a measure of either tion. This practice allows the user to write a comprehensive
the degree of reproducibility or degree of repeatability of the validation statement for the analyzer system including specific
analytical method under normal operating conditions. In this limits for the validated range of application and specific
context, reproducibility refers to the use of the analytical restrictions to the permitted uses of the measurement system.
procedure in different laboratories, as in a collaborative study. Users are cautioned against extrapolation of validation results
3.2.10 quantification limit—the lowest level of a sample beyond the material type(s) and property range(s) used to
property which can be determined with acceptable precision obtain these results.
and accuracy under the stated experimental conditions. 5.6 Users are cautioned that a validated empirically derived
3.2.11 range—the interval between the upper and lower multivariate calibration is applicable only to samples that fall
levels of a property (including these levels) that has been within the subset population represented in the validation set.
demonstrated to be determined with a suitable level of preci- The estimation from an empirically derived multivariate cali-
sion and accuracy using the method as specified. bration can only be validated when the applicability of the
3.2.12 reference value—the metric of a property as deter- calibration is explicitly established for the particular measure-
mined by well-characterized method, the accuracy of which ment for which the estimation is produced. Applicability
has been stated or defined, that is, another, already-validated cannot be assumed.
method.
6. Methods and Considerations
3.2.13 validation—the statistically quantified judgment that
an empirically derived multivariate calibration is applicable to 6.1 When validating an empirically derived multivariate
calibration, it is the responsibility of the user to describe the
the measurement on which the calibration is to be applied and
can performpropertyestimateswith,inthecaseofquantitative measurement system and the required level of agreement
between the estimations produced by the calibration and the
calibrations, acceptable precision, accuracy and bias or, in the
accepted reference method(s).
case of qualitative calibrations, acceptable Positive Fraction
6.2 When validating a measurement system incorporating
Identified and Negative Fraction Identified, as compared with
anempiricallyderivedmultivariatecalibration,itistherespon-
results from an accepted reference method.
sibility of the user to satisfy the requirements of any applicable
4. Summary of Practice
tests specific to the measurement system including any Instal-
4.1 Validating an empirically derived multivariate calibra- lation Qualification (IQ), Operational Qualification (OQ), and
tion (model) consists of four major procedures: validation at Performance Qualification (PQ) requirements; which may be
initial development, revalidation at initial deployment or after mandated by competent regulatory authorities, an applicable
a revision, ongoing periodic revalidation, and qualification of Quality Assurance (QA), or Standard Operating Procedure
each measurement before using the calibration to estimate the (SOP) or be recommended by the instrument or equipment
property(s) of the sample being measured. manufacturer.
6.3 Reference Values and Quality Controls for the Accepted
5. Significance and Use
Reference Method:
5.1 Thispracticeoutlinesauniversallyapplicableprocedure
6.3.1 The reference (or true) value which is compared with
to validate the performance of a quantitative or qualitative,
each respective estimate produced by the empirically derived
empirically derived, multivariate calibration relative to an
multivariate calibration is established by applying an accepted
accepted reference method.
reference method, the characteristics of which are known and
5.2 This practice provides procedures for evaluating the
stated, to the sample from which the measurement system
capability of a calibration to provide reliable estimations
derives the measurement.
relative to an accepted reference method. 6.3.2 To ensure the reliability of the reference values
5.3 This practice provides purchasers of a measurement
provided by an accepted reference method, appropriate quality
system that incorporates an empirically derived multivariate controls should be applied to the accepted reference method.
calibration with options for specifying validation requirements
7. Procedure
to ensure that the system is capable of providing estimations
with an appropriate degree of agreement with an accepted 7.1 The objective of the validation procedure is to quantify
reference method. the performance of an empirically derived multivariate calibra-
5.4 Thispracticeprovidestheuserofameasurementsystem tion in terms of, in the case of quantitative calibrations,
that incorporates an empirically derived multivariate calibra- precision, accuracy and bias or, in the case of qualitative
tionwithprocedurescapableofprovidinginformationthatmay calibrations, Positive Fraction Identified and Negative Fraction
be useful for ongoing quality assurance of the performance of Identified relative to an accepted reference method for each
the measurement system. property of interest. The user must specify, based on the
5.5 Validation information obtained in the application of intendeduseofthecalibration,acceptableprecisionandbiasor
thispracticeisapplicableonlytothematerialtypeandproperty Positive Fraction Identified and Negative Fraction Identified
range of the materials used to perform the validation and only performance criteria before initiating the validation. These
fortheindividualmeasurementsystemonwhichthepracticeis criteria will be dependent on the intended use of the analyzer
completely applied. It is the user’s responsibility to select the and may be based, all or in part, on risk based criteria.
E2617–08
7.1.1 The acceptable performance criteria specified by the 7.3.3.2 Span the ranges of the independent variables over
user may be constant over the entire range of sample variabil- which the calibration will be used; that is, if the range of an
ity.Alternatively,differentacceptableperformancecriteriamay independent variable is expected to vary from a to b, and the
be specified by the user for different sub-ranges of the full standard deviation of the independent variable is c, then the
variations of that independent variable in the set of validation
sample variability.
samples should cover at least 100 % of the range from a to b,
7.2 Validation of calibration is accomplished by using the
and should be distributed as uniformly as possible across the
calibration to estimate the property(s) of a set of validation
range such that the standard deviation in that independent
samples and statistically comparing the estimates for these
variable estimated for the validation samples will be at least
samples to known reference values. Validation requires thor-
95 % of c.
ough testing of the model with a sufficient number of repre-
(1) When validating a calibration for which detection limit
sentative validation samples to ensure that it performs ad-
or quantification limit is an important consideration, the user
equately over the entire range of possible sample variability.
should include a number of validation samples whose proper-
7.3 Initial Validation Sample Set:
ty(s) are close to the detection or quantification limit(s)
7.3.1 For the initial validation of a multivariate model, an
sufficient to validate the respective limit(s) to the statistical
ideal validation sample set will:
degree of confidence required for the application.
7.3.1.1 Contain samples that provide sufficient examples of
7.4 For quantitative calibrations, the validation error for
all combinations of variation in the sample properties which
each property in each sample is given by the Standard Error of
are expected to be present in the samples which are to be
Validation (SEV) and bias for that property.
analyzed using the calibration;
-
7.4.1 The validation bias, e , is a measure of the average
v
7.3.1.2 Contain samples for which the ranges of variation in
difference between the estimates made based on the empirical
the sample properties is comparable to the ranges of variation
model and the results obtained on the same validation samples
expected for samples that are to be analyzed using the model;
using the reference method.
7.3.1.3 Contain samples for which the respective variations
7.4.1.1 If there are single reference values and estimates for
of the sample properties are uniformly and mutually indepen-
each validation sample, the validation bias is calculated as:
dently distributed over their full respective ranges or, when
v
applicable, subranges of variation; and
^
~v — v !
( i i
i51
7.3.1.4 Contain a sufficient number of samples to statisti-
e 5 (1)
v
v
cally test the relationships between the measured variables and
the properties that are modeled by the calibration.
where:
7.3.2 For simple systems, sufficient validation samples can
^
= estimate from the model for the ith sample,
v
i
generallybeobtainedtomeetthecriteriain7.3.1.1-7.3.1.4.For
v = accepted reference value for the ith sample, and
i
complex mixtures, obtaining an ideal validation set may be
v = number of validation samples.
difficult if not impossible. In such cases, it may be necessary to
7.4.1.2 Ifreplicateestimatesandasingleref
...