ISO 16140-4:2020

INTERNATIONAL ISO
STANDARD 16140-4
First edition
2020-07
Microbiology of the food chain —
Method validation —
Part 4:
Protocol for method validation in a
single laboratory
Microbiologie de la chaîne alimentaire — Validation des méthodes —
Partie 4: Protocole pour la validation de méthodes dans un seul
laboratoire
Reference number
©
ISO 2020
© ISO 2020
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting
on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address
below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii © ISO 2020 – All rights reserved

Contents Page
Foreword .iv
Introduction .v
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 2
4 General principles of the single-laboratory detection or quantification method
validation . 4
4.1 General . 4
4.2 Principles of the factorial approach . 5
4.3 Principles of the conventional approach . 5
5 Technical protocol for validation — Factorial approach . 7
5.1 Qualitative methods. 7
5.1.1 Single-laboratory method validation study against a reference method . 7
5.1.2 Single-laboratory method validation study without a reference method .13
5.2 Quantitative methods .15
5.2.1 Single-laboratory method validation study against a reference method .15
5.2.2 Single-laboratory method validation study without a reference method .18
6 Technical protocol for validation — Conventional approach .19
6.1 Qualitative methods.19
6.1.1 Single-laboratory method validation study against a reference method .19
6.1.2 Single-laboratory method validation study without a reference method .20
6.2 Quantitative methods .21
6.2.1 Single-laboratory method validation study against a reference method .21
6.2.2 Single-laboratory method validation study without a reference method .23
7 Summary of acceptability limits .26
Annex A (informative) List of factors and factor levels for factorial method validation .27
Annex B (informative) Calculation of in-house reproducibility for qualitative methods
based on the LOD study described in 6.1.2.3 .29
Annex C (informative) Example of a factorial single-laboratory method validation study for
a quantitative method against a reference method .30
Annex D (informative) Example of a factorial single-laboratory method validation study for
a qualitative method against a reference method .36
Annex E (informative) Example of a factorial single-laboratory method validation study for
the variability of the LOD for a qualitative method without a reference method .40
Annex F (informative) Determination of precision if the artificially contaminated samples
are unstable .43
Annex G (informative) Protocol for single-laboratory validation of alternative methods for
microbiological confirmation and typing procedures .45
Bibliography .46
Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards
bodies (ISO member bodies). The work of preparing International Standards is normally carried out
through ISO technical committees. Each member body interested in a subject for which a technical
committee has been established has the right to be represented on that committee. International
organizations, governmental and non-governmental, in liaison with ISO, also take part in the work.
ISO collaborates closely with the International Electrotechnical Commission (IEC) on all matters of
electrotechnical standardization.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for the
different types of ISO documents should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www .iso .org/ directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject of
patent rights. ISO shall not be held responsible for identifying any or all such patent rights. Details of
any patent rights identified during the development of the document will be in the Introduction and/or
on the ISO list of patent declarations received (see www .iso .org/ patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO’s adherence to the
World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see www .iso .org/
iso/ foreword .html.
This document was prepared by Technical Committee ISO/TC 34, Food products, Subcommittee SC 9,
Microbiology, in collaboration with the European Committee for Standardization (CEN) Technical
Committee CEN/TC 463, Microbiology of the food chain, in accordance with the Agreement on technical
cooperation between ISO and CEN (Vienna Agreement).
A list of all parts in the ISO 16140 series can be found on the ISO website.
Any feedback or questions on this document should be directed to the user’s national standards body. A
complete listing of these bodies can be found at www .iso .org/ members .html.
iv © ISO 2020 – All rights reserved

Introduction
0.1  The ISO 16140 series
The ISO 16140 series has been expanded in response to the need for various ways to validate or verify
test methods. It is the successor to ISO 16140:2003. The ISO 16140 series consists of six parts with the
general title, Microbiology of the food chain — Method validation:
— Part 1: Vocabulary;
— Part 2: Protocol for the validation of alternative (proprietary) methods against a reference method;
— Part 3: Protocol for the verification of reference methods and validated alternative methods in a single
laboratory;
— Part 4: Protocol for method validation in a single laboratory;
— Part 5: Protocol for factorial interlaboratory validation for non-proprietary methods;
— Part 6: Protocol for the validation of alternative (proprietary) methods for microbiological confirmation
and typing procedures.
ISO 17468 is a closely linked International Standard, which establishes technical rules for the
development and validation of standardized methods.
In general, two stages are needed before a method can be used in a laboratory.
— The first stage is the validation of the method. Validation is conducted using a study in a single
laboratory followed by an interlaboratory study (see ISO 16140-2, ISO 16140-5 and ISO 16140-6).
In the case when a method is validated within one laboratory (as described in this document), no
interlaboratory study is conducted.
— The second stage is method verification, where a laboratory demonstrates that it can satisfactorily
perform a validated method. This is described in ISO 16140-3. Verification is only applicable to
methods that have been validated using an interlaboratory study.
In general, two types of methods are distinguished: reference methods and alternative methods.
A reference method is defined in ISO 16140-1:2016, 2.59, as an “internationally recognized and widely
accepted method”. The note to entry clarifies that “these are ISO standards and standards jointly
published by ISO and CEN or other regional/national standards of equivalent standing”.
In the ISO 16140 series, reference methods include standardized reference (ISO and CEN) methods as
defined in ISO 17468:2016, 3.5, as a “reference method described in a standard”.
An alternative method (method submitted for validation) is defined in ISO 16140-1:2016, 2.4, as a
“method of analysis that detects or quantifies, for a given category of products, the same analyte as
is detected or quantified using the corresponding reference method”. The note to entry clarifies that:
“The method can be proprietary. The term ‘alternative’ is used to refer to the entire ‘test procedure
and reaction system’. This term includes all ingredients, whether material or otherwise, required for
implementing the method.”.
This document, ISO 16140-4, addresses validation within a single laboratory. The results are therefore
only valid for the laboratory that conducted the study. In this case, verification (as described in
ISO 16140-3) is not applicable. ISO 16140-5 describes protocols for non-proprietary methods where a
more rapid validation is required or when the method to be validated is highly specialized and the
number of participating laboratories required by ISO 16140-2 cannot be reached. This document
and ISO 16140-5 can be used for validation against a reference method. This document (regarding
qualitative and quantitative methods) and ISO 16140-5 (regarding quantitative methods only) can also
be used for validation without a reference method.
The flow chart in Figure 1 gives an overview of the links between the different parts mentioned above.
It also guides the user in selecting the right part of the ISO 16140 series, taking into account the purpose
of the study and the remarks given above.
Figure 1 — Flow chart for application of the ISO 16140 series
NOTE In this document, the words “category”, “type” and/or “item” are sometimes combined with “(food)”
to improve readability. However, the word “(food)” is interchangeable with “(feed)” and other areas of the food
chain as mentioned in Clause 1.
ISO 16140-6 is somewhat different from the other parts in the ISO 16140 series in that it relates to
a very specific situation where only the confirmation procedure of a method is to be validated [e.g.
the biochemical confirmation of Enterobacteriaceae (see ISO 21528-2)]. The confirmation procedure
advances a suspected (presumptive) result to a confirmed positive result. The validation of alternative
typing techniques (e.g. serotyping of Salmonella) is also covered by ISO 16140-6. The validation study
in ISO 16140-6 clearly defines the selective agar(s) from which strains can be confirmed using the
alternative confirmation method. If successfully validated, the alternative confirmation method can
only be used if strains are recovered on an agar that was used and shown to be acceptable within the
validation study. Figure 2 shows the possibilities where an alternative confirmation method validated
in accordance with ISO 16140-6 can be applied (see text in the boxes).
vi © ISO 2020 – All rights reserved

Figure 2 — Use of validated alternative confirmation methods (see ISO 16140-6)
EXAMPLE An example application of a validated alternative confirmation method is as follows.
An alternative confirmation method based on ELISA has been validated (in accordance with ISO 16140-6) to
replace the biochemical confirmation for Salmonella as described in ISO 6579-1. In the validation study, XLD
(mandatory agar in accordance with ISO 6579-1) plus BGA and a specified chromogenic agar (two optional agars
for second plating in accordance with ISO 6579-1) were used as the agars to start the confirmation. The validated
confirmation method can be used to replace the biochemical confirmation under the following conditions:
— by laboratories using the ISO 6579-1; or
— by laboratories using an ISO 16140-2 validated alternative method that refers to ISO 6579-1 for confirmation; or
— by laboratories using an ISO 16140-2 validated alternative method that starts the confirmation from XLD
and/or BGA agar and/or the specified chromogenic agar.
The validated confirmation method cannot be used under the following conditions:
— by laboratories using an ISO 16140-2 validated alternative method that refers only to agars other than those
included in the validation to start the confirmation (e.g. Hektoen agar and SS agar only); or
— by laboratories using an ISO 16140-2 validated alternative method that refers only to a confirmation
procedure that does not require isolation on agar.
0.2  Validation protocols in the ISO 16140 series
An interlaboratory validation study, in accordance with ISO 16140-2, requires at least eight laboratories
for quantitative methods and at least ten laboratories for qualitative methods. ISO 16140-5 is intended
to be used for interlaboratory studies comprising four to seven laboratories for quantitative methods
and four to nine laboratories for qualitative methods. ISO 16140-5 can only be used for non-proprietary
methods. Table 1 provides an overview of the different protocols.
Table 1 — Overview of different validation protocols described in the ISO 16140 series
Number of participating
With reference method Without reference method
laboratories
This document: This document:
1 —  factorial (see 5.1.1 and 5.2.1), or —  factorial (see 5.1.2 and 5.2.2), or
—  conventional (see 6.1.1 and 6.2.1) —  conventional (see 6.1.2 and 6.2.2)
4 to 7 (quantitative method)/ ISO 16140-5: for non-proprietary ISO 16140-5: for non-proprietary
4 to 9 (qualitative method) methods only quantitative methods only
≥ 8 (quantitative method)/ ISO 16140-2: for the interlaboratory
Not applicable
≥ 10 (qualitative method) study part
The aim of this document is to assess the performance of detection or quantification methods within
a single laboratory, typically across a number of (food) categories and (food) types. Single-laboratory
validation of alternative methods for microbiological confirmation and typing procedures can also
be performed under certain conditions: the general principles are the same as those described in
ISO 16140-6 for the validation of alternative (proprietary) methods for microbiological confirmation
and typing procedures (except there is no interlaboratory study). Further information is given in
Annex G.
The protocols in this document only validate the method for the particular laboratory. A generalization
to other laboratories is not within the scope of these protocols. However, extension to other laboratories
is possible if this document is used as the first phase of validation of a reference method, to be followed
by an interlaboratory study as described in ISO 17468.
If a reference method is available, the validation of a method is conducted by comparing the alternative
method to the reference method. This allows inclusion of naturally contaminated samples in the
validation process and thus provides a more realistic picture of the performance of the method. If no
reference method is available, the validation process is based on samples with known contamination
levels only. This document provides protocols for both situations.
The general principles for single-laboratory validations of detection and quantification methods are the
same as those described in ISO 16140-2 for the validation of alternative (proprietary) methods against
a reference method. This document cannot be used without ISO 16140-1 or ISO 16140-2, as many
definitions and procedures are given in these International Standards. In addition to the validation
parameters described in ISO 16140-2, this document describes the calculation of in-house repeatability
and in-house reproducibility. Calculation of these parameters is not required if an interlaboratory study
is to be conducted after the single-laboratory validation (i.e. if the single-laboratory validation is only
the first phase of validation). Reliability of performance parameters obtained with this document is
comparable to ISO 16140-2. This also means that the workload associated with the technical protocols
for the single laboratory is comparable with the method comparison study of ISO 16140-2.
This document provides two strategies for the single-laboratory method validation of detection and
quantification methods. The first strategy is based on a factorial approach while the second strategy
uses the conventional approach derived from the protocols of ISO 16140-2. In addition, protocols for the
determination of the in-house reproducibility for quantitative methods are described.
The advantages of using a factorial approach, over the conventional approach, are that it takes into
account specific conditions that the laboratory encounters during routine testing and provides more
information on the factors (technicians, culture media, etc.) that vary within the laboratory across
relevant (food) items, while using fewer samples to assess the performance of the method. The factorial
approach offers assessment of the precision of quantitative methods. It allows computation of reliable
and representative single-laboratory method validation parameters such as in-house reproducibility
standard deviation, LOD or RLOD values because it provides information on the variability of these
values under different measurement conditions. The factorial approach requires fewer test results in
order to obtain similar or higher levels of reliability compared to the conventional approach.
viii © ISO 2020 – All rights reserved

INTERNATIONAL STANDARD ISO 16140-4:2020(E)
Microbiology of the food chain — Method validation —
Part 4:
Protocol for method validation in a single laboratory
1 Scope
This document specifies the general principles and the technical protocols for single-laboratory
validation of methods for microbiology in the food chain. The protocols in this document only validate
the method for the laboratory conducting the study.
This document is applicable to single-laboratory validation of:
— methods used in the analysis (detection or quantification) of microorganisms in:
— products intended for human consumption;
— products intended for animal feeding;
— environmental samples in the area of food and feed production, handling;
— samples from the primary production stage;
— methods for the confirmation or typing of microorganisms. This validation will replace only the
confirmation or typing procedure of a specified method (see Annex G).
This document is, in particular, applicable to bacteria and fungi. Some clauses can be applicable to other
(micro)organisms or their metabolites, to be determined on a case-by-case basis.
Single-laboratory validation is required if an interlaboratory validation in accordance with ISO 16140-2
is not appropriate. Possible applications are:
— validation of an in-house method;
— method evaluation study in the validation process of a reference method in accordance with
ISO 17468;
— extension of the scope of an ISO 16140-2 validated method, e.g. category extension or test portion size;
— modifications of existing methods.
Single-laboratory validation is the second step in the standardization of a reference method
(see ISO 17468). It is only applicable to methods that are fully specified with regard to all relevant
parameters (including tolerances on temperatures and specifications on culture media) and that have
already been optimized.
2 Normative references
The following documents are referred to in the text in such a way that some or all of their content
constitutes requirements of this document. For dated references, only the edition cited applies. For
undated references, the latest edition of the referenced document (including any amendments) applies.
ISO 6887 (all parts), Microbiology of the food chain — Preparation of test samples, initial suspension and
decimal dilutions for microbiological examination
ISO 7218, Microbiology of food and animal feeding stuffs — General requirements and guidance for
microbiological examinations
ISO 16140-1:2016, Microbiology of the food chain — Method validation — Part 1: Vocabulary
ISO 16140-2:2016, Microbiology of the food chain — Method validation — Part 2: Protocol for the validation
of alternative (proprietary) methods against a reference method
3 Terms and definitions
For the purposes of this document, the terms and definitions given in ISO 16140-1 and the following apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— ISO Online browsing platform: available at https:// www .iso .org/ obp
— IEC Electropedia: available at http:// www .electropedia .org/
3.1
block
group of settings (3.12) that are conducted in parallel or in a short time interval, and that are used for
the same samples
EXAMPLE Block = settings conducted in parallel =
technician “a” + culture medium “b” + temperature “a” + incubation condition “a”
and
technician “b” + culture medium “a” + temperature “b” + incubation condition “b”.
Note 1 to entry: This definition is based on how ISO 3534-3:2013, 3.1.25, defines “block”. In ISO 3534-3:2013,
3.1.25, the definition is more general as it is defining a block as a set of experimental units that are homogenous
in some sense. The statistical meaning is the same.
3.2
factor
qualitative or quantitative parameter within the method that can be varied at two or more levels within
the limits of the specified method
EXAMPLE Technician.
Note 1 to entry: In this document, only those factors that are in line with the protocol of the method are
considered.
3.3
factor level
value of the factors (3.2) within the experimental design
EXAMPLE Technician “a”, technician “b”, etc.
Note 1 to entry: In this document, each factor is varied at two factor levels: “a” and “b”.
Note 2 to entry: This definition is based on how ISO 3534-3:2013, 3.1.12, defines “factor level”. In ISO 3534-3:2013,
3.1.12, the definition is more general, but the statistical meaning is the same.
3.4
in-house repeatability
measurement precision under a set of in-house repeatability conditions in a specific laboratory
Note 1 to entry: In-house repeatability conditions include the same measurement procedure, same technicians,
same measuring system, same operating conditions, same location and replicate measurements on the same or
similar objects over a short period of time in a particular laboratory.
2 © ISO 2020 – All rights reserved

3.5
in-house reproducibility
measurement precision under a set of in-house reproducibility conditions in a specific laboratory
Note 1 to entry: In-house reproducibility conditions include different technicians, different operating conditions
and replicate measurements on the same or similar objects over a longer period of time in a particular laboratory.
3.6
level of detection
LOD
x
measured analyte concentration, obtained by a given measurement procedure,
for which the probability of detection (3.9) is x
EXAMPLE LOD is the level of detection for which 50 % of tests give a positive result.
Note 1 to entry: The term “level of detection” is used for qualitative methods in microbiology based on replicate
analyses with three different contamination levels of the target analyte in a tested matrix. The replicates
are analysed, and the number of positive results is recorded (e.g. 20 %, 70 % and 100 %) respectively at each
contamination level. These data are then used to determine the number of cells that would give 50 % positive
using a generalized linear model (see ISO 16140-2). This differs from the procedure used for chemical and physical
methods for which a “limit of detection” is defined as the lowest quantity of an analyte that can be distinguished
from the absence of that analyte with a stated confidence level.
[SOURCE: ISO 16140-1:2016, 2.35, modified — Note 1 to entry has been slightly modified.]
3.7
limit of quantification
LOQ
limit of determination
lowest analyte concentration that can be quantified with an acceptable level of
precision and trueness under the conditions of the test
[SOURCE: ISO 16140-1:2016, 2.36]
3.8
orthogonal design
factorial design, in which for every pair of factors (3.2), each combination of factor levels (3.3) occurs
the same number of times across the possible factor levels
Note 1 to entry: This definition is based on how ISO 3534-3:2013, 3.1.31, defines “orthogonal array”, but for
“orthogonal design”, a more general and more theoretical definition is used.
3.9
probability of detection
POD
proportion of positive analytical outcomes for a qualitative method for a given matrix at a given analyte
level or concentration
Note 1 to entry: For qualitative methods, POD represents the probability of detection.
[SOURCE: ISO 16140-1:2016, 2.53, modified — Note 1 to entry has been added.]
3.10
relative level of detection
RLOD
level of detection (3.6) at P = 0,50 (LOD ) of the alternative (proprietary) method divided by the level of
detection at P = 0,50 (LOD ) of the reference method
Note 1 to entry: For purposes of alternative-method acceptance, the derived RLOD is checked with the
acceptability limit for conformity.
[SOURCE: ISO 16140-1:2016, 2.61]
3.11
single-laboratory method validation
in-house method validation
establishment of the performance characteristics of a method for the one particular laboratory in which
the validation is conducted
3.12
setting
combination of factor levels (3.3)
EXAMPLE Technician “a” + culture medium “b” + temperature “a” + etc.
Note 1 to entry: These conditions can be described by the combination of levels of factors varied within the study.
4 General principles of the single-laboratory detection or quantification method
validation
4.1 General
A single-laboratory detection or quantification method validation study is the first step in the
framework of general method validation and is needed to assess the performance of the method across
(food) categories, (food) types and (food) items. The second step in general method validation is an
interlaboratory study to assess the performance of the method across laboratories.
A single-laboratory method validation study is used to demonstrate the performance of the method in
the laboratory that conducted the study. The results are only valid for that particular laboratory.
NOTE Annex G gives the general principles for single-laboratory validation of alternative methods for
microbiological confirmation and typing procedures.
This document describes two approaches for single-laboratory method validation:
— a factorial approach, with:
— performance characteristics derived from ISO 16140-2;
— an orthogonal, factorial study design (see ISO 3534-3);
— more routine settings covered and fewer tests required than the conventional approach;
— a conventional approach, with:
— performance characteristics derived from ISO 16140-2;
— a stepwise procedure;
— a study design derived from ISO 16140-2.
Validation protocols are dependent on whether the method is qualitative or quantitative, and on
whether a factorial or a conventional approach is chosen. The factorial single-laboratory validation
approach can only be used for a fully developed and optimized method. A conventional approach
investigates the method for one specific setting (that is, one set of specific conditions under which the
method is performed). The main differences in approach for the single-laboratory validation covered in
this document are the number of various (food) items and the number of tests required to show that the
method performs adequately. Validation of methods with, and without, a reference method is possible
with the described protocols.
The scope of the validation protocol shall be determined at the start of the process, e.g. validation of
in-house methods, the second step in the validation process in accordance with ISO 17468, extension of
the scope of an ISO 16140-2 validated method, modification of existing methods.
4 © ISO 2020 – All rights reserved

For methods that include a PCR-based detection step, an assessment of the performance characteristics
for the PCR-based detection step is described in ISO 22118. To ensure the reliable detection of the target
organism in the samples tested, the relevant performance parameters of the PCR step should first be
assessed (based on ISO 22118), before validation of the complete analytical procedure (e.g. following
this document).
The selection of (food) categories and (food) types used in the validation study shall be conducted in
accordance with ISO 16140-2:2016, 5.1.3.1. It is recommended that each (food) category relevant to
the test method is also considered in the single-laboratory method validation study. Guidance on the
selection of (food) categories and (food) types is given in ISO 16140-2:2016, Annex A.
The scope of the validation study, results (tables and calculations) of the different parts and the
interpretation of the results, including discrepant results, shall be included in a validation study report.
4.2 Principles of the factorial approach
In a factorial approach, a systematic variation of factors is used to investigate the method performance
under a defined range of conditions that are typically encountered in the routine application of the
method. Typical factors are the technician or the sample storage, which can vary even within the same
laboratory using a given procedure. By investigating the method in a variety of conditions concurrently,
the factorial approach allows generalization of the validation to conditions commonly encountered in
the laboratory and is not just limited to a single condition.
It is necessary to select four major factors that are expected to reflect the typical variation of conditions
encountered in the routine application of the method. A risk analysis of the analytical workflow is
recommended for the selection of the factors. Examples of factors are given in Annex A.
The systematic variation of factors ensures that their combined impact on general performance
parameters, such as precision and sensitivity, can be derived. This is in contrast to a factorial robustness
study, in which the central aim is the detection of specific significant method parameters, so that the
performance of the method can be optimized. Compatibility between different factor levels and the
impact on precision of non-significant effects are not taken into account in such a study.
Compared to the conventional approach as described in ISO 16140-2, the factorial approach requires a
smaller number of (food) items and a smaller number of tests, while allowing for a reliable determination
of validation parameters.
4.3 Principles of the conventional approach
The conventional approach principally follows ISO 16140-2. It is conducted in several steps and does
not vary factors (see Table 2). The conventional approach requires more (food) items and test portions
to be tested than the factorial approach.
Table 2 — Number of tests required for a method validation per (food) category by the factorial
and conventional approach
Factorial approach Conventional approach
Qualitative
method
A R  A R
against a
Factorial study 78 78 Sensitivity study 60 60
reference
(sensitivity + RLOD)
method
Inclusivity/exclusivity 80 0 RLOD study 30 30
a
study
Inclusivity/exclusivity 80 0
a
study
Total number of tests 236 Total number of tests 260
(see 5.1.1)  (see 6.1.1)
Qualitative
method
Factorial study 256 Specificity 20
without a
(sensitivity + LOD )
reference
Inclusivity/exclusivity 80 LOD study 360
method
a
study (LOD + sensitivity)
Total number of tests 336 Inclusivity/exclusivity 80
a
study
Total number of tests 460
(see 5.1.2)  (see 6.1.2)
Quantitative
method
A R  A R
against a
Factorial study 48 48 Relative trueness study 15 15
reference
(relative trueness +
method
accuracy profile +
in-house precision)
Inclusivity/exclusivity 80 0 Accuracy profile study 30 30
a
study
Total number of tests 176 In-house precision study 40 5
Inclusivity/exclusivity 80 0
a
study
Total number of tests 215
(see 5.2.1)  (without LOQ study) (see 6.2.1)
Quantitative
method
Factorial study 48 Relative trueness study 15
without a
(relative trueness +
reference
accuracy profile +
method
in-house precision)
Inclusivity/exclusivity 80 Accuracy profile study 30
a
study
Total number of tests 128 In-house precision study 40
(see 5.2.2)  Inclusivity/exclusivity 80
a
study
Total number of tests 165
(without LOQ study) (see 6.2.2)
Key
A:  number of tests of the alternative method
R:  number of tests of the reference method
a
Inclusivity/exclusivity study requires 80 culture strains (130 for Salmonella) and is carried out only once for all approaches
irrespective of the number of (food) categories.
6 © ISO 2020 – All rights reserved

5 Technical protocol for validation — Factorial approach
5.1 Qualitative methods
5.1.1 Single-laboratory method validation study against a reference method
5.1.1.1 General considerations
The factorial single-laboratory validation can only be used for a fully developed and optimized method.
The validation study consists of two parts:
— a factorial, orthogonal comparison study (sensitivity and RLOD);
— an inclusivity/exclusivity study of the alternative method.
See Annex D for an elaborated example.
5.1.1.2 Factorial, orthogonal method comparison study
5.1.1.2.1 Selection of samples
The method comparison study compares the results obtained by the reference method and that of the
alternative method. The study is conducted using naturally and/or artificially contaminated samples:
usually, only artificially contaminated samples are used.
The requirements are as follows.
— Twelve different (food) items shall be selected for each (food) category: three (food) types per
(food) category shall be selected and four (food) items shall be selected for each (food) type. (Food)
items should be representative for the respective (food) type.
— The selection of (food) items shall take into account: background microbiota and food-processing
factors, such as heat, pH, freezing, smoking, drying (low a ); matrix conditions, such as pH value,
w
a value, aerobic/anaerobic; special sample preparation requirements, such as high fat content or
w
presence of inhibitors, in accordance with the ISO 6887 series.
— Each (food) item shall be contaminated at a minimum of two levels, consisting of at least the
following.
— A low (fractional) level L : the low level should have fractional recovery by the reference
method (fractional recovery at the low level should be between 25 % and 75 % of the number
of test portions tested). Ideally, the low level should be close to the theoretical detection level of
0,7 cfu/test portion (e.g. 0,5 cfu/test portion to 0,9 cfu/test portion).
— A high level L : at the high level (e.g. 5 cfu/test portion to 10 cfu/test portion), 100 % positive
results are expected.
— The four (food) items from each (food) type are allocated at random to four different blocks. Each
block shall contain three (food) items, each at two contamination levels, from three (food) types.
— Use a different strain per block and/or the same strain subjected to different stress factors
[e.g. temperature abuse, acid treatment or chlorination, depending on their relevance for the (food)
type]. Where it is not possible to use different strains for each block, the laboratory needs to provide
an explanation.
— Points to be considered when selecting strains are provided in ISO 16140-2:2016, Annex E.
— Six (food) items out of the twelve different (food) items shall be tested at zero level L (blank, i.e. no
target organism in the test portions).
— The size of the test portion shall be standardized for each study and should be the same for both
methods, if possible. If the size of the test portion allowed by the alternative method is different
from the reference method, contamination levels of the target microorganisms have to be adjusted
accordingly so that the final contamination level (cfu/test portion) is the same for both.
— General protocols for artificial contamination of samples are provided in ISO 16140-2:2016, Annex C.
— Artificial contamination of samples shall be finalized before starting the analyses.
5.1.1.2.2 Selection of method factors
Decisions on the most suitable factors for the particular study should be based on expert knowledge.
For example, optimal conditions are specified in each method (e.g. incubation temperature and duration
at 37 °C and 24 h) and these will give the best results. However, ranges around these, which provide still
acceptable conditions (e.g. for 24 h ± 1 h), are permitted and the study design should test the extremes
of these (e.g. incubate the samples for 23 h or 25 h). Acceptable operating conditions for equipment
are described in ISO 7218.
Relevant method factors that are more difficult to control (e.g. technicians, culture media and incubation
temperature) shall be selected and varied systematically to enable assessment of the accuracy under
routine conditions in the specific laboratory. The choice of these factors and factor levels is crucial to
the reliability of the validation result. These shall reflect the variation within the single laboratory
under routine conditions and should cover the most relevant aspects of the method, such as sample
preparation, sample storage, laboratory technician, laboratory equipment or background microbiota.
Other influences, such as atmosphere and stress conditions, can also be taken into account.
Four relevant factors shall be varied simultaneously, each on two levels.
For methods for culturable microorganisms, the factors and factor levels shown below are to be taken
into consideration. “Technicians” and “culture media” have the greatest impact and shall be included in
all studies as follows.
— Technicians: Tests shall be independently conducted in the single laboratory by two technicians.
— Culture media: Use culture media from two different manufacturers, if available, or two different
batches of culture media (lots), or pre-prepared versus prepared from dehydrated media. The
choices depend on the normal conditions of use of media in the laboratory. For example, two different
batches can be used if only one product is used in the laboratory, even if the product is available
from different manufacturers.
Two other factors shall be taken into account. A non-comprehensive list of grouped potential factors is
provided in Annex A. If possible, one factor from two of the most relevant groups shall be selected.
Factors are studied simultaneously using the study design described in 5.1.1.2.3.
5.1.1.2.3 Experimental design
The twelve (food) items from three (food) types shall be allocated to four blocks: that is, each block
shall include one (food) item from each (food) type. Each (food) item, contaminated at fractional
and high level, is analysed under two different settings (factor level combinations). Each setting is a
combination of levels of four factors, e.g. (food) items 1 to 3 are analysed in setting 1: technician “a”,
background microbiota “b”, culture medium “a”, incubation condition “a” (see Table 3). In addition, six
selected (food) items shall be tested at zero level L (blank).
Tests shall be performed as follows:
— the zero level L (blank) shall be tested using 1 replicate for each of 6 selected (food) items
representing the 3 (food) types (6 tests);
8 © ISO 2020 – All rights reserved

NOTE 1 The zero level L samples are tested to demonstrate that there are no false positive results
[cross reactivity e.g.
...