ISO 3725:2023

INTERNATIONAL ISO
STANDARD 3725
First edition
2023-07
Ships and marine technology —
Aquatic nuisance species — Methods
for evaluating the performance of
compliance monitoring devices for
ballast water discharges
Reference number
© ISO 2023
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting on
the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address below
or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii
Contents Page
Foreword .v
Introduction . vi
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 General . 5
4.1 C ompliance with the discharge standard . 5
4.2 Reference method . 5
4.3 Challenge water . 6
4.4 Test concentrations of organisms. 6
5 E v a lu at ion me t r ic s .7
5.1 General . 7
5.1.1 Overview . 7
5.1.2 Measurement protocols . 7
5.1.3 Categorical outcomes . 7
5.2 Trueness . 8
5.2.1 Overview . 8
5.2.2 Measurement approach . 8
5.2.3 Statistical calculations . 9
5.3 Precision . . . 10
5.3.1 Overview . 10
5.3.2 Measurement approach . 11
5.4 Reliability . 11
6 E valuation test types .12
6.1 Overview .12
6.2 Laboratory tests with prepared challenge water .12
6.2.1 Challenge water .12
6.2.2 Cultured organisms .13
6.2.3 Sample volumes and organism concentrations .13
6.2.4 Sample handling and analysis . 13
6.3 Laboratory tests using natural water with ambient organisms . 14
6.3.1 Natural water . 14
6.3.2 Ambient organisms . 14
6.3.3 Sample volume . 14
6.3.4 Sample handling and analysis . 14
6.4 Field tests using treated water .15
6.4.1 Treated water .15
6.4.2 Organisms present post treatment . 15
6.4.3 Sample volume .15
6.4.4 Sample handling and analysis . 15
6.4.5 Test information and descriptions . 15
7 Experimental design .16
7.1 General . 16
7.2 CMD characteristics . 17
7.3 Known CMD limitations . 17
7.4 Basic evaluation requirements . 18
7.5 Experimental power and sample sizes . 23
7.6 Additional, optional factors for consideration . 24
7.7 Ancillary analyses . 24
8 Test quality management and reporting .25
Annex A (informative) Typically available cultures of organisms .26
iii
Annex B (informative) Additional, optional factors for testing .27
Bibliography .28
iv
Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards
bodies (ISO member bodies). The work of preparing International Standards is normally carried out
through ISO technical committees. Each member body interested in a subject for which a technical
committee has been established has the right to be represented on that committee. International
organizations, governmental and non-governmental, in liaison with ISO, also take part in the work.
ISO collaborates closely with the International Electrotechnical Commission (IEC) on all matters of
electrotechnical standardization.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for the
different types of ISO document should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives).
ISO draws attention to the possibility that the implementation of this document may involve the use
of (a) patent(s). ISO takes no position concerning the evidence, validity or applicability of any claimed
patent rights in respect thereof. As of the date of publication of this document, ISO had not received
notice of (a) patent(s) which may be required to implement this document. However, implementers are
cautioned that this may not represent the latest information, which may be obtained from the patent
database available at www.iso.org/patents. ISO shall not be held responsible for identifying any or all
such patent rights.
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to
the World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT), see
www.iso.org/iso/foreword.html.
This document was prepared by Technical Committee ISO/TC 8, Ships and marine technology.
Any feedback or questions on this document should be directed to the user’s national standards body. A
complete listing of these bodies can be found at www.iso.org/members.html.
v
Introduction
A compliance monitoring device (CMD) is an instrument intended to analyse samples of ballast water,
to estimate whether the concentration of living or viable organisms present in the sample exceeds,
or is at risk of exceeding, the regulated limit [i.e. the discharge standard, (DS)]. Typically, CMDs are
designed for use in shipboard and field locations to provide results rapidly and with less effort relative
to complex analyses. CMDs are instruments that are relatively new to their application in ballast
water testing. They can rely upon standard optical, chemical, or physical measurements, but these
technologies are deployed in unique configurations. They can be packaged in a rugged, transportable
housing, or installed as shipboard equipment. A CMD may operate along a spectrum of water types with
diverse assemblages of organisms. As intended, CMDs provide critical information to vessel inspectors,
ballast water management system (BWMS) commissioning test teams, Port State Control Officers, ship
owners, among others.
This document was developed in response to the need for a standardized approach to evaluate the
performance of CMDs. This evaluation includes:
— laboratory-based tests using prepared sample water amended with cultured organisms as well as
dissolved and particulate materials;
— laboratory-based tests using samples of natural assemblages of organisms, experimentally
manipulated to achieve target concentrations of living or viable organisms (but without manipulation
of dissolved and particulate materials);
NOTE 1 It is recognized that the end user can require laboratory testing with ambient organisms instead of,
or in addition to, cultured organisms. Additionally, the end user can require that both types of laboratory-based
tests are conducted using water that is treated by a BWMS or has undergone a simulated ballast water treatment,
instead of, or in addition to, un-treated water.
— field-based tests using samples of water treated with a BWMS collected aboard a ship.
This standardized approach defines a general test procedure and minimum set of trials to evaluate the
performance of a CMD. The key evaluation metrics are accuracy (hereafter, “trueness” - the agreement
to a reference method), precision, and reliability. While a CMD may report numerical values or estimates
of organism concentrations, trueness and precision are determined based upon the agreement between
the CMD and reference method on the sample disposition (i.e. whether the sample meets or exceeds the
DS).
NOTE 2 This approach is not appropriate to evaluate methods or devices intended to be used as an alternate to
the reference method, i.e. with precise, numerical measurements across a wide range of organism concentrations.
The test methods are adaptable, such that additional factors which are deemed important — e.g.
interferences, organism types, or water characteristics — may be addressed experimentally and
included in the set of performance metrics. This flexibility allows end-users to supplement these
minimal test requirements to examine additional characteristics, such as CMD performance under
different types of BWMS treatments.
vi
INTERNATIONAL STANDARD ISO 3725:2023(E)
Ships and marine technology — Aquatic nuisance species
— Methods for evaluating the performance of compliance
monitoring devices for ballast water discharges
1 Scope
This document specifies methods to evaluate the performance of a specific class of analytical
instruments, known as compliance monitoring devices (CMDs). These instruments are designed and
intended to examine ballast water to determine whether a sample meets or exceeds limits for the
concentration of living or viable organisms. These limits include those specified by the International
Maritime Organization (IMO) Regulation D-2 in the International Convention for the Control and
[4]
Management of Ships' Ballast Water and Sediments or other discharge standards (DS) adopted by
national or regional authorities.
The test methods measure the agreement between the CMD and a reference method to calculate
trueness and precision. Both trueness and precision consider only simple, categorical outcomes (e.g.
“meets” or “exceeds” the DS). The performance metric reliability is quantified by the frequency of
instances when the CMD is not available or is not operating as expected.
The set of tests and trials is based upon the CMD manufacturer claims, such as the DS group(s) targeted
by the CMD, and known limitations, including those based upon the salinity of the sample water.
NOTE Additional tests and trials, if required by the end-user, can follow this general test method. Guidance
on determining experimental power is found in 7.5. This document provides guidance for customizing the tests
to evaluate the claims of the manufacturer or to address optional factors of interest to the end-users.
This document does not set or recommend success criteria of any performance metric, as these are
appropriately defined by the end-users.
2 Normative references
The following documents are referred to in the text in such a way that some or all of their content
constitutes requirements of this document. For dated references, only the edition cited applies. For
undated references, the latest edition of the referenced document (including any amendments) applies.
ISO 11711-1:2019, Ships and marine technology — Aquatic nuisance species — Part 1: Ballast water
discharge sample port
ISO 11711-2:2022, Ships and marine technology — Aquatic nuisance species — Part 2: Ballast water sample
collection and handling
ISO 21748, Guidance for the use of repeatability, reproducibility and trueness estimates in measurement
uncertainty evaluation
ISO/IEC 17025, General requirements for the competence of testing and calibration laboratories
ASTM D1141-98, Standard Practice for Preparation of Substitute Ocean Water
3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
ISO and IEC maintain terminology databases for use in standardization at the following addresses:
— ISO Online browsing platform: available at https:// www .iso .org/ obp
— IEC Electropedia: available at https:// www .electropedia .org/
3.1
accuracy
closeness of agreement between a test result and the accepted reference value
Note 1 to entry: The more specific term, trueness (3.24), is used as a metric throughout this standard.
[SOURCE: ISO 5725-1:1994, 3.6, modified — Note 1 to entry has been replaced.]
3.2
agreement
concurrence between two independent measurements on the outcome of analysis
Note 1 to entry: Analysis outcomes are qualitative or categorical descriptions of whether a sample meets or
exceeds the discharge standard (3.13).
3.3
ambient water
natural water
water collected directly from the natural environment that 1) contains natural communities of
organisms, dissolved and particulate constituents, and 2) has intrinsic characteristics, such as
temperature and salinity
3.4
ballast water
water with its abiotic and biotic constituents taken on board a ship to control trim, list, stability or
stresses of the ship
3.5
ballast water management system
BWMS
equipment that processes ballast water (3.4) such that the water discharged (the treated water) is
intended to meet the specified performance requirements for eliminating, inactivating, or reducing
aquatic organisms
3.6
calibration
analysis, in water, of standards to develop a relationship between raw output of an analytical system
and analyte concentration
3.7
calibration standard
sample containing the analyte of interest at a known concentration either purchased from an external
source or prepared in-house from materials of known purity or concentration, or both, and used to
calibrate the measurement system
3.8
challenge water
water prepared or manipulated (e.g. by adding organisms and abiotic constituents) to achieve minimum
test criteria when testing the performance of equipment, in this case compliance monitoring devices
(3.11)
Note 1 to entry: This protocol shares some characteristics with the minimum water quality requirements
for challenge water for type approval testing (3.25) of the International Maritime Organization and USA, such
as salinity and temperature ranges and abiotic constituents. However, requirements for concentrations and
diversity of organisms are unique to this application.
3.9
colonial organisms
collection of multiple, clonal individuals that are physically connected
Note 1 to entry: Clusters of connected, but non-clonal individuals are typically referred to as aggregated
organisms.
3.10
compliance monitoring device
CMD
instrument and its associated analytical methodology typically used as a rapid assessment of the
concentration of living or viable organisms in treated ballast water for the purpose of determining
compliance or non-compliance with a discharge standard (3.13)
3.11
detection limit
method detection limit
lowermost quantity or concentration measurable by the compliance monitoring device (3.10)
Note 1 to entry: In the context of compliance monitoring device (CMD) evaluation, the detection limit is specified
by the manufacturer. The CMD evaluation may include test samples with concentrations reflecting the stated
detection limit to verify the manufacturer claim (3.16).
Note 2 to entry: In the context of reference method (3.21), the method detection limit is according to the definition
in ISO/IEC Guide 99:2007.
3.12
dissolved organic matter
DOM
mass of organic compounds present in water that are not separated by particle (≤0,7 µm) filtration
Note 1 to entry: Dissolved organic carbon is a related quantity that is commonly measured directly. Although the
two quantities are related, they are distinct and should be clearly identified.
3.13
discharge standard
DS
regulated limits of organism concentrations allowable in discharged ballast water
Note 1 to entry: Regulation D-2 of the International Maritime Organization’s Ballast Water Management
Convention.
Note 2 to entry: The term is generic unless a particular DS is specified.
Note 3 to entry: A DS is also known as a performance standard.
3.14
independent testing organization
testing organization that is free of any conflict of interest with the manufacturer of the compliance
monitoring device (3.10)
3.15
living organism
organism that demonstrates characteristics of life (movement, membrane integrity, etc.)
Note 1 to entry: It is possible that living organisms are not viable (3.27).
3.16
manufacturer claims
specific characteristics of the compliance monitoring device (3.10) that are asserted by the manufacturer
or vendor of the device
Note 1 to entry: Claims typically include the organisms size class(es) or indicator microbe(s) targeted by the
device, limitations based upon organism characteristics (such as autotrophy), water temperature and salinity
ranges, as well as the detection limits, accuracy, and precision of the compliance monitoring device.
3.17
mineral matter
MM
mass of inorganic compounds present in water and separated by particle (≤0,7 µm) filtration
Note 1 to entry: MM is estimated as the mass difference between total suspended solids (3.22) and particulate
organic matter (3.18).
3.18
particulate organic matter
POM
mass of organic matter present in water and separated by particle (≤0,7 µm) filtration
Note 1 to entry: Particulate organic carbon is a related quantity and composes a portion of the mass of POM.
Although the two quantities are related, they are distinct and should be clearly identified.
3.19
precision
agreement between replicate measurements of a sample measured under the same conditions
Note 1 to entry: The same conditions include the same sample, the same instrument unit, and the same analyst,
if applicable.
3.20
reagent-grade, purified water
water meeting the characteristics of Type I or II water, used as the basis for preparing challenge water
for laboratory testing
Note 1 to entry: The characteristics of Type I or II water are defined in ASTM D1193-06.
3.21
reference method
analytical method that produces a value used as a benchmark
Note 1 to entry: Reference methods produce direct measurements of numerical concentrations that are
comparable to the discharge standard (3.13).
Note 2 to entry: Reference methods are typically methods used in ballast water management system (3.5) type
approval testing (3.25).
3.22
total suspended solids
TSS
mass of organic and inorganic matter present in water and separated by particle (≤0,7 µm) filtration
Note 1 to entry: TSS is composed of mineral matter and particulate organic matter.
3.23
trial
complete set of samples and sample analyses associated with a single test condition, such as water
salinity
3.24
trueness
closeness of agreement between the average value obtained from a large series of test results and an
accepted reference value
[SOURCE: ISO 5725-1:1994, 3.7, modified — Notes 1 and 2 to entry have been deleted.]
3.25
type approval testing
testing performed as part of a formal certification of a ballast water management system (3.5) for use
aboard ships
3.26
uncertainty
measurement uncertainty
parameter, associated with the result of a measurement, which characterizes the dispersion of the
values that can reasonably be attributed to the measurand
[SOURCE: ISO 21748:2017, 3.14, modified — “measurement uncertainty” has been added as a preferred
term; Notes 1, 2 and 3 to entry have been deleted.]
3.27
viable
living and capable of reproduction
Note 1 to entry: Manufacturers shall indicate whether their compliance monitoring device (3.10) quantifies living
or viable organisms, and the test should be designed to evaluate their claims using the appropriate reference
method (3.21) for living or viable organisms.
4 General
4.1 C ompliance with the discharge standard
A compliance monitoring device (CMD) determines whether a sample is likely to comply with or exceed
[4]
the discharge standard (DS), such as the IMO Regulation D-2 which sets limits on the concentration of
viable organisms in the following size or taxonomic groups:
— organisms ≥50 µm in minimum dimension;
— organisms ≥10 µm and <50 µm in minimum dimension;
— toxicogenic Vibrio cholerae (serotypes O1 and O139);
— Escherichia coli;
— intestinal enterococci.
NOTE National or regional authorities can define the same or similar categories and concentration limits.
The test methods described in this document are generic: the methods apply to any of the groups
defined in the DS and the corresponding reference method used for each of those defined groups.
This performance evaluation considers the claims of a CMD manufacturer, such as those defining the
targeted group(s), the relevant DS e.g. References [4] and [5], and the stated limitations, such as limits
on the salinity of the sample water.
4.2 Reference method
A reference method is an analytical approach used to quantify living or viable organisms in one of the
categories of a DS (see 4.1). Typically, the reference method is used during type approval (TA) tests,
[6] [7]
such as those prescribed in the test protocols of the IMO or the United States. In general, a reference
method estimates numerical concentrations of living or viable organisms in a single category of DS.
Consequently, each category examined requires a unique set of measurements using the appropriate
reference method.
The numerical results from the reference method shall be reported with estimates of measurement
uncertainty, which shall be determined in accordance with the approaches described in ISO 21748.
Numerical values shall also be converted to one of three categorical values: “meets the DS”, “exceeds
the DS”, or “indeterminant”, which considers the ranges of values with a 95 % confidence interval of
the measurement (see 5.1). This conversion permits a direct comparison between the CMD and the
reference method.
4.3 Challenge water
Challenge water — as defined in this document — is similar to the “challenge water” used for TA testing.
[6][7]
Challenge water for CMD testing is only used in the subset of laboratory trials designed to evaluate
the CMD performance under standardized and simplified conditions. The salinity of challenge water is
-1 1) -1 -1 -1 -1
either fresh (<1 g kg ), brackish (10 g kg to 20 g kg ), or marine (28 g kg to 36 g kg ), and it may
be prepared from natural waters or with purified water amended with sea salts according to ASTM
D1141-98.
Challenge water consists of abiotic characteristics, defined as minimum concentrations of dissolved
organic matter (DOM), particulate organic matter (POM), and mineral matter (MM) and total suspended
[6][7]
solids (TSS), as defined in TA test protocols.
NOTE For other trials, including laboratory tests using ambient water, the salinity, DOM, POM, MM, and TSS
are measured but not manipulated.
For the purpose evaluating the performance of CMD, abiotic challenge water components are as defined
[6][7]
in TA test protocols, but requirements for organism concentrations and diversity are specific to
this test protocol and are defined in 4.4 and 6.2.
4.4 Test concentrations of organisms
The concentrations of living or viable organisms are defined relative to the DS. Concentrations below,
approximately equal to, and above the DS are most relevant to this performance evaluation, and the
target ranges ensure that samples meeting and exceeding the DS are included in the evaluation.
Target ranges are at a minimum defined below:
— below: >0 % and <50 % the DS;
— approximately equal to: ±50 % the DS;
— above: >150 % and <1 000 % the DS.
Concentrations below the DS shall be non-zero and measurable by the reference method (i.e. > limit of
quantification).
−1
NOTE 1 At concentrations 50 % of the DS (e.g. ≤4 organisms ml ), the probability that a random sample
−1 −1
will have ≤9 organisms ml is > 99 %. Likewise, at concentrations >150 % of DS, e.g. ≥16 organisms ml , the
−1
probability that a random sample will have ≥ 11 organisms ml is >95 %.
NOTE 2 Additional concentrations can be added, provided that both concentrations above and below the DS
are included in the evaluation.
-1
1) Salinity is reported here as g kg , which for the purposes of this test protocol is approximately equivalent to
[5]
Practical Salinity Units (PSU) .
5 Evaluat ion metrics
5.1 General
5.1.1 Overview
The evaluation considers three metrics: trueness, precision, and reliability, which are described in 5.2
to 5.4.
NOTE This document uses the terms “trueness” and “precision” to describe the accuracy of a measurement
method, following the terminology in ISO 5725-1. This terminology differs from other common definitions, which
define “accuracy” as the agreement between a measurement and the true quantity e.g. as shown in Reference [9].
5.1.2 Measurement protocols
The evaluation requires measurements from both the reference method and the CMD. Measurements
shall be performed following standard, pre-established protocols. The pre-established protocols shall
have guidance on whether the measurements meet data quality objectives and are acceptable.
EXAMPLE A data quality indicator can exceed the data quality objective, in which case, the protocol
can require that the measurement is rejected and a new sample is analysed. It can be appropriate to reject a
measurement, given measurements are not rejected without cause. It is expected that rejected measurements
are recorded and reported.
The protocol for the CMD or the reference method can require multiple readings or repeated
subsampling and analysis. In this case, the protocols shall indicate the process for reporting the mean
and dispersion (e.g. range, standard deviation) of the set of numerical measurements, or for determining
the overall sample disposition (e.g. meets or exceeds the DS).
5.1.3 Categorical outcomes
Readings from both the CMD and reference method are simplified into three categories:
— “meets the DS”;
— “exceeds the DS”;
— “indeterminate”.
For the CMD, the categorical outcome is based upon guidance from the manufacturer. This guidance
from manufacturers can include instructions for binning numerical measurements into one of the
categories; the CMD may automatically classify the sample disposition and display terminology relating
to one of the three categories.
EXAMPLE A CMD can report the outcome in terms of “risk”, such as “low risk” or “high risk”. These terms are
comparable to “meets the DS” or “exceeds the DS”, respectively.
For the reference method, the categories are assigned based upon the measured concentration and the
uncertainty surrounding the estimate, considering, for example, the range of potential values within
a 95 % confidence interval of the estimate. Estimates with confidence intervals spanning the DS are
considered “indeterminate”.
NOTE The reference method selected is well characterized, such that independent testing organizations
have a historical record of usage and performance monitoring. These data support the estimates of measurement
uncertainty and confidence intervals.
Though the evaluation metrics are based on categorical variables, it is important that all numerical
measurements are collected and reported. For the CMD, all reported numerical information is recorded
for each sample. The numerical results are important, as end-users may perform additional statistical
tests using these measurements. For the reference method, all values used to calculate concentrations
(including sample volumes, reagent volumes, etc.) are recorded with the measurement. Additionally,
the method detection limit and confidence intervals shall be reported for each measurement.
5.2 Trueness
5.2.1 Overview
Trueness is the agreement between measurements of the CMD and an accepted value from a reference
standard or a reference method, which is a substitute for the “true” value. The uncertainty of
measurements of the reference methods shall be determined in accordance with ISO 21748. Trueness is
determined in:
— controlled laboratory tests with prepared challenge water (6.2);
— controlled laboratory tests with ambient water (6.3);
— field tests with water treated with a type-approved ballast water management system (BWMS)
(6.4).
For laboratory tests, organism concentrations are manipulated to yield three samples for each trial (at
each salinity), with concentrations below, approximately equal to, and above the DS, as defined in 4.4.
For field tests, samples of ballast water shall be collected during deballasting — after BWMS treatment
and neutralization processes (if performed) — in accordance with the requirements of ISO 11711-1 and
ISO 11711-2. Although organism concentrations are expected to be below the DS, concentrations are
not manipulated.
5.2.2 Measurement approach
CMDs can provide results as binary outcomes (e.g. “pass/fail”), numerical values, or both. All numerical
data available shall be recorded, but the analysis of trueness is based on categorical outcomes, where a
sample:
— exceeds the DS (“fail”): the sample exceeds the DS, has a high risk of exceeding the DS, or reports
−1 −3
concentrations ≥10 ml or ≥10 m , for example;
— meets the DS (“pass”): the sample meets the DS, has low risks of exceeding the DS, or reports
−1 −3
concentrations <10 ml or <10 m , for example;
— is indeterminate: the sample may exceed or meet the DS, but the determination cannot be made
with confidence, for example, the measurement is not statistically different from the DS.
Statistical analysis is performed by placing all the test outcomes into an error matrix. Figure 1 shows
a generic error matrix used to display measurement agreement between the reference method and the
CMD. Based upon the analysis by both the reference method and the CMD, samples are assigned into
one of nine test outcomes. Values in boxes show the tally of samples for each outcome, and the rows
(R), columns (C), and the entire table (n) are used to calculate Cohen’s kappa (κ) and other metrics, as
described below (5.1.3).
Key
DS discharge standard
∑R sum of row i, where i = 1, 2, or 3 (boxes with light grey shading)
i
∑C sum of column i , where i = 1, 2, or 3 (boxes with light grey shading)
i
n total number of samples (box with dark grey shading)
NOTE 1 Data are hypothetical and used only as an example for calculations. See Formulae (1) to (8).
NOTE 2 Samples in agreement are in boxes along the diagonal, marked with thick borders.
Figure 1 — Generic error matrix with example data
5.2.3 Statistical calculations
Cohen’s kappa (κ) is commonly used to determine measurement agreement, as it adjusts the overall
[10]
observed agreement (P ) to the expected agreement (P ), which is the agreement due to chance.
O E
Formulae (1) to (4) are used to calculate P , P , κ, the standard error of κ (σ ), respectively:
O E κ
∑A ()16++12 6 34
P = = ==07, 6 (1)
O
n 45 45
ΣΣRC⋅
  19⋅20 17⋅18 97⋅
ii      
+ +
       
3 n 84,,46++80 14,,0 16 64
 45   45   45 
 
P = = = == 03, 7 (2)
E ∑
i=1
n 45 45 45
PP−
07,,60− 37 03, 9
OE
κ = = ==06, 2 (3)
1− P 10− ,37 06, 3
E
PP1− 07,,61−076
() () 01, 8
OO
σ = = ==01, 0 (4)
κ
17,9
nP1− 45 10− ,37
() ()
E
where
ΣA is the sum of samples where the reference method and the CMD are in agreement;
ΣR is the sum of samples in row i (where i = 1, 2, or 3);
i
ΣC is the sum of samples in column i;
i
n is the total number of samples.
The confidence interval (I) of κ is based upon the chosen significance level (α). Formula (5) is used
to calculate I based on κ, σ , and z, which is the quantile of a normal distribution corresponding to a
κ
chosen α.
I =±κσ z ⋅ (5)
κ
Typical values of α (α = 0,1, 0,05, or 0,01) correspond to confidence intervals of 90 %, 95 %, or 99 % and
corresponding z values of 1,28, 1,65, or 2,33, respectively. For these z values, confidence intervals (I )
α
are calculated using Formulae (6) to (8), respectively.
I =±κσ 12, 8 ⋅ (6)
01, κ
I =±κσ 16, 5 ⋅ (7)
00, 5 κ
I =±κσ 23, 3 ⋅ (8)
00, 1 κ
NOTE The z values correspond to a one-tailed distribution, where z is determined from 1 – α. This is
appropriate for testing the hypothesis that the observed data exceed a defined threshold. For a two-tailed
distribution, z scores are determined at 1 - α/2. For typical values of α (α = 0,1, 0,05, or 0,01), α/2 equals 0,05,
0,025, or 0,005, and corresponding z values are 1,65, 1,96, or 2,58, respectively.
Values of κ ≥ 0,8 indicate strong agreement between the two methods, but lower thresholds for κ (e.g.
[11,12]
κ ≥ 0,6) can be acceptable in some cases. A set threshold allows a hypothesis test to determine
whether the observed κ (κ ) is significantly greater than the threshold κ (κ ). The number of samples
obs thr
required to determine whether κ exceeds κ depend upon the expected difference between these
obs thr
two values, where large sample sizes are required to detect small differences between κ and κ . An
obs thr
approach for determining samples sizes is described in 7.4.
5.3 Precision
5.3.1 Overview
Precision is determined from multiple readings of a single sample. At a minimum, precision is measured
using a simplified sample, such as the samples of challenge water (see 6.2).
NOTE 1 It is recognized that the end user can require samples of natural water or with a complex matrix, such
as ambient water samples or treated ballast water from a ship or from type approval testing. These samples can
be acceptable for precision, given concentrations remain stable throughout the analysis.
For each organism or organism size class, each trial consists of one sample below the DS and one sample
above the DS, according to 4.4.
NOTE 2 Samples with concentrations approximately equal to the DS are not used to measure precision.
5.3.2 Measurement approach
Samples for precision shall be prepared with sufficient volume to allow ≥10 subsamples for analysis
with the CMD and ≥2 subsamples for analysis using the reference method. Precision of the CMD is the
agreement among ≥10 subsamples from a single sample.
NOTE 1 As all numerical data reported by the CMD are recorded (5.1.2), end-users can additionally calculate
precision using numerical values, with the understanding that numerical comparisons of precision between the
CMD and the reference method are not expected to be comparable.
Analyses using the reference method shall be performed on ≥2 subsamples at the start and the end of
the set of CMD analyses to verify that:
— concentrations are within the target range (whether below or above the DS);
— concentrations remain stable throughout the trial.
A sample shall be well-mixed prior to collecting each subsample, which are analysed in succession to
minimize changes in the organism concentrations over time. Subsamples for the CMD and separate
subsamples for the reference method are collected synchronously, but analysis using the reference
method is only required for the first and last subsample of the CMD analysis.
NOTE 2 Though not required, more frequent sampling and analysis using the reference method can be useful,
for example, if the end-user requires a calculation of precision based upon numerical values. In this case, an
equivalent number of subsamples and analyses (e.g. ≥10 for both the CMD and the reference method) would
provide a direct comparison between the CMD and the reference method.
Organism loss or mortality can cause instability and drive concentrations below the DS (for ≥150 %
DS) or below the limit of detection of the reference method (for the ≤50 % DS) samples. If the status of
the sample changes, the trial shall be invalidated. If changes in organism concentrations invalidate a
trial, a reduced number of subsamples shall be analysed for each sample, given the number of samples
increases so that at ≥10 subsamples are collected and
...