SIST CR 14377:2002

SLOVENSKI STANDARD
01-maj-2002
.DNRYRVW]UDND3ULVWRSNRFHQMHYDQMXPHULOQHQHJRWRYRVWLUHIHUHQþQLKPHULOQLK
PHWRG]D]XQDQML]UDN
Air quality - Approach to uncertainty estimation for ambient air reference measurement
methods
Ta slovenski standard je istoveten z: CR 14377:2002
ICS:
13.040.20 Kakovost okoljskega zraka Ambient atmospheres
2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.

CEN REPORT
CR 14377
RAPPORT CEN
CEN BERICHT
January 2002
ICS
English version
Air quality - Approach to uncertainty estimation for ambient air
reference measurement methods
This CEN Report was approved by CEN on 10 November 2001. It has been drawn up by the Technical Committee CEN/TC 264.
CEN members are the national standards bodies of Austria, Belgium, Czech Republic, Denmark, Finland, France, Germany, Greece,
Iceland, Ireland, Italy, Luxembourg, Malta, Netherlands, Norway, Portugal, Spain, Sweden, Switzerland and United Kingdom.
EUROPEAN COMMITTEE FOR STANDARDIZATION
COMITÉ EUROPÉEN DE NORMALISATION
EUROPÄISCHES KOMITEE FÜR NORMUNG
Management Centre: rue de Stassart, 36  B-1050 Brussels
© 2002 CEN All rights of exploitation in any form and by any means reserved Ref. No. CR 14377:2002 E
worldwide for CEN national Members.

Contents
Foreword.4
1 Introduction .5
2 Assumptions and interpretations in this Report of the EC Air Quality Framework and Daughter
Directives.5
2.1 Annexes to the Daughter Directives .5
2.2 Definitions and interpretation of measurement uncertainty, level of confidence, and
confidence interval, within the Ambient Air Quality Daughter Directives .6
2.2.1 General.6
2.2.2 Uncertainty of measurement .6
2.2.3 Level of confidence .6
2.2.4 Relationship between confidence interval and measurement uncertainty .7
2.3 EC Terms of Reference .8
3 Approaches to uncertainty estimation .8
3.1 General.8
3.2 Guide to the Expression of Uncertainty in Measurement.9
3.3 International Standards ISO 5725 Parts 1 to 6.9
3.4 International Vocabulary of Basic and General Terms in Metrology.10
3.5 Other International Standards .10
4 Recommendations for the assessment of uncertainty of ambient air measurement methods .11
4.1 Introduction .11
4.1.1 General.11
4.1.2 Automated measurement methods.11
4.1.3 Non-automated measurement methods .12
4.1.4 Utilisation of traceable calibration standards and certified reference materials.13
4.1.5 Specification of the regime of field calibrations and QA/QC procedures.13
4.2 Consistent approach to the assessment of uncertainty in the Ambient Air Quality Daughter
Directives.13
4.2.1 General.13
4.2.2 General types of test procedures to be employed .14
4.2.3 Examples of the approach to uncertainty estimation given in this Report .18
4.2.4 Applicability of the general types of tests.18
5 Requirements for on-going quality assurance and quality control of field measurements
covered by the EC Directives .18
5.1 Overview of requirements.18
5.2 Specific requirements of the Framework Directive.18
5.3 Types of organisations involved.19
5.4 Requirement for traceability to national or international accepted standards .19
5.5 Demonstration of the competence of organisations involved with QA/QC activities .20
5.6 The role of quality assurance and quality control for the on-going field measurements.20
5.7 Procedures for the regular auditing of the performance of laboratories involved in QA/QC
activities.21
5.8 Conclusions.22
6 General philosophy for the development of future ambient air measurement methods .22
Annex A Principles of the Reference Methods prescribed in the EC Air Quality Daughter Directives .23
Annex B Data quality objectives contained in the Ambient Air Quality Daughter Directive covering
benzene and carbon monoxide .24
Annex C Reference measurement methods for the implementation of EU Air Quality Directives .25
Annex D Examples illustrating the approach specified in this Report for uncertainty estimation for
ambient air measurement methods .27
Annex E Definitions of measurement uncertainty terms used in this Report.35
Annex F Outline description of a methodology for a proficiency testing scheme which could be used
to underpin European harmonisation of ambient air quality measurements.36
Annex G Members of the Ad-hoc Group on an approach to uncertainty estimation for ambient air
measurement methods .38
Annex H Acknowledgements .39
Bibliography .40
Foreword
This Technical Report has been prepared by Technical Committee CEN/TC 264, "Air quality", the secretariat of
which is held by DIN.
This document is a working document.
This CEN Report has been prepared by an Ad-Hoc-Group of CEN/TC 264 "Air quality" in co-operation with the
European Commission’s Joint Research Centre, Ispra, Italy.
This CEN Report is an informative document.
1 Introduction
The European Framework Directive on Ambient Air-quality Assessment and Management, and its related Daughter
Directives, require measurements to be made using specified Reference Methods (given in outline in Annex A of
this Report), and the results to be reported, on specific air pollutants, with specified data quality objectives, by all
Member States within the EU. The main purpose of this CEN Report is to provide guidance to those CEN/TC 264
working groups which are involved in the preparation of Reference Methods to measure ambient air quality as
required by the EC Daughter Directives, on the uncertainty evaluation to be carried out in order to conform to these
data quality objectives. One further purpose of this Report is to emphasize the essential requirements for
appropriate quality assurance (QA) and quality control (QC) procedures in order to ensure that the ongoing field
measurements are valid – so as to ensure the EC data quality objectives continue to be met.
It is important to understand that these Daughter Directives specify the principles of the Reference Method to be
used, but they rely on the European standardisation body (CEN) to produce detailed Reference Method(s) for each
pollutant as European Standard(s), and to arrange for their publication so that they can be made available to
Member States. As noted above, the principles of these Reference Methods, which are to be used in the
implementation of the Directives, are prescribed in the relevant EC Directives. In some cases, these Methods are
specified as continuous or semi-continuous automated instruments, which comprise fully integrated systems,
usually including a sampling line and the analytical equipment. In other cases, the Reference Methods are
specified as manual or discontinuous methods, which comprise a field sampler and a separate laboratory analytical
component. In addition, it should be noted that in certain scenarios the Directives allow indicative methods to be
used, which are allowed to have poorer measurement uncertainties.
It is also a requirement of some of the Daughter Directives that Member States meet air quality limit values for a
variety of different averaging periods (covering e. g. hourly, daily, and annual averages). For some of the pollutants
specified, however, the requirement is only for annual averages.
The European Framework Directive on Ambient Air-quality, and the associated Daughter Directives, also state that
the approaches given in the Guide to the Expression of Uncertainty in Measurement published by ISO, and given in
the International Standards ISO 5725, are to be used for the estimation of the measurement uncertainties of these
Reference Methods. Both approaches are mentioned in these Directives as alternatives for the estimation of
measurement uncertainty. This Report describes, therefore, in broad terms, the methodology which should be
employed by the relevant CEN working groups for the integration of the approaches described in these documents
into European Standards, whilst still maintaining a valid overarching statistically-based methodology for determining
these measurement uncertainties.
The issues outlined above are covered in this Report. The Report was produced following consultations with an Ad-
hoc Group of experts which were convened to consider the issues. These experts are experienced in both ambient
air monitoring and in measurement uncertainty statistics. The members of this group and its Chairman are listed in
Annex G. It should be noted that all the existing working groups of CEN/TC 264, which are currently involved with
the standardisation of ambient air quality measurements, were represented on this Ad-hoc Group.
It should also be noted that Member States may use other methods, as alternatives to these Reference Methods –
provided they can demonstrate that these other methods produce results which have been shown to be equivalent
to those obtained by the relevant Reference Method. It is not, however, the remit of this Report to discuss the issue
of equivalence, or to define how it may be demonstrated.
2 Assumptions and interpretations in this Report of the EC Air Quality Framework and
Daughter Directives
2.1 Annexes to the Daughter Directives
Each of the Daughter Directives contains an Annex which specifies the data quality objectives required for the air
quality assessment for the particular pollutant species they address. These Annexes specify the ‘accuracy’ or
‘uncertainty’ that must be met by the Member States when reporting results, in order to satisfy the requirements of
the Directives. For clarity, it is assumed in this Report that the two terms accuracy and uncertainty are
synonymous, and therefore in the text of this Report reference will only be made to uncertainty, or to the
uncertainty of measurement, as defined in the Guide to the Expression of Uncertainty of Measurement (1993)
published by ISO – known hereafter in this Report as the GUM. An overview of the GUM is presented in clause 3.2
of this Report.
Included in each of the Annexes of the Daughter Directives, noted above, is an explanatory paragraph on their
accuracy (uncertainty) requirements, in which it is explained that “the Guide to the Expression of Uncertainty of
Measurements 1993, or ISO 5725-1, Accuracy (Trueness and Precision) of Measurement Methods and Results,
shall be used”. As an example of this, the relevant Annex of one of the (draft) Daughter Directives is reproduced in
Annex B of this Report, where it can be seen that the uncertainty values required of the Reference Methods for
benzene and carbon monoxide are listed.
It is also assumed in this Report that the GUM and ISO 5725 are not mutually exclusive, and that there is no
intention in the Daughter Directives to restrict the use by CEN and others to only one or the other. Instead, it is
proposed here that the principles of GUM is employed to identify and combine all uncertainties of the appropriate
Reference Method, and that ISO 5725 (including some of its six parts – not just the one part mentioned in the
Directives) may be used, where appropriate, as an aid to this evaluation (see clause 3 of this Report). It should also
be noted in this context that the GUM is a (voluntary) European Standard (ENV 13005).
It has also been necessary for the Ambient-air working groups in CEN/TC 264 to assume that the data quality
objectives of the Daughter Directives will be met, provided that the uncertainty of the measurement results
obtained with the Reference Methods are within the uncertainties prescribed by these data quality objectives. This
is also assumed in this Report, as it is judged that this is a valid assumption for the relevant European Standards.
There are also other statistical ISO standards that may be useful in this evaluation. Some of these are noted below
in this Report. It is assumed that the statements in the Daughter Directives do not preclude the use of any of these,
or of other relevant standards, where they are applicable, as long as the over-riding principles described in the
GUM are followed.
2.2 Definitions and interpretation of measurement uncertainty, level of confidence, and
confidence interval, within the Ambient Air Quality Daughter Directives
2.2.1 General
As discussed above, each of the EC Ambient Air Quality Daughter Directives specify data quality objectives
(quantified in their Annexes) that EU Member States must conform to, when reporting the results obtained to the
European Commission (EC). The definitions associated with measurement uncertainty that also relate to, or are
referred to, in the Annexes of these Daughter Directives are discussed below for, clarity, and also to serve as a
background to the remainder of this Report.
2.2.2 Uncertainty of measurement
The Air Quality Directives require an evaluation of the ‘uncertainty of the assessment (or measurement) methods’
according to GUM and therefore it is clear that the definition of uncertainty used in GUM shall be used. The GUM
definition of the term ‘uncertainty of measurement’ is therefore reproduced in this Report for completeness as:
Uncertainty of measurement: Parameter, associated with the result of a measurement, that characterizes the
dispersion that would reasonably be attributed to the measurand.
It is also essential to express clearly the conditions under which this uncertainty of the measurement results is to be
evaluated. This issue is also covered in this Report.
2.2.3 Level of confidence
The above term ‘uncertainty of measurement’ is used in the Annexes of the Daughter Directives. The term
‘confidence interval’ is, in addition, used in these Annexes of the Daughter Directives, and this is also defined in the
GUM. The relationship between the terms ‘confidence interval’ and ‘uncertainty of measurement’ is discussed
below in clause 2.2.4. However, it is first useful, for clarification purposes, to discuss the meaning of the term ‘level
of confidence’ which is used throughout these discussions.
As described in the GUM, the usual method of determining the level of uncertainty of a given measurement
parameter is to determine the combined standard uncertainty u of that measured parameter by combining all the
c
individual standard uncertainties, which may be Type A or Type B. It is then necessary to provide a measure of this
uncertainty that defines an interval about the measurement result that is expected to encompass a large fraction of
the distribution of values that could reasonably be attributed to the measurement. This is illustrated
diagrammatically in Figure 1. This measure of uncertainty is termed the expanded uncertainty, denoted by U, which
is obtained by multiplying the standard uncertainty by a coverage factor k:
Uk
u
c
As is well known, the result of the measurement is then traditionally expressed as:
Yy
U
This is interpreted to mean that the best estimate of the value attributed to Y is y and also that y–U to y+U is an
Y
interval that may be expected to encompass a large fraction of the distribution of the value.
More specifically, U may be interpreted as defining the interval about the measurement result that encompasses a
large fraction p of the probability characterised by that result and its associated uncertainty. This fraction p is then
the ‘level of confidence’ of that interval. Generally, and in the specific case of the requirements of the EC Air-quality
Daughter Directives this level of confidence p is defined as 95 % – i.e. 95 % of all the individual measurement
parameters lie within ±U of the best estimated value y and only 5 % lie outside of this interval.
It is clear, however, from this discussion that whatever level of confidence is actually used, it shall be stated
unambiguously, together with the result and its expanded level of uncertainty. It should also be noted that the
coverage factor k which is used as a multiplicand with u to obtain an appropriate level of confidence will, of course,
c
depend on the number of independent (uncorrelated) determinations of that measurand (i.e. the number of degrees
of freedom), which make up the probability distribution (as described in the GUM). Therefore k = 2 shall be used
only when there are a sufficiently large number of such degrees of freedom.
2.2.4 Relationship between confidence interval and measurement uncertainty
The relationship between uncertainty of measurement and the confidence interval I of a statistical distribution of
measurement results is illustrated, in an exemplar manner, in Figure 1, where
T

1 = – 1,96 0 is the lower confidence limit for a large number of degrees of freedom - expressed in this
example at a level of confidence of 95%
T

2 = + 1,96 0 is the upper confidence limit for a large number of degrees of freedom - expressed in
this example at a level of confidence of 95%

0 is the normally defined standard deviation (uncertainty) associated with the statistical
population distribution
The length of the confidence interval I defined in the GUM, for a large number of degrees of freedom expressed at
a level of confidence of 95 %, is then:
IT
T
21 ,96
I
Thus the standard deviation (uncertainty) associated with this confidence interval is:
I



21,96
This then enables the expanded uncertainty U associated with the confidence interval I to be expressed (for a large
number of degrees of freedom, expressed at a level of confidence of 95%) as:
UI
19,,6 
0 5
This interpretation is then fully consistent with the definition and use of the term ‘expanded uncertainty’ within the
GUM, and it is also consistent with the intentions within the relevant Annexes of the Air Quality Daughter Directives.
As noted above, the key terms which are used to describe the statistical procedures associated with measurement
uncertainty are defined in Annex E of this Report.
I = 2  1,96  

1 -



n 

T  -  + T
 
1 2
Figure 1 — Graphical illustration of the uncertainty distribution
of a measurement quantity derived from repeated observations
2.3 EC Terms of Reference
The EC has produced certain Terms of Reference (ToR) which specify requirements for the contents of European
Standards covering ambient air quality, and also to assist in their preparation. The latest draft of these is
reproduced in Annex C of this Report. It should be noted that, although these ToR are still draft, and thus are liable
to revision, they will retain the same under-lying principles given in the current draft. It should also be noted that
because different types of instruments or equipment are available to measure air quality, the EC ToR allow for the
type-approval of instruments, and therefore any instruments which operate on the same principle as those stated
as meeting all the requirements of the Reference Method prescribed in the relevant European Standard are also
acceptable – as long as the latter have been validated by demonstrating that they conform to all the relevant
performance criteria applicable to the Reference Method, and also to the overall requirements of the data quality
objectives in the appropriate EC Daughter Directive.
The Daughter Directives also contain air-quality concentration limit values set by the European Commission. These
are expressed as limits to be met over stated temporal averaging periods, and some of these may be as long as a
year. For discontinuous methods this poses potential difficulties for the development and validation of the relevant
European Standards, not only with respect to the sampling method, but also in evaluation of the uncertainty of the
results of the measurements. In these cases, it should be assumed that the uncertainty of the Reference Method,
which is derived for the shorter averaging period used during the laboratory and field validation trials, applies to the
longer averaging times specified in the Directives. This, although not completely rigorous, is a pragmatic solution.
However, in these circumstances the longest averaging times which it is practical to employ during the laboratory
and field trials should be used.
3 Approaches to uncertainty estimation
3.1 General
As noted above, the Air Quality Daughter Directives place specific data quality objectives on the uncertainty of the
measurements carried out. This uncertainty of the measurement result is expressed in terms of a maximum
allowable percentage of the air quality limit value(s), and it should be calculated with respect to the stated
averaging period(s) referred to in the Directives. The ToR also state that these data quality requirements
(‘objectives’ in Daughter Directives) are criteria against which the Reference Method is assessed using a
combination of laboratory and field tests. The primary concern is therefore to ensure that the assessment of
uncertainty of measurement that is used within the CEN/TC 264 standards to derive uncertainties from these
assessment tests, allows valid comparisons to be made with the Directives’ objectives, both in terms of the
concentration levels actually present, the prescribed limit values, and the associated averaging periods. This has
implications for the subsequent use of the Reference Methods and the on-going quality assurance and quality
control (QA/QC) regime which is adopted, as knowledge of these will be necessary to derive, for example, an
uncertainty on a yearly average when the assessment tests are conducted over a matter of weeks or months.
The Daughter Directives allow the use of the GUM or the International Standards ISO 5725 (or equivalents) to
assess the measurement uncertainty of the Reference Methods. It could be argued that these two approaches are
not compatible. However, this is not necessarily correct, and the two procedures may be used in such a way that
they are complimentary. This approach is explained further in clause 4 of this Report. However, it is useful first to
review briefly the published documents covering uncertainty of measurements that are most relevant to this Report.
3.2 Guide to the Expression of Uncertainty in Measurement
The GUM provides the general concept for the harmonised estimation of measurement uncertainties, agreed on
and adopted by important organisations in the field of general physical metrology and analytical chemistry (e.g.
Bureau International des Poids et Measures (BIPM), International Union of Pure and Applied Chemistry (IUPAC),
International Union of Pure and Applied Physics (IUPAP), International Electrotechnical Commission (IEC), and
International Organisation of Legal Metrology (OIML)), and published by ISO. In overview, the GUM provides a
procedure for assessing the combined uncertainty of a measurement method by assessing and combining all
potential sources of uncertainty. In this procedure it is necessary to:
 Establish a model equation (or measurement equation) and a related uncertainty budget which should list fully
all potential sources of uncertainty. It is also important at this stage to consider carefully the identified
uncertainty contributions of the model equation, so as to ensure that there is no double counting and that any
correlations between uncertainty sources are understood.
 Quantify the individual sources of uncertainty, as standard uncertainties (standard deviations), either by
experiment and statistical analysis of repeated measurement (Type A uncertainties) or by other evaluation
(generally an assumption about the likely distribution describing the uncertainty of the source).
NOTE The above methodology can, in principle, be implemented in all circumstances. However, in some cases,
particularly those involving a complex measurement equation, it may not be practical to establish and evaluate all the
uncertainty components of the measurement equation, and in these cases some components of the uncertainty budget
might potentially be ignored. In these circumstances, the uncertainty evaluations described in clause 3.3 of this Report,
which involve International Standards ISO 5725, may be used, where appropriate, to check and evaluate whether there
are any neglected components of the uncertainty that are of significant magnitude. Any additional components identified
by these means should be incorporated into the uncertainty of the measurement results, in an appropriate manner
according to GUM. Where this is not practical, the field evaluations prescribed by the International Standard
prEN ISO 14956 should, where possible, be used to establish appropriate (additional) components in the uncertainty
budget (see clause 4.1 of this Report).
 Combine the individual uncertainties, taking into account the sensitivity coefficients of each component
determined from the partial differential of the model equation, or by experiment. If correlations are present they
should also be taken into account at this stage (according to methods described within the GUM).
 Expand the combined uncertainty using a coverage factor (usually by a factor derived from the student-t
distribution using the appropriate number of degrees of freedom) to express the expanded uncertainty at a
stated level of confidence, which in the case of the Daughter Directives is 95 %. Note that in this case, as the
model equation will, by definition, match the averaging time and limit values required in the Daughter Directive,
the concept of bias and standard deviation used in certain versions of the text in the Daughter Directives is not
required.
3.3 International Standards ISO 5725 Parts 1 to 6
International Standards ISO 5725 Parts 1 to 6 provide tools and methods for planning, conducting, and analysing,
the results obtained from inter-laboratory studies, which aim to characterise the accuracy of measurement methods
and measurement results. Part 5 of ISO 5725 is the main standard relevant to this Report, but Part 1 (definitions)
and Parts 3 and 4, are also of some relevance. These standards distinguish between:
 repeatability: variations of results within a laboratory using identical methods, equipment, operators etc;
 reproducibility: variations between different laboratories with differing equipment and/or different operators etc.
NOTE: Reproducibility does not define specifically which variables shall be changed and, therefore, this term is not
completely specific. In any specific context therefore the term reproducibility should specify which variables are changed.
These International Standards do not use the term ‘uncertainty’, using instead ‘accuracy (trueness and precision)’.
However, the results obtained from the application of these standards may be used to provide information, which
can then be applied in an over-arching GUM evaluation. ISO 5725 is particularly suited to the assessment of
information on the uncertainty of analytical methods, where complex matrices or large numbers of specific
laboratory-related uncertainty sources mean that it is impractical (or possibly too expensive) to assess, individually,
the contributions of all these sources of uncertainty to the overall uncertainty estimate. Instead, these may be
evaluated by means of a series of well-designed inter-laboratory evaluations as given in ISO 5725-5. The relevant
ISO 5725 investigation should be carried out as a part of the preparation of any European Standard where it is
deemed appropriate to adopt this approach. The information obtained through its application may then be
combined with the information on other potential uncertainty contributions (Type A and Type B) using the GUM
approach. Care is needed, however, in using the results obtained from an ISO 5725-5 study in a GUM evaluation,
so as to ensure that all measures of precision and trueness are expressed correctly in terms of variances and
covariances, and also to avoid incorrect duplicate counting of sources of uncertainty. It should also be noted that
under ISO 5725 a large number of laboratories and test samples may be required to give an acceptable level of
confidence for the derived uncertainties. It is important therefore that any study is large enough to handle correctly
the expected number of influence variables.
This methodology should also enable subsequent (‘equivalent’) measurement methods, which meet the
requirements of the standard, to be adopted. However, these equivalent methods may also require an assessment
of their uncertainties using ISO 5725-5 (or one using the GUM approach, if this is more appropriate) to be carried
out by the user - in order to demonstrate compliance.
3.4 International Vocabulary of Basic and General Terms in Metrology
The International Vocabulary of Basic and General Terms of Metrology (known as the VIM) was first published by
ISO in 1984 under the aegis of BIPM, IEC, ISO and OIML, and was re-issued by ISO in 1993. It is a useful addition
to the GUM, in that it contains the definitions of the key terms used in this Report (some of which are listed in
Annex E).
3.5 Other International Standards
A number of other International Standards provide tools that may be used within a GUM approach to assessing the
uncertainty of a measurement result. These include:
 ISO 9169 Performance characteristics of air quality measuring methods, with its associated standard ISO
6879 Performance characteristics and related concepts for air quality measuring methods, which are both
being revised. This defines laboratory (and field) tests that may be used to determine certain performance
characteristics of air quality measurement methods.
 prEN ISO 14956 Evaluation of the suitability of a measurement method by comparison with a stated
measurement uncertainty, which details the procedure used to calculate the combined uncertainty of the
measurement method from its performance characteristics, specifically related to the air quality measuring
methods given in the (draft) standards ISO 9169 and ISO 6879 (including, for example, the treatment of the
component of measurement uncertainty of the results which arises from interference by other species that may
be present during field monitoring). It also includes a requirement for field tests which are used to establish the
validity of the measurement uncertainty derived from the laboratory tests. This is being progressed as a dual
CEN/ISO standard. The DIS is under revision, to allow for the fact that the variance of the result of a
measurement is, in general, a weighted sum of the variances and the covariance of all the influence variables.
This is necessary in order to make prEN ISO 14956 fully consistent with the GUM.
 ISO/DIS 11222: Determination of the uncertainty of the time average of air quality measurements. In
general, this Draft International Standard is applicable to quantifying the uncertainty of, for example, daily,
monthly, or yearly average (concentration) values of air quality monitoring data where this has sampling times
that are shorter than the required averaging time period. The standard may be used to determine the
uncertainties of long-term average values, from data obtained from validation and calibration procedures
performed over short times. The input data required to enable this standard to be applied is the uncertainty
information attached to all of the QA/QC procedures carried out – i.e. the uncertainties in validation, calibration
and drift control procedures as well as inter-laboratory comparisons. The uncertainties in the average values
obtained by using ISO/DIS 11222 are quantified either as a (combined) standard uncertainty or as an
expanded uncertainty at a stated level of confidence.
4 Recommendations for the assessment of uncertainty of ambient air measurement
methods
4.1 Introduction
4.1.1 General
The assessment of measurement uncertainty shall be based on the approach described in the GUM. The specific
approach to the employed will in detail, however, depend on the type of the measurement method to be employed
to cover the selected ambient air pollutant(s). A summary of the principles of the Reference Methods to be used is
given in Annex A. These methods can be generally divided into automated and non-automated methods, and the
uncertainty estimation procedures will differ to some degree between these, as outlined below.
4.1.2 Automated measurement methods
On-line, automated measurement procedures may be employed where it is practical for the complete measurement
system to be located in the field – i. e. where the results are obtained using a ‘black box’ with its own sample inlet,
which is calibrated and checked at regular intervals. For these systems, possible variations are restricted to the
make of the instrument used, and to the artefacts used for calibration (although for example, in the case of
automated benzene measurements which use gas chromatography, the sampling procedure and the column of the
gas chromatograph may also be varied). In these cases, the uncertainty of the results obtained using automated
methods should be considered to be covered mainly by:
 Performing a set of (mainly) laboratory-based type-approval tests on all the various types (manufacturers and
models) of instrumentation which utilise the principles of the reference measurement technique(s) prescribed
in the appropriate European Standard(s) - in order to assess the intrinsic uncertainty contribution associated
with the results of the measurements obtained with this type of instrumentation. These tests should
encompass all performance criteria that contribute significantly to the uncertainty of the results, as specified in
the relevant European Standard.
 Ensuring that the type-approval tests are designed so that they are as comprehensive and rigorous as
practical. It should be noted, however, that since these tests are surrogates for actual field conditions, it may
not always be possible to achieve this fully. For example, it may sometimes be difficult to mimic, in full, the
effects on the measurement results of transient fluctuations of the influence variables. Therefore, to overcome
this potential limitation the test procedures should be specified so as to be as comprehensive as possible
within the relevant European Standard, and then in addition, well designed field trials should be carried out as
noted below (and prescribed by prEN ISO 14956) to establish whether this uncertainty is robust under field
conditions;
 Ensuring that all the tests are carried out with sufficient rigour (accuracy, precision, linearity etc) so that
uncertainties arising from the test procedures themselves do not contribute significantly to the uncertainty of all
the results obtained during the type-approval procedure. (For example, if a continuous monitor is required by
the relevant European Standard to have a linearity of ±2 % relative value, then the tests which are carried out
to establish this should be such that if they themselves have any non-linearities, these should be demonstrably
smaller than this - by a significant factor).
 Carrying out a set of well-designed field trials, with each type of instrumentation, so as to confirm that the
measurement uncertainty derived from the type-approval testing is valid when the automated method is used
in the field (as prescribed in prEN ISO 14956).
In the above circumstances, it is essential that, when these automated measurement methods are deployed
subsequently in the field, very rigorous and comprehensive QA/QC procedures are specified to ensure, as far as
achievable, that the uncertainty of measurement derived during these type-approval test procedures continues to
be valid for all results obtained from these measurement systems operating in field conditions. These QA/QC tests
in the field should cover, inter alia, those components of the complete Method which are the most difficult to control
within the type-approval tests, and also should target on any components of the method which are excluded by
automated methods for type-approval (e. g. the sample lines and manifolds of the monitoring stations with
automated measurement methods – which are excluded by the EC ToR). These requirements are discussed
further in clause 4.1.4 and 5.
4.1.3 Non-automated measurement methods
For non-automated measurement procedures, consisting of separate sample and analysis steps, it is recognised
that the procedures are of a composite nature – i. e. they may be built up of elements in which significant variations
may exist between organisations carrying out the analyses – due, for example, to different instrumentation,
operators, analytical chemicals, calibration artefacts etc.
For these ‘composite’ measurement procedures, a complete set of type-approval tests will generally not be
practical. Instead, these tests may be applied only to certain parts of instrumentation - such as a sampling pump
where the accuracy of flow over the specified sampling period can be validated - or with a sampler where the
sampling efficiency, blank level etc, can be assessed. In these circumstances therefore, the approach to
assessment of uncertainty may comprise three steps (a, b, and c, see below), depending on the pollutants or the
Working Group concerned.
a) The performance of the method is tested by one/individual laboratories applying the procedure using a series
of tests against pre-specified criteria:
 assessing the standard uncertainty of all the various elements of the measurement procedure that
contribute to the uncertainty of the measurement result;
 and using appropriate within-laboratory control and measurement performance so as to ensure that this
does not affect the test results;
b) The performance of a collaborative test should be carried out following the guidelines in ISO 5725-5, in which
as many of the ‘composite’ elements of the between-laboratory variations as practicable are included in the
determination. This collaborative test should also, where applicable, include both sampling and analysis.
)
Where necessary these tests should be carried out at different concentration levels in order to determine the
influences on the intrinsic method uncertainty of the different pollutant concentrations. In this case, the
comparison should be organized as follows:
 A sufficiently large number of pre-selected laboratories (generally >5) will perform a minimum of six
)
measurements each, at a number of concentrations that are representative of the expected
concentration range.
 The participating laboratories should demonstrate in advance that they are, for example, proficient in the
performance of the required sampling and/or analyses (in terms of, for example, their limits of detection,
capability to analyse certified reference materials or calibration standards with the required accuracy etc).
The sampling and/or analyses should be conducted according to a strict protocol which conforms to
appropriate European quality assurance standards.
 Samples should be stored until analysis under conditions that do not affect the sample stability, and
should be analysed under within-laboratory reproducibility conditions (e. g. separate calibrations for each
sample samples from different levels may be combined). Details of calibration procedures, and
determinations of any desorption efficiency, should also be recorded according to the requirements of the
protocol, in order to enable valid evaluations of the effects of the between-laboratory uncertainties to be
carried out by independent qualified organisations.
...