SIST-TS CEN/TS 17171:2018

SLOVENSKI STANDARD
01-oktober-2018
9RGHQMHRSD]RYDQLKKLGURPHWULþQLKSRGDWNRY1DYRGLOR
Management of observed hydrometric data - Guidance
Mangement gemessener hydrometrischer Datensätze - Empfehlungen
Gestion des données hydrométriques observées - Recommandations
Ta slovenski standard je istoveten z: CEN/TS 17171:2018
ICS:
07.060 Geologija. Meteorologija. Geology. Meteorology.
Hidrologija Hydrology
35.240.70 Uporabniške rešitve IT v IT applications in science
znanosti
2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.

CEN/TS 17171
TECHNICAL SPECIFICATION
SPÉCIFICATION TECHNIQUE
August 2018
TECHNISCHE SPEZIFIKATION
ICS 07.060; 35.240.70
English Version
Management of observed hydrometric data - Guidance
Gestion des données hydrométriques observées - Mangement gemessener hydrometrischer Datensätze -
Recommandations Empfehlungen
This Technical Specification (CEN/TS) was approved by CEN on 26 February 2018 for provisional application.

The period of validity of this CEN/TS is limited initially to three years. After two years the members of CEN will be requested to
submit their comments, particularly on the question whether the CEN/TS can be converted into a European Standard.

CEN members are required to announce the existence of this CEN/TS in the same way as for an EN and to make the CEN/TS
available promptly at national level in an appropriate form. It is permissible to keep conflicting national standards in force (in
parallel to the CEN/TS) until the final decision about the possible conversion of the CEN/TS into an EN is reached.

CEN members are the national standards bodies of Austria, Belgium, Bulgaria, Croatia, Cyprus, Czech Republic, Denmark, Estonia,
Finland, Former Yugoslav Republic of Macedonia, France, Germany, Greece, Hungary, Iceland, Ireland, Italy, Latvia, Lithuania,
Luxembourg, Malta, Netherlands, Norway, Poland, Portugal, Romania, Serbia, Slovakia, Slovenia, Spain, Sweden, Switzerland,
Turkey and United Kingdom.
EUROPEAN COMMITTEE FOR STANDARDIZATION
COMITÉ EUROPÉEN DE NORMALISATION

EUROPÄISCHES KOMITEE FÜR NORMUNG

CEN-CENELEC Management Centre: Rue de la Science 23, B-1040 Brussels
© 2018 CEN All rights of exploitation in any form and by any means reserved Ref. No. CEN/TS 17171:2018 E
worldwide for CEN national Members.

Contents Page
European foreword . 5
Introduction . 6
1 Scope . 7
2 Normative references . 7
3 Terms and definitions . 7
4 Principles of hydrometric data management . 8
4.1 The requirement for data management . 8
4.2 The sequence of data management . 8
4.3 Duty of care . 9
4.3.1 General . 9
4.3.2 Legacy data . 9
4.4 Data types and terminology . 10
4.5 Time series data . 10
4.5.1 General . 10
4.5.2 Time of observation . 10
4.5.3 Derived data terminology . 12
4.5.4 Resolution of data storage . 12
4.5.5 Uncertainty . 13
4.6 Metadata and descriptive material . 13
4.7 Maximizing data utility . 13
4.7.1 Availability of metadata . 13
4.7.2 Data quality control . 14
4.7.3 Data gap infilling . 15
4.7.4 Regular data reviews . 15
4.8 Data storage considerations . 15
4.8.1 Data storage . 15
4.8.2 Changes to data . 15
4.8.3 Data disposal . 15
4.8.4 Electronic data storage system considerations . 16
4.8.5 Tapes, charts and other data in paper format . 16
4.8.6 Data security . 16
4.8.7 Data rescue . 17
4.9 Archiving hydrometric data: national collections. 17
4.10 Electronic transfer of data . 17
5 Metadata . 18
5.1 General . 18
5.2 Monitoring station/site/point metadata . 18
5.2.1 General . 18
5.2.2 Station identifier . 19
5.2.3 Station description . 19
5.2.4 Geographical location . 20
5.2.5 Operating period . 20
5.3 Observation metadata . 20
5.3.1 General . 20
5.3.2 Data set metadata . 20
5.3.3 Data flags and comments . 20
6 Precipitation data . 21
6.1 Raw data . 21
6.1.1 Data to be recorded. 21
6.1.2 Additional raw data for manual observations . 21
6.2 Resolution of data storage . 21
6.3 Data processing and formatting . 21
6.3.1 General . 21
6.4 Quality control of precipitation data . 21
6.4.1 General . 21
6.4.2 Levels of quality control of precipitation data . 21
6.4.3 Methods for quality control of precipitation data . 22
6.4.4 Quality control of frozen precipitation . 22
6.5 Precipitation specific metadata . 22
7 Water level data . 23
7.1 Raw data . 23
7.1.1 Data to be recorded. 23
7.1.2 Types of level data . 23
7.1.3 Regular measurements and event data . 23
7.1.4 Additional raw data for continuous measurements . 24
7.1.5 Groundwater levels and water levels in reservoirs . 25
7.2 Quality control of water level data . 25
7.2.1 General . 25
7.2.2 Levels of quality control of water level data . 25
7.2.3 Methods for quality control of water level data . 25
7.2.4 Data estimation and infilling . 26
7.3 Water level specific metadata . 27
8 Velocity and discharge data . 27
8.1 Raw data . 27
8.1.1 Data to be recorded. 27
8.1.2 Additional raw data for irregular/spot measurements . 28
8.1.3 Additional raw data for regular or fixed site measurements . 28
8.2 Data processing . 29
8.2.1 General . 29
8.2.2 Discharge data derived from water levels . 29
8.2.3 Discharge data from time of flight ultrasonic systems . 30
8.2.4 Discharge data from velocity index method . 31
8.2.5 Discharge data from electromagnetic systems . 31
8.2.6 Combining discharge data from multiple methods . 31
8.3 Resolution of data storage . 31
8.4 Derived data . 31
8.4.1 Daily mean flow . 31
8.4.2 Peak discharges . 32
8.5 Naturalization of discharge data . 32
8.6 Quality control of discharge data . 32
8.6.1 Levels of quality control of discharge data . 32
8.6.2 Methods for quality control of discharge data . 32
8.6.3 Review of extreme flows . 33
8.6.4 Data estimation and infilling . 34
8.7 Discharge specific metadata . 36
8.7.1 Additional monitoring station/site/point metadata . 36
8.7.2 Additional observation metadata . 36
9 Volume data . 37
9.1 General . 37
9.2 Level/storage relationships . 37
9.3 Frequency of sampling and processing . 37
9.4 Resolution of data storage . 37
9.5 Volume data specific metadata . 37
Bibliography . 38

European foreword
This document (CEN/TS 17171:2018) has been prepared by Technical Committee CEN/TC 318
“Hydrometry”, the secretariat of which is held by BSI.
Attention is drawn to the possibility that some of the elements of this document may be the subject of
patent rights. CEN shall not be held responsible for identifying any or all such patent rights.
According to the CEN/CENELEC Internal Regulations, the national standards organisations of the
following countries are bound to announce this Technical Specification: Austria, Belgium, Bulgaria,
Croatia, Cyprus, Czech Republic, Denmark, Estonia, Finland, Former Yugoslav Republic of Macedonia,
France, Germany, Greece, Hungary, Iceland, Ireland, Italy, Latvia, Lithuania, Luxembourg, Malta,
Netherlands, Norway, Poland, Portugal, Romania, Serbia, Slovakia, Slovenia, Spain, Sweden, Switzerland,
Turkey and the United Kingdom.
Introduction
Water management decisions and policies ought to be based upon quantitative knowledge of the
hydrological system. Commonly, such knowledge results from observational hydrometric data, the
collection of which is the subject of other standards, e.g. EN ISO 18365. The subsequent management of
such hydrometric data provides the linkage between field measurement and the eventual use of
processed data to address a wide range of strategic and operational water management applications. As
both the demand for and complexity of hydrometric data increase, it is important that the procedures
and processes used to manage these data are standardized to allow greater integration of data and
ensure their protection for future use.
The availability of high-quality observational data are vital to developing an understanding of the
hydrological cycle. Optimizing data management systems helps ensure that the maximum benefits are
achieved from those resources invested in hydrometric monitoring. Effective standardized procedures
for data transmission, manipulation, quality control, expression of uncertainty and storage are vitally
important and their use should be promoted throughout hydrometric observation networks.
Those responsible for hydrometric data management are encouraged by this Technical Specification to
adopt the ethos of professional stewardship and to remember their role as guardians of an important
national, and sometimes international, resource.
This Technical Specification is designed for use by all organisations and individuals collecting,
processing or storing hydrometric data. Some of the clauses contained in standard are only applicable
for those maintaining national or regional collections of hydrometric data (for example, 4.9). However
most recommendations are widely applicable to all users, including organisations, companies or
individuals involved in: hydropower production, water supply, environmental protection, scientific
research or flood risk management.
This Technical Specification is concerned with general aspects of good practice in data management.
Techniques for managing data are recommended, covering metadata collection, data storage and
quality control. This Technical Specification assumes that the raw data have been collected and
transmitted from the field in line with other European Standards for hydrometry, so this Technical
Specification concentrates on the subsequent processing and management of these hydrometric data.
1 Scope
This document gives recommendations for the management of observed hydrometric data, including
raw data and other data as well as statistics derived from these observations. Although the principles of
data management can be applied to all hydrometric observations, particular focus is placed on
measurements of precipitation, water level (including stage), volume and discharge in open channels.
NOTE The range of sites where water levels, and sometimes flow, are measured includes lakes, reservoirs,
rivers, canals, tidal waters, sewers, wells, and boreholes.
The document covers metadata associated with hydrometric data, including recommendations for the
production and management of descriptive, analytical and statistical material relating to sites where
and measuring techniques, by which hydrometric data are collected. The recommendations of this
document can be applied to some forms of data directly derived from observational records (for
example, summary time series of monthly mean river flows). While not primarily designed for the
management of data resulting from more complex numerical models or spatially aggregated data sets
(for example, remotely-sensed data), many of the recommendations are applicable for such types of
data.
This document does not cover the field collection of data or its transmission, but focuses on the
management of data once they have been received in a hydrometric information management system.
2 Normative references
The following documents are referred to in the text in such a way that some or all of their content
constitutes requirements of this document. For dated references, only the edition cited applies. For
undated references, the latest edition of the referenced document (including any amendments) applies.
EN ISO 772, Hydrometry — Vocabulary and symbols (ISO 772)
3 Terms and definitions
For the purposes of this document, the terms and definitions given in EN ISO 772 and the following
apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— IEC Electropedia: available at http://www.electropedia.org/
— ISO Online browsing platform: available at http://www.iso.org/obp
3.1
data flag
indicator relating to the quality and characteristics of an observation
3.2
derived data
information calculated, or deduced, from raw data (3.5)
3.3
precipitation
water or ice derived from the atmosphere and deposited at ground level
Note 1 to entry: Measured in terms of the depth in millimetres (mm) of its liquid equivalent.
3.4
quality control
process of confirming that the data held are a reliable representation of the variable being measured or
derived
3.5
raw data
data resulting directly from the measurement of variables
3.6
UTC
Coordinated Universal Time
4 Principles of hydrometric data management
4.1 The requirement for data management
Hydrometric data management systems should be designed such that the utility of data to end-users at
all levels is a central consideration. The value of data can be significantly affected by the treatment of
data after collection, so data management systems should be designed to maintain (and where possible
improve) the quality and continuity of records.
As water management decisions and policies are based upon quantitative knowledge of the
hydrological system both now and in the past, the protection of data for future use should be a central
requirement of any data management system. As such, the design of such systems should consider, as
far as possible, both current and future user requirements.
4.2 The sequence of data management
The management of hydrometric data is a continuous sequence (see Figure 1), which starts in the field
at the point of collection and ends if and when data are destroyed. The design of any hydrometric data
management system should consider all stages in this sequence.
The management of data at the point of collection and their subsequent transmission vary depending
on the hydrological variable being recorded, hydrometric methods employed and instrumentation used.
Any data management system used should ensure that adequate data and metadata are recorded at the
point of collection to allow future management and use.
Some data management processes, such as quality control, may occur at more than one stage in the
sequence and be completed by more than one organization. In designing, operating and reviewing data
management systems, all stages in the acquisition, processing and dissemination of data should be
considered.
Figure 1 — Data management sequence
4.3 Duty of care
4.3.1 General
Data often have significant value outside the organization that collected them, and data management
systems should be designed, where possible, to capture all information required for future, sometimes
unforeseen, uses.
Responsibility for the operation of hydrometric monitoring systems may be transferred (for example, to
another organization or other personnel), so data management systems should be capable of preserving
all data and associated metadata required to inform new operators.
4.3.2 Legacy data
The duty of care to manage hydrometric data should be taken to cover any such data currently held by
an organization. In addition to data collected under the direct control of the organization, the duty of
care extends to data collected by predecessors and inherited.
An assessment should be made of all legacy data upon inheritance (or discovery) to determine their
contents and current state. An assessment should be made of their current, and likely future, utility to
the organization and other external parties. While the management (and improvement) of legacy data
can be demanding, the value of the resulting information can be high.
NOTE Long historical records, even those of lower hydrometric quality, can be particularly valuable in
providing an indication of previous hydrological conditions and often improve the value of current data through
extending record lengths.
All legacy data should be actively managed. Where possible, such data should be brought into current
data management systems and managed in accordance with 4.5 to 4.10 and Clause 5 to Clause 9.
Where legacy data are not integrated into current data management systems, they should be secured in
archives maintained either by the organization which collected/inherited them or by independent
bodies (for example, national archives, libraries).
Where the quality of legacy data (including inherited data) are poor, efforts should be made to improve
it. Special attention should be paid to the collation and storage of metadata (both hard copy and digital)
associated with the legacy data. In the case of discharge data, current meter gauging records and details
of derived rating relationships are of particular value and should be maintained. Care should be taken to
preserve operational catchment management information, for example, sluice gate operation, reservoir
releases and resiting of raingauges, which is often vital for understanding apparent anomalies in
hydrometric data.
Where data are considered “at risk” (for example, due to physical deterioration or storage in obsolete
systems/formats) then consideration should be given to a programme of “data rescue” (see 4.8.7).
Due to the difficulty in assessing future, currently unknown, requirements for hydrometric data, legacy
data and associated information holdings should normally be preserved even if there is no immediate
requirement to use the data.
NOTE Where data are deemed surplus to requirements, their archival (alongside metadata describing their
lineage) is highly preferable to disposal. Recommendations for data disposal are given in 4.8.3.
4.4 Data types and terminology
For the purposes of 4.5 onward, hydrometric data should be considered to include:
a) time series of continuous or regular observations of a hydrological variable at a point-in-space;
b) sporadic or single point-in-time observations of a hydrological variable at a point-in-space.
Both of these types of operational data normally have a third type of hydrometric data associated with
them:
c) metadata (including, discovery metadata and temporal metadata) and other descriptive material
which provide information about the measurement, hydrological conditions and/or subsequent
processing of hydrometric data.
Hydrometric data should also be taken to include time series and statistics derived from these data
types. Typical examples of such hydrometric data include: total monthly rainfall, mean annual flood and
Q (the flow equalled or exceeded 95 % of the time).
NOTE Spatially aggregated data (e.g. data produced by remote sensing or areal rainfall estimates) are not
covered by this Technical Specification as their nature and management often differ from those used for other
hydrometric data.
4.5 Time series data
4.5.1 General
For the purposes of this Technical Specification data types a) and b), as well as time series subsequently
derived from such data, are considered to be time series data.
4.5.2 Time of observation
All data points in a time series should be associated with a single point-in-time or period of time (e.g. for
integrations), either by:
a) recording the start point-in-time and interval of a regular time series; or
b) recording the point-in-time associated with every value in a time series.
The representation of dates and times should conform to ISO 8601. When storing raw data, the
standard Gregorian calendar conventions should apply, both to the year (1 January to 31 December)
and day (midnight to midnight). To avoid gaps or overlaps in the records caused by biannual daylight-
saving time adjustments, all times should be recorded in UTC (or to a consistent offset from UTC
throughout the database which is clearly recorded in the metadata). When recording an observation
made at the instant dividing one calendar day from the next, then the time should be given as 00:00:00
(i.e. the start of the new calendar day).
Raw data should be recorded together with the point-in-time at which the observation was made.
Therefore, instantaneous observations should be recorded against the point-in-time the measurement
was taken, and accumulations (e.g. rainfall totals) should be recorded against the end point of the
accumulation.
Processed data (including derived data representing integration over a set period of time, e.g. daily
mean flow or monthly rainfall total) may be recorded against either the start or end of the period in
question. The convention used should always be clearly specified in the metadata associated with the
record or a related data dictionary.
In many European countries, a commonly used convention is to calculate derived data using a
hydrological day/month/year (otherwise known as water or rainfall days/months/years) in addition to
(or instead of) using the calendar day starting 00:00:00. The following norms apply when deriving data
using such conventions:
1) hydrological day = observations made in the 24-h period starting at a given time, recorded against
the start point of the period;
2) hydrological month = observations made during the hydrological days of that month; and
3) hydrological year = observations made during a year of hydrological days starting on a given date,
recorded against the end year of the period.
The starting time for the hydrological day and date of the start of the hydrological year may vary
between countries. Examples of the local conventions in use within Europe are given in Table 1.
Whether using these periods or others, the convention applied should be clearly specified in the
metadata associated with the record or a related data dictionary.
Table 1 — Examples of hydrological day/month/year conventions used in Europe
Hydrological Year
Countries using convention Start of Hydrological Day
(all dates are inclusive)
United Kingdom of Great Britain
09:00 UTC 01 October to 30 September
and Northern Ireland
Italy 08:00 UTC
Slovakia 00:00 UTC 01 November to 31 October
Sweden 06:00 UTC 01 October to 30 September
Portugal 00:00 UTC 01 October to 30 September
While such conventions involve presenting processed daily data (for example, total daily rainfall or
daily mean flow) for the hydrological day, the raw data from which these time series are produced
should always be stored using the normal calendar day. For example, using the above UK convention,
raw hourly rainfall total data recorded at 06:00 on 13 April would be recorded against that time in the
raw data series but subsequently included in the processed daily rainfall total for 09:00 on 12 April to
09:00 on 13 April, presented against the 12 April hydrological day. As a result, to avoid confusion, when
presenting hydrological data for daily (or longer) time periods the associated metadata should always
clearly specify whether they relate to calendar days/months/years or hydrological days/months/years.
4.5.3 Derived data terminology
Common examples of derived data produced from raw discharge time series are daily mean flows and
annual maximum flows.
For any derived data or statistics held, the data management systems should clearly describe the
method of derivation in metadata associated with the data set. There should be clear, non-ambiguous
associations between all derived data sets and the raw time series data from which they are derived.
While many derived data and statistics may be generated on demand, to assist in the future
understanding of data processing, the data derived at each of the important stages in the data
processing (or an unambiguous method for regenerating them) should be stored, even if these are only
interim data. For example, when deriving daily mean discharge from a sub-daily stage series for an
open-channel gauging station, it would be useful to store both the edited stage series, the derived sub-
daily discharge and the subsequent daily time series.
In cases where statistical extremes are extracted at regular intervals prior to averaging (or vice versa),
care should be taken to describe the resulting item of derived data correctly. For example:
a) maximum daily mean discharge (the maximum value of the sequence of daily mean discharges in a
stated period);
b) mean daily maximum discharge (the average value, in a stated period, of a sequence of maximum
instantaneous discharges recorded daily);
c) mean discharge (the average discharge over a stated period); and
d) mean annual runoff (the average value of annual flow volume expressed in millimetres over the
catchment area).
It should be noted that some legacy data may have different time conventions to those currently being
used at a monitoring site and in some cases a combined time series is generated to aid analysis. For
example, long time series of daily discharge observations may include daily mean flows (derived in
accordance with 8.4.1) for recent years and single daily manual observations in the early record. Care
should be taken when analysing such series so as to take into account the different conventions used.
4.5.4 Resolution of data storage
The number of significant figures to which data are stored should reflect the resolution of the original
measurement (not necessarily the instrument), for example, water level readings are usually read to
1 mm or 0,001 m. In the case of raw data, the storage precision should not degrade original
measurement. Additional recommendations related to water level, discharge and volumetric data are
given in 6.2, 8.3 and 9.4.
4.5.5 Uncertainty
NOTE It is important that those responsible for hydrological data management systems have a good
understanding of the component and overall hydrometric uncertainties pertaining to the data for which they are
responsible.
ISO/IEC Guide 98-3, Guide to the expression of uncertainty in measurement (GUM), and DD
CEN ISO/TS 25377, Hydrometric uncertainty guide (HUG), provide guidance on the estimation of
uncertainties in hydrometric measurements and determinations.
A measurement result comprises:
a) an estimate of the measured value; and
b) a statement of the uncertainty of the measurement.
In accordance with DD CEN ISO/TS 25377 (HUG), the uncertainty of a hydrometric measurement or
determination should be estimated as a combined uncertainty, calculated from the various component
uncertainties. For example, the overall uncertainty in a flow determined from a stage-discharge rating
should consist of the combined uncertainties in the stage measurement and the rating.
ISO/IEC Guide 98-3 (GUM) and DD CEN ISO/TS 25377 (HUG) use standard uncertainties (i.e. at the
68 % confidence limit). DD CEN ISO/TS 25377 (HUG) recommends final resultant uncertainty of
measurement to be expressed at the 95 % confidence limit since this is normal hydrometric practice.
Nevertheless, in the first instance, the standard uncertainty is estimated and this result is multiplied by
the coverage factor (k = 2) to express the uncertainty at the 95 % confidence limit. The statement of the
uncertainty should specify the confidence level or number of standard uncertainties for which the
uncertainty estimation has been made.
4.6 Metadata and descriptive material
The metadata and other descriptive material which provide information about the measurement
and/or subsequent processing of hydrometric data are wide ranging and can take many forms. Such
data should be actively managed in accordance with 4.3. Clause 5 provides further recommendations
relating to metadata, but the following principles should be observed.
Metadata should be stored permanently and organized in such a way that they can be linked to any
measurement to which they relate. For example, where the information being stored relates to a
specific monitoring site this should be associated with the site’s unique identification code in a defined
referencing system (see 5.2.2).
Where information relates to a specific time (or time period) this should be clearly recorded alongside
the information. For example, the period of applicability of a particular stage-discharge relation should
be stored along with the definition of the relation.
Each piece of information should have a record of the name/affiliation of the author and date of
preparation. The same information should be recorded for any subsequent changes to information.
Processes should be in place to ensure that metadata records are updated.
4.7 Maximizing data utility
NOTE The recommendations of this subclause are intended to ensure that hydrometric data management
systems are designed and operated to maximize the utility of data both now and in the future.
4.7.1 Availability of metadata
All hydrometric data should be associated with descriptive metadata detailing their origin and
subsequent processing. Such information should be made available to data users.
4.7.2 Data quality control
4.7.2.1 General
Hydrometric data should be subject to defined quality control procedures to ensure that they are fit for
purpose. Quality control procedures should cover the assessment and subsequent alteration of data.
Quality control normally involves editing and applying quality flags and comments to improve the
quality and utility of the data, and should be undertaken to:
a) minimize instrument errors;
b) minimize processing errors and human errors; and
c) ensure the data and their derivatives are plausible.
Any changes to data should be made in accordance with 4.8.2.
NOTE Where more than one organization is involved in processing data, it is essential that responsibilities for
the quality control and changing of data are clearly agreed by the organizations concerned.
Quality control procedures for hydrometric data should be defined according to the following four
levels, with the choice of quality control level ranging from 1 (no quality control) to 4 (extended quality
control) reflecting the importance of the data to anticipated users.
4.7.2.2 Level 1: No quality control
This level is used where data have not been subject to quality control. Wherever the data have not been
subject to quality control (Level 1) they should be flagged as unchecked (see 5.3.3).
4.7.2.3 Level 2: Primary quality control
Sometimes referred to as “initial” or “real time” quality control. The checks implemented under this
level are normally designed to capture the most severe and obvious errors in raw data.
4.7.2.4 Level 3: Secondary quality control
This level of quality control is usually applied in a delayed mode and normally includes a series of more
detailed quality control procedures.
4.7.2.5 Level 4: Extended quality control
The most comprehensive level of quality control usually includes conducting long-term checks on data
to identify and correct underlying issues in the historical series and ensure that the data are as good as
possible.
Quality control procedures
...