Space - Use of GNSS-based positioning for road Intelligent Transport Systems (ITS) - Part 2: Assessment of basic performances of GNSS-based positioning terminals

Like the other ENs of the whole series, this EN deals with the use of GNSS-based positioning terminals (GBPT) in road Intelligent Transport Systems (ITS). GNSS-based positioning means that the system providing position data, more precisely Position, Velocity and Time (PVT) data, comprises at least a GNSS receiver and, potentially, for performance improvement, other additional sensor data or sources of information that can be hybridized with GNSS data.
This new EN proposes testing procedures, based on the replay of data recorded during field tests, to assess the basic performances of any GBPT for a given use case described by an operational scenario. These tests address the basic performance features Availability, Continuity, Accuracy and Integrity of the PVT information, but also the Time-To-First-Fix (TTFF) performance feature, as they are described in EN 16803-1, considering that there is no particular security attack affecting the SIS during the operation. This EN does not cover the assessment tests of the timing performances other than TTFF, which do not need field data and can preferably be executed in the lab with current instruments.
"Record and Replay" (R&R) tests consist in replaying in a laboratory environment GNSS SIS data, and potentially additional sensor data, recorded in specific operational conditions thanks to a specific test vehicle. The dataset comprising GNSS SIS data and potential sensor data resulting from these field tests, together with the corresponding metadata description file, is called a "test scenario". A dataset is composed of several data files.
This EN 16803-3 addresses the "Replay" part of the test scenario data set. It does not address the "Record" part, although it describes as informative information the whole R&R process. This "Record" part will be covered by EN 16803-4 under preparation.
Although the EN 16803 series concerns the GNSS-based positioning terminals and not only the GNSS receivers, the present release of this EN addresses only the replay process of GNSS only terminals. The reason is that the process of replaying in the lab additional sensor data, especially when these sensors are capturing the vehicle’s motion, is generally very complex and not mature enough to be standardized today. It would need open standardized interfaces in the GBPT as well as standardized sensor error models and is not ready to be standardized. But, the procedure described in the present EN has been designed to be extended to GBPT hybridizing GNSS and vehicle sensors in the future.
This EN 16803-3 does not address R&R tests when specific radio frequency signals simulating security attacks are added to the SIS. This case is specifically the topic of EN 16803-3.
Once standardized assessment tests procedures have been established, it is possible to set minimum performance requirements for various intelligent transport applications but it makes sense to separate the assessment tests issue from minimum performance requirements, because the same test procedure may be applicable to many applications, but the minimum performance requirements typically vary from one application to another. So, this EN does not set minimum performance requirements for any application.

Raumfahrt - Anwendung von GNSS-basierter Ortung für Intelligente Transportsysteme (ITS) im Straßenverkehr - Teil 2: Bestimmung der grundlegenden Leistungen von GNSS-basierten Ortungsendgeräten

basierten Ortungsendgeräten (GBPT) in Intelligenten Transportsystemen (ITS) für den Straßenverkehr. GNSS-basierte Ortung bedeutet, dass das System, das die Positionsdaten, genauer gesagt die Daten zu Position, Geschwindigkeit und Zeit (PVT, en: Position, Velocity and Time), liefert, mindestens einen GNSS-Empfänger und gegebenenfalls weitere zur Verbesserung der Leistung dienende zusätzliche Sensordaten oder Informationsquellen umfasst, die sich mit GNSS-Daten hybridisieren lassen.
Dieses neue Dokument schlägt Prüfverfahren vor, die auf der Wiedergabe von in Einsatzprüfungen aufgezeichneten Daten basieren und zur Überprüfung der grundlegenden Leistungsdaten beliebiger GBPT für einen bestimmten, durch ein Einsatzszenario beschriebenen Anwendungsfall dienen. Diese Prüfungen betreffen die grundlegenden Leistungsmerkmale Verfügbarkeit, Stetigkeit, Genauigkeit und Integrität der PVT-Informationen, aber auch das Leistungsmerkmal Time To First Fix (TTFF), wie sie in EN 16803-1 beschrieben sind, unter Berücksichtigung der Tatsache, dass es keinen speziellen Sicherheitsangriff gab, der das SIS während des Einsatzes beeinträchtigt hat. Dieses Dokument befasst sich nicht mit den Prüfungen zur Überprüfung des Leistungsverhaltens der Zeitsteuerung, soweit es nicht um die TTFF geht, die keine Einsatzdaten benötigt und vorzugsweise mit den üblichen Messgeräten im Labor durchgeführt werden kann.
Bei Aufzeichnung/Wiedergabe- bzw. R&R-Prüfungen (en: „Record and Replay“ tests) werden GNSS-SIS-Daten und gegebenenfalls zusätzliche Sensordaten, die mit Hilfe eines speziellen Testfahrzeugs unter bestimmten Betriebsbedingungen aufgezeichnet wurden, in einer Laborumgebung wiedergegeben. Der GNSS-SIS-Daten und gegebenenfalls Sensordaten aus diesen Einsatzprüfungen enthaltende Datensatz wird in Verbindung mit der entsprechenden Metadatenbeschreibungsdatei als „Prüfszenario“ bezeichnet. Ein Datensatz setzt sich aus verschiedenen Datendateien zusammen.
Die vorliegende EN 16803-2 behandelt den „Wiedergabe“-Teil des Datensatzes des Prüfszenarios. Sie behandelt nicht den „Aufzeichnungs“-Teil, obgleich sie zur Information den gesamten R&R-Prozess beschreibt. Dieser „Aufzeichnungs“-Teil wird von EN 16803-4 abgedeckt, die derzeit in Vorbereitung ist.
Obgleich die Normenreihe EN 16803 die GNSS-basierten Ortungsendgeräte und nicht nur die GNSS-Empfänger betrifft, befasst sich die vorliegende Ausgabe dieses Dokuments nur mit dem Wiedergabeprozess von reinen GNSS-Endgeräten. Der Grund hierfür ist, dass der Prozess der Wiedergabe von zusätzlichen Sensordaten im Labor, besonders wenn diese Sensoren die Fahrzeugbewegung erfassen, allgemein sehr komplex und gegenwärtig noch nicht reif für die Normung ist. Für diesen Prozess wären offene genormte Schnittstellen im GBPT sowie genormte Sensorfehlermodelle erforderlich, und er lässt sich derzeit noch nicht normen. Das in der vorliegenden Europäischen Norm beschriebene Verfahren wurde aber so angelegt, dass es sich später auf GBPT erweitern lässt, die GNSS und Fahrzeugsensoren hybridisieren.
Die vorliegende EN 16803-2 behandelt keine R&R-Prüfungen, bei denen dem SIS spezifische Hochfrequenzsignale hinzugefügt werden, um Sicherheitsangriffe zu simulieren. Dieser Fall ist spezieller Gegenstand der EN 16803-3.
Sobald genormte Prüfverfahren für die Überprüfung eingeführt wurden, ist es möglich, Mindestanforderungen an die Leistung verschiedener intelligenter Transportanwendungen festzulegen, es ist jedoch sinnvoll, das Problem der Leistungsüberprüfungen von den Leistungsmindestanforderungen zu trennen, weil ein und dasselbe Prüfverfahren auf viele Anwendungen anwendbar sein kann, die Leistungsmindestanforderungen sich dagegen üblicherweise von Anwendung zu Anwendung unterscheiden. Entsprechend werden in diesem Dokument keine Mindestanforderungen an die Leistung irgendwelcher Anwendungen festgelegt.

Espace - Utilisation du positionnement GNSS pour les systèmes de transport routier intelligents (ITS) - Partie 2 : Évaluation des performances de base des terminaux de positionnement GNSS

Comme les autres EN de la série, la présente EN traite de l'utilisation des terminaux de positionnement GNSS (GBPT) dans les systèmes de transport routier intelligents (ITS). Le positionnement GNSS signifie que le système fournissant les données de position, et plus précisément les données de position, de vitesse et de temps (PVT), comprend au moins un récepteur GNSS et, éventuellement, à des fins d'amélioration des performances, d'autres données de capteurs ou sources d'informations supplémentaires susceptibles d'être hybridées avec les données GNSS.
Cette nouvelle norme EN propose des procédures d'essai, basées sur le rejeu de données enregistrées pendant des essais de terrain, afin d'évaluer les performances de base d'un GBPT dans un cas d'utilisation donné, décrit par un scénario opérationnel. Ces essais concernent les caractéristiques des performances de base Disponibilité, Continuité, Précision et Intégrité des informations PVT, mais également la caractéristique des performances Temps d'acquisition de la première position (TTFF), telles que décrites dans la norme EN 16803-1, en considérant qu'il n'y a pas d'attaque de sécurité particulière affectant le SIS pendant l'opération. La présente norme EN ne couvre pas les essais d'évaluation des performances temporelles autres que le TTFF, qui ne nécessitent pas de données de terrain et peuvent de préférence être réalisés en laboratoire, avec des instruments courants.
Les essais « Enregistrement et rejeu » (R&R) consistent à rejouer, dans un environnement de laboratoire, des données de signaux GNSS dans l'espace, ainsi que des données provenant d’éventuels capteurs supplémentaires, enregistrées dans des conditions opérationnelles particulières grâce à un véhicule d'essai spécifique. L'ensemble de données qui comprend les données de signaux GNSS dans l'espace et les éventuelles données de capteur issues de ces essais de terrain, associé au fichier de description des métadonnées correspondant, est appelé un « scénario d'essai ». Un ensemble de données est composé de plusieurs fichiers de données.
La présente norme EN 16803-2 traite de la partie « Rejouer » de l'ensemble de données du scénario d'essai. Elle ne traite pas de la partie « Enregistrer », bien qu'elle décrive, sous forme informative, l'ensemble du procédé R&R. La partie « Enregistrer » sera couverte par l'EN 16803-4, qui est en cours d'élaboration.
Bien que la série EN 16803 concerne les terminaux de positionnement GNSS, et pas uniquement les récepteurs GNSS, la version actuelle de la présente EN ne concerne que le procédé de rejeu des terminaux exclusivement GNSS. L'explication réside dans le fait que le procédé de rejeu des données de capteurs supplémentaires en laboratoire, notamment lorsque ces capteurs capturent le mouvement du véhicule, est généralement très complexe et pas encore assez abouti pour être normalisé aujourd'hui. Il faudrait des interfaces normalisées ouvertes dans le GBPT ainsi que des modèles d'erreur normalisés pour les capteurs, et le procédé n'est pas prêt à être normalisé. Toutefois, la procédure décrite dans la présente EN a été conçue pour être étendue, à l'avenir, à des GBPT hybridant des signaux GNSS et des capteurs de véhicules. [...]

Vesolje - Uporaba sistemov globalne satelitske navigacije (GNSS) za ugotavljanje položaja pri inteligentnih transportnih sistemih (ITS) v cestnem prometu - 2. del: Ocenjevanje osnovnih tehničnih lastnosti terminalske opreme za določanje položaja, ki uporablja GNSS

General Information

Status
Published
Publication Date
22-Sep-2020
Withdrawal Date
30-Mar-2021
Current Stage
9020 - Submission to 2 Year Review Enquiry - Review Enquiry
Start Date
15-Oct-2025
Completion Date
15-Oct-2025
Standard
EN 16803-2:2020 - BARVE
English language
87 pages
sale 10% off
Preview
sale 10% off
Preview
e-Library read for
1 day

Standards Content (Sample)


SLOVENSKI STANDARD
01-december-2020
Vesolje - Uporaba sistemov globalne satelitske navigacije (GNSS) za ugotavljanje
položaja pri inteligentnih transportnih sistemih (ITS) v cestnem prometu - 2. del:
Ocenjevanje osnovnih tehničnih lastnosti terminalske opreme za določanje
položaja, ki uporablja GNSS
Space - Use of GNSS-based positioning for road Intelligent Transport Systems (ITS) -
Part 2: Assessment of basic performances of GNSS-based positioning terminals
Raumfahrt - Anwendung von GNSS-basierter Ortung für Intelligente Transportsysteme
(ITS) im Straßenverkehr - Teil 2: Bestimmung der grundlegenden Leistungen von GNSS-
basierten Ortungsendgeräten
Espace - Utilisation du positionnement GNSS pour les systèmes de transport routier
intelligents (ITS) - Partie 2 : Evaluation des performances de base des terminaux de
positionnement GNSS
Ta slovenski standard je istoveten z: EN 16803-2:2020
ICS:
03.220.20 Cestni transport Road transport
33.060.30 Radiorelejni in fiksni satelitski Radio relay and fixed satellite
komunikacijski sistemi communications systems
35.240.60 Uporabniške rešitve IT v IT applications in transport
prometu
2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.

EUROPEAN STANDARD
EN 16803-2
NORME EUROPÉENNE
EUROPÄISCHE NORM
September 2020
ICS 03.220.20; 33.060.30; 35.240.60

English version
Space - Use of GNSS-based positioning for road Intelligent
Transport Systems (ITS) - Part 2: Assessment of basic
performances of GNSS-based positioning terminals
Espace - Utilisation du positionnement GNSS pour les Raumfahrt - Anwendung von GNSS-basierter Ortung
systèmes de transport routier intelligents (ITS) - Partie für Intelligente Transportsysteme (ITS) im
2 : Évaluation des performances de base des terminaux Straßenverkehr - Teil 2: Bestimmung der
de positionnement GNSS grundlegenden Leistungen von GNSS-basierten
Ortungsendgeräten
This European Standard was approved by CEN on 15 June 2020.

CEN and CENELEC members are bound to comply with the CEN/CENELEC Internal Regulations which stipulate the conditions for
giving this European Standard the status of a national standard without any alteration. Up-to-date lists and bibliographical
references concerning such national standards may be obtained on application to the CEN-CENELEC Management Centre or to
any CEN and CENELEC member.
This European Standard exists in three official versions (English, French, German). A version in any other language made by
translation under the responsibility of a CEN and CENELEC member into its own language and notified to the CEN-CENELEC
Management Centre has the same status as the official versions.

CEN and CENELEC members are the national standards bodies and national electrotechnical committees of Austria, Belgium,
Bulgaria, Croatia, Cyprus, Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Iceland, Ireland, Italy,
Latvia, Lithuania, Luxembourg, Malta, Netherlands, Norway, Poland, Portugal, Republic of North Macedonia, Romania, Serbia,
Slovakia, Slovenia, Spain, Sweden, Switzerland, Turkey and United Kingdom.

CEN-CENELEC Management Centre:
Rue de la Science 23, B-1040 Brussels
© 2020 CEN/CENELEC All rights of exploitation in any form and by any means Ref. No. EN 16803-2:2020 E
reserved worldwide for CEN national Members and for
CENELEC Members.
Contents Page
European foreword . 5
Introduction . 6
1 Scope . 8
2 Normative references . 8
3 Terms and definitions . 9
3.1 Definitions . 9
3.2 Acronyms . 10
4 Overview of the whole assessment process . 11
4.1 Definition of the general strategy: what kind of tests . 11
4.1.1 Rationale . 11
4.1.2 Record and Replay choice . 12
4.2 Construction of the operational scenarios: how to configure the tests . 13
4.2.1 General . 13
4.2.2 Basic principles . 13
4.2.3 Definition of the operational scenarios . 15
4.3 Definition of the test facilities: which equipment to use . 20
4.3.1 For the record phase . 20
4.3.2 For the replay phase . 21
4.4 Description of the record phase: how to elaborate the data sets of the test
scenarios . 21
4.4.1 General . 21
4.4.2 Test plan . 21
4.4.3 Test bench preparation and good functioning verification . 22
4.4.4 Field test execution . 22
4.4.5 Data control and archiving . 22
4.5 Replay phase: assessing he DUT performances . 24
5 Definition of the metrics . 24
5.1 General considerations . 24
5.2 Basic notation . 25
5.3 Time interpolation procedure . 25
5.4 Accuracy metrics . 26
5.5 Availability and Continuity metrics. 27
5.6 Integrity metrics . 32
5.6.1 Definition of the Protection Level performance metrics . 32
5.6.2 Definition of the Misleading Information Rate metrics . 33
5.7 Timing metrics . 34
5.7.1 Timestamp resolution . 34
5.7.2 Nominal output latency . 34
5.7.3 Nominal output rate . 34
5.7.4 Output latency stability . 34
5.7.5 Output rate stability . 35
5.7.6 Time to first fix . 36
6 Description of the replay phase: how to assess the DUT performances . 37
6.1 General . 37
6.2 Checking of the content of the test scenario . 37
6.3 Setting-up of the replay test-bench . 38
6.4 Validation of the data processing HW and SW by the RF test laboratory . 39
6.5 Replaying of the data . 39
6.6 Computation of the ACAI performances . 42
6.7 Computation of the TTFF performances . 42
6.8 Establishment of the final test report . 47
7 Definition of the validation procedures: how to be sure of the results (checks) . 47
7.1 Definition of the validation . 47
7.2 Pass/Fail criteria for the verification of the test procedures . 49
8 Definition of the synthesis report: how to report the results of the tests . 50
Annex A (informative) Homologation framework . 58
A.1 The road value chain . 58
A.2 Roles of the different stakeholders . 59
A.3 Responsibilities of the different stakeholders . 60
Annex B (informative) Detailed criteria for the testing strategy (trade-off) . 62
B.1 Main criteria for testing strategy . 62
B.2 Metrological quality . 62
B.2.1 Reproductibility . 62
B.2.2 Representativeness . 63
B.2.3 Reliability . 63
B.3 Cost efficiency . 63
B.3.1 Cost of test benches . 63
B.3.2 Cost of the test operations . 64

B.4 Clarity in the sharing of responsibilities . 64
B.5 Scenario-management authority . 64
Annex C (informative) Record and replay testing considerations . 66
C.1 General . 66
C.2 Experimentation considerations . 66
C.3 Equipment justification . 68
C.3.1 Equipment for in-field data collection . 68
C.3.2 Record and Replay Solutions . 71
C.3.3 Recommended equipment . 73
C.4 Presentation of a scenario: rush time in Toulouse . 74
C.5 Quality of the reference trajectory . 76
C.6 Availability, regularity of the DUT's outputs for the metrics computations . 77
Annex D (informative) Perspectives on record and replay of hybridized GBPT . 79
Annex E (informative) Considerations on coordinate systems, reference frames and
projections . 84
Bibliography . 87

European foreword
This document (EN 16803-2:2020) has been prepared by Technical Committee CEN-CENELEC/TC 5
“Space”, the secretariat of which is held by DIN.
This European Standard shall be given the status of a national standard, either by publication of an
identical text or by endorsement, at the latest by March 2021, and conflicting national standards shall be
withdrawn at the latest by March 2021.
Attention is drawn to the possibility that some of the elements of this document may be the subject of
patent rights. CEN shall not be held responsible for identifying any or all such patent rights.
This document has been prepared under a mandate given to CEN and CENELEC by the European
Commission and the European Free Trade Association.
According to the CEN-CENELEC Internal Regulations, the national standards organisations of the
following countries are bound to implement this European Standard: Austria, Belgium, Bulgaria, Croatia,
Cyprus, Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Iceland, Ireland,
Italy, Latvia, Lithuania, Luxembourg, Malta, Netherlands, Norway, Poland, Portugal, Republic of North
Macedonia, Romania, Serbia, Slovakia, Slovenia, Spain, Sweden, Switzerland, Turkey and the United
Kingdom.
Introduction
The EN 16803 series of CEN-CENELEC standards deals with the use of GNSS technology in the intelligent
transport domain and address more particularly the issue of performance assessment.
As recalled in the generic functional architecture of a road ITS based on GNSS, two main sub-systems can
be considered: the positioning system (GNSS-based positioning terminal (GBPT) + external sources of
data) and the road ITS application processing the position quantities output by the terminal to deliver
the final service to the user.

Figure 1 — Generic functional architecture of a Positioning-based road ITS system
This document is the second one of the EN 16803 series.
The performance assessment issue can also be considered at these two levels.
According to Figure 3 in the Introduction of EN 16803-1, the performances of the application cannot be
assessed independently from the GBPT and the adequacy of the GBPT’s performances to the end-to-end
performance of the system cannot be assessed independently from the application. For these two kinds
of assessment, the EN 16803-1 standard proposed a method called “Sensitivity analysis”. In addition, this
first document defined the generic architecture, the generic terms and the basic performance metrics for
the Positioning quantities.
EN 16803-1 can be of interest for many different stakeholders but is targeting mainly the ITS application
developers.
EN 16803-2, EN 16803-3 and EN 16803-4 address specifically the performances of the GBPT itself, as
they can be measured by the metrics defined in EN 16803-1:
— EN 16803-2 proposes a test methodology based on the replay in the lab of real data sets recorded
during field tests, assuming no security attack during the test.
— EN 16803-3 proposes a complement to this test methodology to assess the performance degradation
when the GNSS signal-in-space (SIS) is affected by intentional radio-frequency (RF) perturbations
such as jamming, spoofing or meaconing, also applicable to unintentional RF perturbations.
These 2 (two) ENs are targeting mainly the generalist RF test laboratory that will be in charge of assessing
the performances of GBPTs for different applications.
EN 16803-4 (in preparation) will propose the methodology for the recording of the real data sets and is
targeting mainly the GNSS-specialized test laboratories that will be in charge of elaborating the test
scenarios that will be replayed by the previous category of test laboratories.
1 Scope
Like the other documents of the whole series, this document deals with the use of GNSS-based positioning
terminals (GBPT) in road Intelligent Transport Systems (ITS). GNSS-based positioning means that the
system providing position data, more precisely Position, Velocity and Time (PVT) data, comprises at least
a GNSS receiver and, potentially, for performance improvement, other additional sensor data or sources
of information that can be hybridized with GNSS data.
This new document proposes testing procedures, based on the replay of data recorded during field tests,
to assess the basic performances of any GBPT for a given use case described by an operational scenario.
These tests address the basic performance features Availability, Continuity, Accuracy and Integrity of
the PVT information, but also the Time-To-First-Fix (TTFF) performance feature, as they are described
in EN 16803-1, considering that there is no particular security attack affecting the SIS during the
operation. This document does not cover the assessment tests of the timing performances other than
TTFF, which do not need field data and can preferably be executed in the lab with current instruments.
“Record and Replay” (R&R) tests consist in replaying in a laboratory environment GNSS SIS data, and
potentially additional sensor data, recorded in specific operational conditions thanks to a specific test
vehicle. The data set comprising GNSS SIS data and potential sensor data resulting from these field tests,
together with the corresponding metadata description file, is called a “test scenario”. A data set is
composed of several data files.
This EN 16803-2 addresses the “Replay” part of the test scenario data set. It does not address the
“Record” part, although it describes as informative information the whole R&R process. This “Record”
part will be covered by EN 16803-4 under preparation.
Although the EN 16803 series concerns the GNSS-based positioning terminals and not only the GNSS
receivers, the present release of this document addresses only the replay process of GNSS only
terminals. The reason is that the process of replaying in the lab additional sensor data, especially when
these sensors are capturing the vehicle’s motion, is generally very complex and not mature enough to be
standardized today. It would need open standardized interfaces in the GBPT as well as standardized
sensor error models and is not ready to be standardized. But, the procedure described in the present EN
has been designed to be extended to GBPT hybridizing GNSS and vehicle sensors in the future.
This EN 16803-2 does not address R&R tests when specific radio frequency signals simulating security
attacks are added to the SIS. This case is specifically the topic of EN 16803-3.
Once standardized assessment tests procedures have been established, it is possible to set minimum
performance requirements for various intelligent transport applications but it makes sense to separate
the assessment tests issue from minimum performance requirements, because the same test procedure
may be applicable to many applications, but the minimum performance requirements typically vary from
one application to another. So, this document does not set minimum performance requirements for
any application.
2 Normative references
The following documents are referred to in the text in such a way that some or all of their content
constitutes requirements of this document. For dated references, only the edition cited applies. For
undated references, the latest edition of the referenced document (including any amendments) applies.
EN 16803-1:2016, Space - Use of GNSS-based positioning for road Intelligent Transport Systems (ITS) - Part
1: Definitions and system engineering procedures for the establishment and assessment of performances
EN 16803-3, Space — Use of GNSS-based positioning for road Intelligent Transport Systems (ITS) — Part 3:
Assessment field tests for security performances of GNSS-based positioning terminals
3 Terms and definitions
For the purposes of this document, the following terms and definitions.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
• ISO Online browsing platform: available at http://www.iso.org/obp
• IEC Electropedia: available at http://www.electropedia.org/
3.1 Definitions
3.1.1
GBPT
GNSS-Based Positioning Terminal
term used to define the component that basically outputs PVT
3.1.2
DUT
Device under Test
term used to define a device that is assessed
Note 1 to entry: In the context of EN 16803-2, DUT refers to GPBT.
3.1.3
test scenario
composed of GNSS SIS data and potential sensor data resulting from field tests, complemented by a
metadata description file; a test scenario is a non-empty combination of UTS that allows to assess a GBPT
in the desired environments
Note 1 to entry: Data inside a Test Scenario are raw data, either RF signals from GNSS satellites, or raw data from
other embedded sensors.
Note 2 to entry: A Test Scenario is the whole package that a GNSS-specialized test laboratory delivers to a
Generalist RF test laboratory in charge of performance assessment tests according to the EN 16803 series.
Note 3 to entry: Considering the 6 (six) different environments as defined in EN 16803-1, there’s a combination
of 2^6 - 1 = 63 possible test scenarios; from let’s say “Rural only” test scenario up to “All environment” test scenario
that covers the 6 different environments. See 4.2.2 for more details.
3.1.4
Unitary Test Scenario (UTS)
elementary brick of a Test Scenario, resulting from a specific field test; in other words, a Test Scenario is
composed of a concatenation of several Unitary Test Scenarios
Note 1 to entry: See 4.2.2 for more details.
3.1.5
Uniform Environment Data Set (UEDS)
output of the DUT collected after a replay in laboratory sorted by environment; it is a concatenation of
the output of the DUT for all UTS restricted to a unique environment
Note 1 to entry: See 6.4 for more details.
Note 2 to entry: Considering the 6 (six) different environments as defined in EN 16803-1, there is the same
number of UEDS; i.e. 6.
Note 3 to entry: Data composing a Uniform Environment Data Set are PVT data, as they are output by a GBPT.
Note 4 to entry: Uniform Environment Data Sets are the data sets to which the metrics shall be applied to assess
the performances of the device under test.
3.1.6
GNSS-specialized test laboratory
laboratory in charge of producing Test Scenarios for generalist RF test laboratories
3.1.7
Generalist RF test laboratory
laboratory in charge of assessing the performances of GBPTs thanks to Test Scenario
3.1.8
Benchmark Unitary Test Scenario (B-UTS)
dedicated UTS used specifically for the validation procedure as defined in Clause 7
3.1.9
Benchmark Uniform Environment Data Set (B-UEDS)
each of the UEDS obtained with the benchmark receiver at the GNSS specialised lab (used by the
generalist lab to validate their test platform and procedures)
3.2 Acronyms
Acronym Description
ACAI Availability, Continuity, Accuracy, Integrity
BPM Benchmark Performance Metrics
B-UEDS Benchmark Uniform Environment Data Set
B-UTS Benchmark Unitary Test Scenario
CDF Cumulative Distribution Function
DUT Device Under Test
GBPT GNSS Based Positioning Terminal
GNSS Global Navigation Satellite Systems
I/Q In-phase and Quadrature – I/Q format is an efficient way to store RF signals so that it is
possible to reproduce RF signals in laboratory after modulation. I/Q format is the
format used to store GNSS signals in UTS
ITS Intelligent Transport Systems
KML Keyhole Markup Language
Lab-A GNSS-specialized test laboratory
Lab-B Generalist RF test laboratory
PVT Position Velocity and Time
RAMS Reliability, Availability, Maintainability, and Safety
RF Radio frequency
R&R Record and Replay
SIS Signal In Space: RF signals coming from the GNSS satellites
UTS Unitary Test Scenario
UEDS Uniform Environment Data Set
TTFF Time To First Fix
4 Overview of the whole assessment process
4.1 Definition of the general strategy: what kind of tests
4.1.1 Rationale
Performances and behaviours of GNSS-based positioning terminals not only depend on their design but
also, and strongly, on a lot of external situations and parameters, uncontrolled by the stakeholders.
Among those parameters, we can quote the status of international worldwide space systems (GNSS), the
physical atmospheric conditions, and other environmental conditions in the proximity of the vehicle
(buildings in vicinity, traffic, tree foliage, etc.).
As an example, this situation implies that any realization of one field test procedure of a given product at
a given date and hour, will give a different result than the same test procedure of the same product in the
same location, but at a different date and hour (not stationary stochastic process).
The obvious consequence is that, if a pure field test strategy is targeted as a preferred solution for the
performance assessment aiming certification of devices, the analysis of the tests results would require
specialists, and may frequently result in intangible and unreliable interpretations, the opposite of
metrology.
A solution to avoid this issue is to have a total trust in simulations where all the tests conditions are
controlled, and which could be perfectly repeatable. ETSI addressed a similar issue during its
standardization process targeting the GNSS based Location Based Services. As a conclusion of its work,
ETSI, selected a solution exclusively based on simulations (see [1]).
Considering that the real-life environment remains complex to be simulated, the pure simulation
technique will lead to scenarios with a very great number of parameters to be set-up, inducing risk of
human manipulation errors, and anyway a remaining lack of true representation of the reality.
A standardized performance assessment process, designed for certification, needs to be unquestionable
(undisputable?), repeatable, realistic with respect to the real conditions the GBPT will operate in and
cost-effective. Since the reality of all the physical phenomena impacting the behaviour of GNSS receivers
is quite impossible to be simulated with mathematical models, but still has a significant impact, the
principle of field tests is preferred in this standard to the principle of simulating the GNSS SIS with
constellation simulators.
But, in pure field tests where the device under test (DUT) is on-board the test vehicle, this latter is
stimulated with various environmental features that remain unpredictable and uncontrolled, rarely
reproducible from one run to another and leading to large difficulties to interpret the results.
Consequently, the present EN made the choice of “Record and Replay” (R&R) technique that combines
realism, repeatability and cost effectiveness. R&R technique covers all constellation and all frequencies
by allowing recording lower L-Bands and upper L-Bands of GNSS SIS.
4.1.2 Record and Replay choice
The “Record and Replay” technique starts on the field, by recording of the test data collection
according to an agreed scenario. This step shall guarantee a high-fidelity digitalization of numerous
parameters, in particular the capture of radio signals issued from the worldwide infrastructure
(GNSS) in a realistic local environment of propagation. It ends in the lab to replay as many times as
required the same scenario, using exactly the same radio signal inputs and thus the same
environmental conditions of reception.
With respect to pure simulation, the R&R technique offers a better representation of reality since it comes
from real situations, but less flexibility of tested situations (rare events are unlikely). The chance to
integrate a satellite failure in one recorded test scenario is near to zero. However, as with simulations, a
lot of devices can be tested in exactly the same conditions.
With respect to field tests, the R&R technique offers better metrological features since the repeatability
can be reached.
It also saves money, since the setup of the test bench for recording is similar to one field test, but the
setup of the test bench for replaying is largely less expensive and enables to test a multitude of receivers
with less additional cost.
Finally, a very interesting feature of the R&R approach, is that the interface between the record phase
and the replay phase is a repository of files, offering capabilities like copies and licensing, cut and
paste (in smaller files if a customization by application would become necessary as an example…),
browse, etc. Today, the amount of data are important for storage techniques (2TB/min of recording) but
remains feasible for some hours of recording. It is expected that in few years, improved solution could be
available (currently no data compression is used).
Since the record phase needs recognized skills and experience in GNSS metrology, this work shall be
performed by GNSS-specialized laboratories, ISO/IEC 17025 homologated, and accredited for that job
by a certification authority. They shall follow standardized procedures for recording the data sets that
shall become themselves standardized scenarios that can be replayed by a larger panel of homologated,
but not GNSS-specialized, radio frequency (RF) test laboratories.
Figure 2 — The “Record and Replay” principle
This feature has a true interest in the worldwide economy. First it enables to make in Tokyo a field test
recorded in Paris, or inversely to replay anywhere in Europe and with few expenses a record performed
in Shanghai. Moreover, it is interesting, as an additive business model for GNSS, offering a European
capability to export standardized data sets towards other regional economies, enabling worldwide
competitors to accept and apply the European standards, and even promoting the European
methodologies to be extended to world.
4.2 Construction of the operational scenarios: how to configure the tests
4.2.1 General
Foreword: most of this section is informative in the sense that in provides informative material on how
to proceed to record a scenario that is compatible with the quality required for high-level standards.
This information is necessary to understand the complete R&R process, but this is not the aim of this EN
to standardize the record procedure. The record procedure and its quality framework will be totally and
precisely described in EN 16803-4 (under preparation).
4.2.2 Basic principles
4.2.2.1 Unique data collection for all the metrics
Availability, continuity, accuracy and integrity shall be evaluated from a Uniform Environment Data Set
(UEDS).
The metrics for measuring the GBPT performances with respect to these different performance features
are introduced in EN 16803-1 and precisely described in the present document In Clause 5.
These performances are totally linked in the applicative needs, since a position or velocity output can
make sense for any application only if it is simultaneously available (declared as valid), accurate (at a
certain level), and that we can have trust on it (at a certain level of risk). Therefore, the metrics of the
4 (four) types (except the integrity risk, see 4.2.2.2 below) shall be established on the same data collection
campaign.
4.2.2.2 Particular case of the integrity risk
The integrity functionality is particularly worthy in safety critical or liability critical operations. Its main
interest relies on the ability of detection and of exclusion of positioning process faulty behaviours, caused
by unexpected situations (feared events) susceptible to lead with the highest probability (with regard to
normal operations) towards an accident (safety) or disputable operations like undue payment (liability).
The purpose is to avoid the propagation in the positioning errors of potential rare events (from sensors,
from GNSS particular inputs). The proper behaviour for an integrity-compliant GBPT is to avoid
outputting a declared valid position while a positioning error is greater than the protection level and
especially greater than the acceptable alarm limit for the critical operation. This proper behaviour,
preventing from hazardous misleading information, while unexpected situations are present upstream
the process, avoids the propagation of the effects of rare feared events related to the positioning level in
further computations.
In this standard, only nominal test scenarios are considered, meaning that the test cases are
representative of normal conditions, and, for most of them, do not contain (per definition) rare feared
events that could lead to very hazardous situations for the application.
If the GBPT provides a protection level (PL), supposed to bound the error on the protected component
(e.g. the horizontal position) with a specified target integrity risk (TIR), there will be no problem in
measuring the availability and the performance of this PL, e.g. its size, by the corresponding metrics of
EN 16803-1.
But concerning the integrity risk (IR), the problematic is quite different since it refers to the behaviour of
the GBPT in specific risky operation; So, the other integrity metric concerning the verification of the
−4 −5
integrity risk itself (supposed to be lower than the TIR, which is itself very small, of the order of 10 , 10
−6
or 10 ), will not be correctly estimated with the replay of the test scenarios for 2 (two) reasons:
1) The feared events have not necessarily been recorded in the scenarios.
2) The number of samples is not high enough for estimating the IR with a good level of confidence.
What will be possible to measure, additionally to the PL size, will be the misleading information rate
(MIR), i.e. the ratio of epochs, over the total number of epochs, showing an error greater than the PL. This
MI rate can converge toward the IR when the number of samples is high enough, e.g. several days of test,
which is not economically acceptable.
Consequently, the test procedures that are described in the EN 16803 series are designed to
measure the protection level performance in terms of size, but not the integrity risk. Only the MI
rate over the test period, that is not the IR, can be measured, as an indicator of the reliability of the PL.
Other kinds of tests, involving a large number of simulations including some specific feared events, have
to be designed for assessing the IR.
4.2.2.3 Same data collection for a flexible list of road applications
This EN standard has been setup to deal with a long list of ITS applications without distinctions. However,
the applications can cover local/global areas, slow/high speeds, etc.
While the global set of ITS applications may cover any type of road trips into any type of environment
(and even parking), one selected application could require only some specific conditions, like for instance
low speed near parking locations, or typical places of virtual gantry, etc.
To cover representatively any road applications, the data set collection shall cover a maximum of
different types of road trips, with their typical road environment and motion dynamics:
— long range road transport (transnational deliveries, individual/collective European tourism…);
— very short range deliveries (frequent stops);
— quotidian worker journey with typical traffic jam (urban, rural…);
— etc.
More refined descriptions are given in 4.2.3.
4.2.3 Definition of the operational scenarios
4.2.3.1 General
This section recalls the current definition of an operational scenario issued from EN 16803-1, and then
makes a proposal for defining the operational scenario to be used for recording the operational data to
be replayed for the metrics assessment. This section is just informative and not included in the normative
part of this document. The precise definition of the scenarios will constitute the topic of EN 16803-4.
4.2.3.2 Standardized definition of an operational scenario
4.2.3.2.1 General
The definition of the operational scenario (cf EN 16803-1:2016, 5.1) is recalled here: “description of the
conditions in which the GNSS-based road ITS system is operating and particularly affecting the GBPT”.
The operational environment is also important to define the test conditions in which the system’s
performances and metrics shall be assessed.
As highlighted in the standard EN 16803-1, three sets of parameters would have to be defined in order
to accurately describe an operational scenario and subsequent test cases, since they would likely impact
the assessed performance metrics:
— the set-up conditions of the terminal and of all its sensorial organs (GNSS antenna, inertial
measurement unit, odometers, cameras…) conditioning the sensing properties;
— the trajectory of the vehicle: its location in the world, and its cinematic impacting the capability to
correct or filter any sensor’s defaults;
— the environmental conditions impacting the sensing measurements.
This general definition applies for any kind of tests.
EN 16803-1:2016, 5.2.1 affirms furthermore that the GNSS reference environments shall be used to
1)
interpret test results and to compare results of tests which are similar but executed at different places.
This is the reason why the presentation of the tests results should refer to the high-level categories of
GNSS reference environments, as defined in EN 16803-1:2016, 5.2.2. Construction of the test scenarios
The goal of the standard is to characterize the exclusive part of errors imputable to the terminal’s
processing, when it is operating on the road.
This terminal, when it is in operation or during field tests operates in some environmental conditions,
that it undergoes, as well as the systematic errors due to the GNSS infrastructure or due to the
atmosphere that are not imputable to it. These environmental conditions are, along the road operations,
sometimes favourable and sometimes unfavourable to the GNSS SIS properties, alternatively. Not only
those environmental conditions, individually taken, will influence the capability of the GBPT to provide
all its potential in terms of AIA performances, but also their changing rates.

1) Not to obtain the data set collection, but to compute the metrics.

This EN series relies on a pragmatic manner to get a sufficient combinatory of the test conditions, without
customization to one or another final application, by crossing different roads and different kinds of trip:
— the road selection imposes the GNSS geometry and the propagation channel: it is not possible in field
test to change the topology of a road or of its neighbours, so it is important to integrate in the
selection of roads a variety of landscapes, and landscape changes impacting visibility and
reflection/diffraction effects in the same manner than typical use cases;
— the kinds of trips can impact the kinematic, in particular the proportion and amplitude between low
and high speeds, and this factor, mixed with the GNSS environments, will involve alternative best
uses between inertial or local vision sensors and GNSS sensors, and consequently stimulating the
hybridization systems, and various errors correlation times in the same manner than typical use
cases;
— a cross between road selection and nature of trip for the car will naturally cover the range of the
positioning errors (up to 95 %) by covering the major part of environmental conditions with effects
both on DOPS and on ranging measurements errors;
— no extreme limits of environmental parameters will be looked for, but a preference for worst cases
regarding some usual parameters: thus most of the test records shall be done at the end of spring
and in the afternoon so that no advantageous effects of foliage and ionospheric delays could be found.
Figure 3 illustrates the phi
...

Questions, Comments and Discussion

Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.

Loading comments...