ETSI ES 202 765-4 V1.1.1 (2010-10)

Final draft ETSI ES 202 765-4 V1.1.1 (2010-08)
ETSI Standard
Speech and multimedia Transmission Quality (STQ);
QoS and network performance metrics
and measurement methods;
Part 4: Indicators for supervision of Multiplay services

2 Final draft ETSI ES 202 765-4 V1.1.1 (2010-08)

Reference
DES/STQ-00104-4
Keywords
performance, QoS
ETSI
650 Route des Lucioles
F-06921 Sophia Antipolis Cedex - FRANCE

Tel.: +33 4 92 94 42 00  Fax: +33 4 93 65 47 16

Siret N° 348 623 562 00017 - NAF 742 C
Association à but non lucratif enregistrée à la
Sous-Préfecture de Grasse (06) N° 7803/88

Important notice
Individual copies of the present document can be downloaded from:
http://www.etsi.org
The present document may be made available in more than one electronic version or in print. In any case of existing or
perceived difference in contents between such versions, the reference version is the Portable Document Format (PDF).
In case of dispute, the reference shall be the printing on ETSI printers of the PDF version kept on a specific network drive
within ETSI Secretariat.
Users of the present document should be aware that the document may be subject to revision or change of status.
Information on the current status of this and other ETSI documents is available at
http://portal.etsi.org/tb/status/status.asp
If you find errors in the present document, please send your comment to one of the following services:
http://portal.etsi.org/chaircor/ETSI_support.asp
Copyright Notification
No part may be reproduced except as authorized by written permission.
The copyright and the foregoing restriction extend to reproduction in all media.

© European Telecommunications Standards Institute 2010.
All rights reserved.
TM TM TM TM
DECT , PLUGTESTS , UMTS , TIPHON , the TIPHON logo and the ETSI logo are Trade Marks of ETSI registered
for the benefit of its Members.
TM
3GPP is a Trade Mark of ETSI registered for the benefit of its Members and of the 3GPP Organizational Partners.
LTE™ is a Trade Mark of ETSI currently being registered
for the benefit of its Members and of the 3GPP Organizational Partners.
GSM® and the GSM logo are Trade Marks registered and owned by the GSM Association.
ETSI
3 Final draft ETSI ES 202 765-4 V1.1.1 (2010-08)
Contents
Intellectual Property Rights . 5
Foreword . 5
1 Scope . 6
2 References . 6
2.1 Normative references . 6
2.2 Informative references . 7
3 Symbols and abbreviations . 7
3.1 Symbols . 7
3.2 Abbreviations . 7
4 General Overview. 8
5 Measurement type . 9
6 List of Internet service indicators . 10
6.1 Availability of Internet Access . 10
6.2 Internet Download Bit Rate . 11
6.3 Internet Upload Bit Rate . 13
6.4 Unsuccessful FTP Download session Ratio . 14
6.5 Unsuccessful FTP Upload session Ratio . 14
6.6 Unsuccessful HTTP session Ratio . 15
6.7 Ping Delay . 16
6.8 Internet Login Time. 17
6.9 Web page download Speed . 18
6.10 FTP download Speed. 18
6.11 FTP upload Speed . 20
7 List of voice service indicators . 20
7.1 Voice messaging availability . 21
7.2 Post Dialling Delay . 21
7.3 Pick Up Delay . 21
7.4 Message Provisioning Delay . 22
7.5 Voice message quality . 22
8 List of IPTV indicators . 23
8.1 Channel Availability. 23
8.2 Service Group Channel Availability. 24
8.3 Video Quality . 25
8.4 Audio Quality . 25
8.5 "Black Screen" Occurrences . 26
8.6 Blockiness Occurrences . 26
8.7 Frozen Picture Occurrences . 27
8.8 Lip Desynchronization Occurrences . 27
8.9 Zapping Delay . 27
8.10 Transmission Delay . 28
8.11 IPTV service boot delay . 28
9 List of VoD indicators . . 29
9.1 VoD Service Availability . 29
9.2 Request Conformity . 29
9.3 VoD failure rate . 30
9.4 Video Quality . 30
9.5 Audio Quality . 31
9.6 "Black Screen" Occurrences . 31
9.7 Blockiness Occurrences . 32
9.8 Frozen Picture Occurrences . 32
ETSI
4 Final draft ETSI ES 202 765-4 V1.1.1 (2010-08)
10 Measurement frequency . 33
11 Measurement locations and their distribution . 33
12 Results presentation . 33
Annex A (normative): Principle of artefact detection algorithms . 35
A.1 Detection principle of frozen picture occurrence . 35
A.2 Detection principle of "black screen" occurrence . 36
A.3 Detection principle of blockiness occurrence . 37
Annex B (informative): Comparisons of ES 202 765-4 and TS 102 250-2 (V1.7.1) parameters . 38
Annex C (informative): Bibliography . 42
History . 43

ETSI
5 Final draft ETSI ES 202 765-4 V1.1.1 (2010-08)
Intellectual Property Rights
IPRs essential or potentially essential to the present document may have been declared to ETSI. The information
pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found
in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in
respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web
server (http://webapp.etsi.org/IPR/home.asp).
Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee
can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web
server) which are, or may be, or may become, essential to the present document.
Foreword
This ETSI Standard (ES) has been produced by ETSI Technical Committee Speech and multimedia Transmission
Quality (STQ), and is now submitted for the ETSI standards Membership Approval Procedure.
The present document is part 4 of a multi-part deliverable covering the QoS metrics for telecommunication services and
network performance metrics for transport networks, as identified below:
EG 202 765-1: "General considerations";
ES 202 765-2: "Transmission Quality Indicator combining Voice Quality Metrics";
EG 202 765-3: "Network performance metrics and measurement methods in IP networks";
ES 202 765-4: "Indicators for supervision of Multiplay services".
ETSI
6 Final draft ETSI ES 202 765-4 V1.1.1 (2010-08)
1 Scope
The present document aims at identifying and defining indicators and methodologies for a use in a context of end-user
quality characterisation and supervision of Multiplay services.
In this context the measurements and metric determinations are performed by analysing signals accessible on user-end
services and not on the network.
The present document concerns: Internet access, voice messaging service, IPTV and VoD.
The assessment methods are intrusive and non intrusive.
2 References
References are either specific (identified by date of publication and/or edition number or version number) or
non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the
reference document (including any amendments) applies.
Referenced documents which are not found to be publicly available in the expected location might be found at
http://docbox.etsi.org/Reference.
NOTE: While any hyperlinks included in this clause were valid at the time of publication ETSI cannot guarantee
their long term validity.
2.1 Normative references
The following referenced documents are necessary for the application of the present document.
[1] ETSI EG 202 057-4: "Speech Processing, Transmission and Quality Aspects (STQ); User related
QoS parameter definitions and measurements; Part 4: Internet Access".
[2] ITU-T Recommendation G.1030: "Estimating end-to-end performance in IP networks for data
applications".
[3] ITU-T Recommendation G.1010: "End-user multimedia QoS categories".
[4] IETF RFC 792: "Internet Control Message Protocol".
[5] ETSI TS 102 250-2: "Speech and multimedia Transmission Quality (STQ); QoS aspects for
popular services in GSM and 3G networks; Part 2: Definition of Quality of Service parameters and
their computation".
[6] ITU-T Recommendation P.800: "Methods for subjective determination of transmission quality".
[7] ITU-T Recommendation P.862: "Perceptual evaluation of speech quality (PESQ): An objective
method for end-to-end speech quality assessment of narrow-band telephone networks and speech
codecs".
[8] ITU-T Recommendation P.862.1: "Mapping function for transforming P.862 raw result scores to
MOS-LQO".
[9] ITU-T Recommendation P.862.2: "Wideband extension to Recommendation P.862 for the
assessment of wideband telephone networks and speech codecs".
[10] ITU-T Recommendation P.862.3: "Application guide for objective quality measurement based on
Recommendations P.862, P.862.1 and P.862.2".
[11] ITU-T Recommendation P.800.1: "Mean Opinion Score (MOS) terminology".
[12] ITU-T Recommendation P.505: "One-view visualization of speech quality measurement results".
ETSI
7 Final draft ETSI ES 202 765-4 V1.1.1 (2010-08)
[13] ETSI ES 202 765-2: "Speech and multimedia Transmission Quality (STQ); QoS and network
performance metrics and measurement methods; Part 2 : Transmission Quality Indicator
combining Voice Quality Metrics".
2.2 Informative references
The following referenced documents are not necessary for the application of the present document but they assist the
user with regard to a particular subject area.
[i.1] ETSI TR 102 607: "Speech Processing, Transmission and Quality Aspects (STQ); TCP IP Stack
Parameter Settings for Microsoft Windows XP and Microsoft Windows Vista; Comparison and
Recommendations".
[i.2] ETSI TR 102 505: "Speech Processing, Transmission and Quality Aspects (STQ); Development of
a ReferenceWeb page".
[i.3] ITU-T Recommendation J.144: "Objective perceptual video quality measurement techniques for
digital cable television in the presence of a full reference".
[i.4] ITU-T Recommendation J.247: "Objective perceptual multimedia video quality measurement in
the presence of a full reference".
3 Symbols and abbreviations
3.1 Symbols
For the purposes of the present document, the following symbols apply:
1 kbit/s 1 000 bit/s
1 Mbit/s 1 000 kbit/s
kbps kilobit per second
3.2 Abbreviations
For the purposes of the present document, the following abbreviations apply:
ATM Asynchronous Transfer Mode
BGP Border Gateway Protocol
BRAS Broadband Remote Access Server
CIF Common Intermediate Format
CPE Customer Premises Equipment
DHCP Dynamic Host Control Protocol
DNS Domain Name System
DSLAM Digital Subscriber Line Access Multiplexer
ETSI European Telecommunications Standards Institute
FQDN Fully Qualified Domain Name
FTP File Transfer Protocol
GE Gigabit Ethernet
GSM Global System for Mobile communications
HDMI High Definition Multimedia Interface
HGW Home GateWay
HTTP Hyper Text Transfer Protocol
ICMP Internet Control Message Protocol
IP Internet Protocol
IPTV Internet Protocol Television
ISP Internet Service Provider
ITU-T International Telecommunication Union - Telecommunication standardisation sector
LNS L2TP Network Server
ETSI
8 Final draft ETSI ES 202 765-4 V1.1.1 (2010-08)
MOS Mean Opinión Score
MOS-LQOM Mean Opinion Score - Listening Quality Objective Mixed bandwidths
MOS-LQOM Mean Opinión Store-Listening Quality Objective Mixed
MPEG TS MPEG Transport Stream
MPEG Moving Picture Experts Group
OLT Optical Line Termination
PESQ Perceptual Evaluation of Speech Quality
PSTN Public Switched Telephone Network
QoS Quality of Service
RTP Real Time Protocol
RTT Round-Trip Time
S/PDIF Sony Philips Digital Interface
STB Set Top Box
TCP Transmission Control Protocol
UDP User Datagram Protocol
VoD Video On Demand
VoIP Voice over Internet Protocol
4 General Overview
The present document aims at identifying and defining indicators and methodologies for a use in a context of end-user
quality characterization and supervision of multiplay telephony services such as Internet access, IPTV and VoD. It
completes ES 202 765-2 [13] that was dedicated to voice telephony services.
The present document gives practical requirements of use in the context of service verification and benchmark on a
large and representative scale from the point of view of the potential stakeholders such as the end-users or of the
regulatory authorities. This has been made necessary by the current or recent evolutions of the telecommunication
sector:
• a competitive environment for the offers of multiplay services with a multitude of service providers, with a
quality guarantee not always assured and where clients can very easily change their service providers;
• the development of time varying quality in telecommunications, first in telephony with mobile offers (due to
mobility and irregular network coverage), but now also for multiplay services use in residential context
(mostly due to IP transmission);
• the cohabitation, interaction and competition between services based on different technologies.
The deployment of multiplay offers is increasing but quality guarantee is not always assured.
To achieve the goal mentioned beforehand, there are several existing possibilities, not fully satisfying:
• Subjective tests, with a few human testers assessing the quality of services. This method is very long to run
and not really cheap if we consider that there are many offers to be assessed. And it is not easily applicable in
a context of quality changing over time.
• Objective tests. This is the most reliable way, although it is also based on sampling and can cost a lot of money
in the case of a large deployment of probes or robots.
ETSI
9 Final draft ETSI ES 202 765-4 V1.1.1 (2010-08)
The present document assumes that this last family of methodology answers the needs of a reliable comparison of
multiplay services.
Internet
Internet service
IP core VoIP
VoIP service Service
network
platforms
Access
and
Backhaul
Audiovisual
networks
Service
platforms
IPTV and VoD
services
Game service Game
service
platforms
Figure 1: Possible configuration of architecture of multiplay services
The analysis of multiplay offers requires the simulation of user behaviour. In this context, robots and analyzers have to
use and seek services in a very close way of a customer usage.
What definitely matters is the point of view of the end-users. What they perceive is not only the result of the
transmission across the network, but also artefacts produce by the service platforms or service servers.
In the present document the following services are considered: Internet access, voice messaging (in complement to
telephony services addressed in ES 202 765-2 [13]), IPTV and VoD.
Last important aspect that is addressed in the present document is the practical organization of measurement campaigns
in order to get a realistic and reliable vision of the services as perceived by the end-users. In particular, the questions of
the periodicity of measurement and of the geographical coverage (i.e. more generally the sampling approach). These
aspects are specified in clause 5.
5 Measurement type
Considering the specific perimeter for the characterization of multiplay offer quality 24 hours a day and 7 days a week,
analyses should be realized by robots. In this context, subjective evaluations are not adapted. The robot has to simulate
the use of services by a user.
Besides, the characterization of the offers is considered from the user point of view. So, analyzers shall be connected on
the available accesses of the HGW (Ethernet access, analogical phone access) or/and of the STB (HDMI, S/PDIF). So,
analyses are performed on electric signals.
In general care should be taken by comparing results from measurements obtained by using different setups
(e.g. protocols, service layers).
ETSI
10 Final draft ETSI ES 202 765-4 V1.1.1 (2010-08)
6 List of Internet service indicators
To determine the indicators of internet service, it is necessary to manage the measurements with a Personal
Computer (PC) similar to those currently or mostly used by users. Care should be taken when using a PC which is not
very powerful. On this matters, recommendations are available in TR 102 607 [i.1].
The indicators proposed in the context of end-user quality survey of Internet services are detailed in the following
clauses.
6.1 Availability of Internet Access
Definition This metric represents the probability for a customer that Internet applications are attainable
from his Internet access. It denotes the probability for a customer that his Internet access is
available.
Assessment method This metric provides, for a user, the percentage of time where access to the Internet services
are available.
Availability of Internet Access = 1 –Unavailability of Internet Access

∑ Unavailability duration              ∑ Failure measurement
= 1 - ―――――――――――――― = 1 - ――――――――――――――
Duration of period analysis           Total time of measurement

Internet Access Availability measurement is an attempt since the user access equipment to
reach an Internet service like downloading a web page from a server.
To determine the metric, it is important to test the whole transmission chain which allows to
access to Internet services outside to ISP network.
NOTE: Use the access to ISP mail server does not give a correct view of Internet Access
Availability because edge equipments between ISP network and Internet network
are not involved.
Guidelines In practical way, this indicator can be measured, from the user access, by contacting
different Web sites (national or/and international) hosted on servers outside and within the
ISP network. It is necessary to test the accessibility on several servers to avoid a wrong
measurement interpretation due to Web server breakdown.

The different attempts to reach web servers and measure the successful or unsuccessful
rate should be made periodically. The time interval between 2 sequences of attempts to
reach servers does not be more than 15 minutes. It is better to adjust the periodicity of
analysis between 5 miniutes and 10 minutes.
Unit
% with the resolution of 1 digit after the decimal point
Standardization
reference
Significant
Mandatory
ETSI
11 Final draft ETSI ES 202 765-4 V1.1.1 (2010-08)
Comment This availability excludes provisioning, billing or after sales issues that are part of QoS
parameters of the Customer relationship stages [i.1].

The Internet access availability metric takes into account the availability and the correct
functioning of each network element allowing the access to the service. Network elements to
consider in this context are:
• Access node (DSLAM, OLT).
• Aggregation nodes and links (ATM and/or GE).
• Access server (BRAS, LNS).
• Transmission nodes and links.
• Service Platform (DHCP server, DNS).
A target value for this indicator should be more than 99,95 %
Warning: When the Internet Access Availability is determined by reaching Web servers
outside the ISP network, this indicator needs to be handled with care. Indeed there are a lot
of factors on which the ISP has little or no control: faults in networks of transit providers,
faults at interconnection points, BGP routing errors in peer networks, etc.

6.2 Internet Download Bit Rate
Definition This metric represents Internet download bit rate available to the user. The indicator
evaluates the capacity to use the Internet services.
Assessment method There are several reasons so that Internet download bit rate supplied to the user is lower
than this expected: too long distance between the user and first network access equipment,
bad equipment configuration, degraded link between the user and the network,…
The Internet download bit rate is evaluated by measuring the bit rate during data transfer
from network to user access equipment.
It is important to verify that the server used for the measurement has a sufficient output bit
rate to make this type of measure. It shall send and receive data flow with bit rates higher
than those available on the user access equipment.
ETSI
12 Final draft ETSI ES 202 765-4 V1.1.1 (2010-08)
Guidelines In practical way, this indicator can be measured as follows:

- Data transfers shall be long enough to ensure that results are significant.
10 seconds seem to be a sufficient time period. This means that the file for
downloading test should have a correct size: smaller file for a low-capacity
connection, bigger file for high-capacity connections.
- During this analysis duration (10 seconds), we proceed to the counting of received
bytes (on user side).
- We stop the counting when the analysis duration is over.
- Concerning the reporting, 2 values can be determined: the maximum bit rate during
the faster second and the average bit rate during the analysis period of 10 seconds.
- Preliminary tests can be performed to calibrate the volume of data to be
transferred.
The file type for downloading test could be a jpg image because such files can easily be
generate to different file sizes.

The used protocol for this downloading test should be HTTP.

NOTE: For this measurement, incompressible files should be used for the data transfer.
Unit
bit/s, kbit/s or Mbit/s
Standardization
reference
Significant
Mandatory
Comment 2 types of measurement should be perform:
• Internet Download bit rate measurement in presence of other services (VoIP, IPTV or
VoD); and
• Internet Download bit rate measurement without other services.
For measurement with other services, Internet Download bit rate measurement should be
made from a server located within the ISP network.

The 95 quantile can be determined. This indicator value gives information about the lowest
download bit rate measured during a given period of time (a day, a week or a month).
In a practical way, this indicator is determined by taking into account all measurement results
obtained for the metric "Internet Download Bit Rate" and by determining the bit rate threshold
corresponding to 95 % of the measurements.

NOTE: Care should be taken in the evaluation of the measurement results of this
indicator. Measurements with low (or no) bitrates evidently caused by the
measurement system or other external factors should be excluded from the
statistics.
ETSI
13 Final draft ETSI ES 202 765-4 V1.1.1 (2010-08)
6.3 Internet Upload Bit Rate
Definition
This metric represents Internet upload bit rate available to the user. The indicator evaluates
the capacity to use the Internet services.
Assessment method As for Internet download bit rate, there are several reasons so that Internet upload bit rate
supplied to the user is lower than this expected: too long distance between the user and first
network access equipment, bad equipment configuration, degraded link between the user
and the network,…
The Internet upload bit rate is evaluated by measuring the bit rate during data transfer from
user access equipment to network.
It is important to verify that the server used for the measurement has a sufficient input bit rate
to make this type of measure. It shall send and receive data flow with bit rates higher than
those available on the user access equipment.
Guidelines
In a practical way, the methodology is the same to that used to determine the Internet
Download Bite Rate. Internet Upload Bit Rate can be measured as follows:

- Data transfers shall be long enough to ensure that results are significant.
10 seconds seem to be a sufficient time period. This means that the file for
Uploading test should have a correct size: smaller file for a low-capacity
connection, bigger file for high-capacity connections.
- During this analysis duration (10 seconds), we proceed to the counting of sended
bytes (on user side).
- We stop the counting when the analysis duration is over.
- Concerning the reporting, 2 values can be determined: the maximum bit rate during
the faster second and the average bit rate during the analysis period of 10 seconds.
- Preliminary tests can be performed to calibrate the volume of data to be
transferred.
The file type for Uploading test could be a jpg image because such files can easily be
generate to different file sizes.

The used protocol for this downloading test should be HTTP.

NOTE: For this measurement, incompressible files should be used for the data transfer.
Unit bit/s, kbit/s or Mbit/s
Standardization
reference
Significant Mandatory
Comment 2 types of measurement should be perform:
• Internet upload bit rate measurement in presence of other services (VoIP, IPTV or
VoD); and
• Internet upload bit rate measurement without other services.
For measurement with other services, Internet upload bit rate measurement should be made
to a server located within the ISP network.

The 95 quantile can be determined. This indicator value gives information about the lowest
upload bit rate measured during a given period of time (a day, a week or a month).
In a practical way, this indicator is determined by taking into account all measurement results
obtained for the metric "Internet Upload Bit Rate" and by determining the bit rate threshold
corresponding to 95 % of the measurements.

NOTE: Care should be taken in the evaluation of the measurement results of this
indicator. Measurements with low (or no) bitrates evidently caused by the
measurement system or other external factors should be excluded from the
statistics.
ETSI
14 Final draft ETSI ES 202 765-4 V1.1.1 (2010-08)
6.4 Unsuccessful FTP Download session Ratio
Definition
This metric represents the ratio of unsuccessful FTP download sessions as a measure of the
Internet service accuracy.
This metric applies the definition of the indicator Unsuccessful data transmission ratio
defined in clause 5.3 of EG 202 057-4 [1] to the specific case of FTP download application.
From EG 202 057-4 [1]:
"The unsuccessful data transmission ratio is defined as the ratio of unsuccessful data
transmissions to the total number of data transmission attempts in a specified time period.
A data transmission is successful if a test file is transmitted completely and with no errors.
An attempt to transmit the test file should be considered unsuccessful if it takes longer than
60 s."
number of unsuccessful FTP download tests
Unsuccessful FTP Download Ratio =  ―――――――――――――――――――
number of tests generated
Assessment method This metric is determined by using a file transfer between the user access point and a server
placed outside the ISP domain. The files used for performing this test should have a size
linked with the download access bit rate. File sizes should be calculated or determined to
have file transfer duration of 10 seconds.

NOTE 1: For this measurement, incompressible files should be used for the data transfer.
NOTE 2: Preliminary tests can be performed to calibrate the volume of data to be
transferred to have a transfer duration of 10 seconds.
Unit %
Standardization
EG 202 057-4 [1]
reference ITU-T Recommendation G.1010 [3]
Significant Mandatory
Comment The threshold of 60 seconds refers to the limit for acceptable performance for bulk data
transmission/retrieval of ITU-T Recommendation G.1010 [3].

6.5 Unsuccessful FTP Upload session Ratio
Definition This metric represents the ratio of unsuccessful FTP upload sessions as a measure of the
Internet service accuracy.
This metric applies the definition of the indicator Unsuccessful data transmission ratio
defined in clause 5.3 of EG 202 057-4 [1] to the specific case of FTP upload application.
From EG 202 057-4 [1]:
"The unsuccessful data transmission ratio is defined as the ratio of unsuccessful data
transmissions to the total number of data transmission attempts in a specified time period.
A data transmission is successful if a test file is transmitted completely and with no errors.
An attempt to transmit the test file should be considered unsuccessful if it takes longer than
60 s."
number of unsuccessful FTP upload tests
Unsuccessful FTP Upload Ratio =  ―――――――――――――――――――
number of tests generated
Assessment method As for Unsuccessful FTP Download Ratio, this metric is determined by using a file transfer
between the user access point and a server placed outside the ISP domain. The files used
for performing this test should have a size linked with the upload access bit rate. File sizes
should be calculated or determined to have file transfer duration of 10 seconds.

NOTE 1: For this measurement, incompressible files should be used for the data transfer.
NOTE 2: Preliminary tests can be performed to calibrate the volume of data to be
transferred to have a transfer duration of 10 seconds.
Unit %
Standardization EG 202 057-4 [1]
reference
ITU-T Recommendation G.1010 [3]
Significant
Mandatory
Comment The threshold of 60 seconds refers to the limit for acceptable performance for bulk data
transmission/retrieval of ITU-T Recommendation G.1010 [3].

ETSI
15 Final draft ETSI ES 202 765-4 V1.1.1 (2010-08)
6.6 Unsuccessful HTTP session Ratio
Definition
This metric represents the ratio of unsuccessful web browsing attempts as a measure of the
Internet service accuracy.
This metric applies the definition of the indicator Unsuccessful data transmission ratio
defined in clause 5.3 of EG 202 057-4 [1] to the specific case of web browsing application.
From EG 202 057-4 [1]:
"The unsuccessful data transmission ratio is defined as the ratio of unsuccessful data
transmissions to the total number of data transmission attempts in a specified time period.
A data transmission is successful if a test file is transmitted completely and with no errors."

A web browsing session is considered unsuccessful if it takes longer than 10 s, indicated as
the limit for the loss of users' attention in the ITU-T Recommendation G.1030 [2].

number of unsuccessful HTTP tests
Unsuccessful HTTP Ratio = ―――――――――――――――――――
number of tests generated
Assessment method In practical way, this metric could be determined by downloading web pages from different
servers placed outside the ISP domain. The Webpage defined in TR 102 505 [i.2] should be
in the panel of web pages used for this test. A counter is started when the download request
of the Webpage is sent to the server. This counter is stopped when the page is fully loaded.
The test is considered unsuccessful if the downloading time is upper than 10 seconds.

NOTE: Care should be taken when using Web pages where contents vary. In this
scenario download times vary and measurements are not performed consistently.
Unit %
Standardization EG 202 057-4 [1]
reference ITU-T Recommendation G.1030 [2]
TR 102 505 [i.2]
Significant
Mandatory
Comment The Web pages that should be downloaded by the Test-analyzer could be:

• ETSI portal web page (http://portal.etsi.org/stq/WebReferencePage.asp).
• ad-hoc web page one (small size).
• ad-hoc web page two (medium size).
• ad-hoc web page three (big size).
It is important that the Web server contact for this test is outside the ISP domain; In this
condition, the complete link allowing the access to Web server is tested.

ETSI
16 Final draft ETSI ES 202 765-4 V1.1.1 (2010-08)
6.7 Ping Delay
Definition
This metric is representing the average Round Trip Time (RTT) to reach gaming sites. This
metric indicates the network performance in terms of the transmission parameters (delay and
delay variation).
This metric includes the definition of Delay (one way transmission time) defined in clause 5.5
of EG 202 057-4 [1], for the ping delay.
The delay is half the time, in milliseconds, that is needed for an ICMP Echo Request/Reply
(Ping) to a valid IP address.
The following statistics should be provided: the mean values of the delay in milliseconds and
the standard deviation of this delay when ping delay measurements are performed
successively.
The one-way delay is assessed by measuring half the time for a Echo Reply Message
according to RFC 792 [4].
Ping delay = average ( time (Echo Reply Message))
one_way_delay = average ( ½ time (Echo Reply Message) )
Assessment method
Unit
millisecond
Standardization EG 202 057-4 [1]
reference RFC 792 [4]
Significant
Mandatory
Comment
One Ping Request (TCP or UDP) is considered unanswered if the Ping Reply message is not
received within 5 s after the Ping Request is sent.

2 thresholds are defined based on tests made with intensive gamers:
• Ping delay lower than 40 ms: no trouble perceived in gaming actions,
• Ping delay upper than 80 ms: gaming not possible because delay too high.
When ping delay is between 40 ms and 80 ms, gaming is possible but the actions of game
become inaccurate and anticipation is necessary.
The servers where the Ping Request should be addressed are the following:

• Gaming server.
• Test-server.
As recommended above, the reference used to join these servers should be their IP address
and not their FQDN.
Because Ping messages generated traffic is very low, it is recommended to perform several
requests (5 to 10) successively. This will supply more precise statistics on the network
performances.
Generally ICMP messages tend to be down-prioritised by network operators. So ICMP
messages will not be used to perform this measure.

ETSI
17 Final draft ETSI ES 202 765-4 V1.1.1 (2010-08)
6.8 Internet Login Time
Definition
This indicator is an end to end measurement of service availability in term of capacity for an
Internet customer to access to the Internet.

This metric is compliant with the definition of Login time defined in clause 5.1 of
EG 202 057-4 [1].
From the ETSI Guide:
"The login time is the period starting when the data connection between the test-PC and the
Test Server has been established and finishing when the login process is successfully
completed."
Assessment method This metric is elaborated on the basis of Internet access tests generated by intrusive robots.

A test is considered as successful if the following events are fulfilled:

1) user identification and authorization;
2) a public IP address is attributed to the CPE;
3) successful answer to a DNS request for the ISP homepage;
4) successful ping to the ISP homepage.
Steps 1 to 4 must be completed within 10 seconds.

The value of this metric is the value in milliseconds within which the phases 1 to 4 for an
Internet access tests are successfully accomplished. Unsuccessful tests are excluded in this
indicator.
Unit millisecond
Standardization EG 202 057-4 [1]
reference
Significant
Mandatory
Comment For this indicator, the first attempt shall be separated from the following ones and this shall
be taken into account in the results presentation.

ETSI
18 Final draft ETSI ES 202 765-4 V1.1.1 (2010-08)
6.9 Web page download Speed
Definition
This indicator evaluates the average download performance when the customer is surfing on
the Internet.
This metric is compliant with the definition of Data transmission speed achieved, defined in
clause 5.2 of EG 202 057-4 [1].
From EG 202 057-4 [1]:
"The data transmission speed is defined as the data transmission rate that is achieved
separately for downloading and uploading specified test files between a remote web site and
a user's computer.
The following statistics should be provided separately for download and upload direction:

a) the highest 95 % of the data transmission rate in kbit/s achieved;
b) the lowest 5 % of the data transmission rate in kbit/s achieved;
c) the mean value and standard deviation of the data transmission rate in kibt/s."
Failed attempts have to be excluded by the statistics above. A web browsing session is
considered unsuccessful if it takes longer than 10 s, indicated as the limit for the loss of
users' attention in the ITU-T Recommendation G.1030 [2].
Assessment method As for Unsuccessful HTTP Ratio, this metric could be determined by downloading web pages
from different servers placed outside the ISP domai
...

SLOVENSKI STANDARD
01-oktober-2016
.DNRYRVWSUHQRVDJRYRUDLQYHþSUHGVWDYQLKYVHELQ6740HWULNDNDNRYRVWL
VWRULWHY4R6LQ]PRJOMLYRVWLRPUHåMDWHUPHULOQHPHWRGHGHO,QGLNDWRUML]D
QDG]RURYDQMHVWRULWHY]DYHþXGHOHåHQFHY
Speech and multimedia Transmission Quality (STQ) - QoS and network performance
metrics and measurement methods - Part 4: Indicators for supervision of Multiplay
services
Ta slovenski standard je istoveten z: ETSI ES 202 765-4 V1.1.1 (2010-10)
ICS:
33.040.35 Telefonska omrežja Telephone networks
2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.

SIST-V
...