ETSI EG 202 934 V1.1.1 (2012-01)

SLOVENSKI STANDARD
SIST-V ETSI/EG 202 934:2013
01-marec-2013
Uporabniška skupina - Ocenjevanje celotne kakovosti storitev (QoS), kot jo
dojemajo uporabniki - Definicija kazalnikov QoS za vse stopnje odnosov s
strankami
User Group - The assessment of the overall Quality of Services (QoS) as perceived by
the users - Definition of QoS indexes for all the customer relationship stages
Ta slovenski standard je istoveten z: EG 202 934
ICS:
03.080.01 Storitve na splošno Services in general
03.120.01 Kakovost na splošno Quality in general
SIST-V ETSI/EG 202 934:2013 en
2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.

---------------------- Page: 1 ----------------------

SIST-V ETSI/EG 202 934:2013

---------------------- Page: 2 ----------------------

SIST-V ETSI/EG 202 934:2013
ETSI EG 202 934 V1.1.1 (2012-01)






ETSI Guide
User Group;
The assessment of the overall Quality of Services (QoS)
as perceived by the users;
Definition of QoS indexes for all the
customer relationship stages

---------------------- Page: 3 ----------------------

SIST-V ETSI/EG 202 934:2013
 2 ETSI EG 202 934 V1.1.1 (2012-01)



Reference
DEG/USER-00028
Keywords
QoS, quality, user
ETSI
650 Route des Lucioles
F-06921 Sophia Antipolis Cedex - FRANCE

Tel.: +33 4 92 94 42 00  Fax: +33 4 93 65 47 16

Siret N° 348 623 562 00017 - NAF 742 C
Association à but non lucratif enregistrée à la
Sous-Préfecture de Grasse (06) N° 7803/88

Important notice
Individual copies of the present document can be downloaded from:
http://www.etsi.org
The present document may be made available in more than one electronic version or in print. In any case of existing or
perceived difference in contents between such versions, the reference version is the Portable Document Format (PDF).
In case of dispute, the reference shall be the printing on ETSI printers of the PDF version kept on a specific network drive
within ETSI Secretariat.
Users of the present document should be aware that the document may be subject to revision or change of status.
Information on the current status of this and other ETSI documents is available at
http://portal.etsi.org/tb/status/status.asp
If you find errors in the present document, please send your comment to one of the following services:
http://portal.etsi.org/chaircor/ETSI_support.asp
Copyright Notification
No part may be reproduced except as authorized by written permission.
The copyright and the foregoing restriction extend to reproduction in all media.

© European Telecommunications Standards Institute 2012.
All rights reserved.

TM TM TM
DECT , PLUGTESTS , UMTS and the ETSI logo are Trade Marks of ETSI registered for the benefit of its Members.
TM
3GPP and LTE™ are Trade Marks of ETSI registered for the benefit of its Members and
of the 3GPP Organizational Partners.
GSM® and the GSM logo are Trade Marks registered and owned by the GSM Association.
ETSI

---------------------- Page: 4 ----------------------

SIST-V ETSI/EG 202 934:2013
 3 ETSI EG 202 934 V1.1.1 (2012-01)
Contents
Intellectual Property Rights . 4
Foreword . 4
Introduction . 4
1 Scope . 5
2 References . 5
2.1 Normative references . 5
2.2 Informative references . 5
3 Definitions, symbols and abbreviations . 7
3.1 Definitions . 7
3.2 Symbols . 8
3.3 Abbreviations . 9
4 General context of the information on QoS assessment . 10
4.1 Overall organization of the QoS information . 10
4.2 Aspects impacting the QoS of the utilization stage . 11
4.2.1 Access . 11
4.2.2 Bearer service . 11
4.2.3 Service usage . 11
4.2.4 Presentation and user interface . 11
4.3 Requirements for the QoS parameter assessment . 12
4.4 QoS parameter selection . 12
4.5 Processing of the results . 12
5 Segmentation of the results . 13
6 Definition of a reference threshold for each QoS parameter . 13
7 Aggregation of the results within each CRS . 14
8 Aggregation of the QoS results of the various CRS of a specific service . 15
8.1 Qualitative aggregation . 15
8.2 Quantified aggregation . 16
8.2.1 QoS value ranges . 16
8.2.2 Highest QoS limit of the range . 16
8.2.3 Lowest QoS limit of the range . 17
8.2.4 Definition of a QoS index . 17
Annex A: Weighted aggregation of the results of the CRS of a specific service . 18
Annex B: Graphical representation . 19
B.1 Representation of the results within a CRS . 19
B.2 Representation of the QoS results for the various CRS of a specific service . 21
Annex C: Alternate graphical representation based on the "one-view visualization (OVV)
methodology" described in the ITU-T Recommendation P.505 . 24
C.1 Representation of the results within a CRS . 25
C.2 Representation of the QoS results for the various CRS of a specific service . 26
Annex D: Bibliography . 27
History . 28

ETSI

---------------------- Page: 5 ----------------------

SIST-V ETSI/EG 202 934:2013
 4 ETSI EG 202 934 V1.1.1 (2012-01)
Intellectual Property Rights
IPRs essential or potentially essential to the present document may have been declared to ETSI. The information
pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found
in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in
respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web
server (http://ipr.etsi.org).
Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee
can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web
server) which are, or may be, or may become, essential to the present document.
Foreword
This ETSI Guide (EG) has been produced by ETSI User Group (USER).
Introduction
The proposals contained in the present document are based on actual market realities: state of the art of the service
provided and current users' perceptions of the QoS provided. They aim at providing guidance on how consolidating the
various QoS assessments provided from various sources in a synthetic view that users can easily understand and use.
The intention is not to define values that providers should absolutely comply with in order to get some kind of label but
rather to help the customer to identify which providers are expected to fulfil their requirements. As these requirements
might strongly differ from one user to another one, the guidance are given in such a way that the consolidation can be
carried out differently to better fit the aim of the study and the kind of users under consideration.
ETSI

---------------------- Page: 6 ----------------------

SIST-V ETSI/EG 202 934:2013
 5 ETSI EG 202 934 V1.1.1 (2012-01)
1 Scope
The present document describes a methodology for the aggregation of QoS parameter results in order to obtain a view
onto the overall quality of the ICT services provided by Service Providers (SP) from an individual customer's
perspective; a suggestion for graphical representations including one following ITU-T Recommendation P.505 [i.28] is
proposed in annexes B and C. The purpose is not to identify the best SP but those whose QoS is complying in every
aspect with references considered as satisfactory from the users viewpoint.
The aim of this methodology is to be able to take into account preferences and expectations of individual customers as
well as the subjective and objective QoS assessments.
The present document does not define parameter value threshold nor specific weighting factors for the QoS parameters
but rather suggests possible ways for a weighted aggregation if so wished.
QoS indexes are defined using the QoS parameters provided in EG 202 009-2 [i.2] for all the stages of the customer
relationship including service utilization.
2 References
References are either specific (identified by date of publication and/or edition number or version number) or
non-specific. For specific references,only the cited version applies. For non-specific references, the latest version of the
referenced document (including any amendments) applies.
Referenced documents which are not found to be publicly available in the expected location might be found at
http://docbox.etsi.org/Reference.
NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee
their long term validity.
2.1 Normative references
The following referenced documents are necessary for the application of the present document.
Not applicable.
2.2 Informative references
The following referenced documents are not necessary for the application of the present document but they assist the
user with regard to a particular subject area.
[i.1] ETSI EG 202 009-1: "User Group; Quality of Telecom Services; Part 1: Methodology for
identification of parameters relevant to the Users".
[i.2] ETSI EG 202 009-2: "User Group; Quality of telecom services; Part 2: User related parameters on
a service specific basis".
[i.3] ETSI EG 202 057-1: "Speech Processing, Transmission and Quality Aspects (STQ); User related
QoS parameter definitions and measurements; Part 1: General".
[i.4] ETSI EG 202 057-2: "Speech and multimedia Transmission Quality (STQ); User related QoS
parameter definitions and measurements; Part 2: Voice telephony, Group 3 fax, modem data
services and SMS".
[i.5] ETSI EG 202 057-3: "Speech Processing, Transmission and Quality Aspects (STQ); User related
QoS parameter definitions and measurements; Part 3: QoS parameters specific to Public Land
Mobile Networks (PLMN)".
[i.6] ETSI EG 202 057-4: "Speech Processing, Transmission and Quality Aspects (STQ); User related
QoS parameter definitions and measurements; Part 4: Internet access".
ETSI

---------------------- Page: 7 ----------------------

SIST-V ETSI/EG 202 934:2013
 6 ETSI EG 202 934 V1.1.1 (2012-01)
[i.7] ETSI ES 202 737: "Speech and multimedia Transmission Quality (STQ); Transmission
requirements for narrowband VoIP terminals (handset and headset) from a QoS perspective as
perceived by the user".
[i.8] ETSI ES 202 738: "Speech and multimedia Transmission Quality (STQ); Transmission
requirements for narrowband VoIP loudspeaking and handsfree terminals from a QoS perspective
as perceived by the user".
[i.9] ETSI ES 202 739: "Speech and multimedia Transmission Quality (STQ); Transmission
requirements for wideband VoIP terminals (handset and headset) from a QoS perspective as
perceived by the user".
[i.10] ETSI ES 202 740: "Speech and multimedia Transmission Quality (STQ); Transmission
requirements for wideband VoIP loudspeaking and handsfree terminals from a QoS perspective as
perceived by the user".
[i.11] ETSI EG 202 765-1: "Speech and multimedia Transmission Quality (STQ); QoS and network
performance metrics and measurement methods; Part 1: General considerations".
[i.12] ETSI EG 202 765-2: "Speech Processing, Transmission and Quality Aspects (STQ); QoS and
network performance metrics and measurement methods Part 2 : Transmission Quality Indicator
combining Voice Quality Metrics".
[i.13] ETSI EG 202 765-3:"Speech and multimedia Transmission Quality (STQ); QoS and network
performance metrics and measurement methods; Part 3: Network performance metrics and
measurement methods in IP networks".
[i.14] ETSI EG 202 843: "User Group; Quality of ICT Services; Definitions and Methods for Assessing
the QoS parameters of the Customer Relationship Stages other than utilization".
[i.15] ETSI ES 202 667: "Speech and multimedia Transmission Quality (STQ); Audiovisual QoS for
communication over IP networks".
[i.16] ETSI ES 202 765-4: "Speech and multimedia Transmission Quality (STQ); QoS and network
performance metrics and measurement methods; Part 4: Indicators for supervision of Multiplay
services".
[i.17] ETSI TR 102 805-1 (V1.1.1): "User Group; End-to-end QoS management at the Network
Interfaces; Part 1: User's E2E QoS - Analysis of the NGN interfaces (user case)".
[i.18] ETSI TR 102 805-3 (V1.1.1): "User Group, End-to-end QoS management at the Network
Interfaces; Part 3: QoS informational structure".
[i.19] ETSI TR 102 806: "User Group; Analysis of current End-to-End QoS standardization state".
[i.20] ETSI TR 102 854: "User Group; The assessment of the overall Quality of Services (QoS) as
perceived by the users Review of practical examples of services QoS assessments".
[i.21] ETSI TS 102 250-1: "Speech and multimedia Transmission Quality (STQ); QoS aspects for
popular services in mobile networks; Part 1: Assessment of Quality of Service".
[i.22] ETSI TS 102 250-2: "Speech and multimedia Transmission Quality (STQ); QoS aspects for
popular services in mobile networks; Part 2: Definition of Quality of Service parameters and their
computation".
[i.23] ETSI ES 202 746: "Human Factors (HF); Personalization and User Profile Management; User
Profile Preferences and Information".
[i.24] ETSI TS 102 844: "User Group; Quality of Telecom Services; Conformity assessment;
Requirements for bodies providing QoS assessments and surveys".
[i.25] ETSI TS 102 852: "User Group; Quality of ICT Services; Assessment process of the QoS
parameters of the customer relationship stages".
[i.26] ITU-T Recommendation E.802 (02/2007): "Framework and methodologies for the determination
and application of QoS parameters".
ETSI

---------------------- Page: 8 ----------------------

SIST-V ETSI/EG 202 934:2013
 7 ETSI EG 202 934 V1.1.1 (2012-01)
[i.27] ITU-T Recommendation G.1010 (11/2001): "End-user multimedia QoS categories".
[i.28] ITU-T Recommendation P.505 (11/2005): "One-view visualization of speech quality measurement
results".
3 Definitions, symbols and abbreviations
3.1 Definitions
For the purposes of the present document, the following terms and definitions apply:
cessation: activities associated with the cessation of a telecommunication service from the time it is requested by a
customer, to the time it is completed to the satisfaction of the customer
commercial support: activities associated with the commercial support of a telecommunication service
complaint management: activities associated with the management of customer's complaints to the provider about the
service provided
documentation: activities associated with the provision of documentation to install, set-up and use the various features
of the service as well as to identify and fix possible troubles
NOTE: These activities include: Documentation for service activation and set-up and documentation for service
use.
establishment of the contract (Terms and conditions): activities from the customer decision to contract with the
provider to the time of effective contract
NOTE: Establishment of the contract is meant here for agreeing to the contractual conditions, conditions of use,
customer and provider commitments whether or not there is a formal signature of the contract.
metering/charging/billing: activities associated with the metering, charging and billing of a telecommunication service
to a customer
NOTE: This could be for one call, for a specified period or for a given billing amount, depending whether it is
about a bill or any other type of expense information, e.g. ebill, expense signal, real time expense
information on the provider website, AoC-S supplementary service, etc.
preliminary information/advertisement: activities related to information on the service provided on request of the
prospect or in newspapers, magazines, booklets, etc., to help him choosing the service and provider most appropriate to
his needs
repair-troubleshooting: activities associated with the restoration of a telecommunication service to the customer after
a fault resulting in partial or complete loss of service or service features
service alteration: activities associated with the alteration of a telecommunication service, from the time alteration to a
service is requested by the customer to the time this alteration is carried to the satisfaction of the customer
service provisioning: activities associated with the provision of a telecommunication service, from the time of effective
contract to the time the customer is able to use the service
technical support: activities associated with the technical support of a telecommunication service to help users
experiencing problems in the use of the service
technical upgrade: activities associated with the technical evolution of any component of the service at the provider
initiative
ETSI

---------------------- Page: 9 ----------------------

SIST-V ETSI/EG 202 934:2013
 8 ETSI EG 202 934 V1.1.1 (2012-01)
3.2 Symbols
For the purposes of the present document, the following symbols apply:
NOTE: This list of symbols covers only the parameters Pnnnn used in the present document using the
EG 202 843 [i.14] definitions. Additional ones PTnnnn have been added to cover QoS parameters
concerning the use of telephony service.
P100 Frequency of customer complaints about PI [N/t]: Number of customers' complaints about PI per
million subscribers
P101a Integrity of PI [OR]: Content - Was the relevant information provided as you expected?
P101b Integrity of PI [OR]: Language - Was the information provided clear and understandable without
any ambiguity?
P101c Integrity of PI [OR]: Style - How would you rate the overall style, presentation and
professionalism of the preliminary information provided?
P102 Pricing transparency [OR]: Did you find the pricing information comprehensible?
P103 Availability of PI [%]: Could you retrieve the preliminary information easily?
P200 Frequency of customer complaints about contract establishment [N/t]: Number of customers'
complaints about contract establishment per million subscribers
P201 Integrity of contract information [OR]: How would you rate the integrity of the contractual
document?
P202 Compliance of contractual terms with PI [%]: Was the contract document compliant to the
previously provided preliminary information?
P203 Flexibility for customisation before contract [OR]: How would you rate the flexibility of your
service provider to customise the contract before signature e.g. by applying options?
P204 Ease and flexibility to amend terms after formal contract [OR]: How would you rate the flexibility
of your service provider to further adapt the contract after signature e.g. by applying options?
P300 Frequency of customer complaints about provisioning [N/t]: Number of customers' complaints
about provisioning per million subscribers
P303a Successful provisioning within a specified period [%] - existing subscriber line
P303b Successful provisioning within a specified period [%] - new subscriber line
P309a Provisioning time [Time & %] - existing subscriber line
P309b Provisioning time [Time & %] - new subscriber line
P600 Frequency of customer complaints about service support [N/t]: Number of customers complaints
about service support per million subscribers
P628a Response time of the technical support [Time & %]
P628b Response time of the technical support [Time & %]
P661 Accessibility of the complaint management desk [%]: Concerning your latest attempt to access the
complaint management desk of your service provider: Did you succeed in accessing it?
P662 Recognition of the customer complaints [%]: Concerning your latest attempt to access the
complaint management desk of your service provider: Was your complaint accepted?
P663 Complaint solutions not complete and correct first time [%]: Was the complaint solved to your
satisfaction at the first attempt by the service provider?
P664 Complaint solutions achieved within a specified period [%]: Concerning your latest accepted
complaint: Was the complaint finally solved to your satisfaction by the service provider?
P665 Integrity of complaint resolution [%]: Concerning your latest accepted complaint: Was your
complaint resolved correctly?
P666a Customer perception of the complaint management [OR]: Customer perception of complaint
management (Assurance): How would your rate the service provider's complaint management
related to assurance at all?
P666b Customer perception of the complaint management [OR]: Customer perception of complaint
management (Empathy): How would your rate the service provider's complaint management
related to empathy at all?
P666c Customer perception of the complaint management [OR]: Customer perception of complaint
management (Responsiveness): How would your rate the service provider's complaint
management related to responsiveness at all?
P667 Overall quality of the complaint management process [OR]: How would your rate the overall
handling of the complaint management process?
P706a Fault repair time [Time & %] - Time for 95 % fault repair
P706b Fault repair time [Time & %] - % faults repaired within a 48 hours delay
P707 Frequency of customer complaints related to repair services [N/t]: Number of customers
complaints related to repair services per million subscribers
ETSI

---------------------- Page: 10 ----------------------

SIST-V ETSI/EG 202 934:2013
 9 ETSI EG 202 934 V1.1.1 (2012-01)
P800 Frequency of customer complaints about billing [N/t]: Number of customers complaints about
billing per million subscribers
P801 Accessibility of the tariff information [%]: Concerning your latest attempt to access your
provider's tariff information: Were you able to access the tariff information?
P802 Successful notification of exceeding billing budget [%]: If you are using a notification service
when you reach a predefined budget level: Concerning your latest exceeding of budget: Were you
notified accordingly when you exceeded your budget?
P804 Accessibility of the account management [%]: Concerning your latest attempt to access the
account status at your service provider: Did you succeed in accessing it?
P806 Timeliness of bill delivery [%]: Did you receive all the expected bills throughout the last
6 months?
P807 Bill delivery delay [Time]: If you experienced a delay in bill delivery: How many days was the bill
delayed?
P808 Late notification of amount due [%]: Has the bill been received before the direct debit was
executed?
P809 Modes of billing information transfer [Number]: How many ways do you have to access your
accounting information?
P810 Bill correctness complaints [%]: Percentage of bills resulting in a customer complaint per point of
billing per year
P1004a Contractual cessation achieved [%] within 10 days
P1004b Contractual cessation achieved [%]: time needed (days) to achieved 95 % of cessations requested
P1004c Contractual cessation achieved [%]: time needed (days) to achieved 99 % of cessations requested
P1008 Frequency of customer complaints related to cessation [N/t]: Number of customers' complaints
related to cessation per million subscribers
PT000 Frequency of customer complaints related to use of service [N/t]: Number of customers complaints
related to use of service per million subscribers
PT001a Fault report rate per fixed access lines
PT001b Fault report rate per fixed access lines within 30 days after the delivery
PT002a Unsuccessful call ratio - domestic calls
PT002b Unsuccessful call ratio - international calls
PT003a Call set up time - domestic calls
PT003b Call set up time - international calls
PT004 Speech Quality (MOS)
3.3 Abbreviations
For the purposes of the present document, the following abbreviations apply:
CRS Customer Relationship Stage
ICT Information and Communication Technology
IP-TV Internet Protocol Television
MOS Mean Opinion Score
OR Opinion Rating
OVV One-View Visualization
QoE Quality of Experience
QoS Quality of Service
SLA Service Level Agreement
SP Service Provider
ToIP Telephony over IP
VoD Video on Demand
ETSI

---------------------- Page: 11 ----------------------

SIST-V ETSI/EG 202 934:2013
 10 ETSI EG 202 934 V1.1.1 (2012-01)
4 General context of the information on QoS
assessment
A vast array of documentation is today available on the QoS assessment from the user viewpoint e.g. among others,
ITU-T Recommendation E.802 [i.26], EG 202 009-2 [i.2], EG 202 843 [i.14] or TS 102 852 [i.25]. The use of such
standards should take into account the features of each service and even the specific QoS requirements of each
component of this service as detailed in TR 102 805-1 [i.17], TR 102 805-3 [i.18] and TR 102 806 [i.19]. Some generic
principles to tackle the specificities of services like voice, IP-TV web surfing or emails, are given in
ES 202 765-2 [i.12] and ES 202 765-4 [i.16], TS 102 250-2 [i.22] as well as in ITU-T Recommendation G.1010 [i.27].
Each document provides only a part of the overall QoS as perceived by the user, but there are a lack of documents to
define the overall quality for the whole CRS.
Current studies are also developing the concept of Quality of Experience (QoE) that takes into account in addition to
QoS, the context of use, the user's expectations. In particular QoE takes into account value for money as well as the
psychological aspects that may influence user satisfaction. In particular, users could get from fora or social networks
such as "Facebook", "YouTube" or alike feedback from other users that might influen
...