ETSI TR 102 643 V1.0.0 (2009-11)

ETSI TR 102 643 V1.0.0 (2009-11)
Technical Report


Human Factors (HF);
Quality of Experience (QoE) requirements for
real-time communication services

---------------------- Page: 1 ----------------------
2 ETSI TR 102 643 V1.0.0 (2009-11)



Reference
DTR/HF-00111
Keywords
interaction, quality, service
ETSI
650 Route des Lucioles
F-06921 Sophia Antipolis Cedex - FRANCE

Tel.: +33 4 92 94 42 00  Fax: +33 4 93 65 47 16

Siret N° 348 623 562 00017 - NAF 742 C
Association à but non lucratif enregistrée à la
Sous-Préfecture de Grasse (06) N° 7803/88

Important notice
Individual copies of the present document can be downloaded from:
http://www.etsi.org
The present document may be made available in more than one electronic version or in print. In any case of existing or
perceived difference in contents between such versions, the reference version is the Portable Document Format (PDF).
In case of dispute, the reference shall be the printing on ETSI printers of the PDF version kept on a specific network drive
within ETSI Secretariat.
Users of the present document should be aware that the document may be subject to revision or change of status.
Information on the current status of this and other ETSI documents is available at
http://portal.etsi.org/tb/status/status.asp
If you find errors in the present document, please send your comment to one of the following services:
http://portal.etsi.org/chaircor/ETSI_support.asp
Copyright Notification
No part may be reproduced except as authorized by written permission.
The copyright and the foregoing restriction extend to reproduction in all media.

© European Telecommunications Standards Institute 2009.
All rights reserved.

TM TM TM TM
DECT , PLUGTESTS , UMTS , TIPHON , the TIPHON logo and the ETSI logo are Trade Marks of ETSI registered
for the benefit of its Members.
TM
3GPP is a Trade Mark of ETSI registered for the benefit of its Members and of the 3GPP Organizational Partners.
LTE™ is a Trade Mark of ETSI currently being registered
for the benefit of its Members and of the 3GPP Organizational Partners.
GSM® and the GSM logo are Trade Marks registered and owned by the GSM Association.
ETSI

---------------------- Page: 2 ----------------------
3 ETSI TR 102 643 V1.0.0 (2009-11)
Contents
Intellectual Property Rights . 4
Foreword . 4
1 Scope . 5
2 References . 5
2.1 Normative references . 5
2.2 Informative references . 5
3 Definitions and abbreviations . 8
3.1 Definitions . 8
3.2 Abbreviations . 12
4 What is Quality of Experience (QoE)? . 12
4.1 A definition of QoE for the current document. 13
4.2 QoE and QoS . 13
4.3 Usability and user experience . 14
4.3.1 Technology-centred approach: Quality of Service (QoS) . 15
4.3.2 User-centred approaches: QoE and User-perceived QoS . 15
4.3.2.1 Quality of Perception (QoP) . 15
4.3.2.2 Quality of Experience (QoE). 16
4.3.3 The co-existence of QoS and QoE . 17
4.4 The QoE context of the present document . 17
4.4.1 The Choosing situation . 18
4.4.2 The Usage situation . 19
5 For whom are QoE data important? . 20
6 How can QoE data help? . 20
6.1 To prevent churn . 20
6.2 To prevent product or service rejections . 21
6.3 To optimise a product or service . 21
6.4 By expressing QoE expressed in terms of QoS . 21
7 Providing QoE data for real-time communication services. 21
7.1 Real-time communication services . 21
7.2 Guidelines development and dissemination . 27
7.2.1 Guideline development . 27
7.2.2 Guideline dissemination . 29
8 Web-Based Guideline and Tutorial System for improved dissemination and application of QoE
data . 29
8.1 The main facilities of the web-based system . 29
8.1.1 Navigation facility . 30
8.1.2 Education facility . 31
8.1.3 Dissemination facility . 32
8.2 Types of Guidelines. 32
8.3 Maturity and restrictions of the guidelines . 34
8.4 Maintenance of the Web-based system . 34
9 Towards generic QoE guidelines by including non real-time services . 35
10 Conclusions . 35
11 Recommendations for further work . 35
11.1 Extend towards generic QoE guidelines . 35
11.2 Document the user test framework and guideline derivation methodology in a separate ETSI document . 36
11.3 Update the Web-Based Guideline and Tutorial System . 36
History . 37
ETSI

---------------------- Page: 3 ----------------------
4 ETSI TR 102 643 V1.0.0 (2009-11)
Intellectual Property Rights
IPRs essential or potentially essential to the present document may have been declared to ETSI. The information
pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found
in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in
respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web
server (http://webapp.etsi.org/IPR/home.asp).
Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee
can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web
server) which are, or may be, or may become, essential to the present document.
Foreword
This Technical Report (TR) has been produced by ETSI Technical Committee Human Factors (HF).
ETSI

---------------------- Page: 4 ----------------------
5 ETSI TR 102 643 V1.0.0 (2009-11)
1 Scope
The present document is based on Guidelines produced in Specialist ETSI Task Force 354 and a Web-Based Guideline
access and Tutorial System (WBGTS) (http://portal.etsi.org/stfs/STF_HomePages/STF354/STF354.asp). The main
content of the WBGTS are Quality of Experience (QoE) guidelines for real-time communication services expressed in
Quality of Service (QoS) terms.
The Guidelines and the present document are primarily intended for professionals in network operator, equipment
manufacturer and service provider organisations who are concerned with the user experience of communication
services. Over the last 10 years that include pre-cursor projects to STF 354 about 2000 intended guideline users have
been involved in the development work of the WBGTS. The present document describes an assessment of the current
guidelines and the tutorial system and identifies needs for future work.
2 References
References are either specific (identified by date of publication and/or edition number or version number) or
non-specific.
• For a specific reference, subsequent revisions do not apply.
• Non-specific reference may be made only to a complete document or a part thereof and only in the following
cases:
- if it is accepted that it will be possible to use all future changes of the referenced document for the
purposes of the referring document;
- for informative references.
Referenced documents which are not found to be publicly available in the expected location might be found at
http://docbox.etsi.org/Reference.
NOTE: While any hyperlinks included in this clause were valid at the time of publication ETSI cannot guarantee
their long term validity.
2.1 Normative references
The following referenced documents are indispensable for the application of the present document. For dated
references, only the edition cited applies. For non-specific references, the latest edition of the referenced document
(including any amendments) applies.
Not applicable.
2.2 Informative references
The following referenced documents are not essential to the use of the present document but they assist the user with
regard to a particular subject area. For non-specific references, the latest version of the referenced document (including
any amendments) applies.
[i.1] ANSI T1.522-2000: "Quality of Service for Business Multimedia Conferencing".
[i.2] Birdwhistle, R. (1970): "Kinesics and Context: Essays in Body Motion Communication",
Harmondsworth: Penguin.
[i.3] Blythe, M.; Overbeeke, K., Monk, A., Wright, P. (Eds) (2004): "Funology: From Usability to
Enjoyment" Kluwer Academic.
ETSI

---------------------- Page: 5 ----------------------
6 ETSI TR 102 643 V1.0.0 (2009-11)
[i.4] Brooks P, Brundell P, Hamnes K, Heiestad S, Heim J, Hestnes B, Heydari B, O'Malley C,
Schliemann T, Skjetne JH, Ulseth T (1999): "Final Report. ACTS Project AC314 Vis-à-Vis:
Fitness-for-Purpose of Videotelephony in Face-to-Face Situations", CEC Deliverable
A314/NSS/PB/DS/P/005/b1, June 1999.
[i.5] Brooks P, Schliemann T, Hestnes B, Frowein H, Aaby C, O'Malley C, (2003): "Final Report
Project IST-1999-11577 Eye-2-Eye: Fitness-for-Purpose of Person-Person Communication
Technologies", EC Deliverable IST11577/SEF/DIS/DS/Pub/008/b1, June 2003.
[i.6] Brooks, P. & Hestnes, B. (2003): "User-centred technical guidelines for real-time human
communication services: Requirements and derivation", Proceedings of the 19th International
Symposium on Human Factors in Telecommunication, Berlin, Germany, December 1-4 2003,
pp. 11-18.
[i.7] Brooks, P., Hestnes, B., Heiestad, S., Aaby, C. (2006): "Communicating Quality of Experience
data for the development of multimedia services", Proceedings of the 20th International
Symposium on Human Factors in Telecommunication, Sophia Antipolis, France,
March 21-23 2006.
NOTE: Available from http://www.hft.org/HFT06/HFT_06_programme.htm.
[i.8] Bruce, V. (1996): "The role of the face in communication: Implications for videophone design",
Interacting with Computers, 8(2), 166-176.
[i.9] Ekman, P. & Friesen, W. V. (1975): "Unmasking the face. A guide to recognizing emotions from
facial clues", Englewood Cliffs, New Jersey: Prentice-Hall.
[i.10] ETSI ES 202 667: "Speech and multimedia Transmission Quality (STQ); Audiovisual QoS for
communication over IP networks".
[i.11] ETSI ES 202 737: "Speech and multimedia Transmission Quality (STQ); Transmission
requirements for narrowband VoIP terminals (handset and headset) from a QoS perspective as
perceived by the user".
[i.12] ETSI ES 202 738: "Speech and multimedia Transmission Quality (STQ); Transmission
requirements for narrowband VoIP loudspeaking and handsfree terminals from a QoS perspective
as perceived by the user".
[i.13] ETSI ES 202 739: "Speech and multimediaTransmission Quality (STQ); Transmission
requirements for wideband VoIP terminals (handset and headset) from a QoS perspective as
perceived by the user".
[i.14] ETSI ES 202 740: "Speech and multimedia Transmission Quality (STQ); Transmission
requirements for wideband VoIP loudspeaking and handsfree terminals from a QoS perspective as
perceived by the user".
[i.15] ETSI EG 202 534: "Human Factors (HF); guidelines for real-time person-to-person
communication services".
[i.16] ETSI ETR 160: "Human Factors (HF); Human Factors aspects of multimedia
telecommunications".
[i.17] ETSI ETR 297: "Human Factors (HF); Human Factors in Videotelephony".
[i.18] ETSI TR 102 274: "Human Factors (HF); guidelines for real-time person-to-person
communication services".
[i.19] ETSI TR 102 479: "Telecommunications and Internet converged Services and Protocols for
Advanced Networking (TISPAN); Review of available material on QoS requirements of
Multimedia Services".
[i.20] ETSI TR 102 535: "Human Factors (HF); Guidelines for real-time person-to-person
communication services; Future requirements".
ETSI

---------------------- Page: 6 ----------------------
7 ETSI TR 102 643 V1.0.0 (2009-11)
[i.21] ETSI TS 181 016: "Telecommunications and Internet converged Services and Protocols for
Advanced Networking (TISPAN); Service Layer Requirements to integrate NGN Services and
IPTV".
[i.22] ETSI TS 122 105: "Universal Mobile Telecommunications System (UMTS); Services and service
capabilities (3GPP TS 22.105 version 8.4.0 Release 8)".
[i.23] Heim, J., Asting, T., Brandtzæg, P., Brooks, P., Skjetne, JH., Hestnes, B. et al. (2000): "Initial
Verification of Real-time Communication Requirements", Project IST-1999-11577 Eye-2-Eye:
Fitness-for-Purpose of Person-Person Communication Technologies, EC Deliverable
IST11577/SEF/DIS/DS/5FP/001/b1, July 2000.
[i.24] Hestnes B, Heiestad S, Ulseth T, Schliemann T, Brooks P, Følstad A, Frowein H, Aaby C,
O'Malley C, Brundell P: "Fitness-for-Purpose guidelines for Person-Person Communication",
Project IST-1999-11577 Eye-2-Eye: Fitness-for-Purpose of Person-Person Communication
Technologies". EC Deliverable IST11577/TEL/RAD/DS/Pub/065/b1, March 2003.
[i.25] Hestnes, B., Brooks, P., Heiestad, S., Ulseth, T., Aaby., C. (2003): "Quality of Experience in real-
time person-person communication - User based QoS expressed in technical network QoS terms",
Proceedings of the 19th International Symposium on Human Factors in Telecommunication,
Berlin, Germany, December 1-4 2003, pp. 3-10.
[i.26] Hestnes, B., Brooks, P., Heiestad, S. (2009): "QoE (Quality of Experience) - measuring QoE for
improving the usage of telecommunication services", Telenor Research & Innovation Research
Report, Telenor Research & Innovation-R21/2009, Telenor, Fornebu, Norway.
[i.27] ISO 9241-11 (1998): "Ergonomic requirements for office work with visual display terminals
(VDTs) Part 11: Guidance on usability".
[i.28] ITU-R Recommendation BS 1534-1: "Method for the Subjective Assessment of Intermediate
Quality Level of Coding Systems".
[i.29] ITU-R Recommendation BT.1359-1 (2003): "Method for the Subjective Assessment of
Intermediate Quality Level of Coding Systems".
[i.30] ITU-T Series H: "Audiovisual and multimedia systems, Supplement 1 (05/99) (1999) Application
profile - Sign language and lip-reading real-time conversation using low bit-rate video
communication".
[i.31] ITU-T Recommendation E.800 (2008): "Definitions of terms related to quality of service".
[i.32] ITU-T Recommendation E.860: "Framework of a service level agreement".
[i.33] ITU-T Recommendation F.700 (2000) Framework Recommendation for multimedia services.
[i.34] ITU-T Recommendation G.108 (1999) "Application of the E-model: A planning guide".
[i.35] ITU-T Recommendation G.114 (2003) "One-way transmission time", International.
[i.36] ITU-T Recommendation G.1010 (2001): "End-user multimedia QoS categories".
[i.37] ITU-T Recommendation P.10/G.100: "Amendment 2: New definitions for inclusion in
Recommendation ITU-T P.10/G.100", International Telecommunication Union, Geneva,
Switzerland, 2008.
[i.38] ITU-T Recommendation Y.1541 (2006): "Network performance objectives for IP-based services",
International Telecommunication Union, Geneva, Switzerland.
[i.39] McGurk, H., & McDonald, J. (1976): "Hearing lips and seeing voices", Nature 264, 746-748.
[i.40] Nokia (2004): "Quality of Experience (QoE) of mobile services: Can it be measured and
improved?" White Paper, Nokia Corporation.
[i.41] Reeves, B. and Nass, C. (1996): "The media equation: How people treat computers, television and
new media like real people and places", Cambridge University Press. Communication
Technologies, EC Deliverable IST11577/TEL/RAD/DS/Pub/065/b1, March 2003.
ETSI

---------------------- Page: 7 ----------------------
8 ETSI TR 102 643 V1.0.0 (2009-11)
[i.42] Soldani, D., Li, M., Cuny, R. (2006): "QoS and QoE management in UMTS cellular systems",
John Wiley & Sons Ltd: England.
[i.43] Summerfield, Q. (1992): "Lipreading and audio-visual speech perception", Philosophical
Transactions of the Royal Society of London, B335, 71-78.
[i.44] Wikipedia (2009): http://en.wikipedia.org/.
3 Definitions and abbreviations
3.1 Definitions
For the purposes of the present document, the following terms and definitions apply:
asynchrony: when audio and video information that leaves one communicating party at the same time is received by
the other communicating party at different times
NOTE 1: E.g. typically the audio information arrives before the video information in an asynchronous situation.
NOTE 2: It is calculated as audio delay subtracted from video delay (e.g. if audio delay is 50 ms and video delay is
200 ms, then asynchrony is 150 ms; if audio delay is 100 ms and video delay is 50 ms, then asynchrony is
-50 ms).
audio communication: use of a service that transmits voice in real-time over a telecommunication network, such as
ordinary telephony with a handset and loud-speaking audio conferencing
audio conferencing: telephone service that does not rely on amplification of the voice signal in very close proximity to
the recipient's ear
EXAMPLE: Loud-speaking audio communication.
audio delay: time required for an audio signal generated at the talker's mouth to reach the listener's ear
audio protocol: set of rules defining the way audio information is represented in a network
audio telephony: "ordinary" telephone service using a handset as distinct from loud-speaking audio conferencing
avatar telephony: service for transmitting voice signals in real-time over a telecommunication network in combination
with a graphical (human) representation of the speaker
bandwidth: range of frequencies which can safely be conveyed in a communication channel
burst packet loss: loss of two or more packets in sequence
communication media: types of information with which humans communicate
NOTE: Examples are text, audio and moving image (graphics and video). This is consistent with the "Nature of
information" component of the ETSI definition of a representation medium, which has various possible
coded forms (ETR 160 [i.16]).
communication service: service that is provided via a telecommunication network
NOTE: Examples are audio-telephony, email, videoconferencing, avatar-telephony, audio conferencing.
communication situation: combination of task, motive, content and user (group) characteristics
communication task: what the end-users (want to) do with a communication service
NOTE: E.g. social chatting, buying or selling shares, conducting a job interview, etc.
communicative behaviour: end-user behaviour while using a communication service, including turn taking,
interruptions, verbal and non-verbal back-channels and gaze
ETSI

---------------------- Page: 8 ----------------------
9 ETSI TR 102 643 V1.0.0 (2009-11)
conversational text: See real-time text.
data communication: use of a service that transmits personal computer-based information (e.g. presentation slides)
data conferencing: See data communication.
duration: length of time of the communication task
dyadic communication: (distance) communication between two people
effectiveness: accuracy and completeness with which specified users can achieve specified goals in particular
environments
NOTE: See ISO 9241 [i.27] definition.
efficiency: resources expended in relation to the accuracy and completeness of goals achieved
NOTE: See ISO 9241 [i.27] definition.
end-users: people who use a communication service
fitness-for-purpose: correct balance between technological performance and human performance, such that the
interaction is both sufficient and beneficial for communication and consistent with human expectations
frame-rate: frequency by which a full video frame is updated, sometimes called video temporal resolution or image
frequency
group: (distance) communication between three or more people
NOTE: Either in a point-to-point or a multi-point configuration.
interpersonal perception: extent to which the perception of the other person's attributes (how likeable, intelligent,
friendly, etc.) is positive or negative
media effects: effect a particular communication medium has on an end-users task outcome, communicative behaviour,
attitudes and beliefs
media/medium: See Communication Media/Medium.
monitor size: number in inches of the diagonal of the image screen on a screen
multimedia communication: use of a service that transmits voice, video and data signals in real-time over a
telecommunication network
multimedia conferencing: service for transmitting voice, video and data signals over a telecommunication network
multi-point: distance communication between three or more locations
network quality of service: degree of conformance of the service delivered to a user by a provider with an agreement
between them
NOTE: From ITU-T Recommendation E.860 [i.32].
packet loss: loss of one packet that can be described using a certain statistical model
packet size: magnitude of a data being transmitted over a packet switching network in number of Bytes
personal involvement: extent to which the communication parties are committed to the outcome of the task or perform
the task more on behalf of another party than themselves
point-to-point: communication between two locations
quality of experience (1): measure of user performance based on both objective and subjective psychological measures
of using an ICT service or product
NOTE 1: It takes into account technical parameters (e.g. QoS) and usage context variables (e.g. communication
task) and measures both the process and outcomes of communication (e.g. user effectiveness, efficiency,
satisfaction and enjoyment).
ETSI

---------------------- Page: 9 ----------------------
10 ETSI TR 102 643 V1.0.0 (2009-11)
NOTE 2: The appropriate psychological measures will be dependent on the communication context. Objective
psychological measures do not rely on the opinion of the user (e.g. task completion time measured in
seconds, task accuracy measured in number of errors). Subjective psychological measures are based on
the opinion of the user (e.g. perceived quality of medium, satisfaction with a service).
EXAMPLE: A service provider may conclude that a service with a certain level of QoS used for a particular
communication situation offers users excellent QoE, whist with a different level of QoS provides
poor QoE.
quality of experience (QoE) (2): overall acceptability of an application or service, as perceived subjectively by the
end-user
NOTE 1: Quality of experience includes the complete end-to-end system effects (client, terminal, network, services
infrastructure, etc.).
NOTE 2: Overall acceptability may be influenced by user expectations and context.
NOTE 3: ITU-T Recommendation P.10 [i.37]/G.100 Amendment 2 definition.
quality of service: totality of characteristics of a telecommunications service that bear on its ability to satisfy stated and
implied needs of the user of the service
NOTE: ITU-T Recommendation E.800 [i.31] definition.
Quality of service delivered/achieved by service provider (QoSD): statement of the level of QoS achieved or
delivered to the customer
NOTE 1: Achieved or delivered QoS is expressed by metrics for the pertinent parameters for a service.
NOTE 2: ITU-T Recommendation E.800 [i.31] definition.
Quality of service experienced/perceived by customer/user (QoSE): statement expressing the level of quality that
customers/users believe they have experienced
NOTE 1: The level of QoS experienced and/or perceived by the customer/user may be expressed by an opinion
rating.
NOTE 2: QoSE has two main man components: quantitative and qualitative. The quantitative component can be
influenced by the complete end-to-end system effects (network infrastructure).
NOTE 3: The qualitative component can be influenced by user expectations, ambient conditions, psychological
factors, application context, etc.
NOTE 4: QoSE may also be considered as QoSD (QoS delivered/achieved by service provider) received and
interpreted by a user with the pertinent qualitative factors influencing his/her perception of the service.
NOTE 5: ITU-T Recommendation E.800 [i.31] definition.
Real-time (1): describes information and communication technologies that are able to generate and deliver information
in a time-frame similar to the real-life process that it is assisting
EXAMPLE 1: Real-time charging and billing information is to be generated, processed, and transported to a
desired conclusion in less than 1 second
EXAMPLE 2: Refers to the generation of network management information in a time frame comparative to the
real life process that it is controlling or monitoring
real time (2): Occurring immediately. The term is used to describe a number of different computer features. For
example, real-time operating systems are systems that respond to input immediately. They are used for such tasks as
navigation, in which the computer reacts to a steady flow of new information without interruption. Most general-
purpose operating systems are not real-time because they can take a few seconds, or even minutes, to
...