ETSI EG 202 670 V1.1.2 (2010-03)

ETSI EG 202 670 V1.1.2 (2010-03)
ETSI Guide


Human Factors (HF);
User Experience Guidelines
for real-time communication services
expressed in Quality of Service terms

---------------------- Page: 1 ----------------------
2 ETSI EG 202 670 V1.1.2 (2010-03)



Reference
REG/HF-00133
Keywords
interaction, quality, service
ETSI
650 Route des Lucioles
F-06921 Sophia Antipolis Cedex - FRANCE

Tel.: +33 4 92 94 42 00  Fax: +33 4 93 65 47 16

Siret N° 348 623 562 00017 - NAF 742 C
Association à but non lucratif enregistrée à la
Sous-Préfecture de Grasse (06) N° 7803/88

Important notice
Individual copies of the present document can be downloaded from:
http://www.etsi.org
The present document may be made available in more than one electronic version or in print. In any case of existing or
perceived difference in contents between such versions, the reference version is the Portable Document Format (PDF).
In case of dispute, the reference shall be the printing on ETSI printers of the PDF version kept on a specific network drive
within ETSI Secretariat.
Users of the present document should be aware that the document may be subject to revision or change of status.
Information on the current status of this and other ETSI documents is available at
http://portal.etsi.org/tb/status/status.asp
If you find errors in the present document, please send your comment to one of the following services:
http://portal.etsi.org/chaircor/ETSI_support.asp
Copyright Notification
No part may be reproduced except as authorized by written permission.
The copyright and the foregoing restriction extend to reproduction in all media.

© European Telecommunications Standards Institute 2010.
All rights reserved.

TM TM TM TM
DECT , PLUGTESTS , UMTS , TIPHON , the TIPHON logo and the ETSI logo are Trade Marks of ETSI registered
for the benefit of its Members.
TM
3GPP is a Trade Mark of ETSI registered for the benefit of its Members and of the 3GPP Organizational Partners.
LTE™ is a Trade Mark of ETSI currently being registered
for the benefit of its Members and of the 3GPP Organizational Partners.
GSM® and the GSM logo are Trade Marks registered and owned by the GSM Association.
ETSI

---------------------- Page: 2 ----------------------
3 ETSI EG 202 670 V1.1.2 (2010-03)
Contents
Intellectual Property Rights . 5
Foreword . 5
Introduction . 5
1 Scope . 7
2 References . 7
2.1 Normative references . 7
2.2 Informative references . 8
3 Definitions and abbreviations . 11
3.1 Definitions . 11
3.2 Abbreviations . 15
4 Overview of the real-time communication services addressed in the present document . 16
4.1 Text communication . 20
4.2 Speech communication . 20
4.3 Video communication (including speech) . 20
4.3.1 Face-to-face video communication . 20
4.3.2 Remote inspection or 'see what I see' (SWIS) video communication . 20
4.4 Multimedia communication. 21
4.5 Real-time games . 21
4.6 TV including PC TV and Mobile TV . 21
5 Overview of user experience guidelines expressed in Quality of Service terms . 21
6 Advice on using the Guide . 24
6.1 Requirements for guidelines . 25
6.2 The structure of the guidelines . 25
6.3 Maturity and restrictions of the guidelines . 26
7 The guidelines . 26
7.1 Text communication . 27
7.2 Speech communication . 30
7.3 Video communication (including speech) . 41
7.3.1 Video communication: Face-to-face . 41
7.3.2 Video communication: Remote inspection . 57
7.3.3 Video communication: Multi-point and heterogeneous networks . 63
7.4 Multimedia communication. 64
7.5 Real-time games . 72
7.6 TV . 74
7.6.1 PC TV . 74
7.6.2 Mobile TV . 76
7.7 Service selection . 85
Annex A: Requirements for guidelines and a web-based guideline and tutorial system . 92
A.1 Requirement derivation process . . 92
A.2 Requirements for guidelines . 92
A.2.1 Provide information on key topics of concern that will aid development choices . 92
A.2.2 Provide information on related concepts . 93
A.2.3 Provide user experience data that can be used from different perspectives . 93
A.2.4 Link QoS and user experience variables . 93
A.2.5 Provide information about user behaviour that is feasible to apply . 94
A.3 Requirements for a web-based system . 94
Annex B: Sources for the guidelines . 95
ETSI

---------------------- Page: 3 ----------------------
4 ETSI EG 202 670 V1.1.2 (2010-03)
B.1 Bespoke user studies . 95
B.1.1 Laboratory experiments . 95
B.1.2 Field studies. 96
B.1.3 Survey . 96
B.1.4 Expert review and expert panels . 96
B.2 Existing literature . 97
History . 98

ETSI

---------------------- Page: 4 ----------------------
5 ETSI EG 202 670 V1.1.2 (2010-03)
Intellectual Property Rights
IPRs essential or potentially essential to the present document may have been declared to ETSI. The information
pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found
in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in
respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web
server (http://webapp.etsi.org/IPR/home.asp).
Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee
can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web
server) which are, or may be, or may become, essential to the present document.
Foreword
This ETSI Guide (EG) has been produced by ETSI Technical Committee Human Factors (HF).
The present document is primarily for the HF community.
The present document updates and replaces EG 202 534 [i.11]. New guidelines have been added and some guidelines
from EG 202 534 [i.11] were not considered relevant for the present document (e.g. because of being out of date with
technology developments). In addition, whereas EG 202 534 [i.11] focused on person-to-person (two-way)
communication services, the present document extends to real-time person-to-machine (one-way) communication
services.
Introduction
The real-time communication services that are currently available and in development place different demands on the
communication channel and terminal equipment. Also they continue to evolve for both mobile and fixed usage. The
services offer great potential but also complex choices regarding the most appropriate technologies and media that are
suitable for different communication situations. Taking video communication as an example, there are advances at the
two extremes of usage: for video calls with mobile telephones and conference-room 'Telepresence' systems and between
these extremes video communication is increasingly available through personal computer-based 'Web-conferencing'
services.
TR 102 353 [i.64] identified the need to develop guidelines on real-time communication services. The main aim is to
provide guidelines for network operators, equipment manufacturers and service providers that address the user
experience of different communication services in terms of either:
• the configuration and quality of service (QoS) of a particular communication service;
• the best choice between different communication services, where user experience data exists to enable
recommendations on the basis of of set-up time, reliability, type of user tasks and user situations.
The general requirements for the guidelines contained in the present document are / main inclusion criteria for the
guidelines are:
• Empirical basis - the guidelines are based on user test results where data has been collected on user
performance and/or user opinion. The guidelines are therefore justified on the basis of either a laboratory
experiment, field study, survey or an expert panel of people with empirical knowledge gained through
experience with users or user tests.
• Known QoS - the test systems were of specified QoS parameter values, thereby enabling a statement of
expected user experience for a given service with particular QoS.
• Technology independent - because the user experience results were related to test systems with specified QoS
rather than a particular technology.
ETSI

---------------------- Page: 5 ----------------------
6 ETSI EG 202 670 V1.1.2 (2010-03)
• Vendor independent - because the user experience results were related to test systems with specified QoS
rather than a particular vendor.
• User communication phase - the guidelines concern user experience once communication is established,
therefore the guidelines do not address interface design and call control;
• Stakeholder interest - priority has been given to developing guidelines in areas where stakeholder interest has
been shown or is anticipated. Therefore Guidelines are provided for topics that have been identified as
important for intended guideline users and for which user-based data existed or could be collected. For this
reason, the development of guidelines from published literature is not exhaustive.
The guidelines are grouped into topics that include different user tasks (e.g. decision making, negotiation, persuasion),
technical parameters (e.g. delay, packet loss, frame-rate) and special user groups (e.g. people with speech impairments,
deaf and hearing impaired people, people with cognitive impairments). The user test results are derived primarily from
industry and European Commission Framework Programme research reports, journal articles, conference proceedings
and standards documents.
ETSI

---------------------- Page: 6 ----------------------
7 ETSI EG 202 670 V1.1.2 (2010-03)
1 Scope
The present document provides guidelines for the user experience of real-time communication services. The services
include person-to-person (two-way) communication and person-to-machine (one-way) communication.
The present document is revised from EG 202 534 [i.11] that was restricted to person-to-person communication
services. The revision adds new guidelines that have been requested by stakeholders and omits some of the previous
guidelines that are no longer considered relevant.
The guidelines are based on empirical data about user experience. Most of the data is obtained from scientific papers. A
minority of guidelines are based on existing standardisation documents to ensure that relevant normative and
informative standards material is clear amongst results available from journal articles, conference papers, research
reports, etc.
The present document does not replace any existing standards.
Most of the guidelines derived from scientific papers are specific to a particular context, in that the original user tests
were for specific tasks, users and technical parameters and therefore the results may not be generalisable. Although the
guidelines provide information about the main user experience measure(s) and technical parameter(s) of a particular test
result, it is beyond the scope of the present document to provide all of the variables concerned with each user test.
However, the origin of each empirical source of a guideline is shown and listed in the References.
Those readers interested in the details of a particular guideline empirical source are also directed to a web-based system
(http://portal.etsi.org/stfs/STF_HomePages/STF354/) that provides this information.
2 References
References are either specific (identified by date of publication and/or edition number or version number) or
non-specific.
• For a specific reference, subsequent revisions do not apply.
• Non-specific reference may be made only to a complete document or a part thereof and only in the following
cases:
- if it is accepted that it will be possible to use all future changes of the referenced document for the
purposes of the referring document;
- for informative references.
Referenced documents which are not found to be publicly available in the expected location might be found at
http://docbox.etsi.org/Reference.
NOTE: While any hyperlinks included in this clause were valid at the time of publication ETSI cannot guarantee
their long term validity.
2.1 Normative references
The following referenced documents are indispensable for the application of the present document. For dated
references, only the edition cited applies. For non-specific references, the latest edition of the referenced document
(including any amendments) applies.
Not applicable.
ETSI

---------------------- Page: 7 ----------------------
8 ETSI EG 202 670 V1.1.2 (2010-03)
2.2 Informative references
The following referenced documents are not essential to the use of the present document but they assist the user with
regard to a particular subject area. For non-specific references, the latest version of the referenced document (including
any amendments) applies.
[i.1] Anderson, A., Smallwood, S., MacDonald, R., Mullin, J., Fleming, A. & O'Malley, C. (2000):
"Video data and video links in mediated communication: What do users value? International
Journal of Human-Computer Studies", 52(1), 165-187.
[i.2] ANSI T1.522-2000: "Quality of Service for Business Multimedia Conferencing".
[i.3] Bekkering, E. and Shim, J. (2006). "i2i Trust in Videoconferencing", July 2006/Vol. 49, No. 7
Communications of the ACM, pp. 103-107.
[i.4] Blythe, M.; Overbeeke, K., Monk, A., Wright, P. (Eds) (2004). "Funology: From Usability to
Enjoyment" Kluwer Academic.
[i.5] Brennan, D., Georgeadis, A., Baron, C., Barker, L. (2004): "The Effect of Videoconference-Based
Telerehabilitation on Story Retelling Performance by Brain-Injured Subjects and Its Implications
for Remote Speech-Language Therapy", Telemedicine Journal and e-Health. June 1, 2004,
10(2): 147-154.
[i.6] Brooks, P., Schliemann, T., Hestnes, B., Frowein, H., Aaby, C., O'Malley, C. (2003): ""Final
Report Project IST-1999-11577 Eye-2-Eye: Fitness-for-Purpose of Person-Person Communication
Technologies". EC Deliverable IST11577/SEF/DIS/DS/Pub/008/b1, June 2003.
[i.7] Brooks, P. & Hestnes, B. (2003): "User-centred technical guidelines for real-time human
communication services: Requirements and derivation". Proceedings of the 19th International
Symposium on Human Factors in Telecommunication, Berlin, Germany, December 1-4 2003,
pp. 11-18.
[i.8] Cullum, C., Weiner, M. Gehrmann, H. Hynan, L. (2006): "Feasibility of Telecognitive Assessment
in Dementia. Assessment", Volume 13, No. 4, December 2006 385-390.
[i.9] Cuny, R., Li, M., Kristensson, M. (2006): "Mobile service applications and performance in
UMTS", In Soldani, D., Li, M., Cuny, R. (Eds) "QoS and QoE Management in UMTS Cellular
Systems", John Wiley & Sons, Ltd.
[i.10] Doherty-Sneddon, G., Anderson, A., O' Malley, C., Langton, S., Garrod, S. & Bruce, V. (1997):
"Face-to-face and video mediated communication: A comparison of dialogue structure and task
performance", Journal of Experimental Psychology: Applied , 3(2), 105-125.
[i.11] ETSI EG 202 534: "Human Factors (HF); Guidelines for real-time person-to-person
communication services".
[i.12] ETSI ES 202 667 : "Speech and multimedia Transmission Quality (STQ); Audiovisual QoS for
communication over IP networks".
[i.13] ETSI ES 202 737: "Speech and multimedia Transmission Quality (STQ); Transmission
requirements for narrowband VoIP terminals (handset and headset) from a QoS perspective as
perceived by the user".
[i.14] ETSI ES 202 738: "Speech and multimedia Transmission Quality (STQ); Transmission
requirements for narrowband VoIP loudspeaking and handsfree terminals from a QoS perspective
as perceived by the user".
[i.15] ETSI ES 202 739: "Speech and multimediaTransmission Quality (STQ); Transmission
requirements for wideband VoIP terminals (handset and headset) from a QoS perspective as
perceived by the user".
[i.16] ETSI ES 202 740: "Speech and multimedia Transmission Quality (STQ); Transmission
requirements for wideband VoIP loudspeaking and handsfree terminals from a QoS perspective as
perceived by the user".
ETSI

---------------------- Page: 8 ----------------------
9 ETSI EG 202 670 V1.1.2 (2010-03)
[i.17] ETSI ETR 292: "Terrestrial Trunked Radio (TETRA); Voice plus Data (V+D); Technical
requirements specification; Network management".
[i.18] ETSI TR 102 535: "Human Factors (HF); Guidelines for real-time person-to-person
communication services; Future requirements".
[i.19] ETSI ETR 334: "Human Factors (HF); The implications of human ageing for the design of
telephone terminals".
[i.20] ETSI ETR 297: "Human Factors (HF); Human Factors in Videotelephony".
[i.21] ETSI TS 122 105: "Universal Mobile Telecommunications System (UMTS); Services and service
capabilities (3GPP TS 22.105)".
[i.22] ETSI TS 123 107: "Digital cellular telecommunications system (Phase 2+); Universal Mobile
Telecommunications System (UMTS); LTE; Quality of Service (QoS) concept and architecture
(3GPP TS 23.107 Release 8)".
[i.23] Følstad, A., Brooks, P., Heim, J., Schliemann, T., Wiig., S., Hestnes, B., Heiestad, S., Ulseth, T.,
Frowein, H., Aaby, C., O'Malley, C., Brundell, P., Lonsdale, P. (2002): "Results of Field
Experiments of Communication Media. IST Project 1999-11577. Eye-2-Eye: Fitness-for-purpose
of Person-Person Communication Technologies", CEC Deliverable
IST11577/SEF/DIS/DS/Pub/004/b1, October 2002.
[i.24] France, E. F., Anderson, A. H. & Gardner, M. (2001): "The impact of status and audio
conferencing technology on business meetings", International Journal of Human Computer
Studies, 54: 857-876.
[i.25] Frowein, H., Donker, W., Devoldere, P., Schliemann, T., Brooks, P., Følstad, A., Hestnes, B. et al.
(2003): "Specification of Cost-Benefit Analysis Tool. Project IST-1999-11577 Eye-2-Eye:
Fitness-for-Purpose of Person-Person Communication Technologies", EC Deliverable
IST11577/IvD/RAD/DS/FP5/052/b1, May 2003.
[i.26] Ghinea, G., Thomas, J. (2005). "Quality of Perception: User Quality of Service in Multimedia
Presentations", IEEE Transactions on Multimedia, Vol. 7, No. 4, August 2005, pp. 786-789.
[i.27] Grayson, D. & Coventry, L. (1998) "The Effects of Visual Proxemic Information in Video
Mediated Communication", SIGCHI Bulletin Vol.30 No.3, July 1998.
[i.28] Hellström, G., Aström, L. (2008): "The value of total conversation call with emergency services,
Proceedings of International Symposuim on Human Factors in Telecommunications", Kuala
Lumpa, Malaysia, 17-20 March 2008.
[i.29] Hestnes, B., Brooks, P., Heiestad, S. (2004): "Mobile Eye-phone - a study of relevance,
effectiveness and user-perceived suitability", Fornebu, Telenor R&D Report (Scientific Report
R2/2004).
[i.30] Hestnes, B., Brooks, P., Heiestad, S., Ulseth, T., Aaby., C. (2003): "Quality of Experience in
real-time person-person communication services - User based QoS expressed in technical network
QoS terms", proceedings of the 19th International Symposium on Human Factors in
Telecommunication, Berlin, Germany, December 1-4 2003, pp. 3-10.
[i.31] ISO 9241 (all parts): "Ergonomic requirements for office work with visual display terminals
(VDTs)".
[i.32] ITU-R Recommendation BT.1359-1 (2003): "Method for the Subjective Assessment of
Intermediate Quality Level of Coding Systems".
[i.33] ITU-T Series H: "Audiovisual and multimedia systems, Supplement 1 (05/99) (1999) Application
profile - Sign language and lip-reading real-time conversation using low bit-rate video
communication".
[i.34] ITU-T Recommendation E.800 (2008): "Definitions of terms related to quality of service".
[i.35] ITU-T Recommendation F.700 (2000): Framework Recommendation for multimedia services.
ETSI

---------------------- Page: 9 ----------------------
10 ETSI EG 202 670 V1.1.2 (2010-03)
[i.36] ITU-T Recommendation G.108 (1999): "Application of the E-model: A planning guide".
[i.37] ITU-T Recommendation G.114 (2003): "One-way transmission time", International.
[i.38] ITU-T Recommendation G.1010 (2001): "End-user multimedia QoS categories".
[i.39] ITU-T Recommendation G.1080: "Quality of experience requirements for IPTV services".
[i.40] ITU-T Recommendation P.10/G.100: "Amendment 2: New definitions for inclusion in
Recommendation ITU-T P.10/G.100", International Telecommunication Union, Geneva,
Switzerland, 2008.
[i.41] ITU-T Recommendation Y.1541 (2006): "Network performance objectives for IP-based services",
International Telecommunication Union, Geneva, Switzerland.
[i.42] Jumisko-Pyykkö, S., Vinod Kumar, M. V., Liinasuo, M., & Hannuksela, M. (2006): Acceptance of
audiovisual quality in erroneous television sequences over a DVB-H channel. In Proceedings of
the Second International Workshop in Video Processing and Quality Metrics for Consumer
Electronics".
[i.43] Kilgore, R.M., and Chignell, M., (2006): "Listening to unfamiliar voices in spatial audio: Does
visualization of spatial position enhance voice identification? Proceedings of the 20th International
Symposium on Human Factors in Telecommunication", Sophia Antipolis, France, March 21-23
2006.
NOTE: Available from http://www.hft.org/HFT06/HFT_06_programme.htm).
[i.44] Knoche, H., McCarthy, J. D. and Sasse, M.A. (2005): "Can small be beautiful?: assessing image
resolution requirements for mobile TV", Proc. ACM Multimedia 2005, ACM Press, 829-838.
[i.45] Knoche, H., McCarthy J., Sasse, M. A. (2006): "Reading the Fine Print: The Effect of Text
Legibility on Perceived Video Quality in Mobile TV", ACM Multimedia, ACM, 727-730.
[i.46] Knoche, H.O., Sasse, M.A. (2008): "The sweet spot: how people trade off size and definition on
mobile devices. In Proceeding of the 16th ACM international conference on Multimedia",
Vancouver, British Columbia, Canada: ACM, pp. 21-30. .
[i.47] Mashima, P., Birkmire-Peters, D., Syms, M., Holtel, M., Burgess, L., Peters, L. (2003):
"Telehealth: Voice Therapy Using Telecommunications Technology", American Journal of
Speech-Language Pathology Vol.12 432-439.
[i.48] Mason, S. (2006): "Mobile TV - results from the DVB-H trial in Oxford", EBU TECHNICAL
REVIEW - April 2006.
NOTE: Available at http://tech.ebu.ch/docs/techreview/trev_306-mason.pdf.
[i.49] McCarthy, J., Sasse, M. A. & Miras, D. (2004): "Sharp or Smooth? Comparing the effects of
quantization vs. frame rate for streamed video. Proceedings of CHI 2004", Vienna, Austria,
April 20-24, pp. 535-542.
[i.50] Mulbach, L., Bocker, M. & Prussog, A. (1995): "Telepresence in Videocommunications: A Study
on Stereoscopy and Individual Eye Contact", Human Factors, 37(2), 290-305.
[i.51] Nokia (2004): "Quality of Experience (QoE) of mobile services: Can it be measured and
improved?", White Paper, Nokia Corporation, Finland, 2004.
[i.52] O' Malley, C., Langton, S., Anderson, A., Doherty-Sneddon, G & Bruce, V. (1996): "Comparison
of Face-to-Face and Video-Mediated Interaction. Interacting with Computers", 8(2), 177-192.
[i.53] O'Malley, C., Brundell, P., McFadzean, J., Lonsdale, P., Schliemann, T., Brooks, P. et al. (2002):
"Results of Laboratory Experiments of Communication Media. IST Project 1999-11577.
Eye-2-Eye: Fitness-for-purpose of Person-Person Communication Technologies", CEC
Deliverable IST11577/UON/SOP/DS/Pub/003/b1, December 2002.
ETSI

---------------------- Page: 10 ----------------------
11 ETSI EG 202 670 V1.1.2 (2010-03)
[i.54] Post & Telestyrelsen (2005): "Mobile video communications for people who are deaf: Report on
trial operations with broadband for people with disability", Swedish National Post and Telecom
Agency Report No. PTS-ER-2005:14.
[i.55] Raake, A. (2006). Short- and Long-Term Packet Loss Behavior: Towa
...