IEC TR 63478-1:2023

IEC TR 63478-1 ®
Edition 1.0 2023-11
TECHNICAL
REPORT
colour
inside
User’s quality of experience on multimedia conferencing services –
Part 1: General
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized in any form
or by any means, electronic or mechanical, including photocopying and microfilm, without permission in writing from
either IEC or IEC's member National Committee in the country of the requester. If you have any questions about IEC
copyright or have an enquiry about obtaining additional rights to this publication, please contact the address below or
your local IEC member National Committee for further information.

IEC Secretariat Tel.: +41 22 919 02 11
3, rue de Varembé info@iec.ch
CH-1211 Geneva 20 www.iec.ch
Switzerland
About the IEC
The International Electrotechnical Commission (IEC) is the leading global organization that prepares and publishes
International Standards for all electrical, electronic and related technologies.

About IEC publications
The technical content of IEC publications is kept under constant review by the IEC. Please make sure that you have the
latest edition, a corrigendum or an amendment might have been published.

IEC publications search - webstore.iec.ch/advsearchform IEC Products & Services Portal - products.iec.ch
The advanced search enables to find IEC publications by a Discover our powerful search engine and read freely all the
variety of criteria (reference number, text, technical publications previews. With a subscription you will always have
committee, …). It also gives information on projects, replaced access to up to date content tailored to your needs.
and withdrawn publications.
Electropedia - www.electropedia.org
IEC Just Published - webstore.iec.ch/justpublished
The world's leading online dictionary on electrotechnology,
Stay up to date on all new IEC publications. Just Published
containing more than 22 300 terminological entries in English
details all new publications released. Available online and once
and French, with equivalent terms in 19 additional languages.
a month by email.
Also known as the International Electrotechnical Vocabulary

(IEV) online.
IEC Customer Service Centre - webstore.iec.ch/csc

If you wish to give us your feedback on this publication or need
further assistance, please contact the Customer Service
Centre: sales@iec.ch.
IEC TR 63478-1 ®
Edition 1.0 2023-11
TECHNICAL
REPORT
colour
inside
User’s quality of experience on multimedia conferencing services –

Part 1: General
INTERNATIONAL
ELECTROTECHNICAL
COMMISSION
ICS 33.160.60  ISBN 978-2-8322-7892-5

– 2 – IEC TR 63478-1:2023 © IEC 2023
CONTENTS
FOREWORD . 3
INTRODUCTION . 5
1 Scope . 6
2 Normative references . 6
3 Terms and definitions . 6
3.1 Definitions. 6
3.2 Abbreviated terms . 6
4 Consideration on QoE . 6
4.1 QoE characteristics . 6
4.2 QoE requirements . 7
4.3 QoE parameters . 8
5 QoE measurement framework . 8
5.1 Functional entities . 8
5.2 QoE measurement operations between functional entities. 9
6 QoE measurement models . 10
6.1 General . 10
6.2 Common survey model. 10
6.3 Timeline rating model . 11
6.4 Honeycomb model . 11
Annex A (informative) Visualization of QoE measurement . 13
A.1 QoE measurement timeline . 13
A.2 QoE grading . 13
A.3 QoE measurement table . 14
Annex B (informative) Gap analysis with the existing relevant standards . 15
Bibliography . 20

Figure 1 – Various QoE requirements for multimedia conferencing services . 7
Figure 2 – QoE measurement operations between functional entities . 9
Figure 3 – Common survey model . 10
Figure 4 – Timeline rating model . 11
Figure 5 – Honeycomb model for UI/UX measurement . 12
Figure A.1 – QoE measurement timeline . 13
Figure A.2 – Grading of absolute category rating . 13
Figure A.3 – Grading of Honeycomb rating . 14
Figure A.4 – QoE measurement table . 14
Figure B.1 – Overview of relevant standards . 15

Table 1 – QoE parameters . 8
Table B.1 – List of standards on UX and MM . 15
Table B.2 – List of standards on NP and MM . 16
Table B.3 – List of standards on NP and SA . 18
Table B.4 – List of standards on UX and SA . 19

INTERNATIONAL ELECTROTECHNICAL COMMISSION
____________
USER’S QUALITY OF EXPERIENCE ON MULTIMEDIA
CONFERENCING SERVICES –
Part 1: General
FOREWORD
1) The International Electrotechnical Commission (IEC) is a worldwide organization for standardization comprising
all national electrotechnical committees (IEC National Committees). The object of IEC is to promote international
co-operation on all questions concerning standardization in the electrical and electronic fields. To this end and
in addition to other activities, IEC publishes International Standards, Technical Specifications, Technical Reports,
Publicly Available Specifications (PAS) and Guides (hereafter referred to as “IEC Publication(s)”). Their
preparation is entrusted to technical committees; any IEC National Committee interested in the subject dealt with
may participate in this preparatory work. International, governmental and non-governmental organizations liaising
with the IEC also participate in this preparation. IEC collaborates closely with the International Organization for
Standardization (ISO) in accordance with conditions determined by agreement between the two organizations.
2) The formal decisions or agreements of IEC on technical matters express, as nearly as possible, an international
consensus of opinion on the relevant subjects since each technical committee has representation from all
interested IEC National Committees.
3) IEC Publications have the form of recommendations for international use and are accepted by IEC National
Committees in that sense. While all reasonable efforts are made to ensure that the technical content of IEC
Publications is accurate, IEC cannot be held responsible for the way in which they are used or for any
misinterpretation by any end user.
4) In order to promote international uniformity, IEC National Committees undertake to apply IEC Publications
transparently to the maximum extent possible in their national and regional publications. Any divergence between
any IEC Publication and the corresponding national or regional publication shall be clearly indicated in the latter.
5) IEC itself does not provide any attestation of conformity. Independent certification bodies provide conformity
assessment services and, in some areas, access to IEC marks of conformity. IEC is not responsible for any
services carried out by independent certification bodies.
6) All users should ensure that they have the latest edition of this publication.
7) No liability shall attach to IEC or its directors, employees, servants or agents including individual experts and
members of its technical committees and IEC National Committees for any personal injury, property damage or
other damage of any nature whatsoever, whether direct or indirect, or for costs (including legal fees) and
expenses arising out of the publication, use of, or reliance upon, this IEC Publication or any other IEC
Publications.
8) Attention is drawn to the Normative references cited in this publication. Use of the referenced publications is
indispensable for the correct application of this publication.
9) IEC draws attention to the possibility that the implementation of this document may involve the use of (a)
patent(s). IEC takes no position concerning the evidence, validity or applicability of any claimed patent rights in
respect thereof. As of the date of publication of this document, IEC had not received notice of (a) patent(s), which
may be required to implement this document. However, implementers are cautioned that this may not represent
the latest information, which may be obtained from the patent database available at https://patents.iec.ch. IEC
shall not be held responsible for identifying any or all such patent rights.
IEC TR 63478-1 has been prepared by IEC technical committee 100: Audio, video and
multimedia systems and equipment. It is a Technical Report.
The text of this Technical Report is based on the following documents:
Draft Report on voting
100/4037/DTR 100/4071/RVDTR
Full information on the voting for its approval can be found in the report on voting indicated in
the above table.
The language used for the development of this Technical Report is English.

– 4 – IEC TR 63478-1:2023 © IEC 2023
This document was drafted in accordance with ISO/IEC Directives, Part 2, and developed in
accordance with ISO/IEC Directives, Part 1 and ISO/IEC Directives, IEC Supplement, available
at www.iec.ch/members_experts/refdocs. The main document types developed by IEC are
described in greater detail at www.iec.ch/publications.
A list of all parts in the IEC 63478 series, published under the general title User’s quality of
experience on multimedia conferencing services, can be found on the IEC website.
The committee has decided that the contents of this document will remain unchanged until the
stability date indicated on the IEC website under webstore.iec.ch in the data related to the
specific document. At this date, the document will be
• reconfirmed,
• withdrawn, or
• revised.
IMPORTANT – The "colour inside" logo on the cover page of this document indicates
that it contains colours which are considered to be useful for the correct understanding
of its contents. Users should therefore print this document using a colour printer.

INTRODUCTION
With the global COVID-19 pandemic, business and education meetings have shifted from offline
to online environments. Consequently, a range of multimedia conferencing services has been
developed, enabling users to choose and enjoy conferencing services based on personal
preferences. To ensure optimal user experience, it is essential to measure the quality of
experience (QoE) for multimedia conferencing services. However, there is currently a lack of
standardized methods for measuring a user's QoE in this context. Thus, there is a pressing
need to provide guidelines for measuring and evaluating a user's QoE for multimedia
conferencing services.
Traditionally, quality of service (QoS) has been measured for network data communication,
represented by objective index values like delay, throughput, and jitter. In contrast, QoE
represents a user's level of satisfaction with a specific service and reflects human emotional
quality. As such, QoE is subject to overall service performance from the user's perspective. In
the case of multimedia conferencing services, measuring QoE is challenging due to varying
user preferences/requirements and service/application characteristics. Therefore, a unified
framework is necessary to measure and evaluate a user's QoE for multimedia conferencing
services.
This document aims to provide guidelines for enhancing a user's QoE for multimedia
conferencing services. The series specifies general considerations and requirements to
enhance a user's QoE and measurement methods for associated QoE parameters.
The IEC 63478 series consists of the following parts:
– Part 1: General;
– Part 2: Requirements; and
– Part 3: Measurement methods.
Part 1 of IEC TR 63478-1 (Technical Report) describes general considerations to measure
user's QoE.
Part 2 of IEC 63478-2 (International Standard) describes the requirements to be considered to
measure user's QoE.
Part 3 of IEC 63478-3 (International Standard) describes the measurement methods for QoE
parameters.
– 6 – IEC TR 63478-1:2023 © IEC 2023
USER’S QUALITY OF EXPERIENCE ON MULTIMEDIA
CONFERENCING SERVICES –
Part 1: General
1 Scope
This part of IEC 63478 describes general considerations to be taken for measurement of a
user’s quality of experience (QoE) on multimedia conferencing services.
2 Normative references
There are no normative references in this document.
3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
ISO and IEC maintain terminology databases for use in standardization at the following
addresses:
• IEC Electropedia: available at https://www.electropedia.org/
• ISO Online browsing platform: available at https://www.iso.org/obp
3.1 Definitions
3.1.1
QoE
measure of delight or annoyance of a user's experiences for a multimedia service from the
viewpoint of the entire service experience
3.2 Abbreviated terms
MCC multimedia conferencing client
MCS multimedia conferencing server
QMA QoE measurement agent
QMM QoE measurement manager
QoE quality of experience
UI/UX user interface/experience
4 Consideration on QoE
4.1 QoE characteristics
QoE refers to the quality that a user has experienced and perceived during a multimedia service.
It can be represented by the subjective indicators of service users.
In general, QoE involves the subjective experience of users, and it might be possible to judge
the QoE for a small number of people. However, it might be more accurate if the relevant data
are collected from many people, if possible. A set of QoE parameters can be considered to
measure a user’s QoE. Such QoE parameters are associated with the functions that are

provided by the multimedia conferencing service and also the subjective quality experienced by
the user. In addition, the emotions felt by each user can be considered to measure the user’s
QoE in a variety of ways. For example, QoE can be expressed as a simple expression of user
satisfaction, such as a five-point satisfaction survey. Annex B provides various standards for
measuring a user's QoE.
The QoE measurement is purposed to evaluate the user's satisfaction for a service and to help
the user in choosing a specific service, as per their preference. This QoE measurement can
also be used by service/application developers to provide much more enhanced services to
users.
4.2 QoE requirements
The multimedia conferencing service refers to a service that allows users to participate in an
online conference by using video and audio. It is noted that there can be many different
requirements or preferences for a user’s QoE on multimedia conferencing service, according to
the age of user, the purpose of the service, the device used for the service, and the
communication environment, as shown in Figure 1.

Figure 1 – Various QoE requirements for multimedia conferencing services
• Age
The QoE requirements may depend on the age of the user. If the user is old age, a user
might want large text rather than small text on the UI screen, and the user might focus on
the volume and clarity of the audio, rather than the quality of the video. On the other hand,
if the user is young, the user might focus on a high-quality image or video, rather than a
high-quality voice.
• Purpose
The QoE requirements might also vary according to the purpose of the multimedia
conferencing service. If it is used for group calls, the users want to focus on audio. On the
other hand, the service for purpose of presentation will require an advanced functionality or
UI to share documents, such as screen sharing. In addition, latency is important for a
discussion, less so when giving a presentation.

– 8 – IEC TR 63478-1:2023 © IEC 2023
• Device
The QoE requirements of users might also be different, depending on the type of device
used in the multimedia conferencing service, such as a smartphone, a tablet, a laptop
computer, and a smart TV.
• Communication
The QoE requirements might also be affected by the communication network environment
in which the user is located, such as Wi-Fi, Ethernet, and cellular networks.
4.3 QoE parameters
The QoE parameters are used to evaluate the satisfaction level experienced by a user in a
multimedia conferencing service. These QoE parameters include the specific functions of
multimedia conferencing service and the user’s opinion to represent its satisfaction level.
Table 1 shows some examples of QoE parameters that can be used for QoE measurement.
Table 1 – QoE parameters
QoE parameter Description
Indicates the satisfaction with the quality of the screen that the user has experienced
Video quality
during the multimedia conferencing services.
Indicates the satisfaction with the audio quality experienced by the user during the
Audio quality
multimedia conferencing services.
Refers to the synchronization for video and audio playback and indicates the
Synchronization satisfaction with synchronization that the user has experience during the multimedia
conferencing services.
Accessibility Indicates how much the multimedia conferencing service can be easily accessed.
Indicates how much the service can be easily linked with the other multimedia services,
Inter-connectivity
such as file sharing, screen sharing and whiteboard.
Indicates how much the service supports the compatibility with other applications during
Openness
the multimedia conferencing services.
Represents how much the participant can actively participate in the meeting. For
Participation example, it represents functions such as raising hands during a meeting, chatting to
answer questions and expressing emotions.
Indicates the quality of UI and UX experienced by users. For example, in the UX-
UI/UX
Honeycomb model, the satisfaction level can be expressed by a total of seven scales.

5 QoE measurement framework
5.1 Functional entities
For QoE measurement, the following four functional entities are considered.
• Multimedia conferencing server (MCS)
An MCS is a server to provide a multimedia conferencing service. An MCS performs
multimedia communication with an MCC during a multimedia conferencing service.
• Multimedia conferencing client (MCC)
An MCC is a client for a multimedia conferencing service, and it is implemented by a client
program on the user's device. An MCC performs multimedia communications with an MCS.

• QoE measurement agent (QMA)
A QMA is used to measure the QoE parameter values by interaction with the MCS and the
MCC. For QoE measurement, the QMA must be informed on QoE parameters to be
measured from QMM before the QoE measurement operations begin. Based on this
information, a QMA requests that the MCS and the MCC measure QoE parameters. In
response to the QoE measurement request, the MCS and the MCC will send the measured
QoE parameter values to the QMA, and the QMA will report the aggregated QoE parameter
values to QMM.
• QoE measurement manager (QMM)
A QMM is used to analyse the QoE parameter values that are reported from QMA to measure
a user’s QoE for the multimedia conferencing service. For this purpose, a QMM can request
that the MCS measure the QoE parameters. The QoE measurement results can be used by
the MCS to enhance the associated service or by the user of the multimedia conferenci
...