ISO/IEC 25023:2016

INTERNATIONAL ISO/IEC
STANDARD 25023
First edition
2016-06-15
Systems and software engineering —
Systems and software Quality
Requirements and Evaluation
(SQuaRE) — Measurement of system
and software product quality
Ingénierie des systèmes et du logiciel — Exigences de qualité et
évaluation des systèmes et du logiciel (SQuaRE) — Mesurage de la
qualité du produit logiciel et du système
Reference number
©
ISO/IEC 2016
© ISO/IEC 2016, Published in Switzerland
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized otherwise in any form
or by any means, electronic or mechanical, including photocopying, or posting on the internet or an intranet, without prior
written permission. Permission can be requested from either ISO at the address below or ISO’s member body in the country of
the requester.
ISO copyright office
Ch. de Blandonnet 8 • CP 401
CH-1214 Vernier, Geneva, Switzerland
Tel. +41 22 749 01 11
Fax +41 22 749 09 47
copyright@iso.org
www.iso.org
ii © ISO/IEC 2016 – All rights reserved

Contents Page
Foreword .v
Introduction .vi
1 Scope . 1
2 Conformance . 2
3 Normative references . 2
4 Terms and definitions . 2
5 Abbreviated terms . 4
6 Use of system and software product quality measures . 4
6.1 System/software product quality measurement concepts . 4
6.2 Approach to quality measurement . 5
7 Format used for documenting the quality measures . 7
8 System and software product quality measures . 7
8.1 General . 7
8.2 Functional suitability measures . 8
8.2.1 Functional completeness measures . 8
8.2.2 Functional correctness measures. 8
8.2.3 Functional appropriateness measures . 9
8.3 Performance efficiency measures. 9
8.3.1 Time behaviour measures .10
8.3.2 Resource utilization measures .11
8.3.3 Capacity measures .12
8.4 Compatibility measures .13
8.4.1 Co-existence measures .13
8.4.2 Interoperability measures .13
8.5 Usability measures .14
8.5.1 Appropriateness recognizability measures .14
8.5.2 Learnability measures .15
8.5.3 Operability measures .16
8.5.4 User error protection measures .18
8.5.5 User interface aesthetics measures .18
8.5.6 Accessibility measures . .19
8.6 Reliability measures .19
8.6.1 Maturity measures .20
8.6.2 Availability measures .20
8.6.3 Fault tolerance measures .21
8.6.4 Recoverability measures .22
8.7 Security measures .22
8.7.1 Confidentiality measures .22
8.7.2 Integrity measures .23
8.7.3 Non-repudiation measures .24
8.7.4 Accountability measures . .24
8.7.5 Authenticity measures .24
8.8 Maintainability measures .25
8.8.1 Modularity measures .25
8.8.2 Reusability measures .25
8.8.3 Analysability measures .26
8.8.4 Modifiability measures .27
8.8.5 Testability measures .27
8.9 Portability measures .28
8.9.1 Adaptability measures .28
8.9.2 Installability measures .29
© ISO/IEC 2016 – All rights reserved iii

8.9.3 Replaceability measures .30
Annex A (informative) Considerations for the use of quality measures .31
Annex B (informative) QMEs used to define product or system quality measures .36
Annex C (informative) Detailed explanation of measurement types .39
Bibliography .45
iv © ISO/IEC 2016 – All rights reserved

Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are
members of ISO or IEC participate in the development of International Standards through technical
committees established by the respective organization to deal with particular fields of technical
activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international
organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the
work. In the field of information technology, ISO and IEC have established a joint technical committee,
ISO/IEC JTC 1.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular the different approval criteria needed for
the different types of document should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject
of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent
rights. Details of any patent rights identified during the development of the document will be in the
Introduction and/or on the ISO list of patent declarations received (see www.iso.org/patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation on the meaning of ISO specific terms and expressions related to conformity
assessment, as well as information about ISO’s adherence to the WTO principles in the Technical
Barriers to Trade (TBT) see the following URL: Foreword - Supplementary information
The committee responsible for this document is ISO/IEC JTC 1, Information technology, SC 7, Software
and systems engineering.
This first edition of ISO/IEC 25023, which is a part of the SQuaRE series of standards, cancels and
replaces ISO/IEC TR 9126-2:2003 and ISO/IEC TR 9126-3:2003, with the following changes:
— the quality measures contained in ISO/IEC/TR 9126-2 and ISO/IEC/TR 9126-3 are reviewed and
adopted or rejected according to the practical usefulness;
— in addition, the other quality measures are given for the revised system/software product quality
model in ISO/IEC 25010;
— the internal and external measures are aggregated and represented with a simplified format in
one table.
The SQuaRE series of International Standards consist of the following divisions, under the general title
Systems and software Quality Requirements and Evaluation (SQuaRE):
— ISO/IEC 2500n — Quality Management Division
— ISO/IEC 2501n — Quality Model Division
— ISO/IEC 2502n — Quality Measurement Division
— ISO/IEC 2503n — Quality Requirements Division
— ISO/IEC 2504n — Quality Evaluation Division
— ISO/IEC 25050 to ISO/IEC 25099 — SQuaRE Extension Division
Annexes A, B and C are for information only.
© ISO/IEC 2016 – All rights reserved v

Introduction
This International Standard is a part of the SQuaRE series of International Standards. It provides a set
of quality measures for the characteristics of system/software products that can be used for specifying
requirements, measuring and evaluating the system/software product quality, in conjunction with other
SQuaRE series of International Standards, especially ISO/IEC 25010, ISO/IEC 25030, ISO/IEC 25040 and
ISO/IEC 25041.
The set of quality measures in this International Standard were selected based on their practical value
and are categorized into two levels of reliability. They are not intended to be exhaustive and users of
this International Standard are encouraged to refine them if necessary.
Quality measurement division
This International Standard is a part of the ISO/IEC 2502n series that currently consists of the following
International Standards:
— ISO/IEC 25020 — Measurement reference model and guide: provides a reference model and
guide for measuring the quality characteristics defined in ISO/IEC 2501n quality model division.
— ISO/IEC 25021 — Quality measure elements: provides a format for specifying quality measure
elements and some examples of quality measure elements (QMEs) that can be used to construct
software quality measures.
— ISO/IEC 25022 — Measurement of quality in use: provides measures including associated
measurement functions for the quality characteristics in the quality in use model.
— ISO/IEC 25023 — Measurement of system and software product quality: provides measures
including associated measurement functions for the quality characteristics in the product
quality model.
— ISO/IEC 25024 — Measurement of data quality: provides measures including associated
measurement functions for the quality characteristics in the data quality model.
Figure 1 depicts the relationship between this International Standard and the other International
Standards in the ISO/IEC 2502n division. Developers, evaluators, quality managers, acquirers, suppliers,
maintainers and users of target system/software product can select measures from these International
Standards for the measurement of quality characteristics of interest. This could be for defining
requirements, evaluating system/software products, performing quality management activities or for
other purposes.
vi © ISO/IEC 2016 – All rights reserved

Figure 1 — Structure of the Quality Measurement Division
Outline and organization of SQuaRE series
The SQuaRE series consists of five main divisions and extension division. Outline of each divisions
within SQuaRE series are as follows.
— ISO/IEC 2500n — Quality Management Division. The standards that form this division define
all common models, terms, and definitions referred further by all other standards from SQuaRE
series. The division also provides requirements and guidance for the planning and management of
a project.
— ISO/IEC 2501n — Quality Model Division. The standards that form this division provide quality
models for system/software products, quality in use, and data. A service quality model is under
development. Practical guidance on the use of the quality model is also provided.
— ISO/IEC 2502n — Quality Measurement Division. The standards that form this division include
a system/software product quality measurement reference model, definitions of quality measures,
and practical guidance for their application. This division presents internal measures of software
quality, external measures of software quality, quality in use measures, and data quality measures.
Quality measure elements forming foundations for the quality measures are defined and presented.
— ISO/IEC 2503n — Quality Requirements Division. The standards that form this division help
specify quality requirements. These quality requirements can be used in the process of quality
© ISO/IEC 2016 – All rights reserved vii

requirements elicitation for a system/software product to be developed, designing a process for
achieving necessary quality, or as inputs for an evaluation process.
— ISO/IEC 2504n — Quality Evaluation Division. The standards that form this division provide
requirements, recommendations, and guidelines for system/software product evaluation, whether
performed by independent evaluators, acquirers, or developers. The support for documenting a
measure as an Evaluation Module is also presented.
ISO/IEC 25050 to ISO/IEC 25099 are reserved for SQuaRE extension International Standards, which
currently include ISO/IEC 25051 and ISO/IEC 25060 to ISO/IEC 25069.
viii © ISO/IEC 2016 – All rights reserved

INTERNATIONAL STANDARD ISO/IEC 25023:2016(E)
Systems and software engineering — Systems and software
Quality Requirements and Evaluation (SQuaRE) —
Measurement of system and software product quality
1 Scope
This International Standard defines quality measures for quantitatively evaluating system and software
product quality in terms of characteristics and subcharacteristics defined in ISO/IEC 25010 and is
intended to be used together with ISO/IEC 25010. It can be used in conjunction with the ISO/IEC 2503n
and the ISO/IEC 2504n standards or to more generally meet user needs with regard to software product
or system quality.
This International Standard contains the following:
— a basic set of quality measures for each characteristic and subcharacteristics;
— an explanation of how to apply software product and system quality measures.
It includes, as informative annexes, considerations for the use of quality measures (Annex A), QMEs
used to define product or system quality measures (Annex B), and detailed explanation of measurement
types (Annex C).
This International Standard does not assign ranges of values of the measures to rated levels or to grades
of compliance because these values are defined based on the nature of the system, product or a part
of the product, and depending on factors such as category of the software, integrity level, and users’
needs. Some attributes could have a desirable range of values, which does not depend on specific user
needs but depends on generic factors; for example, human cognitive factors.
The proposed quality measures are primarily intended to be used for quality assurance and
improvement of system and software products during or post the development life cycle process.
The main users of this International Standard are people carrying out quality requirement specification
and evaluation activities as part of the following:
— development: including requirements analysis, design specification, coding and testing through
acceptance during the life cycle process;
— quality management: systematic examination of the software product or computer system, for
example, when evaluating system or software product quality as part of quality assurance, quality
control and quality certification;
— supply: a contract with the acquirer for the supply of a system, software product or software service
under the terms of a contract, for example, when validating quality at qualification test;
— acquisition: including product selection and acceptance testing, when acquiring or procuring a
system, software product or software service from a supplier;
— maintenance: improvement of the software product or system based on quality measurement.
© ISO/IEC 2016 – All rights reserved 1

2 Conformance
Any quality requirement specification or quality evaluation that conforms to this International
Standard shall:
a) select the quality characteristics and/or subcharacteristics to be specified or evaluated as defined
in ISO/IEC 25010;
b) for each selected characteristic or subcharacteristic, all the Generic (G) quality measures defined
in Clause 8 should be used. If any are excluded, then provide a rationale;
c) optionally select any Specific (S) quality measures in Clause 8 that are relevant;
d) if any quality measure is modified, provide the rationale for the changes;
e) define any additional quality measures and QMEs as per ISO/IEC 25021 that are not included in this
International Standard.
3 Normative references
The following documents, in whole or in part, are normatively referenced in this document and are
indispensable for its application. For dated references, only the edition cited applies. For undated
references, the latest edition of the referenced document (including any amendments) applies.
ISO/IEC 25000, Systems and software engineering — Systems and software Quality Requirements and
Evaluation (SQuaRE) — Guide to SQuaRE
ISO/IEC 25010:2011, Systems and software engineering — Systems and software Quality Requirements
and Evaluation (SQuaRE) — System and software quality models
ISO/IEC 25021:2012, Systems and software engineering — Systems and software Quality Requirements
and Evaluation (SQuaRE) — Quality measure elements
4 Terms and definitions
For the purposes of this document, the terms and definitions given in ISO/IEC 25000 and ISO/IEC 25010
and the following apply.
NOTE The essential definitions from ISO/IEC 25000 SQuaRE series and the other ISO standards are
reproduced here.
4.1
external measure (of system or software quality)
measure of the degree to which a system or software product enables the behaviour to satisfy stated
and implied needs for the system including the software to be used under specified conditions
Note 1 to entry: Attributes of the behaviour can be verified and/or validated by executing the system or software
product during testing and operation.
EXAMPLE The failure density against test cases found during testing is an external measure of software
quality related to the reliability of the computer system. The two measures are not necessarily identical since
testing may not find all faults, and a fault may give rise to apparently different failures in different circumstances.
[SOURCE: ISO/IEC 25000:2014, 4.11, modified]
2 © ISO/IEC 2016 – All rights reserved

4.2
internal measure (of software quality)
measure of the degree to which a set of static attributes of a software product satisfy stated and implied
needs for the software product to be used under specified conditions
Note 1 to entry: Static attributes include those that relate to the software architecture, structure and its
components.
Note 2 to entry: Static attributes can be verified by review, inspection, simulation and/or automated tools.
EXAMPLE The number of lines of code, complexity measures and the number of faults found in a walk
through are all internal measures of software quality made on the product itself.
[SOURCE: ISO/IEC 25000:2014, 4.16, modified]
4.3
job
user-defined unit of work that is to be accomplished by a computer
[SOURCE: ISO/IEC/IEEE 24765:2010, 3.1542, modified]
4.4
measure
variable to which a value is assigned as the result of measurement
Note 1 to entry: The term “measures” is used to refer collectively to base measures, derived measures and
indicators.
Note 2 to entry: In this International Standard, whenever the word “measure” is used qualified by a quality
characteristic or sub-characteristic, it refers to a quality measure as defined in 4.8 below.
[SOURCE: ISO/IEC 15939:2007, 2.15, modified]
4.5
measurement
set of operations having the object of determining a value of a measure
Note 1 to entry: Measurement can include assigning a qualitative category such as the language of a source
program (ADA, C, COBOL, etc.).
[SOURCE: ISO/IEC 15939:2007, 2.17, modified]
4.6
measurement function
algorithm or calculation performed to combine two or more quality measure elements
[SOURCE: ISO/IEC 25021:2012, 4.7, modified]
4.7
property to quantify
property of a target entity that is related to a quality measure element and which can be quantified by
a measurement method
Note 1 to entry: A software artifact is an example of a target entity.
[SOURCE: ISO/IEC 25021:2012, 4.11, modified]
4.8
quality measure
derived measure that is defined as a measurement function of two or more values of quality measure
elements
[SOURCE: ISO/IEC 25021:2012, 4.13]
© ISO/IEC 2016 – All rights reserved 3

4.9
quality measure element
QME
measure defined in terms of a property and the measurement method for quantifying it, including
optionally the transformation by a mathematical function
[SOURCE: ISO/IEC 25021:2012, 4.14, modified]
4.10
quality model
defined set of characteristics, and of relationships between them, which provides a framework for
specifying quality requirements and evaluating quality
[SOURCE: ISO/IEC 25000:2014, 4.27]
4.11
quality characteristic (of software product or system)
category of quality attributes that bears on software product or system quality
[SOURCE: ISO/IEC 25000:2014, 4.34, modified]
4.12
task
set or sequence of activities required to achieve a given goal
Note 1 to entry: These activities can be physical or cognitive.
Note 2 to entry: Role and responsibilities can determine goals and tasks.
[SOURCE: ISO/IEC 9241-11:1998, 3.9, modified]
5 Abbreviated terms
The following abbreviated term is used in this International Standard.
QME    quality measure element
6 Use of system and software product quality measures
6.1 System/software product quality measurement concepts
The quality of a system/software product is the degree to which it satisfies the stated and implied needs
of its various stakeholders, and thus provides value. These stated and implied needs are represented in
the SQuaRE series of standards by quality models that categorize system/software product quality into
characteristics, which in some cases are further subdivided into subcharacteristics. The measurable
quality-related properties of a system/software product are called properties to quantify and can be
associated with quality measures. These properties are measured by applying a measurement method.
A measurement method is a logical sequence of operations used to quantify properties with respect to
a specified scale. The result of applying a measurement method is called a quality measure element.
The quality characteristics and subcharacteristics can be quantified by applying measurement
functions. A measurement function is an algorithm used to combine quality measure elements. The
result of applying a measurement function is called a quality measure. In this way, quality measures
become quantifications of the quality characteristics and subcharacteristics. More than one quality
measure can be used for the measurement of a quality characteristic or subcharacteristic (see Figure 2).
4 © ISO/IEC 2016 – All rights reserved

NOTE Target entity can be a system, a software product, data or a user (see ISO/IEC 25010:2011, Figure 5).
Figure 2 — Relationship among quality model, QM, QME, property to quantify, target entity
6.2 Approach to quality measurement
User needs for quality include requirements for system quality in use in specific contexts of use. These
identified needs can be considered when specifying external and internal measures of quality using
software product quality characteristics and subcharacteristics.
Software product quality can be evaluated by measuring internal properties (typically static measures
of intermediate products), or by measuring external properties (typically by measuring the behaviour
of the code when executed), or by measuring quality in use properties (when the product is in real or
simulated use). Appropriate internal properties of the software are a prerequisite for achieving the
required external behaviour and appropriate external behaviour is a prerequisite for achieving quality
in use (see Figure 3).
© ISO/IEC 2016 – All rights reserved 5

Figure 3 — Relationship between types of quality measures
The internal measures can be applied to a non-executable system/software product during its
development stages (such as request for proposal, requirements definition, design specification or
source code) which can be verified by review, inspection, simulation and/or automated tools. Internal
measures provide the users with the ability to measure the quality of the intermediate deliverables
and thereby predict the quality of the final product. This allows the user to identify quality issues and
initiate corrective action as early as possible in the development life cycle. For example, complexity
measures and the number, severity and failure frequency of faults found in a walk through are internal
measures of software quality made on the product itself.
The external measures can be used to measure the quality of the system/software product by
measuring the behaviour of the system of which it is a part. The external measures can only be used
during the testing stages of the life cycle process and during any operational stages. The measurement
is performed when executing the system/software product in the system environment in which it
is tested and/or intended to operate. For example, the number of failures found during testing is an
external measure of software quality related to the number of faults present in the computer system.
It is recommended, where possible, to use internal measures that have a strong relationship with the
target external measures so that they can be used to predict the values of external measures.
This International Standard provides a suggested set of system and software quality measures
(external and internal measures) to be used with the ISO/IEC 25010 quality model. The user of this
International Standard can modify the quality measures defined and can also define and use quality
measures not identified or defined in this International Standard.
NOTE 1 For example, the specific measurement of quality characteristics, such as safety or security, can be
found in the International Standards provided by IEC 65 and ISO/IEC JTC 1/SC 27.
When using a modified or a new quality measure not identified in this International Standard, the user
should specify how the measure relates to the ISO/IEC 25010 quality model or any other substitute
quality model that is being used.
Most quality measures use a measurement function, which normalizes the result value within a range
of 0,0 to 1,0. Closer to 1,0 is better. When this is not true, the interpretation is described in a NOTE.
6 © ISO/IEC 2016 – All rights reserved

Some quality measures produce a result that is relative to a target value that needs to be established as
part of requirements.
NOTE 2 Some measurements are normalized against the target value specified in a requirement specification,
a design specification, or a user documentation. Such target value is able to be determined and required as the
threshold by developers or maintainers to improve architecture, design, implementation, assembles, operational
procedures, user interface or performance of the software product or system. The target value is also able to be
specified as one of agreed requirements by acquirers and suppliers to specify quality requirements or to examine
conformance for acquisition. A requirements specification is usually changed and revised during development and
affects the quality measures based on it. Some of requirements to be specified might be missing or inconsistent,
or some of the target values might be insufficient and need to be changed because it is very difficult to specify
completely both of stated and implied needs derived from stakeholder or system requirements at the beginning
of development. Accordingly, users of quality measures are expected to take account of evolving and revising a
requirements specification and to apply quality measures not at once but iteratively during development and/or
evaluation.
NOTE 3 Some quality measures (such as mean response time) can be difficult to interpret in isolation. The
following are ways that quality measures can be applied so that they are easier to understand and interpret:
a) conformance: comparing measures with a specific business or usage requirements (e.g. the
maximum acceptable response time is 0,5 seconds);
b) benchmarks: comparing measures with a benchmark for the same or a similar product or
system used for the same purpose (e.g. the mean response time of the new system in no more than
the mean response time of the old system);
c) time series: comparing trends over time (e.g. how does the mean response time change during
the day).
7 Format used for documenting the quality measures
The following information is given for each quality measure in the tables in Clause 8:
a) ID: identification code of quality measure; each ID consists of the following three parts:
— abbreviated alphabetic code representing the quality characteristics as capital X and
subcharacteristics as one capital X followed by lowercase x (for example, “PTb” denotes “Time
behaviour” measures for “Performance efficiency”);
— serial number of sequential order within quality subcharacteristic;
— G (Generic) or S (Specific) expressing potential categories of quality measure; where, Generic
measures can be used whenever appropriate and Specific measures could be used when relevant in
a particular situation;
b) Name: quality measure name;
c) Description: the information provided by the quality measure;
d) Measurement function: mathematical formula showing how the quality measure elements are
combined to produce the quality measure.
NOTE Useful QMEs which can be used frequently to construct quality measures are specified briefly in
Annex B to help comprehend and apply measurement function for the quality measures.
8 System and software product quality measures
8.1 General
The quality measures in Clause 8 are listed by quality characteristics and subcharacteristics in the
order used in ISO/IEC 25010.
© ISO/IEC 2016 – All rights reserved 7

Quality measures can be used with different evaluation techniques that could be chosen according
to quality characteristics and evaluation rating levels depending on whether it is used as internal or
external measures. Accordingly, some quality measures listed in Clause 8 can be used at different stages
of evaluation such as static review of design specification or dynamic analysis of executable products.
Quality measures, which may be applicable, are not limited to these listed here. It is recommended
to refer a specific measure or measurement from specific International Standards or guidelines. For
example, functional size measurement is defined in ISO/IEC 14143 and an example of precise time
efficiency measurement can be referred from ISO/IEC 14756.
NOTE 1 This list of quality measures is not finalized and might be revised in future versions of this
International Standard. Readers of this International Standard are invited to provide feedback.
NOTE 2 In this clause, the word measure means quality measure unless otherwise mentioned. For example,
“Functional suitability measures” means “Functional suitability quality measures”.
8.2 Functional suitability measures
Functional suitability measures are used to assess the degree to which a product or system provided
functions that meet stated and implied needs when used under specified conditions.
NOTE 1 Functional suitability is concerned with whether the functions meet stated and implied needs.
NOTE 2 A function referred to here could be an elementary process as defined in functional user requirements
in ISO/IEC 14143.
NOTE 3 Similar measures with other QMEs like functional size can be defined as a way to weight the result
with better accuracy, as unit ratios do not indicate the quantum of functionality that is missing.
8.2.1 Functional completeness measures
Functional completeness measures are used to assess the degree to which the set of functions covers all
the specified tasks and user objectives.
Table 1 — Functional completeness measures
ID Name Description Measurement function
FCp-1-G Functional What proportion of the specified X = 1 – A/B
coverage functions has been implemented?
A = Number of functions missing
B = Number of functions specified
NOTE 1  Functions can be specified in a requirement specification, a design specification, a user manual or all
of these.
NOTE 2  A missing function is detected when the system or software product does not have the ability to
perform a function that is specified.
8.2.2 Functional correctness measures
Functional correctness measures are used to assess the degree to which a product or system provides
the correct results with the needed degree of precision.
8 © ISO/IEC 2016 – All rights reserved

Table 2 — Functional correctness measures
ID Name Description Measurement function
FCr-1-G Functional What proportion of functions X = 1 – A/B
correctness provides the correct results?
A = Number of functions that are incorrect
B = Number of functions considered
NOTE 1  An incorrect function is one that does not provide a reasonable and acceptable outcome to achieve
the specific intended objective.
NOTE 2  The functions considered for evaluation may be all the functions of a product or a specific set of
functions required for a particular usage.
NOTE 3  Developer or maintainer possibly examines an individual function by reviewing or testing and
determines whether the function successfully provides suitable outcomes to specific objectives as defined in
the requirements specification or not. In such a case, the degree of correctness is determined per an individual
function.
8.2.3 Functional appropriateness measures
Functional appropriateness measures are used to assess the degree to which the functions facilitate
the accomplishment of specified tasks and objectives.
Table 3 — Functional appropriateness measures
ID Name Description Measurement function
FAp-1-G Functional What proportion of the functions X = 1 – A/B
appropriateness required by the user provides
A = Number of functions missing or
of usage appropriate outcome to achieve a
incorrect among those that are required for
objective specific usage objective?
achieving a specific usage objective
B = Number of functions required for
achieving a specific usage objective
NOTE 1  This function will typically be considered for the most important or most frequently identified usage
objectives. Thus, this quality measure is first calculated for each of the defined usage objectives that can be
pursued in the system, and then the next quality measure, i.e. FAp-2-G “Functional appropriateness of the
system”, can be calculated collectively across all usage objectives to provide a system measure.
NOTE 2  Users of this International Standard could also consider measuring the proportion of user objectives
that are achievable in order to get a better understanding of the actual impact on user’s intended usage.
FAp-2-G Functional What proportion of the functions
X= A/ n
å
i
appropriateness required by the users to achieve
i = 1 to n
of system their objectives provides
appropriate outcome? A = Appropriateness score for usage objec-
i
tive i, that is, the measured value of FAp-1-G
for i-th specific usage objective
n = Number of usage objectives
8.3 Performance efficiency measures
Performance efficiency measures are used to assess the performance relative to the amount of
resources used under stated conditions. Resources can include other software products, the software
and hardware configuration of the system, and materials (e.g. print paper, storage media).
NOTE 1 The performance efficiency measure is affected strongly and fluctuates depending on the conditions
of use, such as load of processing data, frequency of use, number of connecting sites and so on. Therefore,
performance efficiency measures might include the ratio of estimated or measured value with error fluctuation
to the designed value with allowed error fluctuation range required by specification. It is recommended to list
and to investigate the role played by factors such as “CPU” and memory used by other software, network traffic,
and scheduled background processes. Possible fluctuations and valid ranges for estimated or measured values
can be established and compared to requirement specifications.
© ISO/IEC 2016 – All rights reserved 9

NOTE 2 It is also recommended that a task be identified and defined to be suitable for performance efficiency
or capacity measures; for example, a transaction as a task for a business application, a switching or data packet
sent as a task for a communication application, an event control as a task for a control application and an output
of data produced by a user callable function as a task for a common user application.
8.3.1 Time behaviour measures
Time behaviour measures are used to assess the degree to which the response and processing times
and throughput rates of a product or system when performing its functions meet the requirements.
Table 4 — Time behaviour measures
ID Name Description Measurement function
PTb-1-G Mean response time How long is the mean time taken
X= (A )/ n
å
i
by the system to respond to a user
i = 1 to n
task or system task?
A = Time taken by the system to respond
i
to a specific user task or system task at i-th
measurement
n = Number of responses measured
PTb-2-G Response time How well does the system X = A/B
adequacy response time meet the
A = Mean response time meas
...