ISO/IEC 25062:2006

INTERNATIONAL ISO/IEC
STANDARD 25062
First edition
2006-04-01
Corrected version
2006-10-01
Software engineering — Software product
Quality Requirements and Evaluation
(SQuaRE) — Common Industry Format
(CIF) for usability test reports
Ingénierie du logiciel — Exigences de qualité du produit logiciel et
évaluation (SQuaRE) — Format commun de l'industrie (CIF) pour les
rapports d'essai de rentabilité

Reference number
©
ISO/IEC 2006
PDF disclaimer
This PDF file may contain embedded typefaces. In accordance with Adobe's licensing policy, this file may be printed or viewed but
shall not be edited unless the typefaces which are embedded are licensed to and installed on the computer performing the editing. In
downloading this file, parties accept therein the responsibility of not infringing Adobe's licensing policy. The ISO Central Secretariat
accepts no liability in this area.
Adobe is a trademark of Adobe Systems Incorporated.
Details of the software products used to create this PDF file can be found in the General Info relative to the file; the PDF-creation
parameters were optimized for printing. Every care has been taken to ensure that the file is suitable for use by ISO member bodies. In
the unlikely event that a problem relating to it is found, please inform the Central Secretariat at the address given below.

©  ISO/IEC 2006
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized in any form or by any means,
electronic or mechanical, including photocopying and microfilm, without permission in writing from either ISO at the address below or
ISO's member body in the country of the requester.
ISO copyright office
Case postale 56 • CH-1211 Geneva 20
Tel. + 41 22 749 01 11
Fax + 41 22 749 09 47
E-mail copyright@iso.org
Web www.iso.org
Published in Switzerland
ii © ISO/IEC 2006 – All rights reserved

Contents Page
Foreword. iv
Introduction . v
1 Scope . 1
2 Conformance. 1
3 Normative references . 2
4 Terms and definitions. 2
5 Report Format . 4
5.1 Title page . 4
5.2 Executive summary . 4
5.3 Introduction . 5
5.4 Method . 5
5.5 Results . 11
5.6 Appendices. 14
Annex A (informative) Checklist . 15
Annex B (informative) Glossary. 19
Annex C (informative) Report Template. 23
Annex D (informative) Example . 29
Bibliography . 46

© ISO/IEC 2006 – All rights reserved iii

Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are members of
ISO or IEC participate in the development of International Standards through technical committees
established by the respective organization to deal with particular fields of technical activity. ISO and IEC
technical committees collaborate in fields of mutual interest. Other international organizations, governmental
and non-governmental, in liaison with ISO and IEC, also take part in the work. In the field of information
technology, ISO and IEC have established a joint technical committee, ISO/IEC JTC 1.
International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2.
The main task of the joint technical committee is to prepare International Standards. Draft International
Standards adopted by the joint technical committee are circulated to national bodies for voting. Publication as
an International Standard requires approval by at least 75 % of the national bodies casting a vote.
Attention is drawn to the possibility that some of the elements of this document may be the subject of patent
rights. ISO and IEC shall not be held responsible for identifying any or all such patent rights.
ISO/IEC 25062 was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 7, Software and system engineering.
This corrected version of ISO/IEC 25062 incorporates the following correction:
Figure 1, Appendix C, has been included on page 43.
iv © ISO/IEC 2006 – All rights reserved

Introduction
Usability of software is a key factor in predicting successful deployment of that software. Software
manufacturers subject software to usability testing at various stages in a product’s development; some
companies that purchase software also test products for usability before making purchasing decisions. Testing
often involves (1) subjects who are representative of the target population of users of the software, (2)
representative tasks, and (3) measures of efficiency, effectiveness and subjective satisfaction. When this type
of experimental situation exists, the testing is termed summative, i.e., the results can be expressed as
statistically meaningful measures of central tendency (e.g. mean or median) and variability (e.g. standard
deviation). The Common Industry Format (CIF) for Usability Test Reports is intended for use by usability
professionals to report the results of summative usability testing.
The CIF standardizes the types of information that are captured about testing with users. The level of detail
allows the same or another organization to replicate the test procedure. The major variables are user
demographics, task descriptions, context of the test, including the equipment used, the environment in which
the test is conducted, and the protocol by which the subjects and the test administrator(s) interact, as well as
the particular metrics chosen to code the findings of the study.
The CIF is intended to replace the proprietary formats employed by companies that perform usability testing,
both vendors and purchasers of software. Until now there has been no standard format for reporting usability
testing results. Advantages of using a standardized reporting format include (1) a reduction in training time for
usability staff since an individual only needs to learn to use one form regardless of how many companies he
works for and (2) enhanced potential for increased communication between vendors and purchasing
organizations since readers of CIF-compliant reports will share a common language and expectations.
The purpose of this International Standard is to facilitate incorporation of usability as part of the procurement
decision-making process for interactive software products so that it is easier to judge whether a product meets
usability goals. Examples of decisions include purchasing, upgrading and automating. It provides a common
format for human factors engineers and usability professionals in supplier companies to report the methods
and results of usability tests to customer organizations.
Audience
The Common Industry Format (CIF) is meant to be used by usability professionals within supplier
organizations to generate reports that can be used by customer organizations in the CIF report. The CIF is
also meant to be used by customer organizations to verify that a particular report is CIF-compliant. The
Usability Test Report itself is intended for two types of readers:
— Usability professionals in customer organizations who are evaluating both the technical merit of
usability tests and the usability of the products; and
— Other technical professionals and managers who are using the test results to make business decisions.
The CIF may also be used within a single organization if a formal report of a summative usability test needs to
be generated. In this case additional material such as a list of detailed findings may be included.
The report is in two main sections, an Executive Summary and a main body. The main body contains the
Methods and Results sections and is aimed at the first audience above. These sections (1) describe the test
methodology and results in sufficient technical detail to allow replication by another organization if the test is
repeated, and (2) support application of test data to questions about the product’s expected costs and benefits.
Understanding and interpreting these sections will require technical background in human factors or usability
engineering for optimal use. The second audience is directed to the Introduction, which provides summary
information for non-usability professionals and managers. The Introduction may also be of general interest to
other computing professionals. Decision makers without usability engineering expertise may find the
information in the main body to be useful but should rely on expert interpretation when necessary.
Organization
Clause 1 describes the scope of this specification and the conformance criteria. Clause 4 provides definitions
© ISO/IEC 2006 – All rights reserved v

of the terms used throughout the document. Clause 5 is the main description of the specification.
Additional Information
Annex A provides a checklist that can be used to ensure inclusion of required and recommended information.
A glossary is provided in Annex B to define terminology used in the report format description. A Word template
for report production can be found at: http://www.ncits.org/ref-docs/CIF/CIF_template.dot. A printed version of
the template can be found in Annex C. An example is provided in Annex D illustrating how the format is used
followed by an informative bibliography.
vi © ISO/IEC 2006 – All rights reserved

INTERNATIONAL STANDARD ISO/IEC 25062:2006(E)

Software engineering — Software product Quality
Requirements and Evaluation (SQuaRE) — Common Industry
Format (CIF) for usability test reports
1 Scope
This International Standard is intended to be used to report the measures obtained from a test of usability as
defined in ISO 9241-11: effectiveness, efficiency and satisfaction in a specified context of use.
NOTE Metrics for other more-detailed usability requirements can be found in ISO/IEC 9126 parts 2 and 3.
This International Standard is intended to be used by:
⎯ usability professionals within supplier organizations to generate reports that can be used by customer
organizations;
⎯ customer organizations to verify that a particular report conforms to this International Standard;
⎯ human factors or other usability professionals in customer organizations who are evaluating both the
technical merit of usability tests and the usability of the products; and
⎯ other technical professionals and managers in the customer organization who are using the test results to
make business decisions about product suitability and purchase.
The Executive Summary and Introduction in 5.2 and 5.3 provide summary information for nonusability
professionals and managers.
Subclauses 5.4 and 5.5 describe the test methodology and results in technical detail suitable for replication,
and also support application of test data to questions about the product’s expected costs and benefits.
Understanding and interpreting these sections will require technical background in human factors or usability
engineering for optimal use.
The report format assumes sound practice [1, 2] has been followed in the design and execution of the test.
Test procedures which produce measures that summarize usability should be used, i.e. the test is summative
in nature. Some usability evaluation methods, such as formative tests, are intended to identify problems rather
than produce measures; the format is not structured to support the results of such testing methods.
2 Conformance
A usability test report conforms to this International Standard if it complies with all the requirements in this
International Standard (stated as “shall”). The recommendations (stated as “should”) should be implemented
whenever appropriate.
This International Standard specifies the minimum information that should be provided. Additional information
may be included. For example, if an organization finds that an additional list of findings is useful, the list may
be included even though it is not specified as part of a conformant CIF report.
© ISO/IEC 2006 – All rights reserved 1

3 Normative references
The following referenced documents are indispensable for the application of this document. For dated
references, only the edition cited applies. For undated references, the latest edition of the referenced
document (including any amendments) applies.
ISO/IEC 9126-1:2001, Software engineering — Product quality — Part 1: Quality model
ISO/IEC 9126-2:2001, Software engineering — Product quality — Part 2: External metrics
ISO/IEC 9126-3:2001, Software engineering — Product quality — Part 3: Internal metrics
ISO/IEC 9126-4:2001, Software engineering — Product quality — Part 4: Quality in use metrics
ISO 9241-11:1998, Ergonomic requirements for office work with visual display terminals (VDTs) — Part 11:
Guidance on usability
ISO 13407:1999, Human-centred design processes for interactive systems — Annex C
ISO/IEC 14598-5:1998, Information technology — Software product evaluation — Part 5: Process for
evaluators
4 Terms and definitions
For the purposes of this document, the following definitions apply. The source for definitions 4.1 to 4.9 is
ISO 9241-11:1998.
NOTE Efficiency, effectiveness, and satisfaction are defined in ISO/IEC 9126 in a similar way to the definitions in
ISO/IEC 9241. However, in ISO/IEC 9126 they are defined in terms of the software product where as in ISO/IEC 9241
they are defined in terms of the user’s view. The term efficiency in this International Standard is equivalent to the term
productivity in ISO 9126, as the term efficiency has been defined with a different meaning in ISO 9126.
4.1
usability
the extent to which a product can be used by specified users to achieve specified goals with effectiveness,
efficiency and satisfaction in a specified context of use
NOTE Quality in use is defined in ISO/IEC 9126 in a similar way to the definition of usability in ISO/IEC 9241. Quality
in use may be influenced by any of the quality characteristics, and is thus broader than the definition of usability in
ISO/IEC 9126 that is in terms of understandability, learnability, operability, attractiveness and compliance.
4.2
effectiveness
the accuracy and completeness with which users achieve specified goals
4.3
efficiency
resources expended in relation to the accuracy and completeness with which users achieve goals
NOTE Efficiency in the context of usability is related to “productivity” rather than to its meaning in the context of
software efficiency.
4.4
satisfaction
freedom from discomfort, and positive attitudes towards the use of the product
2 © ISO/IEC 2006 – All rights reserved

4.5
context of use
users, tasks, equipment (hardware, software and materials), and the physical and social environments in
which a product is used
4.6
user
person who interacts with the product
4.7
user group
subset of intended users who are differentiated from other intended users by factors such as age, culture or
expertise that are likely to influence usability

4.8
goal
intended outcome of user interaction with a product
NOTE Specific goals relating to user interaction may be referred to as “task goals”
4.9
task
activities required to achieve a goal
NOTE 1 These activities can be physical or cognitive.
NOTE 2 Job responsibilities can determine goals and tasks.
4.10
accessibility
usability of a product, service, environment or facility by people with the widest range of capabilities
NOTE 1 Although “accessibility” typically addresses users who have a disability, the concept is not limited to disability
issues.
NOTE 2 The usability-oriented concept of accessibility focuses on achieving levels of effectiveness, efficiency and
satisfaction that are as high as possible taking account of the specified context of use, while paying particular attention to
the diversity of the capabilities within the user population, and thus aims to minimize the differences in usability
experienced by individuals.
4.11
assistive technologies
hardware or software that is added to or incorporated within a system that increases accessibility for an
individual
EXAMPLES Braille displays, screen readers, screen magnification software and eye tracking devices are assistive
technologies.
4.12
assist
tester intervention in the form of direct procedural help provided by the test administrator to the test participant
in order to allow the test to continue when the participant could not complete the tasks on their own
© ISO/IEC 2006 – All rights reserved 3

5 Report Format
5.1 Title page
The following information shall be provided:
− Identify report as: ISO/IEC 25062 Common Industry Format for Usability Test Reports and contact
information.
− Name the product and version that was tested.
− Who led the test.
− When the test was conducted.
− Date the report was prepared.
− Who prepared the report.
− Contact name(s) for questions and/or clarifications.
− Supplier phone number.
− Supplier e-mail address.
− Supplier mailing or postal address.
The following information should be provided:
− Customer Company Name.
− Customer Company contact person.

5.2 Executive summary
This section provides a high level overview of the test. The intent of this section is to provide information for
procurement decision-makers in customer organizations. These people might not read the technical body of
this document.
This section shall begin on a new page and end with a page break to facilitate its use as a stand-alone
summary.
A high level overview of the test shall be provided that includes:
− Name and description of the product.
− Summary of method(s) including number(s) and type(s) of participants and tasks.
− Performance and satisfaction results expressed as mean scores or other suitable measure of central
tendency.
The following information should be provided:
− Reason for and nature of the test.
− Tabular summary of performance results.
4 © ISO/IEC 2006 – All rights reserved

− If differences between values or products are claimed, the probability that the difference did not occur
by chance.
5.3 Introduction
This clause provides a description of the product and the test objectives.
5.3.1 Full product description
The following information shall be provided:
− Formal product name and release or version.
− The parts of the product that were evaluated.
− The user population for which the product is intended.
The following information should be provided:
− Assistive technologies that are supported by the product.
− Brief description of the physical and social environment(s) in which the product is intended to be
used.
− The type of user work that is supported by the product.
5.3.2 Test objectives
The following information shall be provided:
− The objectives for the test including any areas of specific interest.
NOTE Possible objectives include testing user performance of work tasks and subjective satisfaction in using the
product to assess whether a product meets specific success criteria.
− Functions and components with which the user directly and indirectly interacted.
The following information should be provided:
− Reason for focusing on a product subset, if the whole product was not tested.
5.4 Method
Sufficient information shall be provided to allow an independent tester to replicate the procedure used in
testing.
5.4.1 Participants
The following information shall be provided:
− The total number of participants tested.
NOTE In order to generate valid summative statistical analyses, it is necessary to test sufficient numbers of
subjects. Eight or more subjects/cell (segment) are recommended for this purpose [9].
− Segmentation of user groups tested, if more than one.
− Key characteristics and capabilities of user group.
© ISO/IEC 2006 – All rights reserved 5

− How participants were selected; whether they had the essential characteristics.
− Differences between the participant sample and the user population.
EXAMPLE Actual users might attend a training course whereas test subjects were untrained.
− Table of participants (row) by characteristics (columns), including demographics, professional
experience, computing experience and user needs for assistive technology.
NOTE Subjects with needs for assistive technology might be individuals with physical or cognitive disabilities.
EXAMPLE The table below is an example; the characteristics that are shown are typical but might not necessarily
cover every type of testing situation.
• Gender Age Education Occupation / Professional Computer Product
role Experience Experience Experience
• • • • • • •
P1
• • • • • • •
P2
• • • • • • •
Pn
For ‘Gender’, the table entries are male or female.
For ‘Age’, the table entries state the chronological age of the participant, or indicate membership in an age range
(e.g., 25-45) or age category (e.g., under 18, over 65) if the exact age is not known.
For ‘Education’, the table entries state the number of years of completed formal education (e.g., in the US, a
high-school graduate would have 12 years of education and a college graduate 16 years). In some instances,
level of education might be stated as ‘highest academic degree’.
For ‘Occupation/role’, the table entries describe the user’s job role when using the product. Use the Role title if
known.
For ‘Professional experience’, the table entries give the amount of time the user has been performing in the role.
For ‘Computer experience’, the table entries describe relevant background such as how much experience the
user has with the platform or operating system, and/or the product domain. This might be more extensive than
one column.
For ‘Product experience’, the table entries indicate the type and duration of any prior experience with the product
or with similar products.
The characteristics and capabilities shall be complete enough so that an essentially similar group of
participants can be recruited.
Characteristics and capabilities should be chosen to be relevant to the product’s usability; they should allow a
customer to determine how similar the participants were to the customers’ user population.
The following information should be provided:
− Description of any users with needs for assistive technology.
Participants should not be from the same organization as the testing or supplier organization.
5.4.2 Context of product use in the test
The following information shall be provided:
6 © ISO/IEC 2006 – All rights reserved

− Any important differences between the evaluated context and the expected context of use.
5.4.2.1 Tasks
The following information shall be provided:
− The tasks for testing.
− The task scenarios for testing.
− Why these tasks were selected.
EXAMPLES The most frequent tasks, the most troublesome tasks.
− The source of these tasks.
EXAMPLES Observation of customers using similar products, product marketing specifications, discussion with
users or design team.
− Any task data given to the participants.
EXAMPLE Source materials for data entry.
− Completion or performance criteria established for each task.
5.4.2.2 Test facility
The following information should be provided:
− The setting and type of space in which the evaluation was conducted.
EXAMPLES Usability lab, cubicle office, meeting room, home office, home family room, manufacturing floor.
− Any relevant features or circumstances that could affect the results.
EXAMPLES Video and audio recording equipment, one-way mirrors, or automatic data collection equipment.
5.4.2.3 Participant’s computing environment
The following information shall provide enough information to replicate and validate the test, including:
− Computer configuration, including model, OS version, required libraries or settings.
− If used, browser name and version; relevant plug-in names and versions.
5.4.2.3.1 Display devices
The following information shall be provided:
− If screen-based, screen size, resolution, and colour setting.
− If print-based, the media size and print resolution.
− If visual interface elements (such as fonts) can vary in size, specify the size(s) used in the test.
5.4.2.3.2 Audio devices
The following information should be provided:
© ISO/IEC 2006 – All rights reserved 7

− If used, the relevant settings or values for the audio bits (determining sound quality), volume, etc.
5.4.2.3.3 Input devices
The following information should be provided:
− If used, the make and model of devices used in the test.
5.4.2.4 Test administrator tools
The following information shall be provided:
− If a standard questionnaire (Section 5.4.4.3 contains a partial list) was used, describe or specify it
here.
NOTE Customized questionnaires are included in an appendix.
The following information should be provided:
− Any hardware or software used to control the test or to record data.
5.4.3 Experimental design
The following information shall be provided:
− The experimental design of the test; the plan for assigning experimental conditions to participants and
the statistical analysis associated with the plan independent variables and control variables.

− The independent variables and control variables.
− The measures for which data were recorded for each set of conditions.
5.4.3.1 Procedure
The following information shall be provided:
− Operational definitions of measures.
− Descriptions of independent variables or control variables.
− Time limits on tasks.
− Policies and procedures for interaction between tester(s) and subjects.
The following information should be provided:
− Sequence of events from greeting the participants to dismissing them.
− Details of nondisclosure agreements, form completion, warm-ups, pretask training, and debriefing.
− Verification that the participants knew and understood their rights as human subjects [1].
− Steps followed to execute the test sessions and record data.
− Number and roles of people who interacted with the participants during the test session.
− Whether other individuals were present in the test environment.
8 © ISO/IEC 2006 – All rights reserved

− Whether participants were paid or otherwise compensated.
5.4.3.2 Participant general instructions
The following information shall be provided:
− Instructions given to the participants (here or in an Appendix).
− Instructions on how participants were to interact with any other persons present, including how they
were to ask for assistance and interact with other participants, if applicable.
5.4.3.3 Participant task instructions
The following information shall be provided:
− Task instruction summary.
5.4.4 Usability metrics
As defined in 4.1, usability is measured by three types of metrics: effectiveness, efficiency, and satisfaction.
The choice of measures depends on the goals of a particular study, characteristics of the users, the specific
tasks, and context-dependent features; e.g., see Dumas and Redish [4].
The following information shall be provided:
− Metrics for effectiveness.
− Metrics for efficiency.
− Metrics for satisfaction.
Effectiveness and efficiency results shall be reported, even when they are difficult to interpret within the
specified context of use. In this case, the report shall specify why the supplier does not consider the metrics
meaningful.
EXAMPLE Suppose that the context of use for the product includes real time, open-ended interaction between close
associates. In this case, Time-On-Task may not be meaningfully interpreted as a measure of efficiency, because for
many users, time spent on this task is “time well spent”.
If it is necessary to provide participants with assists, efficiency and effectiveness metrics shall be provided for
both unassisted and assisted conditions, and the number and type of assists shall be included as part of the
test results.
5.4.4.1 Effectiveness
Effectiveness relates the goals of using the product to the accuracy and completeness with which these goals
can be achieved. Common measures of effectiveness include percent task completion, frequency of errors,
frequency of assists to the participant from the testers, and frequency of accesses to help or documentation
by the participants during the tasks. It does not take account of how the task goals were achieved, only the
extent to which they were achieved. Efficiency relates the level of effectiveness achieved to the quantity of
resources expended.
5.4.4.1.1 Completion rate
The completion rate is the percentage of participants who completely and correctly achieve each task goal. If goals can be
partially achieved (e.g., by incomplete or suboptimum results), then it might also be useful to report the average goal
achievement, scored on a scale of 0 to 100% based on specified criteria related to the value of a partial result. For
example, a spell-checking task might involve identifying and correcting 10 spelling errors and the completion rate might be
© ISO/IEC 2006 – All rights reserved 9

calculated based on the percent of errors corrected. Another method for calculating completion rate is weighting; e.g.,
spelling errors in the title page of the document are judged to be twice as important as errors in the main body of text. The
rationale for choosing a particular method of partial goal analysis should be stated, if such results are included in the
report.
The following information shall be provided:
− The percentage of participants who completely and correctly achieve each task goal.
5.4.4.1.2 Errors
Errors are instances where test participants did not complete the task successfully, or had to attempt portions of the task
more than once. Scoring of data should include classifying errors according to some taxonomy, such as in [10].
5.4.4.1.3 Assists
When participants cannot proceed on a task, the test administrator sometimes gives direct procedural help in order to
allow the test to proceed. Although ‘assist’ measures can be related to both learnability and effectiveness, assists have
been integrated with effectiveness in this specification to comply with the three elements of usability defined in
ISO 9241-11.
The following information shall be provided:
− The unassisted completion rate (i.e., the rate achieved without intervention from the testers).
− The assisted rate (i.e., the rate achieved with tester intervention) where these two metrics differ.
EXAMPLE If a participant received an assist on Task A, that participant should not be included among those
successfully completing the task when calculating the unassisted completion rate for that task. However, if the
participant went on to successfully complete the task following the assist, he could be included in the assisted
Task A completion rate.
When assists are allowed or provided, the number and type of assists shall be included as part of the test results. When
assists are allowed by the study design, the need for presenting the assisted completion rate is not meaningfully
distinguishable from the unassisted rate. In such cases, only a single rate needs to be provided but the number and types
of assists shall be included.
EXAMPLE In an educational setting, students might be encouraged to seek assistance.
In some usability tests, participants are instructed to use support tools such as online help or documentation, which are
part of the product, when they cannot complete tasks on their own. Accesses to product features that provide information
and help are not considered assists for the purposes of this report. It may, however, be desirable to report the frequency of
accesses to different product support features, especially if they factor into participants’ ability to use products
independently.
5.4.4.2 Efficiency
Efficiency relates the level of effectiveness achieved to the quantity of resources expended. Efficiency is
generally assessed by the mean time taken to achieve the task. Efficiency might also relate to other resources
(e.g., total cost of usage). A common measure of efficiency is time on task.
The following information shall be provided:
− The mean time taken to complete each task, together with the range and standard deviation of times
across participants.
5.4.4.2.1 Completion rate/mean time-on-task
Completion Rate/Mean Time-On-Task is another measure of efficiency [10]. The relationship of success rate to time
allows customers to compare fast error-prone interfaces (e.g., command lines with wildcards to delete files) to slow easy
interfaces (e.g., using a mouse and keyboard to drag each file to the trash).
10 © ISO/IEC 2006 – All rights reserved

5.4.4.3 Satisfaction
The following information shall be provided:
− One or more measures of user satisfaction.
Satisfaction describes a user’s subjective response when using the product. User satisfaction might be an
important correlate of motivation to use a product and might affect performance in some cases.
Questionnaires to measure satisfaction and associated attitudes are commonly built using Likert and semantic
differential scales.
A variety of instruments are available for measuring user satisfaction of software interactive products, and
many companies create their own. Whether an external, standardized instrument is used or a customized
instrument is created, subjective rating dimensions such as Satisfaction, Usefulness, and Ease of Use should
be considered for inclusion, as these will be of general interest to customer organizations.
A number of questionnaires are available that are widely used. They include: ASQ [6], CUSI [7], PSSUQ [7],
QUIS [3], SUMI [5], and SUS [2]). While each offers unique perspectives on subjective measures of product
usability, most include measurements of Satisfaction, Usefulness, and Ease of Use. General information on
questionnaire construction are addressed in Dumas and Redish [4].
Suppliers may choose to use validated published satisfaction measures or may submit satisfaction metrics
they have developed themselves.
5.5 Results
5.5.1 Data analysis
The following information shall be provided in sufficient detail to allow replication of the data analysis methods
by another organization if the test is repeated:
− Data collection – the differences between the data that was planned to be collected and the data
that was actually collected.
EXAMPLE How missing data was treated. How data was treated with respect to exclusion of outliers.
− Data scoring – the mapping between the data values that were collected and the values used in
further analysis.
EXAMPLE How errors were categorized. How actual ages map to age ranges. How assisted errors are mapped
to a set of values.
− Data reduction – the method used to compute the measure of central tendency and to characterize
the variation in the data.
EXAMPLE Which measure of central tendency was used (e.g. mean or mode). How variation was measured
(e.g., standard deviation or range).
− Statistical analysis – the statistical procedures used to analyze the data.
EXAMPLE How groups were compared (e.g., t-test. F-test …).
Data that are reported as means shall include the standard deviation and optionally the standard error of the
mean.
5.5.2 Presentation of the results
The following information shall be provided:
© ISO/IEC 2006 – All rights reserved 11

− The measures of effectiveness and efficiency that characterize the performance results per task or
task group in tabular form.
EXAMPLE The following table would be replicated for each task in a task group.
Task A
Unassisted Task Assisted Task Task Time …. Errors Assists Efficiency
Effectiveness Effectiveness (min)
User # [(%)Complete] [(%)Complete]
N
Mean
Standard
Deviation
Min
Max
Various graphical formats are effective in describing usability data at a glance. Bar graphs are useful for
describing subjective data such as that gleaned from Likert scales. A variety of plots can be used effectively to
show comparisons of expert benchmark times for a product vs. the mean participant performance time. The
data might be accompanied by a brief explanation of the results, but a detailed interpretation is discouraged in
the body of the report. If necessary, a detailed interpretation may be included in an appendix.
The following information may be provided:
− Graphs of the measures of effectiveness and efficiency that characterize the performance results.
5.5.2.1 Performance results
The following information should be provided:
− Summary Table(s) of the measures of effectiveness and efficiency that characterize the performance
results across all tasks.
− Graphical Presentation of the measures of effectiveness and efficiency that characterize the
performance results.
− Additional tables of metrics if they are relevant to the product’s design and a particular application
area.
12 © ISO/IEC 2006 – All rights reserved

EXAMPLE The following is a summary table of performance results across all tasks.
Summary
Total Unassisted Total Assisted Total … Total Errors Total Assists Efficiency
Task Task Task
Effectiveness Effectiveness Time
User # [(%)Complete] [(%)Complete] (min)
N
Mean
Standard
Deviation
Min
Max
A table of results may be presented for groups of related tasks (e.g., all program creation tasks in one group,
all debugging tasks in another group) where this is more efficient and makes sense. If a unit task has
subtasks, then the subtasks may be reported in summary form for the unit task.
EXAMPLE If a unit task is to identify all the misspelled words on a page, then the results might be
summarized as a percent of misspellings found.
5.5.2.2 Satisfaction results
The following information shall be provided:
− Summary table(s) of the measures that characterize the satisfaction results.
EXAMPLE The following table is a summary table of partial satisfaction results.
Satisfaction
Participant Ease of Use Usefulness Appearance Clarity …
Number
N
Median
Min
Max
The following information should be provided:
© ISO/IEC 2006 – All rights reserved 13

− Graphical presentation of the measures that characterize the satisfaction results.
5.6 Appendices
The following information shall be provided:
− Custom questionnaires (Sections 5.4.2.4 and 5.4.4.3), if used.
− Participant general instructions (if not in the body of the report); see Section 5.4.3.2.
− Participant task instructions (Section 5.4.3.3).
The following information may be provided:
− Release Notes explaining or updating the test results.

14 © ISO/IEC 2006 – All rights reserved

Annex A
(informative)
Checklist
Use the following checklist to assure that required elements (�) are addressed in your CIF report.
Recommended items are denoted with a ‘�’.
Title Page
� Company Logo or Name
� Identify report as: ISO/IEC 25062 Common Industry Format for Usability Test Reports
� Name the product and version that was tested
� Who led the test
� When the test was conducted
� Date the report was prepared
� Who prepared the report
� Customer Company Name
�Customer Company contact person
� Contact name(s) for questions and/or clarifications
� Supplier phone number
� Supplier e-mail address
� Supplier mailing or postal address
Executive Summary
� Provide a high level overview of the test
� Name and describe the product
� Summary of method(s) including number and type of participants and tasks
� Results expressed as mean scores or other suitable measure of central tendency
� Reason for and nature of the test
� Tabular summary of performance results
� If differences are claimed, the associated statistical probability
� Start on new page; end with page break
Introduction
Full Product Description
� Formal product name and release or version
� Describe what parts of the product were evaluated
� The user population for which the product is intended
� Assistive technologies that are supported by the product
� Brief description of the environment in which it should be used
� The type of user work that is supported by the product
© ISO/IEC 2006 – All rights reserved 15

Test Objectives
� State the objectives for the test and any areas of specific interest
� Functions and components with which the user directly and indirectly interacted
� Reason for focusing on a product subset
Method
Participants
� The total number of participants tested
� Segmentation of user groups tested, if more than one
� Key characteristics and capabilities of user groups
�
...