Software and systems engineering - Software testing - Part 5: Keyword-driven testing

This document defines an efficient and consistent solution for keyword-driven testing by: - giving an introduction to keyword-driven testing; - providing a reference approach to implement keyword-driven testing; - defining requirements on frameworks for keyword-driven testing to enable testers to share their work items, such as test cases, test data, keywords, or complete test specifications; - defining requirements for tools that support keyword-driven testing; these requirements are applicable to any tool that supports the keyword-driven approach (e.g. test automation, test design and test management tools); - defining interfaces and a common data exchange format to ensure that tools from different vendors can exchange their data (e.g. test cases, test data and test results); - defining levels of hierarchical keywords, and advising use of hierarchical keywords; this includes describing specific types of keywords (e.g. keywords for navigation or for checking a value) and when to use "flat" structured keywords; - providing an initial list of example generic technical (low-level) keywords, such as "inputData" or "checkValue"; these keywords can be used to specify test cases on a technical level and can be combined to create business-level keywords as required. This document is applicable to all those who want to create keyword-driven test specifications, create corresponding frameworks, or build test automation based on keywords.

Ingénierie du logiciel et des systèmes — Essais du logiciel — Partie 5: Essais axés sur des mots-clés

General Information

Status
Published
Publication Date
18-Dec-2024
Current Stage
6060 - International Standard published
Start Date
19-Dec-2024
Due Date
13-Jun-2025
Completion Date
19-Dec-2024
Ref Project

Relations

Overview

ISO/IEC/IEEE 29119-5:2024 - Software and systems engineering - Software testing - Part 5: Keyword‑driven testing defines a consistent, vendor‑neutral approach to keyword‑driven testing. The standard introduces keyword‑driven testing concepts, specifies a reference approach to implementation, and sets requirements for frameworks and tools, plus interfaces and a common data‑exchange format so test artifacts (test cases, test data, keywords, results) can be shared and exchanged between tools.

Keywords: keyword-driven testing, test automation, test framework, test data exchange format, hierarchical keywords, ISO/IEC/IEEE 29119-5:2024

Key topics and technical requirements

  • Introduction and layers: Defines layers used in keyword‑driven testing (domain layer, test interface layer, etc.) and guidance on multi‑layer architectures.
  • Types and structure of keywords: Defines simple, composite, navigation/interaction (input) and verification (output) keywords, and when to use hierarchical vs “flat” keyword structures.
  • Framework and tool requirements: Specifies requirements for keyword‑driven frameworks and for any tool that supports the keyword approach (test automation, design and management tools).
  • Interoperability and data exchange: Requires interfaces and a common data exchange format so tools from different vendors can exchange test cases, test data and results.
  • Keyword libraries and example keywords: Advises on levels of hierarchical keywords and provides an initial list of example technical (low‑level) keywords (e.g., inputData, checkValue) that can be combined into higher‑level business keywords.
  • Framework components: Describes components such as keyword‑driven editor, decomposer, data sequencer, manual test assistant, tool bridge, execution engine, keyword library and script repository.
  • Guidance and annexes: Includes informative material for getting started, typical conventions, benefits/issues, roles/tasks, example keywords and data formats.

Practical applications - who should use this standard

  • Test engineers and QA teams adopting or migrating to keyword‑driven testing to improve reusability, maintainability and separation of test logic from implementation.
  • Test automation architects and framework developers building or extending keyword libraries and execution engines.
  • Tool vendors and integrators implementing compliant interfaces and a common data‑exchange format to enable cross‑tool interoperability.
  • QA managers and organizations seeking consistent, repeatable approaches to test design, execution and reporting across projects and toolchains.

Related standards

  • Part of the ISO/IEC/IEEE 29119 family (other parts cover test processes, documentation and techniques). Implementers should align 29119‑5 with the broader 29119 series for full conformity.

ISO/IEC/IEEE 29119‑5:2024 is practical for teams that need standardized, interoperable keyword‑driven testing solutions and for vendors building tools that must exchange keyword‑based test artifacts.

Standard
ISO/IEC/IEEE 29119-5:2024 - Software and systems engineering — Software testing — Part 5: Keyword-driven testing Released:12/19/2024
English language
55 pages
sale 15% off
Preview
sale 15% off
Preview

Standards Content (Sample)


International
Standard
ISO/IEC/IEEE
29119-5
Second edition
Software and systems
2024-12
engineering — Software testing —
Part 5:
Keyword-driven testing
Ingénierie du logiciel et des systèmes — Essais du logiciel —
Partie 5: Essais axés sur des mots-clés
Reference number © ISO/IEC 2024
© ISO/IEC 2024
© IEEE 2024
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting
on the internet or an intranet, without prior written permission. Permission can be requested from either ISO or IEEE at the
respective address below or ISO’s member body in the country of the requester.
ISO copyright office Institute of Electrical and Electronics Engineers, Inc
CP 401 • Ch. de Blandonnet 8 3 Park Avenue, New York
CH-1214 Vernier, Geneva NY 10016-5997, USA
Phone: +41 22 749 01 11
Email: copyright@iso.org Email: stds.ipr@ieee.org
Website: www.iso.org Website: www.ieee.org
Published in Switzerland
© ISO/IEC 2024, © IEEE 2024 – All rights reserved
ii
Contents Page
Foreword .v
Introduction .vi
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 Conformance . 5
4.1 Intended usage .5
4.2 Full conformance .5
4.3 Tailored conformance .5
5 Introduction to keyword-driven testing . 5
5.1 Overview .5
5.2 Layers in keyword-driven testing .8
5.2.1 Overview .8
5.2.2 Domain layer .9
5.2.3 Test interface layer .10
5.2.4 Multiple layers .10
5.3 Types of keywords .11
5.3.1 Simple keywords .11
5.3.2 Composite keywords . 12
5.3.3 Navigation/interaction (input) and verification (output) . 15
5.3.4 Keywords that determine test results . 15
5.4 Keywords and data .16
6 Application of keyword-driven testing . 17
6.1 Overview .17
6.2 Identifying and documenting keywords .17
6.3 Composing test cases .18
6.4 Keywords and data-driven testing .19
6.5 Modularity and refactoring .19
6.6 Keyword-driven testing in the test design and implementation process . 20
6.6.1 Overview . 20
6.6.2 Create test model (TD1) .21
6.6.3 Identify test coverage items (TD2) .21
6.6.4 Derive test cases (TD3) .21
6.6.5 Create test procedures (TD4) . 22
6.7 Converting non-keyword-driven test cases into keyword-driven test cases . 22
7 Keyword-driven testing frameworks .23
7.1 Overview . 23
7.2 Components of a keyword-driven testing framework . 23
7.2.1 Overview . 23
7.2.2 Keyword-driven editor . 25
7.2.3 Decomposer . 25
7.2.4 Data sequencer . 26
7.2.5 Manual test assistant . . 26
7.2.6 Tool bridge . 26
7.2.7 Test execution environment and test execution engine . 26
7.2.8 Keyword library . 26
7.2.9 Data .27
7.2.10 Script repository .27
7.2.11 Documentation and organizational processes .27
7.3 Basic attributes of the keyword-driven testing framework .27
7.3.1 General information on basic attributes .27
7.3.2 Documentation.27

© ISO/IEC 2024, © IEEE 2024 – All rights reserved
iii
7.3.3 Keyword-driven editor (tool) . 28
7.3.4 Decomposer and data sequencer . 29
7.3.5 Manual test assistant (tool) . 29
7.3.6 Tool bridge . 29
7.3.7 Test execution engine . . 29
7.3.8 Keyword library . 30
7.3.9 Script repository . 30
7.4 Advanced attributes of frameworks . 30
7.4.1 General information on advanced attributes . 30
7.4.2 Documentation. 30
7.4.3 Keyword-driven editor (tool) .31
7.4.4 Decomposer and data sequencer .31
7.4.5 Manual test assistant . .31
7.4.6 Tool bridge .31
7.4.7 Test execution environment and test execution engine .31
7.4.8 Keyword library .32
7.4.9 Test data support . 33
7.4.10 Script repository . 33
8 Data interchange .33
Annex A (informative) Typical keyword conventions.34
Annex B (informative) Benefits and issues of keyword-driven testing .35
Annex C (informative) Getting started with keyword-driven testing .37
Annex D (informative) Roles and tasks .39
Annex E (informative) Basic keywords . 41
Annex F (informative) Examples of the application of keywords. 47
Annex G (informative) Example data format for keywords .51
Bibliography .55
IEEE notices and abstract .56

© ISO/IEC 2024, © IEEE 2024 – All rights reserved
iv
Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are
members of ISO or IEC participate in the development of International Standards through technical
committees established by the respective organization to deal with particular fields of technical activity.
ISO and IEC technical committees collaborate in fields of mutual interest. Other international organizations,
governmental and non-governmental, in liaison with ISO and IEC, also take part in the work.
The procedures used to develop this document and those intended for its further maintenance are described
in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for the different types
of document should be noted. This document was drafted in accordance with the editorial rules of the ISO/
IEC Directives, Part 2 (see www.iso.org/directives or www.iec.ch/members_experts/refdocs).
IEEE Standards documents are developed within the IEEE Societies and the Standards Coordinating
Committees of the IEEE Standards Association (IEEE-SA) Standards Board. The IEEE develops its standards
through a consensus development process, approved by the American National Standards Institute, which
brings together volunteers representing varied viewpoints and interests to achieve the final product.
Volunteers are not necessarily members of the Institute and serve without compensation. While the IEEE
administers the process and establishes rules to promote fairness in the consensus development process,
the IEEE does not independently evaluate, test, or verify the accuracy of any of the information contained in
its standards.
ISO and IEC draw attention to the possibility that the implementation of this document may involve the
use of (a) patent(s). ISO and IEC take no position concerning the evidence, validity or applicability of any
claimed patent rights in respect thereof. As of the date of publication of this document, ISO and IEC had not
received notice of (a) patent(s) which may be required to implement this document. However, implementers
are cautioned that this may not represent the latest information, which may be obtained from the patent
database available at www.iso.org/patents and https://patents.iec.ch. ISO and IEC shall not be held
responsible for identifying any or all such patent rights.
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and expressions
related to conformity assessment, as well as information about ISO's adherence to the World Trade
Organization (WTO) principles in the Technical Barriers to Trade (TBT), see www.iso.org/iso/foreword.html.
In the IEC, see www.iec.ch/understanding-standards.
This document was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 7, Software and systems engineering, in cooperation with the Systems and Software
Engineering Standards Committee of the IEEE Computer Society, under the Partner Standards Development
Organization cooperation agreement between ISO and IEEE.
This second edition cancels and replaces the first edition (ISO/IEC/IEEE 29119-5:2016), which has been
technically revised.
The main changes are as follows:
— updated reference to processes (test design and implementation process) according to
ISO/IEC/IEEE 29119-2:2021;
— updated reference to documents (test procedure) according to ISO/IEC/IEEE 29119-3:2021;
— included definitions for terms used in this document, based on ISO/IEC/IEEE 29119-1:2022.
A list of all parts in the ISO/IEC/IEEE 29119 series can be found on the ISO and IEC websites.
Any feedback or questions on this document should be directed to the user’s national standards
body. A complete listing of these bodies can be found at www.iso.org/members.html and
www.iec.ch/national-committees.

© ISO/IEC 2024, © IEEE 2024 – All rights reserved
v
Introduction
The purpose of the ISO/IEC/IEEE 29119 series is to define an internationally agreed set of standards for
software testing that can be used by any organization when performing any form of software testing and
using any life cycle.
This document defines a unified approach for describing test cases in a modular way, which assists with
the creation of items like keyword-driven test specifications and test automation frameworks. The term
"keyword" refers to the elements which are, once defined, used to compose test cases, such as with building
blocks. This document explains the main concepts and application of keyword-driven testing. It also defines
attributes of frameworks designed to support keyword-driven testing.
The concepts relating to software testing defined in ISO/IEC/IEEE 29119-1 are also applicable to this
document.
The test process model on which the keyword-driven testing framework is based is defined in
ISO/IEC/IEEE 29119-2. It comprises test process descriptions that define the software testing processes
at the organizational level, test management level and dynamic test level. Supporting diagrams describing
the processes are also provided in ISO/IEC/IEEE 29119-2. However, this document describes a specific
implementation of the test design and implementation process of ISO/IEC/IEEE 29119-2 for application in
keyword-driven testing, in particular in TD3 (6.6.4) and TD4 (6.6.5).
The templates and examples of test documentation as defined in ISO/IEC/IEEE 29119-3 are also applicable
to this document.
Software test design techniques that can be used during test design are defined in ISO/IEC/IEEE 29119-4.
The application of ISO/IEC/IEEE 29119-4 is assumed when designing test cases that are then described by
keywords according to this document.

© ISO/IEC 2024, © IEEE 2024 – All rights reserved
vi
International Standard ISO/IEC/IEEE 29119-5:2024(en)
Software and systems engineering — Software testing —
Part 5:
Keyword-driven testing
1 Scope
This document defines an efficient and consistent solution for keyword-driven testing by:
— giving an introduction to keyword-driven testing;
— providing a reference approach to implement keyword-driven testing;
— defining requirements on frameworks for keyword-driven testing to enable testers to share their work
items, such as test cases, test data, keywords, or complete test specifications;
— defining requirements for tools that support keyword-driven testing; these requirements are applicable
to any tool that supports the keyword-driven approach (e.g. test automation, test design and test
management tools);
— defining interfaces and a common data exchange format to ensure that tools from different vendors can
exchange their data (e.g. test cases, test data and test results);
— defining levels of hierarchical keywords, and advising use of hierarchical keywords; this includes
describing specific types of keywords (e.g. keywords for navigation or for checking a value) and when to
use "flat" structured keywords;
— providing an initial list of example generic technical (low-level) keywords, such as "inputData" or
"checkValue"; these keywords can be used to specify test cases on a technical level and can be combined
to create business-level keywords as required.
This document is applicable to all those who want to create keyword-driven test specifications, create
corresponding frameworks, or build test automation based on keywords.
2 Normative references
There are no normative references in this document.
3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
ISO, IEC and IEEE maintain terminology databases for use in standardization at the following addresses:
— ISO Online browsing platform: available at https:// www .iso .org/ obp
— IEC Electropedia: available at https:// www .electropedia .org/
— IEEE Standards Dictionary Online: available at https:// ieeexplore .ieee .org/ xpls/ dictionary .jsp
NOTE For additional terms and definitions in the field of systems and software engineering, see
ISO/IEC/IEEE 24765, which is published periodically as a “snapshot” of the SEVOCAB (Systems and software
engineering vocabulary) database and is publicly accessible at https:// www .computer .org/ sevocab.

© ISO/IEC 2024, © IEEE 2024 – All rights reserved
3.1
actual result
set of behaviours or conditions of a test item (3.23), or set of conditions of associated data or the test
environment (3.17), observed as a result of test execution (3.18)
EXAMPLE Outputs to screen, outputs to hardware, changes to data, reports, and communication messages sent.
[SOURCE: ISO/IEC/IEEE 29119-1:2022, 3.5]
3.2
composite keyword
keyword (3.8) that comprises two or more other keywords
3.3
data sequencer
component that replaces keyword (3.8) parameters with data, if necessary, across several hierarchy levels
3.4
decomposer
component that deconstructs the high-level keywords (3.7) across all levels into a sequence of low-level
keywords (3.14) that eventually comprise it
3.5
domain layer
highest level of abstraction for the test item (3.23)
Note 1 to entry: Keywords (3.8) on this level are chosen in a way that is familiar to domain experts.
3.6
expected result
observable predicted behaviour of the test item (3.23) under specified conditions based on its specification
or another source
[SOURCE: ISO/IEC/IEEE 29119-1:2022, 3.35]
3.7
high-level keyword
keyword (3.8) that covers complex activities that can be composed from other keywords and is used by
domain experts to assemble keyword test cases (3.13)
3.8
keyword
one or more words used as a reference to a specific set of actions intended to be performed during the
execution of one or more test cases (3.16)
Note 1 to entry: The actions include interactions with the user interface during the test, verification, and specific
actions to set up a test scenario.
Note 2 to entry: Keywords are named using at least one verb.
Note 3 to entry: Composite keywords (3.2) can be constructed based on other keywords.
Note 4 to entry: Keywords often have parameters so they can be used with different data.
3.9
keyword library
keyword dictionary
repository containing a set of keywords (3.8) reflecting the language and level of abstraction used to write
test cases (3.16)
3.10
keyword-driven testing
testing (3.27) using test cases (3.16) composed from keywords (3.8)

© ISO/IEC 2024, © IEEE 2024 – All rights reserved
3.11
keyword-driven testing framework
test framework (3.20) covering the functional capabilities of a keyword-driven editor, decomposer (3.4), data
sequencer (3.3), keyword library (3.9), data repository, test environment (3.17), and including organizational
processes that define their interaction and use
Note 1 to entry: For manual keyword-driven testing (3.10) the functional capabilities of a manual test assistant are also
covered, while for automated keyword-driven testing the functional capabilities of a test execution engine (3.19), tool
bridge (3.28) and script repository are also covered.
3.12
keyword execution code
implementation of a keyword (3.8) that is intended to be executed by a test execution engine (3.19)
3.13
keyword test case
sequence of keywords (3.8) and the required values for their associated parameters (as applicable) that are
composed to describe the actions of a test case (3.16)
3.14
low-level keyword
keyword (3.8) that covers only one or very few simple actions and is not composed from other keywords
3.15
manual testing
humans performing tests by entering information into a test item (3.23) and verifying the results
Note 1 to entry: Automated testing uses tools, robots, and other test execution engines (3.19) to perform tests. Manual
testing does not use these items.
[SOURCE: ISO/IEC/IEEE 29119-1:2022, 3.46]
3.16
test case
set of preconditions, inputs and expected results (3.6), developed to drive the execution of a test item (3.23)
to meet test objectives (3.24)
[SOURCE: ISO/IEC/IEEE 29119-1:2022, 3.85, modified — Notes 1 to 3 to entry have been removed.]
3.17
test environment
environment containing facilities, hardware, software, firmware, procedures, needed to conduct a test
Note 1 to entry: A test environment can contain multiple environments to accommodate specific test level or test
types (e.g. a unit test environment, a performance test environment).
Note 2 to entry: A test environment can comprise several interconnected systems or virtual environments.
[SOURCE: ISO/IEC/IEEE 29119-1:2022, 3.95]
3.18
test execution
process of running a test on the test item (3.23), producing actual results (3.1)
[SOURCE: ISO/IEC/IEEE 29119-1:2022, 3.99]
3.19
test execution engine
tool that executes test cases (3.16) by passing data to the test item (3.23), triggering the test item to run and
receiving data from the test item
Note 1 to entry: A typical test execution engine can be a part of a test framework (3.20), a capture and playback tool or
a hardware robot.
© ISO/IEC 2024, © IEEE 2024 – All rights reserved
3.20
test framework
structured set of principles, guidelines, and practices used for organizing, selecting and communicating
testing (3.27)
[SOURCE: ISO/IEC/IEEE 29119-1:2022, 3.103]
3.21
test interface
interface to the test item (3.23) used to either stimulate the test item, to get responses [e.g. actual results
(3.1)], or both
Note 1 to entry: The graphical user interface (GUI), application programming interface (API) or service-oriented
architecture (SOA) interfaces are typical test interfaces.
Note 2 to entry: Stimulating the test item can involve passing data into it via computer interfaces or attached hardware.
Note 3 to entry: Getting responses includes getting information from the test item or associated hardware.
3.22
test interface layer
lowest level of abstraction for keywords (3.8), which interacts with the test item (3.23) directly and
encapsulates the atomic (lowest level) interactions at the test interface (3.21)
3.23
test item
test object
work product to be tested
EXAMPLE Software component, system, requirements document, design specification, user guide.
[SOURCE: ISO/IEC/IEEE 29119-1:2022, 3.107]
3.24
test objective
reason for performing testing (3.27)
[SOURCE: ISO/IEC/IEEE 29119-1:2022, 3.114, modified — EXAMPLE has been removed.]
3.25
test procedure
sequence of test cases (3.16) in execution order, with any associated actions required to set up preconditions
and perform wrap-up activities post execution
[SOURCE: ISO/IEC/IEEE 29119-1:2022, 3.120]
3.26
test result
indication of whether or not a specific test case (3.16) has passed or failed, i.e. if the actual results (3.1)
correspond to the expected results (3.6) or if deviations were observed
[SOURCE: ISO/IEC/IEEE 29119-1:2022, 3.122]
3.27
testing
set of activities conducted to facilitate discovery and evaluation of properties of test items (3.23)
Note 1 to entry: Testing activities include planning, preparation, execution, reporting, and management activities,
insofar as they are directed towards testing.
[SOURCE: ISO/IEC/IEEE 29119-1:2022, 3.131]

© ISO/IEC 2024, © IEEE 2024 – All rights reserved
3.28
tool bridge
component that maps and transforms the keywords (3.8) to code and data to be executed by the test execution
engine (3.19)
4 Conformance
4.1 Intended usage
The requirements in this document are contained in Clause 7.
This document provides requirements on the components of frameworks supporting the application of
keyword-driven testing. It also provides requirements on conventions for the definition of keywords.
It is recognized that some organizations, projects, or teams may not need to use all of the components
defined in this document. Therefore, implementation of this document typically involves selecting a set of
components or parts of components suitable for the organization or project.
There are two ways that an implementation can be claimed to conform to this document – full conformance
and tailored conformance. The organization shall assert whether it is claiming full or tailored conformance
to this document.
4.2 Full conformance
Full conformance is achieved by providing evidence that all of the keyword-driven testing requirements
defined in Clause 7 have been met.
4.3 Tailored conformance
When this document is used for implementing components of frameworks that do not qualify for full
conformance, the subset of components for which tailored conformance is claimed should be recorded.
Tailored conformance is achieved by demonstrating that all of the requirements for the recorded subset of
components defined in Clause 7 have been satisfied.
Where tailoring occurs, the justification shall be provided, either directly or by reference, whenever a
requirement defined in Clause 7 is not followed. All tailoring decisions shall be recorded with their rationale,
including the consideration of any applicable risks. Tailoring decisions shall be agreed to by the relevant
stakeholders.
EXAMPLE A tool vendor provides only part of a keyword-driven testing framework in its portfolio, and thus
decides not to implement requirements that are covered by complementary tools (e.g. a vendor only provides an
execution engine, but no keyword-driven editor – then the execution engine can still conform to this document).
5 Introduction to keyword-driven testing
5.1 Overview
Keyword-driven testing is a test case specification approach that is commonly used to support test
automation and the development of test automation frameworks. However, it can also be used if no
automation approach is planned or established.
In principle, keyword-driven testing can be applied at all testing levels (e.g. component testing, system
testing) and all types of testing (e.g. functional testing, reliability testing). Keyword-driven testing benefits
include the following:
— ease of use;
— understandability;
© ISO/IEC 2024, © IEEE 2024 – All rights reserved
— maintainability;
— test information reuse;
— support of test automation;
— potential cost and schedule savings.
The fundamental idea in keyword-driven testing is to provide a set of "building blocks", referred to as
keywords, that can be used to create manual or automated test cases without requiring detailed knowledge
of programming or test tool expertise. The ultimate goal is to provide a basic, unambiguous set of keywords
comprehensive enough so that most, if not all, required test cases can be entirely composed of these
keywords. The vocabulary included in these dictionaries or libraries of keywords is, therefore, a reflection
of the language and level of abstraction used to write the test cases, and not of any standard computer
programming language.
For test automation, each keyword must be implemented in software.
NOTE 1 When keywords are not used, test cases are usually written using natural language or written in a
computer programming language. Compared with natural language, keywords have the advantage of being less
ambiguous and more precise. Compared with a computer programming language, when keywords are well-defined
and well-structured, they have the advantage of being understandable by many people who do not have software
engineering skills.
A keyword is usually defined at the following two levels.
— At a low level, each keyword is associated with a detailed set of one or more actions that describe the
exact steps that are to be performed.
— At a high level, each keyword is ideally assigned a name which is meaningful to non-technical users. This
keyword can require a set of input parameters, which would also belong to this level in the structure.
The keyword and the parameters together comprise a high-level description of the actions associated
with a test case.
Thus, instead of describing each individual action that needs to be taken in each test case, tests can be
defined at a higher level of abstraction using keywords. Higher levels of abstraction can be achieved by using
composite keywords, which are comprised of other keywords to describe associated actions.
An example of the benefits obtained from both manual and automated keyword-driven test cases is
enhanced maintainability. Consider a case where the precise set of actions to carry out a commonly repeated
operation has changed. The modularity introduced by keywords allows modification of only the actions
for the changed operation in the relevant lower-level keyword, leaving the test cases untouched. Without
modularity, it can be necessary to modify each occurrence of this operation in all of the test cases.
Modularization has helped popularize this approach. If test automation is required, a framework can be
created to interpret manually created keyword test cases as executable test automation scripts. This is
achieved by implementing test automation code for each keyword (e.g. keyword execution code).
NOTE 2 Testing tools can be used to support keyword-driven testing, but the available tools can be limited in their
capability to support all the concepts described in this document.

© ISO/IEC 2024, © IEEE 2024 – All rights reserved
Figure 1 — Relationships between keyword-driven testing entities
A test procedure can have multiple test cases in it, and a test case can, in turn, be part of different test
procedures as shown in Figure 1. Test cases can be either manual test cases or keyword test cases. A
keyword test case implements a manual test case.
A keyword test case is typically composed of a sequenced series of keywords. Keywords should be chosen to
be modular and generic so that they can be reused in many test cases. Keywords can also be used more than
once in the same test case. A test case is composed from test actions. Keywords represent test actions.
NOTE 3 It is possible to map several keywords to a single action. It is also possible to define keywords in a way that
each keyword represents one action. In these cases, a one-to-one relationship exists between actions and keywords.
However, a test designer can decide to structure keywords in a different way (e.g. use more than one keyword
to implement an action, or to combine two or more actions into one keyword). Thus, this relationship is not a 1:1
relationship in Figure 1.
Test automation is an option that can be chosen when implementing keyword-driven testing, but a
manual approach is also possible. If keyword test cases are automated, each keyword is implemented by
keyword execution code. Keyword execution code is specific to the chosen tool or test execution engine and
additionally depends on the test interface. For the manual approach, the action described by a keyword is
executed manually, so there is no keyword execution code. That is why in Figure 1 the relationship between
keyword and keyword execution code is 1 to 0.1.
Test automation is typically highly technical and tool-dependent since it depends on the test interface and
on the capabilities of the available tools. In general, keywords can be independent of the test interface (e.g.
user interface) and the tools used to execute the test cases.

© ISO/IEC 2024, © IEEE 2024 – All rights reserved
In this context, automated test scripts can either be generated automatically by a framework or developed
manually by a test automation specialist. Automated test scripts are typically developed by testers with
programming experience.
NOTE 4 When developing automated test scripts, it is beneficial to align the structure (e.g. levels) of the keywords
with the implementation of the automated test scripts.
If a keyword test case or a set of keyword test cases is automated, the framework for keyword-driven testing
generates the automated test script based on the keyword execution code.
NOTE 5 A framework for keyword-driven testing does not necessarily "generate" code. The required code can also
be prepared by testers and be executed by the framework.
Annex B provides an overview of advantages and disadvantages of keyword-driven testing.
5.2 Layers in keyword-driven testing
5.2.1 Overview
Keywords can represent actions at different abstraction levels. For example, one keyword can refer to a very
complex set of activities, like the creation of a contract, which includes a lot of steps, while another keyword
can refer to a very simple action, like pressing a button on a graphical user interface. The first keyword is
close to the business and end user domain, while the second is closer to the test interface. Keywords that
are written at a similar level of detail, and have a similar relationship to the stakeholder's view, are said to
belong to the same abstraction layer.
Keyword-driven testing can be organized by using one or more layers. Typical layers are the end user
domain layer and the test interface layer.
While many implementations of keyword-driven testing comprise two or three abstraction layers, in some
cases it can be necessary to structure keywords in more layers.
The topmost layer is the most abstract layer, which is generally aligned with the wording of the application's
users. In practice, the topmost layer is usually the domain layer. However, in some situations the domain
layer is not required, and another, more abstract layer is used (e.g. if the test cases are supposed to span
several different end user domains, a meta domain layer can be introduced).
The lowest layer is the most detailed layer. It is commonly aligned with the names of test interface elements
(e.g. "SelectMenu"). In practice, this layer is usually, but not always, the test interface layer (e.g. as sometimes
a test interface layer is not required, or for specific reasons, even more detailed layers can be used).
Most keyword-driven test systems have more than one layer due to factors such as having understandable
keyword test cases, maintainability and division of work relying on a multi-layer structure. If only one layer
is implemented, it is commonly either at a low level, which affects the readability of the keyword test cases,
or at a high level, which can result in more keyword execution code.
In Figure 2, an example is given, showing how test cases for two different test interfaces can be structured
by using two layers of keywords.

© ISO/IEC 2024, © IEEE 2024 – All rights reserved
Figure 2 — Example for defining test cases by keywords at several layers
EXAMPLE In Figure 2, two test cases are shown which are designed using a domain layer and a test interface
layer. One of the examples sketches a test case for a GUI application, the other for a camera API. In both examples, the
implementation of the test cases in respect to test automation is done on the test interface layer. From top to bottom,
the example shows a test case for each test item, provides a possible structure of the composite keywords used at the
domain layer for both test items, and outlines th
...

Questions, Comments and Discussion

Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.

Loading comments...

Frequently Asked Questions

ISO/IEC/IEEE 29119-5:2024 is a standard published by the International Organization for Standardization (ISO). Its full title is "Software and systems engineering - Software testing - Part 5: Keyword-driven testing". This standard covers: This document defines an efficient and consistent solution for keyword-driven testing by: - giving an introduction to keyword-driven testing; - providing a reference approach to implement keyword-driven testing; - defining requirements on frameworks for keyword-driven testing to enable testers to share their work items, such as test cases, test data, keywords, or complete test specifications; - defining requirements for tools that support keyword-driven testing; these requirements are applicable to any tool that supports the keyword-driven approach (e.g. test automation, test design and test management tools); - defining interfaces and a common data exchange format to ensure that tools from different vendors can exchange their data (e.g. test cases, test data and test results); - defining levels of hierarchical keywords, and advising use of hierarchical keywords; this includes describing specific types of keywords (e.g. keywords for navigation or for checking a value) and when to use "flat" structured keywords; - providing an initial list of example generic technical (low-level) keywords, such as "inputData" or "checkValue"; these keywords can be used to specify test cases on a technical level and can be combined to create business-level keywords as required. This document is applicable to all those who want to create keyword-driven test specifications, create corresponding frameworks, or build test automation based on keywords.

This document defines an efficient and consistent solution for keyword-driven testing by: - giving an introduction to keyword-driven testing; - providing a reference approach to implement keyword-driven testing; - defining requirements on frameworks for keyword-driven testing to enable testers to share their work items, such as test cases, test data, keywords, or complete test specifications; - defining requirements for tools that support keyword-driven testing; these requirements are applicable to any tool that supports the keyword-driven approach (e.g. test automation, test design and test management tools); - defining interfaces and a common data exchange format to ensure that tools from different vendors can exchange their data (e.g. test cases, test data and test results); - defining levels of hierarchical keywords, and advising use of hierarchical keywords; this includes describing specific types of keywords (e.g. keywords for navigation or for checking a value) and when to use "flat" structured keywords; - providing an initial list of example generic technical (low-level) keywords, such as "inputData" or "checkValue"; these keywords can be used to specify test cases on a technical level and can be combined to create business-level keywords as required. This document is applicable to all those who want to create keyword-driven test specifications, create corresponding frameworks, or build test automation based on keywords.

ISO/IEC/IEEE 29119-5:2024 is classified under the following ICS (International Classification for Standards) categories: 35.080 - Software. The ICS classification helps identify the subject area and facilitates finding related standards.

ISO/IEC/IEEE 29119-5:2024 has the following relationships with other standards: It is inter standard links to ISO/IEC/IEEE 29119-5:2016. Understanding these relationships helps ensure you are using the most current and applicable version of the standard.

You can purchase ISO/IEC/IEEE 29119-5:2024 directly from iTeh Standards. The document is available in PDF format and is delivered instantly after payment. Add the standard to your cart and complete the secure checkout process. iTeh Standards is an authorized distributor of ISO standards.

Die Norm ISO/IEC/IEEE 29119-5:2024 bietet einen umfassenden Rahmen für die Anwendung von keyword-driven testing in der Software- und Systemtechnik. Mit einem klaren Fokus auf Effizienz und Konsistenz definiert das Dokument die grundlegenden Anforderungen und Ansätze, die für die Implementierung von keyword-driven testing erforderlich sind. Ein herausragendes Merkmal der Norm ist die Einführung in das Konzept des keyword-driven testing, das es Testern ermöglicht, ihre Testfall-, Testdaten- und Schlüsselwortdefinitionen auf eine strukturierte Weise zu organisieren. Durch die Bereitstellung eines Referenzansatzes zur Implementierung wird sichergestellt, dass verschiedene Teams innerhalb eines Unternehmens einheitliche Praktiken verfolgen, was die Zusammenarbeit und den Wissensaustausch fördert. Die Norm definiert auch klare Anforderungen an Rahmenbedingungen und Tools, die keyword-driven testing unterstützen. Diese Anforderungen sind so gestaltet, dass sie für eine Vielzahl von Tools anwendbar sind, einschließlich solcher für Testautomatisierung, Testdesign und Testmanagement. Dies erhöht die Flexibilität und Interoperabilität zwischen unterschiedlichen Testtools, was letztlich zu einer effizienteren Testumgebung führt. Besonders hervorzuheben ist die Definition von Schnittstellen und eines gemeinsamen Datenformat-Austauschs, das es ermöglicht, dass Tools von verschiedenen Anbietern Daten wie Testfälle, Testdaten und Testergebnisse nahtlos miteinander austauschen können. Diese Interoperabilität ist entscheidend, um die Effizienz in der Softwareprüfung zu steigern und redundante Arbeiten zu minimieren. Darüber hinaus legt die Norm eine klare Hierarchie für Schlüsselwörter fest und bietet Ratschläge zur Verwendung von hierarchischen Schlüsselwörtern. Die Beschreibung spezifischer Arten von Schlüsselwörtern, wie solche für Navigation oder zur Überprüfung von Werten, sowie die Bereitstellung einer Liste von Beispiel-Keywords auf technischer Ebene, unterstützt Tester bei der Erstellung präziser und zielgerichteter Testspezifikationen. Insgesamt zeigt die ISO/IEC/IEEE 29119-5:2024 Norm eine hohe Relevanz für Fachleute, die an der Erstellung von spezifizierten Tests auf Basis von Schlüsselwörtern interessiert sind. Sie fördert nicht nur die Standardisierung und Effizienz in der Softwaretestung, sondern trägt auch dazu bei, dass Testteams ihre Arbeit effektiver und miteinander verknüpft durchführen können.

La norme ISO/IEC/IEEE 29119-5:2024 offre une approche structurée et cohérente pour le testing basé sur des mots-clés. Son champ d'application est vaste, touchant non seulement les développeurs et les testeurs, mais également les équipes d'assurance qualité cherchant à standardiser leurs pratiques de test. Parmi les forces majeures de cette norme, on trouve la définition claire des exigences relatives aux cadres pour le testing basé sur des mots-clés, permettant ainsi un partage efficace des éléments de travail. Cela inclut les cas de test, les données de test et les spécifications de test complètes, favorisant une collaboration fluide entre les différents intervenants. De plus, la norme établit les exigences pour les outils qui soutiennent cette approche, garantissant leur interopérabilité et leur efficacité. La proposition d'interfaces et d'un format d'échange de données commun est particulièrement pertinente, car elle permet aux outils de différents fournisseurs d'échanger des informations critiques, telles que les cas de test et les résultats de test. Cette normalisation des échanges de données est essentielle dans un écosystème où coexistent divers outils de testification, d'automatisation et de gestion des tests. La norme recommande également l'utilisation de mots-clés hiérarchiques pour structurer les spécifications de test, ce qui facilite la compréhension et la réutilisation des mots-clés techniques. Elle propose une liste initiale de mots-clés techniques génériques, tels que "inputData" et "checkValue", permettant aux utilisateurs de créer des spécifications de test à différents niveaux, du niveau technique au niveau business. En résumé, l'ISO/IEC/IEEE 29119-5:2024 se positionne comme un document indispensable pour quiconque s'engage dans la création de spécifications de test basées sur des mots-clés, la mise en place de cadres adéquats ou le développement d'automatisation de tests, justifiant ainsi sa pertinence dans le domaine de l'ingénierie logicielle et des systèmes.

ISO/IEC/IEEE 29119-5:2024 문서는 키워드 기반 테스트의 표준화를 위한 포괄적이고 효율적인 솔루션을 정의하고 있습니다. 이 표준의 주요 강점은 키워드 기반 테스트에 대한 명확한 소개와 함께 테스트를 구현하기 위한 참조 접근 방식을 제공한다는 점입니다. 이를 통해 개발자는 보다 일관된 방식으로 테스트를 구성할 수 있습니다. 특히, 키워드 기반 테스트 프레임워크에 대한 요구 사항을 정의하여 테스터들이 테스트 케이스, 테스트 데이터, 키워드 또는 전체 테스트 사양과 같은 작업 항목을 공유할 수 있게 하며, 효율적인 협업을 가능하게 합니다. 또한, 키워드 기반 테스트를 지원하는 도구에 대한 요구 사항이 명확히 규정되어 있어, 테스트 자동화, 테스트 설계 및 테스트 관리 도구와 같은 다양한 도구에서 일관된 접근 방식이 가능하게 됩니다. 이 표준은 여러 벤더의 도구 간에 데이터(예: 테스트 케이스, 테스트 데이터 및 테스트 결과)를 교환할 수 있도록 보장하는 인터페이스와 공통 데이터 교환 형식을 정의합니다. 이를 통해 조직 내외부에서 효율적이고 유기적인 정보 흐름이 이루어질 수 있습니다. 또한, 계층적 키워드의 수준을 정의하고 이러한 키워드를 권장하는 점은 키워드 기반 테스트의 구조적 접근 방식을 강화합니다. 특정 유형의 키워드(예: 내비게이션을 위한 키워드 또는 값 검사를 위한 키워드)에 대한 설명도 포함되어 있어, 언제 "평면" 구조의 키워드를 사용할지를 명시적으로 지침을 제공합니다. 마지막으로, "inputData" 또는 "checkValue"와 같은 일반 기술적(저수준) 키워드의 초기 목록을 제공하여, 테스터들이 기술적 수준에서 테스트 케이스를 명확히 정의할 수 있게 도와주며, 필요에 따라 비즈니스 수준의 키워드로 조합할 수 있는 유연성을 제공합니다. ISO/IEC/IEEE 29119-5:2024 표준은 키워드 기반 테스트 사양을 작성하거나 이를 바탕으로 프레임워크 및 테스트 자동화를 구축하려는 모든 이들에게 필수적인 문서입니다.

ISO/IEC/IEEE 29119-5:2024 provides a comprehensive framework for keyword-driven testing, a method that is increasingly relevant in today’s software and systems engineering landscape. This standard effectively outlines the principles and practices necessary for implementing keyword-driven testing, ensuring both consistency and efficiency in test development. One of the standout strengths of this standard is its detailed introduction to keyword-driven testing, which serves as a solid foundation for practitioners new to this approach. By presenting a reference approach, it not only guides organizations on how to implement keyword-driven testing but also helps streamline the process across teams and projects. The clarity and systematic structure of the document facilitate easier adoption of these methodologies. The requirements specified for frameworks supporting keyword-driven testing are particularly noteworthy. These requirements enable testers to share vital work items, such as test cases, test data, and key keywords. By fostering a collaborative environment, this standard encourages knowledge sharing and enhances collective test efficiency. The ability to share complete test specifications is vital for teams working in agile environments where flexibility and rapid iteration are key. Furthermore, the standard’s emphasis on tool compatibility greatly enhances its relevance. By defining requirements for tools that may support the keyword-driven approach-ranging from test automation to test management-it ensures that various testing tools can integrate smoothly. This is vital for organizations using diverse technology stacks and seeking to maintain interoperability among tools from different vendors. The definition of interfaces and a common data exchange format is another critical aspect that positions this standard as an essential resource for organizations engaged in software testing. Effective data exchange is necessary to manage complex testing environments, and this aspect of the standard addresses those needs comprehensively. Moreover, the inclusion of hierarchical keywords and guidance on their usage provides a sophisticated layer to keyword-driven testing. By advising on specific types of keywords and scenarios for their implementation, it allows testers to develop more nuanced and effective test cases. This layered approach can significantly enhance the clarity and organization of test specifications. Additionally, the standard presents an initial list of example low-level technical keywords, such as "inputData" and "checkValue." This inclusion not only aids in direct application but also acts as a springboard for developers and testers to create customized keywords suitable for their projects, ensuring that the standards adapt to varying needs. Overall, ISO/IEC/IEEE 29119-5:2024 is a pivotal standard that defines keyword-driven testing and addresses many practical considerations surrounding its implementation. Its strengths lie in its clarity, comprehensive coverage of requirements, and its relevance in promoting best practices across testing frameworks. This documentation will undoubtedly serve as a valuable resource for organizations aiming to enhance their software testing methodologies through keyword-driven approaches.

ISO/IEC/IEEE 29119-5:2024は、キーワード駆動テストに関する標準的なドキュメントであり、その範囲は非常に広範囲です。この標準は、キーワード駆動テストを効果的かつ一貫した方法で実施するための解決策を定義しています。具体的には、キーワード駆動テストの導入、実装方法の参考アプローチ、テストケースやテストデータ、キーワード、完全なテスト仕様書などの作業アイテムを共有できるフレームワークの要件、さらにテスト自動化やテスト設計、テスト管理ツールに適用可能なツールの要件を定義しています。 特に注目すべきは、異なるベンダーのツールがデータを交換できるインターフェースと共通データ交換フォーマットの定義です。これにより、キーワード駆動テストを採用する際の互換性が確保され、テストケースやテストデータ、テスト結果などの統一的な管理が可能となります。また、階層的なキーワードのレベルを定義し、キーワードの使用に関する具体的なアドバイスも提供されています。これにより、ユーザーはナビゲーション用や値チェック用など、特定のタイプのキーワードを適切に使用し、「フラット」構造のキーワードを使うタイミングを理解することができます。 さらに、特定の低レベルの一般的技術キーワード(例:"inputData" や "checkValue")の初期リストを提供しており、これによりユーザーは技術的レベルでテストケースを指定し、ビジネスレベルのキーワードを酌み取って作成することが可能です。このようなキーワード駆動テストのフレームワークを構築したい、またはそれに基づいてテスト自動化を実施したいと考えるすべての関係者に適用される、非常に有用なドキュメントです。 この標準は、ソフトウェアおよびシステムエンジニアリングの分野におけるテストプロセスの向上に寄与するとともに、効率的なテスト手法を求める多くの組織にとって、その強力な基盤を提供します。キーワード駆動テストの実施において、ISO/IEC/IEEE 29119-5:2024は不可欠な参考文献となるでしょう。