ISO 15746-3:2020

INTERNATIONAL ISO
STANDARD 15746-3
First edition
2020-02
Automation systems and
integration — Integration of advanced
process control and optimization
capabilities for manufacturing
systems —
Part 3:
Verification and validation
Systèmes d'automatisation et intégration — Intégration de contrôles
de processus avancés et capacités d'optimisation des systèmes de
fabrication —
Partie 3: Vérification et validation
Reference number
ISO 15746-3:2020(E)
©
ISO 2020

---------------------- Page: 1 ----------------------
ISO 15746-3:2020(E)

COPYRIGHT PROTECTED DOCUMENT
© ISO 2020
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting
on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address
below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Fax: +41 22 749 09 47
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii © ISO 2020 – All rights reserved

---------------------- Page: 2 ----------------------
ISO 15746-3:2020(E)

Contents Page
Foreword .iv
Introduction .v
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 Abbreviated terms . 2
5 Principle and purpose . 2
5.1 Principle of verification and validation . 2
5.2 Structures of indicators . 3
5.2.1 General. 3
5.2.2 Structure of quantitative indicators . 3
5.2.3 Structure of judgement indicator . 3
5.3 General process for verification and validation. 4
5.4 Verification . 5
5.5 Checkpoints in requirement analysis phase . 5
5.6 Checkpoints of design phase . 8
5.7 Checkpoints in development phase . 9
5.8 Checkpoints in execution phase .11
5.9 Checkpoints in support phase .14
6 Validation .15
Bibliography .19
© ISO 2020 – All rights reserved iii

---------------------- Page: 3 ----------------------
ISO 15746-3:2020(E)

Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards
bodies (ISO member bodies). The work of preparing International Standards is normally carried out
through ISO technical committees. Each member body interested in a subject for which a technical
committee has been established has the right to be represented on that committee. International
organizations, governmental and non-governmental, in liaison with ISO, also take part in the work.
ISO collaborates closely with the International Electrotechnical Commission (IEC) on all matters of
electrotechnical standardization.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for the
different types of ISO documents should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www .iso .org/ directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject of
patent rights. ISO shall not be held responsible for identifying any or all such patent rights. Details of
any patent rights identified during the development of the document will be in the Introduction and/or
on the ISO list of patent declarations received (see www .iso .org/ patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to the
World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see www .iso .org/
iso/ foreword .html.
This document was prepared by Technical Committee ISO/TC 184, Automation systems and integration,
Subcommittee SC 5, Interoperability, integration and architectures for enterprise systems and automation
applications.
A list of all parts in the ISO 15746 series can be found on the ISO website.
Any feedback or questions on this document should be directed to the user’s national standards body. A
complete listing of these bodies can be found at www .iso .org/ members .html.
iv © ISO 2020 – All rights reserved

---------------------- Page: 4 ----------------------
ISO 15746-3:2020(E)

Introduction
As a crucial part of the increasingly complex manufacturing systems, automation and control
applications which are enabled by advanced process control and optimization (APC-O) methodology
and solutions are implemented under the direction of production planning and scheduling. This task
involves initially the specific use of APC-O that will eventually enable the integration of manufacturing
operations management (MOM) with the automation and control of manufacturing process and
equipment.
Automation solutions equipped with both software and hardware components are provided by
different suppliers to accomplish APC-O functions. Due to the diversity of development environments
and the variety of demand focus, the automation solutions from various suppliers tend to be isolated
and relatively independent, which make it harder for the automation solutions to be integrated.
Consequently, various automation solution components that the customers can have access to are filled
with redundant and duplicated functions, resulting in a waste of resources and limited interoperability.
This document offers an interoperability framework for advanced process control and optimization
with the intention of maximizing both the integration and the interoperability of automation solutions.
It is not the intent of this document to suggest that there is only one way of implementing APC-O or to
force users to abandon their current way of implementing APC-O.
The target users of this document include users and providers of advanced process control and
optimization solutions, such as, project solution suppliers, automation systems integrators, production
departments of companies, process engineers, independent software testing organizations,
implementation and consulting service organizations of advanced process control and optimization
software, and relevant government and academic organizations.
© ISO 2020 – All rights reserved v

---------------------- Page: 5 ----------------------
INTERNATIONAL STANDARD ISO 15746-3:2020(E)
Automation systems and integration — Integration of
advanced process control and optimization capabilities for
manufacturing systems —
Part 3:
Verification and validation
1 Scope
This document defines the principle of verification and validation according to the activity models and
workflow of an advanced process control and optimization (APC-O) system, analyses and defines the
general process for verification and validation of APC-O systems, and specifies a set of indicators and
checkpoints used for verification and validation.
2 Normative references
There are no normative references in this document.
3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— ISO Online browsing platform: available at https:// www .iso .org/ obp
— IEC Electropedia: available at http:// www .electropedia .org/
3.1
checkpoint
point where verification (3.6) and validation (3.5) activities needed to be performed throughout the
APC-O lifecycle
3.2
indicator
measurement of an aspect of the system or component
Note 1 to entry: There are two types of indicators: quantitative indicators (3.3) and judgement indicators (3.4).
3.3
quantitative indicator
indicator (3.2) that is calculated using formulae
3.4
judgement indicator
indicator (3.2) that is evaluated using the evaluation method
3.5
validation
process of evaluating an APC-O system to determine whether it satisfies the stakeholders’ requirements
for that system
© ISO 2020 – All rights reserved 1

---------------------- Page: 6 ----------------------
ISO 15746-3:2020(E)

3.6
verification
process of evaluating an APC-O system to determine whether the output of a phase satisfies the
conditions imposed at the start of that phase
4 Abbreviated terms
APC-O Advanced Process Control and Optimization
MOM Manufacturing Operations Management
V&V Verification and Validation
5 Principle and purpose
5.1 Principle of verification and validation
This document provides a specification for both users and suppliers of the APC-O systems. The APC-O
system that conforms to the specification will satisfy the customer requirements and facilitate the
integration between different APC-O systems, as illustrated in Figure 1.
Figure 1 — Verification and validation of APC-O systems
Workflow: The workflow of an APC-O system relates to the phases of the lifecycle.
Output of lifecycle phase: Each work phase has an object.
Work phase: The lifecycle of an APC-O system consists of the following phases:
a) Requirements analysis;
b) Design;
2 © ISO 2020 – All rights reserved

---------------------- Page: 7 ----------------------
ISO 15746-3:2020(E)

c) Development;
d) Execution;
e) Support.
5.2 Structures of indicators
5.2.1 General
Both the quantitative indicators and the judgement indicators are expressed in the manner of the
structure specified in ISO 22400-2. The structure identifies the descriptive element of indicator in the
left column and gives a description of each element in the right column.
5.2.2 Structure of quantitative indicators
Table 1 presents an overview of the main elements of quantitative indicator structure.
Table 2 is an example of quantitative indicator.
Table 1 — Quantitative indicator structure
Name Name of the indicator
ID Unique identification
Description A brief description of the indicator
Scope The object that the indicator is used for, including the lifecycle phases or
the elements/activities within a lifecycle phase
Formula The mathematical formula of the indicator defined in terms of elements
Unit of measure The basic unit or dimension of the indicator
Range Defines the upper and lower logical limits of the indicator
Trend The information that indicates the direction for improvement, e.g. if a
higher or lower value is better
Audience Users and providers of APC-O solutions, such as project solution suppli-
ers, automation systems integrators, production departments of com-
panies, process engineers, independent software testing organizations,
implementation and consulting service organizations of APC-O software,
and relevant government and academic organizations
Table 2 — Example of quantitative indicator
Name Mean squared error (MSE)
ID
Description This checkpoint is used to describe the stable rate
NOTE  Definition of MSE
Formula Mean[(SV − PV) ^2]
Unit of measure %
Range Min: 0 %
Max: 100 %
Trend The lower, the better
Audience Engineers and operators of process
5.2.3 Structure of judgement indicator
Table 3 presents an overview of the main elements of judgement indicator structure.
© ISO 2020 – All rights reserved 3

---------------------- Page: 8 ----------------------
ISO 15746-3:2020(E)

Table 4 is an example of judgement indicator.
Table 3 — Judgement indicator structure
Name Name of the indicator
ID Unique identification
Description A brief description of the indicator
Scope Determine the object that the indicator is used for, which can be a lifecy-
cle phase or the elements/activities within the lifecycle phase
Evaluation method Check whether the object meets the desired requirements
Evaluation result Pass or fail
Audience Users and providers of APC-O solutions, such as project solution suppli-
ers, automation systems integrators, production departments of com-
panies, process engineers, independent software testing organizations,
implementation and consulting service organizations of APC-O software,
and relevant government and academic organizations
Table 4 — Example of judgement indicator
Name Optimal scheme of realization of an APC-O project
ID
Description
Scope Requirement analysis phase
Evaluation method
Evaluation result Pass or fail
Audience Managers of the APC-O project
5.3 General process for verification and validation
The general process of verification and validation in the workflow of the APC-O system is illustrated in
Figure 2.
Figure 2 — General process for verification and validation
4 © ISO 2020 – All rights reserved

---------------------- Page: 9 ----------------------
ISO 15746-3:2020(E)

The aim of verification and validation is to check whether the work meets the needs of users.
a) Define the objective of V&V according to the users’ requirements.
b) Analyse the objective and make sure that the objective applies to the work in this phase.
c) Define the indicators and the checkpoints, the applicable formula and the evaluation method in
order to establish the checklist.
d) Review the indicators and the checkpoints to ensure that the checklist meets the requirements of
V&V. Update the indicators and the checkpoints if necessary.
e) Calculate the indicators, evaluate the checkpoints according to the checklist, and generate the
evaluation report.
f) Assess the evaluation report. There are three situations that indicate whether the evaluation
report fails or passes:
1) Errors have occurred during the process of indicator calculations and checkpoint evaluations,
and modification is needed.
2) The indicators and the checkpoints are not appropriate and need to be adjusted.
3) The work in this phase does not meet the requirements, and modification is needed.
5.4 Verification
The verification process for lifecycle phases complies with the general method of verification
process in 5.3.
5.5 Checkpoints in requirement analysis phase
Table 5 presents an overview of the checklist in the requirement analysis phase.
Tables 6 to 13 describe the indicators in the requirement analysis phase.
Table 5 — Checklist in requirement analysis phase
Number Checkpoint Name ID Evaluation result
1 Requirement analysis Scope of application for
APC-O system
Selection of APC-O modules
User’s goal
Process description
Workflow
Planned coverage of APC-O
modules
Planned return on investment
Planned implementation
efficiency
Table 6 — Scope of application for APC-O system
Name Scope of application for the APC-O system
ID
Description Determine the scope of the APC-O system application based on stakehold-
ers’ requirements
© ISO 2020 – All rights reserved 5

---------------------- Page: 10 ----------------------
ISO 15746-3:2020(E)

Table 6 (continued)
Scope Requirement analysis phase
Evaluation method Check whether the scope of APC-O system application is consistent with
requirements of stakeholders.
EXAMPLE  The object can be distribution column and absorption column.
Q1: What part of the physical plant will the APC-O system be applied to?
Answer to Q1: The APC-O system will be applied to distillation column
and absorption column in this project.
EXAMPLE  The whole process includes totally five units according to the
stakeholders’ requirements, while only three out of the five units will
be applied with APC-O system. This checkpoint is to check whether the
determined three units have been correctly included.
Evaluation result Yes: pass; No: fail, to be modified
Audience Manager of APC-O project
Table 7 — Selection of APC-O modules
Name Selection of APC-O modules
ID
Description Specify the APC-O modules and the system structure
Scope Requirement analysis phase
Evaluation method Check whether the selected modules satisfy the stakeholder’s requirement
Evaluation result Yes: pass; No: fail, to be modified
Audience Engineers of APC-O project, engineers of process
Table 8 — User’s goal
Name User’s goal
ID
Description Analyse and quantify the goal of the users
Scope Requirement analysis phase
Evaluation method Check whether the user’s goal is clear and quantified
Evaluation result Yes: pass; No: fail, to be modified
Audience Managers and Engineers of APC-O project
Table 9 — Process description
Name Process description
ID
Description Specify the characteristics of the equipment and the process
Scope Requirement analysis phase
Evaluation method Check whether the characteristics of the equipment and the process are
described clearly enough for the design of an APC-O system and whether
the process description accurately describe the real process
Check whether the relevant data is complete
Evaluation result Yes: pass; No: fail, to be modified
Audience Engineers of the APC-O project, engineers of process
6 © ISO 2020 – All rights reserved

---------------------- Page: 11 ----------------------
ISO 15746-3:2020(E)

Table 10 — Workflow
Name Workflow
ID
Description Specify the project implementation procedures.
Scope Requirement analysis phase
Evaluation method Check whether the project implement
...