ISO/IEC 40180:2017

INTERNATIONAL ISO/IEC
STANDARD 40180
First edition
2017-10
Information technology — Quality
for learning, education and training
— Fundamentals and reference
framework
Technologies de l'information — Qualité pour l'apprentissage,
l'éducation et la formation — Principes fondamentaux et cadre de
référence
Reference number
©
ISO/IEC 2017
© ISO/IEC 2017, Published in Switzerland
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized otherwise in any form
or by any means, electronic or mechanical, including photocopying, or posting on the internet or an intranet, without prior
written permission. Permission can be requested from either ISO at the address below or ISO’s member body in the country of
the requester.
ISO copyright office
Ch. de Blandonnet 8 • CP 401
CH-1214 Vernier, Geneva, Switzerland
Tel. +41 22 749 01 11
Fax +41 22 749 09 47
copyright@iso.org
www.iso.org
ii © ISO/IEC 2017 – All rights reserved

Contents Page
Foreword .iv
Introduction .v
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 Abbreviated terms . 4
5 Conformance . 4
6 The Quality Reference Framework (QRF) for E-Learning . 4
6.1 General . 4
6.2 Descriptive model of the QRF . 5
6.3 Process model of the QRF . 7
6.3.1 Overview . 7
6.3.2 Description . . . 8
Annex A (informative) Introduction and use of the QRF.16
Annex B (informative) DIN process model (DIN PAS 1032-1) .20
Annex C (informative) French code of practice in E-Learning (AFNOR Z76-001) .47
Annex D (informative) Harmonized model for quality management for learning, education,
and training: mapping CELTSC and QRF .59
Annex E (informative) Specific quality objectives .63
Bibliography .67
© ISO/IEC 2017 – All rights reserved iii

Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are
members of ISO or IEC participate in the development of International Standards through technical
committees established by the respective organization to deal with particular fields of technical
activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international
organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the
work. In the field of information technology, ISO and IEC have established a joint technical committee,
ISO/IEC JTC 1.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular the different approval criteria needed for
the different types of document should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject
of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent
rights. Details of any patent rights identified during the development of the document will be in the
Introduction and/or on the ISO list of patent declarations received (see www.iso.org/patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation on the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to the
World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see the following
URL: www.iso.org/iso/foreword.html.
This document was prepared by Technical Committee ISO/IEC JTC 1, Information technology, SC 36,
Information technology for learning, education and training.
ISO/IEC 40180 cancels and replaces ISO/IEC 19796-1:2005 (initially published under the general title
Information technology — Learning, education and training — Quality management, assurance and
metrics) which has been technically revised.
iv © ISO/IEC 2017 – All rights reserved

Introduction
This document provides the fundamentals and the reference framework for quality assurance,
quality management and quality improvement in IT-enhanced learning, education and training
(called E-Learning). This document primarily consists of the Quality Reference Framework (QRF) for
E-Learning. It is a framework to describe, compare and analyse approaches for quality management
and quality assurance. These approaches can be mapped to QRF. Therefore, the framework is not a
quality management model or guiding management system – it is a framework for the description of
quality approaches. It serves to compare different existing standards and to harmonize these towards
a common quality model. For a better understanding of the QRF, several annexes show samples of its
usage – the annexes are based on the German DIN PAS 1032-1 (Annex B), the French code of practice in
E-Learning (Annex C); and an example of how it can be mapped to other quality systems, such as the
Chinese E-Learning Technology Standards(CELTS) (Annex D).
This document provides the fundamentals for the implementation and adaptation of the general quality
standards within the fields of E-Learning, education and training. This includes specific vocabulary
as well as the Quality Reference Framework (QRF) with its Process Model and Descriptive Model. It
provides sector specific information that may be used to support the better integration of mandatory
and voluntary quality approaches at the local, regional, national and transnational levels. Through the
application of this document, organizations may continue to improve the quality of their processes,
products, services and solutions within the fields of learning, education and training.
This document was carefully developed and revised over a period of more than 10 years to ensure
compatibility with ISO 9001 and ISO 14001 standards and facilitate their use by organizations that
wish to introduce and improve quality assurance, quality management and quality improvement in
learning, education and training together with the implementation of both environmental and quality
management systems to benefit themselves and their customers and stakeholders.
Overview of ISO/IEC 40180:
— Clause 6 describes the process-oriented Quality Reference Framework (QRF) for the comparison
and analysis of quality approaches.
— Annex A (informative) provides an introduction and description of the QRF and illustrates its
relation to the PDCA Cycle and the IDEAL Reference Model. It further provides answers on key
questions in order to provide a clearer understanding of the QRF concept and its use.”
— Annex B (informative) shows the full German process model (DIN PAS 1032-1) as an example of how
the basic model can be extended.
— Annex C (informative) describes the use of the model describing the “French Code of Practice in
E-Learning” (AFNOR Z 76-001) as a second sample of the use of this document.
— Annex D (informative) describes how other quality approaches can be mapped to QRF. Specifically,
the Chinese Model CELTSC is used as an example of the mapping procedure.
— Annex E (informative) describes the use of the model for specific quality objectives such as metadata
quality.
© ISO/IEC 2017 – All rights reserved v

INTERNATIONAL STANDARD ISO/IEC 40180:2017(E)
Information technology — Quality for learning, education
and training — Fundamentals and reference framework
1 Scope
This document provides the fundamentals and the reference framework for quality assurance, quality
management and quality improvement in IT-enhanced learning, education and training (called
E-Learning). It consists mainly of the Quality Reference Framework (QRF) for E-Learning, which is a
common and generic framework to describe, specify and understand critical properties, characteristics
and metrics of quality. The QRF combines an elaborated and extensive process model with a descriptive
model for the processes. This document harmonizes existing approaches, concepts, specifications,
terms and definitions related to quality for E-Learning, education and training.
2 Normative references
There are no normative references in this document.
3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— IEC Electropedia: available at http://www.electropedia.org/
— ISO Online browsing platform: available at http://www.iso.org/obp
3.1
organization
person or group of people that has its own functions with responsibilities, authorities and relationships
to achieve its objectives
[SOURCE: ISO 9000:2015, 3.2.1]
3.2
customer
person or organization who consumes the product (studying and training) directly or indirectly
EXAMPLE Learner, learner’s parents, education institutions and potential employer.
3.3
interested party
person or organization that can affect, be affected by, or perceive itself to be affected by a decision or
activity of ITLET
EXAMPLE Learner, employee, provider, partner, investor, owner, society.
[SOURCE: ISO 9000:2015, 3.2.3, modified, adapted within ITLET context]
3.4
process
set of interrelated or interacting activities which transforms inputs into outputs
Note 1 to entry: Inputs to a process are generally outputs of other processes.
© ISO/IEC 2017 – All rights reserved 1

Note 2 to entry: Processes in an organization are generally planned and carried out under controlled conditions
to add value.
Note 3 to entry: A process where the conformity of the resulting product cannot be readily or economically
verified is frequently referred to as a “special process”.
3.5
product
output of an organization that can be produced without any transaction taking place between the
organization and the customer
Note 1 to entry: Production of a product is achieved without any transaction necessarily taking place between
provider and customer, but can often involve this service (3.6) element upon its delivery to the customer.
Note 2 to entry: The dominant element of a product is that it is generally tangible.
Note 3 to entry: Hardware is tangible and its amount is a countable characteristic (e.g. tyres). Processed materials
are tangible and their amount is a continuous characteristic (e.g. fuel and soft drinks). Hardware and processed
materials are often referred to as goods. Software consists of information regardless of delivery medium (e.g.
computer programme, mobile phone app, instruction manual, dictionary content, musical composition copyright,
driver's licence).
[SOURCE: ISO 9000:2015, 3.7.6]
3.6
service
output of an organization with at least one activity necessarily performed between the organization
and the customer
Note 1 to entry: The dominant elements of a service are generally intangible.
Note 2 to entry: Service often involves activities at the interface with the customer to establish customer
requirements as well as upon delivery of the service and can involve a continuing relationship such as banks,
accountancies or public organizations, e.g. schools or hospitals.
Note 3 to entry: Provision of a service can involve, for example, the following: — an activity performed on a
customer-supplied tangible product (e.g. a car to be repaired); — an activity performed on a customer-supplied
intangible product (e.g. the income statement needed to prepare a tax return); — the delivery of an intangible
product (e.g. the delivery of information in the context of knowledge transmission); — the creation of ambience
for the customer (e.g. in hotels and restaurants);
Note 4 to entry: A service is generally experienced by the customer.
[SOURCE: ISO 9001:2015, 3.7.7]
3.7
quality
degree to which a set of inherent characteristics of an object fulfils requirements
Note 1 to entry: The term “quality” can be used with adjectives such as poor, good or excellent.
Note 2 to entry: “Inherent”, as opposed to “assigned”, means existing in an object.
[SOURCE: ISO 9001:2015, 3.6.2, modified]
3.8
data quality
set of features that concern the collection, analysis, persistence, dissemination, and usage of data
EXAMPLE Relevance, accuracy, timeliness, punctuality, accessibility, clarity, comparability, coherence.
2 © ISO/IEC 2017 – All rights reserved

3.9
quality assessment
measures carried out consistently and systematically in order to insure conformity with the
requirements of a defined specification
3.10
quality assurance
part of quality management focused on providing confidence that quality requirements will be fulfilled
[SOURCE: ISO 9000:2015, 3.3.6]
3.11
quality control
part of quality management focused on fulfilling quality requirements
[SOURCE: ISO 9000:2015, 3.3.7]
3.12
quality improvement
part of quality management focused on increasing the ability to fulfil quality requirements
Note 1 to entry: The quality requirements can be related to any aspect such as effectiveness, efficiency or
traceability.
[SOURCE: ISO 9000:2015, 3.3.8]
3.13
quality management
management with regard to quality
Note 1 to entry: Quality management can include establishing quality policies and quality objectives , and
processes to achieve these quality objectives through quality planning, quality assurance , quality control , and
quality improvement.
Note 2 to entry: [SOURCE ISO 9000:2015, 3.3.4]
3.14
quality objective
objective related to quality
Note 1 to entry: Quality objectives are generally based on the organization's quality policy.
Note 2 to entry: Quality objectives are generally specified for relevant functions, levels and processes in the
organization
Note 3 to entry: [SOURCE ISO 9000:2015, 3.7.2]
3.15
quality planning
part of quality management focused on setting quality objectives and specifying necessary operational
processes and related resources to achieve the quality objectives
Note 1 to entry: Establishing quality plans can be part of quality planning.
[SOURCE: ISO 9000:2015, 3.3.5]
3.16
quality policy
policy related to quality
Note 1 to entry: Generally the quality policy is consistent with the overall policy of the organization, can be
aligned with the organization’s vision and mission and provides a framework for the setting of quality objectives.
© ISO/IEC 2017 – All rights reserved 3

Note 2 to entry: Quality management principles presented in this International Standard can form a basis for the
establishment of a quality policy.
[SOURCE: ISO 9000:2015, 3.5.9]
3.17
total quality management
management approach of an organization, centred on quality, based on the participation of all its
members and aiming at long-term success through customer satisfaction, and benefits to all members
of the organization and to society
4 Abbreviated terms
QRF Quality Reference Framework
RQC Reference Quality Criteria
SCORM Sharable Content Object Reference Model
W3C World Wide Web Consortium
IDEAL Initiating, Diagnosing, Establishing, Acting and Learning model
PDCA Plan-Do-Check-Act cycle
5 Conformance
A quality description conforms to this document if each process included in a QRF instance corresponds
to the appropriate specification in this document and includes all sub-processes. A conforming
description may contain descriptions of processes not included in this document. The description may
contain additional data elements.
This document can assist users who are implementing ISO 9000, ISO 9001 and can be used with other
ISO quality standards, such as ISO 14001.
This document is informed by the concept and philosophy of Total Quality Management and by ISO 9000
and ISO 9001. However, it is not a Management System Standard and does not provide a model to
follow when setting up and operating a quality management system like ISO 9001. Instead, it provides
guidance and support for the introduction of quality management and improving learning quality.
This document is informed by the concept and philosophy of Environmental Management and by
ISO 14001. However, it is not an Environmental Management System Standard and does not provide a
model to follow when setting up and operating an environmental management system like ISO 14001.
Instead, it provides guidance and support for introducing and improving quality processes in learning,
education and training.
6 The Quality Reference Framework (QRF) for E-Learning
6.1 General
The QRF provides a process model and a descriptive model for the description, comparison and
analysis of process-oriented quality approaches. The QRF can be used as a meta-model for approaches
of quality management and quality assurance in learning, education and training. This means that no
assumptions and prescriptive requirements of the quality approaches are made.
4 © ISO/IEC 2017 – All rights reserved

The use scenarios of the QRF can be described as follows:
Scenario 1: Description of a quality approach
— Select a specific quality management or quality assurance approach Q1.
— Identify the processes that are covered within Q1.
— Describe Q1 according to the framework using the description categories.
Scenario 2: Comparison of quality approaches
— Select quality management or quality assurance approaches [Q1.Qn].
— Identify the processes that are covered within [Q1.Qn].
— Describe [Q1.Qn] according to the QRF using the description categories.
— Define a metric to compare [Q1.Qn].
— Perform analysis and comparison.
Scenario 3: Harmonization of quality approaches
— Select quality management or quality assurance approaches [Q1.Qn].
— Identify the processes that are covered within [Q1.Qn].
— Describe [Q1.Qn] according to the QRF using the description categories.
— Define a metric to compare [Q1.Qn].
— Perform analysis and comparison.
— Combine [Q1.Qn] towards a consensus model.
It is important to mention that the QRF process model and its descriptive model shall be used as a
general, descriptive reference framework. In a second step, good practice approaches and profiles
can be generated – these profiles could contain specific recommendations, guidelines, procedures, or
criteria.
NOTE In addition, it is important to underline that the QRF process model and its descriptive model
are not prescriptive and do not require or determine a specific sequence of the processes: In fact, in
practice, many processes are realized in parallel.
Finally, the QRF process model and its descriptive model can be extended and modified to be adapted to
the specific situation, organization, target group and requirements.
6.2 Descriptive model of the QRF
The QRF descriptive model shows the classification and documentation scheme for quality processes
(see Table 1). It is based on CEN CWA 14644. Each process will be described by this scheme.
© ISO/IEC 2017 – All rights reserved 5

Table 1 — QRF descriptive model
Attribute Description Example
ID Unique Identifier ID1234
Category Main Process Course development
Process Name Process name Method selection
Description Description of the process Within this process the didactic concept and
methods are evaluated and selected
Relations Relation to other processes Before the method selection a target group anal-
ysis must be performed; FA.6
Sub-processes/ Sub-processes/sub-aspects/tasks Method identification, method alternatives,
sub-aspects method prioritization
Objective Objective of a Process Adequate selection of one or more didactic
concepts
Method Methodology for this process Method selection shall be based on the target
group
Reference to guideline/
documents Methods are selected based on the teachers’
experience
See Method Guidelines Handbook
Result Expected result of a process Method specification
Documents
Actors Responsible/participating actors Team Didactical Design
Metrics/Criteria Evaluation and Metrics for this process Criteria catalogue 3.2.2–3.2.6
Standards Standards used ISO 9241, IEEE 1484.12.1:2003 Learning Object
Metadata
Annotation/ Further Information, Examples of usage
Example
The overview of the descriptive model of the QRF is presented in Figure 1.
Please note that the QRF process model and its descriptive model are not prescriptive and are not
determining a specific sequence of the processes: In fact and in practice, many processes are realized
in parallel. In addition, the QRF process model and its descriptive model can be extended and modified
and has to be adapted to the specific situation, organization, target group and requirements.
6 © ISO/IEC 2017 – All rights reserved

ID Category Process Description Relation

Sub-processes/
Sub-aspects
Objective
Method
Result
Actors
Metrics/Criteria
Standards
Annotation/Example
Figure 1 — Overview of the QRF descriptive model
6.3 Process model of the QRF
The QRF process model includes the relevant processes within the whole life-cycle of learning,
education and training (Table 2). The QRF process model is divided into seven parts called process
categories. Sub-processes are included with reference to a classification of processes – examples of the
use of classifications are shown in Annexes B and C.
Table 2 — QRF process model
ID Process Category Sub-Processes
NA Needs Analysis Classification
FA Framework Analysis Classification
CD Conception/Design Classification
DP Development/Production Classification
IM Implementation Classification
LP Learning Process Classification
EO Evaluation/Optimization Classification
In each part, the essential processes are described. For selected processes, examples how the reference
model can be used are included; these examples are written in italics.
The use of the process model is shown using the classification of processes from DIN PAS 1032-1, listed
fully in Annex B.
6.3.1 Overview
The overview of the process model of the QRF is presented in Figure 2.
Please note that the QRF process model and its descriptive model are not prescriptive and are not
determining a specific sequence of the processes: In fact and in practice, many processes are realized
in parallel. In addition, the QRF process model and its descriptive model can be extended and modified
and has to be adapted to the specific situation, organization, target group and requirements.
© ISO/IEC 2017 – All rights reserved 7

Figure 2 — Overview of the QRF process model
6.3.2 Description
In this section, the process model of the QRF is described including examples for the use of each category
of the description scheme (see Tables 3 to 9).
8 © ISO/IEC 2017 – All rights reserved

6.3.2.1 Needs analysis
Table 3 — QRF process model: Needs analysis
ID Category Process Description Relation
Identification and description of
requirements, demands,
NA Needs Analysis
and constraints of an education-
al project
Sub-processes/ NA.1 Initiation
Sub-aspects
NA.2 Stakeholder identification
NA.3 Definition of objectives
NA.4 Demand analysis
Objective To describe the needs and demands leading to an educational project
Method Quality Function Deployment
Result Documentation of goals, objectives, needs, and requirements of an educational project
Actors Project manager; specialists, learners, sponsors
Metrics/Criteria Indicators
Standards ISO 9000
Annotation/Example
© ISO/IEC 2017 – All rights reserved 9

6.3.2.2 Framework analysis
Table 4 — QRF process model: Framework analysis
ID Category Process Description Relation
Identification of the framework and
FA Framework Analysis NA, CD
the context of an educational process

FA.1 Analysis of the external context
FA.2 Analysis of staff resources
FA.3 Analysis of target groups
Sub-processes/
Sub-aspects
FA.4 Analysis of the institutional and organizational context
FA.5 Time and budget planning
FA.6 Environment analysis
Objective To describe relevant factors for an educational project
Method Methods of empirical social research; methods in legal and economic research and analysis
Result Documentation and validation of relevant parameters
Actors Project manager; specialists
Metrics/Criteria Check for plausibility; consultation of additional experts
Standards
Annotation/Example
10 © ISO/IEC 2017 – All rights reserved

6.3.2.3 Conception/Design
Table 5 — QRF process model: Conception/Design
ID Category Process Name Description Relations
CD Conception/Design Conception and Design
of an educational process
CD.1 Learning objectives
CD.2 Concept for contents
CD.3 Didactical concept/methods
CD.4 Roles and activities
CD.5 Organizational concept
Sub-processes /
CD.6 Technical concept
Sub-aspects
CD.7 Concept for media and interaction design
CD.8 Media concept
CD.9 Communication concept
CD.10 Concept for tests and evaluation
CD.11 Concept for maintenance
Objective To plan and design the concepts for an educational process
Method Use of Design Guidelines
Result To provide a conception and design for educational processes
Actors Consultant, Media Designers
Metrics/Criteria
Standards
Annotation/Example
© ISO/IEC 2017 – All rights reserved 11

6.3.2.4 Development/Production
Table 6 — QRF process model: Development/Production
ID Category Process Description Relation
Development/
DP Realization of concepts CD
Production
DP.1 Content realization
DP.2 Design realization
Sub-processes/
DP.3 Media realization
Sub-aspects
DP.4 Technical realization
DP.5 Maintenance
Objective To realize the conceptions
Method Implementation manual
Result Educational products and services
Actor IT-specialists, authors, developers
Metrics/Criteria
Standards
Annotation/Example
12 © ISO/IEC 2017 – All rights reserved

6.3.2.5 Implementation
Table 7 — QRF process model: Implementation
ID Category Process Description Relation
Description of the implementation
IM Implementation
of technological components
IM.1 Testing of learning resources
IM.2 Adaptation of learning resources
Sub-processes/
IM.3 Activation of learning resources
Sub-aspects
IM.4 Organization of use
IM.5 Technical infrastructure
To implement appropriate technological components being used in the educational
Objective
process
Method Change/configuration/content management
Result Learning environment including all learning resources
Actor Project manager, IT-manager
Metrics/Criteria Testing beta versions and system
Standards Software validation, e.g. following IEEE; ISO 9000
Annotation/Example
© ISO/IEC 2017 – All rights reserved 13

6.3.2.6 Learning process
Table 8 — QRF process model: Learning process
ID Category Process Description Relation
Realization and use
LP Learning Process
of the learning process
LP.1 Administration
Sub-processes/
LP.2 Activities
Sub-aspects
LP.3 Review of competence levels
Objective To perform the learning process
Method According to the chosen didactical concepts and methods
Result Completed process of learning, education and training
Actor Learners, trainers, tutors
Metrics/Criteria User performance, user satisfaction
Standards
Annotation/Example
14 © ISO/IEC 2017 – All rights reserved

6.3.2.7 Evaluation/Optimization
Table 9 — QRF process model: Evaluation/Optimization
ID Category Process Description Relation
Evaluation/ Description of the evaluation methods,
EO
Optimization principles, and procedures

EO.1 Planning
EO.2 Realization
Sub-processes/
Sub-aspects
EO.3 Analysis
EO.4 Optimization/Improvement
Objective To describe the evaluation of the educational process
Method Evaluation methods (Questionnaires, User tracking, user feedback report)
To perform an evaluation of the educational process; to optimize and improve the
Result
educational process
Actor Evaluators, learners, teachers
Metrics/Criteria
Standards ISO 9000
Annotation/Example
© ISO/IEC 2017 – All rights reserved 15

Annex A
(informative)
Introduction and use of the QRF
Annex A explains the QRF and its introduction and use.
First, a mapping of the QRF with the PDCA and the IDEAL models is provided.
Second, some key questions are answered in order to provide a clearer understanding of the QRF
concept and its use.
[15] [18]
Table A.1 provides a mapping of the QRF with the PDCA Cycle and the IDEAL Model .
Table A.1 — Mapping of the QRF with the PDCA Cycle and the IDEAL Model
QRF (ISO/IEC 40180) PDCA Model IDEAL Model
Needs Analysis Plan Initiate
Framework Analysis
Conception/Design
Development/Production Do Do
Implementation
Learning Process
Evaluation/Optimization Check Evaluate
Evaluation/Optimization Act Act
Figure A.1 provides an overview of the IDEAL Reference Model.
16 © ISO/IEC 2017 – All rights reserved

[18]
Figure A.1 — Phases of the IDEAL Reference Model
In the following, some key questions are answered (FAQ):
Is QRF customer oriented / does it take individualization into account?
In general, QRF is a framework to describe and compare quality approaches. Therefore, it does not take
any position of how “good” quality is achieved. However, it is designed so that guidelines, procedures
and requirements are included and clearly specified (see Table A.2).
© ISO/IEC 2017 – All rights reserved 17

Table A.2 — Example of quality guidelines and methods for content selection/design
ID Category Process Name Description Relations
CD.2 Conception/Design Concept of the Concept of learning and teaching    NA.4 Demand analysis
contents contents
FA.2 Qualifications
Content selection
Sub-processes/
Sub-aspects
Content Design
1 Learner Demand: The goal is to provide content adapted to the needs
and demand of the learner.
Objective
2 Adaptation: Each course shall provide different content presentation
formats and entry points based on the user experience.
A prototype of the content shall be provided to a group of learners’ representa-
tives. In a consensus process, the contents shall be prioritized and agreed on.
Method
For each course, classify groups of learners according to their learning type.
Adapt presentation format and methods according to these learning types.
1 Documentation of planned and agreed contents
Result
2 Periodically, evaluate learning performance of different learners (test
groups).
Curriculum designer, didactic experts, institution accreditation authority,
Actors
teacher, learners’ representatives
Metrics/Criteria The content are measured based on their relevance, importance, exemplarity, …
Standards Higher Education Standards
Annotation/Example
Therefore, negotiations and adaptation should be included when using QRF.
How should specific requirements and needs be included?
Specific requirements should be included in the process. Specific issues can be added to the model in
specific profiles of the model (see Table A.3).
18 © ISO/IEC 2017 – All rights reserved

Table A.3 — Inclusion of French National Body’s document concerning negotiation about
[14]
“Validation of Experiential Learning”
ID Category Process Description Relation
Framework Analysis of staff Description of actors, their
FA.2
Analysis resources qualifications and competences

Roles/Functions
Sub-processes/
Competences/formal qualifications of actors
Sub-aspects
Availability of actors
1 Description of the roles/functions, competences/formal qualifications and
availability of actors and users
Objective
2 Evaluate and certify actors’ qualifications of prior experimental learning.
Analyse learner records and certificates.
Analyse learners’ qualifications through experimental learning: Presenting a
Method
portfolio, or explaining to a jury how professional experience lead to acquire
knowledge and skills required by the qualification.
1 Description of roles/functions of staff
2 Description of competences/formal qualifications/qualifications through
Result
experimental learning
3 Description of availability of staff
Actors Project manager; experts (- > jury), learners
Metrics/Criteria
Project management and documentation guidelines; standards for social research;
Standards
Law : “Loi sur la Validation des Acquis de l'Expérience”
Annotation/Example
NOTE: In order to include the processes of negotiations and individualization, a separate process has been included in
the model.
The model can be adapted (e.g. including a more detailed process model) using classifications of sub-
processes. Examples for the use of sub-processes are given in Annex B.
© ISO/IEC 2017 – All rights reserved 19

Annex B
(informative)
DIN process model (DIN PAS 1032-1)
Annex B contains a sample use of QRF. The German DIN process model (DIN PAS 1032-1) including sub-
processes is described using QRF.
Needs Analysis (NA)
ID Category Process Description Relation
Identification and description of
NA Needs Analysis requirements, demands, and constraints of an
educational project
NA.1 Initiation
NA.2 Stakeholder Identification
Sub-processes/
Sub-aspects
NA.3 Definition of objectives
NA.4 Demand Analysis
Objective To describe the needs and demand leading to an educational project
Method Quality Function Deployment
Result Documentation of goals, objectives, needs, and requirements of an educational project
Actors Project manager; specialists, learners, sponsors
Metrics/Criteria Indicators
Standards ISO 9000
Annotation/Example
20 © ISO/IEC 2017 – All rights reserved

ID Category Process Description Relation
Initiation of an educational project; description
NA.1 Needs Analysis Initiation of necessity or demand for learning/education/ FA.4
training
— Identification of demands and requirements
Sub-process(es)/
Sub-aspects
— Identification of necessities
Description and indication of the processes goals and aims
Objective
Needs for teaching
Methods of marketing research and market analysis, trend analysis, need analysis,
Method interviews, balanced scorecard, skill gap analysis; Assessments, work place analysis,
Auditing
Definition and documentation of the needs, necessities, and demands (for learning/
Result
education/training)
End User
Actors Education and training manager
Experts in education and training
Metrics/Criteria Review of description validity
Standards ISO 9000, ANSI/PMI 99–001–2000 5.1 Initiation
Annotation/Example
© ISO/IEC 2017 – All rights reserved 21

ID Category Process Description Relation
Stakeholder Identifica- Identification, description and evaluation of the
NA.2 Needs Analysis
tion stakeholders groups
— Identification of actors
Sub-processes/
— Identification of interested parties
Sub-aspects
— Identification of users
Description of all potential stakeholders. Influence and cooperation within the educa-
Objective tion and training processes. Importance for and on the development, acceptance and
market capabilities of the finished products
Literature analysis, Staff sheets, Curricula, Workshops
Method Examination regulations, business plans
Interviews, Guidelines
Description of all interested parties, their focused objectives, documentation of ob-
Result
jectives.
Managers
Actors
Initiator
Metrics/Criteria Measurements of acceptance, contracts and applications
Standards ANSI 10.1 and ANSI 10.2
Annotation/Example Who are the respective actors and interested parties? (learners, pupils, students,
educators, teachers, trainers, tutors, managers, enterprises, organizations,
examination boards, regulative bodies in education, universities, sponsors, cooperat-
ing institutions, clients, relevant social groups).

ID Category Process Description Relation
CD.x, IM.5,
Description and evaluation of the objectives of IM.6, LP.1,
NA.3 Needs Analysis Definition of Objectives
relevant stakeholders LP.2, LP.3,
LP.4
— Strategic goals
Sub-processes/
— Tactical goals
Sub-aspects
— Operative goals
Identification, description, and evaluation of goals of the stakeholders
Objective
Transition to a quality system in accordance with ISO 9000
Method Questionnaire, Interview, Polls, Workshop.
Documentation of goals and objectives
Result
operational, consistent goal-systems for quality-measurements in accordance with DIN/
ISO 9000
Actors project or consortia leadership, user, client
Metrics/Criteria Achievement of objectives
Standards ISO 9000, ISO 14001
Annotation/Example
22 © ISO/IEC 2017 – All rights reserved

ID Category Process Description Relation
Specification, description, and evaluation of the
NA.4 Needs Analysis Demand Analysis
demand for qualification
— Description of the qualification demand
Sub-processes/
— Justification of demand
Sub-aspects
— Kind of the qualification demand (formal, informal, prescriptive, taxonomy)
Specification, description and validation of demand and objectives in an educational
Objective
project and their operational description
Method Document analysis, Interview, Workshop, systematic and methodical formulations
Result Requirement specification for project
Actors Project manager, user, supplier, sponsor
Metrics/Criteria Evaluation by experts, Certification through DIN/ISO 9000
Standards ISO 9000, ANSI PMI 99–001–2000a
Annotation/Example
Framework Analysis (FA)
ID Category Process Description Relation
Identification of the framework and the context
FA Framework Analysis NA, CD
of an educational process
FA.1 Analysis of the external context
FA.2 Analysis of staff resources
FA.3 Analysis of target groups
Sub-processes/
Sub-aspects
FA.4 Analysis of the institutional and organizational context
FA.5 Time and budget planning
FA.6 Environment analysis
Objective To describe relevant factors for an educational project
Method Methods of empirical social research; methods in legal and economic research and analysis
Result Documentation and validation of relevant parameters
Actors Project manager; specialists
Metrics/Criteria Check for plausibility; consultation of additional experts
Standards
Annotation/Example
© ISO/IEC 2017 – All rights reserved 23

ID Category Process Description Relation
Identification, description and evaluation of the
Framework Analysis of the
FA.1 external context/framework of the
Analysis external context
educational process
— Legal context/conditions
— Economic context/conditions
Sub-processes/
— Educational context/conditions
Sub-aspects
— Social context/conditions
— Political context/conditions
Identification, description and validation of the external context/framework of the
Objective
educational process
Methods of empirical social/educational research; consultation of specialists; market
Method
analysis
Identification and documentation of relevant aspects of the external context
Legal provisions concerning training intervals in security relevant areas
Development of the E-Learning market
Result
Structures and processes of the organization within which an educational process is
located; organizational culture of learning
Structures and constraints due to established curricula etc.
Planning for updates of context analysis
Actors Project manager; experts (e.g. market researchers, lawyers)
Metrics/Criteria Check on plausibility of results; additional expertise
Standards Project management and documentation guidelines; standards for social research
Annotation/Example
24 © ISO/IEC 2017 – All rights reserved

ID Category Process Description Relation
Framework Analysis of staff Identification and description of actors, their
FA.2
Analysis resources qualifications and competences, and availability

— Roles/functions
Sub-processes/
— Competences/formal qualifications
Sub-aspects
— Availability of actors
Identification of the roles/functions, competen
...