ISO/IEC TR 24372:2021

TECHNICAL ISO/IEC TR
REPORT 24372
First edition
2021-12
Information technology — Artificial
intelligence (AI) — Overview of
computational approaches for AI
systems
Reference number
© ISO/IEC 2021
© ISO/IEC 2021
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting on
the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address below
or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii
© ISO/IEC 2021 – All rights reserved

Contents Page
Foreword .v
Introduction . vi
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 Abbreviated terms . 2
5 General . 3
6 Main characteristics of AI systems . .5
6.1 General . 5
6.2 Typical characteristics of AI systems . 6
6.2.1 Adaptable . 6
6.2.2 Constructive . 6
6.2.3 Coordinated . 6
6.2.4 Dynamic . 6
6.2.5 Explainable . 6
6.2.6 Discriminative or generative . 6
6.2.7 Introspective . . 6
6.2.8 Trained or trainable. 7
6.2.9 Accommodating various data . 7
6.3 Computational characteristics of AI systems . 7
6.3.1 Data-based or knowledge-based . 7
6.3.2 Infrastructure-based. 7
6.3.3 Algorithm-dependent . 8
6.3.4 Multi-step or end-to-end learning-based . 9
7 Types of AI computational approaches . 9
7.1 General . 9
7.2 Knowledge-driven approaches . 10
7.3 Data-driven approaches . 10
8 Selected algorithms and approaches used in AI systems .11
8.1 General . 11
8.2 Knowledge engineering and representation . 11
8.2.1 General . 11
8.2.2 Ontology .12
8.2.3 Knowledge graph .12
8.2.4 Semantic web . . 14
8.3 Logic and reasoning . 14
8.3.1 General . 14
8.3.2 Inductive reasoning . 15
8.3.3 Deductive inference .15
8.3.4 Hypothetical reasoning . 16
8.3.5 Bayesian inference . 17
8.4 Machine learning . 18
8.4.1 General . 18
8.4.2 Decision tree . 18
8.4.3 Random forest . 19
8.4.4 Linear regression .20
8.4.5 Logistic regression . 21
8.4.6 K-nearest neighbour . 21
8.4.7 Naïve Bayes . 22
8.4.8 Feedforward neural network. 22
8.4.9 Recurrent neural network . 23
iii
© ISO/IEC 2021 – All rights reserved

8.4.10 Long short-term memory network. 24
8.4.11 Convolutional neural network . 25
8.4.12 Generative adversarial network . 26
8.4.13 Transfer learning . 27
8.4.14 Bidirectional encoder representations from transformers . 27
8.4.15 XLNet .28
8.5 Metaheuristics .29
8.5.1 General .29
8.5.2 Genetic algorithms .29
Bibliography .31
iv
© ISO/IEC 2021 – All rights reserved

Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are
members of ISO or IEC participate in the development of International Standards through technical
committees established by the respective organization to deal with particular fields of technical
activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international
organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the
work.
The procedures used to develop this document and those intended for its further maintenance
are described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria
needed for the different types of document should be noted. This document was drafted in
accordance with the editorial rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives or
www.iec.ch/members_experts/refdocs).
Attention is drawn to the possibility that some of the elements of this document may be the subject
of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent
rights. Details of any patent rights identified during the development of the document will be in the
Introduction and/or on the ISO list of patent declarations received (see www.iso.org/patents) or the IEC
list of patent declarations received (see https://patents.iec.ch).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to
the World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see
www.iso.org/iso/foreword.html. In the IEC, see www.iec.ch/understanding-standards.
This document was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 42, Artificial intelligence.
Any feedback or questions on this document should be directed to the user’s national standards
body. A complete listing of these bodies can be found at www.iso.org/members.html and
www.iec.ch/national-committees.
v
© ISO/IEC 2021 – All rights reserved

Introduction
Artificial intelligence (AI)-related products, systems and solutions have become more common
in recent years thanks to rapid software and hardware improvements that boost computational
performance, data storage capabilities and network bandwidth. The intent of this document is to look at
1) 2)
computational methods and approaches within AI systems. Based on ISO/IEC 22989 , ISO/IEC 23053
and ISO/IEC TR 24030, this document provides a description of the characteristics of an AI system and
its computational approaches. The illustration of computational approaches in AI systems includes
both machine learning and non-machine learning methods. To reflect state-of-the-art methods used in
AI, this document is structured as follows:
— Clause 5 provides an overall description of computational approaches in AI systems;
— Clause 6 discusses the main characteristics of AI systems;
— Clause 7 provides a general taxonomy of computational approaches, including knowledge-driven
and data-driven approaches;
— Clause 8 discusses selected algorithms used in AI systems, including basic theories and techniques,
main characteristics and typical applications.
By giving an overview of different technologies used by AI systems, this document is intended to help
users understand computational characteristics and approaches used in AI.
1) Under preparation. Stage at the time of publication: ISO/IEC DIS 22989:2021.
2) Under preparation. Stage at the time of publication: ISO/IEC DIS 23053:2021.
vi
© ISO/IEC 2021 – All rights reserved

TECHNICAL REPORT ISO/IEC TR 24372:2021(E)
Information technology — Artificial intelligence (AI) —
Overview of computational approaches for AI systems
1 Scope
This document provides an overview of the state of the art of computational approaches for AI systems,
by describing: a) main computational characteristics of AI systems; b) main algorithms and approaches
used in AI systems, referencing use cases contained in ISO/IEC TR 24030.
2 Normative references
The following documents are referred to in the text in such a way that some or all of their content
constitutes requirements of this document. For dated references, only the edition cited applies. For
undated references, the latest edition of the referenced document (including any amendments) applies.
ISO/IEC 22989, Information technology — Artificial intelligence — Artificial intelligence concepts and
terminology
ISO/IEC 23053, Framework for artificial intelligence (AI) systems using machine learning (ML)
3 Terms and definitions
For the purposes of this document, the terms and definitions given in ISO/IEC 22989 and ISO/IEC 23053
and the following apply.
ISO and IEC maintain terminology databases for use in standardization at the following addresses:
— ISO Online browsing platform: available at https:// www .iso .org/ obp
— IEC Electropedia: available at https:// www .electropedia .org/
3.1
heuristic search
search, based on experience and judgment, used to obtain acceptable results without guarantee of
success
[SOURCE: ISO/IEC 2382:2015, 2123854, modified — Notes to entry removed.]
3.2
fuzzy logic
fuzzy-set logic
nonclassical logic in which facts, inference rules and quantifiers are given certainty factors
[SOURCE: ISO/IEC 2382:2015, 2123795, modified — Notes to entry removed.]
3.3
generator
neural network that produces samples usually to be classified by a discriminator
Note 1 to entry: Generators primarily appear in the context of generative adversarial networks.
© ISO/IEC 2021 – All rights reserved

3.4
discriminator
neural network that classifies samples usually produced by a generator
Note 1 to entry: Discriminators primarily appear in the context of generative adversarial networks.
3.5
generative adversarial network
GAN
neural network architecture comprised of one or more generators and one or more discriminators that
compete to improve model performance
3.6
platform
combination of an operating system and hardware that makes up the operating environment in which
a program runs
[SOURCE: ISO/IEC/IEEE 26513:2017, 3.30]
3.7
perceptron
neural network consisting of one artificial neuron, with a binary or continuous output value that is
determined by applying a monotonic function to a linear combination of the input values and with
error-correction learning
Note 1 to entry: The perceptron forms two decision regions separated by a hyperplane.
Note 2 to entry: For binary input values, the perceptron cannot implement the non-equivalence operation
(EXCLUSIVE OR, XOR).
[SOURCE: ISO/IEC 2382:2015, 2120656, modified — term revised, “or continuous” added to definition
and Notes 3 and 4 to entry removed.]
4 Abbreviated terms
AI artificial intelligence
ASIC application-specific integrated circuit
BERT bidirectional encoder representations from transformers
BPTT back propagation through time
CNN convolutional neural network
CPU central processing unit
DAG directed acyclic graph
DNN deep neural network
ERM empirical risk minimization
FFNN feedforward neural network
FPGA field programmable gate array
GDM gradient descent method
GPU graphics processing unit
© ISO/IEC 2021 – All rights reserved

GPT generative pre-training
IoT internet of things
KG knowledge graph
KNN k-nearest neighbour
LSTM long short-term memory
MFCC Mel-frequency cepstrum coefficient
MLM masked language model
NER named entity recognition
NLP natural language processing
NSP next sentence prediction
OWL web ontology language
QA question answering
RDF resource description framework
RNN recurrent neural network
RTRL real-time recurrent learning
SPARQL SPARQL protocol and RDF query language
SQL structured query language
SRM structure risk minimization
SVM support vector machine
URI uniform resource identifier
XML extensible markup language
5 General
Advances in computational approaches are an important driving force in the maturation of AI to become
capable of processing various tasks. Initial AI methods were primarily rules-based and knowledge-
driven. More recently, data-driven methods such as neural networks have gained prominence.
AI computational approaches continue to evolve in industry and academia and are an important
consideration in AI systems.
Computational approaches for AI systems are often categorized based on various criteria. One such
categorization is by the purpose of the AI system. This purpose-based categorization is adapted from
[1]
studies of AI and includes an exemplary categorization of common types.
a) Search methods. These approaches can be further divided into various types of search: classical,
advanced search algorithms, adversarial search and constraint satisfaction.
1) Classical search algorithms solve problems by a search over some state space and can be
divided into uninformed searches and heuristic searches, which apply a rule of thumb to guide
and speed up the search.
© ISO/IEC 2021 – All rights reserved

2) Advanced search algorithms include those that search in a local subspace, those that are
nondeterministic, those that search with partial observation of the search space and online
versions of search algorithms.
3) Adversarial search algorithms search in the presence of an opponent and are generally used in
games. These include notable algorithms such as alpha-beta pruning and also include stochastic
and partially observable variations.
4) Constraint satisfaction problems are solved when each variable in the problem has a value that
satisfies all the constraints.
b) Logics, planning and knowledge. These approaches can be further divided into three cases: logics,
planning and state space search, and knowledge representation.
1) Logics, such as propositional logic and first-order logic, are used in classical AI to represent
knowledge. Problem solution in such computational systems involves inference over the logic
using algorithms such as resolution.
2) Planning in classical AI systems involves search over some state space as well as algorithmic
extensions to deal with planning in the real world. Methods to deal with the complexity of real-
world planning involve time and resource constraints, hierarchical planning where problems
are solved at abstract levels first before fine-grain details, multi-agent systems that handle
uncertainties and dealing with other agents in the system.
3) Knowledge representation is a kind of data structure for describing knowledge using predicate
logic, “if-then” generation and knowledge frame representation.
c) Uncertain knowledge and reasoning. Approaches in this area deal with potentially missing,
uncertain or incomplete knowledge. They generally use either probability or fuzzy logic to represent
concepts. Probabilistic computational systems reason using Bayes rule, Bayesian networks or (in
time-dependent situations) hidden Markov models or Kalman filters. Another set of computational
approaches is used for decision-making, including those based on utility theory and decision
networks.
d) Learning. Computational approaches in this area deal with the problem of making the computer
learn similarly to a human. Approaches can be grouped into learning from examples, knowledge-
based learning, probabilistic learning, reinforcement learning, deep learning approaches, GANs
and other learning approaches.
1) Learning from examples involves supervised learning approaches that learn a machine
learning model from labelled data. It includes methods such as decision trees, linear and
logistic regression approaches, artificial neural networks, non-parametric approaches (e.g. the
KNN), SVMs and ensemble learning methods (e.g. bagging, boosting and variants of random
forest).
2) Knowledge-based learning approaches include logic-based approaches, explanation-based
learning and inductive logic programming.
3) Probabilistic learning involves computational approaches such as Bayesian methods and
expectation-maximization methods.
4) Reinforcement learning involves computational systems that receive feedback, make decisions
and take actions in environments to maximize the overall reward. Notable algorithms include
temporal difference-learning and Q-learning.
5) Deep learning neural approaches involve modern computational approaches with many hidden
layers, including deep feedforward networks, regularisation, modern optimization methods,
CNNs and sequence learning methods such as LSTM networks.
© ISO/IEC 2021 – All rights reserved

6) GANs involve two competing networks, a generator and discriminator. The generator produces
samples and the discriminator classifies each sample as real or fake. After this iterative
process, trained generators can be used in applications such as creating artificial images.
7) Other learning approaches include unsupervised learning, which involves identifying the
natural structure of data sets; semi-supervised learning, which deals with partially labelled
data sets; online learning algorithms, which continue to learn as they receive data; networks
and relational learning, ranking and preference learning, representation learning, transfer
learning and active learning.
e) Inference. These approaches embody the application of an AI system in estimating parameters
or aspects of (or classifying new or unobserved data based on) learned, acquired or defined
parameters. Bayesian inference is the act of taking statistical inference from a Bayesian point of
view. Approximate inferences, such as variational inference, solves the inference problem by taking
the best approximation of the statistics. Monte Carlo algorithms generate samples from a known
distribution that is difficult to normalize, then infer statistics from generated samples. Causal
inference involves inferencing the causal connections of the observed data.
f) Dimensionality reduction. These computational approaches involve reducing the number of
dimensions of data by either dimensionality reduction (feature extraction) algorithms, which
identify a new smaller number of attributes to represent data, or feature selection, which chooses a
subset of the most appropriate attributes.
g) Communicating, perceiving and acting. Computation approaches in these areas are associated
with the fields of NLP (including tasks such as language modelling, text classification, information
retrieval, information extraction, parsing, machine translation and speech recognition), computer
vision (including image processing and object recognition) and robotics.
These categories and subcategories are not mutually exclusive. For instance, deep learning approaches
[d)5)] can be either supervised [d)1)] or unsupervised [d)7)], reinforcement learning [d)4)] can be
achieved through deep learning [d)5)], and approaches for machine translation or object recognition
[g)] can be learning approaches [d)].
ISO/IEC 22989 specifies concepts and terminologies relevant to AI computational approaches.
ISO/IEC 23053 provides a framework for AI systems using machine learning, encompassing machine
learning algorithms, optimization algorithms and machine learning methods. ISO/IEC TR 24030
collects and analyses AI use cases.
6 Main characteristics of AI systems
6.1 General
Not all AI systems are based on machine learning or neural networks. To demonstrate the breadth of AI
systems, some frequently encountered characteristics of AI systems are described in 6.2 and 6.3. These
characteristics are broadly conceptual and not tied to a specific methodology or architecture. In the
aggregate these characteristics differentiate AI systems from non-AI systems.
Some characteristics of AI systems are common and apply widely to different use cases. Others
are specific to a small number of use cases within a specific industry. This clause contains a list of
characteristics of AI systems which is not exhaustive but contains attributes intrinsic to many AI
systems. While the list is not limited to a specific base technology (such as AI systems built with neural
networks), it does not encompass every type of dynamic AI system.
© ISO/IEC 2021 – All rights reserved

6.2 Typical characteristics of AI systems
6.2.1 Adaptable
Some AI systems adapt to different changes in itself and the environment in which it is deployed. Such
adaptation depends on many factors, including data in the system’s domain, architecture or other
technical decisions made at its implementation.
AI systems often operate on server-side cloud computing environments with access to high-performance
computing and other resources. With the growth of IoT systems capable of general-purpose computing
on GPUs and multi-core CPUs, or AI processing on application-specific processors and accelerators, AI
system adaptability now extends to IoT implementation considerations, such as near-real-time data
processing, optimization for low latency and power-efficient performance.
6.2.2 Constructive
Some AI systems construct or generate a static or dynamic output based on specified input criteria.
This applies to methods including unsupervised learning and generative learning.
6.2.3 Coordinated
Some AI systems coordinate between agents. Agents can also be AI systems in their own right, but do
not need to be. Many simultaneous constraints can govern agent behaviour, including static or dynamic
ways. Coordination can be exhibited either explicitly through direct negotiation among the systems or
implicitly through reaction to changes in the environment.
6.2.4 Dynamic
Some AI systems exhibit dynamic decision-making based on external data sources. These data sources
can come from other software platforms, from physical environments or from other sources.
6.2.5 Explainable
Some AI systems provide a mechanism to explain what precipitated a decision or output. This output
can take many forms and can be explicit or implicit with respect to AI system design.
An explainable AI system can contribute to or complement trustworthiness, accuracy and efficiency.
Explainability can also contribute to comparison and optimization of machine learning model
performance by generating insights into factors that degrade performance. Explainability can be an
important counter to deceptive behaviour in AI systems.
6.2.6 Discriminative or generative
Some AI systems are discriminative, designed primarily to distinguish between possible outputs such
as by excluding prior probabilities. Alternatively, some AI systems are generative, designed primarily to
represent relevant aspects of data, such as by including prior probabilities.
6.2.7 Introspective
Some AI systems self-monitor to adapt to their environment or to provide insight into their functionality,
such as in an audit situation. This self-monitoring can be adaptable, situation-dependent or static, and
can take different forms depending on the system architecture.
To support introspective AI systems, performance monitoring functionality collects and reports
performance metrics regarding CPU, GPU or application-specific processor compute resources, memory
and other system resource usage. This information can be used to configure AI system resources, such
as memory allocation, kernel configuration and load balancing across a multi-processor or hybrid
© ISO/IEC 2021 – All rights reserved

hardware system, and enable an AI system to handle parallelization and acceleration for machine
learning model training or inference.
6.2.8 Trained or trainable
Some AI systems are trained on a data set before deployment or trained dynamically (through
adaptation) as the system is used. Systems with these characteristics have numerous possible system
architectures (e.g. neural networks, hidden Markov models).
6.2.9 Accommodating various data
Some AI systems deal with large amounts of heterogeneous data that are structured or unstructured,
static or streaming. AI systems can draw insights from varied data sets to help humans make better
and more accurate decisions.
6.3 Computational characteristics of AI systems
6.3.1 Data-based or knowledge-based
A characteristic of data-based AI computational approaches is that the computational model is trained
on one or more data sources to acquire knowledge.
Considerations for data used in AI systems include acquisition, storage and access.
a) Data acquisition. An AI application’s use case and task typically dictate the type of data to be
acquired for training. Typical AI system tasks reflected in ISO/IEC 22989 and ISO/IEC 23053
include classification, categorization, (conceptual) clustering, regression, prediction, optimization,
NLP (text or speech), perception and system control or behaviour guidance. Depending on the
application and task, AI system developers can collect training data through intelligent hardware
(e.g. smart bracelet, smart watch, smart phone), IoT sensors (e.g. gravity sensor, temperature
sensor, humidity sensor), cameras, microphones or other sensors.
b) Data storage. Collected data are stored in the format and structure consistent with the AI system
application and task. Storage approaches and constraints can differ during training and evaluation.
In addition, distributed and shared storage can be important data storage considerations.
c) Data access. Rapid access and retrieval of large amounts of data is often necessary in AI systems.
Load-balancing techniques are often used to address challenges in data concurrency and network
overloading.
In addition to perception-based tasks and applications, cognitive intelligence has become an important
aspect of AI systems in which cognitive computing is integrated with industrial knowledge. Using
techniques such as NLP and KG, AI systems can reveal implicit knowledge and give insights into
relations, logic or patterns that are not easily found by human observers.
EXAMPLE Using KGs, accumulated business process data can be converted into organizational experience
and knowledge. This can be used in turn to reduce communication costs among different departments.
6.3.2 Infrastructure-based
AI systems can face simultaneous challenges in computing platform design optimization, computing
efficiency in complex heterogeneous environments, highly parallel and scaled computing frameworks,
and the computing performance of AI applications. One possible solution to such challenges is to use
powerful infrastructures to provide computing capability.
Such infrastructures can include sensor, server, network, processor, storage and other elements. Silicon-
based processors are often used for both training and reasoning in learning approaches. When handling
large amounts of training data or complex DNN structures, the training processes typically need to
execute large-scale calculations on multi-processor systems or processor or accelerator clusters.
© ISO/IEC 2021 – All rights reserved

Compared to training, inference is less computationally intensive, but can still require significant
matrix operations. Training and inference have traditionally been implemented on cloud-based servers,
though use cases that demand real-time processing can implement inference on edge devices.
Depending on the technical architectures, silicon-based processors for AI encompass general-purpose
processors (e.g. CPU, GPU and FPGA), semi-customized processors based on FPGA, fully customized
ASIC processors and brain-like computing processors. Vision processing units, deep-learning processing
units, neural network processing units and other application-specific processors are also suitable for
different AI scenarios and functions.
Sensors with microprocessors to collect, process and transmit information can be used to create a full
awareness of the external environment. Large-scale sensor deployment and application can support
data acquisition in AI applications. Further, specialized sensor requirements are needed for smart
home, smart medical and smart security applications. The development of intelligent sensors for AI
applications relies on important factors such as high precision, high reliability, miniaturization and
integration, and high sensitivity.
6.3.3 Algorithm-dependent
ISO/IEC 22989 defines machine learning as a process of optimizing model parameters through
computational techniques, such that the model's behaviour reflects the data or experience. Machine
learning methods find patterns from observed data or samples and use these patterns to make
predictions on input data without being explicitly programmed. Machine learning methods vary in part
based on differences in learning approaches and computational frameworks.
A machine learning method includes three basic elements: loss functions, learning criteria and
optimization algorithm. Differences across machine learning methods can be viewed as functions of
these elements. For example, linear classification methods, such as perceptron, logistic regression and
SVM, differ in terms of learning criteria and optimization algorithms.
a) From a loss function perspective, machine learning methods can be classified as linear or nonlinear.
A robust method needs a small expected risk or error, which uses loss function to quantify the
difference between predicted data and real data. Common loss functions include 0-1 loss function,
quadratic loss function, cross-entropy loss function, hinge loss function, mean absolute error loss
function, Huber loss function, log-cosh loss function and quantile loss function.
Moreover, other broad categories of loss functions include ranking, distribution-based, classification
and regression.
b) Learning criteria of supervised learning includes ERM and SRM. An ERM reduces the average loss
on training data set. An SRM avoids overfitting problems by introducing parameter regularization
based on ERM to limit the model capability. Unsupervised learning has a variety of learning criteria.
For example, maximum likelihood estimate is often used in density estimation, reconstruction
error minimization is often used in unsupervised feature learning.
c) The task of optimization is to find the optimum machine learning model. It consists of parameter
optimization and hyper-parameter optimization. Common optimization algorithms include
gradient descent method, early stopping, batch gradient descent, stochastic gradient descent, mini-
batch gradient descent, gradient descent methods utilizing the coefficient of momentum, root mean
square propagation and adaptive moment optimization.
In the face of massive data processing and complex knowledge reasoning, a large computing task is
typically divided into smaller computing tasks. Such distributed computational frameworks are
based on cloud computing, edge computing and big data technologies. The deep learning framework
is the basic underlying computational framework for deep learning, which generally includes neural
network architectures and a stable deep learning interface to support the distributed learning. Some
frameworks can be transferred to run on multiple platforms such as cloud computing platforms and
mobile devices.
© ISO/IEC 2021 – All rights reserved

6.3.4 Multi-step or end-to-end learning-based
As opposed to multi-step learning, where a problem is divided into multiple stages to be solved step
by step, end-to-end learning seeks to solve problems such that results are directly obtained from input
data.
Machine learning processes often consist of several independent modules. For example, a typical NLP
application includes segmentation, part of speech tagging, syntactic analysis, semantic analysis and
other independent steps. Each step is an individual task, results from each step impact the next step,
potentially affecting the entire training process.
In end-to-end learning, as can be done with deep learning, a prediction result is obtained from the input
to the output. Typically, errors are transferred through back propagation in each layer of the network.
The representation of each layer is adjusted according to such errors until the network is convergent or
achieves the desired performance. In such end-to-end processes, data labelling before each independent
learning task is no longer needed.
Taking speech recognition as an example, in multi-step speech recognition as shown in Figure 1, speech
is converted into speech feature vectors (e.g. the MFCC features), groups of vectors are then classified
into various phonemes using machine learning, the original texts of speech with maximum probability
are finally restored through phonemes. In this process, feature vectors produced by the feature
computing and phonemes are processed by the acoustic model. The acoustic model and language model
are trained separately.
Figure 1 — Multi-step learning-based speech recognition
For end-to-end learning-based speech recognition as shown in Figure 2, the entire process from feature
extraction to phoneme expression can be directly completed by a DNN. Given enough labelled training
data, including pairs of voice data and text data, at the beginning of recognition process, the end-to-end
learning-based speech recognition can show good performance.
Figure 2 — End-to-end learning-based speech recognition
7 Types of AI computational approaches
7.1 General
Computational approaches in AI can be decomposed into knowledge-driven and data-driven.
Knowledge-driven AI computational approaches primarily consist of a series of rule-based methods.
Taking expert system as an example, learning, reasoning and decision-making for a use case are
realized through a set of conceptualized objects and “if-then” logic rules. A large knowledge base
storing extensive knowledge from domain experts is used to support the expert system.
© ISO/IEC 2021 – All rights reserved

By contrast, data-driven AI computational approaches use large amounts of data as the fundamental
resources processed by algorithms to simulate human thinking and decision-making processes. Typical
data-driven computational approaches include machine learning, which can be decomposed as linear or
logistic regression, probabilistic graphical model, decision tree, neural networks and other approaches.
Figure 3 shows this decomposition of AI computational approaches.
Figure 3 — AI computational approaches
7.2 Knowledge-driven approaches
Knowledge-driven approaches mimic the functions of human intelligence from symbols and logic rules.
The human cognitive process is regarded as a symbolic operation process. This kind of approach has
two basic assumptions:
a) information is represented as symbols;
b) symbols are manipulated by explicit rules (such as logical operations).
7.3 Data-driven approaches
Data-driven approaches rely on data in their AI computational model. Various machine learning
approaches are associated with different types of
...