ISO/IEC 23090-8:2020

INTERNATIONAL ISO/IEC
STANDARD 23090-8
First edition
2020-12
Information technology — Coded
representation of immersive media —
Part 8:
Network based media processing
Reference number
©
ISO/IEC 2020
© ISO/IEC 2020
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting
on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address
below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii © ISO/IEC 2020 – All rights reserved

Contents
Foreword . vii
Introduction . viii
1 Scope . 1
2 Normative references . 1
3 Terms, definitions and abbreviated terms . 1
3.1 Terms and definitions . 1
3.2 Abbreviated terms . 4
4 Conventions . 4
5 Overview . 5
5.1 General . 5
5.2 Architecture . 5
5.3 NBMP workflow . 6
5.3.1 General . 6
5.3.2 Workflow processing model . 6
5.3.3 Task allocation and distribution . 8
5.3.4 Workflow graph . 8
5.4 Relationship between logical definitions, data objects and REST resources . 9
5.5 Description of the defined entities in this document . 10
5.5.1 NBMP APIs . 10
5.5.2 Content format . 10
5.5.3 Definitions . 10
5.5.4 Functional behaviour . 11
6 NBMP descriptions . 11
6.1 NBMP function description (FD) . 11
6.1.1 General . 11
6.1.2 Description . 11
6.1.3 Function group . 12
6.2 NBMP task description (TD) . 13
6.2.1 General . 13
6.2.2 Description . 13
6.2.2 Task lifecycle . 14
6.3 NBMP workflow description (WD) . 15
6.3.1 General . 15
6.3.2 Description . 15
6.3.3 Workflow lifecycle . 16
7 NBMP interfaces . 17
7.1 General . 17
7.2 Workflow APIs . 18
7.2.3 Workflow API operations . 18
7.3 Task APIs . 20
7.3.1 General . 20
7.3.2 Task resource . 20
7.3.3 Task API operations . 21
7.4 Function discovery APIs . 22
7.4.1 General . 22
7.4.2 Function discovery queries . 22
7.4.3 Function discovery API operations . 23
7.5 Supported protocols . 25
8 NBMP descriptors . 25
© ISO/IEC 2020 – All rights reserved iii

8.1 Scheme descriptor .25
8.1.1 General .25
8.1.2 JSON schema .26
8.2 General descriptor .27
8.2.1 General .27
8.2.2 JSON schema .28
8.3 Input descriptor .32
8.3.1 General .32
8.3.2 JSON schema .33
8.3.3 General .38
8.3.4 JSON schema .39
8.4 Processing descriptor .43
8.4.1 General .43
8.4.2 JSON schema .46
8.5 Requirements descriptor .53
8.5.1 General .53
8.5.2 JSON schema .55
8.6 Configuration descriptor .59
8.6.1 General .59
8.6.2 JSON schema .60
8.7 Startup-delay descriptor.63
8.7.1 General .63
8.7.2 JSON schema .64
8.8 Client-Assistance descriptor .64
8.8.1 General .64
8.8.2 JSON schema .65
8.9 Failover descriptor .66
8.9.1 General .66
8.9.2 JSON schema .66
8.10 Events descriptor .67
8.10.1 General .67
8.10.2 JSON schema .68
8.11 Variables descriptor.68
8.11.1 General .68
8.11.2 JSON schema .69
8.12 Monitoring descriptor .70
8.12.1 General .70
8.12.2 JSON schema .70
8.13 Reporting descriptor .71
8.13.1 General .71
8.13.2 JSON schema .72
8.14 Notification descriptor .74
8.14.1 General .74
8.14.2 JSON schema .74
8.15 Assertion descriptor .76
8.15.1 General .76
8.15.2 JSON schema .78
8.16 Request Descriptor .81
8.16.1 General .81
8.16.2 JSON schema .81
8.17 Acknowledge descriptor .82
8.17.1 General .82
8.17.2 JSON schema .82
8.18 Repository descriptor.83
8.18.1 General .83
iv © ISO/IEC 2020 – All rights reserved

8.18.2 JSON schema . 84
8.19 Security descriptor . 85
8.19.1 General . 85
8.19.1 JSON schema . 86
8.20 Step descriptor . 87
8.20.1 General . 87
8.20.2 JSON schema . 87
9 NBMP parameters . 88
9.1 General . 88
9.2 Scheme descriptor parameters . 88
9.3 General descriptor parameters . 89
9.4 Input descriptor parameters . 90
9.5 Output descriptor parameters . 92
9.6 Processing descriptor parameters . 94
9.7 Requirements descriptor parameters . 95
9.7.1 Flow control parameters . 95
9.7.2 Hardware parameters . 95
9.7.3 Security requirements . 96
9.7.4 Workflow/task requirements . 97
9.7.5 Resource estimator parameters . 97
9.8 Startup-Delay descriptor parameters . 97
9.9 Client-Assistant parameters . 98
9.10 Failover parameters . 98
9.11 Events parameters . 99
9.12 Variables parameters . 99
9.13 Monitoring parameters . 100
9.14 Reporting parameters. 100
9.15 Notification parameters . 101
9.16 Assertion parameters . 101
9.17 Request parameters . 103
9.18 Acknowledge parameters . 103
9.19 Repository parameters . 104
9.20 Security parameters . 104
9.21 Step Descriptor parameters . 105
9.22 Configuration descriptor parameters . 106
9.22.1 Generic parameter representation . 106
9.22.2 Example of parameter representation . 107
10 Workflow manager, task and function repository requirements . 110
10.1 Workflow manager requirements . 110
10.2 Function repository requirements. 111
10.3 Task requirements . 111
11 NBMP support for media formats and metadata . 112
11.1 General . 112
11.2 Media formats . 112
11.3 Application formats . 112
11.4 Metadata formats . 112
12 Security considerations in NBMP . 112
12.1 Overview . 112
12.2 Secure and authenticated channels between NBMP source and NBMP workflow
manager . 113
12.2.1 General . 113
12.2.2 Secure communication channel between NBMP source and NBMP workflow manager
................................................................................................................................................................ 113
© ISO/IEC 2020 – All rights reserved v

12.2.3 NBMP source authentication to workflow manager . 113
12.2.4 Workflow manager authentication to NBMP source . 113
12.2.5 Secure channels for task communication . 113
12.2.6 NBMP source authentication/authorization to workflow task . 114
12.2.7 Workflow task authentication to NBMP source . 114
12.2.8 Secure channel for NBMP source and task communication . 114
12.2.9 MPE security . 114
12.2.10 Network security . 114
Annex A (normative) JSON schemas . 115
Annex B (normative) NBMP workflow management . 116
Annex C (informative) Schema for identifying MPEG compatible functions . 119
Annex D (normative) NBMP MIME types . 120
Annex E (informative) Interface for managing function descriptions in function repository
................................................................................................................................................................. 123
Bibliography . 124

vi © ISO/IEC 2020 – All rights reserved

Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are
members of ISO or IEC participate in the development of International Standards through technical
committees established by the respective organization to deal with particular fields of technical activity.
ISO and IEC technical committees collaborate in fields of mutual interest. Other international
organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the
work.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for the
different types of document should be noted. This document was drafted in accordance with the editorial
rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject of
patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent rights. Details
of any patent rights identified during the development of the document will be in the Introduction and/or
) or the IEC list of patent
on the ISO list of patent declarations received (see www.iso.org/patents
declarations received (see http://patents.iec.ch).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to the World
Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT)
see www.iso.org/iso/foreword.html.
This document was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 29, Coding of audio, picture, multimedia and hypermedia information.
A list of all parts in the ISO/IEC 23090 series can be found on the ISO website.
Any feedback or questions on this document should be directed to the user’s national standards body. A
complete listing of these bodies can be found at www.iso.org/members.html.
© ISO/IEC 2020 – All rights reserved vii

Introduction
This document defines a framework that enables initializing and controlling media processing in a
network. A network-based media processing (NBMP) source describes the requested media processing
and provides information about the nature and format of the media data. Based on that, an NBMP
workflow manager establishes the media processing workflow and informs the NBMP source that the
workflow is ready, and that media processing can start. The media source(s) can then start transmitting
their media to the network for processing.
An NBMP workflow can be understood as a connected graph of media processing tasks, each of which
performs a well-defined media processing operation. The workflow manager ensures the correct
operation of the workflow by configuring and monitoring each task as well as the workflow output. The
workflow manager is responsible for the selection of the media processing functions and instantiating
them as tasks based on the workflow description that is received from the NBMP source.
NBMP abstracts the underlying computing platform interactions to establish, load, instantiate and
monitor the media processing entities that will run the media processing tasks. NBMP defines application
programming interfaces (APIs) between an NBMP source and workflow manager; workflow manager and
task(s); and an API to discover appropriate function(s). NBMP is media format and protocol agnostic.
However, it identifies and signals the media, metadata and auxiliary information formats for data
exchanged between media source, the workflow manager and tasks.
Annex C provides schema for identifying MPEG compatible functions.
Annex E provides an interface for managing function descriptions in function repository.
viii © ISO/IEC 2020 – All rights reserved

Information technology — Coded representation of
immersive media —
Part 8:
Network-based media processing
1 Scope
The network-based media processing (NBMP) framework defines the interfaces including both data
formats and application programming interfaces (APIs) among the entities connected through digital
networks for media processing. Users can access and configure their operations remotely for efficient,
intelligent processing. This document describes and manages workflows to be applied to the media data.
This process includes uploading of media data to the network, instantiation of the media processing tasks,
and configuration of the tasks. The framework enables dynamic creation of media processing pipelines,
as well as access to processed media data and metadata in real-time or in a deferred way. The media and
metadata formats used between the media source, workflow manager and media processing entities in a
media processing pipeline are also specified.
2 Normative references
The following documents are referred to in the text in such a way that some or all of their content
constitutes requirements of this document. For dated references, only the edition cited applies. For
undated references, the latest edition of the referenced document (including any amendments) applies.
ISO/IEC 9899, Information technology — Programming languages — C
ISO/IEC 23001–7, Information technology — MPEG systems technologies — Part 7: Common encryption
in ISO base media file format files
IETF RFC 3339:2002, Date and Time on the Internet: Timestamps, https://tools.ietf.org/html/rfc3339
IETF RFC 3986:2005, Uniform Resource Identifier (URI): Generic Syntax,
https://tools.ietf.org/html/rfc3986
IETF RFC 7231, Hypertext Transfer Protocol (HTTP/1.1): Semantics and Content,
https://tools.ietf.org/html/rfc7231
IETF RFC 6381:2011, The ′Codecs′ and ′Profiles′ Parameters for “Bucket” Media Types
3 Terms, definitions and abbreviated terms
3.1 Terms and definitions
For the purposes of this document, the following terms and definitions apply.

ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— ISO Online browsing platform: available at https://www.iso.org/obp
— IEC Electropedia: available at http://www.electropedia.org/
© ISO/IEC 2020 – All rights reserved 1

3.1.1
function description
FD
logical description of the details of a NBMP function including input and output description, media
processing, requirements, etc.
3.1.2
function description document
FDD
document containing function object
3.1.3
function object
FO
function description in JSON format
3.1.4
function repository
storage place where NBMP function description are retrieved from by an NBMP workflow manager or
NBMP source
3.1.5
function resource
FR
REST resource identified with a URL and containing a function description document
3.1.6
media processing entity
MPE
entity that runs one or more media processing task(s)
3.1.7
media sink
entity that consumes the output of the NBMP workflow through existing delivery methods
3.1.8
media source
entity that provides media content to the NBMP workflow to be processed
3.1.9
NBMP function
implementation of a standalone and self-contained media processing operation and the corresponding
description of that operation
3.1.10
NBMP descriptor
a group of NBMP parameters which describe a set of related characteristics of workflow, function or
task
3.1.11
NBMP parameter
variable expressing a characteristic of workflow, function or task
3.1.12
NBMP source
entity that describes media processing in the network
3.1.13
NBMP system
system for processing media across one or more processing entities in the network and consisting of
media source (s), a NBMP source, a NBMP workflow manager, a function repository, media processing
entity(ies) and media sink(s)
2 © ISO/IEC 2020 – All rights reserved

3.1.14
NBMP workflow
graph of one or more connected task(s) that achieve the requested media processing
3.1.15
NBMP workflow manager
entity that provisions tasks and connects them to create, control, manage and monitor a complete
NBMP workflow based on a workflow description document
3.1.16
port
logic input and output endpoints by specifying where the data comes in and out
3.1.17
port mapping
PM
data structure used to make references for NBMP function ports to the workflow input and output
streams, especially to dynamic inputs and outputs
3.1.18
stream ID
unique string for identifying an input or output stream of workflow/function/task
3.1.19
supplementary information
metadata or auxiliary information related to the media data
3.1.20
task
runtime instance of NBMP function that gets executed inside a media processing entity
3.1.21
task description
logical description of the runtime details of a task, including input and output, requirements,
configuration etc.
3.1.22
task description document
TDD
document containing task description object
3.1.23
task object
TO
task description in JSON format
3.1.24
task resource
TR
REST resource identified with an URL and containing task description document
3.1.25
workflow description
logical description of the details of the media processing including input and output description details,
requested media processing, requirements etc.
3.1.26
workflow description document
WDD
document containing workflow description object
© ISO/IEC 2020 – All rights reserved 3

3.1.27
workflow object
WO
workflow description in JSON format
3.1.28
workflow resource
WR
REST resource identified with an URL and containing workflow description document
3.2 Abbreviated terms
API  application programming interface
CPU  central processing unit
DAG directed acyclic graph
DASH dynamic adaptive streaming over HTTP
GPU  graphics processing unit
HTTP hyper-text transfer protocol
JSON JavaScript object notation
MMT MPEG media transport
NBMP network-based media processing
NVP name value pair
PCC  point-cloud compression
RTP  real-time transport protocol
TCP  transmission control protocol
UDP user datagram protocol
URI  uniform resource identifier
URL  uniform resource locator
URN uniform resource name
XML eXtensible Markup Language
4 Conventions
The following naming convention apply in this document:
— Names comply to dash-case convention, i.e. words in a name are separated with ‘-'.
4 © ISO/IEC 2020 – All rights reserved

— Operations, resources, documents, descriptions and descriptors are identified by an upper-case
first letter. In these names, words after ‘-' start with an upper-case letter. All other letters are
lower-case.
— Parameters and their values are identified by lower-case letters. No uppercase letter is used in
these names.
— Parameter values are identified by ‘’, e.g. ‘value’.
— JSON objects comply to dash-case convention with all lower-case letters.
The following legends are used in tables:
— cardinality: 1 = exactly one, 0-1 = zero or one, 0-N = zero or more, 1-N = one or more
— P: parameter
— O: object
— N/A: not applicable
The range of “unsigned integer” is 0 to 2 -1.
5 Overview
5.1 General
The network-based media processing (NBMP) framework enables the creators, service providers and
consumers of digital media to describe media processing operations that are to be performed by the
media processing entities in a network, as shown in Figure 1. It provides a method to describe a workflow
by composing a set of media processing functions that are accessible through NBMP application
programming interfaces (APIs). A media processing entity (MPE) runs processing tasks applied on the
media data and the related metadata received from media sources or other tasks. MPE provides
capabilities for configuring, managing and monitoring processing tasks. A media processing task is a
process applied to media and metadata input(s), producing media data and related metadata output(s)
to be consumed by a media sink or other media processing tasks.
The NBMP framework is media format agnostic and supports any format of media content, including the
existing MPEG codecs and MPEG formats such as ISO/IEC 13818-1 (Reference [10]), ISO/IEC 14496-12,
ISO/IEC 23008-1 (Reference [12]) and ISO/IEC 23009-1 (Reference [13]).
The NBMP framework supports the delivery over IP-based networks using common transport protocols
such as TCP, UDP, RTP (Reference [17]) and HTTP.
The NBMP framework also support the existing delivery methods such as streaming, file delivery, push-
based progressive download, hybrid delivery, multipath and heterogeneous network environments.
5.2 Architecture
NBMP specifies interfaces to create and control media processing workflows in the network. NBMP can
be split into a control plane and a media plane. The control plane covers the following APIs:
― Workflow API is used by NBMP source to create and control a media processing workflow.
© ISO/IEC 2020 – All rights reserved 5

― Function discovery API provides the means for workflow manager and/or NBMP source to
discover media processing Functions that can be loaded as part of a media processing workflow.
― Task API is used by the workflow manager to configure and monitor a task at runtime.
On the media plane, NBMP defines the media formats, the metadata, and the supplementary information
formats between the NBMP source and the task, as well as between the tasks themselves.
The discovery of the NBMP workflow manager and function repository is out of scope.
The NBMP architecture is shown in Figure 1.
Function Function
discovery API description
NBMP workflow Function
NBMP workflow
Function
API discovery API
NBMP source
manager
repository
(Builds workflow/DAG, allocates tasks,
Workflow
Function description
runtime configuration/stream binding)
description
NBMP
Task configuration,
reporting the current task status
Task
API
Media processing entity (MPE)
MPE
Task 1
configuration
Media flow
Media flow
Media source Task 2
Media sink
media
processing
Runtime configuration/
stream/event binding
control flow
data flow
Figure 1 — NBMP reference architecture
5.3 NBMP workflow
5.3.1 General
The workflow manager receives a workflow description document from a NBMP source and builds a
workflow for requested media processing. This subclause describes how media processing functions are
selected, and then the corresponding tasks are configured and distributed to a set of media processing
entities as part of the workflow procedure.
5.3.2 Workflow processing model
This subclause describes the detailed processing model of the workflow manager.
Since the set of functions that are provided by a function repository can be read by an NBMP source, the
workflow description document can be composed in two different ways, based on use cases and actual
needs.
6 © ISO/IEC 2020 – All rights reserved

a) The NBMP source requests the creation of a workflow using a set of functions in the function
repository: with this type of workflow creation request, the NBMP source is responsible for
the selection of functions that are included in the workflow. In this case, the NBMP source
requests the creation of the workflow:
1) using a description of tasks by which the workflow is to be created;
2) specifying the connection map to define the connections of the inputs and outputs of tasks.
The details of the connection map can be found in subclause 8.5. When the workflow
manager receives the above information from NBMP source, it instantiates the tasks
based on function names and connects the tasks according to what is defined in the
connection map.
b) The NBMP source requests the creation of a workflow using a set of keywords by which the
workflow manager constructs the workflow. In this case, the NBMP source may not be aware
of a set of functions to be inserted into the workflow, and requests the creation of the
workflow:
1) using a set of keywords by which the workflow manager finds the appropriate functions;
2) specifying the requirements of the workflow using descriptors from Table 4 in the
workflow description docu
...