ISO/IEC 23006-2:2016

INTERNATIONAL ISO/IEC
STANDARD 23006-2
Third edition
2016-12-01
Information technology — Multimedia
service platform technologies —
Part 2:
MPEG extensible middleware (MXM)
API
Technologies de l’information — Technologies de la plate-forme de
services multimédia —
Partie 2: Intergiciel MPEG extensible (MXM) API
Reference number
©
ISO/IEC 2016
© ISO/IEC 2016, Published in Switzerland
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized otherwise in any form
or by any means, electronic or mechanical, including photocopying, or posting on the internet or an intranet, without prior
written permission. Permission can be requested from either ISO at the address below or ISO’s member body in the country of
the requester.
ISO copyright office
Ch. de Blandonnet 8 • CP 401
CH-1214 Vernier, Geneva, Switzerland
Tel. +41 22 749 01 11
Fax +41 22 749 09 47
copyright@iso.org
www.iso.org
ii © ISO/IEC 2016 – All rights reserved

Contents Page
Foreword .v
Introduction .vi
1 Scope . 1
2 Normative references . 1
3 Terms, definitions and abbreviated terms . 1
3.1 Terms and definitions . 1
3.2 Abbreviated terms . 3
4 Namespace conventions . 4
5 Common MXM interfaces and classes . 7
6 MPEG-M Engines . 8
6.1 General . 8
6.2 Technology Engines . 9
6.2.1 Digital Item Engine . 9
6.2.2 MPEG-21 File Format Engine . 9
6.2.3 REL Engine .10
6.2.4 IPMP Engine .11
6.2.5 GreenMetadata Engine .11
6.2.6 Media Framework Engine .12
6.2.7 Metadata Engine .16
6.2.8 Event Reporting Engine .16
6.2.9 Security Engine .17
6.2.10 Search Engine .18
6.2.11 Contract Engine.20
6.2.12 Overlay Engine .20
6.2.13 Sensory Effect Engine .21
6.2.14 Compact Descriptor for Visual Search Technology Engine .22
6.3 Protocol Engines .23
6.3.1 General.23
6.3.2 Base Protocol Engine .23
6.3.3 Authenticate Services APIs .25
6.3.4 Authorize Services APIs .25
6.3.5 Check With Services APIs .25
6.3.6 Create Services APIs .26
6.3.7 Deliver Services APIs . .26
6.3.8 Describe Services APIs . . .26
6.3.9 Identify Services APIs .26
6.3.10 Negotiate Services APIs .27
6.3.11 Package Services APIs .27
6.3.12 Post Services APIs .27
6.3.13 Present Services APIs .27
6.3.14 Process Services APIs .27
6.3.15 Request Services APIs .27
6.3.16 Revoke Services APIs .28
6.3.17 Search Services APIs .28
6.3.18 Store Services APIs.29
6.3.19 Transact Services APIs .29
6.3.20 Verify Services APIs .29
6.4 MXM Orchestrator APIs .30
6.4.1 General.30
6.4.2 DID Engine Orchestrator APIs .30
6.4.3 Identify Content Engine Orchestrator APIs .30
6.4.4 Identify User Engine Orchestrator APIs .30
© ISO/IEC 2016 – All rights reserved iii

6.4.5 MF Orchestrator Engine APIs .30
Annex A (normative) MXM Configuration .31
Bibliography .39
iv © ISO/IEC 2016 – All rights reserved

Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are
members of ISO or IEC participate in the development of International Standards through technical
committees established by the respective organization to deal with particular fields of technical
activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international
organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the
work. In the field of information technology, ISO and IEC have established a joint technical committee,
ISO/IEC JTC 1.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular the different approval criteria needed for the
different types of ISO documents should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject of
patent rights. ISO shall not be held responsible for identifying any or all such patent rights. Details of
any patent rights identified during the development of the document will be in the Introduction and/or
on the ISO list of patent declarations received (see www.iso.org/patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation on the meaning of ISO specific terms and expressions related to conformity assessment,
as well as information about ISO’s adherence to the World Trade Organization (WTO) principles in the
Technical Barriers to Trade (TBT) see the following URL: www.iso.org/iso/foreword.html.
The committee responsible for this document is ISO/IEC JTC 1, Information technology, SC 29, Coding of
audio, picture, multimedia and hypermedia information.
This third edition cancels and replaces the second edition (ISO/IEC 23006-2:2013), which has been
technically revised.
A list of all parts in the ISO/IEC 23006 series can be found on the ISO website.
© ISO/IEC 2016 – All rights reserved v

Introduction
The ISO/IEC 23006 series is a suite of standards that has been developed for the purpose of enabling the
easy design and implementation of media-handling value chains whose devices interoperate because
they are all based on the same set of technologies, especially MPEG technologies, accessible from the
middleware APIs, elementary services and aggregated services.
The ISO/IEC 23006 series is referred to as MPEG Extensible Middleware (MXM) in its first edition, and
it specifies an architecture (ISO/IEC 23006-1), an API (ISO/IEC 23006-2), a conformance and reference
software (ISO/IEC 23006-3) and a set of protocols which MXM Devices had to adhere (ISO/IEC 23006-
4). It specifies also how to combine elementary services into aggregated services (ISO/IEC 23006-5).
The ISO/IEC 23006 series is subdivided into five parts:
Part 1 — Architecture: specifies the architecture that can be used as a guide to an MPEG-M
implementation;
Part 2 — MPEG Extensible Middleware (MXM) Application Programming Interface (APIs) (this
document): specifies the middleware APIs;
Part 3 — Conformance and Reference Software: specifies conformance criteria and a reference software
implementation with a normative value;
Part 4 — Elementary Services: specifies elementary service protocols between MPEG-M applications;
Part 5 — Service Aggregation: specifies mechanisms enabling the combination of Elementary Services
and other services to build Aggregated Services.
vi © ISO/IEC 2016 – All rights reserved

INTERNATIONAL STANDARD ISO/IEC 23006-2:2016(E)
Information technology — Multimedia service platform
technologies —
Part 2:
MPEG extensible middleware (MXM) API
1 Scope
This document specifies a set of Application Programming Interfaces (called for short MXM APIs) so that
MPEG-M Applications running on an MPEG-M Device can access the standard multimedia technologies
contained in its Middleware as MPEG-M Engines, as specified by ISO/IEC 23006-1.
The MXM APIs belong to two classes:
— the MPEG-M Engine APIs, i.e. the collection of the individual MPEG-M Engine APIs providing access
to a single MPEG technology (e.g. video coding) or to a group of MPEG technologies where this is
convenient;
— the MPEG-M Orchestrator API, i.e. the API of the special MPEG-M Engine (called Orchestrator Engine)
that is capable of creating chains of MPEG-M Engines to execute high-level application calls such as
“Play a video”, as opposed to the typically low-level MPEG-M Engine API calls.
2 Normative references
There are no normative references in this document.
3 Terms, definitions and abbreviated terms
3.1 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— IEC Electropedia: available at http://www.electropedia.org/
— ISO Online browsing platform: available at http://www.iso.org/obp
3.1.1
Aggregated Service
service resulting from the combination of Elementary Services (3.1.2)
3.1.2
Elementary Service
basic unit of service (3.1.13)
3.1.3
content
Digital Item and its component elements, namely resources (e.g. media, scripts, executable), identifiers,
descriptions (e.g. metadata) and event (3.1.6) reports
© ISO/IEC 2016 – All rights reserved 1

3.1.4
contract
set of metadata, licenses (3.1.8), promises and signers agreed by users (3.1.15) of a multimedia value
chain (3.1.16), where a promise is a signed collection of statements about, e.g. obligations, prohibitions
and assertions, and a signer is a user whose signature makes the contract valid
3.1.5
device
hardware/software or simply software apparatus that enables a user (3.1.15) to play a role (3.1.12) in
multimedia value chains (3.1.16)
3.1.6
event
performance of a specified set of functions or operations
3.1.7
entity
one of the following elements in the multimedia value chain (3.1.16): content (3.1.3), contract (3.1.4),
device (3.1.5), event (3.1.6), license (3.1.8), service (3.1.13), and user (3.1.15)
3.1.8
license
collection of authorizations, conditions and payment terms granted by a user (3.1.15) to other users
3.1.9
protocol
set of rules and data format used by two devices (3.1.5) to communicate
3.1.10
resource
individually identifiable asset or a sequence of assets
EXAMPLE Video or audio clip, a 3D synthetic scene, an image, a textual asset, a 2D LASeR scene, a web page,
a single program or a full 24-hour programming of a TV broadcast, a script or executable, etc.
3.1.11
right
ability of a user (3.1.15) to perform an operation in the multimedia value chain (3.1.16)
3.1.12
role
ability of a user (3.1.15) to perform a set of operations in the multimedia value chain (3.1.16)
3.1.13
service
operation performed on an entity (3.1.7) by a user (3.1.15) on behalf of other users
3.1.14
service provider
user (3.1.15) offering services (3.1.13) to other users
3.1.15
user
participant in multimedia value chains (3.1.16)
3.1.16
value chain
collection of users (3.1.15), including creators, end users and service providers (3.1.14), that conform to
this document
2 © ISO/IEC 2016 – All rights reserved

3.1.17
MPEG-M Application
application that runs on an MPEG-M device (3.1.18) and makes calls to the MPEG-M Application API and
MPEG-M Engine APIs (3.1.20)
3.1.18
MPEG-M Device
device (3.1.5) equipped with a selected set of MPEG-M Engines (3.1.19)
3.1.19
MPEG-M Engine
collection of specific technologies that are bundled together to provide a specific functionality that is
needed by MPEG-M Applications (3.1.17)
3.1.20
MPEG-M Engine API
API of a single MPEG-M Engine (3.1.19)
3.1.21
MPEG-M orchestrator API
API of the MPEG-M Orchestrator Engine (3.1.22)
3.1.22
MPEG-M Orchestrator Engine
special MPEG-M Engine (3.1.19) capable of creating chains of MPEG-M Engines
Note 1 to entry: It is also to set-up a sequence of connected MPEG-M Engines for the purpose of executing a high-
level application call such as Play.
3.1.23
MPEG-M Technology
technology that is required to implement an MPEG-M functionality
3.2 Abbreviated terms
AIT Advanced IPTV Terminal
AS Aggregated Service
BBL Bitstream Binding Language
BPMN Business Process Model and Notation
CEL Contract Expression Language
DI Digital Item
DIA Digital Item Adaptation
DID Digital Item Declaration
DIDL Digital Item Declaration Language
DII Digital Item Identification
DIS Digital Item Streaming
ER Event Report
ERR Event Report Request
© ISO/IEC 2016 – All rights reserved 3

ES Elementary Service
IPMP Intellectual Property Management and Protection
IPTV Internet Protocol Television
MDS Multimedia Description Schemes
MPEG Moving Picture Experts Group
MPEG-21 Multimedia Framework [see ISO/IEC 21000 (all parts)]
MPEG-A Multimedia Application Format [see ISO/IEC 23000 (all parts)]
MPEG-M Multimedia Service Platform Technologies [see ISO/IEC 23006 (all parts)]
MPEG-V Multimedia Context and Control [see ISO/IEC 23005 (all parts)]
MPQF MPEG Query Format
REL Rights Expression Language
RTP Real Time Protocol
RTSP Real Time Streaming Protocol
SE Sensory Effect
SEM Sensory Effect Metadata
URI Uniform Resource Identifier
URL Uniform Resource Locator
WSDL Web Services Description Language
XML Extensible Markup Language
XSD XML Schema Definition
XSLT Extensible Stylesheet Language Transformations
4 Namespace conventions
For clarity, throughout this document, consistent namespace prefixes are used.
“xml:” and “xmlns:” are normative prefixes defined in W3C XMLNAMES. The prefix “xml:” is by
definition bound to “http://www.w3.org/XML/1998/namespace”. The prefix “xmlns:” is used only
for namespace bindings and is not itself bound to any namespace name.
“xsi:” prefix is not normative. It is a naming convention in this document to refer to an element of the
http://www.w3.org/2001/XMLSchema-instance namespace. All other prefixes used in either the text
or examples of this document are not normative, e.g. “mpegm:”, “dia:”.
In particular, most of the informative examples in this document are provided as XML fragments
without the normally required XML document declaration and, thus, miss a correct namespace binding
context declaration.
Unless specified otherwise, all unqualified descriptions fragments assume the default namespace
“urn:mpeg:mpegM:schema:02-service-NS:2012”.
4 © ISO/IEC 2016 – All rights reserved

In these descriptions fragments the different prefixes are bound to the namespaces as given in Table 1.
The schema locations of the namespaces in Table 1 are only an informative indication as schema
locations may change over time.
Table 1 — Mapping of prefixes to namespaces used in examples and text
Prefix Corresponding namespace Schema location
mpegm urn:mpeg:mpegM:schema:02-service-NS:2011
mpegmb urn:mpeg:mpegM:schema:01-base-NS:2011
dia urn:mpeg:mpeg21:2003:01-DIA-NS
http://standards.
iso.org/ittf/Public-
lyAvailableStand-
ards/MPEG-21_sche-
ma_files/dia-2nd/
UED-2nd.xsd
erl urn:mpeg:mpeg21:2005:01-ERL-NS
http://standards.
iso.org/ittf/Public-
lyAvailableStand-
ards/MPEG-21_sche-
ma_files/er/er.xsd
fru urn:mpeg:mpegB:schema:FragmentRequestUnits:2007 Defined in ISO/
IEC 23001-2:2008
mpeg7 urn:mpeg:mpeg7:schema:2004
mpeg7s urn:mpeg:mpeg7:systems:2001
cel urn:mpeg:mpeg21:cel:contract:2011
bbl urn:mpeg:mpeg21:2007:01-BBL-NS
http://standards.
iso.org/ittf/Public-
lyAvailableStand-
ards/MPEG-21_sche-
ma_files/dis/bbl.xsd
dii urn:mpeg:mpeg21:2002:01-DII-NS
mpqf urn:mpeg:mpqf:schema:2008 Defined in ISO/
IEC 15938-12
mpeg4ipmp urn:mpeg:mpeg4:IPMPSchema:2002 Defined in ISO/
IEC 14496-
13:2004
ipmpdidl urn:mpeg:mpeg21:2004:01-IPMPDIDL-NS
http://standards.
iso.org/ittf/Public-
lyAvailableStand-
ards/MPEG-21_sche-
ma_files/ipmp/ipmp-
didl.xsd
ipmpmsg urn:mpeg:mpegB:schema:IPMP-XML-MESSAGES:2007 Defined in ISO/
IEC 23001-3:2008
ipmpinfo urn:mpeg:mpeg21:2004:01-IPMPINFO-NS http://standards.
iso.org/ittf/Public-
lyAvailableStand-
ards/MPEG-21_sche-
ma_files/ipmp/ipmp-
general.xsd
didl urn:mpeg:mpeg21:2002:02-DIDL-NS
http://standards.
iso.org/ittf/Public-
lyAvailableStand-
ards/MPEG-21_sche-
ma_files/did/didl.xsd
mpegm-didl urn:mpeg:mpegM:schema:06-didl-NS:2012
© ISO/IEC 2016 – All rights reserved 5

Table 1 (continued)
Prefix Corresponding namespace Schema location
didmodel urn:mpeg:mpeg21:2002:02-DIDMODEL-NS
http://standards.iso.
org/ittf/PubliclyAvail-
ableStandards/MPEG-
21_schema_files/did/
didmodel.xsd
didl-msx urn:mpeg:maf:schema:mediastreaming:DIDLextensions Defined in ISO/
IEC 23000-5:2011
dii urn:mpeg:mpeg21:2002:01-DII-NS
http://standards.
iso.org/ittf/Public-
lyAvailableStand-
ards/MPEG-21_sche-
ma_files/dii/dii.xsd
rel-r urn:mpeg:mpeg21:2003:01-REL-R-NS
http://standards.
iso.org/ittf/Public-
lyAvailableStandards/
MPEG-21_schema_
files/rel-r/rel-r.xsd
rel-sx urn:mpeg:mpeg21:2003:01-REL-SX-NS
http://standards.
iso.org/ittf/Public-
lyAvailableStandards/
MPEG-21_schema_
files/rel-r/rel-sx.xsd
xsd
http://www.w3.org/2001/XMLSchema http://www.
w3.org/2001/
XMLSchema.xsd
xsi http://www.w3.org/2001/XMLSchema-instance
dsig
http://www.w3.org/2000/09/xmldsig# http://www.w3.org/
TR/2002/REC-xm-
ldsig-core-20020212/
xmldsig-core-schema.
xsd
xenc
http://www.w3.org/2001/04/xmlenc#
bpmn
http://www.omg.org/spec/BPMN/20100524/MODEL http://www.
omg.org/spec/
BPMN/20100501/
BPMN20.xsd
bpmnext1 urn:mpeg:mpegM:schema:04-bpmn-ext-mfr-NS:2012
ca urn:mpeg:mpegM:service-type:04-content-adapta-
tion-NS:2012
cel urn:mpeg:mpeg21:2010:01-CEL-NS
cidl urn:mpeg:mpeg-v:2010:01-CIDL-NS
dc
http://purl.org/dc/elements/1.1/ http://dublincore.
org/schemas/xmls/
qdc/2008/02/11/
dc.xsd
dcdv urn:mpeg:mpeg-v:2010:01-DCDV-NS
ebucore urn:ebu:metadata-schema:ebuCore_2010
http://www.ebu.
ch/metadata/
schemas/EBU-
Core/20100820/EBU_
CORE_20100820.xsd
esi urn:mpeg:mpegM:service-type:03-extract-sensory-in-
formation-NS:2012
6 © ISO/IEC 2016 – All rights reserved

Table 1 (continued)
Prefix Corresponding namespace Schema location
etsi urn:dvb:metadata:iptv:sdns:2008-1 Defined in ETSI
[29]
TS 102 034
ipmpinfo-msx urn:mpeg:maf:Schema:mediastreaming:IPMPINFOexten- Defined in ISO/
sions:2007 IEC 23000-5:2011
rs urn:mpeg:mpegM:service-type:01-recog-
nize-speech-NS:2012
saml urn:oasis:names:tc:SAML:2.0:assertion
http://docs.oa-
sis-open.org/se-
curity/saml/v2.0/
saml-schema-asser-
tion-2.0.xsd
samlp urn:oasis:names:tc:SAML:2.0:protocol
http://docs.oa-
sis-open.org/se-
curity/saml/v2.0/
saml-schema-proto-
col-2.0.xsd
sedl urn:mpeg:mpeg-v:2010:01-SEDL-NS
sepv urn:mpeg:mpeg-v:2010:01-SEPV-NS
sev urn:mpeg:mpeg-v:2010:01-SEV-NS
sid urn:mpeg:mpegM:schema:05-sid-NS:2012
ss urn:mpeg:mpegM:service-type:02-synthe-
size-speech-NS:2012
tva urn:tva:metadata:2010 Defined in ETSI
[30]
TS 102 822-3-1
wsdl
http://www.w3.org/ns/wsdl http://www.
w3.org/2002/ws/
desc/ns/wsdl20.xsd
Xsl
http://www.w3.org/1999/XSL/Transform http://www.
w3.org/2007/sche-
ma-for-xslt20.xsd
xhtml
http://www.w3.org/1999/xhtml http://www.w3.org/
MarkUp/SCHEMA/
xhtml11.xsd
5 Common MXM interfaces and classes
The MXM APIs are specified in the Java languages, as follows:
a) a high-level description of the interfaces defining the MXM APIs, provided in the current text;
b) an html format specification of the MXM APIs with a normative value provided as an attachment to
[6]
this document, provided in an attachment or on the official MPEG-M website .
The core part of MXM consists of a number of interfaces and classes which are common to all MPEG-M
Engines. These are:
— MXM: the main class that enables MPEG-M Applications to obtain instances of MPEG-M Engines.
It acts as a factory instantiating the MPEG-M Engines listed in a configuration file (so-called
MXM Configuration File). The MXM Configuration File contains the list of MPEG-M Engines that
are required by an MPEG-M Application; hence, each MPEG-M Application needs to have an MXM
Configuration File (the format of the MXM Configuration file is depicted in Annex A);
© ISO/IEC 2016 – All rights reserved 7

— MXMAbstractEngine: the abstract class at the highest level of an MPEG-M Engine. Every MPEG-M
Engines has to extend this abstract class;
— MXMObject: the basic interface for most of the MXM classes. MXMObject is a wrapper of any
objects exchanged by MPEG-M Engines. It defines the version of MXM and it provides a number
of fundamental methods used to extract the actual class name and type of the object wrapped in
the MXMObject. Furthermore, it provides getters and setters for wrapping/unwrapping an object
into/from an MXMObject;
— MXMAdapter: the basic class implementing the MXMObject interface. By means of an MXMAdapter,
it is possible to convert any object into an MXMObject;
— MXMEngineName: the enumeration listing all possible names (types) of MPEG-M Engines, so that
any entity can be unambiguously identified;
— MXMException: the highest level of exception thrown by MXM. All other exceptions in MXM have to
extend this abstract class;
— MXMInvalidConfigurationException: specialized exception informing an MPEG-M Application
that the MXM Configuration File is trying to use non-compliant MXM Configuration Schema
specifications;
— MXMEngineResponse: the enumeration used as a general-purpose return value for a number of
methods;
— MXMCreatorAndParser: the interface that extends the MXMCreator and MXMParser interfaces
which specify a set of common methods to be implemented by all classes employed to create and
parse data managed by the entire MXM middleware;
— MXMKeys: the class that wraps a key-value construct used to specify an MPEG-M Engine’s parameter.
6 MPEG-M Engines
6.1 General
Throughout Clause 6, the technologies supported by each MPEG-M Engines are detailed. If a specific
profile has not been indicated, it implies that the full standard is supported. Moreover, the APIs of the
aforementioned engines are specified. These APIs have been designed in such a way that any MPEG-M
Application can rely on top of them, independently of their implementation. MPEG-M Part 3 provides a
reference implementation of these MPEG-M Engines; however, this implementation may vary. The only
requirement is that all implementations have to produce the same results.
The MPEG-M Engine APIs can be divided in two subclasses:
— APIs that represent particular data the MPEG-M Engine is specialized on (called “schema handler
APIs”);
— APIs that provide some functionality on the data described above (called “functional APIs”).
The MPEG-M Engines, as already depicted in MPEG-M Part 1, are subdivided in MPEG-M Technology
Engines and MPEG-M Protocol Engines. The difference between them is the way they are acceded:
locally for MPEG-M Technology Engines, remotely in the case of MPEG-M Protocol Engines. A special
case of MPEG-M Technologies Engines are the MPEG-M Orchestrator Engines, which are supposed to
deal with several and different MPEG-M Engines.
8 © ISO/IEC 2016 – All rights reserved

6.2 Technology Engines
6.2.1 Digital Item Engine
6.2.1.1 Technologies
The Digital Item Engine provides access to the technologies specified in Table 2.
Table 2 — Technologies supported by the Digital Item Engine
Standard Profile or technology Reference
Digital Item Declaration Main ISO/IEC 21000-2
MSAF ISO/IEC 23000-7
Digital Item Identification ISO/IEC 21000-3
6.2.1.2 APIs
The Digital Item Engine interface defines the APIs for handling ISO/IEC 21000-2 and ISO/IEC 23000-7
Digital Item Declaration (DID) data structures. Classes implementing the Digital Item Engine interface
shall provide the necessary functionality to
— create Digital Items and retrieve data from them;
— be able to manage:
— Item, Statement, Descriptor from Digital Items;
— Component, Resource from Digital Items;
— License, Metadata, ERRs from Item in Digital Items.
6.2.2 MPEG-21 File Format Engine
6.2.2.1 Technologies
The MPEG-21 File Format Engine provides access to the technologies specified in Table 3.
Table 3 — Technologies supported by the MPEG-21 File Format Engine
Standard Profile or technology Reference
MPEG-21 File Format ISO/IEC 21000-9
6.2.2.2 APIs
6.2.2.2.1 General
The MPEG-21 File Format Engine interface defines the methods for operating over ISO/IEC 21000-
9 MPEG-21 File Format files. Classes implementing the MPEG-21 File Format Engine interface shall
provide the necessary functionality to
— create an MPEG-21 file, and
— access data of an MPEG-21 File.
© ISO/IEC 2016 – All rights reserved 9

6.2.2.2.2 MPEG21 File Creation
Creating an MPEG-21 File involves the following interfaces:
— MPEG21FileCreator: an interface defining the methods to create an MPEG-21 file.
6.2.2.2.3 MP21 File Access
Accessing an MPEG-21 File involves the following interfaces:
— MPEG21FileReader: an interface defining the methods to access an MPEG-21 file.
6.2.3 REL Engine
6.2.3.1 Technologies
The REL Engine provides access to the technologies specified in Table 4.
Table 4 — Technologies supported by the REL Engine
Standard Profile or technology Reference
Rights Expression Language MAM profile ISO/IEC 21000-5
DAC profile ISO/IEC 21000-5
OAC profile ISO/IEC 21000-5
6.2.3.2 APIs
The REL Engine interface defines the methods for handling ISO/IEC 21000-5 Rights Expression
Language (REL) data structures. Classes implementing the REL Engine interface shall provide the
following functionality:
— create Rights Expressions and access data contained in them;
— authorize users to exercise rights.
6.2.3.3 Rights Expression creation and access
6.2.3.3.1 General
Creating and accessing a REL statement involves the following interfaces:
— License: an interface defining the methods to create and parse an r:license element;
— Grant: an interface defining the methods to create and parse an r:grant;
— DigitalResource: an interface defining the methods to create and parse an r:digitalResource;
— ProtectedResource: an interface defining the methods to create and parse an m1x:protectedResource;
— IdentityHolder: an interface defining the methods to create and parse an m1x:identityHolder;
— Issuer: an interface defining the methods to create and parse an r:issuer;
— KeyHolder: an interface defining the methods to create and parse an r:keyHolder.
10 © ISO/IEC 2016 – All rights reserved

6.2.3.3.2 Authorization
Authorizing a user to exercise a right involves the following interfaces:
— AuthorisationManager: an interface defining the methods to authorize a user to exercise a right and
retrieve information from the validation operation;
— AuthorisationResult: an enumeration defining possible result of an authorization.
6.2.4 IPMP Engine
6.2.4.1 Technologies
The IPMP Engine provides access to the technologies specified in Table 5.
Table 5 — Technologies supported by the IPMP Engine
Standard Profile or technology Reference
MPEG-21 IPMP Components Main ISO/IEC 21000-4
MSAF ISO/IEC 23000-5
IPMP XML Messages ISO/IEC 23001-3
6.2.4.2 APIs
6.2.4.2.1 General
The IPMP Engine interface defines the methods for operating over ISO/IEC 21000-4 and ISO/IEC 23000-
5 Intellectual Property Management and Protection data structures. Classes implementing the IPMP
Engine interface shall provide the following functionality:
— create IPMP data structures and access data contained in them.
6.2.4.2.2 IPMP Information creation and access
Creating and accessing IPMP data structures involves the following interfaces:
— IPMPGeneralInfoDescriptor: an interface defining the methods to create and parse an
ipmpinfo:IPMPGeneralInfoDescriptor element;
— IPMPInfoDescriptor: an interface defining the methods to create and parse an
ipmpinfo:IPMPInfoDescriptor element;
— ProtectedAsset: an interface defining the methods to create and parse an ipmpdidl:ProtectedAsset
element;
— RightsDescriptor: an interface defining the methods to create and parse an ipmpinfo:RightsDescriptor
element;
— Tool: an interface defining the methods to create and parse an ipmpinfo:Tool element.
6.2.5 GreenMetadata Engine
6.2.5.1 Technologies
The GreenMetadata Engine provides access to the technologies specified in Table 6.
© ISO/IEC 2016 – All rights reserved 11

Table 6 — Technologies supported by the Green Metadata Engine
Standard Profile or technology Reference
Green Metadata ISO/IEC 23001-11
6.2.5.2 APIs
The Green Metadata TE (Encoder) exposes APIs that are called by the Media Pre-Processor and/or the
Media Encoder to enable the Green Metadata Generator (Encoder) to produce appropriate metadata.
The APIs are called in realtime during pre-processing/encoding. The metadata is used for decoder and
display power reduction, quality recovery and for energy-efficient media selection.
The Green Metadata TE (Decoder) exposes APIs through which it can receive the Green Metadata
stream. The Green Metadata Extractor (Decoder) receives the Green Metadata stream and feeds it to
the Power Optimization Module (Decoder). The metadata is used by the power optimization module to
reduce decoder and display power.
6.2.6 Media Framework Engine
6.2.6.1 Technologies
6.2.6.1.1 General
The Media Framework Engine has to provide access to, at least, the technologies specified in Table 7 to
Table 10.
6.2.6.1.2 Audio
Table 7 — Audio technologies supported by the Media Framework Engine
Standard Profile or technology Reference
MPEG-1 Audio layer II ISO/IEC 11172-3
MPEG-1 Audio layer III ISO/IEC 11172-3
MPEG-4 AAC ISO/IEC 14496-3
MPEG-4 HE-AAC ISO/IEC 14496-3
6.2.6.1.3 Video
Table 8 — Video technologies supported by the Media Framework Engine
Standard Profile or technology Reference
MPEG-1 Video ISO/IEC 11172-2
MPEG-2 Video Main Profile ISO/IEC 13818-2
MPEG-4 Visual Simple Profile ISO/IEC 14496-2
Advanced Simple Profile ISO/IEC 14496-2
MPEG-4 AVC ISO/IEC 14496-10
12 © ISO/IEC 2016 – All rights reserved

6.2.6.1.4 Reconfigurable video
Table 9 — Reconfigurable video technologies supported by the Media Framework Engine
Standard Profile or technology Reference
Codec Configuration Representation ISO/IEC 23001-4
Video tool library ISO/IEC 23002-4
6.2.6.1.5 Muxed content
Table 10 — Muxed content technologies supported by the Media Framework Engine
Standard Profile or technology Reference
MPEG-1 Systems ISO/IEC 11172-1
MPEG-2 Systems Transport Stream ISO/IEC 13818-1
Program Stream ISO/IEC 13818-1
MPEG-4 M4Mux ISO/IEC 14496-1
ISO Base Media File Format ISO/IEC 14496-12
Figure 1 shows the various components of the MPEG-M Media Framework engine at a high level.
Figure 1 — Technologies of the MPEG-M Media Framework Engine
© ISO/IEC 2016 – All rights reserved 13

6.2.6.2 APIs
The MediaFramework is a high level MPEG-M Engine, grouping together several media specific engines
such as Video, Image and Audio Engine. It also implements common functionalities (independent on the
media type) such as resource loading and saving.
The MediaFrameworkEngine holds three main interfaces:
— an interface for accessing the Content (called AccessMedia);
— an interface for creating the Content (called CreationMedia);
— an interface to communicate with Green Metadata engine.
A typical implementation of the AccessMedia interface of the MediaFrameworkEngine first loads a
resource, demultiplex it, check the type of the elementary streams within the resource and call the
associated elementary stream access engines.
A typical implementation of the CreationMedia interface of the MediaFrameworkEngine call the
associated elementary stream creation engines, initialize them with encoding parameters and finally
save the multiplexed resource.
Table 11 shows which class of APIs are provided in the second part of this document for the various
media categories.
Table 11 — MXM Media Framework APIs
Class Type Creation Decoding Rendering
Elementary Image Yes, only if it Yes Yes, only if it is a
is a JPEG texture
Audio Yes
Video Traditional Yes
(MPEG-1,
MPEG-2, etc.)
RVC Yes Yes
Muxed content Yes Yes
The Media Framework Engine acts like a factory that returns an instance of the AccessMediaFramework
or CreationMediaFramework interface.
a) public AccessMediaFramework getAccessMediaFramework(Canvas canvas) throws
MediaFrameworkEngineException;
b) public CreationMediaFramework getCreationMediaFramework() throws
MediaFrameworkEngineException.
In the first case, an optional java.awt.Canvas can be passed to specify where the MediaFramework have
to render the resource.
The AccessMediaFramework holds the following methods:
— boolean consumeResource(List tools, URI inputLocationResource, String mediaType,
OutputStream os, MXMObject useType);
This method is used to consume a given resource.
— List tools: one or more IPMPTools (or other kind of tools) used to manipulate the
resource (e.g. ciphering, filtering) and defining;
— control points involved;
14 © ISO/IEC 2016 – All rights reserved

— name or ID of the tool;
— Tool messages (IPMPMessage) that could contains one or a set of keys;
— URI inputLocationResource: where the resource handled is located;
— String mediaType: what kind of media is handled;
— OutputStream os: where the resource will be eventually stored;
— MXMObject useType: the type of interaction the user requested on the passed resource;
— void open( MXMObject media );
This method is a high-level API that is used to prepare a resource (passed as parameter in a
MXMObject) to be rendered.
— Void play(), void pause(), void stop(), void release(), void setVolume(int newVolume);
These methods are used to manage the media chain created to render the resource. It’s possible to
start, to pause, to stop the rendering of the passed resource; to release the allocated resource and
to set the volume of audio stream inside the resource.
— CaptureFrameFromMedia: return an image captured from media.
The CreationMediaFramework holds the following methods:
— boolean createResource(List tools, URI locationResource, URI newLocationResource).
This method is used to create and handle a given resource:
1) List tools: one or more IPMPTools (or other kind of tools) used to manipulate the
resource (e.g. ciphering, filtering) and defining;
2) control points involved;
3) name or ID of the tool;
4) tool messages (IPMPMessage) that could contain one or a set of keys;
5) URI locationResource: where the resource handled is located;
6) URI newLocationResource: where the resource handled will be located.
GreenMediaFramework specifies a set of APIs that MediaFramework, on both decode and encoder sides,
should support so that an external component can use them in order to reduce the power consumption
required to receive, decode and display the media. It involves the following subinterfaces.
— GreenMediaEncoderFramework: This interface specifies API useful to instruct the Media Framework
to encode media in such a way that the power required for decoding respects the capabilities and
power condition of the decoding device. This is done by allowing external components (mainly the
GreenMetadataManager) to change resolution, framerate and complexity reduction scaling. To
implement a MediaFramework that supports such functionality and the semantics of the relevant
parameters, refer to ISO/IEC 23001-11.
— GreenMediaDecoderFramework: This interface specifies APIs to control the amount of power
required by the decoding Media Framework to decode the media. This allows another component
(mainly the GreenMetadataManager) to reduce the power consumption rate with respect to power
supply conditions (i.e. battery level). Controlled parameters include clock scaling, voltage scaling and
display scaling contrast bounds. To implement a MediaFramework that supports such functionality
and the semantics of the relevant parameters, refer to ISO/IEC 23001-11.
© ISO/IEC 2016 – All rights reserved 15

6.2.7 Metadata Engine
6.2.7.1 Technologies
The Metadata Engine provides access
...