ISO/IEC 15938-6:2020

INTERNATIONAL ISO/IEC
STANDARD 15938-6
Second edition
2020-06
Information technology — Multimedia
content description interface —
Part 6:
Reference software
Technologies de l'information — Interface de description du contenu
multimédia —
Partie 6: Logiciel de référence
Reference number
©
ISO/IEC 2020
© ISO/IEC 2020
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting
on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address
below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Fax: +41 22 749 09 47
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii © ISO/IEC 2020 – All rights reserved

Contents Page
Foreword .iv
Introduction .v
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 Abbreviated terms . 2
5 Copyright disclaimer for software modules . 2
6 XM software architecture . 2
6.1 Block diagrams . 2
6.2 Block descriptions . 4
6.2.1 General. 4
6.2.2 Media database . 4
6.2.3 AV decoders . 4
6.2.4 Media data . 5
6.2.5 Extraction tools . 5
6.2.6 Descriptors (Ds) and description schemes (DSs) . 6
6.2.7 Coding schemes (CSs) . . 7
6.2.8 Search and matching tools. 8
6.2.9 Media transcoders .10
6.2.10 Applications .10
6.2.11 Interface structure.10
7 Systems reference software (BiM) .11
7.1 General .11
7.2 BiM reference software .11
7.2.1 General.11
7.2.2 Package content .11
7.2.3 Installation and execution .11
7.2.4 Software overview .12
7.2.5 Main packages .12
7.3 Access unit navigator .12
7.3.1 General.12
7.3.2 Textual access unit encoder module .13
7.3.3 Textual access unit decoder module .13
8 Description definition language (DDL) reference software .14
9 Video reference software .14
10 Audio reference software .15
11 Multimedia description scheme reference software .17
12 Compilation of the reference software .18
13 Usage information for individual descriptors and description schemes .18
© ISO/IEC 2020 – All rights reserved iii

Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that
are members of ISO or IEC participate in the development of International Standards through
technical committees established by the respective organization to deal with particular fields of
technical activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other
international organizations, governmental and non-governmental, in liaison with ISO and IEC, also
take part in the work.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for
the different types of document should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www .iso .org/ directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject
of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent
rights. Details of any patent rights identified during the development of the document will be in the
Introduction and/or on the ISO list of patent declarations received (see www .iso .org/ patents) or the IEC
list of patent declarations received (see http:// patents .iec .ch).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to the
World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see www .iso .org/
iso/ foreword .html.
This document was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 29, Coding of audio, picture, multimedia and hypermedia information.
This second edition cancels and replaces the first edition (ISO/IEC 15938-6:2003), which
has been technically revised. It also incorporates the Amendments ISO/IEC 15938-6:2003/
Amd.1:2006, ISO/IEC 15938-6:2003/Amd.1:2006/Cor.1:2007, ISO/IEC 15938-6:2003/Amd.2:2007,
ISO/IEC 15938-6:2003/Amd.3:2010 and ISO/IEC 15938-6:2003/Amd.4:2011. The main changes
compared to the previous edition are as follows:
— all previous Amendments have been incorporated and the electronic attachments have been
updated;
— minor editorial corrections have been made throughout the document to fully align with ISO/IEC
Directives.
A list of all parts in the ISO/IEC 15938 series can be found on the ISO website.
Any feedback or questions on this document should be directed to the user’s national standards body. A
complete listing of these bodies can be found at www .iso .org/ members .html.
iv © ISO/IEC 2020 – All rights reserved

Introduction
This document provides a standardized set of technologies for describing multimedia content. It
addresses a broad spectrum of multimedia applications and requirements by providing a metadata
system for describing the features of multimedia content.
The software is available at: https:// standards .iso .org/ iso -iec/ 15938/ -6/ ed -2/ en.
The following are specified in this document:
Description schemes (DS) describe entities or relationships pertaining to multimedia content.
Description schemes specify the structure and semantics of their components, which can be description
schemes, descriptors, or datatypes.
Descriptors (D) describe features, attributes, or groups of attributes of multimedia content.
Datatypes are the basic reusable datatypes employed by description schemes and descriptors.
Systems tools support delivery of descriptions, multiplexing of descriptions with multimedia content,
synchronization, file format, and so forth.
This document contains simulation software for tools defined in ISO/IEC 15938-1, ISO/IEC 15938-2,
ISO/IEC 15938-3, ISO/IEC 15938-4 and ISO/IEC 15938-5. This software has been derived from the
verification models used in the process of developing this series.
Where multimedia content extraction or multimedia content description software is provided, attention
is called to the fact that these software modules are provided for the purpose of creating bitstreams of
descriptors and description schemes with normative syntax. The performance of these software tools
is not indicative of that which can be obtained from implementations where quality and computational
optimization are given priority. The techniques used for extracting descriptors or deriving description
schemes are not specified by this document. This information can be found in the corresponding Part of
the ISO/IEC 15938 series.
The International Organization for Standardization (ISO) and International Electrotechnical
Commission (IEC) draw attention to the fact that it is claimed that compliance with this document may
involve the use of patents.
ISO and IEC take no position concerning the evidence, validity and scope of these patent rights.
The holders of these patent rights have assured ISO and IEC that they are willing to negotiate licences
under reasonable and non-discriminatory terms and conditions with applicants throughout the world.
In this respect, the statements of the holders of these patent rights are registered with ISO and IEC.
Information may be obtained from the patent database available at www .iso .org/ patents.
Attention is drawn to the possibility that some of the elements of this document may be the subject
of patent rights other than those in the patent database. ISO and IEC shall not be held responsible for
identifying any or all such patent rights.
© ISO/IEC 2020 – All rights reserved v

INTERNATIONAL STANDARD ISO/IEC 15938-6:2020(E)
Information technology — Multimedia content description
interface —
Part 6:
Reference software
1 Scope
The ISO/IEC 15938 series operates on and generates conformant bitstreams. This document provides a
specific implementation that behaves in a conformant manner.
NOTE 1 Other implementations that conform to the ISO/IEC 15938 series are possible, which do not necessarily
use the algorithms or the programming techniques of the reference software.
The software contained in this document is known as eXperimentation Model (XM) and is divided into
five categories:
a) Binary format for MPEG-7 (BiM). This software converts DDL (XML) based descriptions to binary
format and vice versa as explained in Clause 5.
b) DDL parser and DDL validation parser. The components of this software module are specified in
Clause 6.
c) Visual descriptors. This software creates standard visual descriptions from associated (visual)
media content as explained in Clause 7.
NOTE 2 The techniques used for extracting descriptors are informative, and the quality and complexity
of these extraction tools has not been optimized.
d) Audio descriptors. This software creates standard descriptions from associated (audio) media
content as explained in Clause 8.
NOTE 3 The techniques used for extracting descriptors are informative, and the quality and complexity
of these extraction tools has not been optimized.
e) Multimedia description schemes. This software modules provide standard descriptions of
multimedia description schemes as specified in Clause 9.
2 Normative references
There are no normative references in this document.
3 Terms and definitions
No terms and definitions are listed in this document.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— ISO Online browsing platform: available at https:// www .iso .org/ obp
— IEC Electropedia: available at http:// www .electropedia .org/
© ISO/IEC 2020 – All rights reserved 1

4 Abbreviated terms
AV audio-visual
CS coding scheme
D descriptor
Ds descriptors
DCT discrete cosine transform
DDL description definition language
DS description scheme
DSs description schemes
MDS multimedia description schemes
MPEG moving picture experts group
MPEG-7 multimedia content description interface standard (the ISO/IEC 15938 series)
XML eXtensible Markup Language
5 Copyright disclaimer for software modules
The source code for this document can be found at https:// standards .iso .org/ iso -iec/ 15938/ -6/ ed -2/ en/ .
Each source code module in this document contains a copyright disclaimer which shall not be removed
from the source code module.
In the text of each copyright disclaimer, is replaced with a reference to its associated
specification, e.g. MPEG-7 Systems (ISO/IEC 15938-1), MPEG-7 Visual (ISO/IEC 15938-3), MPEG-7 Audio
(ISO/IEC 15938-4), MPEG-7 Multimedia description schemes (ISO/IEC 15938-5).
"This software module was originally developed by () and edited by
(), (), in the course of development of the . This software
module is an implementation of a part of one or more tools as specified by the
. ISO/IEC gives users of the free license to this software module
or modifications thereof for use in hardware or software products claiming conformance to the standard>. Those intending to use this software module in hardware or software products are advised
that its use may infringe existing patents. The original developer of this software module and his/her
company, the subsequent editors and their companies, and ISO/IEC have no liability for use of this software
module or modifications thereof in an implementation. Copyright is not released for non
conforming products. retains full right to use the code for his/her own purpose, assign or donate the
code to a third party and to inhibit third parties from using the code for non conforming
products. This copyright notice must be included in all copies or derivative works. Copyright ©20xx-20xx".
=First Name, =Last Name, =Company Name
6 XM software architecture
6.1 Block diagrams
This clause provides information about the XM software architecture. The block diagrams give short
overviews and introduce individual components of the XM software.
2 © ISO/IEC 2020 – All rights reserved

The composing elements of the MPEG-7 reference software are characterized by their functionality and
by their interfaces. They can be configured according to what here is referred as "key applications".
From a functional point of view, they can be distinguished as follows:
— "extraction applications" (a description data base is built from a media database) as illustrated in
Figure 1;
— "search and retrieval applications" (a description is compared with the descriptions in a database to
find the one with the lowest distance) as illustrated in Figure 2;
— "transcoding applications" (a media data base is converted into another media database based on
its description) as illustrated in Figure 3.
Figure 1 — An "extraction application" using the XM reference software modules, with boxes
representing procedural parts and circles representing data structures
Figure 2 — A "search and retrieval application" using the XM reference software modules, with
boxes representing procedural parts and circles representing data structures
© ISO/IEC 2020 – All rights reserved 3

Figure 3 — A "transcoding application" using the XM reference software modules, with boxes
representing procedural parts and circles representing data structures
6.2 Block descriptions
6.2.1 General
In subclauses 6.2.2 through 6.2.11, the blocks of the "key applications" are distinguished. For
elements that are related to specific descriptors or description schemes, the interface is given using a
DummyType example. This represents the XM integration template and not a normative descriptor or
description scheme.
6.2.2 Media database
The media database contains media files, which are supported as input files by the AV decoders. The
database file is read from a file and contains one media filename per line. From this media filename all
additional input and output filenames can be derived.
6.2.3 AV decoders
The XM supports the following AV decoders:
1)
— Still image decoders: ImageMagick® (Ver. 6.8.8 linked as external library, not included in the XM
reference software distribution).
— MPEG-1, MPEG-2 video decoders: (XM directory: Decoders/MPEG2Dec).
1) ImageMagick® is the trademark of a product supplied by ImageMagick Studio LLC. This information is given
for the convenience of users of this document and does not constitute an endorsement by ISO or IEC of the product
named. Equivalent products may be used if they can be shown to lead to the same results.
4 © ISO/IEC 2020 – All rights reserved

— MPEG-1 video motion vector extractor: (XM directory: Decoders/MPEG2Dec). This extracts images
and motion vectors.
— 3D Objects: (XM directory: Media). This reads a 3D object for 3D shape descriptors.
— Key Points: (XM directory: Media). This reads in a list of key points from a file.
— Audio decoders: (XM directory: Media). They read audio files.
6.2.4 Media data
This is the internal XM representation of the raw media data (one class with different structures
depending on the media content type). The class description for media data can be found in the Media
XM directory.
6.2.5 Extraction tools
Extraction tools are specific extraction methods defined for each descriptor and description scheme.
All the source files are available in the ExtractionUtilities XM directory. The extraction tools extract the
descriptions from media data. Because media data can be of significant size, the extraction is performed
on time entities of the media, i.e., if the media is a video the extraction is done frame by frame. Some
of the extraction tools may need OpenCV linked with XM as an external library. The interface of the
DummyType extraction tool (implementation template) is:
//=============================================================================
class DummyTypeExtractionTool: public DescriptorExtractor
{
friend DummyTypeExtractionInterface;
public:
// Null constructor
DummyTypeExtractionTool();
// Also connects the Descriptor (result memory) to the extraction
// If set to "0" it automatically creates the descriptor
DummyTypeExtractionTool(DummyTypeDescriptorInterfaceABC
*DummyType);
// ID of object type
virtual const UUID& GetObjectID(void);
// Object type name
virtual const char *GetName(void);

// This informs the extractor where the source data comes from
virtual int SetSourceMedia(MultiMediaInterfaceABC* media);

// Pointer where the description is stored
virtual DummyTypeDescriptorInterfaceABC*
GetDescriptorInterface(void);
virtual int SetDescriptorInterface(DummyTypeDescriptorInterfaceABC
*aDummyTypeDescriptorInterface);

// initialize descriptor and extraction process (input media must be known)
virtual unsigned long InitExtracting(void);

// performs extraction from input media frame by input media frame
virtual unsigned long StartExtracting(void);

// collects descriptor data after all input media frames were processed
virtual unsigned long PostExtracting(void);

// Extraction object must not be used, only its interface is allowed
// to be used. This function is to get the interface
virtual DummyTypeExtractionInterfaceABC *GetInterface(void);

// access is allowed only by class factories for this
// object. This avoids having to duplicate the
// ID definition in multiple locations. In the future, we may
© ISO/IEC 2020 – All rights reserved 5

// have to do this. PLEASE DO NOT USE THESE UNLESS YOU ARE
// IMPLEMENTING A CLASS FACTORY GENERATING THIS OBJECT
static const UUID myID;
static const char * myName;
private:
// Destructor is private to allow creation of
// object only by using "new"
virtual ~DummyTypeExtractionTool();

DummyTypeExtractionInterface m_Interface;
DummyTypeDescriptorInterfaceABC *m_DescriptorInterface;
MultiMediaInterfaceABC* m_Media;

// only used in this dummy type to show extraction function
unsigned long m_FrameCnt;
#ifdef __HasSubTypes /*include this section if sub-descriptors exist,
remove this section if no sub-descriptors exist*/
SubDummyTypeAExtractionInterfaceABC *m_SubDummyTypeAExtraction;
SubDummyTypeBExtractionInterfaceABC *m_SubDummyTypeBExtraction;
#endif /* __HasSubTypes*/
int m_DummyExtractionParameter;
}; // End class
//=============================================================================

6.2.6 Descriptors (Ds) and description schemes (DSs)
These modules implement the data structure of normative descriptors and description schemes. Low level
video descriptors are using a dedicated C++ class. This class provides methods to access the elements of
the normative descriptions. The source files are located in the descriptors directory. All other normative
Ds and DSs are using the GenericDS class located in the DescriptionSchemes directory. The GenericDS
class does not implement the data structure in a dedicated way, but it is an interface to the XML parser
library which controls the memory for the tree structure of the instantiated D or DS. The interface of the
descriptors class is given for the DummyType descriptor (implementation template) below:
/=============================================================================
class DummyTypeDescriptor: public Descriptor
{
friend DummyTypeDescriptorInterface;
public:
DummyTypeDescriptor();
#ifdef __HasSubTypes /*include this section if sub-descriptors exist,
remove this section if no sub-descriptors exist*/
// constructor which also constructs and/or connects the descriptor object
DummyTypeDescriptor(SubDummyTypeADecriptorInterfaceABC *aSubDummyTypeA,
SubDummyTypeBDescriptorInterfaceABC *aSubDummyTypeB);
#endif /* __HasSubTypes*/
virtual const UUID& GetValueID(void);
virtual const char* GetValueName(void);

virtual const UUID& GetObjectID(void);
virtual const char *GetName(void);

// for reference counting
virtual void addref();
virtual void release();
#ifdef __HasSubTypes /*include this section if sub-descriptors exist,
remove this section if no sub-descriptors exist*/
/* only needed for manual connection with sub components*/
virtual SubDummyTypeADescriptorInterfaceABC
*GetSubDummyTypeADescriptorInterface(void);
virtual unsigned long
SetSubDummyTypeADescriptorInterface(SubDummyTypeADescriptorInterfaceABC
*aSubDummyTypeADescriptorInterface);

virtual SubDummyTypeBDescriptorInterfaceABC
6 © ISO/IEC 2020 – All rights reserved

*GetSubDummyTypeBDescriptorInterface(void);
virtual unsigned long
SetSubDummyTypeBDescriptorInterface(SubDummyTypeBDescriptorInterfaceABC
*aSubDummyTypeBDescriptorInterface);
#endif /* __HasSubTypes*/
// actual descriptor methods, only in this dummy type example
virtual long GetDummyContents(void);
virtual void SetDummyContents(const long val);

// transformation to GenericDS object (MDS implementaion style)
virtual unsigned long
ExportDDL(GenericDSInterfaceABC *aParentDescription);
virtual unsigned long ImportDDL(GenericDSInterfaceABC *aDescription);

// access is allowed only by class factories for this
// object. This avoids having to duplicate the
// ID definition in multiple locations. In the future, we may
// have to do this. PLEASE DO NOT USE THESE UNLESS YOU ARE
// IMPLEMENTING A CLASS FACTORY GENERATING THIS OBJECT
static const UUID myID;
static const char * myName;
virtual DummyTypeDescriptorInterfaceABC *GetInterface(void);

private:
// private destructor to force reference counting mechanism
virtual ~DummyTypeDescriptor();

// reference counter
unsigned long m_refcount;
DummyTypeDescriptorInterface m_Interface;

const bool m_isProprietary;
static const char * valName;
static const UUID valID;
#ifdef __HasSubTypes /*include this section if sub-descriptors exist,
remove this section if no sub-descriptors exist*/
SubDummyTypeADescriptorInterfaceABC *m_SubDummyTypeADescriptorInterface;
SubDummyTypeBDescriptorInterfaceABC *m_SubDummyTypeBDescriptorInterface;
#endif /* __HasSubTypes*/
// This is the actual data the D/DSType stores. In this particular
// dummy example it's just a signed long called m_DummyContents
long m_DummyContents;
};
/=============================================================================

6.2.7 Coding schemes (CSs)
Coding schemes are specific coding and decoding methods defined for individual descriptors (Ds)
and description schemes (DSs). All the source files are available in the CodingSchemes directory. All
normative coding schemes are located in the CodingSchemes directory. Coding schemes are available
for the visual descriptors to encode or to decode a description into its binary representation. The
alternative implementation of the coding scheme box, which is available for all Ds and DSs, allows
encoding and decoding of descriptors into its DDL representation using the GenericDSCS which is an
interface to the "write to file" and "read from
...