ISO/IEC 23000-10:2012

INTERNATIONAL ISO/IEC
STANDARD 23000-10
Second edition
2012-12-15
Information technology — Multimedia
application format (MPEG-A) —
Part 10:
Surveillance application format
Technologie de l'information — Format pour application multimédia
(MPEG-A) —
Partie 10: Format pour application à la surveillance

Reference number
©
ISO/IEC 2012
©  ISO/IEC 2012
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized in any form or by any means,
electronic or mechanical, including photocopying and microfilm, without permission in writing from either ISO at the address below or
ISO's member body in the country of the requester.
ISO copyright office
Case postale 56  CH-1211 Geneva 20
Tel. + 41 22 749 01 11
Fax + 41 22 749 09 47
E-mail copyright@iso.org
Web www.iso.org
Published in Switzerland
ii © ISO/IEC 2012 – All rights reserved

Contents Page
Foreword . iv
Introduction . v
1 Scope . 1
2 Normative references . 1
3 Overview of Repeatedly Used Standards . 1
3.1 ISO Base Media File Format . 1
3.2 AVC File Format . 3
4 Using the Surveillance AF . 3
4.1 General . 3
4.2 Time reference . 3
4.3 File Structure . 3
4.4 File Contents . 4
4.5 Track Structure . 5
4.6 Derivation from the ISO Base Media File Format . 7
5 Derived file formats . 11
6 Metadata . 11
6.1 Introduction . 11
6.2 File Level Metadata . 11
6.3 Track Level Metadata . 12
6.4 Timed Metadata . 13

© ISO/IEC 2012 – All rights reserved iii

Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are members of
ISO or IEC participate in the development of International Standards through technical committees
established by the respective organization to deal with particular fields of technical activity. ISO and IEC
technical committees collaborate in fields of mutual interest. Other international organizations, governmental
and non-governmental, in liaison with ISO and IEC, also take part in the work. In the field of information
technology, ISO and IEC have established a joint technical committee, ISO/IEC JTC 1.
International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2.
The main task of the joint technical committee is to prepare International Standards. Draft International
Standards adopted by the joint technical committee are circulated to national bodies for voting. Publication as
an International Standard requires approval by at least 75 % of the national bodies casting a vote.
Attention is drawn to the possibility that some of the elements of this document may be the subject of patent
rights. ISO and IEC shall not be held responsible for identifying any or all such patent rights.
ISO/IEC 23000-10 was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 29, Coding of audio, picture, multimedia and hypermedia information.
This second edition cancels and replaces the first edition (ISO/IEC 23000-10:2009) and
ISO/IEC 23000-10:2009/Amd 1:2010.
ISO/IEC 23000 consists of the following parts, under the general title Information technology — Multimedia
application format (MPEG-A):
 Part 1: Purpose for multimedia application formats [Technical Report]
 Part 2: MPEG music player application format
 Part 3: MPEG photo player application format
 Part 4: Musical slide show application format
 Part 5: Media streaming application format
 Part 6: Professional archival application format
 Part 7: Open access application format
 Part 8: Portable video application format
 Part 9: Digital Multimedia Broadcasting application format
 Part 10: Surveillance application format
 Part 11: Stereoscopic video application format
 Part 12: Interactive music application format
iv © ISO/IEC 2012 – All rights reserved

Introduction
This part of ISO/IEC 23000 specifies a file format to enable interoperability for surveillance systems. Features
nd
are supported required by wide spread used surveillance systems installations. In the 2 edition support for
audio features has been added, optionally described metadata have been removed and the support for video
significantly extended.
© ISO/IEC 2012 – All rights reserved v

INTERNATIONAL STANDARD ISO/IEC 23000-10:2012(E)

Information technology — Multimedia application format
(MPEG-A) —
Part 10:
Surveillance application format
1 Scope
This part of ISO/IEC 23000 specifies a file format designed to provide for a first level of interoperability for
surveillance systems. The file format provides the overall structure for storing media content and associated
metadata.
2 Normative references
The following documents, in whole or in part, are indispensable for the application of this document. For dated
references, only the edition cited applies. For undated references, the latest edition of the referenced
document (including any amendments) applies.
ISO/IEC 9834-8:2005, Information technology – Open Systems Interconnection – Procedures for the
operation of OSI Registration Authorities: Generation and registration of Universally Unique Identifiers
(UUIDs) and their use as ASN.1 Object Identifier component
ISO/IEC 14496-10, Information technology — Coding of audio-visual objects — Part 10: Advanced Video
Coding
ISO/IEC 14496-12:2008, Information technology — Coding of audio-visual objects — Part 12: ISO base media
file format
ISO/IEC 14496-14:2003, Information technology — Coding of audio-visual objects — Part 14: MP4 file format
ISO/IEC 14496-15:2010, Information technology — Coding of audio-visual objects — Part 15: Advanced
Video Coding (AVC) file format
W3C Recommendation, XML Schema Part 2: Datatypes Second Edition, 28 October 2004
3 Overview of Repeatedly Used Standards
3.1 ISO Base Media File Format
The ISO Base Media File Format [see ISO/IEC 14496-12] is designed to contain timed media information for a
presentation in a flexible, extensible format that facilitates interchange, management, editing, and presentation
of the media. The ISO Base Media File Format is a base format for media file formats.
Also the storage format for AVC coded video – the AVC file format [see ISO/IEC 14496-15] – uses the
techniques from the ISO Base Media File Format. In order to store AVC video together with AAC audio data
© ISO/IEC 2012 – All rights reserved 1

[see ISO/IEC 14496-3] the mp4 file format [see ISO/IEC 14496-14] has been derived from ISO Base Media
File Format.
ISO file
Movie data   Media data
trak (video)
…other boxes
Interleaved, time-ordered, video
trak (audio)   and audio frames

Figure 1 — Example of a simple ISO file used for interchange, containing two streams
The file structure is object-oriented as shown in Figure 1, which means that a file can be decomposed into
constituent objects very simply, and the structure of the objects inferred directly from their type. The file format
is designed to be independent of any particular network protocol while enabling efficient support for them in so
called hint tracks.
It also provides support for metadata in the form of ‘meta’ boxes at the File, Movie and Track level. This allows
support for static (un-timed) metadata. Figure 2 schematically illustrates the location of these un-timed
metadata boxes. However, the ISO Base Media File Format also supports storage of timed metadata. These
metadata can be synchronized with the video tracks and provide additional information e.g. time code values.
Trak
Movie
meta
meta Mdat
meta
Trak
meta
Figure 2 — Support of static un-timed metadata in ISO Base Media Files
Movie fragments can be used to enable features such as instant replay. In general, all data describing timing,
properties and locations of individual video samples are contained in tables within a track. Usually these
tables can only be written if all samples of the track are known. To overcome this burden the ISO Base Media
File Format specifies the usage of movie fragments to extend a presentation in time. In the Surveillance AF
the movie box may contain no or just a limited number of samples (in all the tracks) and the necessary
initialization data. Additional samples are described in one or more movie fragments, depending on the use
case, e.g. to enable instant replay functionality (the file is played while it is still being recorded). Each movie
fragment consists of zero or one track fragments for each track initialized in the movie header.
Each track fragment contains a number of track fragment runs describing the samples individually. If some
properties are identical for all samples in a fragment this value can be stored in the track fragment header (e.g.
sample duration for constant frame rate video).
In order to support storage boundary oriented applications a Surveillance AF using movie fragments should
define track fragment runs with a predictable number of samples in each fragment run and a defined number
of track fragment runs in a track fragment.
While writing the file, the video data chunks (and chunks from other tracks) are appended to the end of the
media data container which might be physically located at the end of the file or in a separate physical file. The
2 © ISO/IEC 2012 – All rights reserved

descriptive data about the media data samples is written to the reserved space for movie fragments – it is
appended to the track fragment run table. Additionally the number of samples is changed in this track
fragment run (see Figure 6). If a new track fragment run is to be created it is appended at the end of the
previous track fragment run. Additionally the size of the track fragment box is changed. The same applies for
creating new track fragments or new movie fragments. If the space reserved for movie and track fragments is
fully used no more samples can be added and a new Surveillance AF fragment should be created.
If the file is to be read while it is still being written the reader can access all needed information in finalized
movie fragments and track fragments. If supported by an application the track fragment run table which is
currently being written can be accessed up to the sample number given with the sample count value of this
track fragment run.
Note that for every video sample a metadata sample must be provided. Therefore the technique described
here must be used for all the video tracks and for all corresponding metadata tracks. When using more than
one video track it must be ensured that all tracks have the same total duration.
When a Surveillance AF fragment is being recorded the duration of this Surveillance AF fragment should be
set to zero to indicate that the duration is currently changing. In this case a player application should scan the
track/movie fragment boxes to calculate the movie duration.
3.2 AVC File Format
In this Surveillance AF support for AVC coded video data is restricted to the main part of ISO/IEC 14496-10,
excluding Annex G (Scalable Video Coding) and Annex H (Multiview Video Coding).
4 Using the Surveillance AF
4.1 General
This Clause provides necessary information for creating and using Surveillance AF fragments.
It describes the box types that Surveillance AF readers will recognize. Other box types may be included but
will not be recognized.
4.2 Time reference
Complete UTC timing information shall be provided according to the “dateTime” data type of W3C
Recommendation “XML Schema Part2: Datatypes” using the “Z” parameter. The time calculation defined in
the ISO base media file format [see ISO/IEC 14496-12] shall be used with 64-bit numbers.
4.3 File Structure
A Surveillance AF contains of a set of self-contained AF fragments which are connected to each other. A
Surveillance AF fragment covers a limited amount of time. Each Surveillance AF fragment shall be identifiable
by an UUID (universal unique identifier) [see ISO/IEC 9834-8]. Each Surveillance AF fragment is linked to a
predecessor and successor fragment through their UUIDs (see Figure 3).
All Surveillance AF data is stored within the Surveillance AF fragments. If a fragment has no predecessor or
successor its value is set the current fragment. Additionally an URI [see RFC 3986] can be given serving as a
hint to the location of the predecessor and successor fragments. A Surveillance AF fragment remains self
contained even if unhinged. Note that there is no requirement to use more than one Surveillance AF fragment.
The concept of using fragments e.g. enables ring buffer architectures.
Each fragment shall be a valid file as defined by this specification.
© ISO/IEC 2012 – All rights reserved 3

Figure 3 — Surveillance AF fragments linked together by means of predecessor id and successor id
All connected Surveillance AF fragments shall use the same number of tracks and the same set of parameters
as timing and media coding settings.
The size of a Surveillance AF fragment can be set as indicated by the application, e.g. providing a constant
number of samples in each Surveillance AF fragment.
Each fragment shall contain the mandatory metadata boxes and may contain additional metadata boxes as
specified in Clause 6.
Managing the storage of Surveillance AF fragments and the connection of fragments to the application is out
of the scope of this specification.
4.4 File Contents
The file format for the Surveillance AF is based on the ISO Base Media File Format and derived file formats. A
Surveillance AF fragment shall contain:
 One box of meta type at file level and one for each media track at track level
4 © ISO/IEC 2012 – All rights reserved

If the Surveillance AF fragment contains media data one track of timed metadata shall be provided for each
media track, not necessarily contained in the very same Surveillance AF fragment.
4.5 Track Structure
If more than one video track from one camera is present these video tracks shall be in the same alternate
group (see 4.6.4 on track selection). Additionally, to each video track a metadata track shall be linked using a
track reference (see 6.4 on the metadata tracks and sample structure).
Different video tracks may contain the same video content coded with different parameters or using a different
coding technology. Alternatively different video tracks may contain different content, e.g. different views of the
area monitored (see Figure 4 and Figure 5).
The same applies to audio tracks.
If there is more than one media track all media tracks shall have the same duration.
NOTE - this does not imply that all tracks have the same number of samples. Different video tracks containing the same
video content may be coded using different frame rates.
ftyp meta mdat
moov
...