ISO/IEC TR 19566-2:2016

TECHNICAL ISO/IEC TR
REPORT 19566-2
First edition
2016-10-15
Information technologies — JPEG
Systems —
Part 2:
Transport mechanisms and packaging
Technologies de l’information — Systèmes JPEG -- —
Partie 2: Mécanismes de transport et paquetage
Reference number
ISO/IEC TR 19566-2:2016(E)
©
ISO/IEC 2016

---------------------- Page: 1 ----------------------
ISO/IEC TR 19566-2:2016(E)

COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2016, Published in Switzerland
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized otherwise in any form
or by any means, electronic or mechanical, including photocopying, or posting on the internet or an intranet, without prior
written permission. Permission can be requested from either ISO at the address below or ISO’s member body in the country of
the requester.
ISO copyright office
Ch. de Blandonnet 8 • CP 401
CH-1214 Vernier, Geneva, Switzerland
Tel. +41 22 749 01 11
Fax +41 22 749 09 47
copyright@iso.org
www.iso.org
ii © ISO/IEC 2016 – All rights reserved

---------------------- Page: 2 ----------------------
ISO/IEC TR 19566-2:2016(E)

Contents Page
Foreword .iv
Introduction .v
1 Scope . 1
2 Normative references . 1
3 Terms, definitions and abbreviated terms . 1
3.1 Terms and definitions . 1
3.2 Abbreviated terms . 3
4 Conventions . 4
4.1 Operators . 4
4.1.1 Arithmetic operators . 4
4.1.2 Logical operators . 4
4.1.3 Relational operators . 4
4.1.4 Precedence order of operators . 4
4.1.5 Mathematical functions . 5
5 General . 5
6 Purpose of the document . 6
7 Technical rationale — Partial representation of image and metadata content.6
8 Use cases and requirements . 7
9 Mapping content to data-bins . 8
9.1 Codestream data-bins . 8
9.2 Metadata-bins . 9
9.3 Designing box-structured file formats for JPIP .10
10 Typical JPIP architecture .11
11 JPIP request and response schemes .12
11.1 JPIP client request .13
11.2 JPIP server response controls .14
12 Design and adaptation of codestream structures for JPIP .15
13 Application example: JPEG over JPIP .15
Bibliography .16
© ISO/IEC 2016 – All rights reserved iii

---------------------- Page: 3 ----------------------
ISO/IEC TR 19566-2:2016(E)

Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are
members of ISO or IEC participate in the development of International Standards through technical
committees established by the respective organization to deal with particular fields of technical
activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international
organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the
work. In the field of information technology, ISO and IEC have established a joint technical committee,
ISO/IEC JTC 1.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular the different approval criteria needed for
the different types of document should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject
of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent
rights. Details of any patent rights identified during the development of the document will be in the
Introduction and/or on the ISO list of patent declarations received (see www.iso.org/patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation on the meaning of ISO specific terms and expressions related to conformity
assessment, as well as information about ISO’s adherence to the WTO principles in the Technical
Barriers to Trade (TBT) see the following URL: Foreword - Supplementary information
The committee responsible for this document is ISO/IEC JTC 1, Information technology, Subcommittee
SC 29, Coding of audio, picture, multimedia and hypermedia information.
A list of all parts of the ISO/IEC 19566 series can be found on the ISO website.
iv © ISO/IEC 2016 – All rights reserved

---------------------- Page: 4 ----------------------
ISO/IEC TR 19566-2:2016(E)

Introduction
Access to images and metadata using channels with limited bit rates can be significantly speed up by
allowing proper decoding of partial representations. In this case, parts of the image can be displayed to
the user or processed by an algorithm as soon as parts of the codestream or of the file are available at
the client side.
Rec. ITU-T T.808 | ISO/IEC 15444-9 standardizes mechanisms called JPIP for incrementally
communicating box-structured files, as well as information found within codestreams that may or
may not be embedded within boxes of a box-structured file. By these means, standardized methods for
accessing meaningful parts of an image are available.
So far, Rec. ITU-T T.808 | ISO/IEC 15444-9 is part of the JPEG 2000 standards. Other standards like JPEG
(Rec. ITU-T T.81 | ISO/IEC 10918-1) or JPEG-XR (ISO/IEC 29199) are either not supported at all, or only
in a very limited way. Consequently, application of JPIP is predominantly restricted to JPEG 2000. In the
ambition to create an ecosystem of tools that can be applied to many or all standards of the JPEG family,
this document gives guidelines for design of future compression standards and transmission formats
such that partial access to images can be provided in a uniform manner based on the concepts and
ideas of JPIP as defined in Rec. ITU-T T.808 | ISO/IEC 15444-9.
© ISO/IEC 2016 – All rights reserved v

---------------------- Page: 5 ----------------------
TECHNICAL REPORT ISO/IEC TR 19566-2:2016(E)
Information technologies — JPEG Systems —
Part 2:
Transport mechanisms and packaging
1 Scope
This document collects important information with the goal of elaborating a system layer for JPEG
standards, referred to as JPEG systems.
This document summarizes the principles of incremental codestream and file transport that are
intended to form the future building blocks JPEG systems. Industrial implementations, future codecs
and systems components are encouraged to follow these guidelines.
2 Normative references
There are no normative references in this document.
3 Terms, definitions and abbreviated terms
3.1 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— IEC Electropedia: available at http://www.electropedia.org/
— ISO Online browsing platform: available at https://www.iso.org/obp/
3.1.1
bit stream
partially encoded or decoded sequence of bits comprising an entropy-coded segment
3.1.2
box
structured collection of data describing the image or the image decoding process
Note 1 to entry: See ISO/IEC 19566-1 for the definition of boxes.
3.1.3
box-based file format
file format whose composing elements are well-defined, hierarchically structured boxes
3.1.4
byte
group of 8 bits
3.1.5
coder
embodiment of a coding process
© ISO/IEC 2016 – All rights reserved 1

---------------------- Page: 6 ----------------------
ISO/IEC TR 19566-2:2016(E)

3.1.6
codestream
sequence of bits, representing a compressed image and associated metadata
3.1.7
coding
encoding or decoding
3.1.8
coding model
procedure (3.1.27) used to convert input data into symbols to be coded
3.1.9
(coding) process
general term for referring to an encoding process (3.1.16), a decoding process (3.1.14) or both.
3.1.10
compression
reduction in the number of bits used to represent source image data
3.1.11
component
two-dimensional array of samples (3.1.28) having the same designation in the output or display device
Note 1 to entry: An image typically consists of several components, e.g. red, green and blue.
3.1.12
continuous-tone image
image whose components (3.1.11) have more than 1 bit per sample (3.1.28)
3.1.13
decoder
embodiment of a decoding process (3.1.14)
3.1.14
decoding process
process which takes as its input compressed image data and outputs a continuous-tone image (3.1.12)
3.1.15
encoder
embodiment of an encoding process (3.1.16)
3.1.16
encoding process
process which takes as its input a continuous-tone image (3.1.12) and outputs compressed image data
3.1.17
entropy-coded (data) segment
independently decodable sequence of entropy-encoded bytes of compressed image data
3.1.18
entropy decoder
embodiment of an entropy decoding (3.1.19) procedure
3.1.19
entropy decoding
lossless procedure which recovers the sequence of symbols from the sequence of bits produced by the
entropy encoder (3.1.20)
3.1.20
entropy encoder
embodiment of an entropy encoding (3.1.21) procedure
2 © ISO/IEC 2016 – All rights reserved

---------------------- Page: 7 ----------------------
ISO/IEC TR 19566-2:2016(E)

3.1.21
entropy encoding
lossless procedure which converts a sequence of input symbols into a sequence of bits such that the
average number of bits per symbol approaches the entropy of the input symbols
3.1.22
Joint Photographic Experts Group
JPEG
informal name of the committee that created this document
Note 1 to entry: The “joint” comes from the ITU-T and ISO/IEC collaboration.
3.1.23
JPEG standards
collection of ISO/IEC/ITU standards developed by the Joint Photographic Experts Group (3.1.22) for still
imaging application as listed in Clause 2
3.1.24
JPEG systems
common elements of a system layer for JPEG standards (3.1.23)
3.1.25
metadata
additional data associated with the image data beyond the image data
3.1.26
pixel
collection of sample (3.1.28) values in the spatial image domain having all the same sample coordinates
EXAMPLE A pixel may consist of three samples describing its red, green and blue value.
3.1.27
procedure
set of steps which accomplishes one of the tasks which comprise an encoding (3.1.16) or decoding process
(3.1.14)
3.1.28
sample
one element in the two-dimensional image array which comprises a component (3.1.11)
3.1.29
superbox
box (3.1.2) that itself contains a contiguous sequence of boxes, and only a contiguous sequence of boxes
3.2 Abbreviated terms
ID identifier
JP2 JPEG 2000 file format
JPEG Joint Photographic Experts Group
JPIP JPEG 2000 interactive protocol
JPX extended JPEG 2000 file format
MJ2 motion JPEG 2000 file format
© ISO/IEC 2016 – All rights reserved 3

---------------------- Page: 8 ----------------------
ISO/IEC TR 19566-2:2016(E)

OSI open systems interconnection
ROI region of interest
XML extensible markup language
4 Conventions
4.1 Operators
NOTE Many of the operators used in this document are similar to those used in the C programming language.
4.1.1 Arithmetic operators
+ Addition
− Subtraction (as a binary operator) or negation (as a unary prefix operator)
* Multiplication
/ Division without truncation or rounding
smod x smod a is the unique value y between −⎾(a-1)/2⏋ and ⎿(a-1)/2⏌
for which y + Na = x with a suitable integer N
umod x umod a is the unique value y between 0 and a-1
for which y + Na = y with a suitable integer N
4.1.2 Logical operators
|| Logical OR
&& Logical AND
! Logical NOT
∈ x ∈ {A, B} is defined as (x == A || x == B)
∉ x ∉ {A, B} is defined as (x != A && x != B)
4.1.3 Relational operators
> Greater than
≥ Greater than or equal to
< Less than
≤ Less than or equal to
== Equal to
!= Not equal to
4.1.4 Precedence order of operators
Operators are listed below in descending order of precedence. If several operators appear in the same
line, they have equal precedence. When several operators of equal precedence appear at the same level
4 © ISO/IEC 2016 – All rights reserved

---------------------- Page: 9 ----------------------
ISO/IEC TR 19566-2:2016(E)

in an expression, evaluation proceeds according to the associativity of the operator, either from right to
left or from left to right.
Operators Type of operation Associativity
(), [ ], . Expression Left to right
− Unary negation
*, / Multiplication Left to right
umod, smod Modulo (remainder) Left to right
+, − Addition and Subtraction Left to right
< , >, ≤, ≥ Relational Left to right
4.1.5 Mathematical functions
⎾x⏋ Ceil of x. Returns the smallest integer that is greater than or equal
to x.
⎿x⏌ Floor of x. Returns the largest integer that is lesser than or equal
to x.
|x| Absolute value. Is −x for x < 0, otherwise x.
sign(x) Sign of x. Zero if x is zero, +1 if x is positive, −1 if x is negative.
clamp(x,min,max) Clamps x to the range (min,max). Returns min if x < min,
max if x > max, or otherwise x.
power(x, a) Raises the value of x to the power of a. x is a non-negative real
number, a is a real number. Power(x,a) is equal to exp[a × log(x)]
where exp is the exponential function and log() is the natural
logarithm. If x is zero and a is positive, power(x,a) is defined to be
zero.
5 General
The purpose of this clause is to give an informative overview of the elements referred to in this
document. Another purpose is to introduce many of the terms which are defined in Clause 3. These
terms are printed in italics upon first usage in this clause.
There are three elements referred to in this document.
a) An encoder is an embodiment of an encoding process. An encoder takes as input digital source image
data and encoder specifications, and by means of a specified set of procedures, generates as output a
codestream.
b) A decoder is an embodiment of a decoding process. A decoder takes as input a codestream, and by
means of a specified set of procedures, generates as output digital reconstructed image data.
c) The codestream is a compressed image data representation which includes all necessary data to
allow a (full or approximate) reconstruction of the sample values of a digital image. Additional data
might be required that define the interpretation of the sample data, such as colour space or the
spatial dimensions of the samples.
© ISO/IEC 2016 – All rights reserved 5

---------------------- Page: 10 ----------------------
ISO/IEC TR 19566-2:2016(E)

6 Purpose of the document
Rec. ITU-T T.808 | ISO/IEC 15444-9 standardizes methods for partial access to codestream and image
files, such that a partial representation of the latter is meaningful and can be decoded by a decoder.
This document gives guidelines for design of future technical systems, compression standards and
transmission formats such that partial access to images can be provided in a uniform manner based on
the concepts and ideas of JPIP as defined in Rec. ITU-T T.808 | ISO/IEC 15444-9. Since not all applications
are compatible with the needs of JPIP, these guidelines are informative, although system designers and
compression experts are encouraged to follow them whenever possible. The actual standardization of
transmission formats and packaging rules is not part of this document and will take place in other
documents.
In order to achieve the purpose of the document, Clauses 7 to 11 give an overview about the use cases of
JPIP and its system architecture, generic concepts and capabilities. Clauses 12 and 13 then summarize
the preconditions for applicability of JPIP and explain concepts for achieving this compatibility.
7 Technical rationale — Partial representation of image and metadata content
Rec. ITU-T T.808 | ISO/IEC 15444-9 (JPIP) has been developed and standardized primarily to overcome
the limitations of linear codestreams in addressing applications for the communication of compressed
media. Traditionally, compressed content is organized as a linear stream of bytes (the codestream),
which may then be wrapped in a file format that provides additional metadata for interpreting and
describing the compressed content. ISO/IEC 19566-1 recommends the use of box-based file formats for
this purpose.
In some cases, the information in a codestream or the boxes in a file can be reorganized, without
altering the meaning of the content. Such reorganization can be useful in communication systems. For
example, a JPEG 2000 codestream can be organized in such a way that information appears in order
of increasing spatial resolution, increasing quality/precision of the representation, increasing image
component indices or increasing location along a raster-like scan. Region-of-interest metadata within
a JPX file can be reordered so that items of greater interest appear earlier, again without altering the
fundamental content. In bandwidth-constrained environments, such reorganization may allow the
recipient of the codestream or complete file to obtain more important information earlier than what
might have occurred without the reorganization.
JPIP standardizes mechanisms to provide a much greater level of flexibility in organization than what
can be achieved within the constraints of the linear codestream and file format structures. Specifically,
JPIP standardizes streaming media types that are capable of communicating the fundamental elements
of one or more codestreams and the boxes of a box-structured file, in any order at all. While resequencing
the content to match the preferences of a particular client application can be very useful, in practice,
the greatest benefits arise from using the JPIP streaming media types to intentionally communicate
only a subset of the content. For example, a server might communicate only a subset of the image
components, a limited image resolution or a limited spatial region of interest by using the relevant JPIP
streaming media type to send selected elements from the compressed codestream. Similarly, a server
might communicate only a limited set of region-of-interest descriptions, a limited set of XML boxes or a
limited set of compositing instructions from an animation.
In addition to streaming media types, Rec. ITU-T T.808 | ISO/IEC 15444-9 standardizes the elements of a
client request syntax that can be used to specify regions, resolutions, components, metadata and higher
level semantic entities of interest, in response to which a server can construct the most appropriate
response stream. Many aspects of the JPIP request language are substantially generic in the sense
that they are meaningful regardless of how the media has been compressed. The streaming media
types, however, should be tailored to match the capabilities and fundamental elements of a particular
compression standard. Nonetheless, there are many principles and even detailed design patterns from
the JPIP standard that can and should be followed in the development of future standards.
Elements of the compressed representation and its containing box-structured file are first mapped
to “data-bins”. A JPIP streaming media type then consists of a sequence of “messages”, each of which
communicates a range of bytes from a single data-bin. A JPIP message may hold the entire data-bin,
6 © ISO/IEC 2016 – All rights reserved

---------------------- Page: 11 ----------------------
ISO/IEC TR 19566-2:2016(E)

a prefix of a data-bin or indeed any range of bytes from the data-bin. Each message commences with
a variable-length “message header” that serves to identify the data-bin, number of bytes and start of
the byte range. In certain circumstances, the information in a message header can be encoded relative
to that in previous messages, but apart from this, messages are fully self-describing so that they can
be arbitrarily rearranged without losing any information. JPIP messages can describe overlapping
byte ranges from any given data-bin. These features are valuable in unreliable and/or distributed
communication environments. While JPIP messages are usually streamed over a communication
channel, they can be stored in files, made available for web-based download, delivered via email,
concatenated, selectively discarded and so forth, all of which provide means for communicating partial
representations of the original content that may be of interest to a wide variety of applications and
consumers.
Although any given sequence of JPIP messages may provide only some of the information from the
original content, a key principle in JPIP is that all messages associated with a single “logical target” are
consistent. That is, wherever two messages that describe byte ranges from the same data-bin overlap,
the overlapping bytes provided by those messages should be identical. This is true even where the
messages are sent in different communication sessions, on different days or after receiving different
instructions from a network client. It is conceivable and potentially very useful to develop applications
in which the content itself is created or compressed dynamically, in response to the needs of client
applications, so that some JPIP messages may be delivered to a client before all the content exists. This
is perfectly acceptable, so long as such dynamic generation of content does not lead to changes in any
data-bin bytes that have already been disseminated via JPIP messages.
8 Use cases and requirements
As introduced above, JPIP is intended to address the needs of applications that can benefit from flexible
resequencing or partial communication of the content from a compressed codestream or from a
surrounding box-structured file format. The use cases and requirements for JPIP are derived from such
applications. The primary use cases are as follows:
a) browsing of large images over a network by an interactive user, where interaction may involve
dynamic selection of a region of interest or a spatial resolution of interest;
b) browsing of volumes (e.g. medical volumes) over a network by an interactive user, where interaction
may involve a volumetric region of interest;
c) browsing of animated media over a network by an interactive user, where interaction may involve a
spatial region of interest, temporal region of interest or a spatial resolution of interest;
d) interactive browsing of large images, volumes or animations together with associated metadata
over a network, where interaction may involve specific metadata of interest (e.g. textual labels,
XML documents or region-of-interest metadata);
e) making available a portion (e.g. region of interest, resolution of interest and/or metadata of
interest) of a large image or other source over a network via a suitably expressed Universal
Resource Identifier (URI);
Beyond these primary use cases, some auxiliary use cases that have similar requirements are the
following:
a) storing a portion (e.g. region of interest, resolution of interest and/or metadata of interest) of a
large image or other source in a file;
b) sending a portion (e.g. region of interest, resolution of interest and/or metadata of interest)
or a large image, volume or animated source over a network based on the needs, privileges of
capabilities of the receiving entity or system.
All use cases require means for unambiguously communicating partial content from a compressed
codestream or a box-structured file format in which codestreams may be embedded. Additionally, the
primary use cases listed above require means for specifying the content that is of interest, i.e. a request
© ISO/IEC 2016 – All rights reserved 7

---------------------- Page: 12 ----------------------
ISO/IEC TR 19566-2:2016(E)

language. To make a pre-defined portion of the content available via a URI, the URI itself is required
to embed elements of the request language. For the other primary use cases, a mechanism is required
to transport user requests to a JPIP capable server. For the auxiliary use cases, no request language is
actually required. For use cases that involve requests, the following additional requirements exist.
a) It should be possible to issue a meaningful JPIP request for a region or resolution of interest, without
prior knowledge of the compressed content, its original dimensions or any compression parameters.
This requirement serves the needs of browsing applications that require high responsiveness since
multiple round-trip communications over a network can be avoided. Effectively, this requirement
means that JPIP should support high-level requests.
b) JPIP should provide mechanisms that can be used to avoid redundant transmission of content
in cases where multiple requests identify overlapping content from a compressed codestream
and/or a box-structured file. This requirement is important for any interactive application where
multiple requests are expected, especially since it is not generally possible for the requesting user
or network client to know how to express requests whose response will be disjoint, without having
detailed prior knowledge of the compressed structure or file format organization. The next two
requirements are largely corollaries of this one, since two requests can only be considered to
overlap if they refer to the same source.
c) Source
...