ISO/IEC TR 19566-9:2024

Technical
Report
ISO/IEC TR 19566-9
First edition
Information technology — JPEG
2024-08
Systems —
Part 9:
JPEG extensions mechanisms to
facilitate forwards and backwards
compatibility
Technologies de l'information — Systèmes JPEG —
Partie 9: Mécanismes d'extension JPEG pour faciliter la
compatibilité ascendante et descendante
Reference number
© ISO/IEC 2024
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting on
the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address below
or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
© ISO/IEC 2024 – All rights reserved
ii
Contents Page
Foreword .iv
Introduction .v
1 Scope . 1
2 Normative references . 1
3 Terms, definitions, and abbreviated terms . 1
3.1 Terms and definitions .1
3.2 Abbreviated terms .2
4 Conventions - Operators . 2
4.1 Arithmetic operators .3
4.2 Logical operators .3
4.3 Relational operators .3
4.4 Precedence order of operators .3
4.5 Mathematical functions .4
5 General extensions mechanisms . 4
5.1 General .4
5.2 Extensions mechanisms for codestreams .4
5.3 Extension mechanisms for file formats.6
5.4 File Type box .7
5.5 JPEG Universal Metadata Box Format (JUMBF) .8
5.6 Compressed boxes .8
6 Extension mechanisms for ITU Recommendation T.81 | ISO/IEC 10918-1 (JPEG 1) . 9
6.1 Application markers .9
6.2 APP11 based extensions .9
6.3 JUMBF Boxes .10
7 Extension mechanisms for Rec. ITU-T T.800 | ISO/IEC 15444 (JPEG 2000) series .10
7.1 General .10
7.2 CAP marker . .11
7.3 Profile indicators .11
7.4 COD marker . 12
7.5 File Type Box . 12
7.6 Reader Requirements Box . 12
8 Extension mechanisms for the ISO/IEC 21122 series (JPEG XS) .13
8.1 General . 13
8.2 CAP marker . . 13
8.3 Marker allocation .14
8.4 Profile, level and sublevel indicators .14
8.5 File format . 15
9 Extension mechanisms for ISO/IEC 18181 (JPEG XL) .15
9.1 General . 15
9.2 Codestream extensions . 15
9.3 New component types . 15
9.4 File Type Box .16
Bibliography . 17

© ISO/IEC 2024 – All rights reserved
iii
Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are
members of ISO or IEC participate in the development of International Standards through technical
committees established by the respective organization to deal with particular fields of technical activity.
ISO and IEC technical committees collaborate in fields of mutual interest. Other international organizations,
governmental and non-governmental, in liaison with ISO and IEC, also take part in the work.
The procedures used to develop this document and those intended for its further maintenance are described
in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for the different types
of document should be noted. This document was drafted in accordance with the editorial rules of the ISO/
IEC Directives, Part 2 (see www.iso.org/directives or www.iec.ch/members_experts/refdocs).
ISO and IEC draw attention to the possibility that the implementation of this document may involve the
use of (a) patent(s). ISO and IEC take no position concerning the evidence, validity or applicability of any
claimed patent rights in respect thereof. As of the date of publication of this document, ISO and IEC had not
received notice of (a) patent(s) which may be required to implement this document. However, implementers
are cautioned that this may not represent the latest information, which may be obtained from the patent
database available at www.iso.org/patents and https://patents.iec.ch. ISO and IEC shall not be held
responsible for identifying any or all such patent rights.
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and expressions
related to conformity assessment, as well as information about ISO's adherence to the World Trade
Organization (WTO) principles in the Technical Barriers to Trade (TBT) see www.iso.org/iso/foreword.html.
In the IEC, see www.iec.ch/understanding-standards.
This document was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 29, Coding of audio, picture, multimedia and hypermedia information .
A list of all parts in the ISO/IEC 19566 series can be found on the ISO and IEC websites.
Any feedback or questions on this document should be directed to the user’s national standards
body. A complete listing of these bodies can be found at www.iso.org/members.html and
www.iec.ch/national-committees.

© ISO/IEC 2024 – All rights reserved
iv
Introduction
This document collects guiding principles on how standards discussed in ISO/IEC TR 19566-1 provide
mechanisms for forwards compatibility, so called extension mechanisms, both on the basis of the codestream
and the file format and lists specific implementations of these guiding principles in particular standards.
The purpose of this document is to provide documentation on these principles for the preparation of future
extensions of these standards, and to ensure consistency of extension principles amongst standards.

© ISO/IEC 2024 – All rights reserved
v
Technical Report ISO/IEC TR 19566-9:2024(en)
Information technology — JPEG Systems —
Part 9:
JPEG extensions mechanisms to facilitate forwards and
backwards compatibility
1 Scope
This document summarizes mechanisms by which existing file formats and codestreams, such as those
specified in ISO/IEC TR 19566-1, can be extended in a forward and backward-compatible way.
2 Normative references
The following documents are referred to in the text in such a way that some or all of their content constitutes
requirements of this document. For dated references, only the edition cited applies. For undated references,
the latest edition of the referenced document (including any amendments) applies.
Rec. ITU-T T.800 | ISO/IEC 15444-1, Information technology — JPEG 2000 image coding system — Part 1: Core
coding system
Rec. ITU-T T.801 | ISO/IEC 15444-2, Information technology — JPEG 2000 image coding system — Part 2:
Extensions
Rec. ITU-T T.802 | ISO/IEC 15444-3, Information technology — JPEG 2000 image coding system — Part 3:
Motion JPEG 2000
ISO/IEC 18181-1, Information technology — JPEG XL image coding system — Part 1: Core coding system
ISO/IEC TR 19566-1, Information technology — JPEG Systems — Part 1: Packaging of information using
codestreams and file formats
ISO/IEC 21122-1, Information technology — JPEG XS low-latency lightweight image coding system — Part 1:
Core coding system
3 Terms, definitions, and abbreviated terms
3.1 Terms and definitions
For the purposes of this document, the terms and definitions given in Rec. ITU-T T.800 | ISO/IEC 15444-1, Rec.
ITU-T T.801 | ISO/IEC 15444-2, Rec. ITU-T T.802 | ISO/IEC 15444-3, ISO/IEC 18181-1, ISO/IEC TR 19566-1,
ISO/IEC 21122-1 and the following apply.
ISO and IEC maintain terminology databases for use in standardization at the following addresses:
— ISO Online browsing platform: available at https:// www .iso .org/ obp
— IEC Electropedia: available at https:// www .electropedia .org/

© ISO/IEC 2024 – All rights reserved
3.1.1
codestream
compressed image data representation which includes all necessary data to allow a (full or approximate)
reconstruction of the sample values of a digital image, which can require additional data to define the
interpretation of the sample data, such as colour space or the spatial dimensions of the samples
3.1.2
file format
encapsulation of a codestream (3.1.1) in a file, that is in a form that allows random access from a file
system of a computer system, along with additional metadata that define the interpretation of the samples
reconstructed by the codestream within
3.1.3
encoder
embodiment of an encoding process, which takes digital source image data and encoder specifications as
input and, by means of a specified set of procedures, generates a codestream (3.1.1) or a file as output
3.1.4
data producer
encoder generating file formats (3.1.2)
3.1.5
decoder
embodiment of a decoding process, which takes a codestream or a file as input and, by means of a specified set
of procedures, generates digital reconstructed image data as output
3.1.6
data consumer
decoder capable of parsing file formats
3.1.7
forward compatibility
design principle allowing future extensions in existing codestreams or files
3.1.8
backward compatibility
design principle for extensions which allows existing decoders to successfully reconstruct data (i.e. images)
from extended codestreams and files, albeit some information embedded in these codestreams or files are
potentially not be fully accessible to them
3.2 Abbreviated terms
ID Identifier
JP2 JPEG 2000 file format
JPEG Joint Photographic Experts Group
JPIP JPEG 2000 Interactive Protocol
XML Extensible Markup Language
4 Conventions - Operators
NOTE Many of the operators used in this document are similar to those used in the C programming language.

© ISO/IEC 2024 – All rights reserved
4.1 Arithmetic operators
+ Addition
− Subtraction (as a binary operator) or negation (as a unary prefix operator)
* Multiplication
/ Division without truncation or rounding.
x smod a is the unique value y between - ()a1− /2 and ()a−12/
smod    
   
for which y+Na = x with a suitable integer N.
umod x mod a is the unique value y between 0 and a-1
for which y+Na = y with a suitable integer N.
4.2 Logical operators
|| Logical OR
&& Logical AND
! Logical NOT
∈ x ∈ {A, B} is defined as (x == A || x == B)
∉ x ∉ {A, B} is defined as (x != A && x != B)
4.3 Relational operators
> Greater than
>= Greater than or equal to
< Less than
<= Less than or equal to
== Equal to
!= Not equal to
4.4 Precedence order of operators
Operators are listed below in descending order of precedence. If several operators appear in the same line,
they have equal precedence. When several operators of equal precedence appear at the same level in an
expression, evaluation proceeds according to the associativity of the operator, either from right to left or
from left to right.
Operators Type of operation Associativity
(), [ ], . Expression Left to Right
− Unary negation
*, / Multiplication Left to Right
umod, smod Modulo (remainder) Left to Right
+, − Addition and Subtraction Left to Right
< , >, <=, >= Relational Left to Right

© ISO/IEC 2024 – All rights reserved
4.5 Mathematical functions
x
  Ceil of x. Returns the smallest integer that is greater than or equal to x.
 
x
  Floor of x. Returns the largest integer that is lesser than or equal to x.
 
|x| Absolute value, is –x for x < 0, otherwise x.
sign(x) Sign of x, zero if x is zero, +1 if x is positive, -1 if x is negative.
clamp(x,min,max) Clamps x to the range [min,max]: returns min if x < min, max if x > max or otherwise x.
power(x,a) Raises the value of x to the power of a. x is a non-negative real number, a is a real number.
Power(x,a) is equal to exp(a×log(x)) where exp is the exponential function and log()
the natural logarithm. If x is zero and a is positive, power(x,a) is defined to be zero.
5 General extensions mechanisms
5.1 General
This clause lists general best practices for cross-standard extensions mechanism on the codestream and on
the file format level.
5.2 Extensions mechanisms for codestreams
As specified in ISO/IEC TR 19566-1, codestreams consist of multiple markers and marker segments.
While markers stand alone, marker segments include a size and following data that allow readers of such
codestreams to skip over data they do not intend to interpret or do not understand.
Generally, the following two types of marker segments are present:
— A capability marker segment that allows registrations of future extensions. Such a marker segment allows
a reader of a particular codestream to identify which extensions are required for a successful decode,
and to abort decoding if it cannot provide the required extensions. That is, capability marker segments
provide early-out conditions to decoders.
— Purely informative comment marker segments that contain additional information a decoder can safely
skip over without compromising the decoding process, but that can be informative to the user of the
media reconstructed by the codestream. Such marker sequences can, for example, embed information
on the software used to create the codestream. A decoder can potentially use this information to enable
workarounds for known vendor-specific defects or display such information upon request of its user.
— Comment markers can be further classified into vendor-specific information, and vendor neutral
information.
Marker sequences often contain bit-fields that enable or disable particular functionalities of a decoder. As
the size of such bit fields is often aligned to multiples of 8 bits, a particular edition of a standard will not
always not require all such bits. Unused bits are reserved for future use by ITU-T/ISO/IEC purposes, and
require encoders to write them as 0. Depending on application, decoders can be either required to abort on
bits that are defined as reserved or ignore them. It can be advisable to reserve one bit (e.g. the topmost bit of
such a bit field) to extend the size of the bit field in future generations of a standard.
The same bitfield can be defined within multiple parts of a series of standards, with some bits only
applicable to a specific part of a standard. It is advisable that those bits within the bitfield that are not used
within a particular standard are marked as “Reserved for ISO/IEC purposes”, and that their default values
are selected as 0 such that an encoder conforming to a part of a standard series writes them as 0 without
compromising the codestream for decoders that support multiple parts of the same series.
The following example lists a bitfield whose topmost bit is used to signal a syntax extension whose origin is
signalled by bits 2 and 3, if it is set, and which is 0 if base signalling is used. For base signalling, bits 6 to 2

© ISO/IEC 2024 – All rights reserved
are currently unused and reserved, and for extended signalling, bits 6 to 4 are reserved. Bits 0 and 1 signal
optional features by the selector bits 7 and (optionally) bits 2 and 3. Assume further that the base signalling
is specified in one part of the standard, and the extension signalling in a further part.
Then, the specification of the bitfield in the base system would read as shown in Table 1.
Table 1 — Example of a bit field (base part)
Field value Function
0000 00x0
Disable feature A
0000 00x1
Enable feature A
0000 000x
Disable feature B
0000 001x
Enable feature B
All other combinations Reserved
In particular, by this bit combinations that employ bits for extended signalling are marked as “Reserved” in
the base part. Assume further that the extension part of the standard also employs “feature A” and “feature
B” which are mutually exclusive to features C and D of the extended. The extension part would then define a
meaning for such extension bits, while preserving the meaning of the existing bits as shown in Table 2.
Table 2 — Example of a bit field (extension part)
Field value Function
0000 00x0
Disable feature A
0000 00x1
Enable feature A
0000 000x
Disable feature B
0000 001x
Enable feature B
1000 01x0
Disable feature C
1000 01x1
Enable feature C
1000 010x
Disable feature D
1000 011x
Enable feature D
All other combinations Reserved
If, however, the extended features C and D of the extension part are orthogonal to features A and B of the
base part, the above signalling cannot be used and instead the extension part will allocate additional bits
from the originally reserved bit set for its purpose. In such a case, the specification of the bit field can read
as shown in Table 3.
Table 3 — Example of a bit field (extension part)
Field value Function
x0xx xxx0
Disable feature A
x0xx xxx1
Enable feature A
x0xx xx0x
Disable feature B
x0xx xx1x
Enable feature B
10x0 01xx
Disable feature C
10x1 01xx
Enable feature C
100x 01xx
Disable feature D
101x 01xx
Enable feature D
All other combinations Reserved
In this definition, bits 2 and 3 would still identify the part within which the extended features C and D
are defined, whereas the features itself are signalled by bits 4 and 5. Yet another part of the same series

© ISO/IEC 2024 – All rights reserved
of specification can then define features E and F, which are mutually exclusive to features C and D, but
orthogonal to A and B of the base part as shown in Table 4.
Table 4 — Example of a bit field (second extension part)
Field value Function
x0xx xxx0
Disable feature A
x0xx xxx1
Enable feature A
x0xx xx0x
Disable feature B
x0xx xx1x
Enable feature B
10x0 10xx
Disable feature E
10x1 10xx
Enable feature E
100x 10xx
Disable feature F
101x 10xx
Enable feature F
All other combinations Reserved
The bits 2 and 3 are here different from the table specified in the first extension, allowing decoders to
distinguish between the feature set C,D of the first extension, and feature set E,F of the second extension.
5.3 Extension mechanisms for file formats
It is advisable to base file format standards on the box-based file format as for example JPX, specified in Rec.
ITU-T T.801 | ISO/IEC 15444-2, or ISOBMFF, as contained in ISO/IEC 14496-12. Both file formats provide
syntax elements known as boxes, see ISO/IEC TR 19566-1 for further details, and the encoding of boxes is
identical in both specifications. It is generally advisable to specify a box-based file format in such a way that
decoders are instructed to skip over boxes they do not implement.
In addition, it is advisable to equip standards with the following specific boxes:
— A signature box as first box in the file format that allows to identify the family of standards a file conforms to.
— A file type box that identifies those standards the contents are compatible to. A reader implementing one
of the listed standards in the compatibility list of standards is expected to make productive use of the
contents of the file. A file type box enables decoders to abort decoding quickly, and/or allows applications
to select a decoder suitable for a particular file. Such file type boxes can list multiple standards or
profiles of standards, and as such also extensions to standards that would be required for a partial or
full decoding.
Rec. ITU-T T.801 | ISO/IEC 15444-2 defines a structure for a file type box using a ‘ftyp’ as box type that
is encouraged to be used in new specifications. It provides “summary” level information to help the user
determine whether it can read and make productive use of the file. Subclause 5.4 provides additional
guidelines on the contents of this box.
— File formats based on the JPX format carry an Image Header box of type ‘ihdr’ that, along with the
image dimensions and number of channels, also carry a compression type C that identifies the type of
the codestream within the file. It is advisable to keep this codestream identifier unique and consistent
throughout specifications. Registration of these codestream formats is currently through the definition
of JPX, within Rec. ITU-T T.801 |ISO/IEC 15444-2. The value to be used for this field is specified in the
corresponding part of the standard defining a JPX based file format. The compression types C are kept
up to date in Rec. ITU-T T.801 | ISO/IEC 15444-2.
— The codestream carrying the encoded sample values is wrapped in a Contiguous Codestream box
of type ‘jp2c’. The box type does not depend on the codestream contained in the box, i.e. it is always
‘jp2c’ even if the codestream is not based on Rec. ITU-T T.80x | ISO/IEC 15444-x. Instead, identification
of the codestream syntax is through the C field of the Image Header box associated to the particular
codestream.
© ISO/IEC 2024 – All rights reserved
— It is advisable to place vendor-specific extensions in UUID boxes. This box type identifies extension data
by means of a 128-bit identifier. UUIDs to identify vendor-specific data are self-registered and do not
require registration through a standardization organization. For example, the uuidgen command line
tool available on many operating systems can be used to create a unique UUID.
— Another mechanism by which generic metadata, and in particular also vendor specific metadata can
be embedded into a file is through XML boxes. Such boxes contain UTC encoded XML data following a
particular XML dialect.
— Finally, ISO/IEC 19566-5 defines the JPEG Universal Metadata Box Format (JUMBF). JUMBF specifies a
metadata embedding framework that is format agnostic. JUMBF defines a generic container, compatible
with ISOBMFF, that can be used to embed domain specific metadata that can be textual or binary, including
media content. JUMBF also defines functionalities that facilitate linking between the metadata elements
themselves and media content. It also specifies a URL schema that can be used for external references
and/or requests into a media asset. Subclause 7.4 illustrates the JUMBF structure in more detail.
5.4 File Type box
The box structure and the ‘ftyp’ box type inform the user that the file is constructed as a set of boxes and
can be parsed as such. All files include the File Type box behind the signature box, as recognizing the File
Type box allows the reader to catalog the file, even if it does not recognize most of the content of the file (e
...