ISO/IEC 21794-1:2020

INTERNATIONAL ISO/IEC
STANDARD 21794-1
First edition
2020-08
Information technology — Plenoptic
image coding system (JPEG Pleno) —
Part 1:
Framework
Technologies de l'information — Système de codage d'images
plénoptiques (JPEG Pleno) —
Partie 1: Cadre
Reference number
ISO/IEC 21794-1:2020(E)
©
ISO/IEC 2020

---------------------- Page: 1 ----------------------
ISO/IEC 21794-1:2020(E)

COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2020
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting
on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address
below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii © ISO/IEC 2020 – All rights reserved

---------------------- Page: 2 ----------------------
ISO/IEC 21794-1:2020(E)

Contents Page
Foreword .iv
Introduction .v
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 Abbreviated terms . 3
5 Conventions . 3
5.1 Conformance language . 3
5.2 Naming conventions for numerical values . 3
6 Framework definition . 4
7 File format architecture . 5
8 Organization of the document . 5
Annex A (normative) JPEG Pleno file format (JPL) . 6
Annex B (informative) JPEG Pleno reference grid system .17
Annex C (informative) Conceptual example .18
Bibliography .20
© ISO/IEC 2020 – All rights reserved iii

---------------------- Page: 3 ----------------------
ISO/IEC 21794-1:2020(E)

Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that
are members of ISO or IEC participate in the development of International Standards through
technical committees established by the respective organization to deal with particular fields of
technical activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other
international organizations, governmental and non-governmental, in liaison with ISO and IEC, also
take part in the work.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for
the different types of document should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www .iso .org/ directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject
of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent
rights. Details of any patent rights identified during the development of the document will be in the
Introduction and/or on the ISO list of patent declarations received (see www .iso .org/ patents) or the IEC
list of patent declarations received (see https:// patents .iec .ch).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to the
World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see www .iso .org/
iso/ foreword .html.
This document was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 29, Coding of audio, picture, multimedia and hypermedia information.
A list of all parts in the ISO/IEC 21794 series can be found on the ISO website.
iv © ISO/IEC 2020 – All rights reserved

---------------------- Page: 4 ----------------------
ISO/IEC 21794-1:2020(E)

Introduction
This document is part of a series of standards for a system known as JPEG Pleno. This document defines
the JPEG Pleno framework. It facilitates the capture, representation, exchange and visualization of
plenoptic imaging modalities. A plenoptic image modality can be a light field, point cloud or hologram,
which are sampled representations of the plenoptic function in the form of, respectively, a vector
function that represents the radiance of a discretized set of light rays, a collection of points with
position and attribute information, or a complex wavefront. The plenoptic function describes the
radiance in time and in space obtained by positioning a pinhole camera at every viewpoint in 3D spatial
coordinates, every viewing angle and every wavelength, resulting in a 7D function.
JPEG Pleno specifies tools for coding these modalities while providing advanced functionality at system
level, such as support for data and metadata manipulation, editing, random access and interaction,
protection of privacy and ownership rights.
© ISO/IEC 2020 – All rights reserved v

---------------------- Page: 5 ----------------------
INTERNATIONAL STANDARD ISO/IEC 21794-1:2020(E)
Information technology — Plenoptic image coding system
(JPEG Pleno) —
Part 1:
Framework
1 Scope
This document specifies the plenoptic image coding system framework architecture and its instantiation
via a generic file format for storage of plenoptic modalities as well as associated metadata descriptors.
2 Normative references
The following documents are referred to in the text in such a way that some or all of their content
constitutes requirements of this document. For dated references, only the edition cited applies. For
undated references, the latest edition of the referenced document (including any amendments) applies.
ISO/IEC 646, Information technology — ISO 7-bit coded character set for information interchange
ISO/IEC 15444-1:2019, Information technology — JPEG 2000 image coding system — Part 1: Core
coding system
ISO/IEC 15444-2:2004, Information technology — JPEG 2000 image coding system — Part 2: Extensions
1)
ISO/IEC 21794-2:—, Information technology — Plenoptic image coding system (JPEG Pleno) — Part 2:
Light field coding
3 Terms and definitions
For the purposes of this document the following terms and definitions apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— ISO Online browsing platform: available at https:// www .iso .org/ obp
— IEC Electropedia: available at http:// www .electropedia .org/
3.1
big-endian
byte ordering for which the most significant byte and least significant byte are sequentially ordered
from lower memory address to higher memory address, respectively
3.2
box
structured collection of data describing the image or the image decoding process
3.3
box content
data wrapped within the box structure
1) Under preparation. Stage at time of publication: ISO/IEC DIS 21794-2:2019.
© ISO/IEC 2020 – All rights reserved 1

---------------------- Page: 6 ----------------------
ISO/IEC 21794-1:2020(E)

3.4
box type
kind of information stored within the box
3.5
coder
embodiment of a coding process
3.6
codestream
coded data representation that includes all necessary data to allow a (full or approximate)
reconstruction of the sample values of a digital image
3.7
coding process
encoding process, decoding process, or both
3.8
complex wavefront
wavefront represented with a complex representation, which can be, for example, real-imaginary or
amplitude-phase
3.9
component
two-dimensional array of samples having the same designation in the output or display device, e. g.,
red, green or blue
3.10
decoder
embodiment of a decoding process
3.11
decoding process
process that takes as its input a codestream and outputs a continuous-tone image
3.12
encoder
embodiment of an encoding process
3.13
encoding process
process that takes as its input a continuous-tone image and outputs a codestream
3.14
hologram
sampled representation of the plenoptic function in the form of a complex wavefront
3.15
holographic display
three-dimensional display that renders a complex optical wavefront
3.16
light field
sampled representation of the plenoptic function in the form of a vector function that represents the
radiance of a discretized set of light rays
3.17
light field data
recorded light field
2 © ISO/IEC 2020 – All rights reserved

---------------------- Page: 7 ----------------------
ISO/IEC 21794-1:2020(E)

3.18
plenoptic function
radiance in time and in space obtained by positioning a pinhole camera at every viewpoint in 3D spatial
coordinates, every viewing angle and every wavelength, resulting in a 7D function
3.19
plenoptic data
sampled representation of the plenoptic function
Note 1 to entry: Example plenoptic functions are light field, point cloud and holographic representation.
3.20
plenoptic object
plenoptic element
plenoptic data representing a part of the scene described by the plenoptic function
3.21
point cloud
sampled representation of the plenoptic function in the form of collection of points with position and
attribute information
3.22
superbox
box that carries other boxes as payload data
3.23
wavefront
locus of spatial points that share the same phase of the light wave
4 Abbreviated terms
3D three dimensional
CGH computer-generated holography
IPR intellectual property rights
JPEG Joint Photographic Experts Group
JPL Jpeg PLeno file format
UUID Universally Unique IDentifier
XML eXtensible Markup Language
5 Conventions
5.1 Conformance language
The keyword "reserved" indicates a provision that is not specified at this time, shall not be used, and
may be specified in the future. The keyword "forbidden" indicates "reserved" and in addition indicates
that the provision will never be specified in the future.
5.2 Naming conventions for numerical values
Integer numbers are expressed as bit patterns, hexadecimal values or decimal numbers. Bit patterns
and hexadecimal values have both a numerical value and an associated particular length in bits.
© ISO/IEC 2020 – All rights reserved 3

---------------------- Page: 8 ----------------------
ISO/IEC 21794-1:2020(E)

Hexadecimal notation, indicated by prefixing the hexadecimal number by "0x", is used instead of binary
notation to denote a bit pattern having a length that is an integer multiple of 4. For example, 0x41
represents an eight-bit pattern having only its second most significant bit and its least significant bit
equal to 1. An octal notation is indicated by prefixing the octal value “\”, this notation is used instead
of binary notation to denote a bit pattern having a length that is an integer multiple of 3. Numerical
values that are specified under a "Code" heading in tables that are referred to as "code tables" are bit
pattern values (specified as a string of digits equal to 0 or 1 in which the left-most bit is considered the
most-significant bit). Other numerical values not prefixed by "0x" are decimal values. When used in
expressions, a hexadecimal value is interpreted as having a value equal to the value of the corresponding
bit pattern evaluated as a binary representation of an unsigned integer (i.e., as the value of the number
formed by prefixing the bit pattern with a sign bit equal to 0 and interpreting the result as a two's
complement representation of an integer value). For example, the hexadecimal value 0xF is equivalent
to the 4-bit pattern '1111' and is interpreted in expressions as being equal to the decimal number 15.
6 Framework definition
The plenoptic image coding system (JPEG Pleno) specifies a framework that facilitates the capture,
representation and exchange of point cloud, light field and holographic imaging modalities. These
[1]
imaging modalities are light representations of the plenoptic function , regardless of which modality
was used to capture or create parts of the entire content:
— Point clouds are a set of data points representing the plenoptic function in a given, often 3D,
coordinate system with associated attributes;
— Light fields are a sampled representation of the plenoptic function by capturing it by either an
array of cameras (resulting in wide baseline light field data) or by a single light field camera that
uses microlenses to sample individual rays of light that contribute to the final image (resulting in
narrow baseline light field data);
— Holograms are a sampled interferometric representation of the plenoptic function that is based on
a wave-based light propagation model.
This framework supports conversations between different modalities throughout an end-to-end
processing chain. This document defines file format and codestream syntax for plenoptic modalities
while providing advanced functionalities at system level. It also supports data and metadata
manipulation, editing, random access and interaction, protection of privacy and ownership.
Figure 1 — JPEG Pleno framework
4 © ISO/IEC 2020 – All rights reserved

---------------------- Page: 9 ----------------------
ISO/IEC 21794-1:2020(E)

The ISO/IEC 21794 series consists of the following Parts (Figure 1).
— This document outlines the overall JPEG Pleno framework and specifies the generic JPEG Pleno
file format.
— ISO/IEC 21794-2 specifies the file format and codestream syntax for coded light fields. Moreover, it
specifies output of JPEG Pleno light field encoders and provides documentation on the encoding and
decoding procedures.
2)
— ISO/IEC 21794-3 specifies conformance testing protocols.
2)
— ISO/IEC 21794-4 specifies reference software.
7 File format architecture
This document shares common definitions for the structure of files (a sequence of objects, called boxes
here, and atoms in other similar file formats), and a common definition of the general structure of an
object (the size and type).
The specification of the file format in Annex A is built upon the structure of ISO/IEC 15444-1 and
ISO/IEC 15444-2 file formats.
All these specifications require that readers ignore objects that are unrecognizable to them.
This document shall take precedence over those on which it is based, in any case where there are
differences or conflicts; however, no such conflicts are known to exist.
For better readability and understanding the syntax description for the different file formats is done in
the same way as in the base formats.
8 Organization of the document
Annex A specifies the JPEG Pleno box-based file format, which includes definitions of boxes to carry
light field, point cloud and holographic data. Every JPEG Pleno file shall conform to the specified file
format syntax.
Annex B specifies the JPEG Pleno reference grid system, a coordinate system that allows for positioning
plenoptic modalities in a global reference grid system, while using a local coordinate system per
plenoptic element to facilitate efficient local addressing.
Annex C provides an example of how plenoptic objects can be signalled using the JPEG Pleno framework.
2) Under preparation.
© ISO/IEC 2020 – All rights reserved 5

---------------------- Page: 10 ----------------------
ISO/IEC 21794-1:2020(E)

Annex A
(normative)

JPEG Pleno file format (JPL)
A.1 General
This annex defines a still image file format that applications may choose to wrap a codestream based
on a JPEG Pleno encoded image. This document is based on the same syntax as the box-based file format
for JPEG 2000 in ISO/IEC 15444-1:2019, Annex I or ISO/IEC 15444-2:2004, Annex M.
This annex:
— specifies a binary container (file) for both light field and metadata;
— specifies a mechanism to indicate light field properties, such as the tone scale or colour space of
the image;
— specifies a mechanism by which readers may recognize the existence of intellectual property rights
information in the file;
— specifies a mechanism by which metadata (including vendor-specific information) can be included
in files specified by this document.
A.2 Specification of the JPL file format
A.2.1 General
The JPL file format provides a foundation for storing application specific data (metadata) in association
with a JPEG Pleno codestream, such as information which is required to display the light field. As many
applications require a similar set of information to be associated with the coded light field data, it is
useful to define the format of that set of data along with the definition of the coding technology and
codestream syntax.
Conceptually, the JPL file format encapsulates the JPEG Pleno codestream along with other core pieces
of information about that codestream. The building-block of the JPL file format is called a "box". All
information contained within the JPL file is encapsulated in boxes. This document defines several types
of boxes; the definition of each specific box type defines the kinds of information that may be found
within a box of that type. Some boxes will be defined to contain other boxes.
A.2.2 File identification
JPL files can be identified using several mechanisms. When stored in traditional computer file systems,
JPL files should be given the file extension ".jpl" (readers should allow mixed case for the alphabetic
characters).
A.2.3 File organization
A JPL file represents a collection of boxes. Some of those boxes are independent, and some of those
boxes contain other boxes, denoted here as "superboxes". The binary structure of a file is a contiguous
sequence of boxes. The start of the first box shall be the first byte of the file, and the last byte of the last
box shall be the last byte of the file.
The binary structure of a box is identical to ISO/IEC 15444-1 and defined in Annex A.3.
6 © ISO/IEC 2020 – All rights reserved

---------------------- Page: 11 ----------------------
ISO/IEC 21794-1:2020(E)

The conceptual structure of a JPL file is shown in Figure A.1. Boxes with dashed borders are optional in
conforming JPL files. However, an optional box may define mandatory boxes within that optional box.
In that case, if the optional box exists, those mandatory boxes within the optional box shall exist. If the
optional box does not exist, then the mandatory boxes within those boxes shall also not exist.
Figure A.1 specifies only the containment relationship between the boxes in the file.
The order of the boxes is not completely arbitrary and shall conform to the following rules:
— The JPEG Pleno Signature box shall be the first box in a JPL file and the File Type box (defined in
ISO/IEC 15444-1) shall immediately follow the JPEG Pleno Signature box.
— The JPEG Pleno Thumbnail box shall be signalled before the JPEG Pleno Light Field, JPEG Pleno Point
Cloud, and JPEG Pleno Hologram superboxes.
— A JPL file can contain an optional XML box signalling catalog information (see A.5.8). An XML box
containing catalogue information should be signalled after the File Type box and before the first
superbox containing plenoptic data.
— The JPEG Pleno Light Field, JPEG Pleno Point Cloud, and JPEG Pleno Hologram superboxes signalling
plenoptic data, shall be signalled as one monolithic block with no preferred ordering, and no other
boxes shall be signalled in between.
The file in Figure A.1 is a strict sequence of boxes. Other boxes may be found between the boxes defined
in this document, except the JPEG Pleno Light Field, JPEG Pleno Point Cloud, and JPEG Pleno Hologram
superboxes. However, all information contained within a JPL file shall be in the box format; byte-
streams not in the box format shall not be found in the file.
A JPL file may also contain an IPR box and one or more XML, UUID and UUID Info boxes. If present these
boxes shall be as defined in ISO/IEC 15444-1.
© ISO/IEC 2020 – All rights reserved 7

---------------------- Page: 12 ----------------------
ISO/IEC 21794-1:2020(E)

Figure A.1 — Conceptual structure of a JPL file
A.2.4 Metadata
One important aspect of the JPL file format is the ability to add metadata to a JPL file. Some of the boxes
provide a set of tools by which applications can add vendor-specific information to the JPL file format,
like the XML box. These boxes are optional in conforming files and may be ignored by conforming
readers.
A.2.5 Conformance with the file format
All conforming files shall contain all boxes required by this document, and those boxes shall be as
defined in this document. Also, all conforming readers shall correctly interpret all required boxes
defined in this document and thus shall correctly interpret all conforming files.
Because all information is encapsulated in boxes, and all boxes have types, the format provides a
simple mechanism for a reader to extract relevant information, while ignoring any box that contains
8 © ISO/IEC 2020 – All rights reserved

---------------------- Page: 13 ----------------------
ISO/IEC 21794-1:2020(E)

information that is not understood by that particular reader. In this way, new boxes can be created,
either through this document or other International Standards or private implementation. Also, any
new box added to a JPL file shall not change the visual appearance of the image.
A.3 Concept of boxes
A.3.1 Key to graphical descriptions
Each box is described in terms of its function, usage and length. The function describes the information
contained in the box. The usage describes the logical location and frequency of this box in the file.
These descriptions are followed by a figure that shows the order and relationship of the parameters
in the box. Figure A.2 provides an example. A rectangle is used to indicate the parameters in the box.
The width of the rectangle is proportional to the number of bytes in the parameter. A shaded rectangle
(diagonal stripes) indicates that the parameter is of varying size. Two parameters with superscripts
and a grey area between them indicate a run of several of these parameters. A sequence of two groups
of multiple parameters with superscripts separated by a grey area indicates a run of that group of
parameters (one set of each parameter in the group, followed by the next set of each parameter in the
group). Optional parameters or boxes will be shown with a dashed rectangle.
Figure A.2 — Example of a box description figure
The figure is followed by a list that describes the meaning of each parameter in the box. If parameters
are repeated, the length and nature of the run of parameters is defined. As an example, in Figure A.2,
0 N–1
parameters C, D, E and F are 8-, 16-, 32-bit and variable length respectively. The notation G and G
i i i
implies that there are N different parameters, G , in a row. The parameters pairs H and J , where i goes
0 0 1 1
from 0 to M-1, specify that the box will contain H , followed by J , followed by H and J , continuing to
M–1 M–1
H and J (M instances of each parameter in total). Finally, the field L is optional and may not be
found in this box.
After the list is a table that either describes the allowed parameter values or provides references to
other tables that describe these values.
Some boxes may carry other boxes as payload data. Such boxes are denoted as superboxes. The payload
size of a superbox is given by the sum of the box lengths of all the boxes it contains.
In addition, in a figure describing the contents of a superbox, an ellipsis (…) will be used to indicate that
the contents of the file between two boxes are not specifically defined. Any box (or sequence of boxes),
unless otherwise specified by the definition of that box, may be found in place of the ellipsis.
For example, the superbox shown in Figure A.3 must contain an AA box and a BB box, and the BB box
must follow the AA box. However, there may be other boxes found between boxes AA and BB. Dealing
with unknown boxes is discussed in A.6.
© ISO/IEC 2020 – All rights reserved 9

---------------------- Page: 14 ----------------------
ISO/IEC 21794-1:2020(E)

Figure A.3 — Example of the superbox description figures
A.3.2 Box definition
Physically, each object in the file is encapsulated within a binary structure called a box. That binary
structure is shown in Figure A.4 and Table A.1.
Figure A.4 — Organization of a box
LBox Box length. This field specifies the length of the box in multiples of bytes, stored as a
4-byte big-endian unsigned integer. This value includes all of the fields of the box, includ-
ing the length and type. If the value of this field is 1, then the XLBox field shall exist and
the value of that field shall be the actual length of the box. If the value of this field is 0, then
the length of the box was not known when the LBox field was written. In this case, this box
contains all bytes up to the end of the file. If a box of length 0 is contained within another
box (its superbox), then the length of that superbox shall also be 0. This means that this
box is the last box in the file. The values 2-7 are reserved for future use by ISO/IEC.
TBox Box type. This field specifies the type of information found in the DBox field. The value of
this field is encoded as a 4-byte big-endian unsigned integer. However, boxes are gener-
ally referred to by an ISO/IEC 646 character string translation of the integer value. For
all box types defined within this document, box types will be indicated as both character
string and as 4-byte hexadecimal integers. Also, a space character is shown in the charac-
ter string translation of the box type as “\040”. All values of TBox not defined within this
document are reserved for future use by ISO/IEC.
XLBox Box extended length. This field specifies the actual length of the box if the value of the
LBox field is 1. This field is stored as an 8-byte big-endian unsigned integer. The value
includes all of the fields of the box, including the LBox, TBox and XLBox fields.
DBox Box contents. This field contains the actual information contained within this box. The
format of the box contents depends on the box type and will be defined individually for
each type. While variable length fields can be held with DBox, the length of DBox shall be
an integer multiple of octets. Bit-padding shall be provided to ensure this.
Table A.1 — Binary structure of a box
Field name Size (bits) Value
0, 1, or
LBox 32
32
8 to (2 – 1)
TBox 32 Variable
64
64 16 to (2 – 1); if LBox = 1
XLBox
0 Not applicable; if LBox ≠ 1
DBox Variable Variable
For example, consider the illustration in Figure A.5 of a sequence of boxes, including one box that
contains other boxes:
10 © ISO/IEC 2020 – All rights reserved

---------------------- Page: 15 ----------------------
ISO/IEC 21794-1:2020(E)

Figure A.5 — Illustration of box lengths
As shown in Figure A.5, the length of each box includes any boxes contained within that box. For
example, the length of Box 1 includes the length of Boxes 2 and 3, in addition to the LBox and TBox fields
for Box 1 itself. In this case, if the type of Box 1 was
...