ISO/IEC 21794-2:2021

INTERNATIONAL ISO/IEC
STANDARD 21794-2
First edition
2021-04
Information technology — Plenoptic
image coding system (JPEG Pleno) —
Part 2:
Light field coding
Technologies de l'information — Système de codage d'images
plénoptiques (JPEG Pleno) —
Partie 2: Codages des champs de lumière
Reference number
©
ISO/IEC 2021
© ISO/IEC 2021
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting
on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address
below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii © ISO/IEC 2021 – All rights reserved

Contents Page
Foreword .iv
Introduction .v
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 Symbols and abbreviated terms . 3
4.1 Symbols . 3
4.2 Abbreviated terms . 7
5 Conventions . 8
5.1 Naming conventions for numerical values . 8
5.2 Operators . 8
5.2.1 Arithmetic operators . 8
5.2.2 Logical operators . 9
5.2.3 Relational operators . 9
5.2.4 Precedence order of operators . 9
5.2.5 Mathematical functions .10
6 General .10
6.1 Functional overview on the decoding process .10
6.2 Encoder requirements .11
6.3 Decoder requirements.11
7 Organization of the document .11
Annex A (normative) JPEG Pleno Light Field superbox .12
Annex B (normative) 4D transform mode .29
Annex C (normative) JPEG Pleno light field reference view decoding .73
Annex D (normative) JPEG Pleno light field normalized disparity view decoding .81
Annex E (normative) JPEG Pleno Light Field Intermediate View superbox .89
Bibiliography .117
© ISO/IEC 2021 – All rights reserved iii

Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that
are members of ISO or IEC participate in the development of International Standards through
technical committees established by the respective organization to deal with particular fields of
technical activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other
international organizations, governmental and non-governmental, in liaison with ISO and IEC, also
take part in the work.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for
the different types of document should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www .iso .org/ directives or www .iec .ch/ members
_experts/ refdocs).
Attention is drawn to the possibility that some of the elements of this document may be the subject
of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent
rights. Details of any patent rights identified during the development of the document will be in the
Introduction and/or on the ISO list of patent declarations received (see www .iso .org/ patents) or the IEC
list of patent declarations received (see patents.iec.ch).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to the
World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see www .iso .org/
iso/ foreword .html. In the IEC, see www .iec .ch/ understanding -standards.
This document was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 29, Coding of audio, picture, multimedia and hypermedia information.
A list of all parts in the ISO/IEC 21794 series can be found on the ISO website.
Any feedback or questions on this document should be directed to the user’s national standards body. A
complete listing of these bodies can be found at www .iso .org/ members .html and www .iec .ch/ national
-committees.
iv © ISO/IEC 2021 – All rights reserved

Introduction
This document is part of a series of standards for a system known as JPEG Pleno. This document defines
the JPEG Pleno framework. It facilitates the capture, representation, exchange and visualization of
plenoptic imaging modalities. A plenoptic image modality can be a light field, point cloud or hologram,
which are sampled representations of the plenoptic function in the form of, respectively, a vector
function that represents the radiance of a discretized set of light rays, a collection of points with
position and attribute information, or a complex wavefront. The plenoptic function describes the
radiance in time and in space obtained by positioning a pinhole camera at every viewpoint in 3D spatial
coordinates, every viewing angle and every wavelength, resulting in a 7D function.
JPEG Pleno specifies tools for coding these modalities while providing advanced functionality at system
level, such as support for data and metadata manipulation, editing, random access and interaction,
protection of privacy and ownership rights.
© ISO/IEC 2021 – All rights reserved v

INTERNATIONAL STANDARD ISO/IEC 21794-2:2021(E)
Information technology — Plenoptic image coding system
(JPEG Pleno) —
Part 2:
Light field coding
1 Scope
This document specifies a coded codestream format for storage of light field modalities as well as
associated metadata descriptors that are light field modality specific. This document also provides
information on the encoding tools.
2 Normative references
The following documents are referred to in the text in such a way that some or all of their content
constitutes requirements of this document. For dated references, only the edition cited applies. For
undated references, the latest edition of the referenced document (including any amendments) applies.
ITU-T Rec. T.800 | ISO/IEC 15444-1, Information technology — JPEG 2000 image coding system — Part 1:
Core coding system
ITU-T Rec. T.801 | ISO/IEC 15444-2, Information technology — JPEG 2000 image coding system — Part 2:
Extensions
ISO/IEC 21794-1:2020, Information technology — Plenoptic image coding system (JPEG Pleno) — Part 1:
Framework
ISO/IEC 60559, Information technology — Microprocessor Systems — Floating-Point arithmetic
3 Terms and definitions
For the purposes of this document the terms and definitions given in ISO/IEC 21794-1 and the
following apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— ISO Online browsing platform: available at https:// www .iso .org/ obp
— IEC Electropedia: available at http:// www .electropedia .org/
3.1
arithmetic coder
entropy coder that converts variable length strings to variable length codes (encoding) and vice versa
(decoding)
3.2
bit-plane
two-dimensional array of bits
3.3
4D bit-plane
four-dimensional array of bits
© ISO/IEC 2021 – All rights reserved 1

3.4
coefficient
numerical value that is the result of a transformation or linear regression
3.5
compression
reduction in the number of bits used to represent source image data
3.6
depth
distance of a point in 3D space to the camera plane
3.7
disparity view
image that for each pixel of the subaperture view contains the apparent pixel shift between two
subaperture views along either horizontal or vertical axis
3.8
hexadeca-tree
division of a 4D region into 16 (sixteen) 4D subregions
3.9
pixel
collection of sample values in the spatial image domain having all the same sample coordinates
EXAMPLE A pixel may consist of three samples describing its red, green and blue value.
3.10
plenoptic function
amount of radiance in time and in space by positioning a pinhole camera at every viewpoint in 3D
spatial coordinates, every viewing angle and every wavelength, resulting in a 7D representation
3.11
reference view
subaperture view that is used as one of the references to generate the intermediate views
3.12
subaperture view
subaperture image
image taken of the 3D scene by a pinhole camera positioned at a particular viewpoint and viewing angle
3.13
texture
pixel attributes
EXAMPLE Colour information, opacity, etc.
3.14
transform
transformation
mathematical mapping from one signal space to another
2 © ISO/IEC 2021 – All rights reserved

4 Symbols and abbreviated terms
4.1 Symbols
Codestream_Body() coded image data in the codestream without Codestream_Header()
Codestream_Header() codestream header preceding the image data in the codestream

DEC
Dt,,sv,u
()
decoded normalized disparity value at view ts, for pixel location vu,
() ()

Dt,,sv,u
()
normalized disparity value at view ts, for pixel location vu,
() ()
DPEC
k pointer to contiguous codestream for normalized disparity view k

scaling parameter to translate quantized normalized disparity maps to pos-
D
shift
itive range
DCODEC disparity view codec type
f focal length
FPW
p fixed-weight merging parameter for view p

Ht,s
()
view hierarchy value for view ts,
()
HCCt,s
()
horizontal camera centre coordinate for view ts,
()
Ht,s
()
D
binary value defining the availability of a normalized disparity view ts,
()
J
0 Lagrangian encoding cost
J
1 Lagrangian encoding cost of spatial partitioning

J
2 Lagrangian encoding cost of view partitioning

KR
pc, sparse filter regressor mask of texture component c for view p
LightField() JPEG Pleno light field codestream

pc,
quantized least-squares merging weight of texture component c for view p ,
LSW
j
jN=…12,, , LS
p
MIDV absolute value of the minimum value over all quantized normalized disparity views
© ISO/IEC 2021 – All rights reserved 3

MMODE
p view merging mode for intermediate view p

MSP
p sparse filter order for view p

NLS
p number of least-squares merging coefficients for intermediate view p

NRT
p regressor template size parameter for sparse filter for view p
NC number of components in an image

N
I number of intermediate views

N
NDV number of reference normalized disparity views

D
N
number of normalized disparity reference views for intermediate view p
p
T
N
number of texture reference views for intermediate view p
p
N
REF number of reference views
N
RES number of prediction residual views

N
sp total available number of regressors for sparse filter
Plev level a particular codestream complies to
Ppih profile a particular codestream complies to

2D image of dimensions VU× , defines the occlusion state-based segmentation
Q
p
at Intermediate view p
Q normalized disparity quantization parameter
R rate or bitrate, expressed in bit per sample
RCODEC prediction residual view codec type
array of bytes containing for a single prediction residual view the RCODEC
RDATA
codestream after header information has been stripped
array of bytes containing for a single prediction residual view the full DCODEC
RENCODING
codestream
RGB colour data for the red, green and blue colour component of a pixel
array of bytes containing for a single prediction residual view the header infor-
RHEADER
mation from the RCODEC codestream
4 © ISO/IEC 2021 – All rights reserved

RPEC
j pointer to contiguous codestream for prediction residual view j
s
coordinate of the addressed subaperture image along the s-axis
S
size of the light field image along the s-axis (COLUMNS)

T
Tr subscript of the column index of the reference view, ii=…12,, ,N in the light
s
p
ii
field array in row-wise scanning order

Dr
subscript of the column index of the reference normalized disparity view,
s
jj
D
jj=…12,, ,N in the light field array in row-wise scanning order
p
SF
p binary variable, determines if sparse filter is used (true) or not (false)

p,0
SPW
j quantized sparse filter coefficients of texture component c for view p , jM=…12,, , SP
p
pc,
de-quantized sparse filter coefficients of texture component c for view p ,

SPW
j
jM=…12,, , SP
p
t coordinate of the addressed subaperture image along the t-axis
T size of the light field image along the t-axis (ROWS)

T
Tr subscript of the row index of the reference view, ii=…12,, ,N in the light field
t
p
ii
array in row-wise scanning order

D
Dr
subscript of the row index of the reference normalized disparity view, jj=…12,, ,N
t
p
jj
in the light field array in row-wise scanning order

D D
ts,
()
view coordinate subscripts for normalized disparity view k
k k
X X
ts,
()
l l view coordinate subscripts for reference view l

I I
ts,
()
view coordinate subscripts for intermediate view p
p p
ts××vu×
kk kk 4D block dimensions at the 4D block partitioning stage

ts××vu×
bb bb 4D block dimensions at the bit-plane hexadeca-tree decomposition stage
TCODEC reference view codec type
array of bytes, containing for a single reference view, the TCODEC codestream,
TDATA
after header information has been stripped
© ISO/IEC 2021 – All rights reserved 5

TENCODING array of bytes, containing for a single reference view the full TCODEC codestream
array of bytes, containing for a single reference view the header information
THEADER
from the TCODEC codestream
TPEC
l pointer to contiguous codestream for reference view l
u sample coordinate along the u-axis within the addressed subaperture image
U size of the subaperture image along the u-axis (WIDTH)
v sample coordinate along the v-axis within the addressed subaperture image
V size of the subaperture image along the v-axis (HEIGHT)

VCCt,s
()
vertical camera centre coordinate for view ts,
()
VPP
p view prediction parameters for intermediate view p

Xt,,sv,,uc
()
texture value at view ts, for pixel location vu, for texture component c
() ()
DEC
decoded texture value at view ts, for pixel location vu, for texture com-
() ()
Xt,,sv,,uc
()
ponent c
ts,
()
Xt ,s
()
result of warping the texture view ts, to view location ts,
W 22 () ()
11 22
Δx
horizontal distance between a pair of camera centres

Δy
vertical distance between a pair of camera centres
colour data for the luminance, the blue chrominance and the red chrominance
YCbCr
component of a pixel
zt,,sv,u
()
depth value at view ts, for pixel location vu,
() ()
T
p
distance based merging weight for reference view iN=…1,, at intermedi-
ˆ
θ
p
i
ate view p
p
distance based factor, used for defining the merging weight, at intermediate
α
i
T
view p for reference view iN=…1,,
p
binary matrix, defining the locations of the non-zero merging weights in merg-
Γ
ing weight matrix Θ at intermediate view p . It is identical between all colour
p
pc,
components c
6 © ISO/IEC 2021 – All rights reserved

pc,
de-quantized least-squares merging weight of texture component c for view p
θ
j
, jN=…12,, , LS
p
sp
θ
pc, sparse filter coefficients at intermediate view p for colour component c

Θ
pc, merging weight matrix for intermediate view p for colour component c

Υ
pc,
locations of the non-zero elements of Ψ
vu,
()
Ψ
vu,
() regressor template at pixel location vu,
()
Dr
Ω
p set of reference normalized disparity views for intermediate view p

occlD
set of occluded pixels, which remain to be inpainted, during normalized dispar-
Ω
p
ity view synthesis at intermediate view p

occlT
set of occluded pixels, which remain to be inpainted, during texture view syn-
Ω
p
thesis at intermediate view p
Tr
Ω
p
set of reference views for intermediate view p
4.2 Abbreviated terms
2D two dimensional
3D three dimensional
4D four dimensional
DCT discrete cosine transform
floating point floating point notation as specified in ISO/IEC 60559
HTTP hypertext transfer protocol
IDCT inverse DCT
IPR intellectual property rights
IV intermediate view; subaperture view that is generated from surrounding refer-
ence view(s)
JPEG Joint Photographic Experts Group
JPL JPEG Pleno file format
LSB least significant bit
© ISO/IEC 2021 – All rights reserved 7

MSB most significant bit
R-D rate-distortion
RV reference view
URL uniform resource locator
XML eXtensible Markup Language
5 Conventions
5.1 Naming conventions for numerical values
Integer numbers are expressed as bit patterns, hexadecimal values or decimal numbers. Bit patterns
and hexadecimal values have both a numerical value and an associated particular length in bits.
Hexadecimal notation, indicated by prefixing the hexadecimal number by "0x", may be used instead
of binary notation to denote a bit pattern having a length that is an integer multiple of 4. For example,
0x41 represents an eight-bit pattern having only its second most significant bit and its least significant
bit equal to 1. Numerical values that are specified under a "Code" heading in tables that are referred to
as "code tables" are bit pattern values (specified as a string of digits equal to 0 or 1 in which the left-
most bit is considered the most-significant bit). Other numerical values not prefixed by "0x" are decimal
values. When used in expressions, a hexadecimal value is interpreted as having a value equal to the
value of the corresponding bit pattern evaluated as a binary representation of an unsigned integer (i.e.
as the value of the number formed by prefixing the bit pattern with a sign bit equal to 0 and interpreting
the result as a two's complement representation of an integer value). For example, the hexadecimal
value 0xF is equivalent to the 4-bit pattern '1111' and is interpreted in expressions as being equal to the
decimal number 15.
5.2 Operators
NOTE Many of the operators used in document are similar to those used in the C programming
language.
5.2.1 Arithmetic operators
+ addition
− subtraction (as a binary operator) or negation (as a unary prefix operator)
× multiplication
/ division without truncation or rounding
s
<< left shift; x< s
>> right shift; x>>s is defined as ⎿x/2 ⏌
++ increment with 1
-- decrement with 1
8 © ISO/IEC 2021 – All rights reserved

umod x umod a is the unique value y between 0 and a–1
for which y+Na = x with a suitable integer N
& bitwise AND operator; compares each bit of the first operand to the corresponding bit
of the second operand
If both bits are 1, the corresponding result bit is set to 1. Otherwise, the corresponding
result bit is set to 0.
^ bitwise XOR operator; compares each bit of the first operand to the corresponding bit
of the second operand
If both bits are equal, the corresponding result bit is set to 0. Otherwise, the correspond-
ing result bit is set to 1.
5.2.2 Logical operators
|| logical OR
&& logical AND
! logical NOT
5.2.3 Relational operators
> greater than
>= greater than or equal to
< less than
<= less than or equal to
== equal to
!= not equal to
5.2.4 Precedence order of operators
Operators are listed in descending order of precedence. If several operators appear in the same line,
they have equal precedence. When several operators of equal precedence appear at the same level in an
expression, evaluation proceeds according to the associativity of the operator either from right to left
or from left to right.
Operators Type of operation Associativity
() expression left to right
[] indexing of arrays left to right
++, -- increment, decrement left to right
!, – logical not, unary negation
© ISO/IEC 2021 – All rights reserved 9

×, / multiplication, division left to right
umod modulo (remainder) left to right
+, − addition and subtraction left to right
& bitwise AND left to right
^ bitwise XOR left to right
&& logical AND left to right
|| logical OR left to right
<<, >> left shift and right shift left to right
< , >, <=, >= relational left to right
5.2.5 Mathematical functions
|x| absolute value, is –x for x < 0, otherwise x
sign(x) sign of x, zero if x is zero, +1 if x is positive, -1 if x is negative
clamp(x,min,max) clamps x to the range [min,max]: returns min if x < min, max if x > max or
otherwise x
⎾x⏋ ceiling of x; returns the smallest integer that is greater than or equal to x
⎿x⏌ floor of x; returns the largest integer that is less than or equal to x
⎿x⏋
rounding of x to the nearest integer, equivalent to sign xx+05.
()
6 General
6.1 Functional overview on the decoding process
This document specifies the JPEG Pleno Light Field superbox and the JPEG Pleno light field decoding
algorithm. The generic JPEG Pleno Light Field superbox syntax is specified in Annex A.
The specified light field decoding algorithm distinguishes two coding modes:
— 4D Transform mode: this mode is specified in Annex B and is based on a 4D inverse discrete cosine
transform (IDCT) and 4D block partitioning and 4D bit-plane hexadeca-tree decoding;
— 4D Prediction mode: this mode is based the prediction of intermediate views based on reference
views and normalized disparity maps. The signalling syntax and decoding of the reference views is
addressed in Annex C, the normalized disparity views in Annex D, and the prediction parameters
and residual views in Annex E. The intermediate views are reconstructed in a decoding process that
involves view warping, view merging and prediction error correction.
The overall architecture (Figure 1) provides the flexibility to configure the encoding and decoding
system depending on the requirements of the addressed use case.
10 © ISO/IEC 2021 – All rights reserved

Figure 1 — Generic JPEG Pleno light field decoder architecture
6.2 Encoder requirements
An encoding process converts source light field data to coded light field data.
In order to conform with this document, an encoder shall conform with the codestream format syntax
and file format syntax specified in the annexes for the encoding process(es) embodied by the encoder.
6.3 Decoder requirements
A decoding process converts coded light field data to reconstructed light field data. Annexes A through
E describe and specify the decoding process.
A decoder is an embodiment of the decoding process. In order to conform to this document, a decoder
shall convert all, or specific parts of, any coded light field data that conform to the file format syntax
and codestream syntax specified in Annex A to E to a reconstructed light field.
7 Organization of the document
Annex A specifies the description of the JPEG Pleno Light Field superbox.
This document specifies two approaches to represent a compressed representation of light field data:
the 4D Transform mode is specified in Annex B and the 4D Prediction mode is specified Annex C,
Annex D and Annex E. Annex C details the signalling of the reference view data, Annex D the signalling
of the normalized disparity views and finally, Annex E the signalling of the prediction parameters to
generate the intermediate views and residual view data to compensate for prediction errors.
© ISO/IEC 2021 – All rights reserved 11

Annex A
(normative)
JPEG Pleno Light Field superbox
A.1 General
This annex specifies the use of the JPEG Pleno Light Field superbox which is designed to contain
compressed light field data and associated metadata. The listed boxes shall comply with their definitions
as specified in ISO/IEC 21794-1.
This document may redefine the binary structure of some boxes defined as part of the ISO/IEC 15444-1
or ISO/IEC 15444-2 file formats. For those boxes, the definition found in this document shall be used for
all JPL files.
A.2 Organization of the JPEG Pleno Light Field superbox
Figure A.1 shows the hierarchical organization of the JPEG Pleno Light Field superbox contained by a
JPL file. This illustration does not specify nor imply a specific order to these boxes. In many cases, the
file will contain several boxes of a particular box type. The meaning of each of those boxes is dependent
on the placement and order of that particular box within the file.
This superbox is composed out of the following core elements:
— a JPEG Pleno Light Field Header box containing parameterization information about the light field
such as size and colour parameters;
— a JPEG Pleno Light Field Reference View box containing the compressed reference views of the
light field;
— a JPEG Pleno Light Field Disparity View box signalling disparity information for all or a subset of
subaperture views;
— a JPEG Pleno Light Field Intermediate View box containing prediction parameters and eventual
compressed residual signals for subaperture views not encoded as reference views.
Table A.1 lists all boxes defined as part of this document. Boxes defined as part of the ISO/IEC 15444-1
or ISO/IEC 15444-2 file formats are not listed. A box that is listed in Table A.1 as “Required” shall exist
within all conforming JPL files. For the placement of and restrictions on each box, see the relevant
section defining that box.
Note that the IPR, XML, UUID and UUID boxes introduced in Annex A can be signalled, as well at the
level of the JPEG Pleno Light Field box, to carry light field specific metadata.
12 © ISO/IEC 2021 – All rights reserved

Figure A.1 — Hierarchical organization of a JPEG Pleno Light Field superbox
© ISO/IEC 2021 – All rights reserved 13

A.3 Defined boxes
A.3.1 Overview
The following boxes shall properly be interpreted by all conforming readers. Each of these boxes
conforms to the standard box structure as defined in ISO/IEC 21794-1:2020, Annex A. The following
clauses define the value of the DBox field. It is assumed that the LBox, TBox and XLBox fields exist for
each box in the file as defined in ISO/IEC 21794-1:2020, Annex A.
Table A.1 — Defined boxes
Box name Type Superbox Required? Comments
JPEG Pleno Light Field ‘jplf’ Yes Yes This box contains a series of boxes
box that contain the encoded light field,
(0x6A70 6C66)
its parameterization and associated
metadata. (Defined in ISO/IEC 21794-
1:2020, Annex A)
JPEG Pleno Profile and ‘jppl’ No Yes This box indicates to which profile
Level box and associated level the file format
(0x6A70 706C)
and codestream complies. (Defined in
Annex A.3.2)
JPEG Pleno Light Field 'jplh' Yes Yes This box contains generic information
Header box about the file, such as the number of
(0x6A70 6C68)
components, bits per component and
colour space. (Defined in Annex A.3.3)
Light Field Header box ‘lhdr’ No Yes This box contains fixed length generic
information about the light field, such
(0x6C68 6472)
as light field dimensions, subaperture
image size, number of components,
codec and bits per component. (De-
fined in Annex A.3.3.2)
Camera Parameter box ‘lfcp’ No No This box signals intrinsic and extrin-
sic camera parameters for calibration
(0x6C66 6370)
of the light field data. (Defined in
Annex A.3.3.3)
Contiguous Codestream 'jp2c' No No This box contains a JPEG Pleno code-
box stream (Defined in Annex A.3.4)
(0x6A70 3263)
JPEG Pleno Light Field ‘lfrv’ Yes No This box contains a series of boxes
Reference View superbox that contain the encoded reference
(0x6C66 7276)
views and their associated parame-
ters. (Defined in Annex C.2)
JPEG Pleno Light Field ‘lfrd’ No No This box signals which views are
Reference View Descrip- encoded as reference views and their
(0x6C66 7264)
tion box encoding configuration. (Defined in
Annex C.3.1)
Common Codestream ‘lfcc’ No No This box contains the redundant part
Elements box of the signalled codestreams. (Defined
(0x6C66 6363)
in Annex C.3.2)
JPEG Pleno Light Field ‘lfdv’ Yes No This box contains a series of boxes
Normalized Disparity that contain the encoded normalized
(0x6C66 6476)
View superbox disparity views and their associated
parameters. (Defined in Annex D.2)
JPEG Pleno Light Field ‘lfdd’ No No This box signals for which views
Normalized Disparity normalized disparity information is
(0x6C66 6464)
View Description box signalled and their encoding configu-
ration. (Defined in Annex D.3.1)
14 © ISO/IEC 2021 – All rights reserved

Table A.1 (continued)
Box name Type Superbox Required? Comments
JPEG Pleno Light Field ‘lfiv’ Yes No This box contains a series of boxes
Intermediate View that contain both the prediction
(0x6C66 6976)
superbox parameters for the intermediate
views and the encoded residual views.
(Defined in Annex E.2)
JPEG Pleno Light Field ‘lfpp’ No No This box signals prediction parame-
Prediction Parameter box ter information for the intermediate
(0x6C66 7070)
views. (Defined in Annex E.3.1)
JPEG Pleno Light Field ‘lfre’ No No This box signals the encoding configu-
Residual View Descrip- ration for the residual views contain-
(0x6C66 7265)
tion box ing the prediction errors. (Defined in
Annex E.3.2)
A.3.2 JPEG Pleno Profile and Level box
The conformance to profiles is indicated in the file type box by the addition of the compatible profiles as
brands within the compatibility list. Derived and application specifications based on this specification
may define additional brands.
The type of the JPEG Pleno Profile and Level box shall be ‘jppl’ (0x6A70 706C) and contents of the box
shall have the organization as in Figure A.2 and format as in Table A.2.
Key
Ppih profile of the codestream
Plev level of the codestream
Figure A.2 — Organization of the contents of a JPEG Pleno Profile and Level box
Table A.2 — Format of the contents of the JPEG Pleno Profile and Level box
Field name Size (bits) Value
Ppih 16 Reserved for future
ISO/IEC use
Plev 16 Reserved for future
ISO/IEC use
A.3.3 JPEG Pleno Light Field Header box
A.3.3.1 General
The JPEG Pleno Header box contains generic information about the file, such as the number of
components, bits per component and colour space. This box is a superbox. Within a JPL file, there shall
be one and only one JPEG Pleno Header box. The JPEG Pleno Header box may be located anywhere
within the file after the File Type box but before the Contiguous Codestream box. It also must be at the
same level as the JPEG Pleno Signature and File Type boxes (it shall not be inside any other superbox
within the file).
The type of the JPEG Pleno Header box shall be 'jplh' (0x6A70 6C68).
© ISO/IEC 2021 – All rights reserved 15

This box contains several boxes. Other boxes may be defined in other documents and may be ignored by
conforming readers. Those boxes contained within the JPEG Pleno Header box that are defined within
this document are shown in Figure A.3:
— The Light Field Header box specifies information about the reference grid geometry, bit depth
and the number of components. This box shall be the first box in the JPEG Pleno Header box and is
specified in A.3.3.2.
— The Bits Per Component box specifies the bit depth of the components in the file in cases where the bit
depth is not constant across all components. Its structure shall be as specified in ISO/IEC 15444-1.
— The Colour Specification boxes specify the colour space of the decompressed image. Their structures
shall be as specified in ISO/IEC 15444-2. There shall be at least one Colour Specification box within
the JPEG Pleno Header box. The use of multiple Colour Specification boxes provides the ability for a
decoder to be given multiple optimization or compatibility options for colour processing. These boxes
may be found anywhere in the JPEG Pleno Header box provided that they come after the Light Field
Header box. All Colour Specification boxes shall be contiguous within the JPEG Pleno Header box.
— The Channel Definition box defines the channels in the image. Its structure shall be as specified in
ISO/IEC 15444-1. This box may be found anywhere in the JPEG Pleno Header box, provided that it
comes after the Light Field Header box.
Key
lhdr Light Field Header box
bppc Bits Per Component box
i
colr Colour Specification boxes
cdef Channel Definition box
Figure A.3 — Organization of the contents of a JPEG Pleno Header box
A.3.3.2 Light Field Header box
A.3.3.2.1 General
This box contains fixed length generic information about the light field, such as light field dimensions,
subaperture image size, number of components, codec and bits per component. The contents of the
JPEG Pleno Header box shall start with a Light Field Header box. Instances of this box in other places in
the file shall be ignored. The length of the Light Field Header box shall be 30 bytes, including the box
length and type fields. Much of the information within the Light Field Header box is redundant with
information stored in the codestream itself.
All references to "the codestream" in the descriptions of fields in this Light Field Header box apply to the
codestream found in the first Contiguous Codestream box in the file. Files that contain contradictory
information between the Light Field Header box and the first codestream are not conforming files.
However, readers may choose to attempt to read these files by using the values found within the
codestream.
The type of the Light Field Header box shall be 'lhdr' (0x6C68 6472) and the contents of the box shall
have the format as in Figure A.4 and Table A.3:
— ROWS (T): The value of this parameter indicates the number of rows of the subaperture view array.
This field is stored as a 4-byte big-endian unsigned integer.
16 © ISO/IEC 2021 – All rights reserved

— COLUMNS (S): The value of this parameter indicates the number of columns of the subaperture
view array. This field is stored as a 4-byte big-endian unsigned integer.
— HEIGHT (V): The value of this parameter indicates the height of the sample grid. This field is stored
as a 4-byte big-endian unsigned integer.
— WIDTH (U): The value of this parameter indicates the width of the sample grid. This field is stored
as a 4-byte big-endian unsigned integer.
— NC: This parameter specifies the number of components in the codestream and is stored as a 2-byte
big-endian unsigned integer. The value of this field shall be equal to the value of the NC field in the
LFC marker in the codestream (as defined in B.3.2.6.3). If no Channel Definition Box is available, the
order of the components for colour images is R-G-B-Aux or Y-U-V-Aux.
— BPC: This parameter specifies the bit depth of the components in the codestream, minus 1, and is
stored as a 1-byte field (Table A.4).
The low 7-bits of the value indicate the bit depth of the components. The high-bit indicates whether
the components are signed or unsigned. If the high-bit is 1, then the components contain signed
values. If the high-bit is 0, then the components contain unsigned values. If the components vary
in bit depth or sign, or both, then the value of this field shall be 255 and the Light Field Header box
shall also contain a Bits Per Component box defining the bit depth of each component (as defined in
A.3.3.2.2).
— C: This parameter specifies the compression algorithm used to compress the image data. It is
encoded as a 1-byte unsigned integer. It the value is 0, the 4D Transform mode coding is activated. If
the value is 1, the 4D Prediction mode is activated. All other values are reserved for ISO/IEC use.
— UnkC: This field specifies if the actual colour space of the image data in the codestream is known.
This field is encoded as a 1-byte unsigned integer. Legal values for this field are 0, if the colour space
of the image is known and correctly specified in the Colour Space Specification boxes within the
file, or 1 if the colour space of the light field is not known. A value of 1 will be used in cases such as
the transcoding of legacy images where the actual colour space of the image data is not known. In
these cases, while the colour space interpretation methods specified in the file may not accurately
reproduce the image with respect to an original, the image should be treated as if the methods do
accurately reproduce the image. Values other than 0 and 1 are reserved for ISO/IEC use.
— IPR: This parameter indicates whether this JPL file contains intellectual property rights information.
If the value of this field is 0, this file does not contain rights information, and thus the file does not
contain an IPR box. If the value is 1, then the file does contain rights information and thus does
contain an IPR box as defined in ISO/IEC 15444-1. Other values are reserved for ISO/IEC use.
Key
ROWS (T) number of rows of the subaperture view array
COLUMNS (S) number of columns of the subaperture view array
HEIGHT (V) subaperture view height
WIDTH (U) subaperture view width
NC number of components
BPC bits per component
C compression type
UnkC colour space unknown
IPR intellectual property
Figure A.4 — Organization of the contents of a Light Field Header box
© ISO/IEC 2021 – All rights reserved 17

Table A.3 — Format of the contents of the Light Field Header box
Field name Size (bits) Value
ROWS 32 1 to (2 – 1)
COLUMNS 32 1 to (2 – 1)
HEIGHT 32 1 to (2 – 1)
WIDTH 32 1 to (2 – 1)
NC 16 1 to 16 384
BPC 8 See Table A.4
UnkC 8 0 to 1
IPR 8 0 to 1
Table A.4 — BPC values
Values (bits)
Component sample precision
MSB LSB
x000 0000 Component bit depth = value + 1. From 1 bit deep to 38 bits
to deep respectively (counting the sign bit, if appropriate).
x010 0101
0xxx xxxx Components are unsigned values.
1xxx xxxx Components are signed values.
1111 1111 Components vary in bit depth.
All other values reserved for ISO/IEC use.
A.3.3.2.2 Bits Per Component box
The Bits Per Component box specifies the bit depth of each component. If the bit depth of all components
in the codestream is the same (in both sign and precision), then this box shall not be found. Otherwise,
this box specifies the bit depth of each individual component. The order of bit depth values in this box is
the actual order in which those components are enumerated within the codestream. The exact location of
this box within the JPEG Pleno Header box may vary provided that it follows the Light Field Header box.
There shall be one and only one Bits Per Component box inside a JPEG Pleno Header box.
The type of the Bits Per Component box shall be 'bpcc' (0x6270 6363). The contents of this box shall be
as in Table A.5 and Figure A.5.
Key
i
BPC bits per component
Figure A.5 — Organization of the contents of a Bits Per Component box
Table A.5 — Format of the contents of the Bits Per Component box
Field name Size (bits) Value
i
BPC 8 See Table A.6
This parameter specifies the bit depth of component i, minus 1, encoded as a 1-byte value (Table A.6).
The ordering of the components within the Bits Per Component box shall be the same as the ordering
18 © ISO/IEC 2021 – All rights reserved

i
of the components within the codestream. The nu
...