ISO/IEC 18477-4:2017

INTERNATIONAL ISO/IEC
STANDARD 18477-4
First edition
2017-10
Information technology — Scalable
compression and coding of
continuous-tone still images —
Part 4:
Conformance testing
Technologies de l'information — Compression échelonnable et codage
d'images plates en ton continu —
Partie 4: Essai de conformité
Reference number
©
ISO/IEC 2017
© ISO/IEC 2017, Published in Switzerland
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized otherwise in any form
or by any means, electronic or mechanical, including photocopying, or posting on the internet or an intranet, without prior
written permission. Permission can be requested from either ISO at the address below or ISO’s member body in the country of
the requester.
ISO copyright office
Ch. de Blandonnet 8 • CP 401
CH-1214 Vernier, Geneva, Switzerland
Tel. +41 22 749 01 11
Fax +41 22 749 09 47
copyright@iso.org
www.iso.org
ii © ISO/IEC 2017 – All rights reserved

Contents Page
Foreword .iv
Introduction .v
1 Scope . 1
2 Normative references . 2
3 Terms, definitions, abbreviated terms and symbols . 2
3.1 Terms and definitions . 2
3.2 Symbols . 8
3.3 Abbreviated terms . 8
4 Conventions . 8
4.1 Conformance language . 8
4.2 Operators . 9
4.2.1 Arithmetic operators . 9
4.2.2 Logical operators . 9
4.2.3 Relational operators . 9
4.2.4 Precedence order of operators . 9
4.2.5 Mathematical functions .10
5 Conventions .10
6 General description .10
6.1 Overview .10
6.2 Parts and profiles .10
6.3 Decoders .11
6.4 Implementation conformance statement .11
6.5 Abstract test suites .11
6.6 Decoder conformance testing procedures .11
7 Copyright .11
8 Conformance files availability and updates .11
Annex A (normative) Decoder conformance testing procedures .12
Annex B (normative) Decoder conformance tests .18
Annex C (normative) Codestream conformance .29
Bibliography .31
© ISO/IEC 2017 – All rights reserved iii

Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are
members of ISO or IEC participate in the development of International Standards through technical
committees established by the respective organization to deal with particular fields of technical
activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international
organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the
work. In the field of information technology, ISO and IEC have established a joint technical committee,
ISO/IEC JTC 1.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular the different approval criteria needed for the
different types of ISO documents should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject of
patent rights. ISO shall not be held responsible for identifying any or all such patent rights. Details of
any patent rights identified during the development of the document will be in the Introduction and/or
on the ISO list of patent declarations received (see www.iso.org/patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation on the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to the
World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see the following
URL: www.iso.org/iso/foreword.html.
This document was prepared by Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 29, Coding of audio, picture, multimedia and hypermedia information.
A list of all parts in the ISO/IEC 18477 series can be found on the ISO website.
iv © ISO/IEC 2017 – All rights reserved

Introduction
The ISO/IEC 18477 series, also known under the term JPEG XT, specifies lossy and lossless codestream
formats for storage of continous-tone high and low dynamic range photographic content. All parts
of the ISO/IEC 18477 series are compatible to the Recommendation ITU-T T.81 | ISO/IEC 10918, also
commonly known as JPEG. That is, any decoder conforming to the latter standard will be able to
reconstruct codestreams from the ISO/IEC 18477 series to an 8 bits/sample image. Additional features
offered by ISO/IEC 18477, such as representation of intermediate or high-dynamic range images,
or lossless reconstruction require an extended decoder that implements, in addition to the Rec.
ITU-T T.81 | ISO/IEC 10918-1, also one or multiple members of ISO/IEC 18477.
This document provides the framework, concepts and methodology for testing codestreams and
implementations, and the criteria to be achieved to claim conformance to the parts and profiles
of ISO/IEC 18477. The objective of this document is to promote interoperability between JPEG XT
decoders, and to test these systems for conformance to one or multiple specifications that are part
of the JPEG XT. Conformance testing is the testing of a candidate implementation for the existence of
specific characteristics required by a standard. It involves testing the capabilities of an implementation
against the conformance requirements in the relevant standard.
The purpose of this document is to define a common test methodology, to provide a framework for
specific abstract test suites (ATS) and to define the procedures to be followed during conformance
testing.
Any organization contemplating the use of the test methods defined in this document should carefully
consider the constraints on their applicability. Conformance testing does not include robustness testing,
acceptance testing, and performance testing, all of which are outside the scope of this text.
© ISO/IEC 2017 – All rights reserved v

INTERNATIONAL STANDARD ISO/IEC 18477-4:2017(E)
Information technology — Scalable compression and
coding of continuous-tone still images —
Part 4:
Conformance testing
1 Scope
This document specifies the framework, concepts, methodology for testing, and criteria to be achieved
to claim conformance to one or multiple parts of ISO/IEC 18477 as listed below. It provides a framework
for specifying abstract test suites and for defining the procedures to be followed during conformance
testing.
This document
— specifies conformance testing procedures for decoding of ISO/IEC 18477-1, 18477-2, ISO/IEC 18477-
6, ISO/IEC 18477-7, ISO/IEC 18477-8 and ISO/IEC 18477-9,
— specifies conformance testing procedures for codestreams to the above International Standards,
— specifies codestreams, decoded images, and error metrics to be used within the decoder testing
procedures, and
— specifies abstract test suites.
This document does not include the following tests:
— testing decoders for conformance to ISO/IEC 18477-3 only. ISO/IEC 18477-6, ISO/IEC 18477-
7, ISO/IEC 18477-8 and ISO/IEC 18477-9 are extensions of ISO/IEC 18477-3 and the required
functionality of ISO/IEC 18477-3 is tested as part of the former standards. Testing codestreams for
conformance to ISO/IEC 18477-3 is specified in C.2;
— testing codestreams for conformance to ISO/IEC 18477-7 beyond testing them for conformance
to individual profiles of this document. Testing such codestreams ("full profile codestreams") for
syntactical correctness is, however, covered by testing them for conformance to ISO/IEC 18477-3;
— testing of the composition of background and foreground for images reconstructed from
ISO/IEC 18477-9 codestreams as this operation is application dependent;
— acceptance testing: the process of determining whether an implementation satisfies acceptance
criteria and enables the user to determine whether or not to accept the implementation. This
includes the planning and execution of several kinds of tests (e.g. functionality, quality, and speed
performance testing) that demonstrate that the implementation satisfies the user requirements;
— performance testing: measures the performance characteristics of an implementation under test
(IUT) such as its throughput, responsiveness, etc. under various conditions.
— robustness testing: the process of determining how well an implementation process data which
contains errors.
The ISO/IEC 18477 series consists of multiple parts, each of which defines one or multiple profiles.
A given IUT (implementation under test) may claim to implement various parts and profiles of
ISO/IEC 18477 at once. To test such implementations, they have to be tested with the Abstract Test
Suites of each part and profile they claim to conform to.
© ISO/IEC 2017 – All rights reserved 1

2 Normative references
The following documents are referred to in the text in such a way that some or all of their content
constitutes requirements of this document. For dated references, only the edition cited applies. For
undated references, the latest edition of the referenced document (including any amendments) applies.
ISO/IEC 18477-1, Information technology — Scalable compression and coding of continuous-tone still
images — Part 1: Scalable compression and coding of continuous-tone still images
ISO/IEC 18477-2, Information technology — Scalable compression and coding of continuous-tone still
images — Part 2: Coding of high dynamic range images
ISO/IEC 18477-3, Information technology — Scalable compression and coding of continuous-tone still
images — Part 3: Box file format
ISO/IEC 18477-6, Information technology — Scalable compression and coding of continuous-tone still
images — Part 6: IDR Integer Coding
ISO/IEC 18477-7, Information technology: Scalable compression and coding of continuous-tone still images,
HDR floating point coding
ISO/IEC 18477-8, Information technology — Scalable compression and coding of continuous-tone still
images — Part 8: Lossless and near-lossless coding
ISO/IEC 18477-9, Information technology — Scalable compression and coding of continuous-tone still
images — Part 9: Alpha channel coding
ISO/IEC 10918-1, Information technology — Digital compression and coding of continuous tone still
images — Requirements and guidelines
3 Terms, definitions, abbreviated terms and symbols
For the purposes of this document, the following terms and definitions apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— IEC Electropedia: available at http://www.electropedia.org/
— ISO Online browsing platform: available at http://www.iso.org/obp
3.1 Terms and definitions
3.1.1
abstract test suite
generic conformance testing concepts and procedures for a given requirement
3.1.2
ASCII
binary encoding of 7-bit characters defined by ISO/IEC 646
3.1.3
base decoding path
process of decoding legacy codestream (3.1.43) and refinement data to the base image (3.1.4), jointly
with all further steps until residual data is added to the values obtained from the residual codestream
3.1.4
base image
collection of sample values obtained by entropy decoding the DCT coefficients of the legacy codestream
(3.1.43) and the refinement codestream, and inversely DCT transforming them jointly
2 © ISO/IEC 2017 – All rights reserved

3.1.5
big endian
order of bytes with the most significant byte first
3.1.6
bit
unit of information representing a single yes/no choice represented by a one or a zero
3.1.7
binary decision
choice between two alternatives
3.1.8
bitstream
partially encoded or decoded sequence of bits (3.1.6) comprising an entropy-coded segment
3.1.9
block
8 × 8 array of samples (3.1.62) or an 8 × 8 array of DCT coefficient values of one component (3.1.19)
3.1.10
box
structured collection of data describing the image or the image decoding process embedded into one or
multiple APP marker segments
Note 1 to entry: See ISO/IEC 18477-3:2015, Annex B for the definition of boxes.
3.1.11
byte
group of 8 bits
3.1.12
coder
embodiment of a coding process (3.1.14)
3.1.13
coding model
procedure used to convert input data into symbols to be coded
3.1.14
coding process
general term for referring to an encoding process, a decoding process, or both
3.1.15
coefficient
values that are the result of a discrete cosine transformation
3.1.16
conformance
fulfillment of the specified requirements, as defined in this document, for a given profile and part of
ISO/IEC 18477
3.1.17
conformance test procedure
process of assessing conformance (3.1.16)
3.1.18
compression
reduction in the number of bits (3.1.6) used to represent source image data
© ISO/IEC 2017 – All rights reserved 3

3.1.19
component
two-dimensional array of samples (3.1.62) having the same designation in the output or display device
3.1.20
continuous-tone image
image whose components (3.1.19) have more than one bit (3.1.6) per sample (3.1.62)
3.1.21
decoder
embodiment of a decoding process (3.1.22)
3.1.22
decoding process
process which takes as its input compressed image data and outputs a continuous-tone image (3.1.20)
3.1.23
dequantization
inverse procedure to quantization by which the decoder (3.1.21) recovers a representation of the DCT
coefficients
3.1.24
downsampling
procedure by which the spatial resolution of a component (3.1.19) is reduced
3.1.25
encoder
embodiment of an encoding process (3.1.26)
3.1.26
encoding process
process which takes as its input a continuous-tone image (3.1.20) and outputs compressed image data
3.1.27
entropy-coded (data) segment
independently decodable sequence of entropy encoded bytes (3.1.11) of compressed image data
3.1.28
entropy decoder
embodiment of an entropy decoding (3.1.29) procedure
3.1.29
entropy decoding
lossless procedure which recovers the sequence of symbols from the sequence of bits (3.1.6) produced
by the entropy encoder (3.1.30)
3.1.30
entropy encoder
embodiment of an entropy encoding (3.1.31) procedure
3.1.31
entropy encoding
lossless procedure which converts a sequence of input symbols into a sequence of bits (3.1.6) such that
the average number of bits (3.1.6) per symbol approaches the entropy of the input symbols
3.1.32
extension image
synonym for residual image (3.1.61)
4 © ISO/IEC 2017 – All rights reserved

3.1.33
grayscale image
continuous-tone image (3.1.20) that has only one component (3.1.19)
3.1.34
high dynamic range
image or image data comprised of more than 8 bits per sample (3.1.62)
3.1.35
Huffman decoder
embodiment of a Huffman decoding (3.1.36) procedure
3.1.36
Huffman decoding
entropy decoding (3.1.29) procedure which recovers the symbol from each variable length code
produced by the Huffman encoder (3.1.37)
3.1.37
Huffman encoder
embodiment of a Huffman encoding (3.1.38) procedure
3.1.38
Huffman encoding
entropy encoding (3.1.31) procedure which assigns a variable length code to each input symbol
3.1.39
implementation
realization of a specification
3.1.40
implementation under test
IUT
implementation (3.1.39) that is being evaluated for conformance (3.1.16)
3.1.41
intermediate dynamic range
image or image data comprised of more than 8 bits per sample (3.1.62)
3.1.42
joint photographic experts group
JPEG
informal name of the committee which created this document
Note 1 to entry: The “joint” comes from the ITU-T and ISO/IEC collaboration.
3.1.43
legacy codestream
collection of markers (3.1.51) and syntax elements defined by ISO/IEC 10918-1 bare any additional
syntax elements defined by the ISO/IEC 18477 standard, i.e. the legacy codestream consists of the
collection of all markers except those APP markers that describe JPEG XT boxes by the syntax defined
in ISO/IEC 18477-3:2015, Annex A
3.1.44
legacy decoding path
collection of operations to be performed on the entropy coded data as described by ISO/IEC 10918-1
jointly with the Legacy Refinement scans before this data is merged with the residual data to form the
final output image
© ISO/IEC 2017 – All rights reserved 5

3.1.45
legacy decoder
embodiment of a decoding process (3.1.20) conforming to ISO/IEC 10918-1, confined to the lossy DCT
process and the baseline, sequential or progressive modes, decoding at most four components to 8 bits
per component
3.1.46
legacy image
arrangement of sample values as described by applying the decoding process (3.1.20) described by
ISO/IEC 10918-1 on the entropy coded data as defined by the said standard
3.1.47
lossless
descriptive term for encoding and decoding processes and procedures in which the output of the
decoding procedure(s) is identical to the input to the encoding procedure(s)
3.1.48
lossless coding
mode of operation which refers to any one of the coding processes (3.1.14) defined in ISO/IEC 18477-8 in
which all of the procedures are lossless (3.1.47)
Note 1 to entry: See ISO/IEC 18477-8:2016, Annex H.
3.1.49
lossy
descriptive term for encoding and decoding processes which are not lossless (3.1.47)
3.1.50
low dynamic range
image or image data comprised of data with no more than 8 bits per sample (3.1.62)
3.1.51
marker
two-byte code in which the first byte is hexadecimal FF and the second byte is a value between 1 and
hexadecimal FE
3.1.52
marker segment
marker (3.1.51) together with its associated set of parameters
3.1.53
pixel
collection of sample values in the spatial image domain having all the same sample coordinates
EXAMPLE A pixel may consist of three samples describing its red, green and blue value.
3.1.54
precision
number of bits (3.1.6) allocated to a particular sample (3.1.62) or DCT coefficient
3.1.55
procedure
set of steps which accomplishes one of the tasks which comprise an encoding or decoding process
3.1.56
quantization value
integer value used in the quantization procedure
3.1.57
quantize
act of performing the quantization procedure for a DCT coefficient
6 © ISO/IEC 2017 – All rights reserved

3.1.58
residual decoding path
collection of operations applied to the entropy coded data contained in the residual data box and
residual refinement scan boxes up to the point where this data is merged with the base image to form
the final output image
3.1.59
residual image
sample values as reconstructed by inverse quantization and inverse DCT transformation applied to the
entropy-decoded coefficients described by the residual scan (3.1.60) and residual refinement scans
3.1.60
residual scan
additional pass over the image data invisible to legacy decoders (3.1.45) which provides additive and/or
multiplicative correction data of the legacy scans to allow reproduction of high dynamic range (3.1.34)
or wide colour gamut data
3.1.61
refinement scan
additional pass over the image data invisible to legacy decoders (3.1.45) which provides additional least
significant bits to extend the precision (3.1.54) of the DCT transformed coefficients (3.1.15)
3.1.62
sample
one element in the two-dimensional image array which comprises a component (3.1.19)
3.1.63
sample grid
common coordinate system for all samples (3.1.62) of an image
Note 1 to entry: The samples at the top left edge of the image have the coordinates (0,0); the first coordinate
increases towards the right, the second towards the bottom.
3.1.64
scan
single pass through the data for one or more of the components (3.1.19) in an image
3.1.65
scan header
marker segment (3.1.52) that contains a start-of-scan marker and associated scan parameters that are
coded at the beginning of a scan (3.1.64)
3.1.66
source image
image used as input to an encoder (3.1.25)
3.1.67
superbox
box (3.1.10) that carries other boxes as payload data
3.1.68
table specification data
coded representation from which the tables used in the encoder (3.1.24) and decoder (3.1.21) are
generated and their destinations specified
3.1.69
testing
process of evaluating conformance (3.1.16)
© ISO/IEC 2017 – All rights reserved 7

3.1.70
uniform quantization
method of reducing the precision (3.1.54) of DCT coefficients to enable more efficient entropy
coding of them
3.1.71
upsampling
procedure (3.1.55) by which the spatial resolution of a component (3.1.19) is increased
3.2 Symbols
Nf Number of components in an image
R Additional bits in the HDR image. 8 + R is the sample precision of the reconstructed HDR or IDR image.
b b
3.3 Abbreviated terms
ASCII American standard code for information interchange
ATS Abstract test suite
DCT Discrete cosine transformation
ETS Executable test suite
HDR High dynamic range
IDR Intermediate dynamic range
IUT Implementation under test
JPEG Joint photographic experts group
LDR Low dynamic range
LSB Least significant bit
MRSE Mean relative square error
MSE Mean square error
MSB Most significant bit
PSNR Peak signal to noise ratio
TMO Tone mapping operator
TCS Test codestream
4 Conventions
4.1 Conformance language
This document consists of normative and informative text.
Normative text is that text which expresses mandatory requirements. The word "shall" is used to
express mandatory requirements strictly to be followed in order to conform to this document and
from which no deviation is permitted. A conforming implementation is one that fulfils all mandatory
requirements.
8 © ISO/IEC 2017 – All rights reserved

Informative text is text that is potentially helpful to the user, but not indispensable and can be removed,
changed or added editorially without affecting interoperability. All text in this document is normative,
with the following exceptions: the Introduction, any parts of the text that are explicitly labelled as
"informative", and statements appearing with the preamble "NOTE" and behaviour described using the
word "should". The word "should" is used to describe behaviour that is encouraged but is not required
for conformance to this document.
The keywords "may" and "need not" indicate a course of action that is permissible in a conforming
implementation.
The keyword "reserved" indicates a provision that is not specified at this time, shall not be used, and
may be specified in the future. The keyword "forbidden" indicates "reserved" and in addition indicates
that the provision will never be specified in the future.
4.2 Operators
NOTE Many of the operators used in this document are similar to those used in the C programming language.
4.2.1 Arithmetic operators
+ Addition
− Subtraction (as a binary operator) or negation (as a unary prefix operator)
* Multiplication
/ Division without truncation or rounding
4.2.2 Logical operators
|| Logical OR
&& Logical AND
! Logical NOT
4.2.3 Relational operators
> Greater than
>= Greater than or equal to
< Less than
<= Less than or equal to
== Equal to
!= Not equal to
4.2.4 Precedence order of operators
Operators are listed below in descending order of precedence. If several operators appear in the same
line, they have equal precedence. When several operators of equal precedence appear at the same level
in an expression, evaluation proceeds according to the associativity of the operator either from right to
left or from left to right.
© ISO/IEC 2017 – All rights reserved 9

Operators Type of operation Associativity
(), [ ], . Expression Left to Right
− Unary negation
*, / Multiplication Left to Right
+, − Addition and Subtraction Left to Right
< , >, <=, >= Relational Left to Right
4.2.5 Mathematical functions
⎾x⏋ Ceil of x. Returns the smallest integer that is greater than or equal to x.
⎿x⏌ Floor of x. Returns the largest integer that is lesser than or equal to x.
|x| Absolute value is –x for x < 0, otherwise x.
sign(x) Sign of x, 0 if x is zero, +1 if x is positive, −1 if x is negative.
clamp(x,min,max) Clamps x to the range [min,max]: returns min if x < min, max if x > max or otherwise x.
a
x Raises the value of x to the power of a. x is a non-negative real number, a is a real
a
number. x is equal to exp[a×log(x)] where exp is the exponential function and log()
a
the natural logarithm. If x is 0 and a is positive, x is defined to be 0.
5 Conventions
The conformance files including codestreams, reference decoded images and descriptive files are
supplied in the form of an electronic attachment, in compressed form. File locations given in this
1)
document are expressed relative to the top level directory tree within this compressed file. A Unix
style file structure and delimiters are assumed.
This document contains instructions for the use of these files. No support can be provided by
ITU | ISO/IEC beyond that offered in this document.
6 General description
6.1 Overview
The ISO/IEC 18477 series, also known as JPEG XT, consists of multiple parts each of which defines
multiple coding tools. This document defines Abstract Test Suites (ATS) defining decoder conformance
tests for ISO/IEC 18477-2, ISO/IEC 18477-6, ISO/IEC 18477-7, ISO/IEC 18477-8 and ISO/IEC 18477-9.
Each part defines one or multiple profiles.
6.2 Parts and profiles
A part of ISO/IEC 18477 defines a set of coding tools designed to satisfy a common set of requirements,
such as the lossy or lossless representation of images consisting of integer or floating point samples. A
part defines a consistent syntax for the representation of data that falls into its requirements.
Profiles define a subset of technologies of a part that meets the needs of a given application within limits
on parameters within a selected technology. Profiles limit the parameters that are available within a
1) Unix is an example of a suitable product available commercially. This information is given for the convenience
of users of this document and does not constitute an endorsement by ISO or IEC of these products.
10 © ISO/IEC 2017 – All rights reserved

given part, and hence constrain the available choices and coding options within such part. The full set
of all possible options within a part is also denoted as Full Profile. Except for the full profile which is a
superset of all other profiles, profiles do not define a hierarchy, and hence are not subsets of each other.
While an implementation that is conforming to Full Profile is capable to decode every other profile, no
other simple relation holds between the remaining profiles; they are independent of each other and
only a superset of the Full Profile.
6.3 Decoders
Conforming implementations of decoders are not required to decode codestreams from all parts and
profiles. JPEG XT decoders may only implement a subset of the parts, and/or for each part, a subset of
the profiles available within this part.
6.4 Implementation conformance statement
Evaluation of conformance for a particular implementation may require a statement of the options that
have been implemented. This will allow the implementation to be tested for conformance against only
the relevant requirements. Such a statement is called an ICS (implementation conformance statement).
This statement shall contain only options within the framework of requirements specified within one
or multiple parts of ISO/IEC 18477.
6.5 Abstract test suites
The abstract test suites (ATS) define general tests for parts and/or profiles within parts of
ISO/IEC 18477. Each ATS includes the following parts and are defined in Annex C:
a) test purpose: what the test requirement is;
b) test method: the procedures to be followed for the given ATS;
c) reference: the portion of the ITU | ISO/IEC document that is being tested by the given ATS.
6.6 Decoder conformance testing procedures
Procedures for testing decoders and the ETS that shall be used are specified in Annexes A and B. These
procedures and ETS will allow an IUT to evaluate conformance to each part and profile within such part.
7 Copyright
These conforming files were originally developed by the parties indicated in the file COPYRIGHT and
README.License. In particular, the original developers of these files and their respective companies
or institutions, the editors and their companies/institutions, and ITU | ISO/IEC have disclaimed liability
for any purposed use of these files or modifications thereof.
8 Conformance files availability and updates
The conformance test images provided as an electronic attachment to this document are the latest
tested versions available at the date at which the text was released.
© ISO/IEC 2017 – All rights reserved 11

Annex A
(normative)
Decoder conformance testing procedures
A.1 General
This annex defines procedures to follow for determining whether a decoder is conforming to a
particular profile within a particular part.
A.2 Decoder test procedure
A.2.1 General
The procedure defined herein will determine whether a decoder is conforming to a certain part and
profile combination. The following steps for testing the set of codestreams are shown in the flow chart
in Figure A.1.
— Select a profile and a part to test against.
— The part and profile define a TCS (test codestream set).
— The codestream and part define an error metric to select.
— Each codestream is decoded using the decoder under test.
— The decoded outputs are format converted if necessary.
— The difference between the decoded outputs and reference outputs are measured with the part-
specific error metric.
— The measurements are compared with the limits for the particular image.
Each of these sets is defined in more detail in a later subsection of this annex. Failure to meet the
tolerance limits for a single image results in the decoder failing to be conforming with the given profile
and class being tested.
12 © ISO/IEC 2017 – All rights reserved

Figure A.1 — Decoder conformance test flow chart
A.2.2 Files for testing
A particular ETS defines the input codestreams, reference images, error metric and error tolerances.
These are specified in Annex B for all parts and profiles.
A.2.3 Decoder settings
Decoders may have mechanisms for supporting various decompression settings. These may be set in
the most advantageous way to achieve conformance. For example, a decoder with a "fast mode" and an
"accurate mode" may be set to the "accurate mode" to determine conformance. These settings should
be noted in any statement of conformance. Settings that allow the output resolution or spatial region of
the reference decoded images to be matched may be changed for each decoded image. The same user
controlled settings for accuracy or quantization reconstruction point of the DCT coefficients shall be
used for all test codestreams.
A.2.4 Output file format conversion
A.2.4.1 General
The reference decoded images are provided in a specific file format defined below. In order to compare
decoded images from the decoder under test with these images, several conversions may be necessary.
These conversions may be done as post-processing steps outside of the decoder solely for determining
conformance. There is no requirement for a conforming decoder to perform these processes as part of
© ISO/IEC 2017 – All rights reserved 13

its normal operation. These conversion shall not introduce a quality change (either loss or gain) except
as required by the specific conversions described in A.2.4.2.
For codestreams using the tools of ISO/IEC 18477-9, the output consists both of an image and an alpha
channel. To assess the correctness of both the image and the alpha channel, the decoder shall either be
instructed to generate two separate output files, one for the image data and one for the alpha channel,
or an implementation-specific post-processing shall be used to separate image data and alpha channel
data, and generate two files for the purpose of reference testing. There is no need for a decoder to
generate two separate streams in the presence of an alpha channel to be conforming to ISO/IEC 18477-
9, and creation of two separate files is only required for the purpose of testing.
A.2.4.2 Sample format conversion
Sample values reconstructed by decoders conforming to ISO/IEC 18477-1 and ISO/IEC 18477-6 are
always integers. Sample values of decoders conforming to ISO/IEC 18477-8 and ISO/IEC 18477-9 may
consist of integers depending on the parameters in the codestream. For the purpose of testing, it is of
advantage to represent the integer output of the IUT in the PNM format defined in A.2.5 as ITU | ISO/IEC
provide tools to measure on such files directly.
Samples of ISO/IEC 18477-2 and ISO/IEC 18477-7 are always, and samples of ISO/IEC 18477-8 and
ISO/IEC 18477-9 may consist of floating point numbers. For the purpose of testing, it is of advantage to
represent the floating point output of the IUT to the PNM format specified in A.2.5.
There is no requirement that a conforming decoder has to generate output to the specified format, and
the representation of the output of the IUT in this format only facilitates the testing process.
The alpha channel data, if present, may require scaling and conversion to integer samples. While
formally the alpha channel consists of floating point samples in the range [0,1], these samples shall be
converted to integer values for the purpose of testing. For that, each sample value shall undergo the
following transformation:
R +8
C = ⎿R ×(2 −1) + 0,5⏌ if the specified output transformation is a linear scaling
i i b
C = R if the output transformation is a half-logarithmic map
i i
Here, R is the reconstructed sample value of the alpha channel, as specified in ISO/IEC 18477-8:2016,
i
Annex A, R is the bit-depth of the alpha channel, defined by the Output Conversion box as a sub-box of
b
the Alpha Merging Specification box, and C is the input to the error metrics defined in A.2.6. In the first
i
case, i.e. an alpha channel converted to integer samples, the error metric of A.2.6.1 shall be used, in the
latter case, the error metric of A.2.6.2 shall be used.
Decoder implementations of ISO/IEC 18477-9 may already include such a conversion of the alpha
channel data or may, at their discretion, skip any linear output transformation. They may, hence, avoid
a scaling of the alpha channel data to [0,1], as specified in ISO/IEC 18477-9, and back to integer, as
necessary by the above formulae in this text. There is no requirement that an ISO/IEC 18477-9 decoder
implementation shall generate a separate alpha channel or shall scale it to a particular range. Scaling
and conversion are only required for the purpose of testing.
NOTE Conversion from YC C to RGB and clipping to range are a mandatory part of the ISO/IEC 18477 series
b r
specifications.
A.2.5 Reference components file format
A.2.5.1 General
This subclause describes the file format, called PNM, of the reference images used for comparison with
the output of the decoder under test. The decoder under test is not required to produce this particular
file format, though it is advantageous to perform a conversion to this file format for testing purposes as
ITU | ISO/IEC provides test tools that are able to decode this format. Any necessary conversion to this
format, as specified by A.2.4, may be applied.
14 © ISO/IEC 2017 – All rights reserved

The format consists of a header and raw data, concatenated to a single file.
A.2.5.2 Header format
The format of the header consists of
— two identifier bytes that specify the data format and encoding:
— which are the ASCII codes for "P6" (hex 0x50 0x36) for three-component integer sample images;
— the ASCII codes "P5" (hex 0x50 0x35) for one-component (grayscale) integer sample images;
— the ASCII codes "PF" (hex 0x50 0x46) for three-component floating-point sample images;
— the ASCII codes "Pf" (hex 0x50 0x66) for one-component floating point sample images;
— followed by any number of white space (ASCII 32, ASCII 8, ASCII 13 or ASCII 10, i.e. blank space, TAB,
CR or LF),
— a width in pixels formatted as ASCII encoded decimal number,
— any number of white space,
— a height in pixels formatted as ASCII encoded decimal number,
— for integer sample formats:
— any number of white space;
R +8
— the maximum sample value, i.e. 2 −1 where R + 8 is the bit precision of the integer samples;
b b
— a single ASCII line feed (hex 0x0a);
— for floating point sample formats:
— any number of white spaces;
— a floating point scale value, represented as decimal fraction. This number serves no particular
purpose in this document and, while present, shall be ignored;
— a single white space;
— the raw data formatted according to the following subclause follows this header immediately.
A.2.5.3 Data format
The binary data appears immediately after the line feed (hex 0x0a) byte in the header. The data is
stored with the most significant byte first, either in the IEC 60559 single-precision format for floating
point images or in binary two’s complement representation for integer sample images, sign extended to
either 1 byte, 2 bytes or 4 bytes. Components are interleaved in this format, in the order "Red, Green,
Blue" or "X, Y, Z", depending on the colour space of the image
A.2.6 Compare decoded and formatted components with reference components
A.2.6.1 Selection of the error metric
Once the decoded image has been converted to a suitable target format, it is compared with the
reference image. This document defines two error metrics to test the validity of the decoded image.
PSNR , defined in A.2.6.2 shall be used to assess the correctness of decoded images consisting
MSE
of integer samples, and hence shall be used to test implementations of ISO/IEC 18477-6, and some
configurations of ISO/IEC 18477-8 and ISO/IEC 18477-9.
© ISO/IEC 2017 – All rights reserved 15

The error metric PSNR defined in A.2.6.3 shall be used to assess the correctness of decoded images
MRSE
consisting of floating point samples, and hence shall be used to test implementations of ISO/IEC 18477-
7, and some configurations of ISO/IEC 18477-8 and ISO/IEC 18477-9.
The reference image is recorded in the format specified in A.2.5. While there is no particular
requirement that the decoded image is stored in the same format, it is helpful to perform this conversion
because ITU | ISO/IEC provide a command line tool that accepts this format and allows to carry out the
computations in this subclause.
A.2.6.2 Error metric for
...