ISO/IEC JTC 1/SC 29 - Coding of audio, picture, multimedia and hypermedia information

This document provides reference software for Rec. ITU-T H.266 | ISO/IEC 23090-3. The reference software includes both encoder and decoder functionality. Reference software is useful in aiding users of a video coding standard to establish and test conformance and interoperability, and to educate users and demonstrate the capabilities of the standard. For these purposes, the accompanying software is provided as an aid for the study and implementation of Rec. ITU-T H.266 | ISO/IEC 23090-3.

This document describes the architecture of systems for the internet of media things. It also includes a comprehensive set of use cases that can be deployed on such an architecture.

This document specifies extensions to existing scene description formats in order to support MPEG media, in particular immersive media. MPEG media includes but is not limited to media encoded with MPEG codecs, media stored in MPEG containers, MPEG media and application formats as well as media provided through MPEG delivery mechanisms. Extensions include scene description format syntax and semantics and the processing model when using these extensions by a Presentation Engine. It also defines a media access function (MAF) API for communication between the Presentation Engine and the media access function for these extensions. While the extensions defined in this document can be applicable to other scene description formats, they are provided for ISO/IEC 12113.

The document specifies a set of tests and procedures designed to indicate whether encoders or decoders meet the requirements specified in ISO/IEC 23090-31.

This document contains simulation software for the MPEG-Immersive Audio standard as defined in ISO/IEC 23090-4.

This document specifies technology that supports the real-time interactive rendering of an immersive virtual or augmented reality audio presentation while permitting the user to have 6DoF movement in the audio scene. It defines metadata to support this rendering and a bitstream syntax that enables efficient storage and streaming of immersive audio content.

This document contains simulation software for the MPEG-H 3D audio standard as defined in ISO/IEC 23008-3.

This document contains the reference software of the ISO/IEC 21122 series. It acts as guidance for implementation of the ISO/IEC 21122 series and as a reference for conformance testing.

This document specifies an image coding technology known as JPEG AI learning-based image coding (JPEG AI) comprising an image coding technology that facilitates the compression and processing of images for both human and machine vision. The scope of the JPEG AI document is the creation of a learning-based image coding standard offering a single-stream, compact compressed domain representation, targeting both human visualization, with significant compression efficiency improvement over image coding standards in common use at equivalent subjective quality, and effective performance for image processing and computer vision tasks. The core coding system describes JPEG AI standard for the human vision reconstruction task and thus specifies the JPEG AI specifies the parsing coded stream and the image reconstruction process. Only the syntax format, semantics, and associated decoding process requirements are specified, while other matters such as pre-processing, the encoding process, system signalling and multiplexing, data loss recovery, post-processing, and video display are considered to be outside the scope of this document. This document is designed to be generic in the sense that it serves a wide range of applications, bit rates, resolutions, qualities and services. In the course of creating this document, several requirements from typical applications have been considered, necessary algorithmic elements have been developed, and these have been integrated into a single syntax. Hence, this document is designed to facilitate image data interchange among different applications and services.

This document specifies the syntax, semantics and decoding processes for MPEG immersive video (MIV), as an extension of ISO/IEC 23090-5. It provides support for playback of a three-dimensional (3D) scene within a limited range of viewing positions and orientations, with 6 Degrees of Freedom (6DoF).

This document specifies carriage of haptic media in ISO base media files.

This document specifies the system layer of the coding. It was developed principally to support the combination of the video and audio coding methods defined in Parts 2 and 3 of ISO/IEC 13818. The system layer supports six basic functions: 1) the synchronization of multiple compressed streams on decoding; 2) the interleaving of multiple compressed streams into a single stream; 3) the initialization of buffering for decoding start up; 4) continuous buffer management; 5) time identification; 6) multiplexing and signalling of various components in a system stream. A Rec. ITU-T H.222.0 | ISO/IEC 13818-1 multiplexed bit stream is either a transport stream or a program stream. Both streams are constructed from PES packets and packets containing other necessary information. Both stream types support multiplexing of video and audio compressed streams from one program with a common time base. The transport stream additionally supports the multiplexing of video and audio compressed streams from multiple programs with independent time bases. For almost error-free environments the program stream is generally more appropriate, supporting software processing of program information. The transport stream is more suitable for use in environments where errors are likely. A Rec. ITU-T H.222.0 | ISO/IEC 13818-1 multiplexed bit stream, whether a transport stream or a program stream, is constructed in two layers: the outermost layer is the system layer, and the innermost is the compression layer. The system layer provides the functions necessary for using one or more compressed data streams in a system. The video and audio parts of this Specification define the compression coding layer for audio and video data. Coding of other types of data is not defined by this Specification, but is supported by the system layer provided that the other types of data adhere to the constraints defined in 2.7.

This document specifies: — How to uniquely identify Digital Items (and parts thereof); — How to uniquely identify IP related to the Digital Items (and parts thereof), for example abstractions; — How to express the relationship between the two above identifiers; — How to deal with varying levels of functional granularity for Digital Item identifiers; — How to uniquely identify description schemes; — The relationship between Digital Items (and parts thereof) and existing identification systems. Annex C contains a list of relevant identification systems. This is not an exhaustive list and is subject to change over time; — How to express the relationship between two Digital Items. This document does not specify: — New identification systems for the content elements for which identification and description schemes already exist and are in use (e.g. this document does not attempt to replace the ISRC, as defined in ISO 3901, for sound recordings); — Normative description schemes for describing content.

This document describes the desired joint behaviour of MPEG-4 Systems (MPEG-4 File Format) and MPEG-4 Audio codecs. It is desired that MPEG-4 Audio encoders and decoders permit finite length signals to be encoded to a file (particularly MPEG-4 files) and decoded again to obtain the identical signal, subject to codec distortions. This enables the use of audio in systems implementations (particularly MPEG-4 Systems), perhaps with other media such as video, in a deterministic fashion. Most importantly, the decoded signal has nothing “extra” at the beginning or “missing” at the end. This permits: a) an exact "round trip" from raw audio to encoded file back to raw audio (excepting encoding artefacts); b) predictable synchronization between audio and other media such as video; c) correct behaviour when performing random access as well as when starting at the beginning of a stream; d) identical behaviour when edits are applied in the raw domain and the encoded domain (excepting encoding artefacts). It is also expected that there be predictable interoperability between encoders (as represented by files) and decoders. There are two kinds of audio "offsets" (or "delay" in the context of transmission): those that are result from the encoding process, and those that are result from the decoding process. This document is primarily concerned with the latter. These issues are resolved by the following: — The handling of composition time stamps for audio composition units is specified. Special care is taken in the case of compressed data, like HE-AAC coded audio, that can be decoded in a backward compatible fashion as well as in an enhanced fashion. — Examples are given that show how a finite length signals can be encoded to an MPEG-4 file and decoded again to obtain the identical signal, excepting codec distortions. Most importantly, the decoded signal has nothing “extra” at the beginning or “missing” at the end.

This document specifies the format of the Session-Based Description document and the media presentation description's (MPD) extension to be used in session-based operations with ISO/IEC 23009-1 (MPEG DASH).

This document specifies the encapsulation of codestreams specified in the JPEG 2000 family of Recommendations | International Standards into file formats derived from ISO/IEC 14496-12, including the file format specified in ISO/IEC 23008-12.

This document specifies advanced video coding for coding of audio-visual objects.

This document defines the JPEG Pleno framework for learning-based point cloud coding. This document is applicable to interactive human visualization, with competitive compression efficiency compared to state of the art point cloud coding solutions in common use, and effective performance for 3D processing and machine-related computer vision tasks, and has the goal of supporting a royalty-free baseline. This document specifies a coded codestream format for storage of point clouds. It provides information on the encoding tools. It also defines extensions to the JPEG Pleno File Format and associated metadata descriptors that are specific to point cloud modalities.

This document specifies the Image File Format, an interoperable storage format for a single image, a collection of images, and sequences of images. The format defined in this document is built on tools defined in ISO/IEC 14496-12 and enables the interchange, editing, and display of images, as well as the carriage of metadata associated with those images. The Image File Format defines structures used to contain metadata, how to link that metadata to the images, and defines how metadata of certain forms is carried. This document also specifies brands for the storage of images and image sequences conforming to High Efficiency Video Coding (HEVC), Advanced Video Coding (AVC), JPEG, Versatile Video Coding (VVC) and Essential Video Coding (EVC). NOTE The storage of HEVC, AVC, VVC and EVC video sequences is out of scope and is provided in ISO/IEC 14496-15.

The network-based media processing (NBMP) framework defines the interfaces including both data formats and application programming interfaces (APIs) among the entities connected through digital networks for media processing. Users can access and configure their operations remotely for efficient, intelligent processing. This document describes and manages workflows to be applied to the media data. This process includes uploading of media data to the network, instantiation of the media processing tasks, and configuration of the tasks. The framework enables dynamic creation of media processing pipelines, as well as access to processed media data and metadata in real-time or in a deferred way. The media and metadata formats used between the media source, workflow manager and media processing entities in a media processing pipeline are also specified.

This document provides context, motivation and use case descriptions for a set of Moving Picture Experts Group (MPEG) standards that collectively deliver media directly to render-based applications such as game engines with a renderer component, or standalone renderers. Emerging examples where such applications are especially relevant include metaverse applications and immersive displays where such displays provide an interface to renderers. This document: — describes the motivators leading to the development of new MPEG standards that facilitate the streaming of media to render-based applications; — differentiates between visual media distributed for video-based applications and visual media distributed to render-based applications; — provides an overview of a media workflow from content production to content distribution; — provides general information on relevant components of render-based systems including game engines and renderers — identifies key components and resources (compute, storage, or network) comprising a heterogeneous set of immersive displays and other render-based applications; — and documents use cases for end-to-end interoperability, including audio, video, graphics and systems aspects for render-based systems and applications.

This document specifies the reference software and conformance suite for carriage of G-PCC data as specified in ISO/IEC 23090-18. The information provided describes the reference software modules and the features that it supports. It includes the status of the development of the reference software for ISOBMFF encapsulation of carriage of G-PCC data. It also provides a description of how the reference software can be utilized and a description of conformance test vectors.

This document specifies information metadata, metrics metadata, clinical data linkage metadata, auxiliary fields, SAM interoperability, protection metadata and programming interfaces of genomic information. It defines: — metadata storage and interpretation for the different encapsulation levels as specified in ISO/IEC 23092-1 (in REF Section_sec_6 \r \h Clause 6 08D0C9EA79F9BACE118C8200AA004BA90B02000000080000000E000000530065006300740069006F006E005F007300650063005F0036000000 ); — metrics metadata containing sequencing data metrics at the dataset and access unit levels as specified in ISO/IEC 23092-1 (in REF Section_sec_7 \r \h Clause 7 08D0C9EA79F9BACE118C8200AA004BA90B02000000080000000E000000530065006300740069006F006E005F007300650063005F0037000000 ); — clinical data linkage metadata stored at the dataset group, dataset and annotation table levels as specified in ISO/IEC 23092-1 (in REF Section_sec_8 \r \h Clause 8 08D0C9EA79F9BACE118C8200AA004BA90B02000000080000000E000000530065006300740069006F006E005F007300650063005F0038000000 ); — protection elements providing confidentiality, integrity and privacy rules at the different encapsulation levels as specified in ISO/IEC 23092-1 (in REF Section_sec_9 \r \h Clause 9 08D0C9EA79F9BACE118C8200AA004BA90B02000000080000000E000000530065006300740069006F006E005F007300650063005F0039000000 ); — how to associate auxiliary fields to encoded reads (in REF Section_sec_10 \r \h Clause 10 08D0C9EA79F9BACE118C8200AA004BA90B02000000080000000F000000530065006300740069006F006E005F007300650063005F00310030000000 ); — interfaces to access genomic information coded in compliance with ISO/IEC 23092-1 and ISO/IEC 23092-2 (in REF Section_sec_12 \r \h Clause 12 08D0C9EA79F9BACE118C8200AA004BA90B02000000080000000F000000530065006300740069006F006E005F007300650063005F00310032000000 ); — mechanisms for backward compatibility with existing SAM content, and exportation to this format (in Annex E).

This document specifies a syntactic description language for describing the structure of binary data. It covers the representation of an SDL specification in plain text, the syntax of the SDL and the semantic rules of the SDL. In scenarios where the usage or interpretation of the SDL are ambiguous or undefined, this document attempts to specify whether such a scenario is considered an invalid SDL specification or will result in undefined behaviour. NOTE While the SDL borrows from and contains some aspects of a general-purpose programming language, it is not intended, nor is it suitable, to be used for such a purpose. This is reflected in the fact that many concepts related to general-purpose programming languages are not addressed in this document. Examples of concepts considered irrelevant to the SDL and therefore not addressed in this document include storage of an SDL specification in a file, compilation, execution, input/output, execution environment and machine architecture.

This document specifies formats for redundant encoding and packaging of live segmented media (REaP). This document specifies: a) formats for Interchangeable Live Media Ingest and stream announcement; b) format and segmentation strategy to generate interchangeable segments; c) formats for generating interchangeable media presentation descriptions or playlists; d) formats for efficient cloud storage access and archiving of live segmented media. e) a protocol for communicating media descriptor files and fragmented media between encoders and packagers. REaP enables the following: 1) failover support and rejoining of distributed components in the workflow (see Annex E); 2) workflows for live with dynamic ad insertion (DAI) with a decisioning system. 3) workflows with digital rights management (DRM) and content protection. 4) Mixing file and live inputs. This document specifies additional constraint to formats defined in ISO/IEC 14496-12, ISO/IEC 23009-1, ISO/IEC 23000-19 and IETF RFC 8216.

This document specifies the framework, concepts, methodology for testing, and criteria to be achieved to claim conformance to multiple parts of the ISO/IEC 21122 series. It lists the conformance testing procedures.

This document describes the reference software and conformance suite for the file format documents in multiple standards. Since these standards share a lot of technology, their reference software and conformance program are being handled together. These standards are: ISO/IEC 14496-12, ISO/IEC 14496-14, ISO/IEC 14496-15, ISO/IEC 14496-30 and ISO/IEC 23008-12. The purpose of the conformance suite is to cover the set of valid features that can be exercised in the file format. Media conformance is not covered, though of course to exercise the file format features, media will be stored.

This document specifies high efficiency video coding.

This document specifies technology for loudness and dynamic range control (DRC). It is applicable to most MPEG audio technologies. It offers flexible solutions to efficiently support the widespread demand for technologies such as loudness normalization and dynamic range compression for various playback scenarios.

This document specifies data formats and APIs for the mission management and control between MThings and end-users/system managers. Specifically, the following interfaces, protocols and associated media-related information representations are within the scope of this document: — structured data formats (XML) representing the mission assigned by the user to the network of IoMT, for the data formats; — structured data formats (XML) representing user commands to one or several MThings, possibly in a modified form (e.g. a subset of 1); — APIs to exchange the data for mission management and control.

This document specifies the syntax, semantics, and decoding for visual volumetric media using video‑based coding methods. Furthermore, this document specifies processes that may be needed for reconstruction of visual volumetric media, and may also include additional processes such as post‑decoding, pre-reconstruction, post‑reconstruction, and adaptation.

This document specifies conformance testing procedures for implementations of ISO/IEC 15938-17 and provides conformance bitstreams. It also provides the reference software for ISO/IEC 15938-17 which is an integral part of this document.

This document defines various code points and fields that establish properties of a video (or still image) representation and are independent of the compression encoding and bit rate. These properties can describe the appropriate interpretation of decoded data or can, similarly, describe the characteristics of such a signal before the signal is compressed by an encoder that is suitable for compressing such an input signal.

This document specifies the Multi-Image Application Format (MIAF), which contains coded images, groups and sequences of images along with their metadata and the information about their relations to each other, all embedded in the High Efficiency Image File (HEIF) format. This document builds on ISO/IEC 23008-12 (HEIF) and specifies the following: — a set of additional constraints on ISO/IEC 23008-12 (HEIF), to simplify its file format options; — specific alpha plane formats; — a set of specific profiles and levels for the supported coding formats; — a set of specific metadata formats; — a set of brands, including application brands indicating conformance with specific profiles; — a set of rules for extending MIAF format to support additional coding formats, profiles, levels and metadata. This document also defines the normative behaviour for a MIAF reader and MIAF renderer. This document (MIAF) is intentionally written to be extensible, and to allow for forward compatibility. The format is also permissive of the presence of other data, such as coding formats, metadata, and derived images.

This document specifies technology that supports the efficient transmission and rendering of haptic signals for the playback of immersive experiences in a wide variety of scenarios. The document describes in detail a robust coded representation of haptic media covering the two most popular haptic perceptions leveraged by devices today: vibrotactile and kinaesthetic. Support for other haptic modalities has also been integrated. The coded representation allows to encode both descriptive and quantized data in a human readable JSON format used for exchange purposes, and a compressed bitstream version, optimised for memory usage and distribution purposes. This approach also allows to meet the expectations for compatibility with both descriptive and quantized formats, as required by the market, as well as interoperability between devices for 3D immersive experiences, mobile applications and other distribution purposes. Information provided in this document related to the decoder is normative, while information related to the encoder and renderer is informative.

This document specifies a framework for establishing trust in media. This framework includes aspects of authenticity, provenance and integrity through secure and reliable annotation of the media assets throughout their life cycle.