ISO/IEC 14496-2:2001

INTERNATIONAL ISO/IEC
STANDARD 14496-2
Second edition
2001-12-01
Information technology — Coding of
audio-visual objects —
Part 2:
Visual
Technologies de l'information — Codage des objets audiovisuels —
Partie 2: Codage visuel
Reference number
©
ISO/IEC 2001
PDF disclaimer
This PDF file may contain embedded typefaces. In accordance with Adobe's licensing policy, this file may be printed or viewed but shall not
be edited unless the typefaces which are embedded are licensed to and installed on the computer performing the editing. In downloading this
file, parties accept therein the responsibility of not infringing Adobe's licensing policy. The ISO Central Secretariat accepts no liability in this
area.
Adobe is a trademark of Adobe Systems Incorporated.
Details of the software products used to create this PDF file can be found in the General Info relative to the file; the PDF-creation parameters
were optimized for printing. Every care has been taken to ensure that the file is suitable for use by ISO member bodies. In the unlikely event
that a problem relating to it is found, please inform the Central Secretariat at the address given below.

©  ISO/IEC 2001
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized in any form or by any means, electronic
or mechanical, including photocopying and microfilm, without permission in writing from either ISO at the address below or ISO's member body
in the country of the requester.
ISO copyright office
Case postale 56 • CH-1211 Geneva 20
Tel. + 41 22 749 01 11
Fax + 41 22 749 09 47
E-mail copyright@iso.ch
Web www.iso.ch
Printed in Switzerland
ii © ISO/IEC 2001 – All rights reserved

Contents
1 Scope.1
2 Normative references.1
3 Terms and definitions .2
4 Abbreviations and symbols.12
4.1 Arithmetic operators.12
4.2 Logical operators.13
4.3 Relational operators.13
4.4 Bitwise operators.13
4.5 Conditional operators.13
4.6 Assignment.13
4.7 Mnemonics.14
4.8 Constants.14
5 Conventions.14
5.1 Method of describing bitstream syntax.14
5.2 Definition of functions.15
5.2.1 Definition of next_bits() function .15
5.2.2 Definition of bytealigned() function.15
5.2.3 Definition of nextbits_bytealigned() function .15
5.2.4 Definition of next_start_code() function .16
5.2.5 Definition of next_resync_marker() function.16
5.2.6 Definition of transparent_mb() function.16
5.2.7 Definition of transparent_block() function.16
5.2.8 Definition of byte_align_for_upstream() function .16
5.3 Reserved, forbidden and marker_bit .16
5.4 Arithmetic precision.17
6 Visual bitstream syntax and semantics .17
6.1 Structure of coded visual data .17
6.1.1 Visual object sequence.18
6.1.2 Visual object.18
6.1.3 Video object.18
6.1.4 Mesh object.24
6.1.5 FBA object.25
6.1.6 3D Mesh Object.29
6.2 Visual bitstream syntax.30
6.2.1 Start codes.30
6.2.2 Visual Object Sequence and Visual Object.34
6.2.3 Video Object Layer .35
6.2.4 Group of Video Object Plane.40
6.2.5 Video Object Plane and Video Plane with Short Header .40
6.2.6 Macroblock.53
6.2.7 Block.58
6.2.8 Still Texture Object.59
6.2.9 Mesh Object.73
6.2.10 FBA Object.75
6.2.11 3D Mesh Object.85
6.2.12 Upstream message.103
6.3 Visual bitstream semantics .104
6.3.1 Semantic rules for higher syntactic structures.104
6.3.2 Visual Object Sequence and Visual Object.104
6.3.3 Video Object Layer .109
© ISO/IEC 2001 – All rights reserved iii

6.3.4 Group of Video Object Plane .120
6.3.5 Video Object Plane and Video Plane with Short Header .120
6.3.6 Macroblock related.131
6.3.7 Block related.134
6.3.8 Still texture object.135
6.3.9 Mesh object.142
6.3.10 FBA object.144
6.3.11 3D Mesh Object.151
6.3.12 Upstream message.162
7 The visual decoding process .164
7.1 Video decoding process .165
7.2 Higher syntactic structures .166
7.3 VOP reconstruction.166
7.4 Texture decoding.166
7.4.1 Variable length decoding.167
7.4.2 Inverse scan.168
7.4.3 DC and AC prediction for intra macroblocks.169
7.4.4 Inverse quantisation.172
7.4.5 Inverse DCT.175
7.4.6 Upsampling of the Inverse DCT output for Reduced Resolution VOP.176
7.5 Shape decoding.177
7.5.1 Higher syntactic structures .177
7.5.2 Macroblock decoding.178
7.5.3 Arithmetic decoding.187
7.5.4 Spatial scalable binary shape decoding.189
7.5.5 Grayscale Shape Decoding .198
7.5.6 Multiple Auxiliary Component Decoding .201
7.6 Motion compensation decoding.201
7.6.1 Padding process.201
7.6.2 Sample interpolation for non-integer motion vectors.205
7.6.3 General motion vector decoding process.207
7.6.4 Unrestricted motion compensation .209
7.6.5 Vector decoding processing and motion-compensation in progressive P- and S(GMC)-VOP .210
7.6.6 Overlapped motion compensation.212
7.6.7 Temporal prediction structure.213
7.6.8 Vector decoding process of non-scalable progressive B-VOPs .214
7.6.9 Motion compensation in non-scalable progressive B-VOPs .214
7.6.10 Motion Compensation Decoding of Reduced Resolution VOP .219
7.7 Interlaced video decoding .224
7.7.1 Field DCT and DC and AC Prediction .224
7.7.2 Motion compensation.225
7.8 Sprite decoding.234
7.8.1 Higher syntactic structures .234
7.8.2 Sprite Reconstruction.235
7.8.3 Low-latency sprite reconstruction.235
7.8.4 Sprite reference point decoding.236
7.8.5 Warping.237
7.8.6 Sample reconstruction.239
7.8.7 GMC decoding.240
7.9 Generalized scalable decoding .241
7.9.1 Temporal scalability.241
7.9.2 Spatial scalability.246
7.10 Still texture object decoding.251
7.10.1 Decoding of the DC subband .251
7.10.2 ZeroTree Decoding of the Higher Bands.252
7.10.3 Inverse Quantisation.257
7.10.4 Still Texture Error Resilience.265
7.10.5 Wavelet Tiling.268
7.10.6 Scalable binary shape object decoding .270
iv © ISO/IEC 2001 – All rights reserved

7.11 Mesh object decoding.276
7.11.1 Mesh geometry decoding .276
7.11.2 Decoding of mesh motion vectors.279
7.12 FBA object decoding.281
7.12.1 Frame based face object decoding.281
7.12.2 DCT based face object decoding .282
7.12.3 Decoding of the viseme parameter fap 1 .284
7.12.4 Decoding of the viseme parameter fap 2 .284
7.12.5 Fap masking.285
7.12.6 Frame Based Body Decoding.285
7.12.7 DCT based body object decoding.286
7.13 3D Mesh Object Decoding .287
7.13.1 Start codes and bit stuffing .288
7.13.2 The Topological Surgery decoding process.288
7.13.3 The Forest Split decoding process.291
7.13.4 Header decoder.292
7.13.5 partition type.293
7.13.6 Vertex Graph Decoder.294
7.13.7 Triangle Tree Decoder.298
7.13.8 Triangle Data Decoder.299
7.13.9 Forest Split decoder.303
7.13.10 Arithmetic decoder .309
7.14 NEWPRED mode decoding.314
7.14.1 Decoder Definition.314
7.14.2 Upstream message.314
7.15 Output of the decoding process .314
7.15.1 Video data.315
7.15.2 2D Mesh data.315
7.15.3 Face animation parameter data.315
8 Visual-Systems Composition Issues.315
8.1 Temporal Scalability Composition.315
8.2 Sprite Composition.316
8.3 Mesh Object Composition .317
8.4 Spatial Scalability composition.318
9 Profiles and Levels.318
9.1 Visual Object Types.318
9.2 Visual Profiles.321
9.3 Visual Profiles@Levels.322
9.3.1 Natural Visual.322
9.3.2 Synthetic Visual.322
9.3.3 Synthetic/Natural Hybrid Visual.324
Annex A (normative) Coding transforms.326
A.1 Discrete cosine transform for video texture.326
A.2 Discrete wavelet transform for still texture.327
A.2.1 Adding the mean.327
A.2.2 Wavelet filter.327
A.2.3 Symmetric extension.328
A.2.4 Decomposition level.329
A.2.5 Shape adaptive wavelet filtering and symmetric extension .329
A.3 Shape-Adaptive DCT (SA-DCT).330
A.3.1 Definition of Forward SA-DCT.330
A.3.2 Definition of Inverse SA-DCT .332
A.4 SA-DCT with DC Separation and ∆∆DC Correction (∆∆DC-SA-DCT) .333
∆∆ ∆∆
A.4.1 Definition of Forward ∆∆∆∆DC-SA-DCT .334
A.4.2 Definition of Inverse ∆∆∆∆DC-SA-DCT.334
Annex B (normative) Variable length codes and arithmetic decoding.336
B.1 Variable length codes.336
B.1.1 Macroblock type.336
© ISO/IEC 2001 – All rights reserved v

B.1.2 Macroblock pattern.338
B.1.3 Motion vector .340
B.1.4 DCT coefficients.342
B.1.5 Shape Coding.352
B.1.6 Sprite Coding.357
B.1.7 DCT based facial object decoding .358
B.1.8 Shape decoding for still texture object .367
B.2 Arithmetic Decoding.368
B.2.1 Aritmetic decoding for still texture object .368
B.2.2 Arithmetic decoding for shape decoding.371
B.2.3 FBA Object Decoding.374
Annex C (normative) Face and body object decoding tables and definitions.376
Annex D (normative) Video buffering verifier.409
D.1 Introduction.409
D.2 Video Rate Buffer Model Definition.409
D.3 Comparison between ISO/IEC 14496-2 VBV and the ISO/IEC 13818-2 VBV (Informative).412
D.4 Video Complexity Model Definition.413
D.5 Video Reference Memory Model Definition.415
D.6 Interaction between VBV, VCV and VMV (informative).416
D.7 Video Presentation Model Definition (informative) .416
Annex E (informative) Features supported by the algorithm .418
E.1 Error resilience.418
E.1.1 Resynchronization.418
E.1.2 Data Partitioning.419
E.1.3 Reversible VLC.419
E.1.4 Decoder Operation.420
E.1.5 Adaptive Intra Refresh (AIR) Method.423
E.1.6 NEWPRED.425
E.2 Complexity Estimation.427
E.3 Resynchronization in Case of Unknown Video Header Format.427
Annex F (informative) Preprocessing and postprocessing.428
F.1 VOP Generation Tools: Automatic and Semi-automatic Segmentations .428
F.1.1 Automatic Segmentation.428
F.1.2 Semi-automatic Segmentation.438
F.1.3 References.446
F.2 Bounding Rectangle of VOP Formation .447
F.3 Postprocessing for Coding Noise Reduction .448
F.3.1 Deblocking filter.448
F.3.2 Deringing filter.450
F.3.3 Further issues.452
F.4 Chrominance Decimation and Interpolation Filtering for Interlaced Object Coding.452
Annex G (normative) Profile and level indication and restrictions .454
Annex H (informative) Patent statements .457
H.1 Patent statements for ISO/IEC 14496 Version 1 .457
H.2 Patent statements for the extensions provided in ISO/IEC 14496 Version 2 .458
Annex I (informative) Encoder Complexity Reduction Based on Intelligent Pre-Quantisation.460
I.1 Introduction.460
I.2 Feature Selection and Pre-quantisation.460
I.3 Model Verification and Threshold Setting.462
I.3.1 H.263 Quantiser.462
I.3.2 MPEG-4 Quantiser.462
Annex J (normative) View dependent object scalability .464
J.1 Introduction.464
J.2 Decoding Process of a View-Dependent Object.464
J.2.1 General Decoding Scheme.464
J.2.2 Computation of the View-Dependent Scalability parameters .466
vi © ISO/IEC 2001 – All rights reserved

J.2.3 VD mask computation.468
J.2.4 Differential mask computation.469
J.2.5 DCT coefficients decoding .469
J.2.6 Texture update.469
J.2.7 IDCT.470
Annex K (normative) Decoder Configuration Information.471
K.1 Introduction.471
K.2 Description of the set up of a visual decoder (informative).471
K.2.1 Processing of decoder configuration information.472
K.3 Specification of decoder configuration information.473
K.3.1 VideoObject.473
K.3.2 StillTextureObject.473
K.3.3 MeshObject.474
K.3.4 FaceObject.474
K.3.5 3DMeshObject.474
Annex L (informative) Rate control .475
L.1 Frame Rate Control .475
L.1.1 Introduction.475
L.1.2 Description.475
L.1.3 Summary.479
L.2 Multiple Video Object Rate Control .479
L.2.1 Initialization.479
L.2.2 Quantisation Level Calculation for I-frame and first P-frame .479
L.2.3 Update Rate-Distortion Model .482
L.2.4 Post-Frameskip Control.482
L.3 Macroblock Rate Control .484
L.3.1 Rate-Distortion Model.484
L.3.2 Target Number of Bits for Each Macroblock .485
L.3.3 Macroblock Rate Control .485
Annex M (informative) Binary shape coding.487
M.1 Introduction.487
M.2 Context-Based Arithmetic Shape Coding.487
M.2.1 Intra Mode.488
M.2.2 Inter Mode.488
M.3 Texture Coding of Boundary Blocks .489
M.4 Encoder Architecture.489
M.5 Encoding Guidelines.490
M.5.1 Lossy Shape Coding.491
M.5.2 Coding Mode Selection.491
M.6 Conclusions.491
M.7 References.492
Annex N (normative) Visual profiles@levels.493
Annex O (informative) 3D Mesh Coding.497
O.1 Introduction.497
O.2 Topological Surgery Representation.497
O.2.1 Simple Polygon Representation.498
O.2.2 Vertex Graph representation .499
O.3 Encoding guidelines for 3D Mesh Coding .500
O.3.1 Topological Surgery Encoding .500
O.3.2 Support for non-manifolds and Non-orientable manifolds .501
O.3.3 Support for Error Resilience .503
O.4 Encoder considerations for efficient compression of Vertex Properties .507
O.5 Progressive Forest Split Representation.508
O.5.1 Encoding the Forest.508
O.5.2 Suppo
...