ISO/IEC 14478-4:1998

INTERNATIONAL ISOAEC
STANDARD 14478-4
First edition
1998-12-15
Information technology - Computer
graphics and image processing -
Presentation Environment for Jlultimedia
Objects (PREMO) -
Part 4:
Modelling, rendering and interac ion
component
Technologies de I’informa tion - lnfographie et traitement d ’images -
Environnement de prksentation d ’objets multimedia (PREMO) -
Partie 4: Composant pour Ia mod&ation, Ie rendu et I’interaction

ISO/IEC 14478-4: 1998(E)
Contents
Foreword . . vii
...........................................
Introduction . Vlll
1 Scope.9
2 Normative references. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3 Definitions. . 10
..........................................
PREMO Part 1 definitions 10
3.1 .
PREMO Part 2 definitions .10
3.2 .
3.3 PREMO Part 3 definitions .
3.4 Additional Definitions .
.12
4 Symbols and abbreviations .
5 Conformance. . . 12
6 Overview of the Modelling, Rendering and Interaction Component. . .12
6.1 Introduction .12
.............................................
..12
6.2 Overview .
Devices for Modelling, Rendering, and Interaction . 16
6.3
Primitives and Coordinates. .17
6.4 .
6.4.1 Introduction
...................................................
0 ISO/IEC 1998
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized
in any form or by any means, electronie or mechanical, including photocopying and microfilm, without
permission in writing from the publisher.
ISO/IEC Copyright Office l Case postale 56 l CH- 12 11 Geneve 20 l Switzerland
Printed in Switzerland
ii
ISO/IEC 14478-4: 1998(E)
0 ISO/IEC
................................................... 17
6.4.2 Coordinates
............................... 17
6.4.3 The Primitive Hierarchy in PREMO
6.431 Overview.l 7
6.432 Captured Primitives. . 18
6.433 Form Primitives.~ 8
...............................................
6.434 Modifier Primitives. 18
6.435 Reference Primitives. . 19
6.436 Structured Primitives. . 19
6.437 Tracer Primitives. . 19
6.4.3.8 Wrapper Primitives. . 19
...................................... 19
6.4.4 Primitives and MRI Devices
Scene. 20
6.5
Interaction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . - . . . 21
6.6
6.7 Coordinators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
6.8 Dependencies on other Parts. . 23
........................................ 23
6.9 Subtyping Diagram.
7 Coordinates .
........................................ 23
7.1 General Coordinates
7.2 Colour. 24
7.3 TimeLocation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
8 Primitives. . . . . . . .
8.1 Introduction. 25
8.2 Captured Primitives. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
8.3 Form Primitives. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
8.3.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
8.3.2 Audio Primitives. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
8.3.3 Geometrie Primitives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
.............................................. 28
8.3.4 Tactile Primitives
................................................ 28
8.3.5 Text Primitives
Modifier Primitives. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
8.4
8.4.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . , . . . . . . . . . . . . . . . . . . . . . 28
8.4.2 Acoustic Modifiers . . . . . . . . . . . . . . . . . _ . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
8.4.3 Structural Modifiers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
8.4.4 TimeFrame Modifiers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
8.4.5 Visual Modifiers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
8.5 Reference Primitives. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
8.5.1 References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
8.5.2 The Name Object Type. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
8.6 Structured Primitives. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1
8.6.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1
8.6.2 Aggregate.3 1
............................................... 32
8.6.3 TimeComposite.
.......................................... 35
8.7 Tracer Primitives.
Wrapper Primitives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
8.8
....................... 36
9 Modelling, Rendering and Interaction Device.
9.1 Introduction .
iii
ISO/IEC 14478-4: 1998(E)
0 ISO/IEC
9.2 MRI ForInat.3 6
-
9.3 E~~iciency.3 6
9.4 Behaviour . .37
10 Modeller.3 7
11 Renderer.3 8
12 MediaEngine.3 8
13 Scene.39
14 Interaction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .~.42
14.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .42
14.2 InputDevice.42
14.3 Router.43
15 Coordinator . . . . . .~.~.~~~~. “.43
16 Functional Specification . . . . . o . . . . . e . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .45
16.1 Introduction . . . e . S . . . . . . . D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .45
16.2 Non-Object data types. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .45
16.3 Exceptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .46
16.4 Objects for coordinate spaces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .47
16.4.1 Coorchcrte Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
16.4.2 C&xlrobject. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .48
16.43 Timehcntiorz Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
16.5 Name Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .49
16.6
Objects for media primitives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .49
16.6.1 Primi~iw Object. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
16.6.2 Cq~turd Object. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . SO
16.63 Objects describing primitives with spatial and/or temporal form.
......... 50
16.631 Form Object
..................................................... SO
16.632 Ob.jects describing form primitives for audio media data
.................. SO
16.633 Ob.jects describing form primitives for geometric media data.
.............. 5 1
16.6.4 Objects describing primitives for the modification of media data
......... 52
16.6.4.1 Moclifkr.Object
................................................... 52
16.6.4.2 ObJet ’ ’ t* s d escribing modifier primitives for audio media data
............... 52
16.6.4.3
Ob.jects describing modifier primitives for structural aspects of media data. . 53
16.6.4.4 Tim Ft-twe ob-ject
................................................ 54
16.6.4.5 Ob-jects describing modifier primitives for visual aspects of media data.
...... S4
16.63 Refereuce Object
.............................................. .55
16.6.6 Objects for organising primitives into structures
...................... 56
16.6.6.1 Str.uctured ob.ject
................................................. 56
16.6.6.2 A,qyegclte ob-ject.
................................................. 56
16.6.6.3 Ob.jects for organising media data within time
.......................... 57
16.6.7 Tracer Object
.................................................. 58
16.6.8 Wrqyerobject
................................................ 59
16.7 Objects for describing properties of devices . . . . . . . . . . . . . . . . . . . . . .59
16.7.1
MRlJhrmat Object. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
16.7.2 l$fkierxyMeaswe Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
16.8 Processing devices for media data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .60
16.8.1 MRCDevice Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .60
IV
ISO/IEC 14478-4: 1998(E)
.............. . . . . . . . ” . . . . . . . . . . . . . . . . . . . . . . . 60
16.8.2 Modeller Object.
16.83 Reizderer Object. . . . * . . . . . . . . . . . . . . . . 61
. . . . . . . . . . . . . . . . .
16.8.4 MdiaEqine Object . . . 61
. . . 63
16.9 Seeuze Object. .
. . . 65
16.10 Objects for supporting interaction.
. . . 65
16.10.1 ItxputDevice Object .
. . . 66
16.10.2 Router Object. .
...............
16.11 Coordinator Object . . . 67
17 Component Specifkation. . a . I . < . . . . . . . . . . .a.* 69
A Overview of PREMO Modelling, Rendering and Interaction Object I ’ypes .
B Diagrammatic Conventions. .
............................................... 73
B.1 Introduction
General Graphical Signatures 73
B.2 .
Conventions for Devices and Communication . 74
B.3
........................ 75
C Relationship between Part 4 and the CGRM
............................................... 75
C.1 Introduction
C.2 Architectural Links. 75
........................................
C.3 Processing Links. 76
..........................................
C.4 Input and Output Primitives . 76
C.5 Storage . 76
A typical example scenario of MRI usage . . . . . . . . . . . . . . . . . . o . . . . . . . . 77
D
0 ISO/IEC
ISO/IEC 14478-4: 1998(E)
Foreword
ISO (the International Organization for Standardization) and IEC (the International
Electrotechnical Commission) form the specialized System for worldwide standardi-
zation. National bodies that are members of ISO or IEC participate in the development
of International Standards through technical committees established by the respective
organization to deal with particular fields of technical activity. ISO and IEC technical
committees collaborate in fields of mutual interest. Other international organizations,
government and non-governmental, in liaison with ISO and IEC, also take part in the
work.
‘In the field of information technology, ISO and IEC have established a joint technical
committee: ISO/IEC JTC 1. Draft International Standards adopted by the joint techni-
cal committees are circulated to the national bodies for voting. Publication as an Inter-
national Standard requires approval by at least 75% of the national bodies casting a
vote.
ISO/IEC 144784 was prepared by Joint Technical Committee ISO/IEC JTCl, hfor-
mation techology, Subcommittee SC24, Conzyuter graphics and image processing.
ISO/IEC 14478 consists of the following Parts under the general title Injkwzation
technology - Computer graphics and image processing - Presentation Environment
for Multimedia Objects (PREMO).
- Pm-t 1: Fundumen.tals of PREMO
- Part 2: Foundation component
- Part 3: Multimedia systerns Services
- Part 4: Modelling, rendering, und interaction component
Annex A forms an integral part of this part of ISO/IEC 14478. Annexes B to D arc for
information only.
vi
ISO/IEC 14478-4: 1998(E)
0 ISO/IEC
Introduction
The Modelling, Rendering and Interaction component of PREMO describes facilities
for the modelling and presentation of, and interaction with, multidimensional data that
utilises multiple media in an integrated way. That is, the data may be composed ofen-
tities that tan be rendered using graphics, Sound, Video or other media, and which may
be interrelated through both spatial coordinates and time.
The objective of this component is to provide developers and users of modelling and
rendering applications with a framework for supporting the definition and use of inter-
operable devices within a distributed setting. It achieves this by:
a) providing an extensible framework of primitives for use in modelling, render-
ing and interaction which encompass multiple media, and which tan be organized
into larger structures and embedded into Scenes.
b) extending the resource and device hierarchies of the PREMO Part 3 (Multi-
media Systems Services) Component to allow modelling, rendering and interac-
tion to be uniformally integrated into a network of objects for managing the
production and utilization of multimedia data.
c) utilizing the property and capability management Services of PREMO Part 3
to characterize the behaviour of modelling, rendering and interaction devices,
allowing an application to be configured from such devices such that constraints
on Performance and functionality are satisfied.
d) building on the Object model and foundation objects of PREMO Part 1 and
Part 2 to allow subsequent components to realize and extend specific modelling,
rendering and interaction functionality.
This component follows PREMO Part3 in describing the external interface of Object
types and other entities involved in modelling, presentation and interaction, but not the
internal structures needed to implement these. That is, it is not the purpose of this com-
ponent to provide a set of building blocks that tan be assembled into a modeller or a
renderer. Rather, the component provides facilities to enable devices, built with vari-
ous applications or Performance trade-offs in mind, to interoperate in a heterogenous
presentation environment.
vii
ISO/IEC 14478-4: 1998(E)
0 ISO/IEC
viii
INTERNATIONAL STANDARD 0 ISO/IEC ISO/IEC 14478-4: 1998(E)
Information technology - Computer graphics and image
processing - Presentation Environment for Multimedia Objects
(PREMO) -
Part 4: Modelling, Rendering, and Interaction Component S
1 Scope
This part of ISO/IEC 14478 describes a set of Object types and non-Object types to provide the construction of, presentation of,
and the interaction with Multimedia information. The multimedia information tan be graphics, Video, audio, or other types of
presentable media. This information tan be enhanced by time aspects. Throughout this document this part of ISO/IEC 14478 will
also be referred to as “Modelling, Rendering and Interaction ”, and abbreviated as MRI.
The Modelling, Rendering and Interaction Component constitutes a framework of ‘Middleware ’, System Software components
lying between the generic operating System and computing environment, and a specific application operating as a client of the
Services and type definitions provided by this component. It provides a framework over the foundation objects and multimedia
Systems Services defined in other Parts of the international Standard for the development of a distributed and heterogeneous net-
work of devices for creating multimedia models, rendering these models, and interacting with this process.
The Modelling, Rendering and Interaction Component encompasses the following characteristics:
a ) Provision of a hierarchy of multimedia primitives as an abstract framework for describing the capabilities of modelling
and rendering devices, and for enabling their interoperation;
b) within the primitive hierarchy, specific Provision for describing the temporal structure of multimedia data through the
stepwise construction of structured primitives from component data;
c) Provision of abstract types for modellers, renderers and other supporting devices, enabling the integration of such devices
or any future subtypes representing real Software or hardware, into a processing network of such devices;
d) provision of an Object type to map synchronization requirements expressed within multimedia primitives into control of
the stream and synchronization mechanisms provided by ISO/IEC 14478-2 and ISO/IEC 14478-3.
The Modelling, Rendering and Interaction Component relies on the Object types and Services defined in PREMO Part 2: Foun-
dation Components (ISO/IEC 1447%2), and PREMO Part 3: Multimedia Systems Services (ISO/IEC 14478-3).
2 Normative references
The following Standards contain provisions which, through reference in this text, constitute provisions of this part of ISO/IEC
14478. At the time of publication, the editions indicated were valid. All Standards are subject to revision, and Parties to agree-
ments based on this part of ISO/IEC 14478 are encouraged to investigate the possibility of applying the most recent editions of
the Standards indicated below. Members of IEC and ISO maintain registers of currently valid International Standards.
ISO/IEC 1 1072: 1992, Information technology - Computer graphics - Computer Graphits Reference Model (CGRM).
ISO/IEC 7942- 1: 1994, Ir$ormation technology - Computer graphics and image processing - Graphical Kernal System (GKS)
- Part 1: Functional description.
ISO/IEC 9592: 1997, Information technology - Computer graphics and image processing - Programmerk Hierarchical Inter-
active Graphits System (PHIGS).

0 ISOIIEC
ISO/IEC 14478=4:1998(E)
ISO/IEC 14478- 1: 1998, btformation technology - Computer graphics and image processing - Presentation Environment for
Multimedia Objects (PREMO) -Part 1: Fundamentals of PREMO
Computer graphics and image processing - Presentation Environment for
ISOIIEC 14478-2: 1998, Information technology -
- Part 2: Foundation component
Multimedia Objects (PREMO)
Computer graphics and image processing - Presentation Environment for
ISOIIEC 14478-3: 1998, Information technology -
- Part 3: Multimedia Systems Services
Multimedia Objects (PREMO)
3 Definitions
31 l PREMO Part 1 definitions
This part of ISO/IEC 14478 makes use of all terms defined in ISO/IEC 14478- 1 (Fundamentals of PREMO).
PREMO Part 2 definitions
32 .
This part of ISO/IEC 14478 makes use of all terms defined in ISO/IEC 14478-2 (Foundation component).
33 l PREMO Part 3 definitions
This part of .ISO/IEC 14478 makes use of all terms defined in ISOAEC 14478-3 (Multimedia Systems Services component).
Additional Definitions
34 .
For the purposes of this part of ISO/IEC 14478, the following definitions apply.
3.2.1 modeller: a virtual device that produces primitives on at least one output Port.
3.2.2 renderer: a device that accepts primitives on at least one of its input ports.
3.2.3 media engine: a virtual device that accepts primitives from at least one of its input ports, and produces primitives on
at least one of its output ports.
3.2.4 presentation: a collection of primitives that tan be perceived by the Operator.
3.2.5 coordinate: a primitive used to define a location in an nD space.
3.2.6 primitive: a structure describing information to be rendered, or information received through interaction.
3.2.6.1 form primitive: a primitive whose presentation has to be constructed by a renderer from an explicit description
in terms of aspects or properties that characterize a class of perceivable representations.
3.2.6.1.1 geometric primitive: a form primitive used to define a shape or extent within nD space.
3.2.6.2 captured primitive: a primitive for which some or all of the perceivable aspects of the primitive have been en-
coded in some format defined externally to the PREMO Standard.
structured primitive: a primitive that contains a collection of other primitives.
3.2.6.3
3.2.6.4 modifier primitive: a primitive that describes a Change to the presentation of another primitive.
3.2.6.4.1 acoustic modifier:
a modifier that changes properties of the Sound generated by other primitives.
3.2.6.4.2 structural modifier: a modifier that affects the spatial and/or temporal aspects of another primitive.
a modifier that affects the (non structural) visual appearance of a primitive.
3.2.6.4.3 visual modifier:
ISO/IEC 14478-4: 1998(E)
0 ISO/IEC
wrapper primitive: a primitive that carries a value drawn from some PREMO non-Object data type.
3.2.6.5
tracer primitive: a primitive that carries an event for use in monitoring and coordinating the transfer of media
3.2.6.6
data across a network.
3.2.7 input device: a device used to obtain data from the Operator.
3.2.8 graphics: the construction, manipulation, analysis and presentation of pictorial representations.
3.2.9 Scene: a device for storing and controlling access to a collection of primitive structures.
3.2.10 primitive structure: a collection of primitives organized into a structure that represents some or all of the data that
describes a multimedia presentation.
3.2.11 coordinator: a MRI device that tan manipulate the streams connecting its components.
3.2.12 router: a device for controlling the flow of data between streams connected to its ports.
temporal extent: the duration in time allocated or used for the presentation of some primitive.
3.2.13
3.2.14 operator-: the external Object that sends or receives information through a virtual device interface.
3.2.15 application: the external Object or client that uses a PREMO System by creating objects, invoking operations on ob--
jects, and using types defined by PREMO. Applications are not modelled in the PREMO System, but their interactions with
.a PREMO System are modelled.
The following alphabetical list gives the sub-clause of each definition.
3.2.6.4.1
acoustic modifiel
3.2.15
application
captured primitive 3.2.6.2
3.2.5
coordinate
3.2.11
coordinator
3.2.6.1
form primitive
oeometric primitive 3.2.6.1.1
c
graphics 3.2.8
input device 3.2.7
media engine 3.2.3
3.2.1
modeller
3.2.6.4
modifier primitive
3.2.14
Operator
3.2.4
presen tation
primitive 3.2.6
primitive structure 3.2.10
renderei 3.2.2
3.2.12
router
3.2.9
Scene
structural modifier 3.2.6.4.2
structured primitive 3.2.6.3
temporal extent 3.2.13
3.2.6.6
tracer primitive
3.2.6.4.3
visual modifier
3.2.6.5
wrapper primitive
ISO/IEC 14478-4: 1998(E) 0 ISO/IEC
4 Symbols and abbreviations
CGRM: Computer Graphits Reference Model.
CSG: Constructive Solid Geometry
GKS: Graphical Kerne1 System.
IEC: International Electrotechnical Commission.
ISO: International Organization for Standardization.
MPEG: Moving Picture Experts Group.
MRI: Modelling, Rendering and Interaction
Multimedia Systems Services
MSS:
PHIGS: Programmers Hierarchical Interactive Graphits System.
PREMO: Presentation Environment for Multimedia Objects.
VRML: Virtual Reality Modeling Language.
nD: Multi-dimensional.
2D: Two-dimensional.
3D: Three-dimensional.
Conformance
A conforming implementation of the PREMO Modellin g, Rendering and Interaction Component shall comply with the general
conformance rules defined in clause 5 of ISO/IEC 14478 1 and the component specification in clause 16.
6 Overview of the Modelling, Rendering and Interaction Component.
Introduction
61 .
This clause presents an overview of the modellin g, rendering, and interaction component (Part 4) of PREMO. It summarises the
concepts defined in the document, and explains how these concepts contribute to the goals set out in the Introduction. More de-
tailed descriptions of the concepts used in the overview are given in subsequent clauses. This part of ISO/IEC 14478 also makes
extensive use of facilities provided by PREMO Parts 1-3, in particular the device and stream concepts introduced in ISO/IEC
144783. A summary of these dependencies is included in this clause.
62 . Overview
The model underlying this part of ISO/IEC 14478 is that a multimedia System consists of modeilers, renderers, and other devices
(some media specific) linked together via streams that carry data of a particular format. These concepts of stream and device arc
those defined in ISO/IEC 14478-3. A device consists of a processing facility, together with a number of ports through which it
tan accept input and produce output, using a format defined by the Port. Figure 1 Shows a high-level view of a (simplified) ex-
ample System in which a graphical user interface is used to control Parameters of an audio-visual presentation System. Rectangles
on the sides of devices represent ports, and the thick lines between such ports represent media flow via streams. Thin lines rep-
resent other forms of interaction, for example Operation invocation. See also Annex B for a table defining the Symbols used in
the figure. The System in Figure 1 consists of:

ISOREC 14478-4: 1998(E)
43 lSO/IEC
from
application
Figure 1 - An audiohisual System
- two modellers, one for audio data and one for graphical data, that might be used to construct and edit primitives via an
interface to the application specific to the needs of each modeller;
- a Video engine, accepting a stream of Video data from the application and constructing a stream of information in .some
Video formst;
- a Scene, which encapsulates primitives produced by the two modellers and which mediates access to the collection of
primitives by both the modellers and the associated renderers;
- an audio engine, that takes primitives from the Scene and converts into a data format that tan be used to drive an audio-
specific device;
- a graphics engine (acting as a mixer or composition tool) uses the Video output from the Video renderer and primitives
from t he scen e to construct a further stream of pri miti ves, integrating the two sets of Source data within some appropriate
visual represe ntation. The p rimitives produced by t he e ngine may be some subset suita .ble for input to a specific renderer;
- two renderers, one for audio and one for graphics, that convert a stream of primitives into a form that tan be processed in
the context beyond the MRI network (in this case, presented to the end user of the application).
The t ‘igure Shows just one way in which such a System might be implemented. A different implementation may collapse the three
engines into a Single device, if it has access to a media engine that tan take both graphical and audio primitives as input and that
tan generate the correspondin g output streams. Another implementation may decompose the graphics engine into, for example,
a number of components that manage specific functionalities such as viewing or clipping. Finally, a high-performance implemen-
tation might collapse all components into a Single device.
Key components of this part are derived from the Object types defined in ISO/IEC 14478-3, in particular the property inquiry and
constraint facilities. These types and facilities may be used by the factory mechanism described in ISO/IEC 14478-2 to produce
ob-jects that meet certain requirements, and by the negotiation and QoS mechanisms for establishing and maintaining a network
of objects that satisfies specific properties. As a result, many of the Object types defined in this part have an associated list of
properties for use in creation and negotiation. For example, modellers and renderers are derived (indirectly) from the VirtualDe-
vice type of ISO/IEC 14478-3, and thus inherit the Property Inquiry Services. Esch renderer and modeller has a collection of prop-
erties that characterise its capabilities in terms of its inputs, Outputs and quality of Service. An application using the facilities of
this part tan request creation of a renderer from a renderer factory by invoking the ‘create-Object’ Operation using the type ‘ren-
derer’ as the name of the object type and passing a structure containing the required capabilities. Alternatively, an application that
is aware that the rendering interface it requires is defined by a specific subtype of the renderer tan request the factory to produce
a renderer of that specific type.
0 ISOIIEC
ISO/IEC 14478-4: 1998(E)
audio
from
application
t J
mouse
router
. . . ._.---.-~.~ __._.
Figure 2 - MRI network including interaction handling
For simplicity, Figure 1 illustrates devices used in presentation only. The same approach of using specialised virtual devices con-
nected by data streams is employed for handling input and interaction. Figure 2 extends the example with a simple framework
g. This introduces two new devices, and a new information flow between two renderers, shown in the outlined
for input handlin
regions of the figure.
- The mouse is an example of an InputDevice that tan provide primitives for processing elsewhere in the network, either
via a stream (as shown) or through a procedural interface or callback mechanism.
- A Router tan be incorporated to allow a data stream to be sent to specific devices depending on some internal state. The
Reuter device achieves this by also subtyping from the Controller Object defined in ISO/IEC 14478-2.
- Although Engirre objects primarily operate on streams of primitives intended for presentation, as the example Shows, an
engine may also have ports that are used to receive (and in the case of the graphics engine, to transmit) primitives used to
carry data about input.
Figure 1 and Figure 2 focus on the main streams and virtual devices involved in a simple MRI network. In addition to these, ISO/
IEC 14478-3 provides Object types for establishing and controlling a collection of streams and devices. These types are called
VirtualConnection and Group, respectively.
An instance of VirtuaZConnection is an Object that represents an abstract view of media transport between devices, allows control
over aspects of the connection, and is responsible for negotiating the connection in terms of formats and quality of Service con-
siderations etc. Several kinds of virtual connections are possible, depending on whether the devices have compatible ports, and
on whether a particular connection is local or networked. General examples of these are given in Annex D of ISO/IEC 14478-3.
If the ports of two devices are not compatible, or the devices are in different Parts of a distributed environment, connection adap-
tors will be employed. These adapters are an implementation concept, not visible to the application, and are not defined as an
Object type in the Profile of ISO/IEC 14478-3.
Independent from issues of distribution and the existente of virtual connections, the Group Object type provides applications with
the ability to manage a collection of objects that are instances of VirtualResource or its subtypes (e.g., devices and connections)
as a Single resource. As Group is subtyped from VirtualResource, this arrangement tan be hierarchical. Groups provides facilities
to acquire the resources needed to establish a number of connectionst to monitor the end-to-end quality of Service, and to provide
an application with a Single access Point for monitoring and controlling the flow of data across the resources that make up the
group.
ISWIEC 1447%4:1998(E)
0 ISO/IEC
A
__ I I
Q, IAih
auuw
dl
nnrtinn
renaerer
;i
t
d
\
B E
i”““’
t
-l-
I‘ SZ- L--l
!
;
A
virtual connection connection adaptor
Figure 3 - Groups and connections in audiohisual System
Figure 3 shows one arrangement ofgroups could be used to implement the audio-visual rendering example. It also illustrates, for
completeness, where connection adaptors may be required. In considering distribution, it has been assumed that the following
sets of components are each located at separate nodes of a distributed System:
a ) the modellers, the Video engine, and the Scene;
b) the audio engine and renderer;
c) the graphics engine, renderer, mouse, and router.
With respect to groups, there are a number of possible arrangements for the network. One such arrangement, consisting of four
groups (labelled A-D) has been shown in the figure. In more detail,
- Oroup A is the outer(most), and contains the other three groups, the mouse and router, and the connections and adaptors
used to link devices in Groups B, C and D;
- group B contains the modellers, the Video engine, and the Scene, plus the virtual connections needed to link devices
within the group;
ISO/IEC 14478-4: 1998(E) 0 ISO/IEC
- «roup C contains the audio engine and renderer, and the virtual connection between them;
e
- group D contains the graphics engine and renderer, and the virtual connection between them.
Although these groups in the example are loosely based around the devices located at each node there is no assumption or re-
quirement in PREMO that there should be such ‘a correspondence. Groups are a logical concept, and in general are independent
of the System Organisation. Although users of this part of ISO/IEC 14478 may need to be aware of groups and connections and
may wish to make use of them explicitly when constructing a MRI network, ISO/IEC 14478-3 also provides the LogicalDevice
Object types that tan be used to hide much of this detail from an application. Furthermore, for applications that distribute presen-
tation data (e.g. primitives, or other medi.a-specific formatsj to multiple renderers or devices, the use of Coordirmtor devices may
also be necessary. The Coordirzntor Object type is defined in this part of PREMO, and is a subtype of MRZJevice. Its role is
described in 6.7.
The purpose of this overview is to illustrate some of the key connections between the Object types defined in this part and those
introduced by ISO/IEC 1447% 1 to ISO/IEC 14478-3. In practice, the Object types mentioned in the discussion will only be a sub-
set of those utilized by an application. For example, in the audio-visual scenario, mouse input ,is passed ‘along a stream to the
graphics renderer, which may be providing immediate visual feedback on the changes made to certain control Parameters (per-
haps by adjusting the drawing of interface elements showing the volume, Speed etc. of theraudio and Video plavback). Other
M
mechanisms, in particular event handlers and reference Points, may also be present to control or mediate interaction between the
components of the system, “but have not been mentioned in this overview.
6 1 t
63 . Devices for Modelling, Rendering, and Interaction
This part of PREMO introduces a hierarchy of devices for processing streams of media data constructed using objects based on
the primitive hierarchy described elsewhere in this part (see 6.4). The root of this local hierarchy is the MRI-Device Object type
that extends the VirtunlDevice Object type defined in Part 3. This means that the devices defined in this part as subtypes of
MRl Device tan be directly integrated into a network of more general devices that may include media-specific input and output
-
devices cis well as more abstract processing nodes. An example of such a network has been developed in 6.2. As virtual devices,
MRl Device and its subtypes contain a number of ports that allow either input or output of data in a particular format. The Formnt
-
Object type is introduced in ISO/IEC 14478-3, and a small subtype hierarchy for certain common media formats is presented as
an informative annex (Annex C) of that part. The Modellin g, Rendering, and Interaction component defines a subtype of the For-
mnt Object type, called MR[-Format, that characterises a data stream that carries the primitives described in 6.4. Subsequent PRE-
MO components, or applications, may specialize this format Object type to define the input and output format of a MRI device
that tan utilise a richer collection of primitives. Figure 4 illustrates the relationship between the Format and MRI_Formnt Object
types, the examples of Format Object types from Annex C of ISO/IEC 14478-3, and any future extensions for processing special-
ised streams of MRI primitives.
Format
Formats for general devices,
including examples in part 3 Annex C
Formats for MRI devices
(application specific) defined
by applications or future PREMO components
Figure 4 - Format hierarchy
This part defines a number of subtypes of MRI-Device as abstract Object types for particular kinds of processing functionality.
Three of these generalize the traditional Computer graphics concerns, of modelling and rendering, to multimedia data processors.
These devices differ in the kind of interface that they are required to support for input and output.
A Modeller is a device that provides at least one output port that tan support MRI Format. In contrast, the input interface
-
of a modeller is typically application-specific. A CSG modeller might offer operations that allow the construction and
0 ISO/IEC ISOLIEC 14478-4: 1998(E)
manipulation of aCSG solid, while a modeller for music scores defined using audio primitives might have a direct manipu-
lation interface through which an end user interactively constructs the Score, which the modeller tan then pass as a stream of
primitives to other MRI devices for processing.
- A ReIzderer is a device that provides at least one input port that tan use MRI Format to accept primitives. The output
-
interface of a renderer is typically application specific. Renderers include devices that output primitives directly to end users
as a presentation (images, Sound etc.) as well as devices that drive further software or hardware components of a System
through some interface, the nature of which is not of concern to this part. This does not preclude a specific renderer from uti-
lising PRIEM0 functionality for its output mechanisms.
~ -4 MediaEngine is both a Modeller and a Renderer. A MediaEnghe is a device that offer at least one input port and at
least one output port that are capable of operating with MRI Format, allowing the engine to accept, process and produce
-
streams of primitives. Media engines, like other MRI devices, may have additional ports that allow the device to accept or
produce data in other formats.
NOTE - 11a ‘ media formst such as MPEG is to be taken directly as input by a renderer, as for example in the audio/visual System shown in
Figure 1, then the raw input stream may need to be converted into a stream of primitives in which the MPEG data is carried within the
primitive structure described shortly. This will allow additional information to be attached, such as a geometric data to characterise the
Position of. the Video data within a graphical Scene. This transformation may either be a capability of a renderer, or may be realized by a
specific device that acts as a translator or pre-processor. Such translators could be introduced by subsequent components, if any, as an Object
type derived from MedinE~zgirze. A translator wouid have exactly two ports, one accepting a stream of an arbierary format, the other generating
a stream of primitives, with the semantics that the translator simply ‘wraps’ the media stream into an appropriate primitive Object.
Primitives and Coordinates
64 .
6.4.1 Introduction
The MSS component, which pr-ovides the base Object types from which MRI devices are derived, contains no Statement on the
content and structure of the data carried on streams and operated on by virtual devices. The contribution of the MRI component
is to add just such information about the data used to realize multimedia presentation and interaction. This data, which is de-
scribed by a hierarchy of PREMO object types, are called primitives. The organization of primitive types into categories or hier-
archies varies widely in practice. This part of ISO/IEC 14478 defines primitive Object types where it is intended that client
applications, or any subsequent components that address the needs of specific domain areas or presentation technologies, extend
this hierarchy in a way most appropriate to their specific needs. By providing a common set of basic primitive types, this part of
PREMO is able to utilize the property and negotiation frameworks defined in ISO/IEC 14478-2 and ISO/IEC 14478-3 for estab-
lishing a network of MRI devices, and ensures that there is common vocabulary for basic MRI device capabilities.
6.4.2 Goordinates
Primitives within the PREMO hierarchy that address specific media such as graphics and audio are defined as abstract Object
types to ensure that they tan adequately be specialized to the needs of specific application areas or implernentations. As such,
primitive object types do not define the structure of primitives in terms of coordinates within the various spaces (cartesian or other
«eometric spaces, properties of Sound, etc.) through which primitives are specified in practice. However, to support inter-opera-
b
bility between implernentations of the primitive hierarchy, this part of ISO/IEC 14478 defines a generic Object type for coordi-
nates within spaces of arbitrary dimensionality, subject to the constraint that the components representing each dimension of a
coordinate are each drawn from the one PREMO type.
A spec ial iz ation of thi s notio n of coord inate, to the representation of colours within a colour model, is also provided by this part.
A map from fou1- col our models in to the dimensions of the colour objec t type is provided.
P ’ng
6.4.3 The Primitive Hierarchy in PREMO
6.4.3.1 Overview
Primitives are structures that primarily carry information that is to be rendered to the user-, or information that characterises input.
PREMO does not attempt to describe a closed set of primitives for modelling and rendering. Instead, the approach of this part is
to provide a general, extensibl
...