ISO/IEC JTC 1/SC 24/WG 9 - Augmented reality continuum concepts and reference model

This document specifies: 1) Constructs for representing and specifying various augmentation and presentation styles. While augmentations can be in modalities other than the visual (e.g. aural, haptic), this work addresses the visual augmentation style only. 2) A model for how to associate the stylization constructs to the augmentation objects. Specifically, the MAR behavior object in ISO/IEC 3721 is extended for this purpose. 3) Other miscellaneous functionalities and abstractions that support the stylization of augmentation objects.

This document specifies the information model for representing the mixed and augmented reality (MAR) scene/contents description, namely, information constructs for: a) representing the virtual reality scene graph and structure such that a comprehensive range of mixed and augmented reality contents can also be represented; b) representing physical objects in the mixed and augmented reality scene targeted for augmentation; c) representing physical objects as augmentation to other (virtual or physical) objects in the mixed and augmented reality scene; d) providing ways to spatially associate aforementioned physical objects with the corresponding target objects (virtual or physical) in the mixed and augmented reality scene; e) providing other necessary functionalities and abstractions that will support the dynamic MAR scene description such as event/data mapping, and dynamic augmentation behaviours; f) describing the association between these constructs and the MAR system which is responsible for taking and interpreting this information model and rendering/presenting it out through the MAR display device.

This document specifies an image-based representation model that represents target objects/environments using a set of images and optionally the underlying 3D model for accurate and efficient objects/environments representation at an arbitrary viewpoint. It is applicable to a wide range of graphic, virtual reality and mixed reality applications which require the method of representing a scene with various objects and environments. This document: — defines terms for image-based representation and 3D reconstruction techniques; — specifies the required elements for image-based representation; — specifies a method of representing the real world in the virtual space based on image-based representation; — specifies how visible image patches can be integrated with the underlying 3D model for more accurate and rich objects/environments representation from arbitrary viewpoints; — specifies how the proposed model allows multi-object representation; — provides an XML based specification of the proposed representation model and an actual implementation example (see Annex A).

This document specifies: — physical and material parameters of virtual or real objects expressed to support comprehensive haptic rendering methods, such as stiffness, friction and micro-textures; — a flexible specification of the haptic rendering algorithm itself. It supplements other standards that describe scene or content description and information models for virtual and mixed reality, such as ISO/IEC 19775 and ISO/IEC 3721-1.

This document defines the framework and information reference model for representing sensor-based 3D mixed-reality worlds. It defines concepts, an information model, architecture, system functions, and how to integrate 3D virtual worlds and physical sensors in order to provide mixed-reality applications with physical sensor interfaces. It defines an exchange format necessary for transferring and storing data between physical sensor-based mixed-reality applications. This document specifies the following functionalities: a) representation of physical sensors in a 3D scene; b) definition of physical sensors in a 3D scene; c) representation of functionalities of each physical sensor in a 3D scene; d) representation of physical properties of each physical sensor in a 3D scene; e) management of physical sensors in a 3D scene; f) interface with physical sensor information in a 3D scene. This document defines a reference model for physical sensor-based mixed-reality applications to represent and to exchange functions of physical sensors in 3D scenes. It does not define specific physical interfaces necessary for manipulating physical devices, but rather defines common functional interfaces that can be used interchangeably between applications. This document does not define how specific applications are implemented with specific physical sensor devices. It does not include computer generated sensor information using computer input/output devices such as a mouse or a keyboard. The sensors in this document represent physical sensor devices in the real world.

This document defines a reference model and base components for representing and controlling a single LAE or multiple LAEs in an MAR scene. It defines concepts, a reference model, system framework, functions and how to integrate a 2D/3D virtual world and LAEs, and their interfaces, in order to provide MAR applications with interfaces of LAEs. It also defines an exchange format necessary for transferring and storing LAE-related data between LAE-based MAR applications. This document specifies the following functionalities: a) definitions for an LAE in MAR; b) representation of an LAE; c) representation of properties of an LAE; d) sensing of an LAE in a physical world; e) integration of an LAE into a 2D/3D virtual scene; f) interaction between an LAE and objects in a 2D/3D virtual scene; g) transmission of information related to an LAE in an MAR scene. This document defines a reference model for LAE representation-based MAR applications to represent and to exchange data related to LAEs in a 2D/3D virtual scene in an MAR scene. It does not define specific physical interfaces necessary for manipulating LAEs, that is, it does not define how specific applications need to implement a specific LAE in an MAR scene, but rather defines common functional interfaces for representing LAEs that can be used interchangeably between MAR applications.

This document defines the scope and key concepts of mixed and augmented reality, the relevant terms and their definitions and a generalized system architecture that together serve as a reference model for mixed and augmented reality (MAR) applications, components, systems, services and specifications. This architectural reference model establishes the set of required sub-modules and their minimum functions, the associated information content and the information models to be provided and/or supported by a compliant MAR system. The reference model is intended for use by current and future developers of MAR applications, components, systems, services or specifications to describe, compare, contrast and communicate their architectural design and implementation. The MAR reference model is designed to apply to MAR systems independent of specific algorithms, implementation methods, computational platforms, display systems and sensors or devices used. This document does not specify how a particular MAR application, component, system, service or specification is designed, developed or implemented. It does not specify the bindings of those designs and concepts to programming languages or the encoding of MAR information through any coding technique or interchange format. This document contains a list of representative system classes and use cases with respect to the reference model.

This document identifies the reference framework for the benchmarking of vision-based spatial registration and tracking (vSRT) methods for mixed and augmented reality (MAR). The framework provides typical benchmarking processes, benchmark indicators and trial set elements that are necessary to successfully identify, define, design, select and apply benchmarking of vSRT methods for MAR. It also provides definitions for terms on benchmarking of vSRT methods for MAR. In addition, this document provides a conformance checklist as a tool to clarify how each benchmarking activity conforms to this document in a compact form by declaring which benchmarking processes and benchmark indicators are included and what types of trial sets are used in each benchmarking activity.