ISO/IEC 18038:2020

INTERNATIONAL ISO/IEC
STANDARD 18038
First edition
2020-04
Information technology — Computer
graphics, image processing and
environmental representation —
Sensor representation in mixed and
augmented reality
Reference number
ISO/IEC 18038:2020(E)
©
ISO/IEC 2020

---------------------- Page: 1 ----------------------
ISO/IEC 18038:2020(E)

COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2020
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting
on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address
below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Fax: +41 22 749 09 47
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii © ISO/IEC 2020 – All rights reserved

---------------------- Page: 2 ----------------------
ISO/IEC 18038:2020(E)

Contents Page
Foreword .v
Introduction .vi
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 Concepts . 4
4.1 Overview . 4
4.2 Scope of physical sensor representation . 5
4.3 Physical sensor types . 6
4.3.1 General. 6
4.3.2 Camera sensor . 7
4.3.3 Chemical sensor . . 7
4.3.4 Electric sensor. 8
4.3.5 Environment sensor. 8
4.3.6 Flow sensor . 8
4.3.7 Force sensor . 9
4.3.8 Light sensor . 9
4.3.9 Movement sensor .10
4.3.10 Navigation sensor .10
4.3.11 Particle sensor .11
4.3.12 Position sensor .11
4.3.13 Pressure sensor.12
4.3.14 Proximity sensor .12
4.3.15 Sound sensor .13
4.3.16 Temperature sensor .13
4.3.17 Other sensors .14
4.4 Sensor representation .14
4.4.1 Overview .14
4.4.2 Precise location and orientation of a physical sensor .16
4.4.3 Sensor properties and interface .19
4.4.4 Sensor representation data model .19
5 Sensor 3D scene graph .19
5.1 Definition of a sensor 3D scene graph .19
5.2 Physical properties and interfaces with real worlds .21
5.2.1 General.21
5.2.2 Physical properties of a physical sensor .21
5.2.3 Physical interfaces with real worlds of a physical sensor .22
5.2.4 A data structure for the physical properties and interfaces for a physical
sensor .23
6 System architecture for physical sensor representation .25
6.1 System architecture for physical sensors .25
6.2 System framework .26
6.2.1 General.26
6.2.2 3D MAR world representation .27
6.2.3 GNSS synchronized 3D virtual worlds .27
6.2.4 Sensor devices and their properties .28
6.2.5 Interfaces with 3D sensor objects .28
6.2.6 Interfaces with physical sensor devices .28
7 XML definition of physical sensor representation .28
7.1 Structure of mixed and augmented reality scene .28
7.1.1 MARScene .28
7.1.2 MARObject .28
© ISO/IEC 2020 – All rights reserved iii

---------------------- Page: 3 ----------------------
ISO/IEC 18038:2020(E)

7.1.3 3D object .28
7.1.4 Shape .28
7.1.5 Physical sensor .28
7.1.6 Sensor type .28
7.1.7 Physical properties .29
7.1.8 Physical interface.29
7.2 XML schema definition.29
7.2.1 MARSceneType .29
7.2.2 GeoPositionType .30
7.2.3 LatitudeType and LongitudeType .30
7.2.4 GeoBoundingBoxType . .31
7.2.5 LengthType .32
7.2.6 OrientationType .33
7.2.7 MARObjectType .35
7.2.8 ThreeDObjectType .35
7.2.9 ShapeType .36
7.2.10 AbstractSensorType .37
7.2.11 PhysicalPropertiesType .40
7.2.12 UUIDType, EventType and ControlType .40
7.2.13 PhysicalInterfaceType .41
7.2.14 IPAddressType and PortType .43
8 Conformance .44
8.1 Conformance criteria .44
8.2 Conformance area .45
Annex A (informative) Examples of physical sensor types and parameters .46
Annex B (informative) Schema for sensor MAR representation .54
Annex C (informative) Example XML schema extension for physical sensor representation .55
Annex D (informative) Example of sensor MAR representation based on the sensor MAR
schema .56
Annex E (informative) Implementation examples of sensor MAR representation .57
Bibliography .61
iv © ISO/IEC 2020 – All rights reserved

---------------------- Page: 4 ----------------------
ISO/IEC 18038:2020(E)

Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that
are members of ISO or IEC participate in the development of International Standards through
technical committees established by the respective organization to deal with particular fields of
technical activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other
international organizations, governmental and non-governmental, in liaison with ISO and IEC, also
take part in the work.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for
the different types of document should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www .iso .org/ directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject
of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent
rights. Details of any patent rights identified during the development of the document will be in the
Introduction and/or on the ISO list of patent declarations received (see www .iso .org/ patents) or the IEC
list of patent declarations received (see http:// patents .iec .ch).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to the
World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT)
This document was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 24, Computer graphics, image processing and environmental data representation.
Any feedback or questions on this document should be directed to the user’s national standards body. A
complete listing of these bodies can be found at www .iso .org/ members .html.
© ISO/IEC 2020 – All rights reserved v

---------------------- Page: 5 ----------------------
ISO/IEC 18038:2020(E)

Introduction
This document defines a representation model for physical sensors to be included in a 3D mixed-reality
world. It defines 3D modelling, rendering, simulation, and interfaces for physical sensors. It defines a
set of principles, concepts, and functionalities for physical sensors applicable to the complete range of
3D mixed reality standards. It includes the following content:
— terms and definition for sensor interfaces;
— requirements and scope;
— a representation model of physical sensors that can be included in a 3D scene;
— 3D modelling, rendering, and simulation of physical sensors in a 3D scene;
— representation of the attributes of physical sensors in a 3D scene;
— representation of I/O data streaming of physical sensors in a 3D scene;
— representation of the interfaces for controlling physical sensors in a 3D scene;
— functionalities and base components;
— relevant physical sensor properties;
— interfaces with virtual and real worlds;
— use cases.
The objectives of this document are as follows:
— provide a reference model for physical sensor-based 3D mixed-reality applications;
— manage and control physical sensors with their physical properties in 3D mixed reality environments;
— provide an exchangeable information model necessary for transferring and storing data between
sensor-based mixed-reality applications;
— support user interfaces with 3D mixed-reality worlds;
— support physical sensor interfaces with 3D mixed-reality worlds.
vi © ISO/IEC 2020 – All rights reserved

---------------------- Page: 6 ----------------------
INTERNATIONAL STANDARD ISO/IEC 18038:2020(E)
Information technology — Computer graphics, image
processing and environmental representation — Sensor
representation in mixed and augmented reality
1 Scope
This document defines the framework and information reference model for representing sensor-based
3D mixed-reality worlds. It defines concepts, an information model, architecture, system functions, and
how to integrate 3D virtual worlds and physical sensors in order to provide mixed-reality applications
with physical sensor interfaces. It defines an exchange format necessary for transferring and storing
data between physical sensor-based mixed-reality applications.
This document specifies the following functionalities:
a) representation of physical sensors in a 3D scene;
b) definition of physical sensors in a 3D scene;
c) representation of functionalities of each physical sensor in a 3D scene;
d) representation of physical properties of each physical sensor in a 3D scene;
e) management of physical sensors in a 3D scene;
f) interface with physical sensor information in a 3D scene.
This document defines a reference model for physical sensor-based mixed-reality applications to
represent and to exchange functions of physical sensors in 3D scenes. It does not define specific
physical interfaces necessary for manipulating physical devices, but rather defines common functional
interfaces that can be used interchangeably between applications.
This document does not define how specific applications are implemented with specific physical sensor
devices. It does not include computer generated sensor information using computer input/output
devices such as a mouse or a keyboard. The sensors in this document represent physical sensor devices
in the real world.
2 Normative references
There are no normative references in this document.
3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— ISO Online browsing platform: available at https:// www .iso .org/ obp
— IEC Electropedia: available at http:// www .electropedia .org/
3.1
3D object
collection of vertices in 3D space, connected by various geometric entities such as triangles, lines,
curved surfaces, etc.
© ISO/IEC 2020 – All rights reserved 1

---------------------- Page: 7 ----------------------
ISO/IEC 18038:2020(E)

3.2
augmented reality
AR
interactive experience of a real-world environment whereby the objects that reside in the real world
are augmented by computer-generated perceptual information
3.3
camera sensor
sensor (3.26) that detects and converts an optical image into an electronic signal
3.4
closed-circuit television
CCTV
video surveillance which uses video cameras to transmit a signal to a specific place on a limited set of
monitors
3.5
chemical sensor
sensor (3.26) that can analyse and provide information about the chemical composition of its
environment, that is, a liquid or a gas phase
3.6
electric sensor
sensor (3.26) that examines the change in electrical or magnetic signals based on an environmental input
3.7
environment sensor
sensor (3.26) that monitors relative ambient humidity, illuminance, ambient pressure and ambient
temperature
3.8
flow sensor
sensor (3.26) for detecting the rate of fluid flow
3.9
force sensor
sensor (3.26) for force detecting resistor whose resistance changes when force or pressure is applied
3.10
globally navigation satellite system
GNSS
satellite navigation system with global coverage
3.11
globally unique identifier
GUID
unique reference number used as an identifier in computer systems
3.12
light sensor
photodetector which detects changes in quantities of optical signal
3.13
mixed and augmented reality
MAR
integration of real and virtual worlds including mixed reality (3.14) and augmented reality (3.2)
2 © ISO/IEC 2020 – All rights reserved

---------------------- Page: 8 ----------------------
ISO/IEC 18038:2020(E)

3.14
mixed reality
MR
merging of real and virtual worlds to generate new environments where physical and synthetic objects
co-exist and interact
3.15
mixed reality system
system that can process mixed reality (3.14) applications with manipulation functions such as read,
write, import, export, modify, display, etc.
3.16
movement sensor
detector to detect a change in position of an object relative to its surroundings or the change in the
surroundings relative to an object
3.17
oxygen sensor
electronic device that measures the proportion of oxygen (O ) in the gas or liquid being analysed
2
3.18
particle sensor
detector to detect, track, and/or identify high-energy particles
3.19
physical device
real device containing a sensor (3.26) which is represented by a virtual device in a virtual environment
3.20
physical sensor
Internet of things (IoT) sensor (3.26) which has the functionality of a physical device (3.19) in a
3D virtual world
3.21
position sensor
sensor (3.26) that permits position measurement
3.22
pressure sensor
sensor (3.26) that measures pressure, typically of gases or liquids
3.23
programmable logic controller
PLC
digital computer used for automation of typically industrial electromechanical processes, such as
control of machinery on factory assembly lines, amusement rides, or light fixtures
3.24
proximity sensor
sensor (3.26) able to detect the presence of nearby objects without any physical contact
3.25
radio frequency identification
RFID
wireless use of electromagnetic fields to transfer data, for the purposes of automatically identifying
and tracking tags attached to objects
3.26
sensor
device to detect events or changes in its environment and send the information to other electronics
© ISO/IEC 2020 – All rights reserved 3

---------------------- Page: 9 ----------------------
ISO/IEC 18038:2020(E)

3.27
sound sensor
sensor (3.26) used to detect the sound intensity of the environment
3.28
temperature sensor
sensor (3.26) to detect a change in temperature
3.30
universally unique identifier
UUID
globally unique identifier
128-bit number used to identify information in computer systems
3.32
virtual world
collection of one or more virtual reality (VR) files and other multimedia content that, when interpreted
by a VR browser, presents an interactive experience to the user consistent with the author's intent
Note 1 to entry: Virtual reality (VR) is understood as an interactive computer-generated experience taking place
in a synthetic and simulated environment.
4 Concepts
4.1 Overview
This clause describes the concepts of sensor-based mixed and augmented reality, including definition,
objectives, sensor type, physical sensor representation, system functions for mixed and augmented
reality (MAR), MAR objects, MAR scene graph, and MAR world.
A mixed-reality world consists of a 3D virtual world and real-world sensors represented as 3D objects
with their physical properties. As a simple example, the conceptual scene of a mixed-reality world
is represented in Figure 1. It displays a heritage site represented by a 3D virtual world with a global
navigation satellite system (GNSS) sensor and a CCTV sensor. The virtual world is of a real heritage
location in a city and the character represents a tourist. GNSS information is displayed for the tourist
and a real CCTV device is located at its real physical location at the heritage site.
Figure 1 — Example sensor-based mixed-reality world
Once real physical sensors are integrated into a 3D virtual world, their physical properties can be
represented precisely in the virtual world. Sensor-based mixed reality is obtained by this convergence
of 3D with physical sensors in the real world. For sensor-based mixed reality, sensors in a 3D virtual
world are defined, and their information should be able to be transferred between applications, and
between a virtual world and a real world. This work is intended to define how to exchange AR/MR
application data in heterogeneous computing environments, and how physical sensors can be managed
and controlled with their physical properties in a 3D virtual world.
4 © ISO/IEC 2020 – All rights reserved

---------------------- Page: 10 ----------------------
ISO/IEC 18038:2020(E)

[11][14][18]
Physical sensors in the real world are many and varied . In order to control them in a 3D scene,
these physical sensors are classified based on their information types and functions. Types of physical
sensor devices include acoustic and sound, automotive and transportation, chemical, electric and
magnetic, environment and weather, flow and fluid, radiation and particle, navigation, position and
angle, speed and acceleration, optical and light, pressure, force and density, thermal and temperature,
proximity and presence, and video. Each sensor is represented as a physical device in a 3D scene
visually and/or functionally depending on the application and the type of sensor. Figure 2 shows an
indoor and an outdoor scene, each with many physical sensors. Each scene represents a corresponding
real world. The information and function of each sensor can be represented in the scenes. MAR scenes
with physical sensors can be used for representing and simulating the functions of the sensors and,
therefore, for managing the sensors using the 3D scene. These can also be used for facility management
[3]
in a real world .
This document focuses on how to represent physical sensors in a 3D scene, what to represent about
each physical sensor, what each sensor can do and the reason why each sensor needs this specification.
When representing a physical sensor in a 3D scene, having the sensor appear in the scene is optional
depending on the type of sensor and the application. Precise location and orientation o
...