kSIST ISO/PRF 24617-10:2024

International
Standard
ISO 24617-10
First edition
Language resource management —
2024-08
Semantic annotation framework
(SemAF) —
Part 10:
Visual information
Gestion des ressources linguistiques - Cadre d'annotation
sémantique —
Partie 10: informations visuelles (VoxML)
Reference number
© ISO 2024
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting on
the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address below
or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii
Contents Page
Foreword .iv
Introduction .v
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 Abbreviated terms . 2
5 Basic semantic assumptions — Habitats and affordances . 3
6 VoxML specification . 4
6.1 Metamodel and VoxML elements . .4
6.2 Representation of VoxML structures .5
6.3 Objects .6
6.4 Actions as programs .7
6.5 Relations .8
6.5.1 General .8
6.5.2 Properties (Attributes) .8
6.5.3 Relations .9
6.5.4 Functions .9
7 Examples of voxemes . 9
7.1 General .9
7.2 Objects .10
7.3 Eventualities as programs . 13
7.4 Properties .14
7.5 Relations . 15
7.6 Functions . 15
8 Using VoxML for simulation modelling of language .16
9 VoxML-based annotation scheme .18
9.1 Overview .18
9.2 Annotation scheme .18
9.2.1 Abstract specification .18
9.2.2 Concrete syntax for the representation of annotation structures .19
9.3 Semantic representation and interpretation . 20
Bibliography .22

iii
Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards
bodies (ISO member bodies). The work of preparing International Standards is normally carried out through
ISO technical committees. Each member body interested in a subject for which a technical committee
has been established has the right to be represented on that committee. International organizations,
governmental and non-governmental, in liaison with ISO, also take part in the work. ISO collaborates closely
with the International Electrotechnical Commission (IEC) on all matters of electrotechnical standardization.
The procedures used to develop this document and those intended for its further maintenance are described
in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for the different types
of ISO document should be noted. This document was drafted in accordance with the editorial rules of the
ISO/IEC Directives, Part 2 (see www.iso.org/directives).
ISO draws attention to the possibility that the implementation of this document may involve the use of (a)
patent(s). ISO takes no position concerning the evidence, validity or applicability of any claimed patent
rights in respect thereof. As of the date of publication of this document, ISO had not received notice of (a)
patent(s) which may be required to implement this document. However, implementers are cautioned that
this may not represent the latest information, which may be obtained from the patent database available at
www.iso.org/patents. ISO shall not be held responsible for identifying any or all such patent rights.
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and expressions
related to conformity assessment, as well as information about ISO’s adherence to the World Trade
Organization (WTO) principles in the Technical Barriers to Trade (TBT), see www.iso.org/iso/foreword.html.
This document was prepared by Technical Committee ISO/TC 37, Language and terminology, Subcommittee
SC 4, Language resource management.
A list of all parts in the ISO 24617 series can be found on the ISO website.
Any feedback or questions on this document should be directed to the user’s national standards body. A
complete listing of these bodies can be found at www.iso.org/members.html.

iv
Introduction
This document standardizes the specification of a semantic annotation scheme for visual information, based
on a modelling language for constructing three-dimensional (3D) visualizations of concepts denoted by
natural language (NL) expressions. This modelling language serves as a semantic basis of interpreting the
semantic forms of annotation structures model-theoretically by constraining the models for interpretation.
This document focuses on the introduction of the modelling language as a semantic basis for interpretation,
since the syntactic specification of the annotation scheme for visual information is a simplified formulation
based on the abstract specification of the spatio-temporal annotation schemes, such as those specified in
ISO 24617-1, ISO 24617-7 and ISO 24617-14. These three standards lay a theoretical basis for this document,
which specifies ways of annotating visual information involving motions and actions that are spatio-
temporally characterized.
The modelling language, named “VoxML” (visual object concept structure modelling language), where “Vox”
abbreviates “visual object concept structure” (VOCS), can be used as the platform for creating multimodal
semantic simulations in the context of human-computer communication. VoxML encodes semantic knowledge
of real-world objects represented as 3D models, and of events and attributes related to and enacted over
these objects. VoxML is intended to overcome the limitations of existing 3D visual markup languages by
allowing for the encoding of a broad range of semantic knowledge that can be exploited by a variety of
systems and platforms, leading to multimodal simulations of real-world scenarios using conceptual objects
that represent their semantic values.
NOTE 1 The main content of this document is based on References [1] and [2]. Reference [1] was developed by the
Brandeis University Computer Science Department in the context of communicating with computers (CwC), a Defence
Advanced Research Projects Agency (DARPA) effort to identify and construct computational semantic elements, for
the purpose of carrying out joint plans between a human and computer through NL discourse.
NOTE 2 This document adopts VoxML as a semantic basis for enriching the model for interpreting the descriptions
of objects, actions and relations involving dynamic visual information.
This document outlines a specification:
a) to formulate the annotation scheme for visual information;
b) to represent semantic knowledge of real-world objects represented as 3D models.
It uses a combination of parameters that can be determined from the object’s geometrical properties as
well as lexical information from NL, with methods of correlating the two where applicable. This information
allows for visualization and simulation software to fill in information missing from the NL input and
allows the software to render a functional visualization of programs being run over objects in a robust and
extensible way. Currently, a voxicon, which is the structured repository of visual object concepts, contains
500 object (noun) voxemes, lexemes or entries of the voxicon, and 10 program (verb) voxemes.
NOTE 3 As this library of available voxemes continues to grow, the specification elements will operationalize an
increasingly large library of various and more complicated programs. A voxeme library and visualization software
where users will be able to conduct visualizations of available behaviours driven by VoxML after parsing and
interpretation is available from Reference [25].

v
International Standard ISO 24617-10:2024(en)
Language resource management — Semantic annotation
framework (SemAF) —
Part 10:
Visual information
1 Scope
This document specifies an annotation language for visual information, based on VoxML (visual object
concept structure modelling language), a modelling language for the visualizations of concepts and actions
denoted by natural language (NL) expressions in three dimensions (3D).
The specification of the VoxML-based annotation scheme conforms to the requirements given in ISO 24617-1,
ISO 24617-7 and ISO 24617-14. The adoption of VoxML, specified in ISO 24617-14 as a semantic basis,
...