ISO/IEC TS 20748-3:2020

TECHNICAL ISO/IEC TS
SPECIFICATION 20748-3
First edition
2020-03
Information technology for learning,
education and training — Learning
analytics interoperability —
Part 3:
Guidelines for data interoperability
Reference number
©
ISO/IEC 2020
© ISO/IEC 2020
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting
on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address
below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Fax: +41 22 749 09 47
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii © ISO/IEC 2020 – All rights reserved

Contents Page
Foreword .iv
Introduction .v
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 Abbreviated terms . 2
5 Introduction to data APIs for LET purposes . 3
5.1 General . 3
5.2 Experience API (xAPI) . 3
5.3 IMS Caliper . 4
5.4 Brief comparison of xAPI and IMS Caliper . 4
5.4.1 Context for the comparison . 4
5.4.2 Detailed comparison . 6
6 Summary of use cases for data interoperability. 9
6.1 General . 9
6.2 Mapping rules for xAPI and IMS Caliper . 9
6.3 Customized profiles for each specification . 9
6.4 xAPI for non-specified data (non-profiled data).10
6.5 Necessary for guideline to use both specifications .10
7 Guideline for data interoperability .10
7.1 Understanding data flows for learning analytics and data collection process .10
7.2 Preparation for data interoperability .12
7.2.1 General.12
7.2.2 xAPI statements and relationship with its profiles .12
7.2.3 IMS Caliper information model and relationship with its profiles .14
7.3 Design guide for the mapping rules for xAPI and IMS Caliper.17
Annex A (informative) Use cases and practices for data interoperability .20
Bibliography .41
© ISO/IEC 2020 – All rights reserved iii

Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that
are members of ISO or IEC participate in the development of International Standards through
technical committees established by the respective organization to deal with particular fields of
technical activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other
international organizations, governmental and non-governmental, in liaison with ISO and IEC, also
take part in the work.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for
the different types of document should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www .iso .org/ directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject
of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent
rights. Details of any patent rights identified during the development of the document will be in the
Introduction and/or on the ISO list of patent declarations received (see www .iso .org/ patents) or the IEC
list of patent declarations received (see http:// patents .iec .ch).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to the
World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see www .iso .org/
iso/ foreword .html.
This document was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 36, Information technology for learning, education and training.
A list of all parts in the ISO/IEC 20748 series can be found on the ISO website.
Any feedback or questions on this document should be directed to the user’s national standards body. A
complete listing of these bodies can be found at www .iso .org/ members .html.
iv © ISO/IEC 2020 – All rights reserved

Introduction
The increasing amount of data being generated from learning environments provides new opportunities
to support learning, education and training (LET) in a number of new ways through learning analytics.
Learning analytics is terminology that is used to refer to both an emerging field of discourse and an
emerging technology. It spans the use of diverse sub-technologies, workflows and practices and
is applied to a wide range of different purposes. For instance, learning analytics are being used to
collect, explore and analyse diverse types and interrelationships of data, such as: learner interaction
data related to usage of digital resources; teaching and learning activity logs; learning outcomes and
structured data about programmes; curriculum and associated competencies.
As an emerging technology, learning analytics address a diverse group of stakeholders and cover a wide
range of applications. Learning analytics raise new interoperability challenges related to data sharing;
privacy, trust and control of data; quality of service, etc. The following issues are identified as general
requirements for learning analytics applications.
For the learner:
— tracking learning activities and progression;
— tracking emotion, motivation and learning-readiness;
— early detection of learner’s personal needs and preferences;
— improved feedback from analysing activities and assessments;
— early detection of learner non-performance (mobilizing remediation);
— personalized learning path and/or resources (recommendation).
For the teacher:
— tracking learners/group activities and progression;
— adaptive teacher response to observed learners' needs and behaviour;
— early detection of learner disengagement (mobilizing relevant support actions);
— increasing the range of activities that can be used for assessing performance;
— visualization of learning outcomes and activities for individuals and groups;
— providing evidence to help teacher improve the design of the learning experience and resources.
For the institution:
— tracking class/group activities and results;
— quality assurance monitoring;
— providing evidence to support the design of the learning environment;
— providing evidence to support improved retention strategies;
— support for course planning.
In addition, learning analytics practice can build upon prior work in LET standardization and innovation
but there are several factors that require special attention. These factors include:
— requirements arising from the analytical process;
— data items required to drive operational LET systems are not always the same as desired for learning
analytics;
© ISO/IEC 2020 – All rights reserved v

— volume, velocity and variety of the data collected for analytics indicate different IT architectures,
which imply different interoperability requirements;
— use of learner data for analytics introduces a range of ethical and other socio-cultural issues beyond
those which arise from exchanging data between operational systems.
Therefore, this document gives a conceptual description of the behaviour of components related
to learning analytics interoperability. In particular, this document specifies learning activity data
interoperability which focuses on xAPI and IMS Caliper for the learning analytics process and
interoperability.
Use cases will be collected to discover problems that arise in data transition points between
heterogeneous learning data in schools, higher education and the workplace.
vi © ISO/IEC 2020 – All rights reserved

TECHNICAL SPECIFICATION ISO/IEC TS 20748-3:2020(E)
Information technology for learning, education and
training — Learning analytics interoperability —
Part 3:
Guidelines for data interoperability
1 Scope
This document specifies guidelines for mapping between different learning analytics data
representations. Using xAPI and IMS Caliper as reference specifications, this document introduces data
API regarding learning analytics as well as guidelines to use the APIs, which can be generalized to other
contexts. Both syntactic and semantic mappings are in scope.
2 Normative references
There are no normative references in this document.
3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— IEC Electropedia: available at http:// www .electropedia .org/
— ISO Online browsing platform: available at http:// www .iso .org/ obp
3.1
assessment
means of measuring or evaluating learner understanding or competency
[SOURCE: ISO/IEC TR 20748-1:2016, 3.2]
3.2
dashboard
user interface based on predetermined reports, indicators and data fields, upon which the end user can
apply filters and graphical display methods to answer predetermined business questions and which is
suited to regular use with minimal training
[SOURCE: ISO TS 29585:2010, 3.3]
3.3
data analysis
systematic investigation of the data and their flow in a real or planned system
[SOURCE: ISO/IEC 2382:2015, 2122686]
3.4
data collection
process of bringing data together from one or more points for use in a computer
EXAMPLE To collect transactions generated at branch offices by a data network for use at a computer centre.
[SOURCE: ISO/IEC 2382:2015, 2122166]
© ISO/IEC 2020 – All rights reserved 1

3.5
data flow
movement of data through the active parts of a data processing system in the course of the performance
of specific work
[SOURCE: ISO/IEC 2382:2015, 2121825]
3.6
data source
functional unit that provides data for transmission
[SOURCE: ISO/IEC 2382:2015, 2124348]
3.7
data storage
means for storing information from which data is submitted for delivery, or into which data is put by
the delivery authority
[SOURCE: ISO/IEC 13888-1:2009, 3.7]
3.8
interoperability
capability to communicate, execute programs, or transfer data among various functional units in a
manner that requires the user to have little or no knowledge of the unique characteristics of those units
[SOURCE: ISO TS 19101-2:2008, 4.17]
3.9
learning analytics
measurement, collection, analysis and reporting of data about learners and their contexts, for purposes
of understanding and optimizing learning and the environments in which it occurs
[SOURCE: ISO/IEC 20748-1:2016, 3.11]
3.10
learning outcome
what a person is expected to know, understand or be able to do at the end of a training programme,
course or module
[SOURCE: ISO/IEC 17027:2014, 2.57]
4 Abbreviated terms
API application programming interface
LET learning, education and training
LMS learning management system
LRS learning record store
LTI learning tools interoperability
VLE virtual learning environment
2 © ISO/IEC 2020 – All rights reserved

5 Introduction to data APIs for LET purposes
5.1 General
In general, many meaningful data are generated through a variety of learning activities using
information and communication technology (ICT) in classrooms and/or online learning environments.
However, at the end of such activities or processes, these data are usually discarded or partially
extracted and recorded. For this reason, it is difficult to track what a learner has done and what skill level
the learner has. This also makes it difficult to provide personalized learning environments (PLE) or to
support adaptive learning. In many cases when this background information is missing, all learners are
targeted to average levels, in terms of one-size-fits-all, and follow-up activities continue in situations
where they do not comprehend a topic on their level of understanding. For example, in the physical
environment for education, summative assessment data such as test scores are recorded manually but
meaningful activity records are not accumulated in the learning processes. It is also difficult to provide
personalized feedback for learning attitudes, preferences or cognitive levels, because it only measures
the students' academic achievements through formalized tests. To overcome these limitations and to
support and motivate individual learners, a new approach using data-based services known as learning
analytics is being developed.
Systematic and accurate data collection is difficult due to diverse platforms and software used within
learning environments. To address this problem, data profiles and/or data collection APIs for collecting
learning data have been developed. However, different specifications for collecting learning data
may cause institutions to use different data APIs among their heterogeneous learning platforms and
software. There are two representative data profiles and APIs: Experience API (known as xAPI) and
IMS Caliper, which allows for detailed data capture about learners’ performance and learning activities/
events in the LET domain.
Enabling data collection regarding learning activities, xAPI and IMS Caliper are introduced in this
document as reference specifications for data APIs, and a comparison is done of the main features of the
specifications and their implications for developing guidelines, which can be generalized into guidelines
for data interoperability.
5.2 Experience API (xAPI)
According to the Advanced Distributed Learning Initiative (ADL), xAPI lets applications share data
about human performance (broadly defined). More precisely, xAPI lets service providers capture (big)
data on human performance, along with associated instructional content or performance context
information (i.e., experience). xAPI applies “activity streams” to tracking data and provides sub-APIs
to access and store information about state and content. This enables nearly dynamic tracking of
activities from any platform or software system – from traditional learning management systems to
mobile devices, simulations, wearables, physical beacons, and more.
xAPI can track micro-behaviours, state and context such as:
— reading an article or interacting with an eBook;
— watching a training video, stopping and starting it;
— training and behaviour data from a simulation;
— performance (user actions) in a mobile app;
— chatting with a mentor;
— physiological measures, such as heart-rate data;
— micro-interactions with e-learning content;
— team performance in a multi-player serious game;
© ISO/IEC 2020 – All rights reserved 3

— quiz scores and answer history by question;
— real-world performance in an operational context.
5.3 IMS Caliper ®
According to IMS Global, Caliper Analytics attempts to address the underlying interoperability
challenges in the learning technology landscape. Caliper provides an information model and a number
of metric profiles, each of which models a learning activity or a supporting activity that helps facilitate
learning. Each profile provides a domain-specific set of terms and concepts that application designers
and developers can draw upon to describe common user interactions in a consistent manner using
a shared vocabulary. Annotating reading materials, playing a video, taking a test, or grading an
assignment submission represent a few examples of the many activities or events that Caliper's metric
profiles attempt to describe.
IMS Caliper can track learning activities based on its profiles such as:
— Annotation profile: models activities related to the annotation of digital resources, such as creating
a bookmark, highlighting selected text, sharing a resource, tagging a document and viewing an
annotation.
— Assignable profile: models activities associated with digital content assigned to a learner for
completion according to specific criteria.
— Assessment profile: models assessment-related activities including interactions with individual
assessment items.
— Forum profile: models learners and others participating in online forum communities. Forums
typically encompass one or more threads or topics to which members can subscribe, post messages
and reply to other messages if a threaded discussion is permitted.
— Grading profile: models grading activities performed by an agent, typically a person or a software
application.
— Media profile: models interactions between learners and rich content such as audio, images and video.
— Reading profile: models activities associated with navigating to and viewing textual content.
— Session profile: models the creation and subsequent termination of a user session established by a
person interacting with a software application.
— Tool use profile: models an intended interaction between a user and a tool.
5.4 Brief comparison of xAPI and IMS Caliper
5.4.1 Context for the comparison
An initial comparison of the core features of the two specifications was developed in August 2016. The
comparison of the core features of the two specifications was based upon:
a) Use cases, scenarios and motivations – identification and clarification of the original scope and
context for xAPI and Caliper.
b) Service endpoints – identification of the types of data exchange that are supported and how this
data is exchanged between the endpoints.
c) Data models – a comparison of the core data features, i.e., this analysis does not work down through
the detailed data structures.
d) Security mechanisms – the data authentication, authorization and encryption mechanisms that are
supported and/or preferred.
4 © ISO/IEC 2020 – All rights reserved

e) Transport mechanisms – the payload exchange technology, i.e., the ways in which the data
definitions are physically exchanged across the networking technology.
f) Vocabularies, metric profiles, profiles and recipes – the mechanisms used to define the data
vocabularies and the tailoring of the specification for specific application domains and use-cases.
g) Data science – identification of the actual learning analytics that can be created and the associated
data science perspectives, e.g., statistical significance.
The key conclusions were:
a) Caliper and xAPI have very different origins. The core xAPI is to enable any type of experience
and evidence tracking, both electronic and physical performance and not limited to just web-
based courses (as is the case for SCORM). Caliper is the manifestation of the IMS learning analytics
framework and the sensor API and profile(s) are the first two components of that framework. xAPI
and Caliper are not equivalent. Adoption should not be ‘one-or-the-other’, instead it should be a
decision as and where appropriate for specific needs.
b) While both xAPI (actor/verb/object) and Caliper (actor/action/activity) use a data model based
upon a triple statement structure, there are considerable differences in the detailed structure and
usage of the object and activity definitions. However, it should be possible for each specification to
make use of the other’s verb/action.
c) A formal processes for the definition and/or modification of the vocabularies, metrics profiles,
profiles and recipes would need to be established with exemplars created to demonstrate the best
practices when producing the corresponding documentation.
d) Any further work on either/both standards would need to include explicit participation of data
scientists with knowledge of learning analytics. It needs to be ensured that the use of xAPI/Caliper
can produce useful learning analytics information and not just data.
From a technology realization perspective, for a next generation, it would simplify common adoption
and convergence to agree common payload binding (currently JSON against JSON-LD), a common
security framework (currently OAuth 1 against APIkey), a common secure transport mechanism
(currently HTTPS) and a common endpoint definition approach (including common agreement on the
use of query parameters and URL construction).
Areas out-of-scope for the 2016 comparison were:
a) A detailed comparison of the approaches used by xAPI and Caliper for a specific use-case.
b) Establishing the scenarios for which xAPI or Caliper should be deployed.
c) Making decisions. This was purely an information-gathering exercise.
© ISO/IEC 2020 – All rights reserved 5

5.4.2 Detailed comparison
— Use cases / scenarios / motivations
xAPI features Caliper features
The core use cases are: The core usages are:
— To enable learning within the SCORM context — To enable the creation of quantitative metrics for
and beyond-the-browser, outside of the LMS and learning;
outside of the SCORM package;
— To provide real-time data messaging to enable
— Distributed content: any type of learning content responsive learning engagement as opposed to
or experience can be delivered from a local just archive-based metrics;
computer, local network or on any remote servers;
— To provide details on student engagement in
— Distributed data: learning data can be stored and learning activities;
shared across one or more systems;
— To resolve the LTI/black-box conundrum.
— Usage and performance data: paradata about
Caliper is IMS’s learning analytics framework of which
learning resources that include not just
the sensor and metric profiles are just two components.
quantitative metrics, but also pedagogic context,
skills, and performance;
— Team-based scenarios: data associated with users
can now be aggregated and associated with a team
or group of users;
— Instructor/facilitator scenarios: instructor or
facilitators may observe and send or receive
feedback or annotations to users during a learning
or performance activity using real-time data
collection displayed in an interface or dashboard.
— To provide system-to-system data transfer
(including non-LRS based) that allows
identification of data ownership, multi-agent
statements, with an extensible data model and
agnostic of security model.
— Service endpoints
xAPI features Caliper features
Supports both ‘reading data from’ and ‘writing data to’ The sensor API is used to write/post data to a Reposi-
an LRS. Explicit support for: tory endpoint:
— Statement API – create/read; — send () – to transmits event data;
— State API – CRUD; — describe () – to transit entity data.
In version 1.x, Caliper does not support reading data
— Agent profile API – CRUD;
from a data repository.
— Activity profile API – CRUD;
EXAMPLE  The sensor API is for writing data to a
repository.
— About resource – read information about the
endpoint.
6 © ISO/IEC 2020 – All rights reserved

— Data models
xAPI features Caliper features
The data model is based upon the statement and this The data model is based upon the {actor, action, activ-
{actor, verb, object} triple: ity} triple:
— An actor is an agent or group (two or more agents);
—  The vocabulary for the action is controlled and
constrained by the metric profiles.
— There are four types of object, i.e., activity, agent,
The data is exchanged either as a set of events or
statement or sub-statement. statements can be
entities with an entity used to describe actors and
composed of sub-statements;
activities. entities provide context for the events. each
event is defined by a ‘metric profile’.
— The vocabulary for the verb, activity types and
extensions are open.
Events do not have explicit dependencies, i.e., they
xAPI can be considered an ‘activity scripting language’.
must be associated through the use of a session.
Caliper can be considered an ‘event scripting language’.
— Security mechanism
xAPI features Caliper features
Support for: Use of API key.
— Basic HTTP authentication;
Use of HTTPS/TLS 1.3 is recommended to secure the
message exchange.
— Use of 2-legged and 3-legged OAuth 1.0 (with
Very little discussion of security.
HMAC-SHA1, RSA-SHA1 and PLAINTEXT) for
statement authorization.
There is lot of information on authorization.
— Transport mechanism
xAPI features Caliper features
The transport is HTTP/HTTPS with JSON payloads. The transport is HTTP/HTTPS with JSON-LD payloads
Supports both the requestor (source) and the respond- (note that the linked data aspects are not subject to
ent (LRS) allocating the unique identifier for a State- conformance). The message is not signed. For conform-
ment that is to be stored. ance, IMS do not address the linked data aspects and
treat the payload as formal JSON.
Statements can be signed and the signature may also
be stored in the LRS. There is a best practices recommendation for using LTI
to provide the sensor endpoint and the corresponding
API key.
© ISO/IEC 2020 – All rights reserved 7

— Vocabularies, metric profiles, profiles and recipes
xAPI features Caliper features
An xAPI profile is similar to an application profile, i.e., A metric profile describes either a learning activity
it is a definition of how to use the xAPI specification or an activity that facilitates learning, e.g., grading.
in a specific application domain. A recipe is used to Profiles are composed of one or more events. Each
describe how xAPI supports a specific use-case in the event specifies a controlled vocabulary of actions as
application domain, e.g., the sequence of statements. well as a set of entities that provide a representation
xAPI profiles have the following characteristics: of the actor, object of the interaction as well as other
elements that together comprise the learning context
— Documentation on how to use the profile;
in which the action is situated.
— IRIs used as part of vocabulary;
For conformance, a vendor must identify the metric
profiles supported by their implementation. support
— Vocabulary metadata (new vocabulary guidance
for metric profiles is optional. Also, an implementation
suggests all should be marked up and published
is only required to support a predefined subset of the
using RDF/JSON-LD) and returned if IRI requested.
actions for a metric profile.
Could this be improved to support conformance
NOTE  There is no established process for the defi-
testing or profile conformance?
nition and/or modification of a metric profile. once
— Recipes, for standardized tracking of common
broader adoption of Caliper has been established IMS
activities;
will also enable domain profiling (as provided for all
deployed IMS specifications) of the metric profiles, i.e.,
— Rules for authentication, security;
refinement of the metric profile for a market specific
requirement.
— Specific launch requirements;
— Specific packaging requirements;
— Storage requirements using Document API;
— Specific sequence of statements or rules (cmi5
uses initialized verb first);
— Granularity or aggregation of statements.
NOTE  There is no established process for the defini-
tion and/or modification of a profile and/or recipe.
— Data science
xAPI features Caliper features
Provides support for identifying an authority (an agent Establishment of metric profiles is meant to reflect
or group of agents) who asserts statement(s) are true. instructionally meaningful interactions. However, the
The authority is achieved using 2-legged or 3-legged current version represents only initial versions of the
OAuth. An LRS must ensure that all statements stored metric profiles.
have an authority.
NOTE  Missing aspects for data science are:
NOTE  missing aspects for data science are:
— The required instrumentation of data sources to — The required instrumentation of data sources to
provide sufficiently rich/useful learning analytics; provide sufficiently rich/useful learning analytics;
— Data provenance and curation; — Data provenance and curation;
— Workflows between the various systems; — Workflows between the various systems;
— Consistent usage/mapping of verbs/actions taking — Consistent usage/mapping of verbs/actions taking
into account evolution of the standards to produce into account evolution of the standards to produce
consistent data science; consistent data science;
— Supporting and processing of extensions to the — Supporting and processing of extensions to the
standards and the process for disseminating/ standards and the process for disseminating/
adopting such extensions. adopting such extensions.
8 © ISO/IEC 2020 – All rights reserved

6 Summary of use cases for data interoperability
6.1 General
The use cases illustrate key features related to data interoperability by focusing on particular
requirements that stakeholders could have and then outlining how such requirements can be reflected
in learning analytics interoperability. A total of seven use cases were received and evaluated.
Use cases considered four main areas:
— mapping rules for xAPI and IMS Caliper;
— customized profiles for each specification;
— xAPI for non-specified data (non-profiled data);
— necessity for a guideline to use both specifications.
The summary of the use cases is presented in this clause. The complete list of use cases is available in
Annex A.
6.2 Mapping rules for xAPI and IMS Caliper
Some institutions and platform providers have developed their own mapping rules for xAPI and IMS
Caliper, which are composed of triple bindings of subject-predicate-object structures to support
different data sources among its tools or environments. There are generally two kinds of mapping rules:
mapping rules between xAPI and IMS Caliper directly and mapping rules for three features among xAPI,
IMS Caliper and data properties of an LMS/VLE. The purpose of mapping to xAPI and IMS Caliper is to
match syntactic structures of learning data and semantics of properties from different data sources.
A common way for mapping to xAPI and IMS Caliper is to match properties per learning event, such as,
viewing topics on a discussion forum, submission of an assignment, trying assessment items, etc. This
work can be understood for syntactic mapping among data structures of xAPI, IMS Caliper or the user's
own data profiles. While the mapping of syntactic structures can be resolved through rules with small
effort, semantic matching between terms or vocabularies of data APIs does not seem easy in terms
of initial building and maintenance for terminologies (or ontologies). For this reason, institutions and
learning platform providers seem to prefer to take syntactic mapping only, leaving the remaining task
for semantic matching works for the data analysis process of learning analytics.
Mapping rules and examples are introduced in 7.3.
6.3 Customized profiles for each specification
International stakeholders require customized profiles to fit their educational circumstances. In
the case of xAPI, while it has simple statement structures to describe diverse learning activities, it
is necessary to manage vocabularies regarding the verbs presented in actions by the system and/or
learners. Some stakeholders of the xAPI domain have built management mechanisms called recipes, but
it is necessary to clarify the meanings of recipe and profile and what is different in both applications.
On the other hand, IMS Caliper has defined very specific properties and lists regarding actions of
systems and/or learners. While this detailed information is helpful for stakeholders to understand
components of learning events, institutions or LMS/VLE providers need to prepare profiling for the
selection of properties and vocabularies from the IMS Caliper specification to make suitable design
data APIs for their services.
While ADL and IMS Global have tried to define specification and profile, ADL separates specification and
profiles with the aim of improving practices for creating profiles as defined in the xAPI specification,
and IMS Global also divides their information model which defines a set of concepts, rules and
relationships for describing learning activities and defines a number of metric profiles, each of which
models a learning activity or a supporting activity that helps facilitate learning.
© ISO/IEC 2020 – All rights reserved 9

The profiles for xAPI and IMS Caliper are described in 7.2 in detail.
6.4 xAPI for non-specified data (non-profiled data)
If a learner receives the message “focus on the capture box in the target direction (or image)” on the app
screen, the learner will take actions following this message and data API can record this to describe
the action, the duration time, count the number of trials, etc. As in this situation, recently emerging
data sources from heterogeneous environments, such as AR/VR content, wearable devices and sensors
around physical spaces, are increased rapidly. One of the interesting use cases is using xAPI to collect
learning data from augmented reality (AR) and virtual reality (VR) apps or content.
In general, AR apps or VR content are very often required operations and interactions by learners, and
these actions are described in narrative ways. One use case in Annex A shows how to extract learning
activity data using the xAPI statement structure without controlled terminology. The reason the data
was extracted without controlled terminology is that they separate the roles for data capturing and
analysing learning data similarly with semantic matching between xAPI and IMS Caliper. It is not
difficult to translate app-dependent data to the xAPI statement following syntactic principles. Once
extracted, learning data from AR/VR apps or content are transported to a learning record store and
moved to an analytics repository system for interpretation of the meanings of the captured data. There
are, however, remaining challenges to the interpretation of verbs regarding activities which came from
different AR/VR apps or content, and there is a requirement for a maintenance mechanism for the
ontology level of those captured data.
This aspect is considered as emerging learning analytics area titled multi-modal learning analytics. For
this requirement the service provider may be considered to define its own profile related to immersive
content. In 7.2 this topic is discussed, however, there are so far just a few cases in real learning
environments.
6.5 Necessary for guideline to use both specifications
A common feature among the use cases is that many institutions and platform providers have different
scales of properties for both xAPI and IMS Caliper specifications. For instance, while only a textbook
player is necessary for coverage related to a reading activity, assessment and duration of learning,
the learning platforms adopted in institutions require more wide coverage regarding the session
information, assignment, discussion forum, assessment, reading, etc. However, it does not mean that
institutions require all of the properties of IMS Caliper or the terminologies of xAPI.
This topic seems to identify issues for each of the data API specifications, not interoperability issues
between xAPI and IMS Caliper. For this reason, ADL has developed a technical document that aims
[11]
to improve practices for creating profiles as defined in the xAPI specification . Also, IMS Global
allows customized profiling as well as extension using the revised IMS Caliper Analytics specification.
Examples of this are introduced in 7.4 in detail.
7 Guideline for data interoperability
7.1 Understanding data flows for learning analytics and data collection process
This guideline addresses the data interoperability issues arising from the key data flows during typical
learning activities.
In ISO/IEC TR 20748-1, there are a total of six processes in the learning analytics workflow that are
supported by privacy and data protection requirements, as noted in Figure 1. Although learning
analytics is primarily based on data collection and analysis, learning and teaching activities within
LET contexts are fundamental to the whole process and need to be considered in order for a feedback
loop to be enabled. Learning and teaching activities provide sources for data collection and subsequent
processes of the learning analytics workflow.
10 © ISO/IEC 2020 – All rights reserved

Figure 1 — Abstract workflow of learning analytics (source ISO/IEC TR 20748-1)
In this document, the focus is on a data collection process for data interoperability. According to
ISO/IEC TR 20748-1, data collection is the process of gathering and measuring information on variables
of interest in learning and teaching activities (as shown in Figure 2). In this process some features, such
as authority and control of data source, interoperability of data, and efficiency of flow and exchange,
are required for a system to work.
Possible flows of data among data collection and data storing and processing involve the following
aspects:
— Learning and teaching activities and related data sources such as learning devices, software
applications and social networks produce various data. The sources include lectures, learning
materials, learning tools, quizzes and assessments, discussion forums, messages, social networks,
homework, prior credit, achievements, system logs, sensors, etc. These learning data need to be
collected or converted to data API specifications such as xAPI and IMS Caliper Analytics.
— The data collected from LET activities, which contains information about learners, is subject to
privacy protection requirements. Here it is important that those collecting such information about
learners do so with the informed consent of the learner (or their parent or legal guardian). Further,
the personal data shall only be used for the goal agreed to and must be protected by necessary means
such as encryption, de-identification, chrono-degradability, pseudomization and anonymization.
NOTE The privacy issues and guidelines are covered in ISO/IEC TS 20748-4.
— Data collection APIs yields data collection instances, possibly via secured data transmission.
— Data collection processes may be subject to conformance testing or conversion prior to storing data
in the temporary data store, such as the event store in IMS Caliper or learning record store in xAPI,
to be used in later processing.
© ISO/IEC 2020 – All rights reserved 11

Figure 2 — Data collection process
7.2 Preparation for data interoperability
7.2.1 General
As introduced in Clauses 5 and 6 there are two representative data APIs used in the LET domain. Prior
to deciding which data API to use for a particular purpose, understanding the structure of the data
API is more important. In addition, understanding how to organize the specification and profile is very
helpful to design customized learning data APIs profiles. This subclause describes the main structure
and profiles of xAPI and IMS Caliper respectively.
7.2.2 xAPI statements and relationship with its profiles
7.2.2.1 xAPI statements
Statements are the evidence for any sort of experience or event which is to be tracked in xAPI. While
statements follow a machine readable JSON format, they can also easily be described using natural
language. This can be extremely useful for the design process. Statements are meant to be aggregated
and analysed to provide a larger meaning for the overall experience than just the sum of its parts.
xAPI statements
...