ISO/TR 21959-1:2018

TECHNICAL ISO/TR
REPORT 21959-1
First edition
2018-11
Road vehicles — Human performance
and state in the context of automated
driving —
Part 1:
Common underlying concepts
Reference number
©
ISO 2018
© ISO 2018
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting
on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address
below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Fax: +41 22 749 09 47
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii © ISO 2018 – All rights reserved

Contents Page
Foreword .iv
Introduction .v
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 Purpose . 1
5 Human performance in the context of automated driving . 2
5.1 General . 2
5.2 Transition from manual to automated driving . 2
5.2.1 Transition process model . 2
5.2.2 Definition of related concepts . 2
5.2.3 Measures for human performance in releasing control to automation . 3
5.3 Transition from automated to manual driving . 4
5.3.1 Transition process models . 4
5.3.2 Definition of related concepts . 6
5.3.3 Measures for human performance in regaining control from automation . 7
6 Human states in the context of automated driving .10
6.1 General .10
6.2 General concepts for mental state related to automated driving .10
6.3 Concepts corresponding to automation related driver states .11
6.4 Concepts corresponding to non-driving related driver states .12
6.5 Driving position and posture .14
7 Driver readiness/availability .15
8 Driver’s experiences and attitudes regarding driving automation system .16
8.1 Prior system image .17
8.2 Education and training .17
8.3 User’s understanding of driving automation system .18
8.3.1 User’s thought about how driving automation system works .18
8.3.2 User’s mental attitude to driving automation system .18
8.4 User’s use of driving automation system .19
8.4.1 User’s behavioural attitude while using driving automation system .19
8.4.2 User’s interaction with driving automation system .19
8.4.3 User’s behaviour/manner at driving automation system .19
Bibliography .20
Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards
bodies (ISO member bodies). The work of preparing International Standards is normally carried out
through ISO technical committees. Each member body interested in a subject for which a technical
committee has been established has the right to be represented on that committee. International
organizations, governmental and non-governmental, in liaison with ISO, also take part in the work.
ISO collaborates closely with the International Electrotechnical Commission (IEC) on all matters of
electrotechnical standardization.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for the
different types of ISO documents should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www .iso .org/directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject of
patent rights. ISO shall not be held responsible for identifying any or all such patent rights. Details of
any patent rights identified during the development of the document will be in the Introduction and/or
on the ISO list of patent declarations received (see www .iso .org/patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to the
World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see www .iso
.org/iso/foreword .html.
This document was prepared by Technical Committee ISO/TC 22, Road vehicles, Subcommittee SC 39,
Ergonomics.
Any feedback or questions on this document should be directed to the user’s national standards body. A
complete listing of these bodies can be found at www .iso .org/members .html.
iv © ISO 2018 – All rights reserved

Introduction
Although automation technology is advancing at a very fast pace, the majority of automated driving
levels (as defined by SAE) still require a human to fulfil specific remaining (driving related) tasks
while being in automated driving mode. The basic requirements with respect to the driver strongly
depend on the level of automation and are subject to human factors research all over the world. The
70 72
SAE documents J3016 [ ] and J3114 [ ] have already introduced working definitions of key concepts
in this field. This document puts an emphasis on common underlying concepts of driver performance
and state in the context of automated driving.
Driver performance includes driver’s activities in transitions both from manual driving to automated
driving and from automated to manual driving, as well as interaction behaviour while using the system.
Driver state here means driver’s internal conditions that may affect performance including knowledge
of and attitudes toward driving automation systems.
Concepts on driver performances in transition from manual to automated driving and from automated
to manual driving are described in Clause 5. Concepts on driver state related to the transition are
described in Clause 6 and a specific concept “readiness/availability” that refers to driver state that
predicts the intervention performance is described in Clause 7. Concepts for driver’s experiences
and attitudes that may affect driver performance and state in the context of automated driving are
described in Clause 8.
TECHNICAL REPORT ISO/TR 21959-1:2018(E)
Road vehicles — Human performance and state in the
context of automated driving —
Part 1:
Common underlying concepts
1 Scope
This document introduces basic common underlying concepts related to driver performance and
state in the context of automated driving. The concepts in this document are applicable to all levels of
automated driving functions that require a human/driver to be engaged or fallback-ready (SAE Level 1,
2 and 3). It can also be used with levels that enable a driver to resume manual control of the vehicle (a
compatible feature for SAE Levels 1 to 5).
Common underlying concepts can be applicable for human factors assessment/evaluations using driving
simulators, tests on restricted roadways (e.g., test tracks) or tests on public roads. The information
applies to all vehicle categories.
This document contains a mixture of information where technical consensus supports such guidance,
as well as discussion of those areas where further research is required to support technical consensus.
These common underlying concepts may be also useful for product descriptions and owner manuals.
The contents in this document are informative, rather than normative, in nature.
2 Normative references
There are no normative references in this document.
3 Terms and definitions
No terms and definitions are listed in this document.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— ISO Online browsing platform: available at https: //www .iso .org/obp
— IEC Electropedia: available at http: //www .electropedia .org/
4 Purpose
The purpose of this document is to provide common underlying concepts for human performance and
state for the researchers and developers of driving automation systems (more specifically SAE levels
1-5) in order to facilitate the sharing of information and knowledge as these systems are developed and
deployed.
This document does not provide design principles on how a human-machine interface (HMI) for
automated driving should be designed or developed. However, common concepts and measures could
be used during the development phase when different HMI designs are evaluated in terms of usability,
user experience and safety.
It is not intended that anything in this document restricts or provides direction regarding the
technology used to create these systems, or the underlying design of these system.
5 Human performance in the context of automated driving
5.1 General
Human performance has two aspects — behaviour being the means and its consequence being the end
[16]
. The focus on consequences, and hence on performance, is especially relevant for situations such as
the transition processes from automated to manual control (Level 0) and vice versa (see Figure 1 to 4).
The following sub-clauses give an overview of possible measures for driver and system initiated
transitions. For transitions between different automation levels (e.g. 4→2 or 3→1) within one vehicle
appropriate measures can be selected or adapted according to the specific circumstances.
5.2 Transition from manual to automated driving
5.2.1 Transition process model
Figure 1 shows a process model for a prototypical transition from manual to automated control, either
initiated by the driver or by the system.
1)
EXAMPLE After entering the highway the driver is informed about the availability of a “ highway pilot function” .
He/she decides to activate automation by a dedicated steering wheel button.
Figure 1 — Driver/system-initiated transition from manual to automated driving (concepts are
further specified in 5.2.2 and 5.2.3)
5.2.2 Definition of related concepts
Manual control: Driving phase where a human driver is performing the Dynamic Driving Task (DDT)
- all of the real-time operational and tactical functions required to operate a vehicle in on-road traffic
[70]
(see definition of Level 0 automation). In cases where lower level automation features are already
active, this phase can be regarded as including the remaining (manual) elements of the DDT required by
1) See: https: //www .daimler .com/innovation/case/autonomous/highway -pilot -2 .html, Hunger, 2017
2 © ISO 2018 – All rights reserved

the driving automation system. For example, driving with adaptive cruise control requires the driver to
perform the lateral control (sub) task as well as the Object and Event Detection and Response (OEDR)
sub-task.
Automation available: If all operational conditions for a driving automation system are fulfilled the
system is ready to be activated by either the user or the system. This system availability may be
signalled to the user via the Driver Vehicle Interface (e.g. screen, tones).
Request to engage automation: Event usually initiated by the user through the driver-vehicle interface
of the vehicle to activate the driving automation system. Apart from user-initiated transitions, system-
initiated transitions from manual to automated control may also be possible, especially after the
driver has temporarily overridden the automated mode by manual intervention. At the end of driver
intervention, the system may automatically activate/resume from suspended to active mode. For
example, in some automated steering control systems, after the driver has transitioned from automated
to manual control by manual use of the steering wheel, when the driver is no longer moving the steering
wheel, the system may automatically activate/resume from manual to automated steering control.
Activation of automation: Onset of the driving automation system activation. There may be a delay
between requesting the activation and the activation itself either due to technical reasons or by
intentionally introducing an activation process as an HMI design feature.
Driver state transition: Process where the driver is releasing control to the driving automation system.
The transition includes physical aspects (releasing hands and feet from primary vehicle controls) as well
as cognitive aspects (ensuring that automation has taken over successfully). The physical transition
phase ends when the driver fully releases manual vehicle control (hands and feet do not have any action
on longitudinal or lateral vehicle control). Behavioural markers for the end of the cognitive transition
are less obvious.
Automated driving: Driving phase where a Level 1 - Level 5 (L1 to L5) system is performing specific
aspects of the DDT.
Acceptable driver state by automation level: Driver state that is required or activity that is allowed by the
driving automation system. The driver state may or may not be monitored by the driving automation
system. Requirements on acceptable driver states are strongly dependent on the automation level.
Sleep is commonly seen as not acceptable by L2/L3 features or physically leaving the driver’s seat is not
acceptable for L2/L3 features.
Non-Driving Related Activity (NDRA): Any activity not related to the monitoring of the driving
automation system and/or the current driving situation is called Non-Driving Related Activity. This
can include activities that take up any of visual, auditory, visual-manual, auditory-manual, manual, or
cognitive capabilities.
Non-Driving Related Task (NDRT): Any activity related to a dedicated task that is different from the
monitoring of the driving automation system and/or the current driving situation is called Non-Driving
Related Task. An activity becomes a task when it has a specific goal, and the task can consist of a series
of activities leading up to this goal. A NDRT can also be called Secondary Task, but only as long as there
is a Primary Task, in this case operating the vehicle. When driving is no longer the driver’s Primary Task
– such as during automated driving at SAE Levels 3 and higher of automated driving – the NDRT stops
being a Secondary Task. Under such circumstances the NDRT itself can be regarded as the primary task.
5.2.3 Measures for human performance in releasing control to automation
Time to activate system: Time interval between events “Automation available” and “Activation of
Automation”
Time to release controls: Time interval between events “Activation of Automation” and “Full release of
vehicle control”
Time to start/resume NDRA: Time interval between events “Activation of Automation” and “Start of NDRA”
Method used to engage driving automation system: Specification of required driver action to fully release
control to driving automation system (e.g. double-pull of stalk at steering column or simultaneous
activation of dedicated steering wheel controls).
5.3 Transition from automated to manual driving
Transitions from automated to manual driving, may have two different “sources”. They may be system
initiated or they may be driver initiated as is presented in the sub-clauses below.
5.3.1 Transition process models
Figure 2 shows the process model for a system-initiated transition from automated to manual vehicle
control with definitions of relevant time periods. This transition model assumes the result of a fully
stabilized vehicle.
2)
EXAMPLE While using a highway pilot system , the function issues a Request to Intervene (RtI) due to an
internal system error. After preparing for taking over manual vehicle control the driver deactivates the Highway Pilot
function and switches to manual driving mode.
Figure 2 — System-initiated transition from automated to manual driving (concepts are further
specified in 5.3.2 and 5.3.3)
In addition to system-initiated transitions, user-initiated transitions without a RtI should be covered,
as Level 1 to Level 3, and some Level 4 or Level 5 systems may be designed to be deactivated by the user
at any point in time during full operation. There are two types of reasons for a user to deactivate the
automation feature which are described below.
2) See: https: //www .daimler .com/innovation/case/autonomous/highway -pilot -2 .html, Hunger, 2017
4 © ISO 2018 – All rights reserved

Figure 3 describes the process of regaining manual vehicle control due to the detection of system
performance limitations. In this case the L1/L2 driving automation system does not issue a RtI to
the driver.
EXAMPLE While using a L2 automation system in a construction zone the driver observes that the system is
following invalid lane markings. He/she decides to immediately take-over control by manually overriding the lateral
steering control (leading to manual driving mode).
Figure 3 — Human-initiated transition of automated to manual driving due to detection of
system performance limits (concepts are further specified in 5.3.2 and 5.3.3)
On the other hand, the driver may want to deactivate the driving automation system without detecting
system performance limitations. For this case the transition process described above can be slightly
adapted (see Figure 4).
EXAMPLE While using a traffic jam pilot feature in heavy traffic on a city freeway, the driver deactivates all
driving automation features using a designated control for that purpose and switches to manual driving in order to
exit the freeway and find a faster route.
Figure 4 — Human-initiated transition of automated to manual driving (without system
performance limit; concepts are further specified in 5.3.2 and 5.3.3)
5.3.2 Definition of related concepts
Request to Intervene (RtI): Notification by a driving automation system to a driver/fallback-ready user
indicating that he/she should take over vehicle control (perform all or parts of the DDT). According
to the requirements of SAE Level 3+ systems, all system related limits should be recognized and
appropriate driver requests to take-over control should be triggered. The RtI time stamp is essential
to analyse the subsequent transition process as it is regarded as the beginning of the driver state
transition process. If a request to intervene is not issued by the driving automation system because
of an automation-external vehicle failure (examples: flat tyre, broken steering system so on), the
(perceivable) failure signal can alternatively be used as a corresponding event. Drivers of Level 2
automation may encounter (silent) system failures or performance limits which are not recognized and
communicated by the driving automation system. In this case the driver may notice suspicious vehicle
control due to a potential system limit which triggers the decision to regain manual control.
Silent system failure and/or silent system limit: System performance limitation of a driving automation
system that is not recognized and communicated to the user. The performance limitation may be due to
internal failure states or incorrect interpretation of the driving environment.
Critical event due to system limit: Situation that can be specified in time and space that the driving
automation system cannot handle safely and that will occur in case the driver does not intervene.
System limits are associated with different time budgets:
— Long-term system limits (e.g. the planned transition of a Highway Pilot when reaching the exit) can
be communicated well in advance to allow for a sufficient preparation. Driver performances with
respect to early communication of long-term system limits are not within primary interest of this
document.
— Mid-term system limits (e.g. approaching a construction site that cannot be handled by the system)
require the driver/fallback-ready user to take-over within a certain time budget. This type of system
limit is central to the definition of SAE Level 3 systems.
— Short-term limits (e.g. due to sensor range) ask the driver for immediate take-over of the DDT. This
type of system limit is central to the definition of SAE Level 2 systems.
6 © ISO 2018 – All rights reserved

Although the majority of system limits will be detected and communicated via a RtI for higher
automation levels, there may be system limits for Level 2 systems without a RtI (see Figure 4).
Take-over mode: The concept refers to the system behaviour after a RtI has been issued. The remaining
performance depends on the automation level and on the design of a particular system. Level 3-5
systems should be able to continue operation after issuing a RtI for a specified period of time.
Depending on the type of system limit, system operation might be of degraded nature. Level 2 systems
may be deactivated immediately after a RtI has been issued. The take-over mode may already contain
measures to reach a minimal risk condition (such as a stopping manoeuvre). However, details on human
performance aspects with respect to a minimal risk manoeuver will be covered in a future, planned
3)
document (ISO/TR 21959-2 ).
Driver state transition: Process of transforming the actual driver state (possibly determined by NDRA)
to a target driver state suitable to effectively take-over manual control. This process can be analysed
on a sensory, motoric and cognitive level. Relevant time markers within this phase are: interrupting/
finishing a non-driving related task, start of visual reorientation, gaze on road centre, hands on wheel/
feet on pedals, etc.
Significant driver intervention: Action initiated by the user of a driving automation system to request
manual control over all or some parts of the DDT. The way manual control can be resumed by the user
depends on the driving automation system design, but usually comprises significant driver intervention
on primary vehicle controls and regular deactivation mechanisms (e.g. based on buttons or switches).
The system reaction towards the driver intervention also depends on the particular design of the
automation function. For higher-level automated driving systems (ADS) which remain active after a
RtI, “significant driver intervention” corresponds to requesting system deactivation. The subsequent
change of the system status may be immediate or delayed (e.g. in order to establish safe conditions for
manual driving). SAE Level 2 functions may deactivate vehicle control of some or all aspects of the DDT
at the time of issuing the RtI. In this case driver intervention takes place in manual mode and typically
corresponds to significant and relevant actions on primary vehicle controls.
Post transition (manual) control: A defined, extended time window to analyse the quality of manual
control driving after a RtI has been issued. The post transition driving phase can be decomposed in the
driver intervention itself and a control stabilization phase.
Completion of driving manoeuver: The kind of take-over action that is expected by the driver to
successfully handle the system limit. The intervention depends on the demands of the take-over
situation. Examples range from easy tasks such as bringing hands back on the steering wheel to follow
the lane to more complex manoeuvers such as manual lane changes or brake manoeuvers etc. Depending
on the take-over situation it can be difficult to determine the exact end of the driver intervention as
there is a gradual transition to the control stabilization phase.
Vehicle control fully stabilized: After the immediate driver intervention phase, manual vehicle control
performance can be below the average performance of an individual driver. The time period until
vehicle control performance is fully re-established is referred to as “control stabilization phase”.
5.3.3 Measures for human performance in regaining control from automation
5.3.3.1 Type of driver intervention
The method for how a particular driving automation system can be overridden or deactivated should
be fully specified. This includes describing thresholds for significant driver intervention on primary
vehicle controls and/or how the system can be deactivated by dedicated controls.
Overview of methods used to resume manual vehicle control (perform part or all of the DDT). Typical
categories are:
— automation feature override or deactivation by brake intervention;
3) Under preparation.
— automation feature override or deactivation by accelerating;
— automation feature override or deactivation by steering intervention;
— automation feature override or deactivation by dedicated controls (e.g. on/off switch); and
— other automation feature override or deactivation.
Overview of different sequences of driver intervention, e.g. 1st action (leading to manual mode), 2nd
action, 3rd action (focus on start of action since actions may also be performed in parallel).
5.3.3.2 Time-related performance measures
The time-based performance measures apply to the transition process depicted in Figure 2, 3 and 4.
The described time periods can be further split up into sub-processes (similarly as is shown for driver
take-over time) but also combined to define new time periods (e.g. driver take-over time + driver
intervention time). For the sake of simplicity, no dedicated concepts are introduced for these derived
measures. Intervention time combined with the previously mentioned term take-over time is also a
relevant time-based metric.
Take-over time: Time interval between onset of RtI and user-initiated intervention or deactivation of an
engaged automation function. This measure is central to human factors research on automated driving
[17] [6] [66]
and has been used by many authors (e.g ; ; ). The transition process can be decomposed into
further sub-processes (all starting with the RtI):
— time to first driver reaction (e.g. interruption of non-driving related task);
— time to start of visual re-orientation;
— time to visually fixate RtI message (if visual HMI is involved);
— time to visually fixate road centre (or other relevant aspects of the scenery);
— time to start to move (at least one) hand to wheel / feet to pedals;
— time to grasp wheel/touch pedals;
— time to start to operate relevant vehicle controls (e.g. blinker) or steering / pedal operation; and
— time to onset to override or deactivate an engaged automation feature by specified methods.
System deactivation time: Higher level automation functions may be deliberately designed to delay the
transition to manual control under certain circumstances (e.g. in order to establish safe conditions for
manual driving). This period of time (not mentioned in Figures 2 to 4) should be considered separately
and is not part of the human-focused “take-over time”.
Decision time: Time interval between detection of a silent system failure and the decision to disengage
the automation feature (see Figure 3).
Intervention time: Time interval required by the driver to handle the imminent take-over situation
by performing an appropriate driving manoeuver. The requirements of the driver intervention vary
from performing complex driving manoeuvers to simple tasks such as maintaining the current vehicle
dynamics. In the latter case the driver intervention time could be very short or even be neglected. In any
case clear behavioural or environmental markers should be defined for the end of the required driver
[61]
intervention. An example is shown in Reference . Intervention time combined with the previously
mentioned term take-over time is also a relevant time-based metric.
Total time budget: The time interval between the RtI and the system limit is defined as “total time
budget”. It represents the maximum time window after a RtI for a successful resumption of manual
control by the driver.
8 © ISO 2018 – All rights reserved

Driving recovery time: Time required by the driver to recover driving with the appropriate reaction
(according to the driving context) from the RtI warning. It is defined as take-over time + intervention
time. Comparison between total time budget and driving recovery time should be used to provide
guidelines that take account of all the driver’s reactions to ensure a successful manoeuvre.
Remaining action time: Time interval between a successfully finished driver intervention and the
system limit. Computationally, it can be calculated as the difference between the total time budget
[17]
and the sum of driver take-over time and driver intervention time. Gold et al. (2013) used a slightly
modified variant of this measure, namely the remaining time to last intervention possibility at the time
of intervention.
Control stabilization time: Time duration it takes for an individual user to reach a similar or comparable
quality level of manual driving performance as in ordinary Level 0 driving by an average driver
(reference condition). For downward transitions to Levels >0, corresponding performance for this
particular level can be used as reference behaviour. Longitudinal and lateral dimensions of vehicle
[30]
control can be considered. An example is shown in Reference .
5.3.3.3 Quality-related measures
Apart from describing the type and timeliness of user response it is essential to assess the quality of
the driver intervention after the RtI. The interpretation of take-over quality measures will always
depend on the (design of) take-over situation under investigation. For example, a driver reaction
that is beneficial in one situation (e.g. swerving to avoid an accident) may be detrimental in another
(e.g. swerving when there is neighbouring traffic). Accordingly, different investigations of take-over
quality should always be compared carefully and with extensive attention to detail. Furthermore, the
described quality measures vary in external validity and sensitivity (usually following the relationship
of increased validity – decreased sensitivity and vice versa). They can further be classified in terms
of objective (e.g. behavioural and electrophysiological measures that do not depend on a person who
measure) and subjective (e.g. self-reported base on the person’s experience) measures.
Safety-oriented, objective take-over quality measures: A variety of measures are available and have
been used to assess driving behaviour in take-over situations with respect to its safety effects on the
individual itself and on other traffic participants. Examples of performance measures for assessing the
[17] [38] [6] [66] [45 [58]
safety-effects of a driver response to a RtI (e.g. used by References ; ; ; ; ]; ) are:
— share of test subjects being able to avoid collision with other traffic participants / prevent run off-
road events;
— description of collision severity;
— omission of visual checks/mirror use;
— operating errors (especially related to system deactivation);
— maximum longitudinal/lateral acceleration values / frequency of strong or emergency braking (e.g.,
2 2
−4 m/s /−6 m/s );
— minimum time to collision / minimum time on / to headway / frequency of “near misses” (TTC
min
<1 s/1,5 s); and
— minimum time to lane crossing (TLC ).
min
Sensitivity-oriented, objective take-over quality measures: As opposed to the safety-oriented measures
that are relevant to the critical situation, other measures can be used to show potential mid- and long-
term detrimental effects of having to regain manual control after an extended period of automated
driving. The potential measures related to lateral and longitudinal control are:
[45] [42]
— Standard deviation (SD) of lateral position (e.g ; );
[8]
— SD of steering wheel angle (e.g .);
[10]
— yaw rate error and SD of yaw rate error (e.g .);
— metrics of distance to other vehicles or objects; and
— metrics for longitudinal control quality, e.g. time headways, speed behaviour (e.g. considering speed
limits).
The measures are recommended to be used on an individual level to compare baseline manual driving
performance with post-transition performance.
Expert-based assessment of take-over quality: In order to assess the overall safety effects of a transition
in a particular traffic situation, a combination of appropriate measures should be taken into account as
single measures may not be sufficient to differentiate safe and unsafe events (for a definition of safety-
[33]
critical events see Reference ). Expert-based assessments of traffic situations have the potential
to weigh and integrate different timing and quality parameters into a global controllability measure.
[54]
This approach has also been recommended in the Response Code of Practice in order to assess the
controllability of driver assistance systems. In the context of automated driving, controllability can be
defined as the driver’s likelihood to safely cope with all possible driving situations occurring during
normal use, at system limits or after system failures. Naujoks et al. (2017) proposed a standardized
rating scheme for controllability of transitions by trained raters that aggregates different aspects of
the take-over situation to one global measure of driving performance based on video footage (see also
[27] [26]
References ; for expert-based assessments of manual driving performance) which differentiates
the following performance levels:
— Rating “1”: perfect performance (i.e, absence of imprecisions and errors);
— Rating “2-3”: imprecisions of vehicle control (e.g., imprecise lane keeping);
— Rating “4-6”: occurrence of driving errors (e.g., late or insufficient braking);
— Rating “7-9”: endangerments of oneself or other road users (e.g., near misses); and
— Rating “10”: collisions or loss of control.
Subjective take-over quality measures: Objective measures are ideally combined with subjective
judgements of the intervention quality, either by the driver him/herself or by an external observer.
Examples of judged aspects can be trust in coping with the situation, experience of being precise, and to
what level the driver feels in control. A potential scale that can easily be adapted for this purpose is the
[48]
“Scale for criticality assessment of driving and traffic scenarios” .
6 Human states in the context of automated driving
6.1 General
The scope of this clause is to introduce a basic structure for how to think about driver states in the
context of automated driving. Rather than covering measures for all possible user states, the overview
is intentionally limited to human states that are related to human performance in transition situations.
6.2 General concepts for mental state related to automated driving
The cognitive aspect of a human state in the context of automated driving primarily refers to the
interpretation of the current driving situation with respect to perception, decision and response
selection. It includes higher-level states such as situation and automation mode awareness (see
Clause 7) during automated driving as well as in transition situations.
The process of perception, decision and response selection is influenced by the way in which attentional
[64]
resources are allocated to the current tasks (e. g. .) as well as by long-term, memory-based factors
such as the mental model a human driver has developed for the system.
10 © ISO 2018 – All rights reserved

Knowledge (or assumptions) about the likelihood of system limits and the kinds of possible transitions
influence the speed with which a transition situation can be correctly interpreted. Similarly, system
trust is considered as an influencing element on the cognitive driver state as it influences the way we
think about and use automation (see Clause 7).
The cognitive state is also influenced by arousal level because total amount of attentional resource is
considered to be influenced by arousal level. The following list describes exemplary impacts on the
human’s cognitive state.
Attention: The element of cognitive functioning in which the mental resources are focused on a specific
[65]
issue, object, or activity. Attention has been described as a function that accomplishes selection and
[62] [4] [63]
its main purpose is to facilitate perceptual processes ( ; ; ). Attention is also said to be a function
[5] [1] [7] [4]
that selects stimuli for further higher-level cognitive processing ( ; ; ; ).
Attentional resource: Amount of attention applied to the target of attention. Physiological measures
such as EFRP (Eye Fixation Related Potential) can be used to assess the amount of attentional resources
[60]
applied to the road environment or to the non-driving related visual (also visual-manual) task .
Attentional resource is also called mental resource. Subsidiary task methods, such as the DTR (Detection
[69]
Response Task ), are used to obtain behavioural measures to assess spare attentional resource. The
[59]
Operation Span Test can also be used to assess spare attentional resource .
(Task) demand: Level of mental activities necessary to achieve the goal of a task. The human’s cognitive
state during automated driving may be affected by the level of task demand requested by the current
(potentially non-driving related) activity and the effort invested by the human. A wide range of
measures have been proposed. Examples of subjective measures are the Rating Scale of Mental Effort
[67] [22] [28] [50]
, the NASA-TLX scale , and the Driving Activity Load Index (DALI) scale ( ; ). In automobile
driving context, road structure (e.g. curve radius) and surrounding traffic (e.g. distance to an adjacent
vehicle) are factors of the demand. It also depends on the driver behaviour task that a driver can select
(e.g. target speed and target time to arrive the destination). The amount of mental resources of a person
to perform the task depends on the task demand. Therefore, the demand can be estimated by workload
measures. An example of a behavioural measure to assess the amount of demand level on the human is
[69]
the standardized DRT to quantify cognitive workload. Another example is the recent use of AttenD
[44] [29]
(Driver Attention System ; ) to measure the effect of multi-modal demand on driver attention
[56] [55]
( ; ).
6.3 Concepts corresponding to automation related driver states
The following concepts relate to driver states associated with monitoring the driving automation
system or the driving environment (see acceptable driver states in 5.2.1 and 5.2.3).
Monitoring the driving environment: The activities and/or automated routines that accomplish real-
time roadway environmental object and event detection, recognition, classification, and response
preparation (excluding actual response), as needed to operate a vehicle (SAE J3016: 3.14.2).
Monitoring the driving automation system performance: The activities and/or automated routines for
evaluating whether the driving automation system is performing part or all of the DDT appropriately.
(SAE J3016: 3.14.2) This is not limited to monitoring the HMI but also includes monitoring the
environment to see if the system performs adequately for the circumstances.
Object and Event Detection and Response (OEDR): The subtasks of the DDT that include monitoring
the driving environment (detecting, recognizing, and classifying objects and events and preparing to
respond as needed) and executing an appropriate response to such objects and events (i.e., as needed to
complete the DDT and/or DDT fa
...