ISO/TS 4654:2025

Technical
Specification
ISO/TS 4654
First edition
Road vehicles — Advanced
2025-07
automatic collision notification
(AACN) systems — Methodology for
creating and validating algorithms
for injury level prediction
Véhicules routiers — Systèmes intelligents de notification
automatique de collision — Méthodologie pour créer et valider
les algorithmes de prédiction du niveau de blessure
Reference number
© ISO 2025
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting on
the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address below
or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii
Contents Page
Foreword .iv
Introduction .v
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 AACN injury severity prediction overview . 3
5 Vehicle and road user data for injury prediction application . 5
5.1 Introduction .5
5.2 Vehicle-related input parameters .6
5.3 Vehicle occupant-related input parameters .6
5.4 Vulnerable road user data .7
6 Injury prediction algorithm development . 8
6.1 General .8
6.2 Applicability of retrospective data for AACN use .8
6.3 Modelling methods .8
6.3.1 Introduction .8
6.3.2 Example implementation of the logistic regression model for creating an injury
risk curve .9
6.3.3 Alternative modelling methods .9
6.3.4 Important considerations for selecting model parameters .10
7 Recommendations on the metrics to validate injury prediction algorithms .10
7.1 Common performance metrics used to assess classification algorithms .10
7.1.1 General .10
7.1.2 Categorising outcomes from model prediction to enable performance assessment .10
7.1.3 Confusion matrix.11
7.1.4 Derived quantities used in assessing performance .11
7.2 Interpreting performance metrics using diagnostic curves in the context of AACN
implementation . 12
7.2.1 Introduction . 12
7.2.2 Receiver Operator Curve . 12
7.2.3 Precision recall curve .14
7.2.4 Effect of class imbalance on ROC and precision recall curves . .14
7.2.5 Connection between ROC, PR curves and triage related terms . 15
7.3 Summary of recommendations on performance metrics .17
8 Recommendation for monitoring the AACN algorithm .18
8.1 Overfitting .18
8.1.1 Overfitting risk .18
8.1.2 Approach for overfitting mitigation. 20
8.2 Ensure the response of the algorithm to obvious physical variables . . 20
8.2.1 Analysis of relationships between variables and model response . 20
8.2.2 Evidence of the algorithm response to obvious physical variables . 23
9 Spatial and temporal validity .23
Annex A (informative) AACN algorithm research publications .24
Annex B (informative) Example implementation of the logistic regression model for creating
an injury risk curve .34
Annex C (informative) Detailed processes involved in assessing the performance of AACN
algorithms prior to implementation .36
Bibliography .40

iii
Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards
bodies (ISO member bodies). The work of preparing International Standards is normally carried out through
ISO technical committees. Each member body interested in a subject for which a technical committee
has been established has the right to be represented on that committee. International organizations,
governmental and non-governmental, in liaison with ISO, also take part in the work. ISO collaborates closely
with the International Electrotechnical Commission (IEC) on all matters of electrotechnical standardization.
The procedures used to develop this document and those intended for its further maintenance are described
in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for the different types
of ISO documents should be noted. This document was drafted in accordance with the editorial rules of the
ISO/IEC Directives, Part 2 (see www.iso.org/directives).
ISO draws attention to the possibility that the implementation of this document may involve the use of (a)
patent(s). ISO takes no position concerning the evidence, validity or applicability of any claimed patent
rights in respect thereof. As of the date of publication of this document, ISO had not received notice of (a)
patent(s) which may be required to implement this document. However, implementers are cautioned that
this may not represent the latest information, which may be obtained from the patent database available at
www.iso.org/patents. ISO shall not be held responsible for identifying any or all such patent rights.
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and expressions
related to conformity assessment, as well as information about ISO's adherence to the World Trade
Organization (WTO) principles in the Technical Barriers to Trade (TBT), see www.iso.org/iso/foreword.html.
This document was prepared by Technical Committee ISO/TC 22, Road vehicles, Subcommittee SC 36 Safety
and impact testing.
Any feedback or questions on this document should be directed to the user’s national standards body. A
complete listing of these bodies can be found at www.iso.org/members.html.

iv
Introduction
This document provides guidance on advanced automatic collision notification (AACN) algorithms for
injury level prediction and related parameters. Guidance on the evaluation of such AACN algorithms is also
presented. This document does not establish a particular AACN injury level prediction algorithm or impose
a specific input data set.
This document contributes to an appropriate implementation, overall, saving lives. Different parties (as
listed below) will benefit from applying this document.
Benefits for implementors (e.g. OEMs, countries) listed below for implementor groups respectively:
— implementors currently not having an AACN algorithm: this document helps to efficiently develop and
evaluate one, facilitating more rapid introduction;
— implementors having AACN algorithm already in a region: implementors can use this document to
demonstrate appropriateness;
— implementors having an AACN algorithm and wanting to enter new market: this document helps to
ensure and demonstrate appropriateness for new market.
Benefits for first respondents, doctors and paramedics:
— advance estimation of expected injury severities in the crash scene;
— unifying advance estimation increases the possibility of using algorithms providing similar estimations
of injury severity;
— reduced time to start medical treatment and improved triage for injured road users involved in a crash.
Benefits for society:
— end users are all road traffic participants involved in a traffic accident. In a collision, car occupants and/
or vulnerable road users can have a better chance to mitigate or survive injuries when there is an AACN
injury level prediction algorithm to facilitate rapid response by dispatching appropriate emergency
services.
v
Technical Specification ISO/TS 4654:2025(en)
Road vehicles — Advanced automatic collision notification
(AACN) systems — Methodology for creating and validating
algorithms for injury level prediction
1 Scope
This document outlines methodologies for creating and evaluating AACN algorithms, using suitable
parameters, to predict the level of injury sustained by road users in a collision.
The injury prediction is used to facilitate emergency response after a collision occurs.
The methodology is based on onboard vehicle data and occupant-related information and applies to vehicle
occupants and vulnerable road users.
This document is applicable to road vehicles having provisions for measuring and communicating crash
related data.
This document provides neither a particular AACN injury level prediction algorithm, nor information on
how to use the estimated probability of injury to decide on further suitable actions (rescue, medical, etc.).
Data format for sending vehicle information and communication protocol between the vehicle and the public
service answering point (PSAP) is outside the scope of this document.
2 Normative references
There are no normative references in this document.
3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
ISO and IEC maintain terminology databases for use in standardization at the following addresses:
— ISO Online browsing platform: available at https:// www .iso .org/ obp
— IEC Electropedia: available at https:// www .electropedia .org/
3.1
advanced automatic collision notification system
AACN system
system that carries out automatic notification of traffic accidents, providing information measured by the
vehicle aiming to predict the level of injury sustained by road users
Note 1 to entry: Additional information (not measured by the vehicle) available just after the crash (3.12) can be used
for the prediction.
3.2
event data recorder
EDR
device or function in a vehicle that records the vehicle’s dynamic, time-series data during the time period just
prior to a crash event (e.g. vehicle speed versus time) or during a crash event (e.g. Δv versus time), intended
for retrieval after the crash event
Note 1 to entry: For the purposes of this definition, the event data do not include audio and video data.

Note 2 to entry: At the time of developing this document, EDR data do not include audio or video information.
[SOURCE: Reference [5], modified — Note 1 to entry was originally part of the definition, Note 2 to entry
was added.]
3.3
injury risk curve
curve giving the probability, for a defined population and for a given input, to sustain a specified severity
of injury
[SOURCE: ISO/TS 18506:2014, 2.1]
3.4
retrospective traffic accident data
sets of historical traffic accident data grouped by analysis category
3.5
triage
rapid process of sorting people depending on their need for immediate medical treatment (as is usually done
in emergencies)
Note 1 to entry: See also 7.2 for use of triage and related definitions in the context of AACN according to this document.
3.6
under-triage
UT
state in which a system has assessed a person as suffering from a minor injury or no injury, when the person
has suffered a severe or fatal injury
Note 1 to entry: See also 7.2 for use of triage (3.5) and related definitions in the context of AACN according to this
document.
3.7
under-triage rate
UTR
value obtained by dividing the number of severe or fatal injuries assessed as minor or no injuries by the
number of cases that actually suffered a severe or fatal injury
Note 1 to entry: See also 7.2 for use of triage (3.5) and related definitions in the context of AACN according to this
document.
3.8
over-triage
OT
state in which a system has assessed a person as suffering from a severe or fatal injury, when the person has
actually suffered a minor injury or no injury
Note 1 to entry: See also 7.2 for use of triage (3.5) and related definitions in the context of AACN according to this
document.
3.9
over-triage rate
OTR
value obtained by dividing the number of minor or no injuries assessed as severe or fatal injuries by the
number of cases that actually suffered a minor or no injury
Note 1 to entry: See also 7.2 for use of triage (3.5) and related definitions in the context of AACN according to this
document.
3.10
algorithm
set of rules or calculations applied to test data that generate an interpretable or reportable result
[SOURCE: ISO 21474-1:2020, 3.2]
3.11
injury level prediction
injury severity prediction
prediction of the level of injuries from a given set of input parameters with certain values of the probability
as threshold
3.12
crash
situation in which the subject vehicle has any contact with at least one other conflict partner either on or off
the trafficway, either moving or stationary (fixed or non-fixed), that is observable or in which kinetic energy
is measurably transferred or dissipated
Note 1 to entry: This excludes roadway features meant to be driven over such as speed bumps and low roadside
barriers (curbs, medians, etc.) within the ground clearance limitations of the vehicle.
Note 2 to entry: A crash can also be a single-vehicle conflict that includes at least one of the following conditions:
vehicle rollover, airbag deployment, injury, more than 90° degrees of horizontal vehicle rotation, or all four tires
leaving the trafficway.
Note 3 to entry: In this document, the terms crash and collision are used interchangeably.
[SOURCE: ISO/TR 21974-1:2018, 3.4, modified — Note 3 to entry has been added.]
3.13
eCall
system to provide notification and relevant coordinate information to public-safety answering points,
by means of wireless communications, that there has been an incident that requires a response from the
emergency services
[SOURCE: ISO 24978:2009, 4.2]
3.14
vulnerable road user
VRU
non-protected road user such as motorcyclists, cyclists, pedestrians and persons with disabilities or reduced
mobility and orientation
[SOURCE: ISO/TR 4804:2020, 3.68]
4 AACN injury severity prediction overview
This clause provides an overview of injury severity prediction in AACN systems for motor vehicle collisions.
Figure 1 shows general factors related to impact severity and injury outcome in the pre-crash, crash and
post-crash phases. The factors are divided into those related to impact severity and those related to injury
mechanisms and outcome. Several factors have related input parameters that can be used for an AACN
system and are also referenced in this document, see for example Table 1 and Table 2.

Figure 1 — Factors related to impact severity and injury outcome
Previous publications, as described in Annex A, document a variety of injury severity prediction models
currently in use around the globe. These models all seek to improve outcomes for people involved in motor
vehicle collisions by supporting the rapid deployment of appropriate emergency response. Generally, models
are based on retrospective traffic accident data, including characteristics of the crash, involved vehicles, and
occupant outcomes. Statistical analysis methods are used to derive a relationship between accident factors
and the degree of injury sustained. This relationship is then used to predict the degree of injury sustained
by a person in a specific accident near real time, to facilitate proper triage.
While the objective and general development approaches of injury severity prediction models are similar,
the literature reveals differences in specific model formulations. Many of these differences are traceable
to the underlying retrospective datasets used in model fitting. For example, one key variable in predicting
occupant injury outcomes is the velocity of involved vehicles before and during a collision. Retrospective
crash data sets vary in how velocities are captured, some leveraging vehicle-based measures of pre-crash
speed or crash change in velocity, Δv, others using police-reported speed estimates, and still others relying
on posted speed limits.
Differences in input accuracy and granularity can affect the significance of factors in predicting injury
outcomes.
Additional differences in models are due to contextual differences where models are implemented. In some
cases, emergency resources are centrally dispatched by a small number of specialized resources. Training
users to interpret and act on model outputs in such environments is easier than in distributed contexts with
many dispatchers managing different types of constrained emergency response resources.

Based on current knowledge, this document describes input data and modelling approaches used in injury
severity prediction. It also describes methods for validating model performance and highlights potential
pitfalls.
5 Vehicle and road user data for injury prediction application
5.1 Introduction
To predict injury risk, algorithms are generally developed using historic collision data. In order to predict
injury at the time of collision, there is a primary requirement for information automatically collected by the
vehicle to be transmitted in near-real time at the time of collision. This information can be supported by
parameters collected shortly after the collision, for example via telecommunications link to the vehicle and
its occupants, or other sources.
This clause explores current and possible future parameters available from historic collision data, near-
real time vehicle sensor data and other data sources. Algorithms can be created from historic collision data
using vehicle and occupant related parameters, and therefore 5.2 and 5.3 are separated by vehicle-related
and occupant-related data parameters.
Based on a review (see NOTE 1 in this subclause) of current EDR regulations, AACN literature, eCall
transmission possibilities and vehicle parameters available in accident databases (national level and in-
depth level) three categories of relevant vehicle-related input parameters can be defined:
— A: considered available and widely used in AACN applications at the time of publication of this document;
— B: considered potentially available and useful at the time of publication of this document;
— C: considered potentially available and useful in the future at the time of publication of this document.
NOTE 1 See Annex A, Table A.2: Studies of existing AACN algorithms which use some parameters; overview of EDR
regulation (US, China, UN ECE & EU); eCall possibilities; status of accidents databases content.
NOTE 2 In this clause, only the vehicle parameters that can be automatically transmitted by the vehicle itself after
the crash are included. Information that can be obtained by a discussion between PSAP member and vehicle occupant
is not included.
NOTE 3 “Available” means that the information is available in the vehicle (EDR or other data recording and storage
systems) at the time of the accident, and that the over the air transmission is feasible. In order to construct an
algorithm, historic collision data is most commonly used. Therefore, available parameters must also be available in
the historic collision data used to create the algorithms.
NOTE 4 "Useful” means that those parameters are relevant to estimate occupant injury severity based on previous
work conducted using accident databases.
For any collision, the type of impact sustained by the vehicle is relevant for injury severity prediction. A list of
relevant inputs for AACN injury level prediction algorithms can be split by the following main impact types:
— frontal impact;
— side impact;
— rear impact;
— roll over.
NOTE 5 "Impact" is used according to the definition in ISO 6813:1998, 3.4.
NOTE 6 An alternative categorization can be done according to the direction of principal force (PDOF), according to
the definition in ISO 6813:1998, 3.4.2.2.

In case of multiple impact detection, parameters corresponding to each impact should be considered. The
implication of multiple impacts occurring even without further impact severity information should be
considered for the injury severity prediction.
5.2 Vehicle-related input parameters
Table 1 shows the key vehicle-related input parameters for each type of detected impact, separated into the
three categories, for usage or future usage in AACN algorithms.
Table 1 — Example vehicle-related input parameters relevant to AACN
Frontal impact detection Rear impact detection Side impact detection Roll over detection
Maximum Δv –
Maximum Δv – longitudinal (+/-) Maximum roll rate
lateral
Maximum Δv – resultant (total) -
Category A
Collision direction
Collision direction Collision direction Collision direction
– driver side
– front – rear – roll over
– non-driver side
Restraint system (e.g. seat belt) status
Time to maximum Time to maximum Time to maximum
Δv – longitudinal Δv - lateral roll rate
Maximum Δv Maximum Δv Maximum Δv
– lateral – longitudinal – lateral
Vehicle speed
Relative velocity
Category B
Airbag or seatbelt pretensioner deployment
Airbag and seatbelt pretensioner deployment times
maximum roll angle
–
(number of rolls)
Multiple events collision
Intrusion magnitude
Category C
Maximum deformation location
5.3 Vehicle occupant-related input parameters
Table 2 shows a range of vehicle occupant-related input parameters, for usage or future usage in AACN
algorithms. Some parameters, such as occupant age, are already widely used in existing AACN algorithms
even though the age information may not be available in real-time. Methods such as telecommunications link
with the vehicle are currently used to assist the emergency services in obtaining this information remotely.
In future it is expected that there will be a greater ability for vehicles to utilize in-vehicle sensors to obtain
occupant characteristic information, which could be leveraged by future AACN systems.
Table 2 — Example vehicle occupant-related input parameters relevant to AACN
Input parameters
Category A –
Category B Age
Sex
Seat occupancy
Restraint status of each occupant (including in child restraints and adults belted)
a
Occupant height and weight information can be available using personalised keys or by inputting the values in advance as
driver information, or by using the seat sensors to detect weight and seat slide position.

TTabablele 2 2 ((ccoonnttiinnueuedd))
Input parameters
Category C Vital signs from biometric sensors (e.g. heart rate, respiratory rate)
Pre-existing health conditions (comorbidities)
a
Occupant characterization (child/adult; size, height & weight)
Occupant seat position (track position captured by in-vehicle sensors)
Atypical occupant seating position (out-position, lying down, rear-facing, etc.), e.g. by in-vehicle
radar
Information available from occupant monitoring technologies (such as camera, radar, infrared), e.g.
occupant consciousness level or pupil monitoring.
a
Occupant height and weight information can be available using personalised keys or by inputting the values in advance as
driver information, or by using the seat sensors to detect weight and seat slide position.
5.4 Vulnerable road user data
Some input parameters considered useful for estimating injury risk for vulnerable road users (VRUs)
are listed in Table 3. At the time of publication of this document there are no known implemented AACN
algorithms for VRUs. This is due to the challenge presented in reliably detecting VRU collisions using vehicle
sensors.
Table 3 includes parameters which could feasibly give an indication of VRU injury severity and how they
may be detected at the time of a collision, now and in future. Table 3 does not contain an exhaustive list and
as instrumentation develops there is likely to be an increasing number of possibilities for data collection
during collisions involving a range of road users.
Table 3 — Example VRU-related input parameters relevant to AACN
VRU-related input parameters Potential detection mechanism
Impact velocity (v ) for striking vehicle Travel speed from event data recorder (EDR)
Collision direction for striking vehicle Systems sensitive to collisions in the direction of
striking vehicle travel
Impact location of VRU on striking vehicle
Inferred from activation of VRU protection system
(e.g. active bonnet, VRU airbags)
Image-based detection technologies
Passenger cars: parking cameras/sensors
Commercial vehicles and buses:
Cameras/sensors to monitor blind spot areas
Secondary impact location of VRU on the ground Image-based detection technologies
Passenger cars: parking cameras/sensors
Commercial vehicles and buses:
Cameras/sensors to monitor blind spot areas
Type of VRU struck (e.g. pedestrian, cyclist, eScooter, other) Classification based on currently available VRU
detection systems
Presence of VRU protection system (e.g. active bonnet, VRU Restraint control module (RCM)
airbags) on striking vehicle
VRU control module
VRU AEB
Multiple collisions System capable of detecting two or more impacted
VRUs
Sequential impact: e.g. loss of control
Detection via forward collision warning systems
(image-based or otherwise)
VRU trajectory Image-based (or other sensor-based) detection
VRU characterization (child/adult; size and weight) Image-based (or other sensor-based) detection
VRU age Image-based (or other sensor-based) detection

TTabablele 3 3 ((ccoonnttiinnueuedd))
VRU-related input parameters Potential detection mechanism
VRU sex Image-based (or other sensor-based) detection
Personal safety devices (e.g. helmet use, type, status, neck Image-based (or other sensor-based) detection
brace)
Pre-impact movement and stance (including gait if relevant) Image-based (or other sensor-based) detection
Other physical attributes (e.g. posture, personal mobility sys- Image-based (or other sensor-based) detection
tems)
Heart rate, breathing rate and other biometric characteristics Biometric sensors (e.g. via personal device)
6 Injury prediction algorithm development
6.1 General
This clause provides general information regarding the use of retrospective traffic accident data (from
accident databases) in the development of injury risk curve(s) in the AACN context. Considerations regarding
the limitations of the data, along with an example is provided.
NOTE See also ISO/TS 18506.
6.2 Applicability of retrospective data for AACN use
Retrospective traffic accident data are limited to the characteristics of the system that generated that data.
A partial list of considerations follows:
— Occupant demographics: some populations can skew older or have high BMIs, which in turn can impact
the outcomes in the available data.
— Vehicle characteristics: regions that have greater dispersion of vehicle mass can also have greater
dispersion of Δv, such as where a significant volume of larger passenger vehicles like full size trucks and
SUVs share the road with smaller cars.
— Emergency response systems: EMS response can vary between and within countries. For example, in
a lightly congested but well-resourced suburban location, EMS response and access to hospitals can
be significantly different than in a crowded urban area or sparsely populated rural area, which can
contribute to differing outcomes even if the same crash dynamics are involved.
— Road systems and quality: as with EMS response, differing road systems may impact response times and
contribute to differing outcomes even if the same crash dynamics are involved.
When using retrospective data sources for the purposes of predicting injury severity, these regional
differences that manifest in the outcome data need to be accounted for. Particular attention should be paid
to the sources of the crash data used to develop an injury prediction model. As time progresses and the
vehicle population shifts toward newer vehicles equipped with newer active and passive safety systems, it
is expected that models used to predict injury will need to change as well. This will require re-evaluating
model performance over time and may periodically require recalibrating model parameters or developing a
new model altogether.
NOTE See also Clause 9 regarding spatial and temporal validity.
6.3 Modelling methods
6.3.1 Introduction
There are many possible methods to choose from when developing an injury prediction algorithm. Since
the fundamental problem is to classify a crash outcome into various severity categories, commonly used
classification methods can be employed. These include (but are not limited to): logistic regression, k-nearest

neighbours' classification, decision trees (including random forests), and neural networks. Of these methods,
logistic regression is among the most commonly used and its output is generally easier to interpret. At the
time of publication, all known implemented algorithms use logistic regression. Predicting injury with a
logistic regression model is summarised in 6.3.2. A full list of studies is found in A.1 and modelling details
are shown in A.2.
6.3.2 Example implementation of the logistic regression model for creating an injury risk curve
Logistic regression is the most frequently adopted method for creating an injury severity prediction
algorithm. Logistic regression predicts the probability of the positive class of a binary outcome variable
occurring using a logistic function. In the context of AACN, this would be the risk of sustaining a given
injury class, for example ISS≥15 (see, for example, Reference [41]) which is often adopted as a threshold for
requiring care at a major trauma centre. The output of a logistic regression is an “S-shaped” risk function.
Logistic regression models can be created using a single input variable or multiple variables, referred to as
multivariate. The mathematical relationship is described by an intercept and coefficients associated with
each input parameter. A detailed example of logistic regression model implementation from the Japanese
standard JIS D 0889 is provided in Annex B.
6.3.3 Alternative modelling methods
At the time of publication of this document, almost all AACN injury prediction algorithms are created using
multivariate logistic regression. At the time of publication, three studies that investigated alternative
modelling approaches were found. A range of methodologies which are prevalent in other fields were
compared based on their predictive performance, demonstrating that there are viable alternative modelling
approaches even though multivariate logistic regression is the most prevalent AACN method adopted at the
time of publication.
In Reference [37] a range of alternative modelling methods are compared to the widely adopted logistic
regression method in the context of AACN algorithms. The same variables were used to predict ISS ≥ 15 in
all models trained and validated using 16 398 vehicles involved in non-rollover collisions from NASS-CDS
(2002-2011). Based on sensitivity and specificity, they found that multivariate logistic regression slightly
outperformed the machine learning approaches which included random forest, AdaBoost, naïve Bayes,
support vector machine (SVM), and classification k-nearest neighbours (kNN). The naïve Bayes model
slightly underperformed compared to the multivariate logistic regression model. 75 % - 80 % of injuries
were missed by the random forest, AdaBoost and kNN algorithms, which were not deemed sensitive enough
to classify injuries consistently in the dataset used with the selected parameters. The SVM model had good
injury detection capabilities (high sensitivity) but triggered too many false alarms, making it unable to
effectively differentiate ISS ≥ 15 cases in the field.
In Reference [35] a literature review of 56 studies was conducted from 2001 to 2021 to investigate different
methods used in road crash severity modelling, with over half (33/56, 59 %) originating from 2018 onwards.
Across these studies, 20 different methods were used. In contrast to Reference [37], random forest approach
overall yielded the best performance (70 % of the times it was applied, 29 % of studies). SVM was best
performing half the time it was applied (16 % of studies), followed by decision tree (best performing 31 %
of the time, used in 14 % of studies). Bayesian networks performed best 67 % of the time but were only
used in 4 % of the studies. kNN performed best 40 % of the time but were only used in 7 % of the studies.
Although none of the injury prediction studies investigated were explicitly AACN algorithms, this paper
is the largest available investigation into methods used in injury prediction and may be informative when
selecting algorithm creation methodologies.
In Reference [28] the effect of different algorithm creation methods on performance in the specific context of
the On Scene Injury Severity Prediction (OSISP) algorithm was investigated on 47 357 eligible registrations
in a Swedish dataset between 2013-2020. 21 parameters accessible in the pre-hospital setting (although
not transmissible via vehicle sensors) were used to create logistic regression, random forest, XGBoost, SVM
and artificial neural network algorithms. A variety of performance metrics (including those discussed in
Clause 7) were used to compare the algorithm performances, with different algorithms performing well
based on the performance metrics selected. While this OSISP study does not include vehicle parameters,
other algorithms in the OSISP family have historically done so.

In summary, there are a range of viable methods for creating AACN injury severity prediction algorithms
which can be considered for use. Different methods can perform best depending on the outcome variable,
predictor variables, and data set being used. It is vital that models are created using relevant data sources,
with informative parameters and appropriate, required training, testing and validation (see Clause 7)
irrespective of the chosen algorithm construction method.
6.3.4 Important considerations for selecting model parameters
Regardless of the classification method chosen, categories will primarily be based on factors related
to crash dynamics. In some cases, data on such factors are retrieved from a vehicle’s sensing diagnostic
module (SDM) or from storage in the event data recorder (EDR). In other cases, surrogates for such factors
are estimated based on available data. These factors may include Δv, crash direction, and seat occupancy,
among others. While some factors, such as occupant age, sex, height and weight may help to create a more
accurate model, these attributes may not be available at the time of the crash. If the implementation of the
model is intended to be effective immediately following a collision, only inputs available at the time of the
collision should be included in the initial prediction. If data is obtained following the collision, but prior or
during emergency resource allocation, then this should be able to be accommodated at the point at which
that data is collected and reported (such as a call centre entering information as it is gathered). Table A.2
provides details on the parameters adopted in existing models at the time of development of this document,
[7] [40]
and further information can be found in a PhD thesis and an associated publication .
7 Recommendations on the metrics to validate injury prediction algorithms
7.1 Common performance metrics used to assess classification algorithms
7.1.1 General
Models should be appropriately validated by assessing performance during construction prior to algorithm
implementation. Depending on the method chosen, validation could include various methods of cross
validation, the creation of a separate test set of data versus training data, and the calculation of several
performance criteria. It is essential to validate the model to ensure that it is accurate, does not over-fit the
training data and continues to be relevant. Trade-offs between over-triage (driven by false positives) and
under-triage (driven by false negatives) should be considered while using performance metrics that are
appropriate for the data source adopted for algorithm construction, especially where a small prevalence of
the injury class exists.
A subset of the retrospective data source used to build the model is withheld from model creation to ensure
that the model’s performance can be tested on unseen data. The created model is then applied to the subset
of withheld data. How well the created model predicts outcomes in the withheld data subset assesses the
performance of the model. The degree of injury predicted from the created model is compared with the
actual degree of injury in the unseen, retrospective data. This process enables the performance of the model
to be assessed, as outlined in this clause and Annex C.
NOTE The degree of injury is obtained from the model, with a minor injury defined as less than a certain threshold
value and a severe injury defined as at or above that threshold value.
7.1.2 Categorising outcomes from model prediction to enable performance assessment
Assessing classification accuracy requires description of correctly and incorrectly predicted outcomes. The
prediction outcomes of a binary classification can be described in four categories: true positives (the model
correctly classifies an event which caused severe injury), true negatives (the model correctly classifies an
event as not causing severe injury), false positive (the model incorrectly classifies an event as causing severe
injury when in fact no injury occurred) and false negative (the model incorrectly classifies an event as
...