ISO/IEC 30107-3:2017

INTERNATIONAL ISO/IEC
STANDARD 30107-3
First edition
2017-09
Information technology — Biometric
presentation attack detection —
Part 3:
Testing and reporting
Technologies de l'information — Détection d'attaque de présentation
en biométrie —
Partie 3: Essais et rapports d'essai
Reference number
©
ISO/IEC 2017
© ISO/IEC 2017, Published in Switzerland
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized otherwise in any form
or by any means, electronic or mechanical, including photocopying, or posting on the internet or an intranet, without prior
written permission. Permission can be requested from either ISO at the address below or ISO’s member body in the country of
the requester.
ISO copyright office
Ch. de Blandonnet 8 • CP 401
CH-1214 Vernier, Geneva, Switzerland
Tel. +41 22 749 01 11
Fax +41 22 749 09 47
copyright@iso.org
www.iso.org
ii © ISO/IEC 2017 – All rights reserved

Contents Page
Foreword .v
Introduction .vi
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
3.1 Attack elements . 2
3.2 Metrics . 3
4 Abbreviated terms . 4
5 Conformance . 5
6 Presentation attack detection overview . 6
7 Levels of evaluation of PAD mechanisms . 6
7.1 Overview . 6
7.2 General principles of evaluation of PAD mechanisms . 7
7.3 PAD subsystem evaluation . 7
7.4 Data capture subsystem evaluation . 8
7.5 Full-system evaluation . 8
8 Artefact properties . 9
8.1 Properties of presentation attack instruments in biometric impostor attacks . 9
8.2 Properties of presentation attack instruments in biometric concealer attacks .10
8.3 Properties of synthesized biometric samples with abnormal characteristics .10
9 Considerations in non-conformant capture attempts of biometric characteristics .11
9.1 Methods of presentation.11
9.2 Methods of assessment .11
10 Artefact creation and usage in evaluations of PAD mechanisms .11
10.1 General .11
10.2 Artefact creation and preparation .12
10.3 Artefact usage .13
10.4 Iterative testing to identity effective artefacts .13
11 Process-dependent evaluation factors .13
11.1 Overview .13
11.2 Evaluating the enrolment process .14
11.3 Evaluating the verification process .14
11.4 Evaluating the identification process .14
11.5 Evaluating offline PAD mechanisms .15
12 Evaluation using Common Criteria framework .15
12.1 General .15
12.2 Common Criteria and biometrics .17
12.2.1 Overview .17
12.2.2 General evaluation aspects .17
12.2.3 Error rates in testing .17
12.2.4 PAD evaluation .18
12.2.5 Vulnerability assessment .18
13 Metrics for the evaluation of biometric systems with PAD mechanisms .19
13.1 General .19
13.2 Metrics for PAD subsystem evaluation .20
13.2.1 General.20
13.2.2 Classification metrics .20
13.2.3 Non-response metrics .21
13.2.4 Efficiency metrics .22
© ISO/IEC 2017 – All rights reserved iii

13.2.5 Summary .22
13.3 Metrics for data capture subsystem evaluation.22
13.3.1 General.22
13.3.2 Classification metrics .22
13.3.3 Non-response and capture metrics .22
13.3.4 Efficiency metrics .23
13.3.5 Summary .23
13.4 Metrics for full-system evaluation.23
13.4.1 General.23
13.4.2 Accuracy metrics . .23
13.4.3 Efficiency metrics .24
13.4.4 Summary .24
Annex A (informative) Classification of attack types .25
Annex B (informative) Examples of artefact species used in a PAD subsystem evaluation for
fingerprint capture devices .31
Bibliography .32
iv © ISO/IEC 2017 – All rights reserved

Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are
members of ISO or IEC participate in the development of International Standards through technical
committees established by the respective organization to deal with particular fields of technical
activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international
organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the
work. In the field of information technology, ISO and IEC have established a joint technical committee,
ISO/IEC JTC 1.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular the different approval criteria needed for
the different types of document should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject
of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent
rights. Details of any patent rights identified during the development of the document will be in the
Introduction and/or on the ISO list of patent declarations received (see www.iso.org/patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation on the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to the
World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see the following
URL: www.iso.org/iso/foreword.html.
This document was prepared by Technical Committee ISO/IEC JTC 1, Information technology, SC 37,
Biometrics.
A list of all parts in the ISO 30107 series can be found on the ISO website.
© ISO/IEC 2017 – All rights reserved v

Introduction
The presentation of an artefact or of human characteristics to a biometric capture subsystem in a
fashion intended to interfere with system policy is referred to as a presentation attack. ISO/IEC 30107
(all parts) addresses techniques for the automated detection of presentation attacks. These techniques
are called presentation attack detection (PAD) mechanisms.
As is the case for biometric recognition, PAD mechanisms are subject to false positive and false negative
errors. False positive errors wrongly categorize bona fide presentations as attack presentations,
potentially flagging or inconveniencing legitimate users. False negative errors wrongly categorize
presentation attacks (also known as attack presentations) as bona fide presentations, potentially
resulting in a security breach.
Therefore, the decision to use a specific implementation of PAD will depend upon the requirements
of the application and consideration of the trade-offs with respect to security, evidence strength, and
efficiency.
The purpose of this document is as follows:
— to define terms related to biometric presentation attack detection testing and reporting, and
— to specify principles and methods of performance assessment of biometric presentation attack
detection, including metrics.
This document is directed at vendors or test labs seeking to conduct evaluations of PAD mechanisms.
Biometric performance testing terminology, practices, and methodologies for statistical analysis have
been standardized through ISO and Common Criteria. Metrics such as FAR, FRR, and FTE are widely
used to characterize biometric system performance. Biometric performance testing terminology,
practices, and methodologies for statistical analysis are only partially applicable to the evaluation of
PAD mechanisms due to significant, fundamental differences between biometric performance testing
concepts and PAD mechanism testing concepts. These differences can be categorized as follows:
a) Statistical significance
Biometric performance testing utilizes a statistically significant number of test subjects representative
of the targeted user group. Error rates are not expected to vary significantly when adding more test
subjects or using a completely different group. Generally, taking more measurements increases the
accuracy of the error rates.
In PAD testing, many biometric modalities can be attacked by a large or indeterminate number of
potential presentation attack instrument (PAI) species. In these cases, it is very difficult or even
impossible to have a comprehensive model of all possible presentation attack instruments. Hence, it
could be impossible to find a representative set of PAI species for the evaluation. Therefore, measured
error rates of one set of presentation attack instruments cannot be assumed to be applicable to a
different set.
PAI species present a source of systematic variation in a test. Different PAI may have significantly
different error rates. Additionally, within any given PAI species, there will be random variation across
instances of the PAI series. The number of presentations required for a statistically significant test
will scale linearly with the number of PAI species of interest. Within each PAI species, the uncertainty
associated with a PAD error rate estimate will depend on the number of artefacts tested and the number
of individuals.
EXAMPLE 1 In fingerprint biometrics, many potent artefact materials are known, but any material or material
mixture that can present fingerprint features to a biometric sensor is a possible candidate. Since artefact
properties such as age, thickness, moisture, temperature, mixture rates, and manufacturing practices can
have a significant influence on the output of the PAD mechanism, it is easy to define tens of thousands of PAI
species using current materials. Hundreds of thousands of presentations would be needed for a proper statistical
analysis – even then, resulting error rates could not be transferred to the next set of new materials.
vi © ISO/IEC 2017 – All rights reserved

b) Comparability of test results across systems
In biometric performance testing, application-specific error rates based on the same corpus of biometric
samples can be used to compare different biometric systems or different configurations. The meaning
of “better” and “worse” is generally understood.
By contrast, when using error rates to benchmark PAD mechanisms, terms such as “better” can be
highly dependent on the intended application.
EXAMPLE 2 In a given testing scenario with 10 PAI species (presented 100 times), System detects 90 % of
attack presentations and System detects 85 %. System detects all presentations for 9 PAI species but fails to
2 1
detect all presentations with the 10th PAI species. System detects 85 % of all PAI species. Which is better? In a
security analysis, System would be worse than System , because revealing the 10th PAI species would orient an
1 2
attacker such that he could use this method to defeat the capture device all the time. However, if attackers could
be prevented from using the 10th PAI species, System would be better than System , because individual rates
1 2
indicate that it is possible to overcome System with all PAI species.
c) Cooperation
Many biometric performance tests address applications such as access control in which subjects
are cooperative. Errors due to incorrect operation are an issue of a lack of knowledge, experience
or guidance rather than intent. Significant uncooperative behaviour in a group is not part of the
underlying “biometric model” and would render the determined error rates almost useless for biometric
performance testing.
PAD tests include subjects whose behaviour is not cooperative. Attackers will try to find and exploit any
weakness of the biometric system, circumventing or manipulating its intended operation. Presentation
attack types, based on the experience and knowledge of the tester, can change the success rates for an
attack dramatically. Hence, it can be difficult to define testing procedures that measure error rates in a
fashion representative of cooperative behaviour.
d) Automated testing
In biometric performance testing, it is often possible to test comparison algorithms using databases
from devices or sensors of similar quality. Performance can be measured in a technology evaluation
using previously collected corpuses of samples as specified in ISO/IEC 19795-1.
In PAD testing, data from the biometric sensor (e.g. digitized fingerprint images) may be insufficient
to conduct evaluations. Biometric systems with PAD mechanisms often contain additional sensors
to detect specific properties of a biometric characteristic. Hence, a database previously collected for
a specific biometric system or configuration may not be suitable for another biometric system or
configuration. Even slight changes in the hardware or software could make earlier measurements
useless. It is generally impractical to store multivariate synchronized PAD signals and replay them in
automated testing. Therefore, automated testing is often not an option for testing and evaluating PAD
mechanisms.
e) Quality and performance
In biometric performance testing, performance is usually linked directly to biometric data quality.
Low-quality samples generally result in higher error rates while a test with only high-quality samples
will generally result in lower error rates. Hence, quality metrics are often used to improve performance
(dependent on the application).
In PAD testing, even though low biometric quality can cause an artefact to be unsuccessful, there is no
reason to assume a certain quality level from artefacts in general. Samples from artefacts can exhibit
better quality than samples from human biometric characteristics. Absent a model of attacker skill,
it seems valid (at least in a security evaluation) to assume a “worst case” scenario where the attacker
always uses the best possible quality. That way, one can at least determine a guaranteed minimal
detection rate for the specific test set while reducing the number of necessary tests at the same time.
It is then a matter of rating the attack potential of successful artefacts (effort and expertise for the
needed quality) in order to assess the security level, as is the practice in Common Criteria evaluations.
© ISO/IEC 2017 – All rights reserved vii

Based on the differences a) through e), the following general comments regarding error rates and
metrics related to PAD mechanisms can be derived:
— In an evaluation, PAI species are analysed/rated separately.
— Attack presentation classification error rates other than 0 % for a PAI species only prove that the PAI
can be successful. A different tester might achieve a higher or lower attack presentation classification
error rate. Further, training to identify the relevant material and presentation parameters could
increase the attack presentation classification error rate for this PAI species. The experience and
knowledge of the tester, as well as the availability of the necessary resources, are significant factors
in PAD testing and are taken into account when conducting comparisons or performance analysis.
— Error rates for PAD mechanisms are determined by the specific context of the given PAD
mechanism, the set of PAI species, the application, the test approach, and the tester. Error rates
for PAD mechanisms are not necessarily comparable across similar tests, and error rates for PAD
mechanisms are not necessarily reproducible by different test laboratories.
viii © ISO/IEC 2017 – All rights reserved

INTERNATIONAL STANDARD ISO/IEC 30107-3:2017(E)
Information technology — Biometric presentation attack
detection —
Part 3:
Testing and reporting
1 Scope
This document establishes:
— principles and methods for performance assessment of presentation attack detection mechanisms;
— reporting of testing results from evaluations of presentation attack detection mechanisms;
— a classification of known attack types (in an informative annex).
Outside the scope are:
— standardization of specific PAD mechanisms;
— detailed information about countermeasures (i.e. anti-spoofing techniques), algorithms, or sensors;
— overall system-level security or vulnerability assessment.
The attacks considered in this document take place at the sensor during presentation. Any other attacks
are considered outside the scope of this document.
2 Normative references
The following documents are referred to in the text in such a way that some or all of their content
constitutes requirements of this document. For dated references, only the edition cited applies. For
undated references, the latest edition of the referenced document (including any amendments) applies.
ISO/IEC 2382-37, Information technology — Vocabulary — Part 37: Biometrics
ISO/IEC 19795-1:2006, Information technology — Biometric performance testing and reporting — Part 1:
Principles and framework
ISO/IEC 30107-1:2016, Information technology — Biometric presentation attack detection — Part 1:
Framework
3 Terms and definitions
For the purposes of this document, the terms and definitions given in ISO/IEC 2382-37 and
ISO/IEC 30107-1 and the following apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— IEC Electropedia: available at http://www.electropedia.org/
— ISO Online browsing platform: available at http://www.iso.org/obp
© ISO/IEC 2017 – All rights reserved 1

3.1 Attack elements
3.1.1
presentation attack
attack presentation
presentation to the biometric data capture subsystem with the goal of interfering with the operation of
the biometric system
Note 1 to entry: An attack presentation might be a single attempt, a multi-attempt transaction, or some other
type of interaction with a subsystem.
3.1.2
bona fide presentation
interaction of the biometric capture subject and the biometric data capture subsystem in the fashion
intended by the policy of the biometric system
Note 1 to entry: Bona fide is analogous to normal or routine, when referring to a bona fide presentation.
Note 2 to entry: Bona fide presentations can include those in which the user has a low level of training or
skill. Bona fide presentations encompass the totality of good-faith presentations to a biometric data capture
subsystem.
3.1.3
attack type
element and characteristic of a presentation attack, including PAI species, concealer or impostor attack,
degree of supervision, and method of interaction with the capture device
3.1.4
test approach
totality of considerations and factors involved in PAD evaluation
Note 1 to entry: Elements of a test approach are given in Clauses 7 to 11.
Note 2 to entry: A test approach refers to all processes, factors, and aspects specified in the course of the
evaluation.
3.1.5
item under test
IUT
implementation that is the object of a test assertion or test case
Note 1 to entry: The IUT is the equivalent of TOE in Common Criteria evaluations.
3.1.6
PAI species
class of presentation attack instruments created using a common production method and based on
different biometric characteristics
EXAMPLE 1 A set of fake fingerprints all made in the same way with the same materials but with different
friction ridge patterns would constitute a PAI species.
EXAMPLE 2 A specific type of alteration made to the fingerprints of several data capture subjects would
constitute a PAI species.
Note 1 to entry: The term “recipe” is often used to refer to how to make a PAI species.
Note 2 to entry: Presentation attack instruments of the same species may have different success rates due to
variability in the production process.
2 © ISO/IEC 2017 – All rights reserved

3.1.7
PAI series
presentation attack instruments based on a common medium and production method and a single
biometric characteristic source
EXAMPLE A set of fake fingerprints all made in the same way with the same materials and with the same
friction ridge pattern.
Note 1 to entry: Depending on the experimental goals, an evaluation may utilize series from one source or from
several. While tests involving several biometric sources may demonstrate generality of a PAI species, they add
variation associated with individual human traits.
3.1.8
target of evaluation
TOE
within Common Criteria, the IT product that is the subject of the evaluation
Note 1 to entry: The TOE is the equivalent of IUT in Common Criteria evaluations.
3.1.9
attack potential
measure of the capability to attack a TOE given the attacker’s knowledge, proficiency, resources and
motivation
3.2 Metrics
3.2.1
attack presentation classification error rate
APCER
proportion of attack presentations using the same PAI species incorrectly classified as bona fide
presentations in a specific scenario
3.2.2
bona fide presentation classification error rate
BPCER
proportion of bona fide presentations incorrectly classified as presentation attacks in a specific
scenario
3.2.3
attack presentation non-response rate
APNRR
proportion of attack presentations using the same PAI species that cause no response at the PAD
subsystem or data capture subsystem
EXAMPLE A fingerprint system may not register or react to the presentation of a PAI due to the PAI’s lack of
realism.
3.2.4
bona fide presentation non-response rate
BPNRR
proportion of bona fide presentations that cause no response at the PAD subsystem or data capture
subsystem
3.2.5
attack presentation acquisition rate
APAR
proportion of attack presentations using the same PAI species from which the data capture subsystem
acquires a biometric sample of sufficient quality
© ISO/IEC 2017 – All rights reserved 3

3.2.6
impostor attack presentation match rate
IAPMR
proportion of impostor attack presentations using the
same PAI species in which the target reference is matched
3.2.7
concealer attack presentation non-match rate
CAPNMR
proportion of concealer attack presentations using
the same PAI species in which the reference of the concealer is not matched
3.2.8
impostor attack presentation identification rate
IAPIR
proportion of impostor attack presentations using
the same PAI species in which the targeted reference identifier is among the identifiers returned or,
depending on intended use case, at least one identifier is returned by the system
Note 1 to entry: An attacker might be both an impostor (trying to match an existing non-self enrolee) and a
concealer (obscuring his real biometric sample with a PAI).
3.2.9
concealer attack presentation non-identification rate
CAPNIR
proportion of concealer presentation attacks using
the same PAI species in which the reference identifier of the concealer is not among the identifiers
returned or, depending on intended use case, in which no identifiers are returned
Note 1 to entry: In a negative identification system, such as a black-list, the concealer could intend that no
identifiers are returned to avoid scrutiny by a human operator.
3.2.10
PAD subsystem processing duration
PS-PD
duration required for the PAD subsystem to classify PAD data
3.2.11
data capture subsystem processing duration
DCS-PD
duration required for the data capture subsystem to acquire a sample, inclusive of PAD subsystem
processing duration (if applicable)
3.2.12
full-system processing duration
FS-PD
duration required for the data capture subsystem and comparison subsystem to acquire and process a
sample, inclusive of PAD subsystem processing duration (if applicable)
4 Abbreviated terms
The abbreviated terms shown in Table 1 are used in this document.
Table 1 — Abbreviated terms
APCER Attack Presentation Classification Error Rate
APAR Attack Presentation Acquisition Rate
APNRR Attack Presentation Non-Response Rate
4 © ISO/IEC 2017 – All rights reserved

Table 1 (continued)
BPCER Bona Fide Presentation Classification Error Rate
BPNRR Bona Fide Presentation Non-Response Rate
CCRA Common Criteria Recognition Arrangement
CAPNIR Concealer Attack Presentation Non-Identification Rate
CAPNMR Concealer Attack Presentation Non-Match Rate
DCS-PD Data Capture Subsystem Processing Duration
EAL Evaluation Assurance Level
FTA Failure to Acquire Rate
FTE Failure to Enrol Rate
FAR False Accept Rate
FNIR False Negative Identification Rate
FPIR False Positive Identification Rate
FRR False Reject Rate
FSDPP Fingerprint Spoof Detection Protection Profiles
FS-PD Full-System Processing Duration
IAPIR Impostor Attack Presentation Identification Rate
IAPMR Impostor Attack Presentation Match Rate
IUT Item Under Test
PS-PD PAD Subsystem Processing Duration
PAD Presentation Attack Detection
PAI Presentation Attack Instrument
PAIS Presentation Attack Instrument Species
TOE Target of Evaluation
5 Conformance
To conform to this document, an evaluation of PAD mechanisms shall be planned, executed and reported
in accordance with the mandatory requirements as follows:
— Clause 6 to 11.1;
— Clause 13.1;
— for evaluations of PAD mechanisms in enrolment, see 11.2;
— for evaluations of PAD mechanisms in verification, see 11.3;
— for evaluations of PAD mechanisms in positive or negative identification, see 11.4;
— for PAD subsystem evaluations, see 13.2;
— for data capture subsystem evaluations, see 13.3;
— for full-system evaluations of verification systems, see 13.4.2.1;
— for full-system evaluations of positive identification systems, see 13.4.2.2;
— for full-system evaluations of negative identification systems, see 13.4.2.3.
© ISO/IEC 2017 – All rights reserved 5

6 Presentation attack detection overview
This document describes two types of presentation attackers: biometric impostors (a.k.a. impersonators)
and biometric concealers. These types of attackers differ in that biometric impostors typically need
to defeat PAD subsystems, pass quality checks, and match through comparison subsystems, whereas
biometric concealers do not need to match through comparison subsystems.
While the desired impersonation or concealment outcome may lend itself towards a sub-set of attack
types, any type of PAI can be used by either type of attacker.
Evaluations of PAD mechanisms and resulting reports shall specify the type of presentation attacker,
biometric impostor or biometric concealer, considered in an evaluation.
Evaluations of PAD mechanisms are classifiable as one of three general types, increasing in specificity,
as follows:
— generic, broad evaluations of PAD mechanisms of any device for an unknown application;
— application-focused evaluations of PAD mechanisms in which the set/range of attack types is
selected to be appropriate to the application, such as those discussed in Clause 11;
— product-specific evaluations of PAD mechanisms, used to test a supplier’s claim of performance
against a specific category of attack types.
Evaluations of PAD mechanisms and resulting reports shall describe the type of evaluation conducted
as well as the attack types to be tested.
7 Levels of evaluation of PAD mechanisms
7.1 Overview
Evaluation of PAD mechanisms is determined by the item under test (IUT). PAD evaluations and
resulting reports shall fully describe the IUT, including all configurations and settings as well as
the amount of information available to the evaluator about PAD mechanisms in place. IUTs shall be
categorized as follows:
— PAD subsystem;
— data capture subsystem;
— full system.
A PAD subsystem is a hardware and/or software that implements a PAD mechanism and makes an
explicit declaration regarding the detection of presentation attacks. Results of the PAD mechanism are
accessible to the evaluator and are an aspect of the evaluation.
EXAMPLE 1 A PAD subsystem could be a fingerprint device that logs a PAD score or decision when a PAI is
presented.
A data capture subsystem, consisting of capture hardware or/and software, couples PAD mechanisms
and quality checks in a fashion opaque to the evaluator. The evaluator may not necessarily know
whether the data capture subsystem utilizes presentation attack detection. Acquisition may be for the
purpose of enrolment or recognition, but no comparison takes place in the data capture subsystem.
EXAMPLE 2 A data capture subsystem could be an iris collection device that fails to acquire a sample from an
iris artefact, where it is impossible to determine whether failure to acquire is due to a liveness check or quality
check (the implementation does not provide this level of transparency).
NOTE For simplicity, the term “quality check” encompasses feature extraction, segmentation, or any other
automated processing function used to validate the utility of a biometric sample.
6 © ISO/IEC 2017 – All rights reserved

A full system adds biometric comparison to the PAD subsystem or data capture subsystem, comprising
a full end-to-end system. This leads to additional failure points for the PAI beyond PAD mechanisms
and quality checks. In a full system, there might be one or multiple PAD mechanisms at different points
in the system.
Evaluations of PAD mechanisms and resulting reports shall specify the applicable evaluation level,
whether PAD subsystem, data capture subsystem, or full system. The resulting reports should discuss
how the evaluation level influenced PAD testing.
7.2 General principles of evaluation of PAD mechanisms
Evaluations of PAD mechanisms shall cover a defined variety of attack types by utilizing a representative
set of presentation attack instruments and a representative set of bona fide capture subjects.
For the set of presentation attack instruments, evaluations of PAD mechanisms should be based on the
appropriate evaluation type (see Clause 6) and on relevant attack types. Not all PAD mechanisms are
designed to address all possible presentation attacks.
EXAMPLE A PAD mechanism designed to recognize an artificial biometric characteristic is not likely to be
effective for detecting an altered biometric characteristic.
Once the types are defined, the number and range of presentation attack instruments to be evaluated
should be specified. Establishing whether a specific attack type reproducibly succeeds does not require
a very large number of presentations.
The evaluator shall define the parameters of the attack presentation to fully characterize the range of
attacker interactions with the IUT, to include the temporal boundaries of the presentation.
A representative set of bona fide capture subjects is required to determine the frequency with which
the PAD mechanism incorrectly classifies bona fide presentations. This is a critical part of PAD testing
since a PAD mechanism could erroneously classify bona fide presentations as attack presentations. A
high classification error rate for bona fide capture subjects would reduce system usability.
The representativeness of bona fide presentations should consider test subject selection and size as
described in ISO/IEC 19795-1:2006, 6.5 and 6.6. Particularly, total numbers of bona fide presentations
should exceed that required by Rule of 30.
In an evaluation of PAD mechanisms, the evaluator shall (1) define bona fide presentations and
representative capture subjects for the target application and population and (2) provide a rationale for
these definitions.
NOTE Defining "bona fide" presentations and representative capture subjects can be a challenge in
evaluations of PAD mechanisms. In some cases, the evaluator may define bona fide presentations as those that
conform to vendor or implementer specifications. However, in certain applications, bona fide or representative
capture subject interaction with data capture devices may encompass a wide range of behaviours and
conditions. For example, a vendor may define a conformant presentation to a fingerprint sensor as one conducted
with clean fingerprints. While one could conduct a test in which all capture subjects without perfectly clean
fingerprints are excluded, it is reasonable to expect that operational systems have some tolerance for a range of
regular, reasonable, or typical fingerprint conditions. Otherwise, operational systems would have excessively
high false rejection or failure to enrol (FTE) rates. This is particularly relevant to PAD testing, because bona
fide presentation classification errors may be most frequently encountered among data capture subjects
whose interactions with data capture devices, while sufficient for enrolment or biometric recognition, are only
marginally conformant with vendor specifications.
7.3 PAD subsystem evaluation
PAD subsystem evaluations measure the ability of the PAD subsystem to correctly classify both
attack presentations and bona fide presentations. An effective attack presentation will be incorrectly
classified as a bona fide presentation, resulting in the defeat of the PAD subsystem.
© ISO/IEC 2017 – All rights reserved 7

PAD subsystem evaluations may focus on the effectiveness of the sensor (mostly hardware or possibly
internal firmware) in terms of refusal to acquire a sample, including cases with or without automated
indications of refusal. Such evaluations focus on rejecting presentation attack instruments. The output
of the PAD subsystem could be discrete, such as a pass/fail to each PAI utilized.
Alternatively, PAD subsystem evaluations may focus on the effectiveness of a PAD algorithm (exemplified
[8]
by LivDet ). This type of PAD subsystem evaluation can be performed offline with a corpus of samples;
the PAD subsystem determines whether samples come from an attack. Such tests are typically based on
a collected database, analogous to technology tests in biometric performance evaluations.
If the PAD subsystem returns a PAD score, false-negative and false-positive error rates can be expressed
parametrically as functions of the decision threshold (e.g. through a detection error trade-off curve).
Clause 10 provides an overview of factors that need to be considered when designing a test for PAD
subsystems designed to recognize artefacts.
7.4 Data capture subsystem evaluation
In data capture subsystems, presentation attacks may fail for reasons other than detection in the
PAD subsystem. For example, the data capture subsystem may fail to respond to a presentation
attack, or a quality subsystem may reject the presentation attack. In data capture subsystems whe
...