ISO/IEC JTC 1/SC 37/WG 5 - Biometric testing and reporting

This document establishes: — machine-readable records for documenting the output of a biometric test; — formats for data that ISO/IEC 19795 series tests are required to report; and — an ASN.1 syntax for test reports. This document does not: — require, prohibit, or otherwise specify the format of biometric samples or templates used in a test; — require, prohibit or otherwise specify the encapsulation of biometric samples or templates used in a test; or — regulate metrics for tests. NOTE The reportable metrics are established in ISO/IEC 19795-1.

This document addresses: — requirements for planning, executing and reporting the influence of user interaction on biometric system performance based on scenario test methodologies, considering three kinds of factors: a) factors related to the design, position or condition of the capture device, b) factors depending on users and user attributes, c) factors depending on the interaction of users with the biometric system; — specifications for the definition, establishment and measurement of conditions needed for evaluation, including those relating to equipment; — requirements for establishing a reference evaluation condition (REC) and target evaluation condition(s) (TEC) to compare the influence of user interaction factors; — a specification of the biometric evaluation including requirements for test population, test protocols, data to record, test results; and — procedures for carrying out the overall evaluation. This document does not: — determine which parameters ought to be analysed for a specific biometric modality. This is currently covered in ISO/IEC TR 19795-3; — specify requirements for performing a vulnerability analysis modifying user interaction influence factors; — include procedures for performing usability testing.

This document: — establishes general principles for testing the performance of biometrics systems in terms of error rates and throughput rates for purposes including measurement of performance, prediction of performance, comparison of performance, and verifying conformance with specified performance requirements; — specifies performance metrics for biometric systems; — specifies requirements on the recording of test data and reporting of test results; and — specifies requirements on test protocols in order to: — reduce bias due to inappropriate data collection or analytic procedures; — help achieve the best estimate of field performance for the expended effort; — improve understanding of the limits of applicability of the test results. This document is applicable to empirical performance testing of biometric systems and algorithms through analysis of the comparison scores and decisions output by the system, without requiring detailed knowledge of the system’s algorithms or of the underlying distribution of biometric characteristics in the population of interest. Not within the scope of this document is the measurement of error and throughput rates for people deliberately trying to subvert the intended operation of the biometric system (e.g. by presentation attacks).

This document provides guidance for performance testing of biometrics when this technology is used on mobile devices with local biometric authentication to improve authentication assurance. This document aims to: — Provide guidance for affordable and cost-efficient testing and reporting methods for performance assessment at a full system level of biometric systems embedded in mobile devices with offline evaluation of false accept rate (FAR) claims. — Define modality-specific considerations of these methods. This document is applicable to: — verification use cases related to secure transactions. This document is not applicable to: — privacy aspects; — secure authentication from mobile device to server; — testing and reporting for presentation attack detection (PAD) mechanisms in mobile devices; — performance testing of biometric sub-systems such as acquisition sub-system or comparison sub-system; — continuous authentication.

The purpose of ISO/IEC TR 29189:2015 is to identify and characterize those aspects of performance testing that are unique to examiner assisted biometric applications.

ISO/IEC 19795-6:2012: provides guidance on the operational testing of biometric systems; specifies performance metrics for operational systems; details data that may be retained by operational systems to enable performance monitoring; and specifies requirements on test methods, recording of data, and reporting of results of operational evaluations. ISO/IEC 19795-6:2012 does not: cover testing of operational systems in the laboratory or address vulnerability testing.

ISO/IEC 19795-4:2008 prescribes methods for technology and scenario evaluations of multi-supplier biometric systems that use biometric data conforming to biometric data interchange format standards. It specifies requirements needed to assess performance available from samples formatted according to a standard interchange format (SIF), performance available when samples formatted according to a SIF are exchanged, performance available from samples formatted according to a SIF, relative to proprietary data formats, SIF interoperability by quantifying cross-product performance relative to single-product performance, performance available from multi-sample and multimodal data formatted according to one or more SIFs, and performance interoperability of biometric capture devices. In addition, ISO/IEC 19795-4:2008 includes procedures for establishing an interoperable set of implementations, defines procedures for testing interoperability with previously established sets of implementations, and gives testing procedures for the measurement of interoperable performance. It does not establish a conformance test for biometric data interchange formats, or provide test procedures for online data collection.