ISO/IEC 30116:2016

INTERNATIONAL ISO/IEC
STANDARD 30116
First edition
2016-10-01
Information technology — Automatic
identification and data capture
techniques — Optical Character
Recognition (OCR) quality testing
Technologies de l’information — Techniques automatiques
d’identification et de capture des données — Essais de qualité des
caractères pour reconnaissance optique
Reference number
©
ISO/IEC 2016
© ISO/IEC 2016, Published in Switzerland
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized otherwise in any form
or by any means, electronic or mechanical, including photocopying, or posting on the internet or an intranet, without prior
written permission. Permission can be requested from either ISO at the address below or ISO’s member body in the country of
the requester.
ISO copyright office
Ch. de Blandonnet 8 • CP 401
CH-1214 Vernier, Geneva, Switzerland
Tel. +41 22 749 01 11
Fax +41 22 749 09 47
copyright@iso.org
www.iso.org
ii © ISO/IEC 2016 – All rights reserved

Contents Page
Foreword .iv
Introduction .v
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 Abbreviated terms . 3
5 Quality grading . 3
6 Measurement methodology for OCR-B . 3
6.1 Overview of methodology . 3
6.2 Obtaining the test image . 3
6.2.1 Measurement conditions . 3
6.2.2 Raw image . 3
6.2.3 Reference grey-scale image . 3
6.2.4 Binarized image . 4
6.3 Reference reflectivity measurements . 4
6.3.1 General requirements . 4
6.3.2 Light sources . 4
6.3.3 Effective resolution . 4
6.3.4 Optical geometry . 4
6.3.5 Inspection area . 8
6.4 Basis of symbol grading . 8
6.5 Capture the raw image . 9
6.6 Image assessment parameters and grading . 9
6.6.1 Determining the document horizontal axis . 9
6.6.2 Character best-fit algorithm . . 9
6.6.3 Position of a character .11
6.6.4 Character evaluation value (CEV) in the best-fit location .11
6.6.5 Background noise . .12
6.6.6 Contrast PCS of the characters .12
7 Reporting the grade .12
Annex A (normative) OCR-B character centreline coordinates .14
Annex B (normative) Threshold determination method .20
Annex C (normative) OCR reference decode algorithm .24
Annex D (informative) Example calculation of character evaluation value (CEV) .25
Bibliography .29
© ISO/IEC 2016 – All rights reserved iii

Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are
members of ISO or IEC participate in the development of International Standards through technical
committees established by the respective organization to deal with particular fields of technical
activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international
organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the
work. In the field of information technology, ISO and IEC have established a joint technical committee,
ISO/IEC JTC 1.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular the different approval criteria needed for
the different types of document should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject
of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent
rights. Details of any patent rights identified during the development of the document will be in the
Introduction and/or on the ISO list of patent declarations received (see www.iso.org/patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation on the meaning of ISO specific terms and expressions related to conformity assessment,
as well as information about ISO’s adherence to the World Trade Organization (WTO) principles in the
Technical Barriers to Trade (TBT) see the following URL: www.iso.org/iso/foreword.html.
The committee responsible for this document is ISO/JTC 1, Information technology, Subcommittee SC 31,
Automatic identification and data capture techniques.
iv © ISO/IEC 2016 – All rights reserved

Introduction
For the inspection of ID documents, i.e. MRTDs (Machine Readable Travel Documents) according to
ISO/IEC 7501 (all parts)/ICAO Doc 9303 (all parts) and driving licences according to ISO/IEC 18013
(all parts), a reliable and ergonomic document inspection technology is essential. Considering RFID
interoperability, strong improvement has been reached introducing mechanisms for interoperability
evaluation and testing of MRTDs and reader devices. Similar standards for optical reading would
improve the reliability of OCR. This is especially important because OCR of the document’s MRZ (Machine
Readable Zone) is essential for accessing BAC (Basic Access Control) and/or SAC (Supplementary Access
Control) protected passports.
Thus, reliable OCR makes the performance of automated border control systems, as well as of many
other applications, more predictable. Furthermore, the evaluation of document reader products can be
done much easier. This standardization project defines test methods to evaluate OCR document quality.
Furthermore, it defines requirements ensuring the compliance to the applicable OCR standards. The
project applies experiences from other domains such as bar code reading and possibly other test
methods for OCR. Where conflicts in the specification work between MRTDs and driving licenses may
arise, satisfying the definitions for MRTDs is given preference.
© ISO/IEC 2016 – All rights reserved v

INTERNATIONAL STANDARD ISO/IEC 30116:2016(E)
Information technology — Automatic identification and
data capture techniques — Optical Character Recognition
(OCR) quality testing
1 Scope
This document
— specifies the methodology for the measurement of specific attributes of OCR-B character strings,
— defines a method for evaluating these measurements and deriving an overall assessment of character
string quality,
— defines a reference decode algorithm for OCR-B, and
— gives information on possible causes of deviation from optimum grades to assist users in taking
appropriate corrective action.
This document applies to OCR-B as defined in ISO 1073-2, but its methodology can be applied partially
or wholly to other OCR fonts.
2 Normative references
There are no normative references in this document.
3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— IEC Electropedia: available at http://www.electropedia.org/
— ISO Online browsing platform: available at http://www.iso.org/obp
3.1
binarized image
binary (black/white) image created by applying the global threshold to the pixel (3.5) values in the
reference grey-scale image
3.2
document reference edge
physical (i.e. mechanical) end of the surface with the MRZ whose position is determined by putting a
black background under the surface with the MRZ and sliding the document up against a physical stop
3.3
inspection area
rectangular area which contains the entire symbol (3.11) to be tested inclusive of its quiet zones
3.4
character outline limits
outlines of an ideal printed image of a character
Note 1 to entry: This is a qualitative evaluation utilized in ISO 1831 that is replaced in this document with SWT.
© ISO/IEC 2016 – All rights reserved 1

3.5
pixel
individual light-sensitive element in a light-sensitive array
Note 1 to entry: Examples of light-sensitive array are CCD (charge coupled device) or CMOS (complementary
metal oxide semiconductor) device.
3.6
raw image
matrix of the reflectance values in x and y coordinates across a two-dimensional image, derived from
the discrete reflectance values of each pixel (3.5) of the light-sensitive array
3.7
reference grey-scale image
raw image (3.6) convolved with a synthesized circular aperture
3.8
scan grade
result of the assessment of a single scan of an OCR symbol, derived by taking the lowest grade achieved
for any measured parameter of the reference grey-scale and binarized images (3.1)
3.9
stroke width
nominal dimension perpendicular to the direction of the line making up an OCR character
3.10
stroke width template
inner and outer character boundaries defined by circles whose centres follow the line created by the
character centreline coordinates defined in Annex A
3.11
symbol
group of OCR characters comprising the entire machine-readable entity (e.g. Machine Readable Zone
(MRZ) as specified in ICAO 9303, sizes ID-1, ID-2 and ID-3) including quiet zones and the document
reference edge (3.2)
Note 1 to entry: Document sizes are defined in ISO/IEC 7501 (all parts) (ICAO 9303) as TD1, TD2 and TD2,
whereas the same sizes are defined in ISO/IEC 7810 as ID-1, ID-1 and ID-3. In this document, we use the terms
ID-1, ID-2 and ID-3.
3.12
X-tolerance
0,08 mm for Size I with a nominal stroke width (3.9) of 0,35 mm
Note 1 to entry: 0,08 mm for Size I with a nominal stroke width of 0,35 mm was originally defined in
ISO 1831:1980, Table 2.
3.13
Y-tolerance
0,15 mm for Size I with a nominal stroke width (3.9) of 0,35 mm
Note 1 to entry: 0,15 mm for Size I with a nominal stroke width of 0,35 mm was originally defined in
ISO 1831:1980, Table 2.
2 © ISO/IEC 2016 – All rights reserved

4 Abbreviated terms
COL character outline limits
CEV character evaluation value
MRZ machine readable zone
SWT stroke width template
5 Quality grading
Quality grades for best-fit, PCS, position and background noise are determined as one of three levels:
recommended, needs attention and not recommended. The parameter with the lowest grade is the
grade of the symbol.
6 Measurement methodology for OCR-B
6.1 Overview of methodology
The basis of the measurement methodology is the evaluation of reflectance from the symbol. This
methodology is also intended to correlate with conditions encountered in OCR scanning systems. The
method starts by obtaining the raw image, which is a high-resolution grey-scale image of the symbol
captured under controlled illumination and viewing conditions.
6.2 Obtaining the test image
6.2.1 Measurement conditions
A test image of the symbol shall be obtained in a configuration that mimics the typical scanning
situation for that symbol, but with substantially higher resolution (see 6.3.3), uniform illumination and
at best focus. The reference optical arrangement is defined in 6.3.4. Alternative optical arrangements
may be used provided that the measurements obtained with them can be correlated with the use of the
reference optical arrangement.
Ambient light levels shall be controlled in order not to influence the measurement results. Whenever
possible, measurements shall be made on the symbol in its final configuration, i.e. the configuration
in which it is intended to be scanned. For MRTD evaluation, optically personalized samples shall be
used. This includes that all layers available at a document including laminations, security features and
protective layers shall be present.
Two principles govern the design of the optical set-up. First, the test image’s grey-scale shall be
nominally linear and not be enhanced in any way. Second, the image resolution shall be adequate to
produce consistent readings, which generally requires that the character stroke-widths span at least
10 image pixels.
6.2.2 Raw image
The raw image is a matrix of the actual reflectance values for each pixel of the light-sensitive array,
from which are derived the reference grey-scale image and the binarized image which are evaluated for
the assessment of symbol quality.
6.2.3 Reference grey-scale image
The reference grey-scale image is obtained from the raw image by processing the individual pixel
reflectance values through a synthetic circular aperture equal to 0,2 mm.
© ISO/IEC 2016 – All rights reserved 3

6.2.4 Binarized image
The binarized image is obtained from the reference grey-scale image by applying the algorithm defined
in Annex B.
6.3 Reference reflectivity measurements
6.3.1 General requirements
Equipment for assessing the quality of symbols in accordance with this subclause shall comprise a
means of measuring and analysing the variations in the reflectivity of a symbol on its substrate over an
inspection area which shall cover the full height and width of the symbol.
The measured reflectance values shall be expressed in percentage terms by means of calibration and
reference to recognized national standards laboratories, where 100 per cent should correspond to the
reflectance of a barium sulphate or magnesium oxide reference sample.
It should be ensured that all materials visible to the camera or close to the optical path are reflection-
free, at least in IR illumination. In particular, the background the symbol is attached to shall be IR
absorbing. The environment temperature shall be between 20°C and 25°C.
6.3.2 Light sources
Measurements shall be made using light emitting diode (LED) light sources at 890 nm and 940 nm
wavelengths.
All illumination elements shall have a diameter of 25 mm or less and may be shaped as circles, squares
or similar.
6.3.3 Effective resolution
The effective resolution of an instrument that implements this document shall be sufficient to ensure
that the parameter grading results are consistent irrespective of the rotation of the symbol. The effective
resolution is the product of the resolution of the light-sensitive array and of the magnification of the
associated optical system and effected by distortions introduced by the optical system. The reference
optical arrangement requires an effective resolution of not less than 10 pixels per stroke width.
6.3.4 Optical geometry
A reference optical geometry is defined for reflectivity measurements and consists of
— flood incident illumination, uniform across the inspection area, from a set of four light sources
arranged at 90-degree intervals around a circle concentric with the inspection area and in a plane
parallel to that of the inspection area, at a height which will allow incident light to fall on the centre
of the inspection area at an angle of 45° to its plane, and
— a light collection device, the optical axis of which is perpendicular to the inspection area and passes
through its centre, and which focuses an image of the test symbol on a light-sensitive array.
The light reflected from the inspection area shall be collected and focused on the light-sensitive array.
Implementations may use alternative optical geometries and components, provided that their
performance can be correlated with that of the reference optical arrangement defined in this subclause.
Figure 1 and Figure 2 illustrate the principle of the optical arrangement, but are not intended to
represent actual devices; in particular, the magnification of the device is likely to differ from 1:1. For
example, it is possible to use a 10 MP industrial camera without IR cut filter with a sensor size of ½”. The
image could be captured from a distance of approximately 350 mm and the lens chosen appropriately.
The resulting magnification then would be 1:21.
4 © ISO/IEC 2016 – All rights reserved

Key
1 light-sensing element
2 lens providing 1:1 magnification (measurement A = measurement B)
3 inspection area
4 light sources
ϑ angle of incidence of light relative to plane of symbol = 45°
Figure 1 — Reference optical arrangement — Side view
© ISO/IEC 2016 – All rights reserved 5

Key
1 light source
2 symbol
Figure 2 — Reference optical arrangement — Top view
6 © ISO/IEC 2016 – All rights reserved

Figure 3 — Reference optical arrangement — Angles and tolerances
When setting up a reference optical arrangement, the following considerations shall be made. The
symbol (e.g. the MRZ of an ID-3 MRTD) has a size of approximately 24 mm × 125 mm (marked as a
bar in the bottom of Figure 3). For other document sizes (ID-1, ID-2), the same size of the object to be
captured shall be used. All areas not covered by the travel document but visible to the camera shall be
made of IR-absorbing material. The small rectangles at the left and right border of Figure 3 represent
the illumination elements.
The nominal illumination angle shall be 45° as given in Figure 1. This angle is measured in the middle
axis of the MRZ zone. An angle of 45° directly determines that the horizontal and vertical distance of
the illumination from the centre of the symbol is identical. This distance should be 200 mm as depicted
in Figure 3.
© ISO/IEC 2016 – All rights reserved 7

For the small side of the symbol (24 mm), the minimal and maximal illumination angles at the symbol
borders are 43,3° and 46,8°, as shown in Table 1.
The difference between the nominal angle (in the centre) and the angles at the borders is much higher
for the long side of the symbol; 37,3° and 55,5°.
Table 1 — Dark pixel portion for threshold of 4.5
Angle Short side (24 mm) Long side (125 mm)
α 45° 45°
α′ 46,8° 55,5°
α″ 43,3° 37,3°
Figure 4 — Reference optical arrangement — Reflection considerations
Figure 4 shows the position where the direct reflection of the illumination at the symbol (i.e. caused by
plain lamination) will not be visible to the camera. The distance between the camera and the symbol
should be approximately 350 mm.
6.3.5 Inspection area
The inspection area within which all measurements shall be a rectangular area framing the complete
symbol. The centre of the inspection area shall be as close as practicable to the centre of the field of
view. For example, the MRZ in a passport shall be placed accordingly.
6.4 Basis of symbol grading
OCR symbol quality assessment shall be based on the measurement and grading of parameters of the
reference grey-scale image, the binarized image derived from it and the application of the reference
decode algorithm to these. Quality grading of these parameters shall be used to provide a relative
measure of symbol quality under the measurement conditions used.
8 © ISO/IEC 2016 – All rights reserved

6.5 Capture the raw image
Centre the symbol in the field of view and align the average bottom edge of the characters with the
sensor as precisely as possible, but always with less than +/−5° deviation.
Find and replace the brightest and darkest 0,005 % pixels in the overall image with the median of the
9 pixels consisting of itself and its 8 immediate neighbours.
Apply the aperture defined in 6.2.3 to the raw image to create a reference grey-scale image.
6.6 Image assessment parameters and grading
6.6.1 Determining the document horizontal axis
The application shall define the MRZ region in relation to the document reference edge.
6.6.2 Character best-fit algorithm
These steps should be followed in order to find the best-fit position of the character outline limit (SWT)
gauges on a character image captured from a machine-readable character string in order to find the
location of the characters.
Using the binarized image, determine four corner positions that bound the character image. From these
points, establish four more points that are further away from the character by the nominal stroke
width. These four points define the range over which the SWT gauges will be moved in order to find the
best-fit position. An example is shown in Figure 5.
Key
1 vertical range
2 horizontal range
Figure 5 — Sample corner positions
Create the SWT for each defined character in Annex A by moving a circle of radius of the appropriate
tolerance value around the centreline of the character and saving the outermost points (note that this is
equivalent to finding the points perpendicular to the centre line at a distance equal to the radius of the
circle). An example of this process is in Figure 6. Figure 7 illustrates the tolerances of the SWT boxes as
derived from ISO 1831.
© ISO/IEC 2016 – All rights reserved 9

Figure 6 — SWT creation example
a)  X-tolerance. Radius 6,75 “sq” inside 10,75 b)  Y-tolerance. Radius 5 “sq” inside 12,5 “sq”
“sq” outside outside
NOTE 1 These are the tolerances from ISO 1831:1980, Table 2.
NOTE 2 “Sq” is the count of squares from the original drawings (see Annex A) where each square equals
1/50 mm.
Figure 7 — SWT tolerance gauges
10 © ISO/IEC 2016 – All rights reserved

Overlay the original captured image with the SWT Y-tolerance gauges such that the horizontal axis of
the gauges is parallel with the document reference edge. Starting with the four extreme positions, move
the SWT Y-tolerance gauges right and left and up and down relative to the document reference edge to
each test position.
At each test position, sum up the reflectance values of each SWT Y-tolerance gauge resolution pixel
within the region defined by the minimum gauge. The test position with the lowest sum is used to
determine the position of the character. If there is more than one test position with the same lowest
sum (e.g. the inside of the minimum gauge is all black), then for each equivalent test position, sum up
the reflectance values of each gauge resolution pixel outside the maximum gauge. The equivalent test
position with the highest sum is used to determine the position of the character. If there is more than
one test position with the same highest sum, then compute the average of these positions and use it to
determine the position of the character.
6.6.3 Position of a character
Using the test position determined in 6.6.2, the location of a character is the origin of the SWT, where
the origin is defined as (0,0) for every character in Annex A.
The position of every character is determined and graded according to an application specific profile.
The character with the lowest grade determines the position grade.
6.6.4 Character evaluation value (CEV) in the best-fit location
A pixel is outside or inside a border if more than 50 % of the pixel area is outside or inside the border,
respectively.
For each tolerance template on every character, use the optimal position of the template placed over the
binarized character in the image. The following lists the variables to be used to calculate the CEV grades:
a) CEV_X_Inside — number of white pixels inside the X-tolerance inner boundary;
b) CEV_X_Outside — number of black pixels outside the X-tolerance outer boundary;
c) CEV_Y_Inside — number of white pixels inside the Y-tolerance inner boundary;
d) CEV_Y_Outside — umber of black pixels outside the Y-tolerance outer boundary;
e) Y_Boundary_Area — total number of image pixels of any color inside the Y-tolerance outer
boundary;
f) Y_Inside_Total — total number of image pixels of any color inside the Y-tolerance inner boundary;
g) Character_Region_Total — total number of image pixels in the rectangular area defined by
the Y-tolerance outer template of the chosen character plus a one-stroke width boundary. The
Character Region Total is computed as the product of the width of the Y-tolerance outer boundary
plus two times the nominal stroke width rounded to the nearest number of pixels and the height of
the Y-tolerance outer boundary plus the two times the nominal stroke width rounded to the nearest
number of pixels.
From these measurements, compute the following graded parameters:
a) Character_Inside_Fit = CEV_Y_Inside / Y_Inside_Total
b) Character_Outside_Fit = CEV_Y_Outside / (Character_Region_Total − Y_Boundary_Area)
Grade the total results as follows.
a) Character_Inside_Fit > 10 % need attention
b) Character_Inside_Fit > 20 % not recommended
© ISO/IEC 2016 – All rights reserved 11

c) Character_Outside_Fit> 1 % need attention
d) Character_Outside_Fit > 2 % not recommended
The character with the lowest grade determines the character evaluation value (CEV) Grade of the
entire MRZ.
NOTE CEV_X_Inside and CEV_X_Outside are useful for process control.
See Annex C for the OCR reference decode algorithm. See Annex D for an example calculation of
character evaluation value (CEV).
6.6.5 Background noise
Find the background noise, N.
Measure the maximum and minimum reflectance levels inside a rectangular area equal to the
height and width of a character (1,4 mm × 2,4 mm) centred vertically between the two lines of text
in the MRZ starting at the left end of the MRZ. Calculate N (noise) as the difference of the maximum
reflectance minus the minimum reflectance all divided by the maximum reflectance within the box
[(Rmax − Rmin) / Rmax] and save. Move the box a half character width and repeat. Continue to the end
of the MRZ. Find the largest N.
It is recommended that the backgro
...