SIST ISO 21248:2019

INTERNATIONAL ISO
STANDARD 21248
First edition
2019-03
Information and documentation —
Quality assessment for national
libraries
Information et documentation — Evaluation de qualité pour les
bibliothèques nationales
Reference number
ISO 21248:2019(E)
©
ISO 2019

---------------------- Page: 1 ----------------------
ISO 21248:2019(E)

COPYRIGHT PROTECTED DOCUMENT
© ISO 2019
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting
on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address
below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Fax: +41 22 749 09 47
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii © ISO 2019 – All rights reserved

---------------------- Page: 2 ----------------------
ISO 21248:2019(E)

Contents Page
Foreword .v
Introduction .vi
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 Quality assessment in national libraries .17
4.1 Mission and functions of national libraries .17
4.1.1 General.17
4.1.2 Mission statement for national libraries .18
4.1.3 Core functions of national libraries .18
4.1.4 Additional functions of national libraries .19
4.2 Target groups of national libraries .20
4.3 Quality criteria in national libraries .21
4.3.1 Quality criteria that apply to all types of libraries .21
4.3.2 Specific quality criteria for national libraries .21
4.4 Methods for quality assessment in national libraries .21
5 Performance indicators for national libraries .22
5.1 General .22
5.2 Use of performance indicators .22
5.2.1 General.22
5.2.2 Selection of performance indicators .22
5.2.3 Limitations of performance measurement .23
5.3 Criteria for performance indicators .23
5.4 Descriptive framework .24
5.4.1 General.24
5.4.2 Name .24
5.4.3 Background.24
5.4.4 Objective of the performance indicator .24
5.4.5 Definition of the performance indicator .24
5.4.6 Method(s).24
5.4.7 Interpretation and use of results .25
5.4.8 Source(s) .25
5.4.9 Examples and further reading (optional).25
5.5 List of performance indicators for national libraries .25
6 Impact assessment in national libraries .27
6.1 Overview .27
6.2 Definition and description of library impact .27
6.2.1 General.27
6.2.2 Definition of library impact .27
6.2.3 Effects of library impact .27
6.2.4 Challenges in assessing impact .30
6.2.5 Use of impact assessment results . .30
6.3 Methods for assessing impact of national libraries .31
6.3.1 General.31
6.3.2 Inferred evidence .31
6.3.3 Observed evidence .33
6.3.4 Solicited evidence .34
6.3.5 Combined methods .36
6.4 Assessing the economic value of national libraries .37
6.4.1 General.37
6.4.2 Calculating the value of library benefits to users .37
6.4.3 Economic impact analysis .40
© ISO 2019 – All rights reserved iii

---------------------- Page: 3 ----------------------
ISO 21248:2019(E)

6.5 Assessing impact on a specified target population .40
6.5.1 General.40
6.5.2 Assessing impact on researchers .40
6.5.3 Assessing impact on educators and learners .41
6.5.4 Assessing impact on the general public .43
6.5.5 Assessing impact on the library and information network .44
6.5.6 Assessing impact on publishers and authors .45
6.5.7 Assessing impact on public administration .46
6.5.8 Assessing impact on business .47
Annex A (normative) Description of performance indicators .50
Annex B (informative) Examples of impact surveys .123
Annex C (informative) Performance indicators in the structure of the balanced scorecard .133
Bibliography .135
iv © ISO 2019 – All rights reserved

---------------------- Page: 4 ----------------------
ISO 21248:2019(E)

Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards
bodies (ISO member bodies). The work of preparing International Standards is normally carried out
through ISO technical committees. Each member body interested in a subject for which a technical
committee has been established has the right to be represented on that committee. International
organizations, governmental and non-governmental, in liaison with ISO, also take part in the work.
ISO collaborates closely with the International Electrotechnical Commission (IEC) on all matters of
electrotechnical standardization.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for the
different types of ISO documents should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www .iso .org/directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject of
patent rights. ISO shall not be held responsible for identifying any or all such patent rights. Details of
any patent rights identified during the development of the document will be in the Introduction and/or
on the ISO list of patent declarations received (see www .iso .org/patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to the
World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see www .iso
.org/iso/foreword .html.
This document was prepared by Technical Committee ISO/TC 46, Information and documentation,
Subcommittee SC 8, Quality — Statistics and performance evaluation.
Any feedback or questions on this document should be directed to the user’s national standards body. A
complete listing of these bodies can be found at www .iso .org/members .html.
© ISO 2019 – All rights reserved v

---------------------- Page: 5 ----------------------
ISO 21248:2019(E)

Introduction
This document is concerned with the evaluation of national libraries.
It provides guidance on the use of performance indicators regarding the quality of services in national
libraries and on methods for assessing the impact and value of national libraries.
Clause 5 resumes the work done in ISO/TR 28118:2009 where, for the first time, a specific evaluation
method was described for national libraries. This document establishes a set of performance indicators
that are adapted to the mission and functions of national libraries. Annex A specifies how the indicators
are intended to be calculated and used.
In Clause 6, this document describes methods for identifying and proving the impact of national
libraries on individuals, institutions and on society. The methods described in Clause 6 do not reflect all
possible methods or evaluation techniques, but are those seen to be most effective for assessing impact
of national libraries. Annex B gives examples of impact surveys, considering different user groups of
national libraries.
The texts in this document are partly based on ISO 11620 and ISO 16439.
vi © ISO 2019 – All rights reserved

---------------------- Page: 6 ----------------------
INTERNATIONAL STANDARD ISO 21248:2019(E)
Information and documentation — Quality assessment for
national libraries
1 Scope
This document defines terms for the quality assessment of national libraries and specifies the following
methods for the assessment:
— performance measurement, and
— impact assessment.
The results of both methods are of special interest for comparison over time within the same library.
Comparisons between libraries are possible if differences in the mandate, tasks and constituencies of
the libraries are taken into account.
Not all methods described in this document apply to all national libraries. Limitations of the applicability
of individual methods are specified in the descriptions.
This document is not intended to exclude the use of performance indicators or of methods for impact
assessment not specified in it.
This document does not cover web archiving, but refers to ISO/TR 14873 for statistics and quality
issues for this new task of national libraries.
2 Normative references
There are no normative references in this document.
3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— ISO Online browsing platform: available at https: //www .iso .org/obp
— IEC Electropedia: available at http: //www .electropedia .org/
3.1
access
successful request of a library-provided online service
Note 1 to entry: An access is one cycle of user activities that typically starts when a user connects to a library-
provided online service and ends by a terminating activity that is either explicit (by leaving the database through
log-out or exit) or implicit (timeout due to user inactivity).
Note 2 to entry: Accesses to the library website (3.46) are counted as virtual visits (3.97).
Note 3 to entry: Requests of a general entrance or gateway page should be excluded.
Note 4 to entry: If possible, requests by search engines should be excluded.
[SOURCE: ISO 2789:2013, 2.2.1]
© ISO 2019 – All rights reserved 1

---------------------- Page: 7 ----------------------
ISO 21248:2019(E)

3.2
accessibility
ease of reaching and using a service or facility
[SOURCE: ISO 11620:2014, 2.2]
3.3
anecdotal evidence
evidence based on anecdotes or stories, obtained informally from personal observations and
experiences, not collected systematically nor empirically tested
[SOURCE: ISO 16439:2014, 3.5]
3.4
availability
degree to which content, documents, facilities or services are actually provided by the library at the
time required by users
[SOURCE: ISO 11620:2014, 2.6]
3.5
benefit
helpful or good effect, or something intended to help
[SOURCE: ISO 16439:2014, 3.7]
3.6
book
non-serial printed document in codex form
[SOURCE: ISO 2789:2013, 2.3.4]
3.7
capital expenditure
expenditure which results in the acquisition of, or addition to, fixed assets
Note 1 to entry: This includes expenditure on building sites, new buildings and extensions, furnishings and
equipment for new and expanded buildings, computer systems (hardware and software), etc. When applicable,
local and national sales/purchase taxes [e.g. value added tax (VAT)] are included.
[SOURCE: ISO 2789:2013, 2.6.1]
3.8
closed stacks
stacks that are not accessible to users
Note 1 to entry: Antonym: open stacks (3.63).
[SOURCE: ISO/TR 11219:2012, 2.19]
3.9
commercial publication
document in print or electronic format that is generally produced, distributed and sold for the purpose
of profit
Note 1 to entry: Excludes publications by non-profit organizations.
[SOURCE: ISO 2789:2013, 2.3.7, modified — Note 1 to entry has been added.]
2 © ISO 2019 – All rights reserved

---------------------- Page: 8 ----------------------
ISO 21248:2019(E)

3.10
conservation
preservation measures and actions applied to prevent, arrest or delay deterioration of a document or
other material object
[SOURCE: ISO 5127:2017, 3.12.1.01]
3.11
content unit
computer-processed uniquely identifiable textual or audio-visual piece of published work that may be
original or a digest of other published work
Note 1 to entry: This includes documents or parts of documents (e.g. articles, abstracts, content tables, images)
and descriptive records.
Note 2 to entry: Adapted from COUNTER code of practice, Release 3:2008.
Note 3 to entry: PDF, Postscript, HTML and other formats of the same content unit will be counted as separate items.
[SOURCE: ISO 2789:2013, 2.3.9]
3.12
contingent valuation
method for assessing the economic value of non-profit institutions and projects of public utility
depending on potential users’ responses to survey questions such as what they are willing to pay for a
benefit (3.5) or feature (willingness-to-pay), or what they would accept as financial compensation if a
certain benefit or feature was missing (willingness-to-accept)
Note 1 to entry: The method is used in environmental protection and health care, for example.
[SOURCE: ISO 16439:2014, 3.9]
3.13
cultural economics
branch of economics that studies the relation of culture to economic outcomes and explores cultural
phenomena as economic factors
[SOURCE: ISO 16439:2014, 3.12]
3.14
data mining
computational process that identifies patterns by analysing quantitative data (3.73) from different
perspectives and dimensions, categorizing it, and summarizing potential relationships and impacts
[SOURCE: ISO 16439:2014, 3.13]
3.15
digital document
information unit with a defined content, born digital or digitized, that has been created or digitized by
the library or acquired in digital form as part of the library collection (3.43)
Note 1 to entry: This includes eBooks, electronic patents, networked audio-visual documents and other digital
documents, e.g. reports, cartographic and music documents, preprints, etc. Databases and electronic serials are
excluded.
Note 2 to entry: A digital document can be structured into one or more files.
Note 3 to entry: A digital document consists of one or more content units (3.11). Before digitization (3.16), the
library must decide which content units should be searchable afterwards, e.g. articles in serials, or songs on
records.
[SOURCE: ISO 2789:2013, 2.3.13]
© ISO 2019 – All rights reserved 3

---------------------- Page: 9 ----------------------
ISO 21248:2019(E)

3.16
digitization
process of converting analogue materials into digital form
Note 1 to entry: Digitization for document supply from the library collection (3.43)to a user or institution is
excluded.
Note 2 to entry: Digitization for preservation purposes is included.
Note 3 to entry: Mass digitization is included.
Note 4 to entry: Purchase of electronic copies for replacing print copies is excluded.
[SOURCE: ISO 2789:2013, 2.3.15]
3.17
document
recorded information or material object which can be treated as a unit in a documentation process
Note 1 to entry: Documents can differ in form and characteristics.
[SOURCE: ISO 5127:2017, 3.1.1.38, modified — Note 1 to entry of ISO 2789:2013, 2.3.17 has been
included.]
3.18
document processing
everything done to a document after it is acquired by a library, before it is placed on the shelves, including
acquisition, descriptive cataloguing, subject indexing, if applicable binding, physical processing, and
shelving
Note 1 to entry: For electronic documents, the end of processing will either coincide with cataloguing when
the URL for a document on an external server has been added, or with the installation of the document on the
library’s server and the URL being added in the catalogue.
[SOURCE: ISO/TR 28118:2009, 2.13, modified — Note 1 to entry has been added.]
3.19
document retrieval
process of recovering specific documents from a storage (3.87) place
[SOURCE: ISO 5127:2017, 3.10.1.03, modified — The term “placement” has been excluded in the
definition.]
3.20
economic impact
effect of a policy, decision, institution, or event on the economy of a given area
Note 1 to entry: Economic impact is usually measured in terms of changes in economic growth (output or value
added) and associated changes in jobs (employment) and income (wages).
[SOURCE: ISO 16439:2014, 3.16, modified — The term “decision” has been added while “programme”
has been excluded in the definition.]
3.21
economic impact of libraries
positive influence of libraries on the local, regional or national economic life
4 © ISO 2019 – All rights reserved

---------------------- Page: 10 ----------------------
ISO 21248:2019(E)

3.22
educational services
learning sessions and learning materials and programmes in all formats for children and adults for the
purpose of enhancing skills in library and information use
Note 1 to entry: This includes the provision of services for schools and the cooperation with other libraries in
preparing and offering educational services.
Note 2 to entry: Education of librarians is excluded.
3.23
effectiveness
measure of the degree to which given objectives are achieved
Note 1 to entry: An activity is effective if it maximizes the results it was established to produce.
[SOURCE: ISO 11620:2014, 2.15]
3.24
efficiency
measure of the utilisation of resources to realize a given objective
Note 1 to entry: An activity is efficient if it minimizes the use of resources, or produces better performance with
the same resources.
[SOURCE: ISO 11620:2014, 2.16]
3.25
electronic collection
digital collection
all resources in electronic form in the library collection (3.43), whether born digital or digitized
Note 1 to entry: The electronic collection includes databases, electronic serials, and digital documents (3.15). Free
Internet resources (3.31) which have been catalogued by the library in its online catalogue (3.61) or a database are
excluded.
Note 2 to entry: The resources can be networked, installed on stand-alone workstations or stored on physical
carriers.
[SOURCE: ISO 2789:2013, 2.3.21, modified — The term “digital collection” has been added as an
accepted term.]
3.26
electronic document delivery
electronic transmission of a document or part of a document from the library collection
(3.43) to a user, mediated by library staff not necessarily via another library
Note 1 to entry: Electronic transmission of documents to members of the population to be served (3.67) is
included. Fax transmission is excluded.
Note 2 to entry: May be split up as to transmission with or without charge to the user.
Note 3 to entry: Unmediated downloading by users from the electronic collection (3.25) of the library is excluded.
[SOURCE: ISO 2789:2013, 2.2.7]
3.27
electronic service
library service delivered via electronic means whether from local servers or provided via networks
Note 1 to entry: Electronic library services include the online catalogue (3.61), the library website (3.46), the
electronic collection (3.25), electronic lending, electronic document delivery (mediated), electronic reference
service (3.77), user training (3.94) by electronic means, services for mobile devices (3.51), services for interactive
use (including services on social networks), and Internet access offered via the library.
© ISO 2019 – All rights reserved 5

---------------------- Page: 11 ----------------------
ISO 21248:2019(E)

Note 2 to entry: This does not include booking physical services (e.g. rooms or library tours) by electronic means.
[SOURCE: ISO 2789:2013, 2.2.8]
3.28
evaluation
process of estimating the effectiveness, efficiency, utility and relevance of a service or facility
[SOURCE: ISO 11620:2014, 3.19]
3.29
event
pre-arranged activity with cultural, educational, social, political, scholarly, or other intent
EXAMPLE Exhibitions, author visits, literary discussions, workshops, etc.
Note 1 to entry: Only events arranged by the library on its own or in partnership with other institutions are
included, whether inside or outside the library premises. Events inside the library premises organized by
institutions outside the library without the library’s cooperation are excluded.
Note 2 to entry: User training (3.94) lessons and library tours are excluded.
Note 3 to entry: Ongoing programmes are included. Each session of a programme is counted as one event.
Note 4 to entry: Virtual events are included.
[SOURCE: ISO 2789:2013, 2.2.9]
3.30
focus group interview
focus group discussion
group interview in the form of a moderated discussion among a small number of selected individuals on
topics introduced by the moderator
[SOURCE: ISO 16439:2014, 3.23]
3.31
free Internet resource
Internet resource with unrestricted (open) access for which no payment is required
[SOURCE: ISO 2789:2013, 2.3.23]
3.32
full-time equivalent
FTE
measurement equal to one staff person working a full-time work schedule for one year
Note 1 to entry: For example, if out of three persons employed as librarians, one works quarter-time, one works half-
time, and one works full-time, then the FTE of these three persons would be 0,25 + 0,5 + 1,0 = 1,75 librarians (FTE).
Note 2 to entry: Not all libraries may use the same number of hours per year to determine an FTE. Thus, any
comparative measures between libraries may need to consider any differences in hours.
[SOURCE: ISO 11620:2014, 2.23]
3.33
goal
desired state of affairs to be achieved by the implementation of agreed policies
[SOURCE: ISO 11620:2014, 2.24]
6 © ISO 2019 – All rights reserved

---------------------- Page: 12 ----------------------
ISO 21248:2019(E)

3.34
homepage
main or opening screen of a website to which all other pages for users are linked
[SOURCE: ISO 5127:2017, 3.3.3.26]
3.35
impact
difference or change in an individual or group resulting from the contact with library services
Note 1 to entry: The change can be tangible or intangible.
[SOURCE: ISO 16439:2014, 3.25]
3.36
income generated
income generated by library operations and fro
...