ISO/TR 20983:2003

TECHNICAL ISO/TR
REPORT 20983
First edition
2003-11-01

Information and documentation —
Performance indicators for electronic
library services
Information et documentation — Indicateurs de performance pour les
services électroniques des bibliothèques




Reference number
ISO/TR 20983:2003(E)
©
ISO 2003

---------------------- Page: 1 ----------------------

ISO/TR 20983:2003(E)
PDF disclaimer
This PDF file may contain embedded typefaces. In accordance with Adobe's licensing policy, this file may be printed or viewed but
shall not be edited unless the typefaces which are embedded are licensed to and installed on the computer performing the editing. In
downloading this file, parties accept therein the responsibility of not infringing Adobe's licensing policy. The ISO Central Secretariat
accepts no liability in this area.
Adobe is a trademark of Adobe Systems Incorporated.
Details of the software products used to create this PDF file can be found in the General Info relative to the file; the PDF-creation
parameters were optimized for printing. Every care has been taken to ensure that the file is suitable for use by ISO member bodies. In
the unlikely event that a problem relating to it is found, please inform the Central Secretariat at the address given below.


©  ISO 2003
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized in any form or by any means,
electronic or mechanical, including photocopying and microfilm, without permission in writing from either ISO at the address below or
ISO's member body in the country of the requester.
ISO copyright office
Case postale 56 • CH-1211 Geneva 20
Tel. + 41 22 749 01 11
Fax + 41 22 749 09 47
E-mail copyright@iso.org
Web www.iso.org
Published in Switzerland

ii © ISO 2003 — All rights reserved

---------------------- Page: 2 ----------------------

ISO/TR 20983:2003(E)
Contents Page
Foreword. iv
Introduction . v
1 Scope. 1
2 Normative references . 1
3 Terms and definitions. 1
4 Relationships with other indicators . 4
4.1 The networked environment . 4
4.2 Categories of performance indicators. 5
4.3 The relevance of ISO 11620 to electronic library services . 5
4.3.1 Indicators that apply equally to traditional and electronic library services . 5
4.3.2 Indicators that apply only to traditional library services. 6
4.3.3 Indicators for traditional library services where analogous indicators can be found for
electronic library services. 7
5 Use of performance indicators . 7
5.1 Purposes. 7
5.2 Selection of performance indicators. 8
5.3 Limitations . 8
5.3.1 Measuring the impact of electronic library services. 8
5.3.2 Degree of accuracy. 8
5.3.3 Comparability of performance indicator data . 9
5.3.4 Indicators of temporary importance . 9
Annex A List of performance indicators for electronic library services . 10
Annex B Descriptions of performance indicators . 12
Bibliography . 31
Alphabetical index . 32

© ISO 2003 — All rights reserved iii

---------------------- Page: 3 ----------------------

ISO/TR 20983:2003(E)
Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards bodies
(ISO member bodies). The work of preparing International Standards is normally carried out through ISO
technical committees. Each member body interested in a subject for which a technical committee has been
established has the right to be represented on that committee. International organizations, governmental and
non-governmental, in liaison with ISO, also take part in the work. ISO collaborates closely with the
International Electrotechnical Commission (IEC) on all matters of electrotechnical standardization.
International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2.
The main task of technical committees is to prepare International Standards. Draft International Standards
adopted by the technical committees are circulated to the member bodies for voting. Publication as an
International Standard requires approval by at least 75 % of the member bodies casting a vote.
In exceptional circumstances, when a technical committee has collected data of a different kind from that
which is normally published as an International Standard (“state of the art”, for example), it may decide by a
simple majority vote of its participating members to publish a Technical Report. A Technical Report is entirely
informative in nature and does not have to be reviewed until the data it provides are considered to be no
longer valid or useful.
Attention is drawn to the possibility that some of the elements of this document may be the subject of patent
rights. ISO shall not be held responsible for identifying any or all such patent rights.
ISO/TR 20983 was prepared by Technical Committee ISO/TC 46, Information and documentation,
Subcommittee SC 8, Quality — Statistics and performance evaluation.
iv © ISO 2003 — All rights reserved

---------------------- Page: 4 ----------------------

ISO/TR 20983:2003(E)
Introduction
This Technical Report is concerned with the evaluation of electronic library services.
The successful provision of these services depends on close collaboration between publishers, network
providers, and librarians. Their needs for data about various aspects of performance are different, and the
ability to collect data may not rest with the person most interested in that data. For example, the publisher may
be best placed to collect data on the usage of a particular service; whereas the librarian may have most
interest in using that data in developing and evaluating the service.
The main purpose of this Technical Report is to spread knowledge about the practice of evaluating electronic
library services. Although electronic library services have been developed over more than thirty years, the
technical environment has changed significantly in the last ten years. The need for performance indicators for
electronic services was acknowledged in ISO 11620: 1998 Information and documentation - Library
performance indicators. It was also acknowledged that at the time of publication of that International Standard,
there were no indicators that had been tested and documented, or that were in widespread use. Although
there has been significant progress in the last few years on the definition, testing and documentation of
indicators, it is still too soon to determine which indicators will be most generally useful, or become widely
used. This Technical Report provides a standardized terminology and concise definitions and descriptions of a
selection of performance indicators, in the same format as that used in ISO 11620. Publications listed in the
Bibliography [2,3,5] provide more detailed information on data collection and analysis.
Only a small number of the indicators presented here are indicators of service quality: work on this area is not
yet mature enough to be included. The indicators that are included have been tested in libraries, or are simple
adaptations of similar indicators. In each case they have been judged against the criteria established in
ISO 11620. That is, each indicator is judged to have informative content, to be reliable, valid, appropriate,
practical, and in some circumstances may be used for comparative purposes.
The indicators presented in this Report are a representative selection of those which have been tested. A
further selection would have taken longer to prepare, and to publish. Developments in the measurement and
evaluation of electronic library services will be monitored by a Working Group, which will propose additional
indicators, and modifications or adaptations to the indicators here as they are tested and validated. This
Technical Report should be considered as work in progress towards the development of an International
Standard. It is intended that, where appropriate, the contents of this Technical Report will be incorporated in a
future revision of ISO 11620.

© ISO 2003 — All rights reserved v

---------------------- Page: 5 ----------------------

TECHNICAL REPORT ISO/TR 20983:2003(E)

Information and documentation — Performance indicators for
electronic library services
1 Scope
This Technical Report is applicable to all types of libraries in all countries. Limitations on the applicability of
individual performance indicators are listed in the scope clause of the description of each indicator (see
Annex B).
Indicators may be used for comparison over time for the services provided by the same library. Comparisons
between libraries and services should only be made with caution, taking into account differences in the
constituencies of the libraries, with good understanding of the indicators used, and careful interpretation of the
data.
This Technical Report does not include indicators for the evaluation of the impact of libraries either on
individuals or on society.
Throughout the text, the names of indicators are printed with initial capitals for significant words, e.g.
Workstation Use Rate. This helps to distinguish the names from supporting text.
2 Normative references
The following referenced documents are indispensable for the application of this document. For dated
references, only the edition cited applies. For undated references, the latest edition of the referenced
document (including any amendments) applies.
ISO 2789:2003, Information and documentation — International library statistics
ISO 5127:2001, Information and documentation — Vocabulary
ISO 11620:1998, Information and documentation — Library performance indicators
ISO 11620:1998/Amd. 1:2003, Information and documentation — Library performance indicators —
Amendment 1: Additional performance indicators for libraries
3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
3.1
database
collection of electronically stored data or unit records (facts, bibliographic data, texts) with a common user
interface and software for the retrieval and manipulation of the data [ISO 2789]
NOTE 1 The data or records are usually collected with a particular intent and are related to a defined topic. A database
may be issued on CD-ROM, diskette, or other direct access method, or as a computer file accessed via dial-up methods
or via the Internet.
© ISO 2003 — All rights reserved 1

---------------------- Page: 6 ----------------------

ISO/TR 20983:2003(E)
NOTE 2 Licensed databases are counted separately even if access to several licensed database products is effected
through the same interface.
3.2
digital document
information unit with a defined content that has been digitized by the library or acquired in digital form as part
of the library collection [ISO 2789]
NOTE 1 This includes eBooks, electronic patents, networked audiovisual documents and other digital documents, e.g.
reports, cartographic and music documents, pre-prints etc. Databases and electronic serials are excluded.
NOTE 2 Items incorporated in databases are covered by 3.1.
NOTE 3 A digital document can be structured into one or more files.
3.3
document
recorded information or material object which can be treated as a unit in a documentation process [ISO 5127]
NOTE Documents may differ in their physical form or characteristics.
3.4
document downloaded
full text of a document or part of a document in the electronic collection that is delivered to a user [ISO 2789]
3.5
electronic collection
all resources in electronic form in the library collection [ISO 2789]
NOTE The electronic collection includes databases, electronic serials, and digital documents. Free Internet resources
which have been catalogued by the library in its OPAC or a database should be counted separately.
3.6
electronic resource
a document in electronic form which forms a separate item with a distinctive title, whether issued in one or
several units
NOTE This includes electronic books, serials, maps, but not databases.
3.7
electronic service
electronic library service which is either supplied from local servers or accessible via networks [ISO 2789]
NOTE Electronic library services comprise the OPAC, the library web site, the electronic collection, electronic
document delivery (mediated), electronic reference service, user training on electronic services and Internet access
offered via the library.
3.8
entry
a unit of bibliographic or factual information or multimedia objects included in a searchable database
3.9
information request
information contact that involves the knowledge or use of one or more information sources (such as printed
and non-printed materials, machine-readable databases, the library’s own and other institutions’ catalogues)
by library staff [ISO 2789]
NOTE 1 May also involve recommendations, interpretation or instruction in the use of such sources.
NOTE 2 The request can be delivered personally or by means of telephone, regular mail, fax or electronic media (via e
mail, the library web site, instant messaging or other networked communications mechanisms).
2 © ISO 2003 — All rights reserved

---------------------- Page: 7 ----------------------

ISO/TR 20983:2003(E)
NOTE 3 It is essential that libraries do not include directional and administrative inquiries, eg for locating staff or
facilities, regarding opening times or about handling equipment such as reader printers or computer terminals.
NOTE 4 Inquiries are also excluded if library staff simply help the user to locate items of stock that have already been
identified bibliographically.
3.10
library collection
all documents provided by a library for its users [ISO 2789]
NOTE 1 Comprises documents held locally and remote resources for which permanent or temporary access rights
have been acquired.
NOTE 2 Access rights may be acquired by the library itself, by a consortium and/or through external funding.
NOTE 3 Acquisition is to be understood as deliberately selecting a document, securing access rights and including it in
the OPAC or other databases of the library. Interlibrary lending and document delivery and excluded.
NOTE 4 Does not include links to Internet resources for which the library has not secured access rights by legal
agreements (eg legal deposit right), license or other contractual and/or co-operative agreement. Free Internet resources
which have been catalogued by the library in its OPAC or a database should be counted separately.
3.11
population to be served
number of individuals for whom the library is set up to provide its services and materials [ISO 11620]
NOTE For public libraries this will normally be the population of the legal service area (authority); for libraries of an
institution of higher education this will normally be the total of academic and professional staff plus students.
3.12
rejected session (turnaway)
unsuccessful request of a database or the OPAC because of requests exceeding simultaneous user limit
[ISO 2789]
NOTE Rejection through entry of wrong passwords is excluded.
3.13
remote session
a successful request of a database or the OPAC established from outside the library building (adapted from
[1]
EQUINOX )
3.14
session
successful request of a database or the OPAC [ISO 2789]
NOTE 1 A session is one cycle of user activities that typically starts when a user connects to a database or the OPAC
and ends with explicit (by leaving the database through log-out or exit) or implicit (timeout due to user inactivity)
termination of activities in the database. The average timeout period would be 30 minutes. If another time period is used
this should be reported.
NOTE 2 Sessions on the library web site are counted as virtual visits.
NOTE 3 Requests of a general entrance or gateway page should be excluded.
NOTE 4 If possible, requests by search engine should be excluded.
3.15
user training
training programme set up with a specified lesson plan, which aims at specific learning outcomes for the use
of library services [ISO 2789]
© ISO 2003 — All rights reserved 3

---------------------- Page: 8 ----------------------

ISO/TR 20983:2003(E)
NOTE 1 User training can be offered as a tour of the library, as library tuition, or as a web based service for users.
NOTE 2 The duration of a lesson is irrelevant.
3.16
virtual visit
a user's request of the library web site from outside the library premises regardless of the number of pages or
elements viewed [ISO 2789]
3.17
web site
electronic service that has a unique domain on the Internet and consists of a collection of digital documents
[ISO 2789]
NOTE 1 The pages of a web site are usually interconnected by the use of hypertext links.
NOTE 2 Excludes the documents that fit the definitions of electronic collection and external Internet resources that may
be linked from the library web site.
3.18
workstation
computer that may stand alone or be networked, or a dumb terminal [ISO 2789]
4 Relationships with other indicators
4.1 The networked environment
The networked environment provides a different frame of reference for the provision of services: the
boundaries between internal and external provision are different. For example, consider document delivery. In
traditional library services this is represented by the interlending of physical documents between libraries. In
the electronic world, document delivery is accomplished using a network. The network may be provided by a
variety of external providers. Special equipment has to be installed in the libraries at both ends of document
delivery process and staff have to be trained to use it. The suppliers of documents may be publishers or
libraries; the end users may be directly in communication with the suppliers.
Again, the provision of equipment within the organisation of which the library forms a part, may be outside the
library's own control: however, the library’s performance may be judged on its service delivery even though it
does not control some of the infrastructure essential to the delivery of the service.
It is because the boundaries within which electronic services operate differ from those in which traditional
library services operate that comparisons between traditional and electronic library services are difficult, if not
impossible. Since the extent to which different libraries have adopted electronic library services will differ then
comparisons between libraries are increasingly difficult. The position is further complicated by the changes in
expectations, experience and behaviour of users and of publishers. New generations of users will increasingly,
at least in the developed world, be familiar with the use of computers and Internet applications generally and
will expect electronic services as a matter of course. There will, however be for some considerable time
significant numbers of older users who are less familiar with the concepts and practice of computer usage and
will need careful support if they are not to be disadvantaged.
On the other hand, publishers are operating in a variety of ways to deal with the possibilities of electronic
publishing. Long established practices in the supply of printed materials do not operate in the same way when
original works are often generated in electronic format. The economics of publication in electronic formats are
significantly different from those in printed formats and different charging models are being tried. The
preferred format varies for electronic publications between types of publication, and may also vary from year
to year; and the financial arrangements between library and publisher also vary between publications and
from year to year.
4 © ISO 2003 — All rights reserved

---------------------- Page: 9 ----------------------

ISO/TR 20983:2003(E)
Furthermore, the ability to measure the usage of publications in electronic format differs radically from that
associated with print formats. It may be possible, although expensive, for a library to account for every use
made of a printed document within a library: it may well be impossible for the library to count every use made
of electronic documents. The publisher may be able count use made of the electronic format, while having no
idea of the use made of printed documents. Effective performance measurement thus depends on new
arrangements and collaboration between the copyright owners, publishers, agents, librarians and users.
A final point relates to ‘Per capita’ indicators. Some electronic library services (web-sites, OPACs, ‘free’
information services) can be accessed by anyone, and are not limited to the population to be served. In these
cases, usage per capita ratios would be artificially inflated, if the data were collected automatically. If a library
is interested in knowing what proportion of its population to be served is using its services remotely, those
data could be collected by a conventional survey technique.
4.2 Categories of performance indicators
In defining performance indicators for electronic library services it is natural to start by seeking analogies with
performance indicators for traditional library services. Reference to ISO 11620:1998 and its amendment
ISO 11620/Amd. 1 show a list of 34 indicators. These are grouped in a number of categories according to
Table 1.
Table 1
Service, activity or aspect measured Sub-category
User perception General
Public services General
Providing documents
Retrieving documents
Lending documents
Document delivery from external sources
Enquiry and reference services
Information searching
User Education
Facilities
Technical services Acquiring documents
Processing documents
Cataloguing
Promotion of services
User services Deployment of staff

In drawing up this technical report it has proved possible to use the same categories for the proposed
performance indicators with only minor modifications (see Annex A). The applicability of the indicators in
ISO 11620 is discussed in the following subclause.
4.3 The relevance of ISO 11620 to electronic library services
4.3.1 Indicators that apply equally to traditional and electronic library services
Some indicators are the same for both traditional and electronic library services. An obvious example is User
Satisfaction. Others need only very slight changes to existing definitions and descriptions of methods to make
it clear that they are relevant to both types of service: an example is Cost per Title Catalogued. Similarly, the
group of indicators relating to providing documents: here the definitions and descriptions would need to be
modified to include documents in electronic formats. There are other indicators that have analogues in the
networked environment (see 4.3.3), but more substantial revision would be needed to accommodate different
formats, and different bases for measurement. See Table 2.
© ISO 2003 — All rights reserved 5

---------------------- Page: 10 ----------------------

ISO/TR 20983:2003(E)
Table 2
Service, activity or aspect Reference in
Performance indicator
measured ISO 11620
User perception: General User Satisfaction B.1.1.1
Public services: Providing Titles Availability B.2.2.1
documents
Required Titles Availability B.2.2.2
B.2.2.3
Percentage of Required Titles in the
Collection
Required Titles Extended Availability B.2.2.4
Public services: Enquiry and Correct Answer Fill Rate B.2.6.1
reference services
Title Catalogue Search Success Rate B.2.7.1
Public services: Information
searching
Subject Catalogue Search Success Rate B.2.7.2
Public services: Facilities Facilities Availability B.2.9.1
Facilities Use Rate B.2.9.2
Seat Occupancy Rate B.2.9.3
Automated Systems Availability B.2.9.4
Median Time of Document Acquisition B.3.1.1
Technical services: Acquiring
documents
Technical services: Cataloguing Cost per Title Catalogued B.3.3.1
User services; Deployment of staff User Services Staff per Capita B.5.1.1
User Services Staff as a Percentage of B.5.1.2
Total Staff

4.3.2 Indicators that apply only to traditional library services
The indicators in this group are so closely related to the physical format of documents that they have no
precise equivalent in the electronic world. See Table 3.
Table 3
Service, activity or aspect Reference in
Performance indicator
measured ISO 11620
Public services: Providing In-library Use per Capita B.2.2.5
documents
Document Use Rate B.2.2.6
Proportion of Stock not Used B.2.2.7
Shelving Accuracy B.2.2.8
Public services: Retrieving Median Time of Document Retrieval from B.2.3.1
documents Closed Stacks
Median Time of Document Retrieval from B.2.3.2
Open Access Areas
Public services: Document delivery Speed of Interlibrary Lending B.2.5.1
from external sources
Technical services: Processing Median Time of Document Processing B.3.2.1
documents

6 © ISO 2003 — All rights reserved

---------------------- Page: 11 ----------------------

ISO/TR 20983:2003(E)
4.3.3 Indicators for traditional library services where analogous indicators can be found for
electronic library services
In this group, the indicators for electronic services are based on the same concepts as those for traditional
services, but the basis for measurement is different. For example, Library Visits per Capita would be replaced
by an indicator relating virtual visits to the population to be served. The various indicators for lending
documents have analogues now that digital lending is possible.
The fact that analogues can be found suggests that, in a future revision of ISO 11620, it may be necessary to
devise composite indicators that integrate traditional and networked services. For example, the indicator
Library Visits per Capita could be redefined to include both physical and virtual visits. Two composite
indicators are included in this Technical Report: Percentage of Virtual Visits to Total Visits, and Percentage of
Information Requests Submitted Electronically. Librarians may wish to create other composite measures and
indicators if that proves helpful in their own circumstances. See Table 4.
Table 4
Service, activity or aspect Reference in
Performance indicator
measured ISO 11620
Public services: General Percentage of Target Population Reached B.2.1.1
Cost per User B.2.1.2
Public services: General Library Visits per Capita B.2.1.3
Cost per Library Visit B.2.1.4
Public services: Lending Collection Turnover B.2.4.1
documents
Loans per Capita B.2.4.2
Documents on Loan per Capita B.2.4.3
Cost per Loan B.2.4.4
Loans per Employee B.2.4.5
Proportion of Stock on Loan B.2.4.6

5 Use of performance indicators
5.1 Purposes
The performance indicators described in this Technical Report are used as tools to compare the effectiveness,
efficiency and quality of the library's services and products to the library's mission and goals. They can be
used for evaluation purposes in the following areas:
 comparing a single library's performance over years
 support for management decisions, e. g. reallocating resources, introducing new services, reducing or
deleting existing services
 demonstrating the library's performance and its co
...

SLOVENSKI STANDARD
SIST-TP ISO/TR 20983:2005
01-november-2005
,QIRUPDWLNDLQGRNXPHQWDFLMD±.D]DOFLXVSHãQRVWLHOHNWURQVNLKNQMLåQLþQLK
VWRULWHY
Information and documentation -- Performance indicators for electronic library services
Information et documentation -- Indicateurs de performance pour les services
électroniques des bibliothèques
Ta slovenski standard je istoveten z: ISO/TR 20983:2003
ICS:
01.140.20 Informacijske vede Information sciences
35.240.30 Uporabniške rešitve IT v IT applications in information,
informatiki, dokumentiranju in documentation and
založništvu publishing
SIST-TP ISO/TR 20983:2005 en
2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.

---------------------- Page: 1 ----------------------

SIST-TP ISO/TR 20983:2005

---------------------- Page: 2 ----------------------

SIST-TP ISO/TR 20983:2005


TECHNICAL ISO/TR
REPORT 20983
First edition
2003-11-01

Information and documentation —
Performance indicators for electronic
library services
Information et documentation — Indicateurs de performance pour les
services électroniques des bibliothèques




Reference number
ISO/TR 20983:2003(E)
©
ISO 2003

---------------------- Page: 3 ----------------------

SIST-TP ISO/TR 20983:2005
ISO/TR 20983:2003(E)
PDF disclaimer
This PDF file may contain embedded typefaces. In accordance with Adobe's licensing policy, this file may be printed or viewed but
shall not be edited unless the typefaces which are embedded are licensed to and installed on the computer performing the editing. In
downloading this file, parties accept therein the responsibility of not infringing Adobe's licensing policy. The ISO Central Secretariat
accepts no liability in this area.
Adobe is a trademark of Adobe Systems Incorporated.
Details of the software products used to create this PDF file can be found in the General Info relative to the file; the PDF-creation
parameters were optimized for printing. Every care has been taken to ensure that the file is suitable for use by ISO member bodies. In
the unlikely event that a problem relating to it is found, please inform the Central Secretariat at the address given below.


©  ISO 2003
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized in any form or by any means,
electronic or mechanical, including photocopying and microfilm, without permission in writing from either ISO at the address below or
ISO's member body in the country of the requester.
ISO copyright office
Case postale 56 • CH-1211 Geneva 20
Tel. + 41 22 749 01 11
Fax + 41 22 749 09 47
E-mail copyright@iso.org
Web www.iso.org
Published in Switzerland

ii © ISO 2003 — All rights reserved

---------------------- Page: 4 ----------------------

SIST-TP ISO/TR 20983:2005
ISO/TR 20983:2003(E)
Contents Page
Foreword. iv
Introduction . v
1 Scope. 1
2 Normative references . 1
3 Terms and definitions. 1
4 Relationships with other indicators . 4
4.1 The networked environment . 4
4.2 Categories of performance indicators. 5
4.3 The relevance of ISO 11620 to electronic library services . 5
4.3.1 Indicators that apply equally to traditional and electronic library services . 5
4.3.2 Indicators that apply only to traditional library services. 6
4.3.3 Indicators for traditional library services where analogous indicators can be found for
electronic library services. 7
5 Use of performance indicators . 7
5.1 Purposes. 7
5.2 Selection of performance indicators. 8
5.3 Limitations . 8
5.3.1 Measuring the impact of electronic library services. 8
5.3.2 Degree of accuracy. 8
5.3.3 Comparability of performance indicator data . 9
5.3.4 Indicators of temporary importance . 9
Annex A List of performance indicators for electronic library services . 10
Annex B Descriptions of performance indicators . 12
Bibliography . 31
Alphabetical index . 32

© ISO 2003 — All rights reserved iii

---------------------- Page: 5 ----------------------

SIST-TP ISO/TR 20983:2005
ISO/TR 20983:2003(E)
Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards bodies
(ISO member bodies). The work of preparing International Standards is normally carried out through ISO
technical committees. Each member body interested in a subject for which a technical committee has been
established has the right to be represented on that committee. International organizations, governmental and
non-governmental, in liaison with ISO, also take part in the work. ISO collaborates closely with the
International Electrotechnical Commission (IEC) on all matters of electrotechnical standardization.
International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2.
The main task of technical committees is to prepare International Standards. Draft International Standards
adopted by the technical committees are circulated to the member bodies for voting. Publication as an
International Standard requires approval by at least 75 % of the member bodies casting a vote.
In exceptional circumstances, when a technical committee has collected data of a different kind from that
which is normally published as an International Standard (“state of the art”, for example), it may decide by a
simple majority vote of its participating members to publish a Technical Report. A Technical Report is entirely
informative in nature and does not have to be reviewed until the data it provides are considered to be no
longer valid or useful.
Attention is drawn to the possibility that some of the elements of this document may be the subject of patent
rights. ISO shall not be held responsible for identifying any or all such patent rights.
ISO/TR 20983 was prepared by Technical Committee ISO/TC 46, Information and documentation,
Subcommittee SC 8, Quality — Statistics and performance evaluation.
iv © ISO 2003 — All rights reserved

---------------------- Page: 6 ----------------------

SIST-TP ISO/TR 20983:2005
ISO/TR 20983:2003(E)
Introduction
This Technical Report is concerned with the evaluation of electronic library services.
The successful provision of these services depends on close collaboration between publishers, network
providers, and librarians. Their needs for data about various aspects of performance are different, and the
ability to collect data may not rest with the person most interested in that data. For example, the publisher may
be best placed to collect data on the usage of a particular service; whereas the librarian may have most
interest in using that data in developing and evaluating the service.
The main purpose of this Technical Report is to spread knowledge about the practice of evaluating electronic
library services. Although electronic library services have been developed over more than thirty years, the
technical environment has changed significantly in the last ten years. The need for performance indicators for
electronic services was acknowledged in ISO 11620: 1998 Information and documentation - Library
performance indicators. It was also acknowledged that at the time of publication of that International Standard,
there were no indicators that had been tested and documented, or that were in widespread use. Although
there has been significant progress in the last few years on the definition, testing and documentation of
indicators, it is still too soon to determine which indicators will be most generally useful, or become widely
used. This Technical Report provides a standardized terminology and concise definitions and descriptions of a
selection of performance indicators, in the same format as that used in ISO 11620. Publications listed in the
Bibliography [2,3,5] provide more detailed information on data collection and analysis.
Only a small number of the indicators presented here are indicators of service quality: work on this area is not
yet mature enough to be included. The indicators that are included have been tested in libraries, or are simple
adaptations of similar indicators. In each case they have been judged against the criteria established in
ISO 11620. That is, each indicator is judged to have informative content, to be reliable, valid, appropriate,
practical, and in some circumstances may be used for comparative purposes.
The indicators presented in this Report are a representative selection of those which have been tested. A
further selection would have taken longer to prepare, and to publish. Developments in the measurement and
evaluation of electronic library services will be monitored by a Working Group, which will propose additional
indicators, and modifications or adaptations to the indicators here as they are tested and validated. This
Technical Report should be considered as work in progress towards the development of an International
Standard. It is intended that, where appropriate, the contents of this Technical Report will be incorporated in a
future revision of ISO 11620.

© ISO 2003 — All rights reserved v

---------------------- Page: 7 ----------------------

SIST-TP ISO/TR 20983:2005

---------------------- Page: 8 ----------------------

SIST-TP ISO/TR 20983:2005
TECHNICAL REPORT ISO/TR 20983:2003(E)

Information and documentation — Performance indicators for
electronic library services
1 Scope
This Technical Report is applicable to all types of libraries in all countries. Limitations on the applicability of
individual performance indicators are listed in the scope clause of the description of each indicator (see
Annex B).
Indicators may be used for comparison over time for the services provided by the same library. Comparisons
between libraries and services should only be made with caution, taking into account differences in the
constituencies of the libraries, with good understanding of the indicators used, and careful interpretation of the
data.
This Technical Report does not include indicators for the evaluation of the impact of libraries either on
individuals or on society.
Throughout the text, the names of indicators are printed with initial capitals for significant words, e.g.
Workstation Use Rate. This helps to distinguish the names from supporting text.
2 Normative references
The following referenced documents are indispensable for the application of this document. For dated
references, only the edition cited applies. For undated references, the latest edition of the referenced
document (including any amendments) applies.
ISO 2789:2003, Information and documentation — International library statistics
ISO 5127:2001, Information and documentation — Vocabulary
ISO 11620:1998, Information and documentation — Library performance indicators
ISO 11620:1998/Amd. 1:2003, Information and documentation — Library performance indicators —
Amendment 1: Additional performance indicators for libraries
3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
3.1
database
collection of electronically stored data or unit records (facts, bibliographic data, texts) with a common user
interface and software for the retrieval and manipulation of the data [ISO 2789]
NOTE 1 The data or records are usually collected with a particular intent and are related to a defined topic. A database
may be issued on CD-ROM, diskette, or other direct access method, or as a computer file accessed via dial-up methods
or via the Internet.
© ISO 2003 — All rights reserved 1

---------------------- Page: 9 ----------------------

SIST-TP ISO/TR 20983:2005
ISO/TR 20983:2003(E)
NOTE 2 Licensed databases are counted separately even if access to several licensed database products is effected
through the same interface.
3.2
digital document
information unit with a defined content that has been digitized by the library or acquired in digital form as part
of the library collection [ISO 2789]
NOTE 1 This includes eBooks, electronic patents, networked audiovisual documents and other digital documents, e.g.
reports, cartographic and music documents, pre-prints etc. Databases and electronic serials are excluded.
NOTE 2 Items incorporated in databases are covered by 3.1.
NOTE 3 A digital document can be structured into one or more files.
3.3
document
recorded information or material object which can be treated as a unit in a documentation process [ISO 5127]
NOTE Documents may differ in their physical form or characteristics.
3.4
document downloaded
full text of a document or part of a document in the electronic collection that is delivered to a user [ISO 2789]
3.5
electronic collection
all resources in electronic form in the library collection [ISO 2789]
NOTE The electronic collection includes databases, electronic serials, and digital documents. Free Internet resources
which have been catalogued by the library in its OPAC or a database should be counted separately.
3.6
electronic resource
a document in electronic form which forms a separate item with a distinctive title, whether issued in one or
several units
NOTE This includes electronic books, serials, maps, but not databases.
3.7
electronic service
electronic library service which is either supplied from local servers or accessible via networks [ISO 2789]
NOTE Electronic library services comprise the OPAC, the library web site, the electronic collection, electronic
document delivery (mediated), electronic reference service, user training on electronic services and Internet access
offered via the library.
3.8
entry
a unit of bibliographic or factual information or multimedia objects included in a searchable database
3.9
information request
information contact that involves the knowledge or use of one or more information sources (such as printed
and non-printed materials, machine-readable databases, the library’s own and other institutions’ catalogues)
by library staff [ISO 2789]
NOTE 1 May also involve recommendations, interpretation or instruction in the use of such sources.
NOTE 2 The request can be delivered personally or by means of telephone, regular mail, fax or electronic media (via e
mail, the library web site, instant messaging or other networked communications mechanisms).
2 © ISO 2003 — All rights reserved

---------------------- Page: 10 ----------------------

SIST-TP ISO/TR 20983:2005
ISO/TR 20983:2003(E)
NOTE 3 It is essential that libraries do not include directional and administrative inquiries, eg for locating staff or
facilities, regarding opening times or about handling equipment such as reader printers or computer terminals.
NOTE 4 Inquiries are also excluded if library staff simply help the user to locate items of stock that have already been
identified bibliographically.
3.10
library collection
all documents provided by a library for its users [ISO 2789]
NOTE 1 Comprises documents held locally and remote resources for which permanent or temporary access rights
have been acquired.
NOTE 2 Access rights may be acquired by the library itself, by a consortium and/or through external funding.
NOTE 3 Acquisition is to be understood as deliberately selecting a document, securing access rights and including it in
the OPAC or other databases of the library. Interlibrary lending and document delivery and excluded.
NOTE 4 Does not include links to Internet resources for which the library has not secured access rights by legal
agreements (eg legal deposit right), license or other contractual and/or co-operative agreement. Free Internet resources
which have been catalogued by the library in its OPAC or a database should be counted separately.
3.11
population to be served
number of individuals for whom the library is set up to provide its services and materials [ISO 11620]
NOTE For public libraries this will normally be the population of the legal service area (authority); for libraries of an
institution of higher education this will normally be the total of academic and professional staff plus students.
3.12
rejected session (turnaway)
unsuccessful request of a database or the OPAC because of requests exceeding simultaneous user limit
[ISO 2789]
NOTE Rejection through entry of wrong passwords is excluded.
3.13
remote session
a successful request of a database or the OPAC established from outside the library building (adapted from
[1]
EQUINOX )
3.14
session
successful request of a database or the OPAC [ISO 2789]
NOTE 1 A session is one cycle of user activities that typically starts when a user connects to a database or the OPAC
and ends with explicit (by leaving the database through log-out or exit) or implicit (timeout due to user inactivity)
termination of activities in the database. The average timeout period would be 30 minutes. If another time period is used
this should be reported.
NOTE 2 Sessions on the library web site are counted as virtual visits.
NOTE 3 Requests of a general entrance or gateway page should be excluded.
NOTE 4 If possible, requests by search engine should be excluded.
3.15
user training
training programme set up with a specified lesson plan, which aims at specific learning outcomes for the use
of library services [ISO 2789]
© ISO 2003 — All rights reserved 3

---------------------- Page: 11 ----------------------

SIST-TP ISO/TR 20983:2005
ISO/TR 20983:2003(E)
NOTE 1 User training can be offered as a tour of the library, as library tuition, or as a web based service for users.
NOTE 2 The duration of a lesson is irrelevant.
3.16
virtual visit
a user's request of the library web site from outside the library premises regardless of the number of pages or
elements viewed [ISO 2789]
3.17
web site
electronic service that has a unique domain on the Internet and consists of a collection of digital documents
[ISO 2789]
NOTE 1 The pages of a web site are usually interconnected by the use of hypertext links.
NOTE 2 Excludes the documents that fit the definitions of electronic collection and external Internet resources that may
be linked from the library web site.
3.18
workstation
computer that may stand alone or be networked, or a dumb terminal [ISO 2789]
4 Relationships with other indicators
4.1 The networked environment
The networked environment provides a different frame of reference for the provision of services: the
boundaries between internal and external provision are different. For example, consider document delivery. In
traditional library services this is represented by the interlending of physical documents between libraries. In
the electronic world, document delivery is accomplished using a network. The network may be provided by a
variety of external providers. Special equipment has to be installed in the libraries at both ends of document
delivery process and staff have to be trained to use it. The suppliers of documents may be publishers or
libraries; the end users may be directly in communication with the suppliers.
Again, the provision of equipment within the organisation of which the library forms a part, may be outside the
library's own control: however, the library’s performance may be judged on its service delivery even though it
does not control some of the infrastructure essential to the delivery of the service.
It is because the boundaries within which electronic services operate differ from those in which traditional
library services operate that comparisons between traditional and electronic library services are difficult, if not
impossible. Since the extent to which different libraries have adopted electronic library services will differ then
comparisons between libraries are increasingly difficult. The position is further complicated by the changes in
expectations, experience and behaviour of users and of publishers. New generations of users will increasingly,
at least in the developed world, be familiar with the use of computers and Internet applications generally and
will expect electronic services as a matter of course. There will, however be for some considerable time
significant numbers of older users who are less familiar with the concepts and practice of computer usage and
will need careful support if they are not to be disadvantaged.
On the other hand, publishers are operating in a variety of ways to deal with the possibilities of electronic
publishing. Long established practices in the supply of printed materials do not operate in the same way when
original works are often generated in electronic format. The economics of publication in electronic formats are
significantly different from those in printed formats and different charging models are being tried. The
preferred format varies for electronic publications between types of publication, and may also vary from year
to year; and the financial arrangements between library and publisher also vary between publications and
from year to year.
4 © ISO 2003 — All rights reserved

---------------------- Page: 12 ----------------------

SIST-TP ISO/TR 20983:2005
ISO/TR 20983:2003(E)
Furthermore, the ability to measure the usage of publications in electronic format differs radically from that
associated with print formats. It may be possible, although expensive, for a library to account for every use
made of a printed document within a library: it may well be impossible for the library to count every use made
of electronic documents. The publisher may be able count use made of the electronic format, while having no
idea of the use made of printed documents. Effective performance measurement thus depends on new
arrangements and collaboration between the copyright owners, publishers, agents, librarians and users.
A final point relates to ‘Per capita’ indicators. Some electronic library services (web-sites, OPACs, ‘free’
information services) can be accessed by anyone, and are not limited to the population to be served. In these
cases, usage per capita ratios would be artificially inflated, if the data were collected automatically. If a library
is interested in knowing what proportion of its population to be served is using its services remotely, those
data could be collected by a conventional survey technique.
4.2 Categories of performance indicators
In defining performance indicators for electronic library services it is natural to start by seeking analogies with
performance indicators for traditional library services. Reference to ISO 11620:1998 and its amendment
ISO 11620/Amd. 1 show a list of 34 indicators. These are grouped in a number of categories according to
Table 1.
Table 1
Service, activity or aspect measured Sub-category
User perception General
Public services General
Providing documents
Retrieving documents
Lending documents
Document delivery from external sources
Enquiry and reference services
Information searching
User Education
Facilities
Technical services Acquiring documents
Processing documents
Cataloguing
Promotion of services
User services Deployment of staff

In drawing up this technical report it has proved possible to use the same categories for the proposed
performance indicators with only minor modifications (see Annex A). The applicability of the indicators in
ISO 11620 is discussed in the following subclause.
4.3 The relevance of ISO 11620 to electronic library services
4.3.1 Indicators that apply equally to traditional and electronic library services
Some indicators are the same for both traditional and electronic library services. An obvious example is User
Satisfaction. Others need only very slight changes to existing definitions and descriptions of methods to make
it clear that they are relevant to both types of service: an example is Cost per Title Catalogued. Similarly, the
group of indicators relating to providing documents: here the definitions and descriptions would need to be
modified to include documents in electronic formats. There are other indicators that have analogues in the
networked environment (see 4.3.3), but more substantial revision would be needed to accommodate different
formats, and different bases for measurement. See Table 2.
© ISO 2003 — All rights reserved 5

---------------------- Page: 13 ----------------------

SIST-TP ISO/TR 20983:2005
ISO/TR 20983:2003(E)
Table 2
Service, activity or aspect Reference in
Performance indicator
measured ISO 11620
User perception: General User Satisfaction B.1.1.1
Public services: Providing Titles Availability B.2.2.1
documents
Required Titles Availability B.2.2.2
B.2.2.3
Percentage of Required Titles in the
Collection
Required Titles Extended Availability B.2.2.4
Public services: Enquiry and Correct Answer Fill Rate B.2.6.1
reference services
Title Catalogue Search Success Rate B.2.7.1
Public services: Information
searching
Subject Catalogue Search Success Rate B.2.7.2
Public services: Facilities Facilities Availability B.2.9.1
Facilities Use Rate B.2.9.2
Seat Occupancy Rate B.2.9.3
Automated Systems Availability B.2.9.4
Median Time of Document Acquisition B.3.1.1
Technical services: Acquiring
documents
Technical services: Cataloguing Cost per Title Catalogued B.3.3.1
User services; Deployment of staff User Services Staff per Capita B.5.1.1
User Services Staff as a Percentage of B.5.1.2
Total Staff

4.3.2 Indicators that apply only to traditional library services
The indicators in this group are so closely related to the physical format of documents that they have no
precise equivalent in the electronic world. See Table 3.
Table 3
Service, activity or aspect Reference in
Performance indicator
measured ISO 11620
Public services: Providing In-library Use per Capita B.2.2.5
documents
Document Use Rate B.2.2.6
Proportion of Stock not Used B.2.2.7
Shelving Accuracy B.2.2.8
Public services: Retrieving Median Time of Document Retrieval from B.2.3.1
documents Closed Stacks
Median Time of Document Retrieval from B.2.3.2
Open Access Areas
Public services: Document delivery Speed of Interlibrary Lending B.2.5.1
from external sources
Technical services: Processing Median Time of Document Processing B.3.2.1
documents

6 © ISO 2003 — All rights reserved

---------------------- Page: 14 ----------------------

SIST-TP ISO/TR 20983:2005
ISO/TR 20983:2003(E)
4.3.3 Indicators for traditional library services where analogous indicators can be found for
electronic library services
In this group, the indicators for electronic services are based on the same concepts as those for traditional
services, but the basis for measurement is different. For example, Library Visits per Capita would be replaced
by an indicator relating virtual visits to the population to be served. The various indicators for lending
documents have analogues now that digital lending is possible.
The fact that analogues can be found suggests that, in a future revision of ISO 11620, it may be necessary to
devise composite indicators that integrate traditional and networked services. For example, the indicator
Library Visits per Capita could be redefined to include both physical and virtual visits. Two composite
indicators are included in this Technical Report: Percentage of Virtual Visits t
...

RAPPORT ISO/TR
TECHNIQUE 20983
Première édition
2003-11-01


Information et documentation —
Indicateurs de performance pour les
services électroniques des bibliothèques
Information and documentation — Performance indicators for electronic
library services




Numéro de référence
ISO/TR 20983:2003(F)
©
ISO 2003

---------------------- Page: 1 ----------------------
ISO/TR 20983:2003(F)
PDF – Exonération de responsabilité
Le présent fichier PDF peut contenir des polices de caractères intégrées. Conformément aux conditions de licence d'Adobe, ce fichier
peut être imprimé ou visualisé, mais ne doit pas être modifié à moins que l'ordinateur employé à cet effet ne bénéficie d'une licence
autorisant l'utilisation de ces polices et que celles-ci y soient installées. Lors du téléchargement de ce fichier, les parties concernées
acceptent de fait la responsabilité de ne pas enfreindre les conditions de licence d'Adobe. Le Secrétariat central de l'ISO décline toute
responsabilité en la matière.
Adobe est une marque déposée d'Adobe Systems Incorporated.
Les détails relatifs aux produits logiciels utilisés pour la création du présent fichier PDF sont disponibles dans la rubrique General Info
du fichier; les paramètres de création PDF ont été optimisés pour l'impression. Toutes les mesures ont été prises pour garantir
l'exploitation de ce fichier par les comités membres de l'ISO. Dans le cas peu probable où surviendrait un problème d'utilisation,
veuillez en informer le Secrétariat central à l'adresse donnée ci-dessous.


©  ISO 2003
Droits de reproduction réservés. Sauf prescription différente, aucune partie de cette publication ne peut être reproduite ni utilisée sous
quelque forme que ce soit et par aucun procédé, électronique ou mécanique, y compris la photocopie et les microfilms, sans l'accord écrit
de l'ISO à l'adresse ci-après ou du comité membre de l'ISO dans le pays du demandeur.
ISO copyright office
Case postale 56 • CH-1211 Geneva 20
Tel. + 41 22 749 01 11
Fax. + 41 22 749 09 47
E-mail copyright@iso.org
Web www.iso.org
Publié en Suisse

ii © ISO 2003 — Tous droits réservés

---------------------- Page: 2 ----------------------
ISO/TR 20983:2003(F)
Sommaire Page
Avant-propos. iv
Introduction . v
1 Domaine d'application. 1
2 Références normatives. 1
3 Termes et définitions . 1
4 Liens avec d'autres indicateurs. 5
4.1 Environnement en réseau . 5
4.2 Catégories d'indicateurs de performance . 6
4.3 Pertinence de l'ISO 11620 pour les services électroniques . 6
4.3.1 Indicateurs communs aux services traditionnels et électroniques. 6
4.3.2 Indicateurs qui ne s'appliquent qu'aux services traditionnels des bibliothèques. 7
4.3.3 Indicateurs s'appliquant aux services traditionnels des bibliothèques pour lesquels des
indicateurs analogues peuvent être trouvés pour les services électroniques . 8
5 Utilisation des indicateurs de performance . 8
5.1 Objectifs. 8
5.2 Choix d'indicateurs de performance. 9
5.3 Restrictions . 9
5.3.1 Mesure de l'impact des services électroniques des bibliothèques. 9
5.3.2 Degré de précision. 9
5.3.3 Comparabilité des données des indicateurs de performance . 10
5.3.4 Les indicateurs à usage provisoire. 10
Annexe A Liste des indicateurs de performance pour les services électroniques des
bibliothèques. 11
Annexe B Description des indicateurs de performance . 13
Bibliographie . 33
Index alphabétique. 34

© ISO 2003 — Tous droits réservés iii

---------------------- Page: 3 ----------------------
ISO/TR 20983:2003(F)
Avant-propos
L'ISO (Organisation internationale de normalisation) est une fédération mondiale d'organismes nationaux de
normalisation (comités membres de l'ISO). L'élaboration des Normes internationales est en général confiée
aux comités techniques de l'ISO. Chaque comité membre intéressé par une étude a le droit de faire partie du
comité technique créé à cet effet. Les organisations internationales, gouvernementales et non
gouvernementales, en liaison avec l'ISO participent également aux travaux. L'ISO collabore étroitement avec
la Commission électrotechnique internationale (CEI) en ce qui concerne la normalisation électrotechnique.
Les Normes internationales sont rédigées conformément aux règles données dans les Directives ISO/CEI,
Partie 2.
La tâche principale des comités techniques est d'élaborer les Normes internationales. Les projets de Normes
internationales adoptés par les comités techniques sont soumis aux comités membres pour vote. Leur
publication comme Normes internationales requiert l'approbation de 75 % au moins des comités membres
votants.
Exceptionnellement, lorsqu'un comité technique a réuni des données de nature différente de celles qui sont
normalement publiées comme Normes internationales (ceci pouvant comprendre des informations sur l'état
de la technique par exemple), il peut décider, à la majorité simple de ses membres, de publier un Rapport
technique. Les Rapports techniques sont de nature purement informative et ne doivent pas nécessairement
être révisés avant que les données fournies ne soient plus jugées valables ou utiles.
L'attention est appelée sur le fait que certains des éléments du présent document peuvent faire l'objet de
droits de propriété intellectuelle ou de droits analogues. L'ISO ne saurait être tenue pour responsable de ne
pas avoir identifié de tels droits de propriété et averti de leur existence.
L'ISO/TR 20983 a été élaboré par le comité technique ISO/TC 46, Information et documentation, sous-comité
SC 8, Qualité — Statistiques et évaluation de la performance.
iv © ISO 2003 — Tous droits réservés

---------------------- Page: 4 ----------------------
ISO/TR 20983:2003(F)
Introduction
Le présent Rapport technique a pour objet l'évaluation des services électroniques des bibliothèques.
La fourniture réussie de ces services dépend d'une collaboration étroite entre les éditeurs, les fournisseurs
d'accès et les bibliothécaires. Leurs besoins de collecte de données sur les divers aspects de la performance
sont différents, et les destinataires de ces données n'ont pas forcément la maîtrise de leur collecte. Par
exemple, l'éditeur peut être mieux placé pour collecter les données sur l'utilisation d'un service, tandis que
c'est sans doute le bibliothécaire qui sera le plus intéressé par l'emploi de ces données pour développer et
évaluer ce service.
L'objectif principal du présent Rapport technique est de diffuser les connaissances sur l'évaluation pratique
des services électroniques des bibliothèques. Bien que ces services aient été développés depuis plus de
trente ans, l'environnement technique a changé de façon significative au cours de la dernière décennie.
L'ISO 11620:1998, Information et documentation — Indicateurs de performance des bibliothèques, a reconnu
la nécessité de disposer d'indicateurs de performance pour les services électroniques. Il était en même temps
reconnu qu'au moment de la publication de cette Norme internationale, on ne disposait pas de tels indicateurs
qui aient été testés et publiés, ou qui aient été utilisés largement. Bien que l'on ait beaucoup progressé ces
dernières années dans la définition, dans les tests et la publication d'indicateurs, il est encore trop tôt pour
déterminer quels sont ceux qui s'avéreront les plus généralement utiles, ou qui seront d'un usage de plus en
plus répandu. Le présent Rapport technique fournit une terminologie normalisée ainsi que des définitions
concises et des descriptions d'une sélection d'indicateurs de performance dans une présentation identique à
celle de l'ISO 11620. Les publications référencées dans la bibliographie [2, 3, 5] fournissent une information
plus détaillée sur la collecte et l'analyse des données.
Seuls quelques indicateurs présentés ici sont des indicateurs de qualité du service: la réflexion sur ce sujet
n'est pas assez avancée pour être intégrée ici. Les indicateurs qui ont été retenus ont été testés dans des
bibliothèques, ou bien sont de simples adaptations d'indicateurs semblables. Dans tous les cas, ils ont été
retenus selon les critères établis dans l'ISO 11620. A savoir que chaque indicateur dispose d'un contenu
informatif, est fiable, valide, approprié, applicable, et peut dans certaines circonstances être employé pour des
comparaisons.
Les indicateurs figurant dans le présent Rapport constituent une sélection représentative de ceux qui ont été
testés. Une sélection plus élaborée aurait demandé trop de temps pour leur préparation et leur publication.
Les avancées dans la mesure et l'évaluation des services électroniques des bibliothèques seront suivies par
un groupe de travail, qui proposera des indicateurs supplémentaires, ainsi que des modifications ou des
adaptations des indicateurs présentés ici au fur et à mesure qu'ils seront testés et validés. Il convient de
considérer le présent Rapport technique comme un travail préparatoire en vue de la rédaction d'une Norme
internationale. Il est entendu que, le moment venu, son contenu sera intégré dans une révision ultérieure de
l'ISO 11620.
© ISO 2003 — Tous droits réservés v

---------------------- Page: 5 ----------------------
RAPPORT TECHNIQUE ISO/TR 20983:2003(F)

Information et documentation — Indicateurs de performance
pour les services électroniques des bibliothèques
1 Domaine d'application
Le présent Rapport technique s'applique à tous les types de bibliothèques dans tous les pays. Les restrictions
apportées à l'application d'indicateurs de performance particuliers sont énumérées dans la rubrique Domaine
d'application qui figure dans la description de chaque indicateur (voir Annexe B).
Les indicateurs peuvent être utilisés pour des comparaisons au cours du temps pour les services fournis par
la même bibliothèque. On peut aussi, sous réserve de prudence, comparer des bibliothèques entre elles, à
condition de prendre en compte leurs différences d'organisation, de faire preuve d'une bonne compréhension
des indicateurs utilisés, et d'interpréter les données avec précaution.
Le présent Rapport technique ne comporte pas d'indicateur pour l'évaluation de l'impact des bibliothèques sur
les individus ou sur la société.
2 Références normatives
Les documents de référence suivants sont indispensables pour l'application du présent document. Pour les
références datées, seule l'édition citée s'applique. Pour les références non datées, la dernière édition du
document de référence s'applique (y compris les éventuels amendements).
ISO 2789:2003, Information et documentation — Statistiques internationales de bibliothèques
ISO 5127:2001, Information et documentation — Vocabulaire
ISO 11620:1998, Information et documentation — Indicateurs de performance des bibliothèques
ISO 11620:1998/Amd. 1:2003, Information et documentation — Indicateurs de performance des bibliothèques,
Amendement 1: Indicateurs complémentaires de performance des bibliothèques
3 Termes et définitions
Pour les besoins du présent document, les termes et définitions suivants s'appliquent.
3.1
base de données
ensemble de données ou enregistrements informatiques (données factuelles ou bibliographiques, textes),
dotés d'une interface commune et d'un logiciel pour récupérer ou exploiter les données [ISO 2789]
(Terme anglais: database)
NOTE 1 Les données ou les références sont généralement classées avec un objectif précis et concernent un sujet
défini. Une base de données peut être publiée sur cédérom, disquette ou sur d'autres supports, ou en tant que fichier
informatique accessible en ligne ou par Internet.
NOTE 2 Les bases de données soumises à licence sont comptabilisées à l'unité, même si l'accès à plusieurs bases de
données soumises à licences est proposé par la même interface.
© ISO 2003 — Tous droits réservés 1

---------------------- Page: 6 ----------------------
ISO/TR 20983:2003(F)
3.2
document numérique
unité documentaire au contenu identifié qui a été numérisée par la bibliothèque ou acquise sous forme
numérique comme élément de la collection de la bibliothèque [ISO 2789]
(Terme anglais: digital document)
NOTE 1 Cela comprend les livres électroniques, les brevets électroniques, les documents audiovisuels en réseau et
les autres documents numériques, par exemple les rapports, les documents cartographiques et musicaux, les pré-
publications, etc. Les bases de données et les périodiques électroniques sont exclus.
NOTE 2 Les éléments faisant partie des bases de données sont décrits en 3.1.
NOTE 3 Un document numérique peut être structuré en un ou plusieurs fichiers.
3.3
document
information enregistrée ou objet matériel qui peut être traité comme une unité dans une chaîne documentaire
[ISO 5127]
(Terme anglais: document)
NOTE Les documents peuvent varier dans leur forme et leurs caractéristiques.
3.4
document téléchargé
texte intégral ou extrait d'un document de la collection électronique qui est fourni à un usager [ISO 2789]
(Terme anglais: document downloaded)
3.5
collection électronique
toutes les ressources de la bibliothèque qui sont sous forme électronique [ISO 2789]
(Terme anglais: electronic collection)
NOTE La collection électronique comprend les bases de données, les périodiques électroniques et les documents
numériques. Les ressources Internet gratuites qui ont été cataloguées par la bibliothèque et figurent dans le catalogue
interrogeable en ligne par le public ou dans une base de données doivent être comptabilisées séparément.
3.6
ressource électronique
document sous forme électronique qui constitue un tout séparé avec un titre distinct, qu'il soit publié en une
ou plusieurs unités
(Terme anglais: electronic resource)
NOTE Cela comprend les livres, publications en série, et cartes électroniques, à l'exclusion des bases de données.
3.7
service électronique
service en ligne fourni par des serveurs locaux ou accessible à distance par les réseaux [ISO 2789]
(Terme anglais: electronic service)
NOTE Les services électroniques de bibliothèque comprennent le catalogue interrogeable en ligne par le public, le
site web de la bibliothèque, la collection électronique, la fourniture de documents électroniques par l'intermédiaire de la
bibliothèque, un service de référence électronique, la formation des usagers aux services électroniques et l'accès à
Internet proposé par la bibliothèque.
2 © ISO 2003 — Tous droits réservés

---------------------- Page: 7 ----------------------
ISO/TR 20983:2003(F)
3.8
enregistrement
unité d'information bibliographique ou factuelle ou objet multimédia inclus dans une base de données
interrogeable
(Terme anglais: entry)
3.9
demande d'information
opération de médiation auprès du public par le personnel de la bibliothèque à partir d'une ou de plusieurs
sources d'informations (tels que les documents imprimés et non imprimés, les bases de données
électroniques, les catalogues de la bibliothèque et d'autres institutions) [ISO 2789]
(Terme anglais: information request)
NOTE 1 Concerne également des recommandations, interprétations et instructions pour l'utilisation de ces ressources.
NOTE 2 La demande peut être adressée de vive voix, par téléphone, par courrier, par télécopie ou par voie
électronique (messagerie électronique, site web de la bibliothèque, message instantané ou tout autre moyen de
communication en réseau).
NOTE 3 Il est essentiel que les bibliothèques ne comptabilisent pas les demandes de renseignement, d'orientation et
les questions administratives telles que trouver un agent de la bibliothèque ou repérer des équipements, préciser les
horaires d'ouverture ou expliquer le fonctionnement des équipements (imprimantes ou ordinateurs par exemple).
NOTE 4 Les questions relatives à la simple localisation de documents du fonds déjà identifiés dans le catalogue ne
sont pas comptées.
3.10
collection
tous les documents mis à disposition des usagers par une bibliothèque [ISO 2789]
(Terme anglais: library collection)
NOTE 1 Cela comprend les documents détenus localement et les ressources accessibles à distance pour lesquelles
des droits permanents ou temporaires ont été acquis.
NOTE 2 Les droits d'accès peuvent être acquis par la bibliothèque elle-même, par un consortium et/ou par un
financement extérieur.
NOTE 3 Les acquisitions doivent être considérées comme le résultat d'une sélection de documents, de droits d'accès
négociés, et être incluses dans le catalogue interrogeable en ligne par le public ou dans d'autres bases de données de la
bibliothèque. Le prêt entre bibliothèques et la fourniture de documents à distance sont exclus.
NOTE 4 Cela n'inclut pas les liens aux ressources Internet pour lesquels la bibliothèque n'a pas négocié de droits
d'accès par des autorisations légales (par exemple, droit lié au dépôt légal), des licences ou des accords soit contractuels
soit de coopération. Il convient de comptabiliser séparément les liens avec des ressources Internet gratuites qui ont été
cataloguées par la bibliothèque et qui sont présentées sur le catalogue interrogeable en ligne par le public ou dans une
base de données.
3.11
population à desservir
ensemble des individus auxquels la bibliothèque est destinée à fournir ses services et ses documents
[ISO 11620]
(Terme anglais: population to be served)
NOTE En règle générale, pour les bibliothèques publiques, il s'agit de la population relevant de la même collectivité
territoriale; pour les bibliothèques d'un établissement d'enseignement supérieur, cela comprend l'ensemble des
personnels enseignants et administratifs et des étudiants.
© ISO 2003 — Tous droits réservés 3

---------------------- Page: 8 ----------------------
ISO/TR 20983:2003(F)
3.12
session rejetée
tentative infructueuse de connexion à une base de donnée ou à un catalogue interrogeable en ligne par le
public lorsque le nombre de requêtes dépasse le nombre autorisé d'utilisateurs simultanés [ISO 2789]
[Terme anglais: rejected session (turnaway)]
NOTE Les rejets dus à des mots de passe non conformes sont exclus.
3.13
session à distance
[1]
interrogation d'une base de données depuis l'extérieur des locaux de la bibliothèque (adapté d'EQUINOX )
(Terme anglais: remote session)
3.14
session
connexion effective à une base de données ou à un catalogue interrogeable en ligne par le public [ISO 2789]
(Terme anglais: session)
NOTE 1 Une session est un cycle qui débute lorsque l'usager se connecte à une base de données ou à un catalogue
interrogeable en ligne, et qui se termine de façon explicite (par déconnexion ou fonction quitter) ou implicite (mise en veille
après une période déterminée de non utilisation). Le délai moyen de déconnexion spontanée est de 30 minutes. Si un
autre délai est utilisé, il convient de le mentionner.
NOTE 2 Les sessions sur le site web de la bibliothèque sont comptabilisées comme visites virtuelles.
NOTE 3 Les connexions à une page d'accueil sont exclues.
NOTE 4 Si possible, il convient d'exclure les recherches effectuées par les moteurs de recherche.
3.15
formation des usagers
plan de formation mis en place selon un programme spécifique et qui a pour but de développer l'utilisation des
services de la bibliothèque [ISO 2789]
(Terme anglais: user training)
NOTE 1 La formation des usagers peut être proposée sous la forme d'une visite de la bibliothèque, de présentation de
la bibliothèque ou comme service sur le web à destination des usagers.
NOTE 2 La durée des formations ne doit pas être prise en compte.
3.16
visite virtuelle
recherche faite par un usager sur le site web de la bibliothèque depuis l'extérieur de la bibliothèque quel que
soit le nombre de pages ou d'informations consultées [ISO 2789]
(Terme anglais: virtual visit)
3.17
site web
service électronique ayant un nom de domaine unique sur Internet et se composant d'une ensemble de
documents numériques [ISO 2789]
(Terme anglais: web site)
NOTE 1 Les pages du site web sont généralement reliées grâce à des liens hypertextes.
4 © ISO 2003 — Tous droits réservés

---------------------- Page: 9 ----------------------
ISO/TR 20983:2003(F)
NOTE 2 Sont exclus les documents qui répondent à la définition de collection électronique et les ressources externes
d'Internet pour lesquelles un lien a été établi à partir du site web de la bibliothèque.
3.18
poste de travail
ordinateur qui fonctionne seul ou en réseau, ou en terminal passif [ISO 2789]
(Terme anglais: workstation)
4 Liens avec d'autres indicateurs
4.1 Environnement en réseau
Le fonctionnement en réseau définit un cadre de référence différent pour la fourniture de services: les
frontières varient selon l'origine de l'approvisionnement, fourniture interne ou externe. Considérons par
exemple la fourniture de documents. Dans les services traditionnels de la bibliothèque, le prêt de documents
physiques entre bibliothèques en est une des formes. Dans le monde électronique, on procure le document
grâce à un réseau. L'accès à ce dernier peut être fourni par divers prestataires extérieurs. Un équipement
spécifique doit être installé dans les bibliothèques aux deux bouts de la chaîne de fourniture du document, et
le personnel doit être formé à son utilisation. Les fournisseurs de documents peuvent être des éditeurs ou des
bibliothèques; les utilisateurs finals peuvent entrer en liaison directe avec les fournisseurs.
Encore une fois, la mise en œuvre d'un équipement à l'intérieur de l'établissement dont la bibliothèque fait
partie peut être réalisée en dehors du contrôle de la bibliothèque: pourtant la performance de celle-ci sera
évaluée sur sa capacité à fournir un service, même si elle ne contrôle pas certains aspects essentiels de
l'infrastructure.
Du fait que les frontières diffèrent entre les services électroniques et les services traditionnels des
bibliothèques, les comparaisons entre ces services sont difficiles, sinon impossibles. Puisque le degré
d'intégration des services électroniques varie selon les bibliothèques, les comparaisons sont de plus en plus
ardues. La situation est rendue encore plus complexe par les évolutions dans les attentes, dans l'expérience
et le comportement des usagers et des éditeurs. Les nouvelles générations d'usagers vont être, au moins
dans les pays développés, de plus en plus familiarisées avec l'informatique et l'utilisation d'Internet en
général; ils considéreront les services électroniques comme une évidence. Toutefois, pendant une période
encore assez longue, de nombreux usagers plus âgés resteront moins familiers avec les concepts et la
pratique de l'informatique, et auront besoin d'un soutien attentif si l'on ne veut pas qu'ils soient désavantagés.
D'un autre côté, les éditeurs travaillent dans de nombreuses directions pour prendre en compte les
possibilités de l'édition électronique. Les pratiques de fourniture de documents imprimés, établies de longue
date, n'ont plus le même caractère opérationnel maintenant que les originaux sont souvent produits
directement en format électronique. L'économie de l'édition électronique diffère de manière significative de
celle de l'imprimé, et différents modes de tarification sont testés. Le format privilégié pour les publications
électroniques varie selon le type de publication mais peut aussi évoluer d'une année sur l'autre; et les
modalités financières des contrats entre la bibliothèque et l'éditeur peuvent aussi changer selon la publication
et d'une année sur l'autre.
De plus, la capacité à mesurer l'utilisation des publications diffère radicalement entre documents
électroniques et imprimés. S'il est possible pour une bibliothèque, bien que cela soit onéreux, de décompter
chaque utilisation faite d'un document imprimé dans ses locaux, il pourra bien lui être impossible de compter
chaque utilisation d'un document électronique. D'un autre côté, l'éditeur pourra être capable de compter
l'utilisation au format électronique, tout en n'ayant aucune idée de l'utilisation faite des documents imprimés.
Une mesure effective de la performance repose donc sur de nouveaux accords et sur la collaboration entre
les titulaires de droits d'auteurs, les éditeurs, les agents, les bibliothécaires et les usagers.
Un dernier point concerne les indicateurs «par personne de la population à desservir». Certains services
électroniques des bibliothèques (sites web, CIEL, services d'information libres) peuvent être utilisés par
quiconque, et ne sont pas réservés à la population à desservir. Dans ce cas, les ratios d'utilisation «par
personne de la population à desservir» seraient artificiellement élevés si les données étaient collectées
© ISO 2003 — Tous droits réservés 5

---------------------- Page: 10 ----------------------
ISO/TR 20983:2003(F)
automatiquement. Si une bibliothèque souhaite connaître la proportion de sa population à desservir qui utilise
ses services à distance, une méthode d'enquête classique sera plus appropriée afin de collecter de telles
données.
4.2 Catégories d'indicateurs de performance
En définissant des indicateurs de performance pour les services électroniques des bibliothèques, il est naturel
de commencer par rechercher les analogies avec les indicateurs de performance pour les services
traditionnels. L'ISO 11620:1998 et son amendement ISO 11620/DAM 1 déterminent une liste de 34
indicateurs. Ils sont regroupés en un certain nombre de catégories, selon le Tableau 1.
Tableau 1
Service, activité ou aspect mesuré Sous-catégorie
Opinion des usagers Généralités
Services offerts au public Généralités
Fourniture de documents
Recherche de documents
Prêt de documents
Fourniture de documents provenant de sources externes
Service de renseignement et de référence
Recherche d'information
Formation des usagers
Installations
Services techniques Acquisition de documents
Traitement des documents
Catalogage
Promotion des services
Services aux usagers Disponibilité et emploi des ressources humaines

Durant l'élaboration du présent Rapport technique, il a été démontré qu'il était possible d'utiliser les mêmes
catégories pour les indicateurs de performance proposés, sous réserve de modifications mineures (voir
Annexe A). L'applicabilité des indicateurs de l'ISO 11620 est discutée dans le paragraphe suivant.
4.3 Pertinence de l'ISO 11620 pour les services électroniques
4.3.1 Indicateurs communs aux services traditionnels et électroniques
Certains indicateurs sont les mêmes à la fois pour les services traditionnels et les services électroniques des
bibliothèques. L'indicateur Satisfaction des usagers en est un exemple évident. Pour d'autres, de très légers
changements dans la définition et la description des méthodes permettent à l'évidence de les rendre
pertinents pour les deux types de service: par exemple, le Coût par titre catalogué. De même pour le groupe
des indicateurs relatifs à la fourniture de documents: là les définitions et les descriptions devront être
modifiées pour inclure les documents électroniques. D'autres indicateurs ont des équivalents dans
l'environnement en réseau (voir 4.3.3), mais une révision plus substantielle serait nécessaire pour harmoniser
les différents formats et les différentes unités de mesure et méthodes d'estimation. Voir Tableau 2.
6 © ISO 2003 — Tous droits réservés

---------------------- Page: 11 ----------------------
ISO/TR 20983:2003(F)
Tableau 2
Service, activité ou aspect Référence dans
Indicateur
...