ISO/IEC 14143-2:2011

INTERNATIONAL ISO/IEC
STANDARD 14143-2
Second edition
2011-09-01
Information technology — Software
measurement — Functional size
measurement
Part 2:
Conformity evaluation of software size
measurement methods
to ISO/IEC 14143-1
Technologies de l'information — Mesurage du logiciel — Mesurage de
la taille fonctionnelle
Partie 2: Évaluation de la conformité des méthodes de mesure de taille
de logiciel à l'ISO/CEI 14143-1

Reference number
©
ISO/IEC 2011
©  ISO/IEC 2011
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized in any form or by any means,
electronic or mechanical, including photocopying and microfilm, without permission in writing from either ISO at the address below or
ISO's member body in the country of the requester.
ISO copyright office
Case postale 56  CH-1211 Geneva 20
Tel. + 41 22 749 01 11
Fax + 41 22 749 09 47
E-mail copyright@iso.org
Web www.iso.org
Published in Switzerland
ii © ISO/IEC 2011 – All rights reserved

Contents Page
Foreword . iv
Introduction . v
1  Scope . 1
2  Normative references . 1
3  Terms and definitions . 1
4  Conformity evaluation . 3
4.1  General . 3
4.2  Overview . 4
4.3  Evaluator characteristics . 4
4.3.1  Evaluator organization . 4
4.3.2  Conformity evaluation team . 5
4.4  Inputs to conformity evaluation . 5
4.4.1  List of inputs . 5
4.4.2  Candidate FSM Method documentation . 5
4.4.3  Conformity evaluation plan . 6
4.4.4  Conformity evaluation procedure . 6
4.4.5  Conformity evaluation checklist . 6
4.5  Tasks and steps of the conformity evaluation procedure . 7
4.5.1  Guidance . 7
4.5.2  Tasks and Steps . 8
4.6  Conformity evaluation output . 11
4.7  Conformity evaluation result . 12
Annex A (informative) Evaluator capability . 13
A.1  Conformity evaluation team . 13
A.2  Demonstration of competence . 13
Annex B (informative) Example of a conformity evaluation checklist . 15
B.1  Introduction . 15
B.2  Conformity evaluation checklist . 15
Annex C (informative) Example of a conformity evaluation report . 26
C.1  Introduction . 26
C.2  Executive summary . 26
C.3  Conformity evaluation checklist . 27
C.4  Original conformity evaluation plan . 28
C.5  Justifications for results . 28
C.6  Conformity evaluation procedure . 28
C.7  Qualifications of conformity evaluation team . 28
C.8  Record of the liaison with the owner during the conformity evaluation process . 28

© ISO/IEC 2011 – All rights reserved iii

Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are members of
ISO or IEC participate in the development of International Standards through technical committees
established by the respective organization to deal with particular fields of technical activity. ISO and IEC
technical committees collaborate in fields of mutual interest. Other international organizations, governmental
and non-governmental, in liaison with ISO and IEC, also take part in the work. In the field of information
technology, ISO and IEC have established a joint technical committee, ISO/IEC JTC 1.
International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2.
The main task of the joint technical committee is to prepare International Standards. Draft International
Standards adopted by the joint technical committee are circulated to national bodies for voting. Publication as
an International Standard requires approval by at least 75 % of the national bodies casting a vote.
Attention is drawn to the possibility that some of the elements of this document may be the subject of patent
rights. ISO and IEC shall not be held responsible for identifying any or all such patent rights.
ISO/IEC 14143-2 was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 7, Software and systems engineering.
This second edition cancels and replaces the first edition (ISO/IEC 14143-2:2002), of which it constitutes a
minor revision.
ISO/IEC 14143 consists of the following parts, under the general title Information technology — Software
measurement — Functional size measurement:
― Part 1: Definition of concepts
― Part 2: Conformity evaluation of software size measurement methods to ISO/IEC 14143-1
― Part 3: Verification of functional size measurement methods [Technical Report]
― Part 4: Reference model [Technical Report]
― Part 5: Determination of functional domains for use with functional size measurement [Technical Report]
― Part 6: Guide for use of ISO/IEC 14143 series and related International Standards
iv © ISO/IEC 2011 – All rights reserved

Introduction
Functional Size Measurement (FSM) is a technique used to measure the size of software by quantifying the
1)
Functional User Requirements of the software . The first published method to embrace this concept was
Function Point Analysis, developed by Allan Albrecht in the late 1970s. Since then, numerous extensions and
variations of the original method have been developed. The end user may have many variants from which to
choose, each with its own advantages in specific situations. ISO/IEC 14143-1 was developed to define the
concepts of FSM and provides a basis against which the user can compare all variants. This part of
ISO/IEC 14143 was developed to provide a process for checking whether a Candidate FSM Method conforms
to the provisions of ISO/IEC 14143-1. The output from this process can assist prospective users of the
Candidate FSM Method in judging whether it is appropriate to their needs.

1)
Refer to ISO/IEC 14143-1.
© ISO/IEC 2011 – All rights reserved v

INTERNATIONAL STANDARD ISO/IEC 14143-2:2011(E)

Information technology — Software measurement — Functional
size measurement
Part 2:
Conformity evaluation of software size measurement methods
to ISO/IEC 14143-1
1 Scope
This part of ISO/IEC 14143:
a) establishes a framework for the conformity evaluation of a Candidate FSM Method against the provisions
of ISO/IEC 14143-1;
b) describes a process for conformity evaluation of whether a Candidate FSM Method meets the (type)
requirements of ISO/IEC 14143-1 such that it is an actual FSM method, i.e. they are of the same type;
c) describes the requirements for performing a conformity evaluation in order to ensure repeatability of the
conformity evaluation process, as well as consistency of decisions on conformity and the final result;
d) aims to ensure that the output from the conformity evaluation process is objective, impartial, consistent,
repeatable, complete and auditable;
e) provides informative guidelines for determining the competence of the conformity evaluation teams;
f) provides an example checklist to assist in the conformity evaluation of a Candidate FSM Method; and
g) provides an example template for the conformity evaluation report.
2 Normative references
The following referenced documents are indispensable for the application of this document. For dated
references, only the edition cited applies. For undated references, the latest edition of the referenced
document (including any amendments) applies.
ISO/IEC 14143-1, Information technology — Software measurement — Functional size measurement —
Part 1: Definition of concepts
3 Terms and definitions
For the purposes of this document, the terms and definitions given in ISO/IEC 14143-1 and the following apply.
3.1
Candidate FSM Method
documented software size measurement method submitted for conformity evaluation according to
ISO/IEC 14143-1
© ISO/IEC 2011 – All rights reserved 1

3.2
evaluation checklist
list of questions, each of which is designed to check for conformity of a product, process or service to one or
more provisions within a particular International Standard
NOTE In the case of this part of ISO/IEC 14143, the product being evaluated for conformance is the Candidate FSM
Method and the provisions are those of ISO/IEC 14143-1.
3.3
evaluation procedure
series of tasks and steps that, when completed, enable the evaluation team to determine if the product,
process or service being evaluated is conformant to a particular standard
3.4
evaluation sponsor
person or organization that requires the evaluation to be performed and provides financial or other resources
to carry it out
3.5
exclusive requirement
mandatory requirement (deprecated)
requirement of a normative document that must necessarily be fulfilled in order to comply with that document
[ISO/IEC Guide 2:2004, definition 7.5.1]
3.6
optional requirement
requirement of a normative document that must be fulfilled in order to comply with a particular option permitted
by that document
NOTE An optional requirement may be either
a) one of two or more alternative requirements, or
b) an additional requirement that must be fulfilled only if applicable and may otherwise be disregarded.
[ISO/IEC Guide 2:2004, definition 7.5.2]
3.7
owner
person or organization that owns the copyright for the Candidate FSM Method
3.8
provision
expression in the content of a normative document, that takes the form of a statement, an instruction, a
recommendation or a requirement
NOTE These types of provision are distinguished by the form of wording they employ; for example, instructions are
expressed in the imperative mood, recommendations by the use of the auxiliary “should”, and requirements by the use of
the auxiliary “shall”.
[ISO/IEC Guide 2:2004, definition 7.1]
3.9
recommendation
provision that conveys advice or guidance
[ISO/IEC Guide 2:2004, definition 7.4]
2 © ISO/IEC 2011 – All rights reserved

3.10
requirement
provision that conveys criteria to be fulfilled
[ISO/IEC Guide 2:2004, definition 7.5]
NOTE A requirement is denoted by the word “shall” and when used includes both the exclusive and applicable
optional requirements.
4 Conformity evaluation
4.1 General
4.1.1 Conformity evaluations are conducted by a conformity evaluation team that has the competencies
described in this part of ISO/IEC 14143. This part of ISO/IEC 14143 assumes familiarity with the concepts and
definitions described in ISO/IEC 14143-1.
The conformity evaluation is performed by cross-referencing each component of a Candidate FSM Method
against the corresponding provisions of ISO/IEC 14143-1. The components of the Candidate FSM Method are
then evaluated for their conformity.
The output from the conformity evaluation includes a decision for each provision evaluated. Only the
requirements (shalls) are considered when determining if the Candidate FSM Method conforms to
ISO/IEC 14143-1. The recommendations (shoulds) of ISO/IEC 14143-1 may also be investigated to provide
additional information to end users of the Candidate FSM Method.
The output from the conformity evaluation process is the conformity evaluation report. The report may be used
to:
a) inform end users that a Candidate FSM Method conforms to ISO/IEC 14143-1 in accordance with this
part of ISO/IEC 14143, and is therefore an FSM Method, and
b) assist end users in making informed judgements about which method best suits their needs.
4.1.2 This part of ISO/IEC 14143 may be used for first party (supplier), second party (user or purchaser) or
third party (independent body), conformity evaluations.
NOTE The relationship between the owner, sponsor and evaluator depends on the type of evaluation that is
performed, i.e. first, second or third party.
4.1.3 While conformance of a Candidate FSM Method to ISO/IEC 14143-1 may be claimed without
referencing this part of ISO/IEC 14143, this part provides a conformity evaluation process that may be used to
add credibilty to such claims. This part places requirements upon a conformity evaluation procedure and is
usable for first, second or third party claims of conformance. Its provisions are particularly suitable for those
who require third party conformity evaluation. Customers desiring to use or aquire an FSM Method evaluated
for conformance in accordance with this part, should explicitly cite this International Standard when requesting
the evaluation.
4.1.4 Conformity evaluation should not be construed as guaranteeing that the FSM Method is free from
non-conformities; it only signifies that evidence of non-conformance was not found during the conformity
evaluation process.
4.1.5 A Candidate FSM Method shall be determined as conforming if it successfully completes a conformity
evaluation procedure which satisfies the requirements of 4.5.
NOTE 1 Conformity of a Candidate FSM Method is based on evaluation against requirements of ISO/IEC 14143-1.
This part of ISO/IEC 14143 defines a process that may be used in evaluating whether a Candidate FSM Method conforms
to the requirements of ISO/IEC 14143-1.
© ISO/IEC 2011 – All rights reserved 3

NOTE 2 An International Standard on conformity evaluation or test methods, such as this one, does not imply any
obligation to carry out any kind of test. It defines the process by which the evaluation, if required and referred to (for
example in a regulation, or in contract documents), should be carried out.
4.2 Overview
4.2.1 The objective of a conformity evaluation shall be to determine if the Candidate FSM Method conforms
to all the requirements of ISO/IEC 14143-1. Although the conformity evaluation procedure may also evaluate
the implementation of the recommendations of ISO/IEC 14143-1, the results of this evaluation shall not
contribute to the determination of conformity.
4.2.2 A conformity evaluation shall be valid only for the particular version of a Candidate FSM Method that
was the subject of the conformity evaluation process. Each new version of a method, including a Local
Customisation, is considered to be another Candidate FSM Method, and requires a separate conformity
evaluation. If a conformity evaluation team can identify the similarities and/or differences between a Candidate
FSM Method and a previously evaluated version of the same method, they may use the output report from a
previous conformity evaluation as the basis for the new conformity evaluation. If any non-conformities have
been reported for a previously evaluated version of the same Candidate FSM Method, then the conformity
evaluation team shall consider such non-conformities during the current conformity evaluation process.
NOTE If the conformity evaluation team bases a conformity evaluation on a previous evaluation report, then they
need to be aware of the risks involved as the two versions may have differences that have not been noted. The conformity
evaluation team needs to ensure that the net effect of all changes is taken into account during the evaluation.
4.2.3 The conformity evaluation team shall verify that the Candidate FSM Method Documentation is
complete, as defined in sub-clause 4.4.2.1, and correct for the version of the Candidate FSM Method being
evaluated.
4.2.4 The conformity evaluation team should liaise with the evaluation sponsor during the conformity
evaluation process.
4.2.5 If the owner can be contacted, then the conformity evaluation team shall:
a) liaise with the owner during the conformity evaluation process;
b) document the subject of the liaison with the owner, within the conformity evaluation report and where
appropriate cross-reference the provision or evaluation activity to which it relates.
4.2.6 The evaluation team shall determine whether information received from the owner during the liaison
would result in a different version of the method than that submitted for this conformity evaluation. In this case
section 4.2.2 shall apply.
4.2.7 If the owner of the Candidate FSM Method can be contacted, then the owner shall be provided with
the opportunity to respond to the findings of the conformity evaluation and to add comments to the conformity
evaluation report before its publication.
4.2.8 If the owner of the Candidate FSM Method does not respond to the findings of the conformity
evaluation report within a reasonable time period, then the conformity evaluation team may proceed with
publication of the report. This time period should be agreed upon by the owner and the conformity evaluation
team at the outset of the conformity evaluation process.
4.3 Evaluator characteristics
4.3.1 Evaluator organization
In cases of a third party conformity evaluation, the third party evaluator organization shall be competent for the
functions which they have to perform.
4 © ISO/IEC 2011 – All rights reserved

4.3.2 Conformity evaluation team
The conformity evaluation team shall be responsible for ensuring that all activities in the conformity evaluation
process are completed. These activities shall include, but are not limited to, the following:
a) developing the conformity evaluation plan;
b) developing or acquiring the conformity evaluation procedure;
c) developing or acquiring the conformity evaluation checklist;
d) performing the conformity evaluation procedure;
e) producing the conformity evaluation report.
NOTE Confidence in the evaluation result is directly related to the competence of the conformity evaluation team.
Annex A describes the characteristics of a competent conformity evaluation team, and the mechanisms that may be used
to demonstrate the team's competence to perform conformity evaluation in accordance with the requirements of this part of
ISO/IEC 14143.
4.4 Inputs to conformity evaluation
4.4.1 List of inputs
As a minimum, the inputs to the conformity evaluation process shall include the following:
a) parts 1 and 2 of ISO/IEC 14143;
b) Candidate FSM Method documentation;
c) conformity evaluation plan;
d) conformity evaluation procedure;
e) conformity evaluation checklist.
4.4.2 Candidate FSM Method documentation
4.4.2.1 The Candidate FSM Method documentation shall include all materials necessary for the proper
use of the Candidate FSM Method, in the same format and content that would be supplied to the users of the
method. Where a Candidate FSM Method is embedded within a software tool and the processes used to
measure software size are not explicit to the user, then in order to be evaluated, the Candidate FSM Method
shall include documentation to describe these processes. If the owner is contactable then the evaluation team
shall confirm that the Candidate FSM Method Documentation provided as input to the evaluation is correct
and complete. If the owner cannot be contacted then the evaluation sponsor and the conformity evaluation
team shall agree on the materials that will comprise the Candidate FSM Method documentation.
NOTE Such material may include manuals, guidelines, examples, case studies, and any other tools that are
necessary for proper use of the method.
4.4.2.2 The Candidate FSM Method documentation shall be uniquely identifiable and should clearly state
the:
a) name and version number of the Candidate FSM Method that it describes,
b) name(s) of author(s), if applicable,
c) date of publication, and
d) name and contact details of the publisher.
© ISO/IEC 2011 – All rights reserved 5

NOTE The process for evaluating the conformity of a Candidate FSM Method requires the unique identification of
both the Candidate FSM Method and of the version being evaluated. This identification requires information that is not
essential to the measurement of software size. Therefore, this part of ISO/IEC 14143 introduces exclusive requirements
which are not present in ISO/IEC 14143-1, but which are deemed essential to the conformity evaluation process. That is, a
Candidate FSM Method does not have to uniquely identify its documentation in order to be an FSM Method. However, in
order for the conformity evaluation process to be auditable, the report must be able to uniquely identify the Candidate FSM
Method documentation that was evaluated. This will only be possible if the Candidate FSM Method conforms to the
requirements of this clause.
4.4.3 Conformity evaluation plan
The conformity evaluation team shall develop the conformity evaluation plan in consultation with the
evaluation sponsor. At a minimum, it shall include the following:
a) activities, schedule and resources required for the conformity evaluation process;
b) list of inputs that uniquely identifies each of the inputs to the conformity evaluation process;
c) names and contact details of the conformity evaluation team members;
d) name and contact details of the evaluator organization, in the case of third party assessment;
e) name(s) and contact details of the evaluation sponsor(s);
f) roles and responsibilities of all persons involved in the conformity evaluation process;
g) relationship of the conformity evaluation team members and the evaluator organization to any other
parties involved.
4.4.4 Conformity evaluation procedure
The conformity evaluation team should develop the conformity evaluation procedure in consultation with the
evaluation sponsor. The conformity evaluation procedure shall provide detailed descriptions of:
a) each of the tasks and steps to be performed by the conformity evaluation team and the evaluation
sponsor as part of the conformity evaluation procedure (refer Section 4.5);
b) how the inputs are used within the conformity evaluation procedure to produce the conformity evaluation
output.
4.4.5 Conformity evaluation checklist
4.4.5.1 The conformity evaluation team should develop the conformity evaluation checklist in consultation
with the evaluation sponsor. They may use as a basis for their checklist, an existing checklist - such as the
one provided in Annex B. The conformity evaluation checklist shall consist of a set of evaluation questions that
can be used to evaluate the Candidate FSM Method against all the requirements of ISO/IEC 14143-1. The
conformity evaluation checklist may also include a set of additional evaluation questions that can be used to
evaluate the Candidate FSM Method against all the recommendations of ISO/IEC 14143-1. When determining
the conformity of a Candidate FSM Method, the conformity evaluation team shall use only the responses to
evaluation questions relating to the requirements of ISO/IEC 14143-1. The conformity evaluation team shall
decide the appropriate structure and presentation of the conformity evaluation checklist.
NOTE Annex B contains an example of a conformity evaluation checklist that satisfies the requirements of this part of
ISO/IEC 14143 by providing checks for the requirements of ISO/IEC 14143-1. It also exceeds these by providing checks
for the recommendations of ISO/IEC 14143-1.
4.4.5.2 The conformity evaluation checklist shall be structured so that:
a) it contains evaluation questions that correspond to each of the requirements of ISO/IEC 14143-1, against
which the characteristics of the Candidate FSM Method are evaluated,
6 © ISO/IEC 2011 – All rights reserved

b) each requirement corresponds to at least one evaluation question,
c) it includes a matrix that cross-references each requirement of ISO/IEC 14143-1 to the corresponding
evaluation questions, and
d) the set of evaluation questions that correspond to a particular requirement fully evaluate all aspects of
that requirement.
NOTE 1 ISO/IEC 14143-1 contains a set of requirements against which the conformity evaluation team evaluates the
Candidate FSM Method, but the structure of ISO/IEC 14143-1 does not facilitate an efficient conformity evaluation. The
conformity evaluation checklist is intended to provide a more effective mechanism for evaluating the conformity of a
Candidate FSM Method to ISO/IEC 14143-1. The checklist does this by providing a set of evaluation questions for which
there are clear and simple responses, such as ‘yes’ or ‘no’. To support a complete evaluation for each requirement in
ISO/IEC 14143-1, there must be at least one evaluation question in the checklist that evaluates the Candidate FSM
Method against that requirement. To evaluate conformity to a particular requirement, the checklist may contain several
evaluation questions.
NOTE 2 In some circumstances, an evaluation question may correspond to multiple requirements, but multiple
coverage evaluation questions should ideally be kept to a minimum.
4.4.5.3 If the conformity evaluation checklist includes evaluation questions which evaluate the
recommendations of ISO/IEC 14143-1, the evaluation checklist should be structured so that:
a) it contains evaluation questions that correspond to each of the recommendations of ISO/IEC 14143-1,
against which the characteristics of the Candidate FSM Method are evaluated,
b) each recommendation corresponds to at least one evaluation question,
c) the matrix also cross-references each recommendation of ISO/IEC 14143-1 to the corresponding
evaluation questions, and
d) the set of evaluation questions that correspond to a particular recommendation fully evaluate all aspects
of that recommendation.
4.4.5.4 If the conformity evaluation checklist contains evaluation questions relating to the
recommendations of ISO/IEC 14143-1, then these evaluation questions shall be grouped separately from
those relating to the requirements of ISO/IEC 14143-1.
4.4.5.5 The conformity evaluation checklist should be structured in terms of format, sequencing and
grouping of questions such that it facilitates the conformity evaluation procedure.
4.4.5.6 Each evaluation question on the conformity evaluation checklist shall include a cross-reference to
the corresponding provision(s) of ISO/IEC 14143-1.
4.4.5.7 Evaluation questions that correspond to optional requirements within ISO/IEC 14143-1 shall
include the phrase “if applicable”. Evaluation questions that do not correspond to optional requirements shall
not contain the phrase “if applicable”.
4.5 Tasks and steps of the conformity evaluation procedure
4.5.1 Guidance
Parties making use of the Candidate FSM Method should be able to derive from the contents of the Candidate
FSM Method a common understanding of its meaning and intent. The Candidate FSM Method should be so
clear and precise that it results in accurate and uniform interpretation.
If the owner can be contacted, then difficulties arising from 4.5.1 during the evaluation shall be referred to the
owner of the Candidate FSM Method for clarification. If the difficulties still cannot be resolved then the
evaluation question shall be deemed as not being able to be resolved.
© ISO/IEC 2011 – All rights reserved 7

4.5.2 Tasks and Steps
The conformity evaluation procedure shall include the tasks and steps listed below.
a) The conformity evaluation steps listed below shall be conducted for each evaluation question.
1) If the evaluation question has ‘if applicable’ as an option, then determine whether this evaluation
question is applicable to the Candidate FSM Method being evaluated. If not, then this evaluation
question does not contribute to the evaluation result for this method. Record that the evaluation
question does not contribute to the evaluation. No further steps for this evaluation question are
necessary.
2) Identify all relevant information in the Candidate FSM Method documentation. If no relevant
information can be located, and is still not located after liaising with the owner then this evaluation
question shall be deemed as not being able to be resolved, - proceed to step 4.5.2 a) 5).
3) Record the location of the relevant information (identified in step 4.5.2 a)
...