ISO/IEC 14143-2:2002

INTERNATIONAL ISO/IEC
STANDARD 14143-2
First edition
2002-11-15
Information technology — Software
measurement — Functional size
measurement —
Part 2:
Conformity evaluation of software size
measurement methods to
ISO/IEC 14143-1:1998
Technologies de l'information — Mesurage du logiciel — Mesurage de
la taille fonctionnelle —
Partie 2: Évaluation de la conformité des méthodes de mesure de taille
de logiciel à l'ISO/CEI 14143-1:1998

Reference number
©
ISO/IEC 2002
PDF disclaimer
This PDF file may contain embedded typefaces. In accordance with Adobe's licensing policy, this file may be printed or viewed but shall not
be edited unless the typefaces which are embedded are licensed to and installed on the computer performing the editing. In downloading this
file, parties accept therein the responsibility of not infringing Adobe's licensing policy. The ISO Central Secretariat accepts no liability in this
area.
Adobe is a trademark of Adobe Systems Incorporated.
Details of the software products used to create this PDF file can be found in the General Info relative to the file; the PDF-creation parameters
were optimized for printing. Every care has been taken to ensure that the file is suitable for use by ISO member bodies. In the unlikely event
that a problem relating to it is found, please inform the Central Secretariat at the address given below.

©  ISO/IEC 2002
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized in any form or by any means, electronic
or mechanical, including photocopying and microfilm, without permission in writing from either ISO at the address below or ISO's member body
in the country of the requester.
ISO copyright office
Case postale 56 • CH-1211 Geneva 20
Tel. + 41 22 749 01 11
Fax + 41 22 749 09 47
E-mail copyright@iso.ch
Web www.iso.ch
Printed in Switzerland
ii © ISO/IEC 2002 – All rights reserved

Contents Page
Foreword.iv
Introduction .v
1 Scope.1
2 Normative references.2
3 Terms and definitions.2
4 Conformity evaluation.4
4.1 Overview.4
4.2 Evaluator characteristics.5
4.2.1 Evaluator organization.5
4.2.2 Conformity evaluation team.5
4.3 Inputs to conformity evaluation .6
4.3.1 List of inputs.6
4.3.2 Candidate FSM Method documentation .6
4.3.3 Conformity evaluation plan.7
4.3.4 Conformity evaluation procedure .7
4.3.5 Conformity evaluation checklist.7
4.4 Tasks and steps of the conformity evaluation procedure.9
4.4.1 Guidance.9
4.4.2 Tasks and Steps.9
4.5 Conformity evaluation output.13
4.6 Conformity evaluation result .14
Annex A (informative) Evaluator capability .15
A.1 Conformity evaluation team.15
A.2 Demonstration of competence .15
A.2.1 Individual declaration .15
A.2.2 Evaluation practice .15
A.2.3 Software size measurement concepts.16
A.2.4 Software size measurement practice.16
Annex B (informative) Example of a conformity evaluation checklist .17
B.1 Introduction .17
B.1.1 Background .17
B.1.2 Structure .17
B.1.3 Instructions.17
B.2 Conformity evaluation checklist.17
B.2.1 Part 1 - requirements .17
B.2.2 Part 2 - recommendations.25
B.2.3 Part 3 - cross-reference between provisions of ISO/IEC 14143-1:1998 and evaluation questions .27
Annex C (informative) Example of a conformity evaluation report .29
C.1 Introduction .29
C.2 Executive summary .29
C.3 Conformity evaluation checklist.30
C.4 Original conformity evaluation plan.31
C.5 Justifications for results .31
C.6 Conformity evaluation procedure .31
C.7 Qualifications of conformity evaluation team.31
C.8 Record of the liaison with the owner during the conformity evaluation process.32

© ISO/IEC 2002 – All rights reserved iii

Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical Commission)
form the specialized system for worldwide standardization. National bodies that are members of ISO or IEC
participate in the development of International Standards through technical committees established by the
respective organization to deal with particular fields of technical activity. ISO and IEC technical committees
collaborate in fields of mutual interest. Other international organizations, governmental and non-governmental, in
liaison with ISO and IEC, also take part in the work. In the field of information technology, ISO and IEC have
established a joint technical committee, ISO/IEC JTC 1.
International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 3.
The main task of the joint technical committee is to prepare International Standards. Draft International Standards
adopted by the joint technical committee are circulated to national bodies for voting. Publication as an International
Standard requires approval by at least 75 % of the national bodies casting a vote.
Attention is drawn to the possibility that some of the elements of this part of ISO/IEC 14143 may be the subject of
patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent rights.
ISO/IEC 14143-2 was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 7, Software and system engineering.
ISO/IEC 14143 consists of the following parts, under the general title Information technology — Software
measurement — Functional size measurement:
— Part 1: Definition of concepts
— Part 2: Conformity evaluation of software size measurement methods to ISO/IEC 14143-1:1998
— Part 3: Verification of functional size measurement methods
— Part 4: Reference model
— Part 5: Determination of functional domains for use with functional size measurement
Annexes A, B and C of this part of ISO/IEC 14143 are for information only.
iv © ISO/IEC 2002 – All rights reserved

Introduction
Functional Size Measurement (FSM) is a technique used to measure the size of software by quantifying the
1)
Functional User Requirements of the software . The first published method to embrace this concept was Function
Point Analysis, developed by Allan Albrecht in the late 1970s. Since then, numerous extensions and variations of
the original method have been developed. The end user may have many variants from which to choose - each with
its own advantages in specific situations. ISO/IEC 14143-1:1998 was developed to define the concepts of FSM
and provides a basis against which the user can compare all variants. This part of ISO/IEC 14143 was developed
to provide a process for checking whether a Candidate FSM Method conforms to the provisions of ISO/IEC 14143-
1:1998. The output from this process can assist prospective users of the Candidate FSM Method in judging
whether it is appropriate to their needs.
_________________________
1) Refer to ISO/IEC 14143-1:1998, Information technology — Software measurement — Functional size measurement —
Part 1: Definition of concepts.

© ISO/IEC 2002 – All rights reserved v

INTERNATIONAL STANDARD ISO/IEC 14143-2:2002(E)

Information technology — Software measurement — Functional
size measurement —
Part 2:
Conformity evaluation of software size measurement methods to
ISO/IEC 14143-1:1998
1 Scope
1.1 This part of ISO/IEC 14143:
a) establishes a framework for the conformity evaluation of a Candidate FSM Method against the provisions
of ISO/IEC 14143-1:1998,
b) describes a process for conformity evaluation of whether a Candidate FSM Method meets the (type)
requirements of ISO/IEC 14143-1:1998 such that it is an actual FSM method, i.e. they are of the same
type,
c) describes the requirements for performing a conformity evaluation in order to ensure repeatability of the
conformity evaluation process, as well as consistency of decisions on conformity and the final result,
d) aims to ensure that the output from the conformity evaluation process is objective, impartial, consistent,
repeatable, complete and auditable,
e) provides informative guidelines (refer Annex A) for determining the competence of the conformity
evaluation teams,
f) provides an example checklist (refer Annex B) to assist in the conformity evaluation of a Candidate FSM
Method, and
g) provides an example template (refer Annex C) for the conformity evaluation report.
Conformity evaluations are conducted by a conformity evaluation team that has the competencies described
in this part of ISO/IEC 14143. This part of ISO/IEC 14143 assumes familiarity with the concepts and
definitions described in ISO/IEC 14143-1:1998.
The conformity evaluation is performed by cross-referencing each component of a Candidate FSM Method
against the corresponding provisions of ISO/IEC 14143-1:1998. The components of the Candidate FSM
Method are then evaluated for their conformity.
The output from the conformity evaluation includes a decision for each provision evaluated. Only the
requirements (shalls) are considered when determining if the Candidate FSM Method conforms to ISO/IEC
14143-1:1998. The recommendations (shoulds) of ISO/IEC 14143-1:1998 may also be investigated to provide
additional information to end users of the Candidate FSM Method.
The output from the conformity evaluation process is the conformity evaluation report. The report may be used
to:
a) inform end users that a Candidate FSM Method conforms to ISO/IEC 14143-1:1998 in accordance with
this part of ISO/IEC 14143, and is therefore an FSM Method, and
b) assist end users in making informed judgements about which method best suits their needs.
© ISO/IEC 2002 – All rights reserved 1

1.2 This part of ISO/IEC 14143 may be used for first party (supplier), second party (user or purchaser) or third
party (independent body), conformity evaluations.

NOTE  The relationship between the owner, sponsor and evaluator depends on the type of evaluation that is performed, i.e.
first, second or third party.
1.3 While conformance of a Candidate FSM Method to ISO/IEC 14143-1:1998 may be claimed without
referencing this part of ISO/IEC 14143, this part provides a conformity evaluation process that may be
used to add credibility to such claims. This part places requirements upon a conformity evaluation
procedure and is usable for first, second or third party claims of conformance. Its provisions are
particularly suitable for those who require third party conformity evaluation. Customers desiring to use or
aquire an FSM Method evaluated for conformance in accordance with this part, should explicitly cite this
International Standard when requesting the evaluation.
1.4 Conformity evaluation should not be construed as guaranteeing that the FSM Method is free from non-
conformities; it only signifies that evidence of non-conformance was not found during the conformity
evaluation process.
1.5 A Candidate FSM Method shall be determined as conforming if it successfully completes a conformity
evaluation procedure which satisfies the requirements of sub-clause 4.4 of this part of ISO/IEC 14143.

NOTES
1 Conformity of a Candidate FSM Method is based on evaluation against requirements of ISO/IEC 14143-1:1998. This part
of ISO/IEC 14143 defines a process that may be used in evaluating whether a Candidate FSM Method conforms to the
requirements of ISO/IEC 14143-1:1998.
2 An International Standard on conformity evaluation or test methods, such as this one, does not imply any obligation to carry
out any kind of test. It defines the process by which the evaluation, if required and referred to (for example in a regulation,
or in contract documents), should be carried out.

2 Normative references
The following normative documents contain provisions which, through reference in this text, constitute provisions of
this part of ISO/IEC 14143. For dated references, subsequent amendments to, or revisions of, any of these
publications do not apply. However, parties to agreements based on this part of ISO/IEC 14143 are encouraged to
investigate the possibility of applying the most recent editions of the normative documents indicated below. For
undated references, the latest edition of the normative document referred to applies. Members of ISO and IEC
maintain registers of currently valid International Standards.
ISO/IEC 14143-1:1998, Information technology - Software measurement - Functional size measurement - Part 1:
Definition of concepts
ISO/IEC Guide 2:1996, Standardization and related activities – General vocabulary

3 Terms and definitions
For the purposes of this part of ISO/IEC 14143, the terms and definitions given in ISO/IEC 14143-1:1998 and the
following apply.
2 © ISO/IEC 2002 — All rights reserved

3.1
Candidate FSM Method
documented software size measurement method submitted for conformity evaluation according to ISO/IEC
14143-1:1998
3.2
evaluation checklist
list of questions, each of which is designed to check for conformity of a product, process or service to one or more
provisions within a particular International Standard
NOTE In the case of this part of ISO/IEC 14143, the product being evaluated for conformance is the Candidate FSM
Method and the provisions are those of ISO/IEC 14143-1:1998.

3.3
evaluation procedure
series of tasks and steps that, when completed, enable the evaluation team to determine if the product, process or
service being evaluated is conformant to a particular standard

3.4
evaluation sponsor
person or organization that requires the evaluation to be performed and provides financial or other resources to
carry it out
3.5
exclusive requirement
(deprecated: mandatory requirement): requirement of a normative document that must necessarily be fulfilled in
order to comply with that document
[ISO/IEC Guide 2:1996, definition 7.5.1]

3.6
optional requirement
requirement of a normative document that must be fulfilled in order to comply with a particular option permitted by
that document
NOTE An optional requirement may be either:
a) one of two or more alternative requirements, or
b) an additonal requirement that must be fulfilled only if applicable and may otherwise be disregarded.
[ISO/IEC Guide 2:1996, definition 7.5.2]

3.7
owner
person or organization that owns the copyright for the Candidate FSM Method

3.8
provision
expression in the content of a normative document, that takes the form of a statement, an instruction, a
recommendation or a requirement
© ISO/IEC 2002 — All rights reserved 3

NOTE These types of provision are distinguished by the form of wording they employ e.g. instructions are expressed in the
imperative mood, recommendations by the use of the auxilliary "should", and requirements by the use of the auxiliary "shall".
[ISO/IEC Guide 2:1996, definition 7.1]

3.9
recommendation
provision that conveys advice or guidance
[ISO/IEC Guide 2:1996, definition 7.4]

3.10
requirement
provision that conveys criteria to be fulfilled
[ISO/IEC Guide 2:1996, definition 7.5]
NOTE A requirement is denoted by the word “shall” and when used includes both the exclusive and applicable optional
requirements.
4 Conformity evaluation
4.1 Overview
4.1.1 The objective of a conformity evaluation shall be to determine if the Candidate FSM Method conforms to all
the requirements of ISO/IEC 14143-1:1998. Although the conformity evaluation procedure may also evaluate the
implementation of the recommendations of ISO/IEC 14143-1:1998, the results of this evaluation shall not contribute
to the determination of conformity.
4.1.2 A conformity evaluation shall be valid only for the particular version of a Candidate FSM Method that was
the subject of the conformity evaluation process. Each new version of a method, including a Local Customisation,
is considered to be another Candidate FSM Method, and requires a separate conformity evaluation. If a conformity
evaluation team can identify the similarities and/or differences between a Candidate FSM Method and a previously
evaluated version of the same method, they may use the output report from a previous conformity evaluation as the
basis for the new conformity evaluation. If any non-conformities have been reported for a previously evaluated
version of the same Candidate FSM Method, then the conformity evaluation team shall consider such non-
conformities during the current conformity evaluation process.
NOTE If the conformity evaluation team bases a conformity evaluation on a previous evaluation report, then they need to
be aware of the risks involved as the two versions may have differences that have not been noted. The conformity evaluation
team needs to ensure that the net effect of all changes is taken into account during the evaluation.
4.1.3 The conformity evaluation team shall verify that the Candidate FSM Method Documentation is complete, as
defined in sub-clause 4.3.2.1, and correct for the version of the Candidate FSM Method being evaluated.
4.1.4 The conformity evaluation team should liaise with the evaluation sponsor during the conformity evaluation
process.
4.1.5 If the owner can be contacted, then the conformity evaluation team shall:
a) liaise with the owner during the conformity evaluation process;
4 © ISO/IEC 2002 — All rights reserved

b) document the subject of the liaison with the owner, within the conformity evaluation report and where
appropriate, cross-reference the provision or evaluation activity to which it relates.
4.1.6 The evaluation team shall determine whether information received from the owner during the liaison would
result in a different version of the method than that submitted for this conformity evaluation. In this case section
4.1.2 shall apply.
4.1.7 If the owner of the Candidate FSM Method can be contacted, then the owner shall be provided with the
opportunity to respond to the findings of the conformity evaluation and to add comments to the conformity
evaluation report before its publication.
4.1.8 If the owner of the Candidate FSM Method does not respond to the findings of the conformity evaluation
report within a reasonable time period, then the conformity evaluation team may proceed with publication of the
report. This time period should be agreed upon by the owner and the conformity evaluation team at the outset of
the conformity evaluation process.

4.2 Evaluator characteristics
4.2.1 Evaluator organization
In cases of a third party conformity evaluation, the third party evaluator organizations shall be competent for the
functions which they have to perform.
4.2.2 Conformity evaluation team
The conformity evaluation team shall be responsible for ensuring that all activities in the conformity evaluation
process are completed. These activities shall include, but are not limited to, the following:
a) developing the conformity evaluation plan;
b) developing or acquiring the conformity evaluation procedure;
c) developing or acquiring the conformity evaluation checklist
d) performing the conformity evaluation procedure;
e) producing the conformity evaluation report.
NOTE Confidence in the evaluation result is directly related to the competence of the conformity evaluation team. Annex A
describes the characteristics of a competent conformity evaluation team, and the mechanisms that may be used to demonstrate
the team's competence to perform conformity evaluation in accordance with the requirements of this part of ISO/IEC 14143.
© ISO/IEC 2002 — All rights reserved 5

4.3 Inputs to conformity evaluation
4.3.1 List of inputs
As a minimum, the inputs to the conformity evaluation process shall include the following:
a) parts 1 and 2 of ISO/IEC 14143;
b) Candidate FSM Method documentation;
c) conformity evaluation plan;
d) conformity evaluation procedure;

e) conformity evaluation checklist.

4.3.2 Candidate FSM Method documentation
4.3.2.1  The Candidate FSM Method documentation shall include all materials necessary for the proper use of the
Candidate FSM Method, in the same format and content that would be supplied to the users of the method. Where
a Candidate FSM Method is embedded within a software tool and the processes used to measure software size are
not explicit to the user, then in order to be evaluated, the Candidate FSM Method shall include documentation to
describe these processes. If the owner is contactable, then the evaluation team shall confirm that the Candidate
FSM Method documentation provided as input to the evaluation is correct and complete. If the owner cannot be
contacted, then the evaluation sponsor and the conformity evaluation team shall agree on the materials that will
comprise the Candidate FSM Method documentation.
NOTE Such material may include manuals, guidelines, examples, case studies, and any other tools that are necessary for
proper use of the method.
4.3.2.2  The Candidate FSM Method documentation shall be uniquely identifiable and should clearly state the:
a) name and version number of the Candidate FSM Method that it describes,
b) name(s) of author(s), if applicable,
c) date of publication, and
d) name and contact details of the publisher.
NOTE The process for evaluating the conformity of a Candidate FSM Method requires the unique identification of both the
Candidate FSM Method and of the version being evaluated. This identification requires information that is not essential to the
measurement of software size. Therefore, this part of ISO/IEC 14143 introduces exclusive requirements which are not present
in ISO/IEC 14143-1:1998, but which are deemed essential to the conformity evaluation process. That is, a Candidate FSM
Method does not have to uniquely identify its documentation in order to be an FSM Method. However, in order for the
conformity evaluation process to be auditable, the report must be able to uniquely identify the Candidate FSM Method
documentation that was evaluated. This will only be possible if the Candidate FSM Method conforms to the requirements of this
clause.
6 © ISO/IEC 2002 — All rights reserved

4.3.3 Conformity evaluation plan
The conformity evaluation team shall develop the conformity evaluation plan in consultation with the evaluation
sponsor. At a minimum, it shall include the following:
a) activities, schedule and resources required for the conformity evaluation process;
b) list of inputs that uniquely identifies each of the inputs to the conformity evaluation process;
c) names and contact details of the conformity evaluation team members;
d) name and contact details of the evaluator organization, in the case of third party assessment;
e) name(s) and contact details of the evaluation sponsor(s);

f) roles and responsibilities of all persons involved in the conformity evaluation process;
g) relationship of the conformity evaluation team members and the evaluator organization to any other parties
involved.
4.3.4 Conformity evaluation procedure
The conformity evaluation team should develop the conformity evaluation procedure in consultation with the
evaluation sponsor. The conformity evaluation procedure shall provide detailed descriptions of:
a) each of the tasks and steps to be performed by the conformity evaluation team and the evaluation sponsor as
part of the conformity evaluation procedure (refer section 4.4)
b) how the inputs are used within the conformity evaluation procedure to produce the conformity evaluation output.

4.3.5 Conformity evaluation checklist
4.3.5.1  The conformity evaluation team should develop the conformity evaluation checklist in consultation with the
evaluation sponsor. They may use as a basis for their checklist, an existing checklist - such as the one provided in
Annex B. The conformity evaluation checklist shall consist of a set of evaluation questions that can be used to
evaluate the Candidate FSM Method against all the requirements of ISO/IEC 14143-1:1998. The conformity
evaluation checklist may also include a set of additional evaluation questions that can be used to evaluate the
Candidate FSM Method against all the recommendations of ISO/IEC 14143-1:1998. When determining the
conformity of a Candidate FSM Method, the conformity evaluation team shall use only the responses to evaluation
questions relating to the requirements of ISO/IEC 14143-1:1998. The conformity evaluation team shall decide the
appropriate structure and presentation of the conformity evaluation checklist.
NOTE Annex B contains an example of a conformity evaluation checklist that satisfies the requirements of this part of
ISO/IEC 14143 by providing checks for the requirements of ISO/IEC 14143-1:1998. It also exceeds these by providing checks
for the recommendations of ISO/IEC 14143-1:1998.

© ISO/IEC 2002 — All rights reserved 7

4.3.5.2  The conformity evaluation checklist shall be structured so that:
a) it contains evaluation questions that correspond to each of the requirements of ISO/IEC 14143-1:1998, against
which the characteristics of the Candidate FSM Method are evaluated,
b) each requirement corresponds to at least one evaluation question,
c) it includes a matrix that cross-references each requirement of ISO/IEC 14143-1:1998 to the corresponding
evaluation questions, and
d) the set of evaluation questions that correspond to a particular requirement, fully evaluate all aspects of that
requirement.
NOTES
1 ISO/IEC 14143-1:1998 contains a set of requirements against which the conformity evaluation team evaluates the
Candidate FSM Method, but the structure of ISO/IEC 14143-1:1998 does not facilitate an efficient conformity evaluation. The
conformity evaluation checklist is intended to provide a more effective mechanism for evaluating the conformity of a Candidate
FSM Method to ISO/IEC 14143-1:1998. The checklist does this by providing a set of evaluation questions for which there are
clear and simple responses, such as ‘yes’ or ‘no’. To support a complete evaluation for each requirement in ISO/IEC 14143-
1:1998, there must be at least one evaluation question in the checklist that evaluates the Candidate FSM Method against that
requirement. To evaluate conformity to a particular requirement, the checklist may contain several evaluation questions.
2 In some circumstances, an evaluation question may correspond to multiple requirements, but multiple coverage
evaluation questions should ideally be kept to a minimum.

4.3.5.3  If the conformity evaluation checklist includes evaluation questions which evaluate the recommendations
of ISO/IEC 14143-1:1998, the evaluation checklist should be structured so that:
a) it contains evaluation questions that correspond to each of the recommendations of ISO/IEC 14143-1:1998,
against which the characteristics of the Candidate FSM Method are evaluated,
b) each recommendation corresponds to at least one evaluation question,
c) the matrix also cross-references each recommendation of ISO/IEC 14143-1:1998 to the corresponding
evaluation questions, and
d) the set of evaluation questions that correspond to a particular recommendation, fully evaluate all aspects of that
recommendation.
4.3.5.4  If the conformity evaluation checklist contains evaluation questions relating to the recommendations of
ISO/IEC 14143-1:1998, then these evaluation questions shall be grouped separately from those relating to the
requirements of ISO/IEC 14143-1:1998.

4.3.5.5  The conformity evaluation checklist should be structured in terms of format, sequencing and grouping of
questions such that it facilitates the conformity evaluation procedure.

4.3.5.6  Each evaluation question on the conformity evaluation checklist shall include a cross-reference to the
corresponding provision(s) of ISO/IEC 14143-1:1998.

4.3.5.7  Evaluation questions that correspond to optional requirements within ISO/IEC 14143-1:1998 shall include
the phrase "if applicable". Evaluation questions that do not correspond to optional requirements shall not contain
the phrase "if applicable".
8 © ISO/IEC 2002 — All rights reserved

4.4 Tasks and steps of the conformity evaluation procedure
4.4.1 Guidance
Parties making use of the Candidate FSM Method should be able to derive from the contents of the Candidate FSM
Method a common understanding of its meaning and intent. The Candidate FSM Method should be so clear and
precise that it results in accurate and uniform interpretation.
If the owner can be contacted, then difficulties arising from 4.4.1 during the evaluation shall be referred to the
owner of the Candidate FSM Method for clarification. If the difficulties still cannot be resolved then the evaluation
question shall be deemed as not being able to be resolved.
4.4.2 Tasks and Steps
The conformity evaluation procedure shall include the tasks and steps listed below.
a) The conformity evaluation steps listed below shall be conducted for each evaluation question.
1) If the evaluation question has ‘if applicable’ as an option, then determine whether this evaluation question
is applicable to the Candidate FSM Method being evaluated. If not, then this evaluation question does not
contribute to the evaluation result for this method. Record that the evaluation question does not contribute
to the evaluation. No further steps for this evaluation question are necessary.
2) Identify all relevant information in the Candidate FSM Method documentation. If no relevant information
can be located, and is still not located after liaising with the owner, then this evaluation question shall be
deemed as not being able to be resolved, - proceed to step 4.4.2 a) 5).
3) Record the location of the relevant information (identified in step 4.4.2 a) 2)) against the evaluation
question. Each recorded location shall include:
i) in the case of text, the page number, lowest level heading and paragraph or line number,
ii) in the case of tables, the page number, table name and row,
iii) in the case of diagrams, the page number, diagram name and number, or
iv) any other details necessary to locate the relevant information.
NOTE 1 Recording the information used to evaluate conformity to a provision of ISO/IEC 14143-1:1998 provides the
evaluation sponsor with a clear statement of precisely what was evaluated. For the purpose of auditing, it is recommended that
the identifier of the evaluation questions be recorded at the appropriate location on the Candidate FSM Method documentation.
4) Consider all located information as a whole and determine if it satisfies the requirements of the evaluation
question. If so, then the Candidate FSM Method shall pass this evaluation question - proceed to step
4.4.2 a) 7). If the evaluation question was not able to be resolved - proceed to step 4.4.2 a) 5) otherwise
proceed to step 4.4.2 a) 6).
5) If the evaluation question could not be resolved, then the conformity evaluation team shall record:
i) the locations of the information, or the absence of the information, that left the evaluation question
unable to be resolved, and
ii) the justification for the decision that left the evaluation question unable to be resolved.
© ISO/IEC 2002 — All rights reserved 9

6) If any evaluation question was not passed, then the conformity evaluation team shall record:
i) the locations of the information, or the absence of the information, which caused the evaluation
question not to be able to be passed, and
ii) the justification for the decision for not passing the evaluation question.
7) If the evaluation question was passed at every step in this procedure to reach this step, then the
conformity evaluation team shall record that the Candidate FSM Method passed this evaluation question.
NOTE 2 Figure 1 provides an informative diagrammatic representation of the evaluation procedure for each evaluation
question.
10 © ISO/IEC 2002 — All rights reserved

for each
evaluation
question
question does not
record "question
1) is question
contribute to
No is not
applicable?
evaluation
applicable”
Yes
2) can relevant
No
information be located?
(Clarify with owner if
necessary)
Yes
consider all located
3) record
are there additional No
information for this
location of locations with relevant
question
information?
information
Yes
5) record locations
4) does information of , or absence of
6) record locations
Unresolved
No satisfy requirements of
information that
of information
prevented
question?
prevented passing,
resolution, and
and justify
justify
Yes
7) record
question
passed
question not
question question
passed could not be
passed
resolved
Figure 1 — Example of diagrammatic procedure for use with each evaluation question
© ISO/IEC 2002 — All rights reserved 11

b) The conformity evaluation steps listed below shall be conducted for each provision in ISO/IEC 14143-1:1998.
1) Determine if the Candidate FSM Method has passed all corresponding evaluation questions.
2) If the Candidate FSM Method has passed all corresponding evaluation questions, then it shall be deemed-
to-satisfy this provision, and the result shall be recorded as the ‘provision has been satisfied’.
3) If the Candidate FSM Method has one or more corresponding evaluation questions which were not
passed, then the provision shall be deemed as not having been satisfied.
4) If the Candidate FSM Method has one or more corresponding evaluation questions which could not be
resolved, then the provision shall be deemed ‘unable to be evaluated’.
5) If the provision was deemed as not having been satisfied, then the conformity evaluation team shall record
the justification for the decision.
6) If the provision was deemed as ‘unable to be evaluated’, then the conformity evaluation team shall record
the justification for the decision.
NOTE 3 Figure 2 provides an informative diagrammatic representation of the evaluation procedure for each provision.
1) have all questions
for each provision of 4) were one or more questions
(corresponding to this provision) No 3) did any questions not pass? No
ISO/IEC 14143-1:1998 unable to be resolved?
passed?
Yes Yes Yes
2) record that 5) record 6) record
provision was justification for justification for
"satisfied" decision decision
provision unable
provision satisfied provision not satisfied
to be evaluated
Figure 2 — Example of diagrammatic procedure for use with each evaluation provision
NOTE 4 The conformity evaluation team must be able to justify their decision not to pass a provision. The justifications would
be required for the following types of decisions where the evaluation team were unable to:
• pass an evaluation question,
• resolve an evaluation question,
• deem a provision as satisfied, or
• evaluate a provision.
12 © ISO/IEC 2002 — All rights reserved

These justifications are essential to the review of the conformity evaluation process, both by the evaluation sponsor and by

those who may want to use a specific software sizing method. This is required in order for the conformity evaluation to be
understood and respected.
c) If all requirements, of ISO/IEC 14143-1:1998 have been recorded as having been satisfied, then the Candidate
FSM Method shall be deemed to have successfully completed this conformity evaluation procedure.
d) If any requirements, of ISO/IEC 14143-1:1998 have not been able to be recorded as being satisfied, or have
been recorded as being unable to be evaluated, then the Candidate FSM Method shall be deemed not to have
successfully completed this conformity evaluation procedure.
e) If the owner of the Candidate FSM Method has added comments to the conformity evaluation report, then the
evaluation team shall review those comments to determine if any steps of the conformity evaluation process
need to be repeated before the conformity evaluation report is published.
4.5 Conformity evaluation output
4.5.1  The conformity evaluation process shall include the production of a conformity evaluation report. The report
is the means of recording the detailed evidence to support the conformity evaluation decision against each
provision of ISO/IEC 14143-1:1998.

4.5.2  As a minimum, the conformity evaluation report shall include the following sections:
a) executive summary, which shall include as a minimum the following information:
1) full identification details of the Candidate FSM Method;
2) name of the evaluator organization;
3) type of assessment ie. first, second or third party;
4) date(s) of the conformity evaluation;
5) result of the evaluation.
b) completed conformity evaluation checklist;
c) results (including all the information that contributed to any decisions made);
d) conformity evaluation plan;
e) justifications for decisions where a requirement was not deemed as satisfied or was unable to be evaluated;
f) conformity evaluation procedure;
g) qualifications of conformity evaluation team;
h) record of liaison with the owner during the conformity evaluation process.

4.5.3  The conformity evaluation plan section shall include the original conformity evaluation plan, and describe
and justify any deviations from that plan.
© ISO/IEC 2002 — All rights reserved 13

4.5.4  For each provision that was not deemed as satisfied or was unable to be evaluated, the justification for
results section shall include the following:
a) list of evaluation questions that contributed to that decision;
b) locations of all information that contributed to that decision;
c) justification for that decision.
4.5.5  The section describing the qualifications of conformity evaluation team shall contain information to assist
both the evaluation sponsor and the readers of the report to assess the competence of the conformity evaluation
team.
NOTE Annex C contains an example template of a conformity evaluation report that exhibits the minimum requirements of
this part of ISO/IEC 14143.
4.6 Conformity evaluation result
If, after applying the requirements of this pa
...