ISO 10303-31:1994

INTERNATIONAL
ISO
STANDARD
10303-31
First edition
1994-12-15
Industrial automation Systems and
integration - Product data representation
and exchange -
Part 31:
Conformance testing methodology and
framework: General concepts
Systemes d ’automatisation
industrielle et intkgration - Repr&en ta tion
et khange de donnkes de
produits -
Partie 3 1: Mk thodologie et cadre g&Gral pour /es essais de conformitk
Concepts g&Graux
Reference nun-her
ISO 10383-31:1994(E)
Contents
Page
1 Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Applicability of this series of Parts . . . . . . . . . . . . . . . . . . . . . . . . .
1.1
12 . Applicability of this part of ISO 10303 . . . . . . . . . . . . . . . . . . . . . . .
13 . Coverage of this series of Parts . . . . . . . . . . . . . . . . . . . . . . . . . . .
14 . Coverage of this part of ISO 10303 . . . . . . . . . . . . . . . . . . . . . . . . .
2 Normative references . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
31 . Terms defined in ISO 10303-1 . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. Other definitions . . . . . . . . . . . . . . . . . . . . . * . . . . . . . . . . . . .
32 4
3.2.1 abstract test case (ATC)
........................... 4
3.2.2 abstract test group
...............................
3.2.3 abstract test method
.............................. 4
(laboratory) accreditation
3.2.4
........................... 5
3.2.5 accreditation body
...............................
assessor
3.2.6 ,
..................................... 5
3.2.7 attestation of conformity
........................... 5
3.2.8 basic tests.
................................... 5
3.2.9 capabilities of an IUT
............................. 5
3.2.10 capability tests
.................................
3.2.11 certificate of conformance
........................... 5
3.2.12 certificate of conformity; certificate of conformance
............. 5
3.2.13 certification body
............................... 5
3.2.14 certification mark
............................... 5
3.2.15 certification of conformity
........................... G
3.2.16 certification System
.............................. 6
3.2.17
client (of a testing laboratory)
........................ 6
3.2.18 comparability (of results)
........................... 6
3.2.19
conformance
.................................. G
3.2.20 conformance assessment process
....................... 6
3.2.21
conformance log
................................ 6
>
3.2.22 conformance testing
.............................. 6
3.2.23
(conformance) test report
........................... 6
0 ISO 1994
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or
utilized in any ferm or by any means, electronie or mechanical, including photocopying and
microfilm, without permission in writing from the publisher.
International Organization for S tandardization
Case Postale 56 l CH- 1211 Geneve 20 l Switzerland
Printed in Switzerland
ii
ISO lom-31:1994(E)
c ISO
........................... 6
3.2.24 conforming implementation
............................ 6
3.2.25 conformity; conformance
control board .
3.2.26
.......................... 6
3.2.27 declaration of conformance
. 7
declaration of conformity; (manufacturer ’s) declaration of conformance
3.2.28
executable test case .
3.2.29
.............................. 7
3.2.30 executable test Suite
3.2.31 fail (verdict) .
............................... 7
3.2.32 falsification testing
3.2.33 Implementation Under Test (IUT) .
3.2.34 inconclusive (verdict ) .
in-hause testing .
3.2.35
................. 7
3.2.36 manufacturer ’s decla,ration of conformance
................................
3.2.37 non-conformance
non-conformity; non-conformance .
3.2.38
.................................. 7
3.2.39 pass (verdict)
PIXIT proforma . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2.40
3.2.41 postprocessor .
3.2.42 preprocessor . 8
3.2.43 proficiency testing .
Protocol Implementation extra Information for Testing (PIXIT) . 8
3.2.44
3.2.45 repeatability (of results) .
3.2.46 resolution tests .
3.2.47 selected abstract test Suite . 8
3.2.48 selected executable test Suite .
System Under Test (SUT) . 8
3.2.49
3.2.50 test campaign. .
..................................... 8
3.2.51 testcase
3.2.52 test case error .
test purpose . 9
3.2.53
3.2.54 test realiser .
test report . 9
3.2.55
test verdict . 9
3.2.56
3.2.57 testing laborstory .
(test) verdict . 9
3.2.58
3.2.59 verdict criteria, .
............................... 9
3.2.60 verification testing
4 Abbreviations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
5 Conformance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
51 . The meaning of conformance in ISO 10303 . . . . . . . . . . . . . . . . . . . . 10
Conformance requirements . r ,, 0 . . o l . D l . o o n . n . . n e . . . . . 0 . . 10
52 .
. Protocol Implementation Conformance Statement . . m . . l . . . . . . . . . . 10
54 . A conforming System . . e . . . . . . . . . . . . . . . . . . . . . . . . . . . l . .
6 Conformance testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
. . .
c ISO
. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
62 . Types of conformance tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
Basic tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
6.2.1
6.2.2 Capability tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
. Protocol Implementation extra InformaJtion for Testing . . . . . . . . . . . . .
. Conformance assessment process overview . . . . . . . . . . . . . . . . . . . . .
65 . Preparation for testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. Test campaign . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
67 . Analysis of results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
. Conformance test report production . . . . . . . . . . . . . . . . . . . . . . . . 16
. Intrinsic properties of the conformance assessment process . . . . . . . . . . . .
Repeatabilitv of results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
6.9.1
6.9.2 Comparability of results . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Auditability of results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
6.9.3
c
7 Abstract test methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
8 Abstract and executable test suites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
81 . Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
82 . Test purposes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. Abstract test cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
84 . Executable test cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
. Relaltionship between abstract and executable test cases . . . . . . . . . . . . . 20
Annexes
A Information Object registration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Optional conformance requirement s . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
B
Resolution tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
C
Supporting organisations
D
................................. 24
D.l Introduction
.....................................
D.l.l Purpose 24
.....................................
D.1.2 Scope .
D.1.3 Intended readership
..............................
Background.
D.l.4 25
..................................
D.1.5 General philosophy of conformance testing
................. 25
Responsible authorities
D.2 25
...............................
D.2.1 Control board
.................................
D.2.2 Accreditation body 28
...............................
D.2.3 Testing laboratory
...............................
D.2.4 Certification body ,, . ,, 29
.............. ............
D.3 Administration and certification 30
..........................
D.3.1 Guidelines for role of the certification body
................. 30
D.3.2 Sponsoring a laboratory accreditation programme
............. 30
__ _
ISO 10303=31:1994(E)
c ISO
Conditions for testing laboratorv accreditation .
D.3.3
................. 31
D.3.4 Criteria for accrediting testing laboratories
Testing laboratory assessment .
D.3.5
D.4 The process of certification . 34
Initial contact . 34
D.4.1
Certificate of conformity content and presentation .
D.4.2
National responsibilities . 35
D.4.3
D.4.4 International (ISO TC184/SC4) responsibilities . . . . . . . . . . . . . . . 35
. . . . . . . . . . . . . . . . . . . 38
E Bibliography . . . . . . . .
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
Figures
1 Relationship between conformance testing Standards and other series of Parts . . . . 2
2 Overview of the conformance assessment process . . . . . . . . . . . . . . . . . . . . 15
Example structure of an abstract test Suite . . . . . . . . . . . . . . . . . . . . . . . 19
D-1 Infrastructure for conformance testing . . . . . . . . . . . . . . . . . . . . . . . . . . 26
D.2 Sample certificate of conformity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
D.3 Sample form for additional claims made by the client . . . . . . . . . . . . . . . . . .
V
c ISO
ISO loso3-31:1994(E) 0
Foreword
The International Organization for Standa,rdization (ISO) is a worldwide federation of national
Standards bodies (ISO member bodies). The work of prepa,ring International Standards is nor-
mally carried out t hrough IS 0 t echnical commit t ees. Esch member body interested in a subject
for which a technical committee has been established has the right to ‘be represented on that
International organizations, governmental and non-governmental, in liaison with
committee.
ISO, also take part in the work. ISO collaborates closely with the International Electrotechnical
on all matters of electrotechnical standardization.
Commission (IEC)
Draft International Standards adopted by technical committees are circulated to the member
Publication as an International Sta)ndard requires approval by at least 75%
bodies for voting.
of the member bodies casting a vote.
International Standard ISO 10303-31 was prepared by Technical Committee ISO/TC 184, 11~-
Subcommittee SC2, Industrial data and global
dustrial automation Systems and integration,
manufacturing programnzing languages.
ISO 10303 consists of the following Parts under the general title Industrial automation Systems
and integration - Product data representation and ezchange:
- Part 1, Overview and fundamental principles;
- Part 11, Description methods: The KWRWS 1a)nguage reference manual;
- Part 21, Implementation methods: Clear text encoding of the exchange structure;
- Part 22, Implementation methods: Standard data a#ccess interface specification;
- Part 31, Conformance testing methodology and framework: General concepts;
- Part 32, Conformance testing methodology and framework: Reyuirements on testing
laboratories and client s;
- Part 41, Integrated generic resources: Fundamentals of product description and support;
- Part 42, Integrated generic resources: Geometrie and topological represent ation;
- Part 43, Integrated generic resources: Representation structures;
- Part 44, Integrated generic resources: Product structure configuration;
- Part 45, Integrated generic resources: Materials;
-
Part 46, Integrated generic resourceso Visual presentation;
- Part 47, Integrated generic. resources: Shape Variation tolerantes;
- Part 49, Integrated generic resources: Process structure and properties;
vi
ISO 1030%31:1994(E)
c ISO
-
Part 101, Integrated application resources: Draughting;
-
Part 104, Integrated application resources: Finite element analysis;
- Part 105, Integrated application resources: Kinematics;
- Part 201, Application protocol: Explicit draughting;
- Part 202, Application protocol: Associative draughting;
-- Part 203, Application protocol: Configuration controlled design;
- Part 207, Application protocol: Sheet metal die planning and design;
- Part 210, Application protocol: Printed circuit assembly product design data;
- Part 213, Application protocol: Numerital control process Plans for machined Parts.
The structure of this International Standard is described in ISO 10303-1. The numbering of the
Parts of this International Standard reflects its structure:
- Part 11 speci-fies the description methods;
- Parts 21 and 22 specify the implementation methods;
- Parts 31 and 32 specify the conformance testing methodology and framework;
- Parts 41 to 49 specify the integrated generic resources;
- Parts 101 to 105 specifv the integrated application resources;
”
- Parts 201 to 213 specify the application protocols.
Should further Parts be published, they will follow the Same numbering Pattern.
Annex A forms an integral part of this part of ISO 10303. Annexes B, C, D, and E are for
information only.
vii
c ISO
ISO mo%31:1994(E)
Introduction
ISO 10303 is an International Sta$ndard for the computer-interpretahle representation and ex-
Change of product data. The objective is t ’o provide a neutral mechanism capable of describing
product data throughout the life cycle of a product independent from any pa,rticular System.
The nature of this description makes it suitable not onlv for neutral file exchange, but) also as a
e
basis for implementing and sharing product databases and archiving.
This International Standard is organized as a series of Parts, each published separately. The
Parts of ISO 10303 fall into one of the following series: description methods, integrated resources,
application protocols, abstract test suites, implementation methods, and conformance testing.
The series are described in ISO 10303-1. This part of ISO 10303 is a member of the conformance
testing series.
This part of ISO 10303 introduces the series of Parts of ISO 10303 devoted to conformance test-
ing, provides a framework and describes the general concepts for conformance testing of imple-
mentations of ISO 10303. This part of ISO 10303 is based in palrt upon material in ISO 9646-1
which provides the Same function for conformance testing in Open Systems Interconnection.
The concepts have been modified for use in this particular dornain. This part therefore plays
a similar role for this series of Parts as does ISO 9646-1 for ISO 9646. Subsequent Parts con-
cerning conformance testing of implementations of ISO 10303 are also based upon subsequent
Parts of ISO 9646. ISO 10303-31 applies not only to exchange structures but, with the current
knowledge available, to other implementation methods also.- Subsequentl parts required for the
establishment of conformance testing Services address:
- requirements on testing la,boratories and clients (ISO 10303-32);
- abstract test suites (ISO 10303-33);
- abstract test methods (ISO 10303-34).
This part also incorporates aspects from the development of the ISO 9000 and EN 45000 series
of Standards and from a number of ISO/IEC guides:
ISO/IEC Guide 2: C -reneral terms and their definitions concerning standardization and
a>
related activities;
b) ISO/IEC Guide 25: G eneral requirements for the competence of calibration and testing
laboratories;
ISO/IEC Guide 38: General requirements for the acceptance of testing laboratories;
d) ISO/IEC Guide 40: General requirements for the acceptance of certification bodies;
ISO/IEC Guide 42: G
ui l d e 1mes ’ for a step-by-step approach to an international certifi-
e)
catiorn system;
f) ISO/IEC Guide 43: D
evelopment and Operation of laboratory proficiency testing;
0.0
Vlll
c ISO ISO 10303-31:1994(E)
ISO/IEC Guide 45: Guidelines for the presentation of test results.
g>
The objective of product data exchange cannot be completely achieved unless Systems tan be
tested to determine whether they conform to the relevant product data exchange Standards.
There is an industrial need to establish conformance testing Services for implernentations of ISO
10303. This part of ISO 10303 provides a foundation for the subsequent Parts in this series which
are required to establish conformance testing Services and hence, meet that industrial need.
Conformance testing is a type of testing defined as the testing of a candidate product for the
existente of specific characteristics required by a Standard in Order to determine the extent
to which that product is a conforming implementation. It involves testing the capabilities of
an implementation against bot11 the conformance requirements in the relevant Standard(s) and
what the client states the implementation ’s capabilities are.
An abstract test Suite is standardised for each application protocol of ISO 10303, for use by
suppliers or implernentors in self-testing, by users of product data exchange products, or by
other third Party testing organisations. This should lead to comparability and wide acceptance
of test reports produced by different testing laboratories, and thereby minimise the need for
repeated conformance testing of the Same System.
The Standardisation of abstract test suites requires international definition and acceptance of
a common test methodology, together with appropriate test methods and procedures. It is the
purpose of this series of Parts to define the methodology, to provide a framework for specifying
abstract test suites, and to define the procedures to be followed during conformance testing.
Details of test methods are addressed in this series of Parts; however, any Organisation con-
templating the use of test methods defined in this series of Parts should carefully consider the
constraints on their applicability.
Conformance testing does not include some types of testing which may be appropriate for imple-
mentations of ISO 10303. These include robustness testing, interoperability testing, acceptance
testing and Performance testing. These test methods do not form part of conformance test-
ing because there are no appropriate conformance requirements in the Standard against which
to test. After the results from conformance testing are available, additional testing may be
performed.
Conformance testing does not provide judgements on how a System is implemented, how reliable
it is, how it provides any requested Service, nor the environment of the implementation. It does
not, except in an indirect way, prove anything about the logical design of the Standard itself.
Conformance is not sufficient to guarantee interoperalbility; however, it increases the probability
that different implernentations are able to interoperate. In Order to be able to interoperate,
it is necessary that two implernentations that conform to a given application protocol support
compatible Options within that application protocol.
The complexity of most Standards makes exhaustive testing impractical on both technical and
economic grounds. For this Same reason of complexity, proof of correctness (verification testing)
is also impractical. Falsification testing does not guarantee conformance to a Standard since it
detects errors rather than the absence of errors. Conformance testing gives confidence that an
implementation has the required capabilities.
1X
This page i ntentionall y left blank

INTERNATIONAL STANDARD @ISO ISO 10303=31:1994(E)
Industrial automation Systems and integration -
Product data representation and exchange -
Part 31 :
Conformance testing methodology and framework:
General concepts
1 Scope
This pa,rt of ISO 10303, which introduces the series of pa ’rts devoted to conformance testing,
specifies a* general methodology a,nd fra.mework for testing the conformance of an implemen-
tation of ISO 10303. During conformance testing, such aln iI~lpleI~~ent~a~tion is termed an IUT
(implementation under test ).
NOTE - Figure 1 is a. pictlorial representation of tlhe relation bet(ween tlhe various Standards and
some of the concepts of conformance tlesting. The roles of other Parts of ISO 10303 are documented
in ISO 10303-1.
1.1 Applicability of this series of Parts
This series of pa.rts is applicable to t ’he phases of t,he conforma,nce testing process, these phases
being characterised bv t,he following maOjor a#ctivities:
c
a) the definition of a ’bstract teste suites for ISO 10303 applicaltion protocols;
the definitlion of abstract test methods for ISO 10303 implementation methods;
W
the conformance a,ssessment, process carried out bv a testing laboratory for a client,
culminating in the production of a conformance test report.
1.2 Applicability of this part of ISO 10303
This part of ISO 10303 is aNpplica8ble t,o all1 of the a ’bove act,iviCes, providing introductory ma-
terial, normative requirements 011 ealch part of this series, a ’nd definitions of common terms and
concepts.
1.3 Coverage of this series of Parts
This series of park specifies the requirements for and gives guida8nce on the procedures to be
foliowed in conformance testing for ISO 10303. ‘This series of pa.rts includes onlv such information
c
a,s 3s necessa,ry to meet the following objectives:
-.
c ISO
ISO lo3ozb31:1994(E)
application
implementation
protocol
method
(10303-20 series) (10303-200 series)
i
abstract test Suite
ISO 10303-33
(ISO 10303-1200
series)
abstract test ISO 10303-31
ISO 10303-34
method
I
conformance
assessment
process
-
Figure 1 - Relationship between conformance testing Standards and other series of
Parts
to achieve an adequate level of confidence in the tests als a mea ’sure of conformame;
to achieve comparability between the results of the corresponding test,s applied in
b)
different places at differentl times;
to facilitate communication between the Parties responsible for the activities described
in (a) and (b).
Requirements for procurement and contracts are outside the scope of this series of parts.
Testing by means of test methods which are specific to particular applications or Systems is
outside the scope of this series of Parts.
The framework established by this Part of ISO 10303 includes the concept of executable test
suites. These, by their very nature, cannot be standa ’rdised; consequently, Standardisation of
executable test suites is outside the scope of this series of parts.

c ISO ISO 10303=31:1994(E)
1.4 Coverage of this part of ISO 10303
This part of ISO 10303 provides introductory material, which is expanded further in the remain-
ing Parts of this series of Parts of ISO 10303, including:
an exposition of the meaning of conformance in the context of ISO 10303;
a description of basic and capability tests;
b)
an introduction to the conformance assessment process;
an introduction to the abstract test methods and their applicability;
an introduction to the concepts of abstract test Suite design.
e)
The procedures required for testing the conformance of an implementation of ISO 10303 are
outside the scope of this part of ISO 10303, but are addressed in ISO 10303-34.
The following types of testing are all outside the scope of this part of ISO 10303.
(User) acceptance testing: the process of determining whether an implementation sat-
isfies acceptance criteria and enables the user to determine whether to accept the System.
This includes the planning and execution of several kinds of tests (e.g., functional, volume,
Performance tests) to demonstrate the implemented Software satisfies the user requirements.
interoperability testing: related to acceptance testing, but applied to the examination of
the information exchalnge and sharing between two specific IUTs and the ability of each IUT
to use such informaltion.
Performance testing: measures the Performance characteristics of an IUT, such as its
throughput, response time, number of transactions, and responsiveness under various condi-
tions.
robustness testing: the process of determining how well an IUT processes data which
cont ains errors.
This part of ISO 10303 provides a framework for certification (an administrative procedure
which may follow conformance testing) in annex D. However, there is no ISO requirement for
an implementation of ISO 10303 to undergo certification or conformance testing.
2 Normative references
The following Standards contain provisions which, through reference in this text, constitute
provisions of this part of ISO 10303. At the time of publication, the editions indicated were
valid. All Standards are subject to revision, and Parties to agreements based on this part of
are encouraged to investigate the possibility of applying the most recent editions of
ISO 10303
the Standards indicated below. Members of IIX and ISO maintain registers of currently valid
International Standards.
~@ISO
ISO 1oXW.%l:l994(E)
l -‘) , kformation techn&gy - Open Systems fnterconn,ectz ’on - A bstract ,S ’yntax
ISO/IEC 8824-1.
Notation One (ASN.1) - Part 1: Specijkation of Basic Notation,.
ISO 10303~1:1994, Industrial automation systems and integration - Product data represen,tation
and exchange - Part 1: Overview and fundamental principles.
ISO 10303-32: --l) , Industrial automation systems afnd integration - Product data representation
and exchaJnge - Part 32: Conformance testing methodobogy m2d framework: ReqCements on
testing laboratories and clients.
3 Definit ions
3.1 Terms defined in ISO 10303-1
This part of ISO 10303 makes use of the following terms defined in ISO 10303-1:
-
abstract test Suite;
-
exchange structure;
-
implementation method;
- PICS proforma;
- Protocol Implementation Conforma,nce Statement (PICS).
3.2 Other definitions
For the purposes of this part of ISO 10303, the following definitions applv.
*
3.2.1 abstract test case (ATC): a specific&ion, encapsulating at least one test purpose, tlialt
provides the formal basis from which executable test cases a ’re derived. It is independent of both
the implementation and the values.
3.2.2 abstract test group:
a named set of related abstract test cases.
3.2.3 abstract test method: the description of how an implementation is to be tested, given
at the appropriate level of abstralction to make the description independent of alny particular
implementation of testing tools or procedures, but with sufficient deta,iI to enable these tools
and procedures to be produced.
3.2.4 (laboratory) accreditation: the formalised initial and continuing process of ensuring
a testing laboratory is competent to carry out specific (types of j tests.
l)Tc, be published.
ISO 10303=31:1994(E)
c ISO
NOTE - The term “laboratory accreditation” covers the recognition of both the technical com-
petence and the impartiality of a testing laboratory. Accreditation is normally awarded following
successful laboratory assessmentl and is followed by appropriate Surveillance.
3.2.5 accreditation body: a body that conducts and administers laboratory accreditation
and grants accreditation.
3.2.6 assessor: an expert selected to conduct assessment during accreditation of a particular
laboratory.
3.2.7 attestation of conformity: action by a third-Party testing laboratory, demonstrating
that the specific IUT tested is in conformitv with a1 specific Standard or other normative docu-
Y
ment.
NOTE - Compare with declaration oi confomaity and certijkation of conformity.
3.2.8 basic tests: limited tests performed to determine whether it is appropriate to perform
thorough testing.
3.2.9 capabilities of an IUT: the set of functions and Options in the relevant Standard that
is supported by the IUT.
3.2.10 capability tests: tests performed to determine the capabilities of an IUT, designed to
determine whether an implementation conforms to a particular feature of an application protocol
as described in the tese purpose.
3.2.11 certificate of conformance: see certificate of confornzity.
3.2.12 certificate of conformity; certificate of conformance: a document issued under the
rules of a certifkation System indicating that adequate confidence is provided that an IUT is
in conformity with a specific Standard or technical specification as determined through use of a
specified test method.
3.2.13 certification body: an imparlial body possessing the necessary competence and relia-
bility to operate a certification System, and in which the interests of all Parties concerned with
the function of the System arc ripresented.
c
NOTE - The certifkation body may decentralise its activities and rights to certify conformity.
3.2.14 certification mark: the certification body ’s sign, Symbol, or letter that identifies a
product(s) or Service(s) as being certified.
3.2,15 certification of conformity: action bv a third Party. demonstrating that adequate
T
confidence is provided that an identified IUT is in conformity with a specific Standard or other
normative docurnent 0
NOTE - Co mpare . with declaration of conformity and attcstation of conformity.

c ISO
ISO 1o3osmlXM(E)
a procedural and managerial systern for carrying out certifica-
3.2.16 certification System:
tion of conformity that is overseen by a certification body.
.*
NOTE - Certification Systems may be operated at the national, regional or international level.
3.2.17 client (of a testing laboratory): the Organisation that submits an implementation
for conformance testing.
3.2.18 comparability (of results): chalracteristic of conformance assessment processes such
that execution on the Same SUT, in different testing laboratories, leads to the same Overall
summary.
3.2.19 conformance: see co~jhnzity.
3.2.20 conformance assessment process: the process of accomplishing the conformance test-
ing activities necessary to determine the conformance of an implementation to an application
protocol.
3.2.21 conformance log: a record of information, produced als al result of a test campaign,
that is sufficient to make and verifv t he assignment of test verdicts.
3.2.22 conformance testing: the testing of a candidate product for the existente of specific
characteristics required by a stialndard in Order to determine the extent to which that product is
a conforming implement ation.
3.2.23 (conformance) test report: a document written at the end of the conformance assess-
ment process, that provides the Overall summary of the conformance of the IUT to the Standard
for which conformance testing was carried out, and that gives the details of the testing.
3.2.24 conforming implementation: an implementation which satisfies the conformalnce re-
quirements, consistent with the capabilities stated in the PICS.
3.2.25 conformity; conformance:
the fulfilment by an implement ation of all requirement s
specified.
3.2.26 control board: an impartial body possessing the necessary competence and reliability
to approve and maintain proficient test suites for the testing programme, aJnd in which the
interests of all Parties concerned with the function of the tests are represented.
3.2.27 declaration of conformance: see declaxhon of corzfo~~nlity.
3.2.28 declaration of conformity; (manufacturer ’s) declaration of conformance: a state-
ment by a supph claiming under his sole responsibility that an IUT’ is in conformity with a
specific Standard or other normative document m

NOTES
1 - Compare with attestation of conformity and certification of conformity.
2 - The term “self certification” should notl be used, in Order to avoid any confusion with the concept
of certification which should imply the involvement of a third Party.
3.2.29 executable test case: an instantiation of an abstract test case with va ues. 8: 3.6.41
(SG4: 19th J anuary 1989)
3.2.30 executable test Suite: the set of executable test cases necessary to perlarm conlor-
mance testing of an implementation against a Standard or group of Standards.
3.2.31 fail (verdict): a test verdict given when the observed test outcome demonstrates non-
conformance with respect to either the test purpose or at leaist one of the conformance require-
ments in the relevant Standard(s).
3.2.32 falsification testing: a test method developed to find errors in the implementation. If
errors are found, one tan correctly deduce the implementation does not conform to the Standard;
however, the absence of errors does not necessarily imply the converse. Falsification testing tan
only demonstrate non-conformance.
NOTE - Compare with verijication testing.
3.2.33 Implementation Under Test (IUT): that part of a product which is to be studied
under testing, which should be an implementation of one or more characteristics of the stan-
dard(s) based on a given implementation method.
verdict): a test verdict given when the observed test outcome is such
3.2.34 inconclusive (
that neither a pass nor a fail verdict tan be given.
3.2.35 in-house testing: the testing undertaken by the client (of a testing laboratory) using
the executable test Suite supplied by the testing laboratory Prior to Submission for the formal
Phase of the Service.
3.2.36 manufacturer ’s declaration of conformance: see cleckratio~z 0s co72~orn2 ’Ltz.~.
3.2.37 non-conformance: see non-conformity.
3.2.38 non-conformity; non-conformance: the failure of an implementation to fulfil one or
more requirements specified.
XZ.39 pass (verdict): a test verdict given when the observed test outcome gives evidente of
conformance to the conforrnance requirernent on which the test purpose is focused and is valid
with respect to the releva)nt Standard(s) and with respect to the PICS.
I I
c ISO
ISO lo3o&-31:1994(E)
a document, in the form of a questionnaire, written and provided
3.2.40 PIXIT proforma:
by the testing laboratory which, when completed during the preparation for testing, becomes
the PIXIT.
3.2.41 postprocessor: a Software unit that translates product information from an indepen-
dent public domain product data format to the internal format of a particular Computer system.
3.2.42 preprocessor: a soft)ware unit that transMes product informat,ion from the internal
format of a particular cornputer System to an independent public domajin product da,ta format.
3.2.43 proficiency testing: determinat ’ion of laboratory testing performante by means of
inter-laboratory test comparisons or by testing Systems for which the results have already been
determined.
NOTE - This does not ferm part of conformance testing: see D.3.5.4.
3.2.44 Protocol Implementation extra Information for Testing (PIXIT): a Statement
made by the client which contaNins or references all of the information (in a,ddition to that given
in the PICS) related to the IUT and its corresponding SUT, which will enable the testing
laboratory to run an appropriate test Suite a#gainst that IUT.
3.2.45 repeatability (of results): characteristic of an abstract test case and derived exe-
cutable test c.ase(s), such that repeated executions on the Same SIJT under the Same conditions
lead to the same test verdict; a ’nd, by extension, a chalracteristic of an abstract test Suite and
derived executable test Suite(s).
ts: tests performen to determine in depth whether or not an implemen-
3.2.46 resolution tes
tation satisfies specific req uirement s.
3.2.47 selected abstract test Suite: the set of abstra,ct test cases selected using a specific
PICS.
3.2.48 selected executable test Suite: the set of executable test ca,ses selected using a spe-
cific PICS.
3.2.49 System Under Test (SUT): the Computer hardware,
software and communication
network required to support the IUT.
3.2.50 test campaign: the process of running the executable test Suite for a particular IUT.
3.2.51 test case: use of this term is deprecated: see Abstract Test Gase or Executable Test
Gase.
3.2.52 test case error: a Statement made with respect to an abstract test case when a)n error
is detected in the abstract test case itself or its executable equivalent.
c ISO ISO 10303-31:1994(E)
3.2.53 test purpose: a precise description of an objective which an abstract test case is de-
signed t o achieve.
3.2.54 test realiser: an Organisation which takes responsibility for providing, in a form inde-
pendent of the clients of a testing laboratory and their IUTs, a means of testing IUTs.
3.2.55 test report: see conformance test report.
3.2.56 test verdict: see verdict.
3.2.5 7 testing laboratory: an Organisation that carries out the conformance assessment pro-
cess.
NOTE - A testing laboratory tan be a third Party, a user Organisation, an admini strative organi-
sation, or an identifiable part of a supplier Organisation.
3.2.58 (test) verdict: a Statement of “Pas?, “fail ”, or “inconclusive” concerning conformance
of an IUT with respect to an executable test case and the abstract test case from which it was
derived.
3.2.59 verdict criteria: information defined within an abstract test case which enables the
testing laboratory to assign a verdict.
3.2 .60 verification testing: the process of proving mathematically whether an IUT is correct,
con sistent and complete.
NOTE - Compare with falsijication testing.
4 Abbreviations
For the purposes of this series of Parts the following abbreviations apply.
ATC Abstract Test Case
CTR Conformance Test Report
IUT Implementation Under Test
PICS Protocol Implementation Conformance Statement
PIXIT Protocol Implementation extra Information for ‘Testing
SUT System Under Test
c ISO
ISO mo3a:1994(E)
5 Conformance
5.1 The meaning of conformance in ISO 10303
In the context of ISO 10303, an implementation exhib,its conformance if it camplies with the
conformance requirements of the applicable Part(s) of ISO 10303.
Conformance of an implementaJtion is expressed either a,s conformance to sin application protocol
combined with an implementation method or as conformance to a description method. Confor-
mance of aln implementation shall be determined by using an executable test Suite generated
from the applicable Standard abstract test Suite. EaLch abstract test Suite is documented in the
ISO 10303-1200 series of Parts and is referenced normatively by the corresponding application
c
protocol.
5.2 Conformance requirements
The conformance requirements in aN stalndaird caln be:
mandatory requirements: these a,re to be observed in all cases;
a>
conditional requirements: these are to be observed if certain conditions set out, in the
b)
Standard apply;
optional requirements: these tan be selected to suit the implementation, provided that
any requirements applicable to the Option are observed.
NOTE - More information on Options is provided in annex ß.
Eurthermore, conformance requirements in at Standard tan be sta ’ted:
positively: they state what shall be done;
a>
negatively (prohibitions): they state wha,t shall not be done.
b)
5.3 Protocol Implementation Conformance Statement
To evaluate the conforrnance of a particular implementation, it is necessa,ry to have a staltement
of the Options which have been implemented so that the implernentation tan be tested for con-
formante against relevant requirement s, and agajns t t hose requirement s only. Such a st a)tement
is called a Protocol Implementation Conformance Statement (PICS). The- Options within the
PICS shall only be stated within the framework of requirements specified in the relevant Parts
of ISO 10303. The PICS shall not include Options beyond this framework.
The PICS is used by the testing laboratory to have a better understanding of the SIJT for the
conformance assessment process and to help to identify the boundaries of the domain of testing.
c ISO ISO 10303-31:1994(E)
NOTE - In Order to evaluate two or more Systems for successful interoperability, it is recommended
that a comparison be made of the PICSs of these Systems, comparing each Option to determine if
both Systems Claim to support it. If the Systems use different Versions of the relevant Standard,
as indicated in the PICSs, the differentes between the Versions need to be identified and their
implications for interoperability need to be considered, including their use in combination with
other Standards.
The PICS is generated by the client using a PICS proforma. The PICS proformais a standardised
document included in the relevant part of ISO 10303. It is a questionnaire used as a framework
by the client to document those SUT capabilities necessary to undertake conformance testing.
A PICS is needed for each International Standard providing a PICS proforma that has been
irnplemented in a System.
5.4 A conforming System
A conforming System or implementation is one which satisfies the conformance requirernents of
the implemented part of ISO 10303, consistent with the PICS. Such an implementation shall
have passed all tests constituting the domain of testing, including any optional requirements
stated as supported in the PICS.
Use of the term ‘conformity’ is ambiguou s unl .ess used with the ide ntifi cation of bot h im .plem .en-
t ation method, the application protocol, and (where appropriate) the conformance clas s.
6 Conformance testing
6.1 Introduction
This clause introduces two enabling technologies of the conformance assessment process: the dif-
ferent types of conformance tests used and the PINT. It also describes the phases and properties
of the conformance assessment process.
6.2 Types of conformance tests
111 principle, the objective of conformance testing is to establish whether the implementation
being tested conforms to the requirements stated in the relevant application protocol. This
series of Parts distinguishes two types of tests, according to the extent to which they provide an
indication of conformance.
Basic tests, which provide preliminary evidente that an IUT conforms. It is optional
a>
to specify these tests in the abstract test Suite; however, if the Standard abstract test Suite
identi
...