SIST EN ISO/IEC 9646-1:1997

2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.Information technology - Open Systems Interconnection - Conformance testing methodology and framework - Part 1: General concepts (ISO/IEC 9646-1:1994)Informationstechnik - Kommunikation offener Systeme - Methodik der Konformitätsprüfung - Teil 1: Allgemeine Konzepte (ISO/IEC 9646-1:1994)Technologies de l'information - Interconnexion de systemes ouverts - Cadre général et méthodologie des tests de conformité OSI - Partie 1: Concepts généraux (ISO/IEC 9646-1:1994)Information technology - Open Systems Interconnection - Conformance testing methodology and framework - Part 1: General concepts (ISO/IEC 9646-1:1994)35.100.01Medsebojno povezovanje odprtih sistemov na splošnoOpen systems interconnection in generalICS:Ta slovenski standard je istoveten z:EN ISO/IEC 9646-1:1996SIST EN ISO/IEC 9646-1:1997en01-december-1997SIST EN ISO/IEC 9646-1:1997SLOVENSKI
STANDARD



SIST EN ISO/IEC 9646-1:1997



SIST EN ISO/IEC 9646-1:1997



SIST EN ISO/IEC 9646-1:1997



INTERNATIONAL STANDARD ISOJIEC 9646-1 Second edition 1994-12-15 Information technology - Open Systems Interconnection - Conformance testing methodology and framework - Part 1: General concepts Technologies de I’informa tion - Interconnexion de systkmes ouverts - Cadre g&&al et m&hodologie des tests de conformit OS/ - Partie I: Concepts g&Graux Reference number lSO/IEC 9646-l :I 994(E) SIST EN ISO/IEC 9646-1:1997



ISO/IEC 9646-l: 1994(E) Contents Page Foreword . .i v Introduction. . . 1 Scope . .l 2 Normative references. . .2 3 Definitions . .3 3.1 Reference model definitions . .3 3.2 Terms defined in other International Standards, ITU-T Recommendations and Technical Reports. . .3 3.3 Conformance testing definitions . .4 4 Abbreviations . .10 5 ThemeaningofconforrnanceinOSI. . .ll 5.1 Introduction . .ll 5.2 Conformance requirements . .ll 5.3 Static conformance requirements. . 11 5.4 Dynamic conformance requirements. . .12 5.5 Conformance requirements related to transfer syntaxes . .12 5.6 Implementation Conformance Statements (ICSs) . .12 5.6.1 Introduction. . .12 5.6.2 Protocol Implementation Conformance Statement (PICS) . .12 5.6.3 Profile Implementation Conformance Statement (profile ICS). . .12 5.6.4 System Conformance Statement (SCS) . .13 5.7 Aconforming system . .13 5.8 Interworking and conformance . .13 6 Confomnance and testing, . “14 6.1 Objectives of conformance testing. . .14 6.1.1 Introduction . .14 6.1.2 Basic Interconnection Tests. . .14 6.1.3 Capability tests. . .14 6.1.4 Behaviour tests. . .15 6.1.5 Conformance resolution tests . .15 6.2 Extra information for testing . .15 6.2.1 Implementation Extra Information for Testing (IXIT) . .15 6.2.2 Protocol Implementation Extra Information for Testing (PIXIT) . .16 0 ISO/IEC 1994 All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying and microfilm, without permission in writing from the publisher. ISO/IEC Copyright Office l Case Postale 56 l CH-1211 Geneve 20 l Switzerland Printed in Switzerland ii SIST EN ISO/IEC 9646-1:1997



0 ISO/IEC ISO/IEC 9646-l: 1994(E) 6.2.3 Profile Implementation Extra Information for Testing (profile IXIT) . .16 6.3 Confkxk&~ &k&&t*ii&ei, b&-;ik*~ : : : : : : : : : : : : : : : : : : : : : : : . .16 6.4 Use of Basic Interconnection Tests and capability tests in the test campaign18 6.5 Analysis of results . .18 6.5.1 Test outcomes and test verdicts. . .18 6.5.2 Conformance test reports . .18 6.5.3 Repeatability of results . .19 6.5.4 Comparability of results . .19 6.5.5 Auditability of results . .19 7 Testmethods.l 9 7.1 Introduction.19 7.2 Classification of real open systems and IUTs for conformance testing . .19 7.2.1 Classification of SUTs . .19 7.2.2 Identification of the IUT . .21 7.3 Abstract testing methodology . .21 7.3.1 General . .21 7.3.2 Points of Control and Observation (PCOs) . .23 7.4 Abstract testing functions . .23 7.5 Overview of Abstract Test Methods. . .2 4 7.5.1 End-system IUTs in the Single-Party Testing context . .24 7.5.2 The Local and Distributed test methods . .24 7.5.3 The Coordinated and Remote test methods. . .24 7.5.4 Variants of abstract test methods . .24 7.5.5 IUTs in the Multi-Party Testing context. . 24 7.6 Applicability of test methods to real open systems. . .24 7.7 Applicability of the test methods to OS1 protocols and layers . .26 8 Testsuites.2 6 8.1 Structure. . .2 6 8.2 Abstract and executable test cases . .26 8.3 Base specification conformance testing specifications . .28 8.4 Profile Test Specifications (PTS) . .28 9 Relationships between parts, concepts and roles . .29 10 Comp~ance.2 9 Annex A Op~oIls.3 1 B Indexofparts 1.2.4.5.6and7 . .32 . . . 111 SIST EN ISO/IEC 9646-1:1997



ISO/IEC 9646-l: 1994(E) 0 ISO/IEC Foreword IS0 (the International Organization for Standardization) and IEC (the Inter- national Electrotechnical Commission) form the specialized system for worldwide standardization. National bodies that are members of IS0 or IEC participate in the development of International Standards through technical committees established by the respective organization to deal with particular fields of technical activity. IS0 and IEC technical committees collaborate in fields of mutual interest. Other international organizations, governmental and non-governmental, in liaison with IS0 and IEC, also take part in the work. In the field of information technology, IS0 and IEC have established a joint technical committee, ISO/IEC JTC 1. Draft International Standards adopted by the joint technical committee are circulated to national bodies for voting. Publication as an International Standard requires approval by at least 75 % of the national bodies casting a vote. International Standard ISO/IEC 9646-l was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology, Subcommittee 21, Open Systems Interconnection, data management and open distributed processing. This second edition cancels and replaces the first edition (ISO/IEC 9646-l: 1991) which has been technically revised. It also consolidates Technical Corrigendum 1. ISO/IEC 9646 consists of the following parts, under the general title Information technology - Open Systems Interconnection - Conformance testing methodol- ogy and framework: - Part I: General concepts - Part 2: Abstract Test Suite specification - Part 3: The Tree and Tabular Combined Notation - Part 4: Test realization - Part 5: Requirements on test laboratories and clients for the conformance assessment process - Part 6: Protocol profile test specification - Part 7: Implementation conformance statements Annexes A and B of this part of ISO/IEC 9646 are for information only. SIST EN ISO/IEC 9646-1:1997



0 ISO/IEC ISO/IEC 9646=1:1994(E) Introduction The objective of OS1 will not be completely achieved until systems can be tested to determine whether they conform to the relevant protocol and profile specifications. The relevant ones can be OS1 International Standards, ITU-T Recommendations or International Standardized Profiles. Standardized Abstract Test Suites should be developed for each International Standard or ITU-T Recommendation which specifies an OS1 protocol, for use by suppliers or implementors in self-testing, by users of OS1 products, by telecommunications administrations and recognized private operating agencies, or by other third party testing organizations. Standardized Profile Test Specifications should be developed for each OS1 profile defined in an International Standardized Profile, or in an International Standard or ITU-T Recommendation, to specify how to combine and select from these Abstract Test Suites for the purpose of testing conformance of systems to that profile. This should lead to comparability and wide acceptance of test results produced by different test laboratories, and thereby minimize the need for repeated conformance testing of the same system. The standardization of test suites requires international definition and acceptance of a common testing methodology, together with appropriate testing methods and procedures. It is the purpose of ISO/IEC 9646 to define the methodology, to provide a framework for specifying conformance test suites, and to define the procedures to be followed during testing. Conformance testing involves testing both the capabilities and behaviour of an implementation, and checking what is observed against the conformance requirements in the relevant International Standards or ITU-T Recommendations and if appropriate in the related International Standardized Profiles and against what the implementor states the implementation capabilities are. Conformance testing does not include assessment of the performance nor the robustness or reliability of an implementation. It cannot give judgements on the physical realization of the Abstract Service Primitives, how a system is implemented, how it provides any requested service, nor the environment of the protocol implementation. It cannot, except in an indirect way, prove anything about the logical design of the protocol itself. The purpose of conformance testing is to increase the probability that different OS1 implementations are able to interwork. However it should be borne in mind that the complexity of most protocols makes exhaustive testing impractical on both technical and economic grounds. Also, testing cannot guarantee conformance to a specification since it detects errors rather than their absence. Thus conformance to a test suite alone cannot guarantee inter-working. What it does do is give confidence that an implementation has the required capabilities and that its behaviour conforms consistently in representative instances of communication. SIST EN ISO/IEC 9646-1:1997



ISO/IEC 9646-l: 1994(E) 0 ISOAEC It should be noted that the OS1 basic reference model (ISO 7498: 1984 or CCITT X.200 (1984)) states (in 4.3): “Only the external behuviour of Open Systems is retained as the standard of behaviour of real Open Systems” This means that although aspects of both internal and external behaviour are described in OS1 International Standards and ITU-T Recommendations, it is only the requirements on external behaviour that have to be met by real open systems. Although some of the methods defined in ISO/IEC 9646-2 do impose certain limitations on the implementor, for example that there be some means of realizing control and observation at one or more Service Access Points, it should be noted that other methods defined herein do not impose such limitations. However, in the case of partial OS1 end-systems which provide OS1 protocols up to a specific layer boundary, it is desirable to test not only the external behaviour of the implemented protocol entities, but also the ability of those entities to support correct external behaviour in higher layers. Detailed investigation of relative benefits, efficiency and limitations of all methods is addressed in various parts of ISO/IEC 9646. However, any organization contemplating the use of test methods defined in ISOIIEC 9646-2 in a context such as certification should carefully consider the limitations on their applicability and the benefits of each. Testing is voluntary as far as ISO/IEC and ITU-T are concerned. Requirements for testing in procurement and other external contracts are not a matter for standardization. This part of ISO/IEC 9646 is also to be published by ITU-T as Recommendation X.290 but not as identical text. SIST EN ISO/IEC 9646-1:1997



INTERNATIONAL STANDARD 0 ISO/IEC ISO/IEC 9646-l: 1994 (E) Information technology - Open Systems Interconnection - Conformance testing methodology and framework - Part 1: General concepts 1 Scope 1.1 ISO/IEC 9646 is a multi-part International Standard which specifies a general methodology for testing the conformance of products to OS1 specifications which the products are claimed to implement. The methodology applies to testing conformance to a) the specification of an OS1 protocol; b) the specification of a transfer syntax used in combination with a specific OS1 protocol; c) the specification of an OS1 protocol profile, including the testing of conformance to any specified information objects used in combination with one or more of the protocols, as specified in the profile; d) the specifications of a combination of OS1 protocols, possibly used in combination with a specified transfer syntax and/or one or more specified information objects. The OS1 specification to which conformance is tested may be contained in an International Standard, an ITU-T Recommendation, or an International Standardized Profile. 1.2 ISO/IEC 9646 is in principle, applicable to conformance testing for Integrated Services Digital Network (ISDN). 1.3 ISO/IEC 9646 is applicable to conformance testing for those implementations of a protocol or set of protocols that require communication between two or more real open systems to achieve their purpose (e.g. Message Handling Service (MHS)), Directory Services, ISDN, Transaction Processing, Routeing, and Systems Management). 1.4 ISO/IEC 9646 is applicable to the different phases of the conformance testing process, these phases being characterized by three major activities. These activities are: a) the specification of Abstract Test Suites (ATSs) for particular OS1 protocols and Profile Test Specifications (PTSs) for particular OS1 profiles; b) realization of a Means of Testing (MOT) able to run an executable form of an Abstract Test Suite; c) the conformance assessment process carried out by a test laboratory for a specific client on the basis of an Implementation Conformance Statement (ICS), culminating in the production of a System Conformance Test Report (SCTR) and one or more Protocol Conformance Test Reports (PCTR), one for each ATS used; the results are given in terms of the relevant protocol specification(s) and test suite(s) used. This part is applicable to all three activities, providing tutorial introductory material, together with definitions of common terms and concepts. NOTE - ISO/IEC 9646-2 deals with the requirements and guidance for the specification of ATSs, independent of test notation. ISO/IEC 9646- 3 defines the recommended test notation. ISOAEC 9646-4 deals with requirements and guidance for realization of the means of testing. ISO/ IEC 9646-5 deals with requirements and guidance for test laboratories and their clients for the conformance assessment process. ISO/IEC 9646 6 deals with the requirements and guidance for PTSs based upon ATSs for each of the protocols involved. ISO/IEC 9646-7 deals with the requirements and guidance for ICSs, System Conformance Statements (SCSs), their proformas and Requirements Lists (RLs). 1.5 ISO/IEC 9646 specifies the requirements for and gives guidance on the procedures to be followed in OS1 conformance testing. 1.6 ISO/IEC 9646 includes only such information as is necessary to meet the following objectives: a) to achieve an adequate level of confidence in the tests as a guide to conformance; b) to achieve comparability between the results of the corresponding tests on a particular OS1 implementation applied in different places at different times; c) to facilitate communication between the parties responsible for the activities described in 1.4 above. 1 SIST EN ISO/IEC 9646-1:1997



ISO/IlK 9646-l: 1994(E) 0 1somc 1.7 This part of ISO/IEC 9646 includes tutorial introductory material which provides a) an exposition of the meaning of conformance in the context of OSI; b) a description of the major categories of conformance tests; c) an introduction to the conformance assessment process; d) an introduction to the Abstract Test Methods and their applicability; e) an introduction to the concepts of test suite design. In addition, this part describes the relationship between the other parts of ISO/IEC 9646 and the activities involved in conformance testing, and introduces the concept of compliance with respect to the other parts of ISO/IEC 9646. 1.8 The following are outside the scope of ISO/IEC 9646: a) certification, an administrative procedure which may follow conformance testing; b) requirements for procurement and contracts; c) testing by means of test methods which are specific to particular applications, protocols or systems; d) testing by means other than PDU exchange; NOTE - ISO/IEC 9646 does not apply fully to Physical layer protocols. Nevertheless, many of the concepts apply to all protocols. 2 Normative references The following standards contain provisions which, through reference in this text, constitute provisions of this part of ISO/IEC 9646. At the time of publication, the editions indicated were valid. All standards are subject to revision, and parties to agreements based on this part of ISO/IEC 9646 are encouraged to investigate the possibility of applying the most recent editions of the standards listed below. Members of IEC and IS0 maintain registers of currently valid International Standards. IS0 7498: 1984, Information processing systems - Open Systems Interconnection - Basic Reference Model. (See also CCITI’ Recommendation X.200 (1984)) ISOITR 8509: 1987, Information processing systems - Open Systems Interconnection - Service conventions. (See also CCITT Recommendation X.210 (1988)) ISO/IEC 8825: 1990, Information technology - Open Systems Interconnection - Specification of Basic Encoding Rules for Abstract Syntax Notation One (ASN.1). (See also CCI‘IT Recommendation X.209 (1988)) ISO/IEC 9646-2: 1994, Information technology - Open Systems Interconnection - - Conformance testing methodology and framework - Part 2: Abstract Test Suite spectfication. (See also ITU-T Recommendation X.291 J)) ISO/IEC 9646-3: 1992, Information technology - Open Systems Interconnection - Conformance testing methodology and framework - Part 3: The Tree and Tabular Combined Notation (TTCN). (See also ITU-T Recommendation X.292 (1993)) ISO/IEC 9646-3 Amd 1: J, Information technology - Open Systems Interconnection - Conformance testing methodology and framework - Part 3: Th.e Tree and Tabular Combined Notation - Amendment 1: TTCN extensions. ISO/IEC 9646-4: 1994, Information technology - Open Systems Interconnection - Conformance testing methodology and framework - Part 4: Test realization. (See also ITU-T Recommendation X.293 J)) ISO/IEC 9646-5: 1994, Information technology - Open Systems Interconnection - * Conformance testing methodology and framework - Part 5: Requirements on test laboratories and clients for the conformance assessment process. (See also ITU-T Recommendation X.294 -I)) ISO/IEC 9646-6: 1994, Information technology - Open Systems Interconnection - I Conformance testing methodology and framework - Part 6: Protocol profile test specification. (See also ITU-T Recommendation X.295 -I)). ISOIIEC 9646-7: J), Information technology - Open Systems Interconnection - Conformance testing methodology and framework - Part 7: Implementation Conformance Statements. (See also ITU-T Recommendation X.296 J)). 1) To be published. 2 SIST EN ISO/IEC 9646-1:1997



0 ISO/IEC ISO/IEC 9646-l: 1994(E) ISOIIEC TR 10000-l: 1990, Information technology - Framework and taxonomy of International Standardized Profiles, Part 1 - Framework. 3 Definitions 3.1 Reference model definitions This part of ISO/IEC 9646 is based upon the concepts developed in the Basic Reference Model for Open Systems Interconnection (IS0 7498 1 CCITT X.200), and makes use of the following terms defined in that International Standard: a) (N)-entity b) (N)-layer c) (N)-protocol d) (N)-protocol-data-unit e) (N)-relay f) (N)-service g) (N)-service-access-point h) Application layer i) Application-service-element j) Data Link layer k) Network layer 1) Physical layer m) Presentation layer n) real open system 0) real system p) (N)-service data unit q) Session layer r) subnetwork s) transfer syntax t) Transport layer 3.2 Terms defined in other International Standards, ITU-T Recommendations and Technical Reports 3.2.1 This part of ISO/IEC 9646 uses the following terms defined in the OS1 Service Conventions (ISO/TR 8509 1 CCITT X.210): a) service-user b) service-provider c) service primitive 3.2.2 This part of ISO/IEC 9646 uses the following term defined in the Specification of Basic Encoding Rules for Abstract Syntax Notation One (ASN.l) (IS0 8825 1 CCITT X.209): - encoding 3.2.3 This part of ISO/IEC 9646 uses the following terms defined in the Framework and Taxonomy of International Standardized Profiles (ISO/IEC TR 10000-l): a) International Standardized Profile b) profile c) base standard SIST EN ISO/IEC 9646-1:1997



ISO/IIK 9646-1: 1994(E) 0 ISO/IEC NOTES 1 - In ISO/IEC 9646 the term profile is used to mean “protocol profile” as defined in 3.3.82 . 2 - In ISO/IEC 9646 the more general term “base specification” is used with similar meaning to base standard but independent of the status of the specification, as defined in 3.3.10 . 3.3 Conformance testing definitions For the purposes of this part of ISO/IEC 9646 the following definitions apply: 3.3.1 abnormal (test case) termination: The term used to describe the result of execution of an abstract test case when it has been prematurely terminated by the test system. 3.3.2 abstract (N)-service-primitive [(N)-ASP]: An implementation-independent descri .ption of an service-user and a service-provider at an (N)-service boundary, as defined in an OS1 service defin ition. interaction between a 3.3.3 abstract test case: A complete and independent specification of the actions required to achieve a specific test purpose, defined at the level of abstraction of a particular Abstract Test Method, starting in a stable testing state and ending in a stable testing state. This specification may involve one or more consecutive or concurrent connections. NOTES 1 The specification should be complete in the sense that it is potentially observable test outcome (i.e. sequence of test events). sufficient to enable a test verdict to be assigned unambiguously to each 2 The specification should be independent in the sense that it should be possible to execute the derived executable test case in from other such test cases (i.e. the specification should always include the possibility of starting and finishing in the “idle” state). isolation 3.3.4 abstract test case error: A test case error resulting from an error in the abstract test case. 3.3.5 (abstract) test method [ATM]: The description of how an IUT is to be tested, given at an appropriate level of abstraction to make the description independent of any particular realization of a Means of Testing, but with enough detail to enable abstract test cases to be specified for this test method. 3.3.6 abstract test suite [ATS]: A test suite composed of abstract test cases. 3.3.7 abstract test suite [ATS] specification: A specification that contains a standardised ATS together with related information. 3.3.8 abstract testing context: Either a Multi-Party or Single-Party Testing context. 3.3.9 abstract testing methodology: An approach to describing and categorizing Abstract Test Methods, 3.3.10 base specification: A specification of a protocol, abstract syntax, encoding rules, or information object. 3.3.11 basic interconnection test [BIT]: A test of an IUT which has limited scope to determine whether or not there is sufficient conformance to the relevant protocol(s) for interconnection to be possible, without trying to perform thorough testing. 3.3.12 behaviour test: A test to determine the extent to which one or more dynamic conformance requirements are met by the IUT. 3.3.13 capability (of an implementation): A set of functions in the relevant protocol(s) which is supported by the implementation. 3.3.14 capability test: A test to verify the existence of one or more claimed capabilities of an IUT. NOTE - Capability testing involves checking all mandatory capabilities and those optional ones that are stated in the ICS as supported, but not checking those optional ones which are stated in the ICS as not supported by the KIT. 3.3.15 client (of a test laboratory): An organization that submits a system or implementation for conformance testing. 3.3.16 common (sub)profile: A profile or specified subset of a profile which is included by reference within one or more other profiles. 3.3.17 comparability (of results): Characteristic of conformance assessment processes, such that their execution on the same IUT, in different testing environments, leads to the same overall summary of conformance for the specified IUT. 3.3.18 comprehensive testing service: A service, offered to clients by a test laboratory, to perform the conformance assessment process for one or more OS1 protocol(s), with a choice of test methods sufficient to make the service applicable to all real open systems that claim to implement the specified protocols. 3.3.19 conformance assessment process: The complete process of accomplishing all conformance testing activities necessary to assess the conformance of an implementation or a system to one or more OS1 specifications. 4 SIST EN ISO/IEC 9646-1:1997



0 ISOIIEC ISO/IEC 9646-l: 1994(E) 3.3.20 conformance log: A human-readable record of information produced as a result of a test campaign, which is sufficient to record the observed test outcomes and verify the assignment of test results (including test verdicts). 3.3.21 conformance resolution test: A non-standardized, possibly system-specific test to fulfil a test purpose for which a standardized abstract test case is not defined, in order to investigate the behaviour of an OS1 protocol implementation with respect to one or more particular conformance requirements. 3.3.22 (conformance) test suite: A complete set of test cases, possibly combined into nested test groups, that is needed to perform dynamic conformance testing for one or more OS1 protocols. NOTE - It should cover both capability testing and behaviour testing. It may be qualified by the adjectives: abstract or executable, as appropriate. Unless stated otherwise, an “abstract test suite” is meant. 3.3.23 conformance testing: Testing the extent to which an IUT is a conforming implementation. 3.3.24 conformance testing specification: One or more specifications that contain a standardized ATS, together with its related TSS&TP, partial IXIT proforma, and TMP specification, if any. 3.3.25 conforming implementation: An IUT which satisfies both static and dynamic conformance requirements, consistent with the capabilities stated in the ICS(s). 3.3.26 conforming system: A real system which satisfies both static and dynamic conformance requirements consistent with the capabilities stated in the KS(s) referenced by the SCS. 3.3.27 Coordinated test method: An Abstract Test Method in which the Upper Tester is within the SUT and for which a standardized TMP is defined for the TCP, enabling the control and observation to be specified solely in terms of the Lower Tester activity, including the control and observation of Test Management PDUs. 3.3.28 Distributed test method: An Abstract Test Method in which the Upper Tester is within the SUT and there is a PC0 at the upper service boundary of the IUT. 3.3.29 dynamic conformance requirement: One of the requirements which specifies what permitted by the relevant specification(s) in instances of communication. 3.3.30 embedded testing: Testing specified for a single-protocol within a multi-protocol IUT including the specification of the protocol activity above the one being tested, but without specifying control or observation at service boundaries within the multi-protocol IUT. observable behaviou r is NOTE - This definition assumes that the protocols of the IUT are ordered in a conti
...