ISO/IEC 10641:1993

ISO/IEC
INTERNATIONAL
STANDARD
First edition
1993-07-15
Information technology - Computer
graphics and image processing -
Conformance testing of implementations
of graphics standards
- Traitemen t informa tis6 de /‘image et des
Technologies de I’informa tion
graphiques - Essais de conformit de la mise en application des normes
graphiques
Reference number
lSO/IEC 10641 :I 993(E)
0 ISO/IEC
Page
Contents
1 Scope .
2 Normative references .
3 Definitions .
G
...........................................................................
4 Overview
4.1 Conformance testing .
(i
4.2 Graphics conformance testing .
4.2.1 The impact of registration .
............................ 8
4.2.2 Interfaces in computer graphics conformance testing
1I)
4.3 The testing process .
...................................
5 Conformance testing requirements within graphics standards
6 Graphics test suite .
6.1 Test software .
..........................................
6.1.1 Determination of testing domain
.................................................
6.1.2 Structure of a test suite
..............................................
6.1.3 Maintenance of a test suite
IS
..............................................
6.1.4 Portability of test software
.........................................
6.1.5 Language bindings and encodings
IS
6.2 Test documentat& .
............................................
6.2.1 Test Requirements document
.............................................
6.2.2 Test Specifications document
............................................
6.2.3 Test suite and documentation
1X
.................................................................
7 Graphics test service
7.1 Procedures and guidelines .
7.1.1 Acceptance of a test suite .
7.1.2 Establishment of test procedures .
7.1.2.1 Testing Control Board .
7.1.2.2 Testing Control Board procedures .
............................................
7.1.2.3 Applying for testing
.............................................
7.1.2.4 Testing procedures
.....................................
7.1.2.5 Preparation of the test report
0 ISO/IEC 1993
All rights reserved. No part of this publication may be reproduced or utilized in any form or
by any means, electronic or mechanical, including photocopying and microfilm, without per-
mission in writing from the publisher.
l Case Postale 56 l CH-1211 Gen&e 20 l Switzerland
I SO/I EC Copyright Off ice
Printed in Switzerland
ii
0 ISO/IEC ISO/IEC 10641r1993 (E)
................................................
7.1.2.6 Confidentiality
...............................................
7.1.2.7 Documentation
...........................................
7.1.2.8 Archiving of records
...................................................
7.1.2.9 Checklists
.................................................
7.2 Adoption of a test report format
..............................................................
7.3 Issue of licences
......................................................
7.4 Maintenance requirements
..............................................................
8 Establishing a test service
....................................................
8.1 Components of a test service
.................................................
8.1.1 The testing laboratory
....................................................
8.1.2 Accreditation body
.....................................................
81.3 Certification body
.................................................
8.1.4 Testing support service
8.1.5 CIient .
...................................................
8.2 Harmonisation of test services
Annexes
.........................................................
A Overview of the testing process
.................................................
B Overview of test suite development process
I
. . .
0 ISO/IEC
Foreword
IS0 (the International Organization for Standardization) and IEC (the International Electrotechnical Commission) form
the specialized system for worldwide standardization. National bodies that are members of IS0 or IEC participate in the
development of International Standards through technical committees established by the respective organization to deal
with particular fields of technical activity. IS0 and IEC technical committees collaborate in fields of mutual interest.
Other international organizations, governmental and non-governmental, in liaison with IS0 and IEC, also take part in the
work.
In the field of information technology, IS0 and IEC have established a joint technical committee, ISO/IEC JTC 1. Draft
International Standards adopted by the joint technical committee are circulated to national bodies for voting. Publication
as an International Standard requires approval by at least 75 % of the national bodies casting a vote.
International Standard ISO/IEC 10641 was prepared by Joint Technical Committee ISOLIEC JTC 1, Information tech-
nology, Sub-Committee SC 24, Computer graphics and image processing.
Annexes A and B of this International Standard are for information only.
iv
ISOAEC 10641:1993 (E)
0 ISO/IEC
Introduction
This International S tandard addresses conform nance testing of implementations of graphics stand‘ards. Conformance testing is
the method used to
determine the adherence or non-adherence of an implementation under test (WI’) to a staklard.
This International Standard specifies an approach for St2 ndards of products that
testing the conformance to computer graphics
claim to implement these standards. It addresses the conformance testing processes for all classes of graphics standards
This International Standard defirtes a general framework of procedures and guidelines for conformance testing, together with
definitions of terms and concepts.
The framework given in this International Standard, together with the Test Requirements document for a particular graphics
standard, provides a description of the procedures to be followed to achieve successful conformance testing of products for
conformance to a particular graphics standard.
The concept of conformance is central to every standard. The aims and benefits of a standard can be realized if there is a
means of testing for conformance.
The main reasons for introducing a document on conformance testing in the area of computer graphics are:
- To promote standards that are developed in a way such that products can be tested for conformance to the standards’
requirements;
- To promote that conformance is addressed in each standard;
- To promote test suites that are appropriately defined for testing products for conformance to all areas of the
standard, and are of high quality;
- To promote test methods for similar standards that are developed in a consistent way;
- To promote conformance testing that is carried out in a consistent way throughout the international graphics
community.
Users of this International Standard include:
- Developers of graphics standards;
- Implementors of graphics standards;
- Developers of graphics test suites;
- Testing laboratories;
- Certification bodies;
- Accreditation bodies.
Annexes A and B contain diagrams illustrating the relationships among the users of this International Standard and the
information shared by them.
This page intentionally left blank

INTERNATIONAL STANDARD
Information technology - Computer graphics and
image processing - Conformance testing of
implementations of graphics standards
Scope
This International Standard specifies a general framework for testing conformance to a computer graphics standard. The general
framework described in this International Standard addresses the following six components:
- Conformance in the standard itselfr
- Test Requirements document, defining what shall be tested for a computer graphics standard;
- Test Specifications document, addressing the test technique and the content of each test
- Test method, defining the implementation of the Test Specification document, including the test software;
- Test procedures, defining the application of the test software, which consists of the procedures to be used in
conformance testing;
- The establishment of test services.
This International Standard is applicable to all standards within the scope of the subcommittee within ISO/IEC JTCl responsible
for computer graphics and image processing.

0 ISOAEC
2 Normative references
The following st‘andcuds contain provisions which, through reference in this text, constitute provisions of this International
Stand‘ard. At the time of publication, the editions indicated were valid. All stLandcards care subject to revision, ‘and p‘arties to
agreements bczsed on this International Stand,ud of the standcards indicated below. Members of IEC and IS0 mintain registers of currently valid International Standczrds.
Gruphicul Kernel System (MS) functionul
ISO/IEC 7942: 1985, Informution processing systems - Computer gruphios -
description.
ISO/IEC 86320 1: 1992, Infbrmution technology - Computer gruphics - Metufile for the storuge und transfer of picture description
inf~wnution -Purt 1: Functional specijkution.
ISO/lEC 8632-2: 1992, Inf~wnzution technology - Computer gruphics - Metufile for the storuge und trunsfer of picture description
information -Part 2: Churucter encoding.
ISO/IEC 8632-3: 1992, Informxztion technology - Computer gruphics - Metufile for the storqe und trunsfer of picture description
inf~wnution -Purt 3: Binary encoding.
ISO/IEC 8632-4: 1992, Informution technology - Computer gruphics - Metufile for the storage und trunsfer of picture description
infwmution -Purt 4: Cleur text encoding.
Gruphicul Kernel System (GIG) lunguqe
ISO/IEC 8651-l: 1988, Information processing systems - Computer gruphics -
bindings - Purt I: FORTRAN.
- Gruphicul Kernel System (GKS) lunguuge
ISO/IEC 865 l-2: 1988, Information processing systems - Computer gruphics
bindings - Purt 2: Puscul.
Gruphicul Kernel System (GKS) lunguuge
ISO/IEC 865 l-3: 1988, Information processing systems - Computer gruphics -
bindings - Part 3: Adu.
ISO/IEC 865 l-4: 199 1, Information technology - Computer gruphics - Gruphicul Kernel System @KS) lunguuge bindings - Purt
4: c.
- Gruphicul Kernel System for Three Dimensions
ISO/IEC 8805: 1988, Information processing systems - Computer gruphics
(GKS-3D) functional description.
ISO/IEC 8806-l: 1988, Information processing - Computer gruphics - Gruphicul Kernel Systemfor Three Dimensions (GKS-3D)
language bindings - Part I: FORTRAN.
ISOIIEC 8806,4:-l), Inform&on technology - Computer gruphics - Gruphicul Kernel Systemfbr Three Dimensions (GKS-3D)
lunguuge bindings - Purt 4: C.
ISO/IEC 9592~1:1989/Amd.l: 1992, Informution processing systems - Computer gruphics - Progrummer’s Hierurchicul
Inteructive Gruphics System (PHI(X) - Purt 1: Functionul description Amendment i.
IS()/IEC 9592-2: 1989/Amd. 1: 1992, Information processing systems - Computer gruphics - Progrummer’s Hierurchkul
Interactive C;ruphics System (PHIGS) - Purt 2: Archive file $rmut Amendment I.
ISO/IEC 9592-3: 1989/&n& 1: 1992, Inf~wmtion processing systems - Computer gruphics - Progrummer’s Hierurchicul
Interuc:tive Ifruphir:s System (PHIC;S) - Purt 3: Cleur text encoding of urchive file Amendment 1.
IsO/IEC 9592-4: 1992, Inf(lrmtion proc:essing systems - Computer gruphics - Proq-ummer’s Hierurchicul Inteructive Gruphics
System (PHIGS) - Part 4: Plus Lumiere und Su$k:es, PHIGS PLUS.
1) To be published.
0 ISO/IEC
ISO/IEC 9593-1: 1990, Informtion prclcessing systems - Computer gruphics - Programmer’s Hierurchicul Inteructive (iruphil:s
System (PHIGS) language bindings - Part 1.’ FORTRAN.
ISOfIEC 9593-3: 1990, Information technology - Computer gruphics - Progrummer’s Hierurchicul Interactive Gruphics System
(PHIGS) lunguuge bindings - Purt 3: Adu.
ISO/IEC 9593-J: 1992, Inftwmution technology - Computer gruphics - Progrummer’s Hierurchicul Interutrtive Gruphics System
(PHIGS) lunguuge bindings - Purt 4: C.
ISO/IEC 9636- 1: 199 1, Information technology - Computer gruphics - Inte@xing techniques j?lr dialogues with gruphicul
devices (WI) - Functionul specijkution - Purt I : Overview, &files and confx-munce.
ISO/IEC 9636-2: 1991, Informution technology - Computer gruphics - Interfclcing techniques for diulogues with gruphicul
devices (WI) -Functionul specification - Part 2: Control.
ISO/IEC 9636-3: 1991, Information technology - Computer gruphics - Interjktring techniques for dialogues with gruphicul
devices (WI) - Functionul specification - P&t 3: Output. -
ISO/IEC 9636-4: 1991, Inf~~rmution technology - Computer gruphics - Inter$ucing techniques for dialogues with gruphicrul
devices (WI) - Functionul specification - Purt 4: Segments.
ISO/IEC 9636-S: 1991, Informution technology - Computer gruphics - Interfking techniques for diulogues with gruphicul
devices (WI) - Functional specificution - P&t 5: Input und echoing.
ISO/IEC 9636-6: 1991, Infi~rmution technology - Computer gruphics - Inte$ucing techniques for dialogues with gruphicul
devices (CGI) - Functional specificution - Purt 6: Raster. -
ISOLIEC 9637-l :-2~,Informution technology - Computer gruphics - Interfking techniques for dialogues with gruphicul devices
(CM) - Data streum binding - Part I: Character encoding.
ISO/IEC 9637-2: 1992, Information technology - Computer gruphics - Inter-fcrcing techniques for dialogues with gruphicul
devices (WI) - Dutu streum binding - Purt 2: Binury encoding
- Computer Gruphics Reference Model.
ISO/IEC 11072: 1992, Information technology - Computer gruphics
ISO/IEC Guide 2: 199 1, Generul terms and their definitions concerning stundurdizution und reluted uctivities.
ISO/IEC Guide 23: 1982, Methods of indicating conformity with stundurds for third-party certijkution systems.
ISO/IEC Guide 25: 1990, Generul requirements for the competence of culibrution und testing luborutories.
ISOfIEC Guide 2811982, General rules for u model third-purty certification system for products.
ISO/IEC Guide 45:198S, Guidelines for the presentution of test results.
2) To be published.
0 ISO/IEC
3 Definitions
For the purposes of this International Standard, the following definitions apply.
3.1 accreditation: Formal recognition that a testing laboratory is competent to ccv~y out specific tests or specific types of
tests.
3.2 accreditation body: A body which conducts ‘and administers a laboratory accreditation system and giants accreditation.
3.3 application programmer interface (API) standard: A standard which provides interface.
3.4 candidate implementation: Implementation that is tested for conform,ulce to a given standard.
3.5 certificate (of conformity): A document attesting that a product or a service is conforming to one or more specific
st,ulda.rds or technicczl specifications.
3.6 certification: Procedure resulting in the issuLvlce of a certificate.
3.7 certification body: Body that conducts certificates of conformity “[ISO/IEC Guide 21”; body that issues certificates of
conformity.
3.8 certification criteria: Criteria that determine whether a certificate is issued.
3.9 client: Anyone requesting conform,ulce testing.
3.10 configuration: An interleaved combination of hardw‘are and software including the host computer, the host operating
system, the device-independent graphics package, the device driver h‘andling graphic‘al input/output on a specific device and
the programming hanguage.
3.11 conformance: Fulfillment by a product, process or service of all requirements specified “[ISO/EC Guide 2,
conformity]“; adherence of an implementation to the requirements of one or more specific standards or technical specifications.
3.12 conformance testing: Test to evaluate the adherence or non-adherence of a ccvldidate implementation to a st‘andard.
3.13 data stream encoding: Specific representation of the syntax of a graphics st‘andard suitable for use over a data
interface.
3.14 essential feature: Features that, when grouped together, perform a minimal set of operations expected by a st‘andard.
3.15 falsification: Test method that attempts to find errors in a c‘andidate implementation to determine if it is incorrect.
3.16 full conformance: The implementation of all required features error-free.
3.17 implemmtation conformance: Description of what is required of implementors in order to have the implementation
conform to a standard.
3.18 implementation under test (IUT): A cculdidate implementation being tested.
3.19 language binding: The progmmmer’s interface to the functions in a graphics system through a programming language.
3.20 metafile standards: Smndards defining graphical elements and corresponding encodings for storage ‘and tr,ulsfer of
graphical information.
0 ISOAEC
3.21 minimal conformance: The implementation of all essential features error-free.
3.22 operator: The external object that observes the contents of the display and generates physical input values.
3.23 operator interface: Interface provided by the physical environment to the operator.
3.24 program conformance: Description of what is required of a progr,un in order to conform to a standard.
3.25 test case: Smallest unit of a test program that tests one feature of a candidate implementation.
3.26 test method: Specified technical procedure for performing a test “[ISO/IEC Guide 21”; a defined technical procedure
used to design a test suite for a given stand‘ard.
3.27 test procedures: Defines the procedures to be followed when applying a test suite to a product for the purposes of
conformance testing.
3.28 test program: A program consisting of a set of test cases.
3.29 test report: A document that presents the test results and other information relev‘ant to the tests (e.g., configuration
description ‘and detected errors).
Describes the features ‘and functions defined in a pcuticu1‘a.r stand‘ard to which a
3.30 test requirements document:
c‘andidate implementation shall be tested for conform,ulce.
3.31 test result: Output of a test ase.
3.32 test script: Document describing the test software and its v,uious test cases for operator guidance ‘and decision support.
3.33 test service: Service offered by testing laboratories.
3.34 test specifications document: Describes procedures, tools iand test ceases that Test Requirements document.
3.35 test software: Set of test progr,uns cznd corresponding documentation that care used for conform,ulce testing.
for
cvld test procedures that check an implementation
3.36 test suite: The combination of test softw‘are, test documentation,
conform,ulce to a standcard.
standards and testing set up for each or type of products to
3.37 testing control board: A bo‘ard of experts in
group
problems raised (as a result of conformance testing.
3.38 testing laboratory: Laboratory that performs tests “[ISO/IEC Guide 21”; a laboratory that measures conform,ulce of
implementations of standards to a group or type of standards.
3.39 testing support service: Orgculization responsible for a p‘articular test service (e.g., maintenance of test suite, licensing).
3.40 validation: Testing for conformance.
3.41 verification: Method to prove the correctness of a candidate implementation against standards or technical specifications.

ISOAEC 10641:1993 (E)
0 ISO/IEC
4 Overview
Correct utilization of standards allows applications to be moved among different computers and graphics devices with minimal
ch,ulge. The degree of portability achieved is affected by the degree of support for the sca.me sets of functions, levels, and
implementation dependent features of (an implementation. Conformcvlce testing using test suites encourages implementors to
use the standczrds correctly by checking for deviations ‘and omissions from the standards. When all deviations and omissions
,are eliminated, the implementation conforms to the stand~ard. This greatly diminishes or removes the work involved to achieve
portability between dissimilar systems. In rncvly countries, an implementation of a standard that has obtained a certificate is
favoured for purchase by the public. In addition, the certificate may be mandated for sales into certain markets, such as the
government.
Even before a standard becomes official, there is a strong effort to implement products that (claim to) conform to the proposed
standczrd. This is because even d&t standcards create a strong user dem,uld. For ex‘atnple, vendors started advertising
implementations of ca.n IS0 graphics standard at least one ye(zr before the st‘and‘ard wcas actu‘ally approved by ISO/IEC.
Shortly
after that, these implementations were available on the marketplace. Without defined test methods, some of these packages
be‘ar only superficial resemblance to a correct implementation of the standard. Other implementations might conform to many
aspects of the standard but might still be deficient in subtle La.reas. Thus, test suites that test implementations to determine
conformance to standcuds ,a.re needed.
4.1 Conformance testing
Conform,ulce testing is a way of scrutinizing implementations of a computer standczrd to determine whether or not deviations
from the standard exist. Standards usually contain two main ingredients: semantics and syntax. The semantics is the
functional description that defines precisely what must be done, but not how it is to be done. The syntax defines the
mechanism by which these functions an be accessed. Syntax may consist of verbs in a progr,unming language to access the
function, or (in the c t,?ke the form of data stream encodings for data interchange st,uld However, a third ingredient is sometimes overlooked in standards - conformance. The conformance or classification and
designation clause, in conjunction with the rest of the standard, specifies the requirements an implementor shall adhere to in
order to conform to the stand‘ard and sets the groundwork for the development of conformance tests.
A test suite is the combination of test softw‘are, test script, and test procedures, aI of which check an implementation for
conformance to a sta.nd,ud.
The approach usually used in developing test suites for testing conformance of implementations is called falsification testing.
This method uses sample c A test suite tries to find
errors in the implementation. If errors are found, one can correctly deduce that the implementation does not conform to the
standard. However, the absence of errors does not necesa-ily prove that the implementation is correct. The absence of errors
implies either that the implementation conforms to the st‘and errors. Thus, f‘alsification testing can determine non-conform,ulce to a st‘andca.rd but ccan never cassure complete conformance.
Falsification testing is the method used to test conformance to graphics standards.
A Test Requirements document describes the features ‘and functions defined in a particukar standard to which ‘an implementation
It is vital to the correctness and completeness of a test suite that the Test
under test (IUT) sh‘all be tested for conform,ulce.
Requirements document be developed in p‘araIle1 with the standczrd. This guarantees communication between the standards
developers and test suite developers, which ensures that features in the standard ,a.re defined in a manner which makes testing
possible. It also ensures that the tests (are more comprehensive because the test developers are directly involved in the
specification of the standczrds and can more thoroughly understand what needs to be tested.
4.2 Graphics conformance testing
Generally, the sem‘antics for standards are specified using the English language in n,urative form. As suitable formal
specification techniques mature ‘and experience in their usage grows, it is desirable that the semantics of standards be specified
using such techniques. This would reduce the probability of errors ,uld ambiguities in the standard. Additionally, if formal
specifications existed, formal proofs of correctness could be developed which prove conformance to the standard. This is
contrasted with falsification testing which can only indicate non-conform(ulce to the standcard.
0 ISOAEC
The absence of formal specifications hcas a profound effect on conform,ulce testing. Without a method for defining an
umambiguous definition of a stand‘ard (a form~aI specification), there is a d‘anger that test routines may become the definition
of the standdard. This is because (any questions of interpretation of the English hanguage sem‘antics or questions of (ambiguities
must be resolved by the developer of a test suite prior to tr,2nscribing the semMtics into vczlidation softw,ue. The possibility
that the criteria for testing conforn-uance may be b‘ased on subjective interpretation by a test developer necessitates the
establishment of a Testing Control Bo(zrd for each functiomal stand‘ard. This Bo‘ard will oversee the development ‘and
m,?inten,ulce of the tests ‘and h,urdle questions of interpretation.
Syntax in graphics standards may consist of hanguage bindings that allow a progr,unmer to access graphics functions in the
hanguage he is using (e.g., FORTRAN, C, Pascal, Ada). Conform,ulce testing aims to ensure that the functions perform
correctly and also that the hanguage bindings have been implemented correctly. Syntax in data interch‘ange standcards, such as
ISO/IEC 8632-l (CGM), specify data encodings for the CGM functionality. For these stand‘ards, conform,ulce tests czirn to
ensure that these encodings (are implemented correctly.
The development of test suites for graphics standards is unique due to the pictorial nature of graphics output. Tests for
conformance to other standards, such as progmmming language standards, can automatic,2lly compare the desired results with
the actual results ‘and c‘an easily m,?ke a pass/fail determination with respect to conform‘ance of the IUT. However, in the
present method of evaluating conformdance of proposed standard-conforming graphics implementations, visual analysis of
Examples of questions (asked during this ‘analysis pictorial information is required.
“is the line style recognizable as a series of dashed lines”, etc. If API standards were more precise,
recognizable cas text”,
testing could be more automated ‘and less subjective judgement would be required.
4.2.1 The impact of registration
Conformance tests concentrate strictly on implementation conform,ulce. They test that the implementation meets all mandatory
requirements of a star&-d, as well as optional requirements. For example, ISO/IEC 7942 (GKS) requires that four m‘andatory
linetypes exist for the polyline primitive. Thus the GKS test suite ensures that the four m,uldatory linetypes (are implemented
correctly including the ‘assignment of correct vczlues for each linetype (i.e., linetype 1 = solid, linetype 2 = dashed, etc.).
In addition, ‘an implementor may supply cvly number of linetypes by ‘assigning negative values to them. The additional linetypes
may be specific to his implementation. Optiomally, he may choose to share these linetypes with other implementations so that
users c‘an obtain portability. In this c‘ase positive v(alues are assigned to the linetypes and documented in the IS0 Register of
Graphical Items.3)
3) For the purpose of this International Standard and according to the rules for the designation and operation of registration
authorities in the ISO/IEC JTCl procedures, the IS0 and IEC Councils have designated the National Institute of Standards
and Technology (Computer Systems Laboratory), A-266 Technology Building, Gaithersburg, MD 10899, USA, to act as
registration authority.
0 ISO/IEC
4.2.2 Interfaces in computer graphics conformance testing
ISO/IEC 11072 defines the external interfaces to the computer graphics environment in an overall structure as shown in
Figure 1.
f
Application
.
b
Application
--I-B-----B-B
---
IntiXce-
Computer Graphics
Environments
rator
---m-v-------- ape
--
I&G -
Operator
Figure 1 - External Interfaces
The txtxn;~l in terikxs to computer graphics arc:
- Opentor interface. which is the interface provided by the lowest internal awironrncnt ot’ the wnputcr gxphics
environment to the operator:
- Application interface. which is the interface provided by the upper most environment of the computer graphics
environment to the application:
- Data capture metafile interfaces. which ‘are the interfaces provided by each of the defined internal environments of
the computer graphics environment for importing and exporting al1 or part of the data elements defined bv ISO/IEC
d
11072;
- Audit tnil metafile interface, which is the interface provided to record or pl;~h;~ck the tlow of information across
d
the application interface.
In addition. interfaces may be defined between the internal environments of the computer graphics environment that rnav also
d
be subject to conformance testing. The vi.rtuaI device interface is an example of this form of interface.
All interfaces defined in a graphics standard need to be tested. For application programmer interface (API) st;md;ucis. at ktst
the operator interface a.nd the :ipplication interface should be tested. Examples of API standards ;ue ISO/IEC 7942 (GKS) and
ISO/IEC 9592 (PHIGS).
Figure 2 is ‘an example of a device-independent graphics system. In figure 2 the application interface is the interface between
the application program and the device-independent graphics package (i.e., implementations of GKS or PHIGS). Test programs
at this interface ensure that the proper information is returned within the application program when calls to c;u-Alate
implementations ‘are invoked.
The operator interface is the interface between a graphics device ‘and a hum&an being looking at the pictures produced. Testins
at this interface can be less subjective when API standards define the visaal appearance of graphics primitives prtxisely. Fvr
example, the tester might not be able to differentiate between dashed and dotted lines (both m,uldatory line styles for the
polyline primitive) because the implementor might choose to m‘ake dots ‘and dashes that differ only slightly in length. However.
the standards should specify that dashes should be “n” times the width of dots so that the difference is apparent ‘and can be
objectively evaluated by the tester. In gener‘al, test suite developers should write tests that do Las much automated checking of
results as possible. However, testing at the operator interface is still subjective and does not lend itself to automated checkins
because it requires interpretation of computer-generated output.
ISOAEC 10641:1993 (E)
0 ISO/IEC
ISO/IEC 8632-l 19 (CGM) is a graphical data inttxchc2nge standard. In figure 2, CGM is a file that is produced by a generator
‘and rendered by ‘an interpreter. ISO/IEC IS 9636-l (CGI) is the interface between the device-independent graphics package
,uld the device-dependent device drivers. CGI resides at the virtual device interface. Tests for the CGI are at this interface.
Graphics Systems Architecture
Database
Application Programming interface
C
Metafile
Device - Independent
+ G
Interpreter
M
Graphics Package
(Reader) I
Virtual
- Device
Interface
Metafile
Generator
(Writer)
Metafile
Terminal
Plotter
CGI Virtual Device
Figure 2 - Reference Model
One thing that must be remembered in graphics conformance testing is that the c‘andidate implementation is an interleaved
combination of h‘ardw‘are ‘and softw implementation must pcass tests for each host system it is intended for 4.3 The testing process
A client who wishes to have his implementation tested for conform,ulce, using a p‘articular graphics test service, applies to accredited testing laboratory. The client signs a contract with the testing laboratory ‘and receives the test suite. The client may
then test the implementation and make changes to it to correct any errors. Formal testing of the implementation by the testing
laboratory is then ccvried out. Upon completion of this formal testing, the testing laboratory issues a drczft test report. The draft
test report is agreed upon by the client cznd the testing laboratory prior to issuing the final test report. The client may then
submit the test report to the certification body. If the criteria for issuing a certificate have been met, then a certificate will be
issued. Clients may submit test reports to certification bodies in MY country to receive a certificate. (See figure A.1 .)
0 ISO/IEC ISO/IEC 10641:1993 (E)
5 Conformance testing requirements within graphics standards
The standards developers should avoid specifying non-testable features in a stand‘ard. A standard itself should be precise (and
clecar.
Often, applications do not use all of the features within a stand‘ard. For instance, specific workstations might have only limited
input capabilities. A hard-copy plotter h(as no input devices, and applications generating pictures on a plotter would not require
input. In order to accommodate these applications it is often desirable to pcvtition a stand above ex‘ample input functions would be one set. Implementors the entire standczrd. These sets All graphics standcvds shall contain a conformcvlce clause. The conform,ulce clause of a graphics standard shall be contained
within a clause entitled “Classification ‘and Designation” (this clause is ‘also entitled “Conformance” in older standards). The
concepts of program conform,ulce ‘and implementation conform,ulce shcall be defined in this clause.
The st In the absence of sets of functions, implementation conform‘ance shall
implementation ‘arid a conforming application progr,un.
be defined as implementing at least all of the functions ‘and their minimal requirements. If different sets of functions care
defined in a sta.nd,arcl, the conform,ulce clause shall expkain which sets are required to be implemented for implementation
conform,ulce.
In the conformcznce clause, standards developers shall explicitly state what is required of This information is necessary for implementations and
‘and a progr,unmer using a standard-conforming implementation.
applications to conform to the standard.
The conform,ulce clause is a high-level description of what is
Requirements required of implementors ‘and progr‘ammers, cvld refers to other clauses in the stand‘ard. The conform,ulce clause may specify
sets of functions, which may take the form of profiles or levels or other requirements. The conformLance clause may specify
minimal requirements for certain functions ‘and minimczl requirements for implementation-dependent vczlues.
Progr,un conform,ulce shall be defined set of functions if optional requirements exist. Progmm conformance places the responsibility on the progiammer to avoid the
use of extensions to achieve portability.
Functional standards shall allow extensions only if the following conditions - Functiomally conforming programs shall still execute correctly;
- Extensions sh‘all not re-define sem,ultics for existing functions;
- Extensions shall not cause standard-conforming functions to execute incorrectly;
- The implementation shall provide a mech,ulism that enables an application to check whether can application is
program conforming;
- The mech‘anism for determining progr,un conform‘ance and the extensions shall be clearly described in the
documentation, (and the extensions shall be m‘arked as such;
- Extensions sh If further restrictions need to be imposed on extensions, these shaI1 be stated in the conform(ulce clause.
The conform,ulce clause of each standard shall address the following three 1. Implementation-independent r,ulges, data, minimum or mLwimum allowed sizes, etc.;
2. Values which may be different for different conforming implementations of the standard;
3. Features reserved for registration.
The conformance clause of graphics standards shcal.l require that at least one standczrd kanguage binding or encoding be provided
in order to be a standczrd conforming implementation.
Some standards require (an implementor to supply documentation. For instculce, a stand‘ard may require that implementation-
defined features be documented to explain how they are implemented. In this cease assessing the documentation is part of the
conform,ulce testing process.
Each graphics stculd‘ard, which shall contain a conform,ulce clause, shall define implementation conform,ulce. If the stLuldard
is intended to be used by progr~ammers, progr,cun conform,ulce shLall ‘also be defined.
The (sub)clause on implementation conform,ulce of each graphics stand‘ard shaI1 m,uldate the implementation of a minimum
set of functionality. However, implementors may supply extensions that rnczke their implementations more useful for a certain
class of users. These additional functions may eventually migrate into future revisions of the stand The (sub)clause on progr,un conform,ulce shall m,uldate that only the functions defined in the standard shall be used. If
programmers use the extensions they do not conform to the standard. Use of the extensions must be avoided in order to
achieve portability. However, there may be situations where the progrcvnmer wishes to use extensions:
- If the programmer kIlOI% 5 can application will never have to run on different hardwcue or system software;
- Tl le application wi .U not meet requirements w ithout extensions.
If both implementation stand‘ards without losing the benefits of portability.
EssentiaI features Lare features that, when grouped together, perform a minimal set of operations expected by a standard. A
standLard may specify essenti‘al features. If so, they shall be stated in the conform,ulce clause.
Users must have a high degree of confidence that the implementation correctly implements the essential features of the standard.
Since certification bodies may choose to allow implementations to contain errors and still obtain certificates, the essential
features must be implemented error-free for minimal conform,ulce to be obtained ‘and to acquire a certificate.
All required
features must be implemented error-free for full conformance to be obtained.
It may be desirable to test interfaces between the different standcuds to ensure that the family of graphics stand,?rds works
correctly as a complete system. Interfaces between different standards are outside the scope of each individuczl standard, and
thus ccannot be mandated. Therefore, each product claiming to conform to the standard shall be tested sepcarately for each
functional standard.
The conformance clauses of functional standards shall be developed by both the functional standards developers and the
working group responsible for validation ‘and testing within ISO/IEC. This is because the functional stand‘ards developers are
experts on the details of the implementation requirements of the functional st‘a.ndc?rds, and the working group responsible for
validation ‘and testing is the expert on the general principles of conform,ulce ‘and testing. The combination of knowledge from
both groups will result in a complete ‘and realistic conformance clause for each functional standard.
Products implementing computer graphics standczrds ,a.re embedded in a h,udw‘are and software environment. This environment
differs cvnong implementations of a standard. All important components from which m,ulufacturers will m,?ke up the
environment for a product meeting a specific computer graphics stand‘ard should be listed in the conform‘ance clause of that
standcard.
0 ISOAEC
6 Graphics test suite
Products claiming to meet the requirements of computer graphics stand‘ards (are usuczlly implemented ‘and sold products. Test suites (are necessary to determine whether (an implementation conforms to the standard. A graphical test suite
is a set of softw‘are files cvld operator procedures that ccan be used to determine whether stand‘ard f,zils to adhere to a standard.
6.1 Test software
A test suite is designed to test the conform,ulce of implementations of a graphics standLard. Testing is done by applying a
number of test cases to the graphics system under test (are determined from the graphics standcard by the test suite developer.
The IUT is tested against the requirements of the standard or a set of standards. A test suite should not assess the perform,ulce
of the IUT with respect to the functionaI standard ‘and hanguage binding stand‘ard unless perform,ulce requirements are specified
in the graphics standard, although implementation dependencies or machine dependencies may be demonstrated through the
execution of the test cases. For conformance testing, only the requirements specified in the standard should not check cvly product properties that are not described by the st(and The results of conformance testing apply only to the graphics implementation and envir
...