ISO/TC 69/SC 6/WG 1 - Accuracy of measurement methods and results

This document — introduces conditions, constraints and resources necessary to evaluate a measurement method or a result; — defines an organizational scheme for the acquisition of trueness and precision data by study; — provides the necessary definitions, statistical model and principles for ISO 5725 (all parts). — is not applicable to proficiency testing or production of the reference item that has their own standards (ISO 13528, respectively and ISO Guide 35). This document is concerned exclusively with measurement methods which yield results on a continuous scale and give a single value as the test result, although this single value may be the outcome of a calculation from a set of observations. It defines values which describe, in quantitative terms, the ability of a measurement method to give a true result (trueness) or to replicate a given result (precision). Thus, there is an implication that exactly the identical item is being measured, in exactly the same way, and that the measurement process is under control. This document may be applied to a very wide range of test items, including gas, liquids, powders and solid objects, manufactured or naturally occurring, provided that due consideration is given to any heterogeneity of the test item. This document does not include methods of calculation that are described in the other parts.

This document provides a) a discussion of alternative experimental designs for the determination of trueness and precision measures including reproducibility, repeatability and selected measures of intermediate precision of a standard measurement method, including a review of the circumstances in which their use is necessary or beneficial, and guidance as to the interpretation and application of the resulting estimates, and b) worked examples including specific designs and computations. Each of the alternative designs discussed in this document is intended to address one (or several) of the following issues: a) a discussion of the implications of the definitions of intermediate precision measures; b) a guidance on the interpretation and application of the estimates of intermediate precision measures in practical situations; c) determining reproducibility, repeatability and selected measures of intermediate precision; d) improved determination of reproducibility and other measures of precision; e) improving the estimate of the sample mean; f) determining the range of in-house repeatability standard deviations; g) determining other precision components such as operator variability; h) determining the level of reliability of precision estimates; i) reducing the minimum number of participating laboratories by optimizing the reliability of precision estimates; j) avoiding distorted estimations of repeatability (split-level designs); k) avoiding distorted estimations of reproducibility (taking the heterogeneity of the material into consideration). Often, the performance of the method whose precision is being evaluated in a collaborative study will have previously been assessed in a single-laboratory validation study conducted by the laboratory which developed it. Relevant factors for the determination of intermediary precision will have been identified in this prior single-laboratory study.

This document provides statistical techniques for the determination of the reproducibility of the level of detection for a) binary (qualitative) test methods for continuous measurands, e.g. the content of a chemical substance, and b) binary (qualitative) test methods for discrete measurands, e.g. the number of RNA copies in a sample. The reproducibility precision is determined according to ISO 5725 (all parts). Precision estimates are subject to random variability. Accordingly, it is important to determine the uncertainty associated with each estimate, and to understand the relationship between this uncertainty, the number of participants and the experimental design. This document thus provides not only a description of statistical tools for the calculation of the LOD reproducibility precision, but also for the standard error of the estimates.

This document provides detailed descriptions of statistical methods for proficiency testing providers to use to design proficiency testing schemes and to analyse the data obtained from those schemes. This document provides recommendations on the interpretation of proficiency testing data by participants in such proficiency testing schemes and by accreditation bodies. The procedures in this document can be applied to demonstrate that the measurement results obtained by laboratories, inspection bodies, and individuals meet specified criteria for acceptable performance. This document is applicable to proficiency testing where the results reported are either quantitative measurements or qualitative observations on test items. NOTE The procedures in this document can also be applied for the assessment of expert opinion where the opinions or judgments are reported in a form which can be compared objectively with an independent reference value or a consensus statistic. For example, when classifying proficiency test items into known categories by inspection - or in determining by inspection whether proficiency test items arise, or do not arise, from the same original source - and the classification results are compared objectively, the provisions of this document that relate to nominal (qualitative) properties can be applied.

This document introduces five statistical methods for evaluating the precision of binary measurement methods and their results. The five methods can be divided into two types. Both types are based on measured values provided by each laboratory participating in a collaborative study. In the first type, each laboratory repeatedly measures a single sample. The samples measured by the laboratories are nominally identical. The second type is an extension of the first type, where there are several levels of samples. For each statistical method, this document briefly summarizes its theory and explains how to estimate the proposed precision measures. Some real cases are illustrated to help the readers understand the evaluation procedures involved. For the first and second types of methods, five and three cases are presented, respectively. Finally, this document compares the five statistical methods.

1.1 This document — specifies basic methods for estimating the bias of a measurement method and the laboratory bias when a measurement method is applied; — provides a practical approach of a basic method for routine use in estimating the bias of measurement methods and laboratory bias; — provides a brief guidance to all personnel concerned with designing, performing or analysing the results of the measurements for estimating bias. 1.2 It is concerned exclusively with measurement methods which yield measurements on a continuous scale and give a single value as the measurement result, although the single value can be the outcome of a calculation from a set of observations. 1.3 This document applies when the measurement method has been standardized and all measurements are carried out according to that measurement method. NOTE In ISO/IEC Guide 99:2007(VIM), "measurement procedure" (2.6) is an analogous term related to the term "measurement method" used in this document. 1.4 This document applies only if an accepted reference value can be established to substitute the true value by using the value, for example: — of a suitable reference material; — of a suitable measurement standard; — referring to a suitable measurement method; — of a suitable prepared known sample. 1.5 This document applies only to the cases where it is sufficient to estimate bias on one property at a time. It is not applicable if the bias in the measurement of one property is affected by the level of any other property (i.e. it does not consider interferences by any influencing quantity). Comparison of the trueness of two-measurement methods is considered in ISO 5725-6.

1.1 This document — amplifies the general principles for designing experiments for the numerical estimation of the precision of measurement methods by means of a collaborative interlaboratory experiment; — provides a detailed practical description of the basic method for routine use in estimating the precision of measurement methods; — provides guidance to all personnel concerned with designing, performing or analysing the results of the tests for estimating precision. NOTE Modifications to this basic method for particular purposes are given in other parts of ISO 5725. 1.2 It is concerned exclusively with measurement methods which yield measurements on a continuous scale and give a single value as the test result, although this single value can be the outcome of a calculation from a set of observations. 1.3 It assumes that in the design and performance of the precision experiment, all the principles as laid down in ISO 5725-1 are observed. The basic method uses the same number of test results in each laboratory, with each laboratory analysing the same levels of test sample; i.e. a balanced uniform-level experiment. The basic method applies to procedures that have been standardized and are in regular use in a number of laboratories. 1.4 The statistical model of ISO 5725-1:1994, Clause 5, is accepted as a suitable basis for the interpretation and analysis of the test results, the distribution of which is approximately normal. 1.5 The basic method, as described in this document, (usually) estimates the precision of a measurement method: a) when it is required to determine the repeatability and reproducibility standard deviations as defined in ISO 5725-1; b) when the materials to be used are homogeneous, or when the effects of heterogeneity can be included in the precision values; and c) when the use of a balanced uniform-level layout is acceptable. 1.6 The same approach can be used to make a preliminary estimate of precision for measurement methods which have not reached standardization or are not in routine use.

The purpose is to give some indications of the way in which accuracy data can be used in various practical situations by: giving a standard method of calculating the repeatability limit, the reproducibility limit and other limits, providing a way of checking the acceptability of test results obtained under repeatability or reproducibility conditions, describing how to assess the stability of results within a laboratory over a period of time, describing how to assess whether a given laboratory is able to use a given standard measurement method in a satisfactory way, describing how to compare alternative measurement methods.

ISO 13528:2005 complements ISO Guide 43 (all parts) by providing detailed descriptions of sound statistical methods for organizers to use to analyse the data obtained from proficiency testing schemes, and by giving recommendations on their use in practice by participants in such schemes and by accreditation bodies. ISO 13528:2005 can be applied to demonstrate that the measurement results obtained by laboratories do not exhibit evidence of an unacceptable level of bias. It is applicable to quantitative data but not qualitative data.

Provides basic methods for estimating the bias of a measurement method and the laboratory bias when a measurement method is applied. In order that the measurements are made in the same way, it is important that the measurement method has been standardized. Can be applied only if the accepted reference value can be established as a conventional true value, e.g. by measurement standards or suitable reference materials or by referring to a reference measurement method or by preparation of a known sample.

Specifies four intermediate measures due to changes in observation conditions (time, calibration, operator and equipment) within a laboratory. These intermediate measures can be established by an experiment within a specific laboratory or by an interlaboratory experiment. Furthermore, discusses the implications of the definitions of intermediate precision measures, presents guidance on the interpretation and application of the estimates of intermediate precision measures in practical situations, discusses the connections between trueness and measurement conditions.

Amplifies the general principles to be observed in designing experiments for the numerical estimation of the precision of measurement methods by means of a collaborative interlaboratory experiment, provides a detailed practical description of the basic method for routine use in estimating the precision of measurement methods, provides guidance to all personnel concerned with designing, performing or analysing the results of the tests for estimating precision. Annex B provides practical examples of estimating the precision of measurement methods by experiment.

The purpose is to outline the general principles to be understood when assessing accuracy (trueness and precision) of measurement methods and results, and in applications, and to establish practical estimations of the various measures by experiment. Is concerned exclusively with measurement methods which yield measurements on a continuous scale and give a single value as the test result. May be applied to a very wide range of materials, including liquids, powders and solid objects, manufactured or naturally occurring, provided that due consideration is given to any heterogeneity of the material.