ISO/TR 11462-4:2022

TECHNICAL ISO/TR
REPORT 11462-4
First edition
2022-02
Guidelines for implementation of
statistical process control (SPC) —
Part 4:
Reference data sets for measurement
process analysis software validation
Lignes directrices pour la mise en œuvre de la maîtrise statistique des
processus (MSP) —
Partie 4: Jeu de données pour la validation des logiciels d'analyse de
processus de mesure
Reference number
© ISO 2022
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting on
the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address below
or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii
Contents Page
Foreword .iv
Introduction .v
1 Scope . 1
2 Normative references . 1
3 Terms and definitions, and symbols and abbreviated terms . 1
3.1 Terms and definitions . 1
3.2 Symbols and abbreviated terms . 2
3.3 Abbreviated terms . 4
4 Overview of the test examples . 4
4.1 Overview . 4
4.2 Notes . 5
4.2.1 Notes on the accuracy of the test examples and results . 5
4.2.2 Note on outlier detection . 5
4.2.3 Note on capability indices . 5
4.2.4 Note on the model of the measurement and correlations . 5
4.2.5 Note on other reference data sets . 5
4.2.6 Note on systematic errors . 6
5 Reference data sets description and evaluation . 6
5.1 Test data set 1 – example of linearity study with at least three standards . 6
5.1.1 Test data set 1 – information . 6
5.1.2 Test data set 1 – data, calculations and results. 6
5.2 Test data set 2 – attribute measurement process – operator influence (ISO 22514-7) .12
5.2.1 Test data set 2 – information .12
5.2.2 Test data set 2 – data, calculations and results.12
5.3 Test data set 3 – attributive measurements – capability calculations using
reference values – calculation of the uncertainty range (ISO 22514-7) .13
5.3.1 Test data set 3 – information . 13
5.3.2 Test data set 3 – data, calculations and results.13
5.4 Test data set 4 – measurement process capability with three reference standards
(VDA 5) . 16
5.4.1 Test data set 4 – information . 16
5.4.2 Test data set 4 – data, calculations and results. 16
5.5 Test data set 5 – Measurement Process Capability of a CMM (VDA 5 and ISO 15530-
3) . 19
5.5.1 Test data set 5 – information . 19
5.5.2 Test data set 5 – data, calculations and results. 20
5.6 Test data set 6 – measurement process capability of automated test device .23
5.6.1 Test data set 6 – information . 23
5.6.2 Test data set 6 – data, calculations and results.23
5.7 Test data set 7 – measurement process capability of a multiple-point measuring
Instrument (VDA 5) . 27
5.7.1 Test data set 7 – information . 27
5.7.2 Test data set 7 – data, calculations and results. 27
Bibliography .32
iii
Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards
bodies (ISO member bodies). The work of preparing International Standards is normally carried out
through ISO technical committees. Each member body interested in a subject for which a technical
committee has been established has the right to be represented on that committee. International
organizations, governmental and non-governmental, in liaison with ISO, also take part in the work.
ISO collaborates closely with the International Electrotechnical Commission (IEC) on all matters of
electrotechnical standardization.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for the
different types of ISO documents should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject of
patent rights. ISO shall not be held responsible for identifying any or all such patent rights. Details of
any patent rights identified during the development of the document will be in the Introduction and/or
on the ISO list of patent declarations received (see www.iso.org/patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to
the World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT), see
www.iso.org/iso/foreword.html.
This document was prepared by Technical Committee ISO/TC 69, Applications of statistical methods,
Subcommittee SC 4, Applications of statistical methods in product and process management.
A list of all parts in the ISO 11462 series can be found on the ISO website.
Any feedback or questions on this document should be directed to the user’s national standards body. A
complete listing of these bodies can be found at www.iso.org/members.html.
iv
Introduction
The test examples were developed for the assessment of systems performing a measurement system
analysis (MSA). They allow MSA software developers to evaluate their systems. Thus, the end user of
those systems can be sure that the data sets are evaluated correctly with a high level of reliability.
In order to cover as wide a spectrum as possible, suitable data sets were prepared individually for
various constellations. The evaluation results of those data sets are documented and commented on
the following pages.
The results were verified multiple times using different computer programs. This turns the data sets
and the results into references for validation of the software. The data sets are listed in the related
clauses of this document or can be accessed via https://standards.iso.org/iso/tr/11462/-4/ed-1/en.
v
TECHNICAL REPORT ISO/TR 11462-4:2022(E)
Guidelines for implementation of statistical process
control (SPC) —
Part 4:
Reference data sets for measurement process analysis
software validation
1 Scope
This document describes examples for software validation for software implementing the standards of
ISO 22514-7 on the capability of measurement processes. In detail, the following standards are covered:
— ISO 22514-7.
It provides data sets and test results for testing the implementation of the evaluation methods described
in these standards. This includes:
a) the calculation of standard uncertainties from other sources (other than experiments – type B –
ISO/IEC Guide 98-3);
b) the estimation of uncertainty components using repeated measurements on reference parts;
c) the estimation of uncertainty components using repeated measurements on multiple parts with
different operators and their evaluation using the ANOVA method;
d) the combination of uncertainty components using the Gaussian law of uncertainty propagation;
e) the calculation of measurement process capability indices;
f) the influence of operators on attributive measurements;
g) the uncertainty range and capability indices for attributive measurements.
The test examples are intended to cover the calculation of the measuring system capability and
measurement process capability according to ISO 22514-7.
2 Normative references
The following documents are referred to in the text in such a way that some or all of their content
constitutes requirements of this document. For dated references, only the edition cited applies. For
undated references, the latest edition of the referenced document (including any amendments) applies.
ISO 22514-2, Statistical methods in process management — Capability and performance — Part 2: Process
capability and performance of time-dependent process models
3 Terms and definitions, and symbols and abbreviated terms
3.1 Terms and definitions
For the purposes of this document, the terms and definitions given in ISO 22514-2 apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— ISO Online browsing platform: available at https:// www .iso .org/ obp
— IEC Electropedia: available at https:// www .electropedia .org/
3.2 Symbols and abbreviated terms
Symbols used in this standard are identical to symbols used in ISO 22514-7.
a half width of a distribution of possible values of input quantity
a maximal form deviation
OBJ
α significance level
B bias
i
C capability index for attributive measurement
attr
C measurement process capability index
MP
C measuring system capability index
MS
d average interval
d interval from the last reference value, for which all operators have assessed the result as
LR
unsatisfied to the first reference value, for which all operators have the result as approved
d interval from the last reference value, for which all operators have assessed the result as
UR
approved to the first reference value, for which all operators have the result as unsatisfied
e residuals
nj
K number of repeatability measurements
k coverage factor
k coverage factor from the calibration certificate
CAL
L lower specification limit
l measured length
M the number of subgroups
M maximum permissible error (of the measuring system) (MPE-value)
PE
m frequencies in Bowker-test
ij
N number of standards
n sample size of each subgroup
Q attributive measurement process capability ratio
attr
Q measurement process capability ratio
MP
Q measuring system capability ratio
MS
Q capability ratio limit for measuring system
MS_max
Q capability ratio limit for measurement process
MP_max
R resolution of measuring system
E
ˆ
σ
sample standard deviation
T temperature
U upper specification limit
U uncertainty on the calibration of standards
CAL
u standard uncertainty on the coefficient of expansion
α
u standard uncertainty from the operator´s repeatability
AV
u standard uncertainty from the measurement bias
BI
u calibration standard uncertainty on a standard
CAL
u standard uncertainty from maximum value of repeatability or resolution
EV
u standard uncertainty from repeatability on standards
EVR
u standard uncertainty from repeatability on test parts
EVO
u standard uncertainty from reproducibility of the measuring system
GV
u standard uncertainty from interactions
IAi
u standard uncertainty from linearity of the measuring system
LIN
u standard uncertainty calculated based on maximum permissible error
MPE
u combined standard uncertainty from other influence components not included in the analysis
MS-REST
of the measuring system
u standard uncertainty from test part inhomogeneity
OBJ
u standard uncertainty from resolution of measuring system
RE
u standard uncertainty from other influence components not included in the analysis of the
REST
measurement process
u standard uncertainty from the stability of measuring system
STAB
u standard uncertainty from temperature
T
u standard uncertainty from temperature expansion coefficients
TA
u standard uncertainty from temperature difference between workpiece and measuring system
TD
U expanded measurement uncertainty on an attributive measurement
attr
u combined standard uncertainty of attributive measuring
attr
U expanded measurement uncertainty of the measuring system
MS
u combined standard uncertainty on measuring system
MS
U expanded measurement uncertainty of the measurement process
MP
u combined standard uncertainty on measurement process
MP
u standard uncertainty from effect of spindle clamping
MX
u standard uncertainty from resolution
RA
u standard uncertainty from repeatability
RE
th
x
i reference quantity value
i
x reference quantity value of the standard (master) at the upper specification limit
mU
x reference quantity value of the standard (master) in the centre of the specification
mm
x reference quantity value of the standard (master) at the lower specification limit
mL
x
arithmetic mean of the conventional true values
th
y
j measurement value
j
y
arithmetic mean of the measured values
3.3 Abbreviated terms
ANOVA analysis of variance
MSA measurement systems analysis
MPE maximum permissible error
4 Overview of the test examples
4.1 Overview
For an overview of the test examples see Table 1.
Table 1 — List of the test data sets
Test
Sub- Character- Decimal Source/ Refer-
data set Description of data set
clause istics type points ence
number
All uncertainty components mentioned in the
22514-7 are covered. Combination of type A ISO 22514-7
1 5.1 Variable 2
and type B evaluation, including Linearity + additions
and GRR studies
Test on influence of operators based on ex-
2 5.2 Attributive --- ISO 22514-7
perimental data
Calculation of uncertainty range and capabil-
3 5.3 Attributive --- (6) ity of the attributive measurement process ISO 22514-7
based on experimental data
Measurement process capability with three
reference standards
Linearity study, GRR with ANOVA
4 5.4 Variable 4 VDA 5
Multiple uncertainty components: resolu-
tion, calibration, repeatability, linearity, bias,
operators, part-interaction
Table 1 (continued)
Test
Sub- Character- Decimal Source/ Refer-
data set Description of data set
clause istics type points ence
number
Measurement process capability of a CMM
Repeatability and bias with one standard
VDA 5 and
5 5.5 Variable 4
Multiple uncertainty components: resolu- ISO 15530-3
tion, calibration, repeatability, linearity, bias,
temperature
Measurement process capability of automat-
ed test device
Multiple measurements on one standards and
6 5.6 Variable 4 10 parts VDA 5
Multiple uncertainty components: resolu-
tion, calibration, repeatability, linearity, bias,
MPE(gauge)
Measurement process capability of a multi-
ple-point measuring instrument
GRR with ANOVA
7 5.7 Variable 4 Multiple uncertainty components: resolution, VDA 5
calibration, repeatability, linearity, bias, MPE
(sensor), reproducibility, part-interaction,
temperature, error of temperature compen-
sation
4.2 Notes
4.2.1 Notes on the accuracy of the test examples and results
Capability indices are always given with two digits (rounded).
4.2.2 Note on outlier detection
Each test data set was tested for outliers using Grubbs’ test for outliers (according to ISO 5725-2) with a
level of significance of 1 % and no outliers were detected.
4.2.3 Note on capability indices
There are various different capability indices given in the relevant different standards and guidelines.
All are based on the ratio of the specification interval and the measurement uncertainty. Only the
expansion factors and limit values vary. In this standard only the capability indices according to
ISO 22514-7 are used.
4.2.4 Note on the model of the measurement and correlations
Although ISO/IEC Guide 98-3 provides the possibility of including non-linear models and correlations
between input quantities, correlations and non-linearities are not covered by the ISO 22514-7.
Therefore, only a linear model with sensitivity coefficients of one for every input quantity as well as no
correlations are considered in this standard and its examples.
4.2.5 Note on other reference data sets
[1]
ISO/TR 12888 provides multiple examples especially for the case of GRR studies .
4.2.6 Note on systematic errors
According to ISO/IEC Guide 98-3 any systematic error is compensated and the uncertainty of the
systematic error is included into the measurement budget and is part of the combined uncertainty.
5 Reference data sets description and evaluation
5.1 Test data set 1 – example of linearity study with at least three standards
5.1.1 Test data set 1 – information
Test data set for ISO 22514-7 capability of measurement processes with a linearity and ANOVA study.
This example has been taken from ISO 22514-7:2021, Annex A (the data originally come from ISO 11095).
The uncertainties arising from the object and the temperature were added.
5.1.2 Test data set 1 – data, calculations and results
5.1.2.1 Calculation of the measuring system capability
5.1.2.1.1 Components of type B which are not taken into account by experiments
Resolution
The uncertainty component caused by resolution is u =0,001 μ44 m .
RE
The uncertainty component u is much smaller than u , see behind Table 4. Therefore, the
RE EVR
component u is not used.
RE
Object
The maximum expected error due to the clamping of the part during the measurement is
a =0,001 5 μm .
OBJ
The uncertainty component is therefore:
a
OBJ
u = =0,000 μ866 m
OBJ
√3
Calibration
It is assumed according to the calibration certificate that the calibration uncertainty u is 0,005 μm.
CAL
5.1.2.1.2 Components of Type A which are derived from a linearity study with at least 3
standards
An experiment is carried out on an imaging system (an optical microscope with a measuring device).
The data listed in Table 2 are measured values and true values of intervals in the range of 0,5 μm to
12 μm.
Table 2 — Values from repeated measurements on reference materials
Values y from K = 4 repeatability measurements on N = 10 reference
nj
Conventional true values x of
n materials
the 10 reference materials
y y y y
n1 n2 n3 n4
6,19 6,31 6,27 6,31 6,28
9,17 9,27 9,21 9,34 9,23
Table 2 (continued)
Values y from K = 4 repeatability measurements on N = 10 reference
nj
Conventional true values x of
n
materials
the 10 reference materials
y y y y
n1 n2 n3 n4
1,99 2,21 2,19 2,22 2,20
7,77 8,00 7,81 7,95 7,84
4,00 4,27 4,15 4,15 4,15
10,77 10,93 10,73 10,92 10,89
4,78 4,95 4,87 5,00 5,00
2,99 3,24 3,17 3,21 3,21
6,98 7,14 7,07 7,18 7,20
9,98 10,23 10,02 10,07 10,17
Data in Table 2 are plotted in Figure 1.
Key
X reference value (µm)
Y measured value (µm)
Figure 1 — Plot of measured and true values
5.1.2.1.3 Calculation of means and residuals
For each reference material the mean value y , the bias B and the residuals e to e are calculated.
n i,n n1 n4
See Table 3 for the calculated values.
Table 3 — Calculation of means and residuals
Conventional true Residuals
Mean values
values x of the 10 B
i,n
n
y e e e e
reference materials n n1 n2 n3 n4
6,19 6,292 5 0,102 5 0,017 5 −0,022 5 0,017 5 −0,012 5
9,17 9,262 5 0,092 5 0,007 5 −0,052 5 0,077 5 −0,032 5
1,99 2,205 0 0,215 0 0,005 0 −0,015 0 0,015 0 −0,005 0
7,77 7,900 0 0,130 0 0,100 0 −0,090 0 0,050 0 −0,060 0
4,00 4,180 0 0,180 0 0,090 0 −0,030 0 −0,030 0 −0,030 0
10,77 10,867 5 0,097 5 0,062 5 −0,137 5 0,052 5 0,022 5
4,78 4,955 0 0,175 0 −0,005 0 −0,085 0 0,045 0 0,045 0
2,99 3,207 5 0,217 5 0,032 5 −0,037 5 0,002 5 0,002 5
6,98 7,147 5 0,167 5 −0,007 5 −0,077 5 0,032 5 0,052 5
9,98 10,122 5 0,142 5 0,107 5 −0,102 5 −0,052 5 0,047 5
Data in Table 3 are plotted in Figure 2.
Key
X value of reference part
Y bias
1 mean bias over all reference parts
2 uncertainty from linearity
individual error
mean bias of the reference part
Figure 2 — Plot of deviations and conventional true values
5.1.2.1.4 ANOVA table
Given values:
N = 10       Number of standards (Factor A)
K = 4        Number of repeatability measurements
Calculated values:
B =0,152  Arithmetic mean of all biases.
i
The components are calculated by an ANOVA, see Table 4.
Table 4 — ANOVA table
Sum of Degrees of Mean Estimated Test sta- Critical
Estimator
squares freedom squares variance tistic value
Source
σ
SS ν MS S F F
Factor A 0,077 39 9 0,008 599 0,001 121 2,089 6 2,210 7 0,033 480 9
Residual error 0,123 45 30 0,004 115 0,004 115 0,064 148 3
Total 0,200 84 39 --
5.1.2.1.5 Estimation of uncertainty components
Estimated uncertainties from Table 4 and mean bias:
Bi
uncertainty due to bias                          u ==0,087 76
BI
ˆ
uncertainty due to linearity                      u ==σ 0,033 48
LINA
uncertainty due to repeatability on references    u ==σˆ 0,064 15
EVRRES
5.1.2.1.6 Determination of the combined and expanded uncertainty
The uncertainty components of the measuring system are listed in Table 5 where the standard
uncertainty of the measuring system is calculated as the Euclidian distance of the following components:
22 22
uu=+ uu++ u
MS CALEVR LINBI
Because u << u the standard uncertainty of the resolution u is excluded from the calculation of
RE EVR RE
u .
MS
Table 5 — Uncertainty budget of the measuring system
u
Uncertainty component Symbol Type Remark Rank
μm
Resolution of the measuring system B (0,001 44) 5
u << u
RE EVR
Calibration uncertainty u B 0,005 00 4
CAL
Repeatability on reference standard A 0,064 15 2
u
EVR
Uncertainty from linearity u A 0,033 48 3
LIN
Uncertainty from Bias A 0,087 76 1
u
BI
Measuring system 0,113 85
u
MS
The combined uncertainty of the measuring system: u =0,114 μm
MS
and the expanded uncertainty: U =0,228 μm .
MS
5.1.2.2 Experimental determination of the measurement process uncertainty
In addition to the estimated uncertainty components from the measuring system found in Table 4, it
can be useful to determine some additional uncertainty components ( uu,, u ) from the
EVOAVIAi
measurement process by the evaluation of the results from this process under the real conditions. In
this example (estimation of uncertainty components from different operators, repeatability and
interaction between operators) the following data are collected, see Table 6.
Table 6 — ANOVA test data set in µm
Operator 1 Operator 2 Operator 3
Part
Measure- Measure- Measure- Measure- Measure- Measure- Measure- Measure- Measure-
no.
ment 1 ment 2 ment 3 ment 1 ment 2 ment 3 ment 1 ment 2 ment 3
1 8,120 8,435 8,480 8,200 8,290 8,245 8,525 8,435 8,345
2 7,445 6,815 7,490 7,300 7,120 7,075 7,535 7,355 7,085
3 9,965 10,010 9,560 9,660 9,340 9,250 9,830 9,695 9,515
4 6,140 5,960 6,365 6,095 6,185 6,185 6,140 6,140 6,050
5 5,690 5,600 5,780 5,080 5,340 5,440 5,780 5,735 5,555
6 2,855 2,450 2,585 2,315 2,585 2,315 2,630 2,360 2,585
7 10,685 10,595 10,775 10,450 10,840 11,050 10,865 11,000 11,180
8 6,725 6,275 6,545 6,240 6,120 6,300 6,590 6,500 6,725
9 4,970 5,105 5,510 5,015 5,285 5,150 5,060 5,195 5,105
10 9,875 10,100 9,875 10,080 9,800 9,970 10,190 9,785 9,965
From the measurements in Table 6 the following analysis of variance table can be calculated, see Table 7.
Table 7 — ANOVA table
Degrees of Sum of Mean Estimated Test sta- Critical
Uncertainty
freedom squares Square variance tistic value
Uncertainty
F
component


ν SS MS F
σ ²
u =+ σ ²
i
i i
α = 5 %
Operator 2 0,519 1 0,259 5 0,007 38 0,085 91 6,810 3,150
Part to part 9 526,877 5 58,541 9 6,500 43 n/a 1 536,234 2,040
Interaction be-
tween operator 18 0,685 9 0,038 1 0,002 05 0,045 29 1,193 1,778
and part
Reproducibility 60 1,917 3 0,032 0 0,031 95 0,178 76 --- ---
Since the interaction between operator and part is not significant (F < F ) pooling is used. Then a
modified variance table can be developed there, see Table 8
Table 8 — Modified ANOVA table
Degrees of Sum of Mean Estimated Test sta- Critical
Uncertainty
freedom squares Square variance tistic value
Uncertainty
component F


ν SS MS F
σ ²
u =+ σ ²
i
i i
α = 5 %
Operator 2 0,519 06 0,259 53 0,007 54 0,086 82 7,776 3,114
Part to part 9 526,877 50 58,541 94 6,500 95 n/a 1 754,088 2,002
Reproducibility 78 2,603 22 0,033 37 0,033 37 0,182 69 --- ---
The uncertainty components of the measurement process are then found:
u =0,086 82μm
AV
u =0,182 69 μm
EVO
Temperature
The length of the object is l =10 μm .
The maximum temperature difference is δ =01, K .
T
The average temperature during the measurement is T =°21 C .
−−61
The expansion coefficient is α =⋅11,510 K .
−−71
The uncertainty of α is uK=⋅11,510 .
α
δα ⋅⋅ l
T
The uncertainty from temperature differences u is u = =0,000 00664 μm .
TD TD
Tu−°20 C ⋅⋅  l
α
The uncertainty on expansion coefficients is u = =0,000 00664 μm .
TA
The influence from temperature is therefore uu=+²² u =0,000009 39 μm .
TTDTA
Table 9 — Uncertainty budget of the measurement process
u
Uncertainty component Symbol Type Remark Rank
μm
Resolution of the measuring system u B (0,001 4) << u 7
RE EVO
Calibration uncertainty B 0,005 0 6
u
CAL
Repeatability on reference standard u A (0,064 1) << u 4
EVR EVO
Uncertainty from linearity A 0,033 5 5
u
LIN
Uncertainty from Bias u A 0,087 8 2
BI
Reproducibility of operators A 0,086 8 3
u
AV
Repeatability on test parts u A 0,182 7 1
EVO
Uncertainty from interactions A pooling --
u
IAI
Inhomogeneity of measurand u B 0,000 9 8
OBJ
Temperature B 0,000 0 9
u
T
Measurement process u 0,223 1
MP
The components u and u are both set to 0.
STAB REST
The uncertainty components of the measurement process are given in Table 9 where the standard
uncertainty of the measurement process is calculated as the Euclidian distance of the following
components:
22 22 22 2
uu=+uu++uu++uu+
MP CALLIN BI AV EVOOBJ T
The combined uncertainty of the measurement process: u =0,223 μm
MP
and the expanded uncertainty: U =0,446 μm .
MP
5.1.2.2.1 Assessing the capability of measuring system and the measurement process
With the specification interval: UL−= 11−29 μm = μm .
()
The capability ratios are:
2⋅U
20⋅ ,227 70μm
MS
%%Q = ⋅=100 ⋅=100 %%51,
MS
UL− 11−2 μm
()
2⋅U 20⋅ ,446 14μm
MP
%%Q = ⋅=100 ⋅=100 %%99,
MP
UL− ()11−2 μm
The capability indices are:
02,,⋅−UL 02⋅−11 2 μm
() ()
C = = =39, 5
MS
2⋅⋅ku 220⋅⋅ ,113 85μm
MS
04,,⋅−()UL 04⋅−()11 2 μm
C = = =40, 3
MP
2⋅⋅ku 220⋅⋅ ,223 07μm
MP
The calculated statistics are listed in Table 10.
Table 10 — Uncertainty and capability for test data set 1
Measuring system Measurement process
UL− UL−
Specification interval 9,00 µm 9,00 µm
Combined standard uncertainty u 0,114 μm u 0,228 µm
MS MP
Expanded measurement uncertainty U 0,223 µm U 0,446 µm
MS MP
Capability ratio %Q 5,1 % %Q 9,9 %
MS MP
Capability index C 3,95 C 4,03
MS MP
5.2 Test data set 2 – attribute measurement process – operator influence (ISO 22514-7)
5.2.1 Test data set 2 – information
Test data set for attributive measurement processes without reference values. Only the influence of
different operators is determined. The example was taken from ISO 22514-7:2021, 12.2.
5.2.2 Test data set 2 – data, calculations and results
Typically at least 40 different test parts are tested 3 times by 2 different operators, called A and B.
Each of the 120 different measurement results on the 40 parts, which the operator A or operator B has
achieved, is assigned to one of the following three classes.
— Class 1: all three test results on the same part gave the result “good”;
— Class 2: the three test results on the same part gave different results;
— Class 3: all three test results on the same part gave the result “bad”.
The test results are given in Table 11.
Table 11 — Test result from an attribute measurement process
Frequency Operator B
Class 1 Class 2 Class 3
n
ij
Result ‘+++’ Different results Result ‘- - -’
Class 1
7 3 1
Result ‘+++’
Class 2
Operator A 10 4 7
Different results
Class 3
2 1 5
Result ‘- - -’
The two operators in Table 11 are tested using a Bowker-Test of symmetry. If there are no significant
differences between operators, the resulting frequencies in Table 10 are sufficiently symmetrical with
respect to main diagonal. The hypothesis H : mm= (i, j = 1, …, 3 with i ≠ j) says that the frequencies
0 ij
m and m which lies symmetrical with respect to the main diagonal are identical.
ji
Test statistic
nn− ²
()
()10−3 ²²()21− ()17− ²
ij ji
χ = = + + =88,603
∑
nn+ 10+3 21+ 17+
ij ji
ij> u
is compared to 1−α fractile in the χ distribution with 3 degrees of freedom. The null hypothesis
()
test states that changes from one category to another are random in nature. The hypothesis on
symmetry is rejected on the level if the test value is greater than the 1−α fractile in the χ
()
distribution with 3 degrees of freedom. In this case, the hypothesis is rejected because the calculated
value 8,603 is greater than the value 7,815 which is the 95 % fractile of the χ ()3 distribution.
5.3 Test data set 3 – attributive measurements – capability calculations using reference
values – calculation of the uncertainty range (ISO 22514-7)
5.3.1 Test data set 3 – information
Test data set for attributive measurement processes. Capability calculations using reference values
including the calculation of the uncertainty range.
5.3.2 Test data set 3 – data, calculations and results
This method is based on signal detections and therefore requires workpieces with known reference
values. When about 25 % of the workpieces is at or close to the lower specification limit and 25 % of
the workpieces is at or close to the upper specification limit, the area of risk around the specification
limits can be addressed. The purpose of this method is to determine the uncertainty range, in which an
operator is unable to make an unambiguous decision. Figure 2 illustrates the test results of an attribute
measurement process obtained from a set of reference values.
Figure 3 — Test data set
Symbols
In Figure 3, the reference measurement values are introduced in the form of a code. A green plus sign
means that the operator has indicated the result from the test piece as approved. A grey minus sign
means that the operator has indicated the result from the test piece as not approved. A green smiley
means that all three operators have indicated the result from the test piece as approved or rejected in
all three tests, and that this assessment is consistent with the reference value. A red smiley indicates
a case where at least one of the operators has come to a test result, which is not consistent with the
reference value.
Steps for the calculation of the capability index:
Step 1:
Sort the table according to the measured reference size. In Figure 2, a sorting in descending order is
made – from the highest reference value descending to the lowest reference value.
Step 2:
Select the last reference value for which all operators have assessed all the results as being
unsatisfactory (not approved). This is the transition from symbol “–” to symbol “X”.
0,566 152 mm –
0,561 457 mm X
Step 3:
Select the first reference value for which all operators the first time assessed all results being approved.
This is the transition from symbol “X” to the symbol “+”.
0,543 077 mm X
0,542 704 mm +
Step 4:
Select the last reference value for which all operators last time assessed all the results as being
approved. This is the transition from the “+” symbol to the symbol “X”.
0,470 832 mm +
0,465 454 mm X
Step 5:
Select the first reference value for which every operator has again first assessed all the results as
unsatisfactory (not approved). This is the transition from symbol “X” to the symbol “–”.
0,449 696 mm X
0,446 697 mm –
Step 6:
Calculate the d interval from the last reference value, for which all operators have assessed the
UR
result as unsatisfied (not approved) to the first reference value, for which all operators have the result
as approved.
d =−0,,566 152 0 542704 mm =0,023448 mm
()
UR
Step 7:
Calculate the d interval from the last reference value, for which all operators have assessed the result
LR
as approved to the first reference value, and for which all operators have the result as unsatisfied (not
approved).
d =−()0,,470 832 0 446697 mm =0,024135 mm
LR
Step 8:
Calculate the average “d” of the two intervals:
dd+ 0,,023 448 + 0 024135 mm
()
UR LR
d = = =0,023791 5mm
2 2
Step 9:
Calculate the uncertainty range and capability ratio/ index:
d
U == 0,011 90mm =ˆ 11,90 μm
attr
U
attr
u == 0,,005 95mm =595 μm
ˆ
attr
2⋅U
0,023 7915 mm
attr
%%Q = ⋅=100 ⋅=100 %,23 79%
attr
UL− 01, mm
where UL−=01, mm
02, ⋅−()UL 02,,⋅01 mm
C = = =16, 8
attr
2⋅⋅ku 220⋅⋅ ,005 95mm
attr
Table 12 shows the uncertainty and capability statistics.
Table 12 — Uncertainty and capability for test data set 3
Specification interval UL− 100,00 µm
Measurement uncertainty 5,95 µm
u
attr
Expanded measurement uncertainty U 11,90 µm
attr
Capability ratio 23,79 %
%Q
attr
Capability index C 1,68
attr
5.4 Test data set 4 – measurement process capability with three reference standards
(VDA 5)
5.4.1 Test data set 4 – information
Test data set for the Measurement Process Capability using 3 reference standards. The example was
nd
taken from the VDA 5, 2 edition and the data are listed in Table 13.
5.4.2 Test data set 4 – data, calculations and results
An instrument measuring boltholes requires that the capability of the measurement process for inside
diameters is evaluated and documented. Uncertainties from test part or the temperature are regarded
as negligible and are not considered in the evaluation.
Table 13 — Information about measuring system and measurement process
Information about measuring system and measurement process
Nominal dimension 30,000 mm
Upper specification limit U 30,008 mm
Lower specification limit L 30,003 mm
Resolution of the measuring system R (1 digit = 0,000 1 mm) 0,1 μm
E
Calibration uncertainty U 0,026 μm
CAL
Coverage factor k 2
CAL
Linearity 0
Reference quantity value of the standard at the upper specification limit x 30,007 6 mm
mu
Reference quantity value of the standard in the centre of the specification x 30,005 0 mm
mm
Table 13 (continued)
Information about measuring system and measurement process
Reference quantity value of the standard at the lower specification limit x 30,002 5 mm
ml
Capability ratio limit measuring system Q 15 %
MS_max
Capability ratio limit measurement process Q 30 %
MP_max
5.4.2.1 Test data set 4 – evaluating the capability of measuring system
In order to determine the standard uncertainties from repeatability on standards and from
measurement bias, an experiment was conducted performing 10 repeated measurements on each of
three reference standards. The reference values and measuring results are listed in Table 14.
Table 14 — Reference value and measurement values
Dimensions in millimetres
Standard 1 Standard 2 Standard 3
Reference value 30,007 6 30,005 0 30,002 5
Measurement value 1 30,007 5 30,005 0 30,002 5
Measurement value 2 30,007 5 30,005 1 30,002 4
Measurement value 3 30,007 7 30,005 1 30,002 4
Measurement value 4 30,007 5 30,005 0 30,002 3
Measurement value 5 30,007 6 30,005 2 30,002 5
Measurement value 6 30,007 6 30,005 1 30,002 4
Measurement value 7 30,007 6 30,005 0 30,002 3
Measurement value 8 30,007 5 30,005 1 30,002 3
Measurement value 9 30,007 6 30,005 1 30,002 4
Measurement value 10 30,007 6 30,005 2 30,002 4
The information about the measuring system and the measured quantity values gained in the
experiment leads to the following uncertainty budget and overview of results, see Table 15.
Table 15 — Uncertainty budget of the measuring system
u
Uncertainty component Symbol Type Remark Rank
μm
Resolution of the measuring system u B (0,03) << u 3
RE EVR
Calibration uncertainty u B 0,01
CAL
Repeatability on reference standard u A 0,07 1
EVR
--
Uncertainty from linearity u B
LIN
Uncertainty from Bias u A 0,06 2
BI
Measuring system u 0,10
MS
where
22 2
uu=+ uu+
MS CALEVR BI
The calculated uncertainty and capability statistics are given in Table 16.
Table 16 — Results of the measuring system
Specification interval UL− 5,00 µm
Combined standard uncertainty 0,10 µm
u
MS
Expanded measurement uncertainty 0,20 µm
U
MS
Capability ratio 7,86 %
%Q
MS
Capability index 2,55
C
MS
Due to a percentage resolution %R of 2,00 % and a capability ratio %Q of 7,86 %, the capability of
E MS
the measuring system of the instrument measuring boltholes is confirmed. After the capability of the
measuring system is confirmed, the measurement process is analysed.
5.4.2.2 Test data set 4 – analysing the measurement process
The operator influence, the repeatability on test parts and their interactions are determined
experimentally under operational conditions. In this experiment, 2 repeated measurements are
performed on each of 10 test parts by 3 operators. For the measured values, see Table 17.
Table 17 — Measurement values
Dimensions in millimetres
Operator A Operator B Operator C

Trial 1 Trial 2 Trial 1 Trial 2 Trial 1 Trial 2
1 30,005 4 30,005 5 30,005 7 30,005 8 30,005 8 30,005 7
2 30,005 6 30,005 8 30,005 9 30,005 4 30,005 7 30,005 8
3 30,005 3 30,005 4 30,005 5 30,005 5 30,005 6 30,005 9
4 30,004 1 30,004 2 30,004 3 30,004 4 30,004 5 30,004 2
5 30,005 1 30,005 3 30,005 5 30,004 9 30,005 2 30,004 9
6 30,005 0 30,005 2 30,005 4 30,005 5 30,005 5 30,005 3
7 30,004 9 30,005 0 30,004 9 30,005 2 30,005 1 30,005 1
8 30,005 6 30,005 6 30,005 7 30,005 9 30,005 8 30,005 7
9 30,005 4 30,005 5 30,005 6 30,005 7 30,005 4 30,005 6
10 30,005 8 30,005 8 30,005 9 30,006 1 30,005 7 30,006 1
Based on the recorded measured quantity values, the individual standard uncertainties can be
determined and allocated by using the method of ANOVA, see Table 18.
Table 18 — ANOVA table (without the non-significant interaction term)
Degrees
Sum of Estimated Test sta- Critical
of free- Mean Square Uncertainty
squares variance tistic value
Uncertainty
dom
component
F


ν SS MS F
σ ²
u =+ σ ²
i
i i
α = 5 %
−7 −7 −8 −5
Operator 2 7,519 3,191
3,423
...