ISO/TR 29901:2007

TECHNICAL ISO/TR
REPORT 29901
First edition
2007-11-01
Selected illustrations of full factorial
experiments with four factors
Illustrations choisies de plans d'expérience factoriels complets à quatre
facteurs
Reference number
©
ISO 2007
PDF disclaimer
This PDF file may contain embedded typefaces. In accordance with Adobe's licensing policy, this file may be printed or viewed but
shall not be edited unless the typefaces which are embedded are licensed to and installed on the computer performing the editing. In
downloading this file, parties accept therein the responsibility of not infringing Adobe's licensing policy. The ISO Central Secretariat
accepts no liability in this area.
Adobe is a trademark of Adobe Systems Incorporated.
Details of the software products used to create this PDF file can be found in the General Info relative to the file; the PDF-creation
parameters were optimized for printing. Every care has been taken to ensure that the file is suitable for use by ISO member bodies. In
the unlikely event that a problem relating to it is found, please inform the Central Secretariat at the address given below.

©  ISO 2007
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized in any form or by any means,
electronic or mechanical, including photocopying and microfilm, without permission in writing from either ISO at the address below or
ISO's member body in the country of the requester.
ISO copyright office
Case postale 56 • CH-1211 Geneva 20
Tel. + 41 22 749 01 11
Fax + 41 22 749 09 47
E-mail copyright@iso.org
Web www.iso.org
Published in Switzerland
ii © ISO 2007 – All rights reserved

Contents Page
Foreword. iv
Introduction . v
1 Scope . 1
2 Normative references . 1
3 Terms and definitions. 2
4 Symbols and abbreviated terms . 4
5 Generic description of full factorial designs . 4
5.1 Overview of the structure of the four factor examples in Annexes A through E. 4
5.2 Overall objective(s) of the experiment. 4
5.3 Response variable(s). 5
5.4 Factors affecting the response(s). 5
5.5 “Full” factorial design . 5
5.6 Analyse the results — Numerical summaries and graphical displays. 6
5.7 Present the results. 7
5.8 Perform confirmation runs . 7
6 Description of Annexes A through E. 7
6.1 Comparing and contrasting the examples. 7
6.2 Experiment summaries . 7
Annex A (informative) Solder bar experiment . 8
Annex B (informative) Direct mail marketing campaign. 17
Annex C (informative) Button tactility experiment. 25
Annex D (informative) Optimizing a customer PVC formulation. 34
Annex E (informative) Genetic algorithms for DNA sequencing experiment. 44
Bibliography . 52

Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards bodies
(ISO member bodies). The work of preparing International Standards is normally carried out through ISO
technical committees. Each member body interested in a subject for which a technical committee has been
established has the right to be represented on that committee. International organizations, governmental and
non-governmental, in liaison with ISO, also take part in the work. ISO collaborates closely with the
International Electrotechnical Commission (IEC) on all matters of electrotechnical standardization.
International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2.
The main task of technical committees is to prepare International Standards. Draft International Standards
adopted by the technical committees are circulated to the member bodies for voting. Publication as an
International Standard requires approval by at least 75 % of the member bodies casting a vote.
In exceptional circumstances, when a technical committee has collected data of a different kind from that
which is normally published as an International Standard (“state of the art”, for example), it may decide by a
simple majority vote of its participating members to publish a Technical Report. A Technical Report is entirely
informative in nature and does not have to be reviewed until the data it provides are considered to be no
longer valid or useful.
Attention is drawn to the possibility that some of the elements of this document may be the subject of patent
rights. ISO shall not be held responsible for identifying any or all such patent rights.
ISO/TR 29901 was prepared by Technical Committee ISO/TC 69, Applications of statistical methods.

iv © ISO 2007 – All rights reserved

Introduction
The Six Sigma and international statistical standards communities share a philosophy of continuous
improvement and many analytical tools. The Six Sigma community tends to adopt a pragmatic approach
driven by time and resource constraints. The statistical standards community arrives at rigorous documents
through long-term international consensus. The disparities in time pressures, mathematical rigor and statistical
software usage have inhibited exchanges, synergy and mutual appreciation between the two groups.
The present document takes one specific statistical tool (full factorial designs with four factors, 2 designs) and
develops the topic somewhat generically (in the spirit of International Standards) but then illustrates it through
the use of five detailed and distinct applications. The generic description focuses on the commonalities across
2 designs. These commonalities hold more generally for arbitrary numbers of factors, but a value of four was
chosen for this Technical Report. The annexes containing the five illustrations follow the basic framework but
also identify the nuances and peculiarities in the specific applications. Each example offers at least one
“wrinkle” to the problem, which is generally the case for real Six Sigma applications. It is thus hoped that
practitioners can identify with at least one of the five examples, if only to remind them of the basic material on
factorial designs that was encountered during their Six Sigma training. Each of the five examples is developed
and analysed using statistical software of current vintage. The explanations throughout are devoid of
mathematical detail — such material can be readily obtained from the many design and analysis of
experiments textbooks available (such as those given in the Bibliography).

TECHNICAL REPORT ISO/TR 29901:2007(E)

Selected illustrations of full factorial experiments
with four factors
1 Scope
This Technical Report describes the steps necessary to specify, to use and to analyse 2 full factorial designs
through illustration, with five distinct applications of this methodology.
Depending on the application, a number of factors other than four may be considered in the experiment.
NOTE 1 Each of these five illustrations is similar in that sufficient resources were available to implement the design.
Other commonalities among the five examples are noted (e.g. study objective, two levels for factors, response variable(s),
factors effecting the response). The individual illustrations have some salient features that are distinct such as
presence/absence of repetitions, centre points, interactions, or different types of response variables. Each illustration
takes place in a different environment such as marketing, software, manufacturing, telecommunications and chemical
processing.
NOTE 2 For the purposes of this Technical Report, the selection of four factors with two levels (aside from centre
points) was made in advance. Furthermore, the detailed use of response surface designs as a follow-up or augmentation
of the existing designs was excluded from this Technical Report, although their use is noted in some of the illustrations.
Likewise, Taguchi designs and blocking designs were not included.
NOTE 3 Full factorial experiments are often employed by individuals (so-called “black belts” or “green belts”)
associated with Six Sigma methods. Six Sigma methods are concerned with problem solving and continuous improvement.
A full factorial experiment with four factors is one of many tools available to Six Sigma practitioners, but hitherto has not
been addressed in detail in ISO International Standards.
2 Normative references
The following referenced documents are indispensable for the application of this document. For dated
references, only the edition cited applies. For undated references, the latest edition of the referenced
document (including any amendments) applies.
ISO 3534-1:2006, Statistics — Vocabulary and symbols — Part 1: General statistical terms and terms used in
probability
ISO 3534-2, Statistics — Vocabulary and symbols — Part 2: Applied statistics
ISO 3534-3:1999, Statistics —Vocabulary and symbols — Part 3: Design of experiments
3 Terms and definitions
For the purposes of this document, the terms and definitions in ISO 3534-1, ISO 3534-2, ISO 3534-3 and the
following apply.
3.1
analysis of variance
ANOVA
technique which subdivides the total variation of a response variable into meaningful components associated
with specific sources of variation
NOTE Adapted from ISO 3534-3:1999, definition 3.4.
3.2
binomial distribution
discrete distribution having the probability mass function
n!
x nx−
PX()==x p(1−p)
xn!( −x)!
where x = 0, 1, …, n and with indexing parameters n = 1, 2, …, and 0 < p <1.
NOTE Adapted from ISO 3534-1:2006, definition 2.46.
3.3
block
collection of experimental units more homogeneous than the full set of experimental units
NOTE Adapted from ISO 3534-3:1999, definition 1.11.
3.4
centre point
vector of factor level settings of the form (a , a , ., a ), where all a equal 0, as notation for the coded levels of
1 2 k i
the factors
NOTE Adapted from ISO 3534-3:1999, definition 1.36.
3.5
design matrix
matrix with rows representing individual treatments (possibly transformed according to the assumed model)
which can be extended by deduced levels of other functions of factor levels (interactions, quadratic terms,
etc.) but are dependent upon the assumed model
NOTE Adapted from ISO 3534-3:1999, definition 2.7.1.
3.6
factor
predictor variable that is varied with the intent of assessing its effect on the response variable
NOTE Adapted from ISO 3534-3:1999, definition 1.5.
3.7
full factorial experiment
experiment consisting of all possible treatments formed from two or more factors, each being studied at two or
more levels
NOTE Adapted from ISO 3534-3:1999, definition 2.1.
2 © ISO 2007 – All rights reserved

3.8
interaction
effect for which the apparent influence of one factor on the response variable depends upon one or more
other factors
NOTE Adapted from ISO 3534-3:1999, definition 1.17.
3.9
level
potential setting, value or assignment of a factor
NOTE Adapted from ISO 3534-3:1999, definition 1.6.
3.10
normal distribution
continuous distribution having the probability density function
()x−µ
−
1 2
2σ
fx()= e
σ 2π
where −∞ < x < ∞ and with parameters −∞ < µ < ∞ and σ > 0
NOTE Adapted from ISO 3534-1:2006, definition 2.50.
3.11
predictor variable
variable that can contribute to the explanation of the outcome of an experiment
NOTE Adapted from ISO 3534-3:1999, definition 1.3.
3.12
randomization
process used to assign treatments to experimental units so that each experimental unit has an equal chance
of being assigned a particular treatment
NOTE Adapted from ISO 3534-3:1999, definition 1.29.
3.13
replication
performance of an experiment more than once for a given set of predictor variables
NOTE Adapted from ISO 3534-3:1999, definition 1.27.
3.14
split-plot design
design in which a group of experimental units (plot) to which the same level assigned to the principal factor is
subdivided (split) so as to study one or more additional principal factors within each level of that factor
NOTE Adapted from ISO 3534-3:1999, definition 2.3.6.
4 Symbols and abbreviated terms
The symbols and abbreviated terms used in this Technical Report are as follows:
y Response variable
A, B, C, D Factors
AB, AC, AD, BC, BD, CD 2-way interactions
ABC, ACD, BCD 3-way interactions
ABCD 4-way interactions
+1/−1 High and low settings
2 Four factors each with two levels
σ Standard deviation
5 Generic description of full factorial designs
5.1 Overview of the structure of the four factor examples in Annexes A through E
This Technical Report provides general guidelines on the design, conduct and analyses of two-level full
factorial designs and illustrates the steps with five distinct applications given in Annexes A through E. Each of
these five examples follows the basic structure given in Table 1.
The steps given in Table 1 apply to design and analysis of experiments in general, although this Technical
Report focuses on 2 full factorial designs. Each of the seven steps is explained in general below. Specific
explanations of the substance of these steps is provided in the examples in Annexes A through E.
Table 1 — Basic steps in experimental design
1 State the overall objective(s) of the experiment
2 Describe the response variable(s)
3 List the factors that might affect the response(s)
4 Select a “full” factorial design
5 Analyse the results – Numerical summaries and graphical displays
6 Present the results
7 Perform a confirmation run
5.2 Overall objective(s) of the experiment
Experiments are conducted for a variety of reasons. The primary motivation for the experiment should be
clearly stated and agreed to by all parties involved in the design, conduct, analysis and implications of the
experimental effort. There may be secondary objectives which could be addressed with the full factorial
experiment.
The ultimate outcome of the experiment could be to take immediate action on factor levels or to obtain a
predictive model, both of which dictate some elements of the analyses.
4 © ISO 2007 – All rights reserved

5.3 Response variable(s)
Associated with the objective of an experiment is a measurable outcome or performance measure. A
response of interest could involve maximization (larger is better), minimization (smaller is better) or meet a
target value (be close to a specified value). The response variable (denoted here by the variable y) should be
intimately (if not directly) related to the objective of the experiment. For some situations, there may be multiple
characteristics of interest to be considered, although there typically is a primary response variable associated
with the experiment. In other cases, multiple responses must be considered; however, for purposes of this
document, a single response is considered in each example.
5.4 Factors affecting the response(s)
The response variable likely depends in some unknown way on a variety of conditions that occur or could be
set in the course of generating a response variable outcome. These conditions are presumed to relate to
controllable factors that may be continuous (temperature, concentration) or discrete (two assembly lines A
or B, two vendors, two packaging styles, and so forth). For 2 experiments, we simplify the experimental
design process by selecting two levels for each factor to be varied in the experiment. For discrete factors with
only two possible settings, the levels of this factor are just these two settings. For continuous factors, there is
discretion in choosing the two specific values. In some cases, the two settings could be the historical value
and a proposed value. In other cases, the two settings could be a nominal adjustment from the historical
setting. In any event, the settings should be sufficiently far removed to have an opportunity to reveal an impact
subject to the inherent uncertainty, while not being so disparate that the settings are unreasonable from a
practical, safety or sensibility standpoint. The setting of levels of continuous factors benefits from the domain
expert collaborating on the experiment.
There may be additional factors that could impact that response variable but may be deemed less important
than the chosen four factors or are too difficult or expensive to control. Finally, it is the case that the factors
are to be set independently from each other. It could, however, be discovered that the factors interact (the
setting of one factor impacts how a second factor affects the response).
5.5 “Full” factorial design
This classical design consists of 16 runs obtained by considering all combinations of four factors with two
possible levels. Table 2 provides the basic layout in a standard order for ease of understanding. Each row of
the table represents one set of experimental conditions that when run will produce a value of the response
variable y. The four factors are designated as A, B, C and D. For an individual factor, the level “−1” is the “low”
setting, or one of the two levels if the factor is categorical. The level “+1” is the “high” setting, or the other level
of the categorical factor. The column “y” is a placeholder for the response value once a run has occurred.
In addition to providing the explicit 16 experimental runs to be conducted, Table 2 provides an example of a
randomized run order. The experiment should not be run in the standard order (for example, an increasing
trend in the response could be confused with the effect of factor D on the response). The final two columns
provide abbreviated names for the 16 runs. The first name coincides with the “+” or “−” levels of the factors
given in the order for ABCD. The second naming convention includes a lower case letter for each factor at its
high level. By convention, the four settings at the low level are designated as “(1)” since otherwise the
abbreviation would be null.
Table 2 — Layout of a generic 2 full factorial design
Row Name Alternate
A B C D y Run order
number of run name of run
1 −1 −1 −1 −1 y 6 − − − − (1)
2 +1 −1 −1 −1 y 14 + − − − a
3 −1 +1 −1 −1 y 4 − + − − b
4 +1 +1 −1 −1 y 11 + + − − ab
5 −1 −1 +1 −1 y 9 − − + − c
6 +1 −1 +1 −1 y 2 + − + − ac
7 −1 +1 +1 −1 y 3 − + + − bc
8 +1 +1 +1 −1 y 1 + + + − abc
9 −1 −1 −1 +1 y 8 − − − + d
10 +1 −1 −1 +1 y 13 + − − + ad
11 −1 +1 −1 +1 y 7 − + − + bd
12 +1 +1 −1 +1 y 10 + + − + abd
13 −1 −1 +1 +1 y 15 − − + + cd
14 +1 −1 +1 +1 y 16 + − + + acd
15 −1 +1 +1 +1 y 5 − + + + bcd
16 +1 +1 +1 +1 y 12 + + + + abcd
In the situation with all four factors being continuous (quantitative), some experimenters elect to run additional
“centre” points. The levels of the centre point are at the midpoint of the two levels of each factor. The inclusion
of centre points facilitates statistical testing of effects, using the variability of the responses at the centre points
as a guideline. It also provides the means to test for non-linearity (curvature) of the response.
In the situation where a particular factor is difficult or expensive to change from one level to the other, the
experiment could be conducted in “blocks”, one block for each level of the difficult factor.
One final situation concerns replication of the runs in the design. Multiple replications at each experimental
setting accommodate estimates of the overall inherent variability which can then be pooled for statistical
testing purposes.
5.6 Analyse the results — Numerical summaries and graphical displays
At the completion of the conduct of the experiment, the y values would be replaced by the actual observed
i
responses. Many existing statistical software packages exist to facilitate the generation of output to aid in the
understanding of the results of the experiment. Of immediate concern is the determination of the impact of the
four factors individually on the response variable. Thus, the main effects for A, B, C and D are to be estimated
as well as the following interaction terms:
⎯ two-way: AB, AC, AD, BC, BD, CD;
⎯ three-way: ABC, ABD, ACD, BCD;
⎯ four-way: ABCD.
6 © ISO 2007 – All rights reserved

In addition to the actual estimates, it is useful to arrange the estimated effects from largest to smallest in the
form of a Pareto Chart. The effects can also be presented on a normal or half-normal plot to identify the
stronger effects. Depending on the inclusion of centre points or replication, the experimental error can be
estimated directly and in turn the standard error of the effects can be determined. In the absence of these
experimental runs, the experimental error can be estimated indirectly by presuming that three-way and four-
way interactions are negligible. Effects plots, interaction plots and residual plots are also generally considered
in assessing the results of an experiment. These are illustrated in Annexes A through E.
5.7 Present the results
Frequently, the purpose of the experiment is to develop a predictive model so as to explore alternate settings
of the factors. Contour plots are available from most software packages. Moreover, with a predictive model,
tentative consideration of optimal settings can be identified. The predictive model will consist of a function
(typically linear) of the estimated effects and possibly interactions. Alternate mathematical models may be
envisaged depending on examination of the residuals (collection of predictive values minus observed values).
5.8 Perform confirmation runs
Follow-up experiments may prove useful to demonstrate that the lessons learned from the 2 full factorial
experiments are verified in subsequent runs. A natural follow-on experiment could be to identify a promising
direction in which to adjust the factors for improved performance in the response variable.
6 Description of Annexes A through E
6.1 Comparing and contrasting the examples
Five distinct examples of 2 full factorial designs are illustrated in Annexes A to E. Each of these examples
follows the same general template as given in Table 1 and follows a version of the standard design given in
Table 2.
6.2 Experiment summaries
Table 3 summarizes the five examples detailed in the annexes and indicates aspects of the analyses which
were unique to each experiment.
Table 3 — Experiment summaries found by annex
Annex Experiment Problem-specific aspects
Mixture of discrete and continuous factors; repetitions;
A Solder bars
important 2-way interaction
Proportion response variable, standard errors based on
B Direct mail campaign
binomial distribution
C Button tactility Centre points; curvature in response
D PVC formulation optimization Split-plot; centre points; contour plots
Randomized order not necessary; set up for interim analyses;
E Genetic algorithms
replication; dispersion effect

Annex A
(informative)
1)
Solder bar experiment
A.1 General
The material solder can be produced in a number of forms. One of the most common is a bar of solid solder.
The bars are sold by mass, namely 0,5 kg, 1,0 kg and 2,0 kg.
A.2 Overall objective for the experiment
The process had recently been relocated to a new site and subsequently, on the underside of the bars there
were a large number of “rosettes” (small pits on the surface of the solder bar). This led to numerous customer
complaints, customers assuming that the solder was of an inferior standard. The current level of rosettes on
bars was assessed to be about 30 per bar.
The operational staff had tried over several months various “one-factor-at-a-time” experiments, none of which
produced a large enough improvement for the producer to say to their customers that they had solved the
problem. It was decided to design and then run an experiment to understand the factors responsible for the
defect and then how to manage the process so as to minimize the number of rosettes.
A.3 Description of the process
Solder bars are produced by pouring molten solder into moulds pre-selected according to the target mass of
the bars to be produced.
Ingots of solder are firstly melted in an electric furnace at about 290 °C. An operator then produces the bars
by filling a “kettle” with the molten solder and then by pouring the solder from the kettle into moulds. A kettle is
similar to the kitchen utensil of the same name but with extra spouts, thus enabling several bars to be poured
simultaneously.
The bars are cast upside down so the bottom of each bar is uppermost and open to the atmosphere. The
moulds are arranged in banks on a table with water-cooling integrated into it that can be turned on or off.
Before casting, the moulds can be “smoked”. This is a layer of carbon applied with an oxyacetylene torch.
After a short period of time during which the bars solidify, they are de-moulded, stacked and then packed.
A.4 Response variable
A.4.1 Choice of variable
The response chosen for the experiment was the average number of rosettes counted on the surface of the
bars. The objective was to minimize the average number of rosettes.

1) This example has been kindly donated by Cookson Electronics (Fry's Metals Glasgow).
8 © ISO 2007 – All rights reserved

A.4.2 Measurement of the response variable
Approximately 160 bars were produced in each experimental “run”. The number of rosettes was counted
across all of the solder bars and the arithmetic mean of the number of rosettes per bar calculated for each
experimental run. This became the output for that experimental run.
A.4.3 Relationship of the response variable to the objective of the experiment
The mean number of rosettes per bar is directly linked to the objective of the experiment.
A.5 Factors affecting the response
A.5.1 Description of each factor (continuous/discrete) to be varied
The factors chosen to be varied within the experiment were determined using the knowledge of the production
operators and the technical staff. These were:
A: the casting temperature of the molten solder;
B: water-cooling, applied or not;
C: the pouring rate of the molten solder into the moulds; and
D: whether the moulds were smoked or not.
The experiment was to be run using these four factors each at two levels.
Factor A was clearly a factor that could be adjusted to any given temperature. Factor C was, in theory, one
that could be varied as a continuous variable. However, in practice, it was handled as a categorical factor and
the rate was either “Normal” or “Maximum”.
The other two factors were categorical. Water cooling, factor B, was either “Off” or “On” and the moulds,
factor D, were either not smoked “No” or smoked “Yes”.
A.5.2 Selection of levels (related to size of effect to be determined)
The factors used in the experiment and their associated levels are given in Table A.1.
Table A.1 — Factors and their associated levels
Factor Level 1 Level 2
A: Casting temperature (°C) 260 320
B: Water cooling Off On
C : Pouring rate Normal Maximum
D: Mould conditioning No Yes
The casting temperatures selected were determined to be the extremes that would be considered during
production. This was a result of discussions with the technical and metallurgical staff.
The levels for the remaining factors were deemed to be discrete, e.g. “On” or “Off”.
Prior to the experiment, it was really not known if the selected factors would have any effect on the response,
and so this might be regarded as a “screening” experiment. The considered opinion of the technical
department was that these factors were very likely to be linked to the response and that the levels chosen for
the casting temperature were wide enough to generate an observable change in the response.
A.5.3 Other factors noted but not incorporated due to issues with controllability or relevance
Other factors that might have been considered, but were not included in the experiment, are given in
Table A.2.
Table A.2 — Factors not included in the experiment
Factor Reason for exclusion
Air temperature Air temperature was not regarded as important to the response and the factory was not
temperature controlled and this would have been very difficult to control.
Humidity Considered irrelevant and the factory did not posses air conditioning so therefore it would
have been impossible to control.
Air cleanliness The atmosphere within the factory was naturally smoky and it would have been impossible
to control the level of cleanliness.
Operator There was only one operator involved with this process and so the inclusion of different
operators for this experiment was seen as irrelevant.
Day The day of the week was regarded as irrelevant for this experiment.
A.6 Full factorial design
A.6.1 Design matrix (standard order, run order)
The design selected was a full factorial that allowed for all interactions to be examined. Table A.3 shows the
design.
10 © ISO 2007 – All rights reserved

Table A.3 — Design matrix
A: B: C: D:
Standard order Run order
Casting Mould
Water cooling Pouring rate
temperature °C conditioning
320 On Maximum No 8 1
320 Off Maximum No 6 2
260 On Maximum No 7 3
260 On Normal No 3 4
260 On Maximum Yes 15 5
260 Off Normal No 1 6
260 On Normal Yes 11 7
260 Off Normal Yes 9 8
260 Off Maximum No 5 9
320 On Normal Yes 12 10
320 On Normal No 4 11
320 On Maximum Yes 16 12
320 Off Normal Yes 10 13
320 Off Normal No 2 14
260 Off Maximum Yes 13 15
320 Off Maximum Yes 14 16
The experiment was randomized and the column headed “Run order” was the order in which the runs were
performed during the experiment. The column “Standard order” specifies where the particular factor
combination would be found in the “design matrix” for the 4-factor 2-level design.
A.6.2 Centre points
Centre points were not selected for this experiment, partly because three of the four factors were taken as
categorical (binary) factors, making centre points inconvenient to arrange.
A.6.3 Replication and repetition
The factory could not afford any more experimental time than that needed to conduct the 16 run experiment
shown above. Therefore, there was no replication of any of the experimental runs. However, once an
experimental run had been set up, about 160 bars were produced, providing abundant repetition.
Repetition is important to overcome effects of measurement uncertainty and to improve the reliability of the
analysis. The decision taken to average the number of rosettes was seen as appropriate in this case. At other
times different decisions might be taken, e.g. to calculate a standard deviation from the repeated
measurements for each replicated run. These repetitions provide the analyst with information about factors
that might influence the process variability.
A.7 Analysis of results
A.7.1 Data acquired through the experiment and analysis considerations
The results from the experiment are given in Table A.4. The response analysed is the mean number of
2)
rosettes per bar per run. They were keyed into a software program called MINITAB™ .
Table A.4 — Experiment results
A: B: C: D:
Casting Mould Standard Mean rosettes
Water cooling Pouring rate Run order
temperature °C conditioning order per bar
320 On Maximum No 8 1 85,1
320 Off Maximum No 6 2 77,9
260 On Maximum No 7 3 89,1
260 On Normal No 3 4 87,7
260 On Maximum Yes 15 5 92,2
260 Off Normal No 1 6 100,4
260 On Normal Yes 11 7 84,7
260 Off Normal Yes 9 8 75,4
260 Off Maximum No 5 9 84,5
320 On Normal Yes 12 10 15,3
320 On Normal No 4 11 80,6
320 On Maximum Yes 16 12 8,1
320 Off Normal Yes 10 13 1,0
320 Off Normal No 2 14 84,8
260 Off Maximum Yes 13 15 87,6
320 Off Maximum Yes 14 16 2,3
A decision was taken that only the first-order interactions, e.g. AB, would be analysed, together with the main
effects. Therefore, the higher order interactions were initially considered to be unimportant and insignificant
(subsequently confirmed).
2) MINITAB™ is the trade name of a product supplied by Minitab Inc. This information is given for the convenience of
users of this document and does not constitute an endorsement by ISO of the product named. Equivalent products may be
used if they can be shown to lead to the same results.
12 © ISO 2007 – All rights reserved

A.7.2 Estimation of effects
Figure A.1 — Estimated effects and ANOVA table
The above output (in Figure A.1) indicates that there are significant main effects (“Casting temperature” and
“Mould conditioning”) as well as an interaction between these two factors.
In addition, the probability plot in Figure A.2 indicates the significant factors.
Figure A.2 — Normal probability plot of the standardized effects
The size of the effects and their significance can be seen in the Pareto chart in Figure A.3.
A.7.3 Pareto chart of effects
Figure A.3 — Pareto chart of the standardized effects
The vertical line on the Pareto chart indicates the point beyond which the size of the effect would be
considered significant (at the 0,05 level). It clearly supports the analysis that factors A, D and their interaction
are significant in influencing the occurrence of rosettes.
14 © ISO 2007 – All rights reserved

The plot shows the “standardized” effect. This means the actual effect has been standardized in the statistical
sense.
A.7.4 Estimation of experimental error and standard error of effects
It was mentioned above that only the two-factor interactions were considered. The degrees of freedom of
those discounted interactions (ABC, ABD, ACD, BCD and ABCD) are all assigned to the estimation of the
residual error. Therefore, the degrees of freedom in the ANOVA table in Figure A.1 show 5 for the residual
error. If all of the terms had been used, it would have resulted in zero degrees of freedom to estimate the
residual error and would have prevented any statistical analysis to have taken place in this experiment.
The standard error of the effects have been calculated using the usual formula, i.e. σ /√n. The estimate of σ
e e
can be found by taking the square root of the adjusted mean square (Adj MS) value for the residual
error, i.e. √41,24 or 6,422. Since each level of each factor occurs 8 times in the experiment, the standard error
can be calculated as 6,422 /√8, or 2,271.
A.8 Presentation of results — Effects plots
The main effects plot is shown in Figure A.4 clearly indicating that the “Casting temperature” needs to be set
at the 320 °C level and that moulds need to be conditioned to reduce the level of rosettes.

Figure A.4 — Main effects plot
Figure A.5 is an interaction plot between the factors. The interaction of note is that between the “Casting
temperature” and the “Mould conditioning”, indicating that it is important to have both of these factors set to
their “higher” settings to really reduce the level of rosettes.
Figure A.5 — Interaction plot
A.9 Presentation of results — Optimization recommendations
The above analysis indicates that it is important to have the casting temperature set at 320 °C and the moulds
to be conditioned for the response to be minimized. The other factors and their interactions were unimportant.
Consequently, it was decided to run the process with the settings shown in Table A.5.
Table A.5 — Process settings
Factor Setting
A: Casting temperature (°C) 320
B: Water cooling Off
C: Pouring rate Normal
D: Mould conditioning Yes
A.10 Confirmation results
A run was conducted with the chosen settings described in Table A.5 above. The results were that no rosettes
were found.
There was only one draw-back to running the process in this way, which was that the bars showed slight
symptoms of oxidation owing to the high casting temperature. This gave a slight “yellowing” of the bars
instead of the usual bright shiny appearance. A subsequent experiment was then called for to determine a
temperature below 320 °C at which the process could be run and still hold a very low occurrence of rosettes.

16 © ISO 2007 – All rights reserved

Annex B
(informative)
3)
Direct mail marketing campaign
B.1 General
Direct mail material for a direct marketing campaign can be sent in a variety of packaging styles and with
various textual and graphical cues to encourage a high response rate. Historically, the company had
experimented with a small subset of their campaigns in which one aspect of the package would be varied. The
company was receptive to an approach that was more sophisticated than their previous “one factor at a time”
style.
B.2 Overall objective for the experiment
The objective of the experiment was to maximize the magazine subscription response rate based on a direct
mailing campaign.
B.3 Description of the process
A typical mailing involves solicitations to 400 000 potential subscribers. The actual package mailed can be
varied regarding outer envelope design and other features that had previously enhanced response rate. For
this study, the parent company opted to devote 40 000 packages to the experiment. The mailing itself is
subcontracted to a third party that specializes in assembling these packages.
B.4 Response variable
B.4.1 Choice of variable
The response is the number of people who subscribed and paid (either by credit card or cheque).
B.4.2 Measurement of the response variable
Exactly 2 500 mailings went out for each combination of package contents. Subscribers were tracked to
determine which package of materials they had been sent.
B.4.3 Relationship of the response variable to the objective of the experiment
The publisher expected a response rate of about 2 %. Furthermore, the publisher wanted to be fairly certain to
be able to recognize a 0,5 % increase and at least a decent chance of detecting a 0,25 % increase. The
response rate is the critical factor in addressing the objective of the experiment as the package variations
have roughly the same cost in preparation. Of secondary interest is the proportion of positive responders who
pay immediately as well, which avoids a later billing.

3) This example has been adapted from the article “Using a Fractional Factorial Design to Increase Direct Mail
Response,” by J. Ledolter and A. J. Swersey (2006), Quality Engineering, 18, pp. 469-475. Used with permission from the
publisher, the American Society of Quality, Milwaukee, Wisconsin.
B.5 Factors affecting the response
B.5.1 Description of each factor (continuous/discrete) to be varied
The factors chosen to be varied within the experiment were determined using the publisher’s assessment of
possible improvements and an appreciation of cost-effective adjustments. These were:
A: Presence/absence of an “Act Now to Respond/Pay today” insert;
B: Additional option of paying by credit card (rather than only by personal cheque);
C: Offer wording strength;
D: Presence/absence of phrase on outer envelope with mild profanity.
Each of these factors are categorical with two levels.
B.5.2 Selection of levels (related to size of effect to be determined)
The factors used in the experiment and their associated levels are given in Table B.1.
Table B.1 — Factors and their associated levels
Factor Level 1 Level 2
A: Act now No insert Insert
B: Payment Personal cheque Cheque or credit card
C: Offer phrase No mention Mention “hot off press”
D: Profanity No profanity Mild profanity

B.5.3 Other factors noted but not incorporated due to issues with controllability or relevance
Other factors that might have been considered, but were not included in the experiment, are shown in
Table B.2.
Table B.2 — Other factors considered
Factor Reason for exclusion
Enclosure of bumper sticker An attractive but costly bumper sticker mentioning the magazine could induce some to
respond favourably to the campaign. The size of the bumper sticker impacts the
mailing package and postal costs. No significant improvement with bumper sticker was
expected.
Guarantee Money-back guarantee for part or all of the subscription was considered. This factor
suggests a defeatist attitude and was rejected for consideration. Moreover, the
implementation of a money-back policy incurs costs that directly impact profits.
Testimonials Celebri
...