Many engineers spend substantial portions of their time in testing.
They test materials, processes, parts, and products. They test
to evaluate concepts or physical principles, to solve problems,
and to improve systems. The amount of resources spent in testing
and the importance of this activity for the success of the design
effort means that there is great value to any technique that can
help engineers become more effective and efficient in these activites.
The above figure summarizes an experiment by Swedish engineers
that illustrates what may be possible using modern approaches
to testing. This experiment showed engineers for SKF, the Swedish
ball bearing manufacturer, how to quadruple the life of a ball
bearing. An important aspect of this experiment is that the solution
might never have been tried by engineers using the traditional
"one factor at a time" approach to testing that many
engineers have been taught.
To understand this experiment, it helps to know that a ball bearing
consists of four basic elements: outer ring, inner ring, balls,
and a cage that keeps the balls a constant distance apart. The
SKF design team decided to test simultaneously three different
modifications to a current design. They tested a modified heat
treatment for the inner ring in comparison to the standard --
the top vs. the bottom of the cube. Second, they tested a modified
cage design in comparison to the standard -- the back vs. the
front faces of the cube. Third, they tested a modification of
the outer ring osculation in comparison to the standard -- the
right vs. the left hand sides of the cube. (In other contexts,
"osculation" means "kissing". Here, it describes
whether the ball "kisses" the outer ring in one point
or two.) The design team made four inner rings with the modified
heat treatment and four with the standard, four modified cages
and four standard, and four modified outer rings and four standard.
They then assembled them in the eight combinations indicated by
the eight points of the cube. The eight resulting ball bearings
were then run to failure under several times the maximum rated
load. Six of the eight ball bearings failed in the first 26 hours
of the test. In less than a week, they had all failed.
The results were dramatic: The combination of modified inner and
outer rings quadrupled the lifetime of the ball bearing. The lifetimes
in the upper right edge of the cube -- 85 and 128 -- are roughly
four times the lifetimes of the other six ball bearings tested.
Moreover, the cage design seemed to have little impact on product
life. This allowed SKF to save money by using the cheaper cage.
It is important to note that this solution might never have been
found without a modern experimental strategy such as the one in
the above figure. Many engineers are trained to test "one
factor at a time," holding all others constant. An engineer
following that advice might never have tested the modified inner
ring with the modified outer ring, and therefore might have stopped
without finding the dramatic solution presented in the figure.
This situation, where the whole is different from the sum of the
parts, is called "interaction," and is relatively common
in both physical and social processes. The solution to many engineering
problems requires getting several things right simultaneously,
and the "one factor at a time" approach often becomes
an obstacle to solution.
Fortunately, some fairly simple experimental strategies can dramatically
increase the chances of solving a problem in a reasonable period
of time. They also often increase the quality of the answer obtained.
In many cases they require relatively little time and other resources.
The modern field of experimental design provides tools that help
experimenters (a) explore a wide range of alternatives with minimal
effort and (b) organize the results in simple ways that are easy
to understand and use. For industrial situations where tests are
often relatively inexpensive and quick to perform, it is best
to think of a sequence of quick, simple experiments. Later experiments
build on the knowledge gained earlier. A sequential approach like
this fits well with the way that engineering and scientific knowledge
is actually developed. When testing takes longer or is more expensive,
good planning using appropriate experimental design techniques
can increase the chances that the experiment will resolve the
issue it was designed to answer. If reasonable physical theory
is available for the phenomena under study, this can be refined
with appropriately designed experiments. In many cases unnecessary
assumptions are made that only complicate the engineering analysis.
Meanwhile, other critical assumptions are overlooked. These issues
can be explored with experimentation. In other cases it is easier
and quicker to use graphics, such as the above figure, supplemented
with simple algebra to summarize the experimental results and
decide what to do next. Mathematical summaries of results, whether
with simple algebra or higher math, provide substantial advantages
for building the proprietary technology base of an organization
and improving the quality of future engineering efforts with similar
Designed experiments have a fairly long history. They have helped
people solve important problems in agriculture since the 1920s,
for example, They have helped in the production of beer (Guiness
stout) since the early 1900s, in textiles since at least the 1930s,
and in the chemical industry since at least the 1940s, to name
only a few. More recently, they helped the US semiconductor industry
reverse a slide towards extinction. In 1982, US companies had
54% of the world semiconductor market; Japanese companies had
27%. By 1986, Japanese were in the lead and the US was forecasted
to have only 17% of world markets in the year 2000. There was
great concern about this in the US. In 1987, the Semiconductor
Industry Association founded SEMATECH to sponsor and conduct research
aimed at assuring leadership in semiconductor manufacturing technology
for the US (www.sematech.org).
Shortly thereafter, the ominous trend was reversed. By 1993,
the US was back in the lead. The President of SEMATECH proclaimed
that statistical methods, including experimental design, had played
a major role in this reversal and were "a competitive necessity"
[William J. Spencer and Paul A. Tobias, "Statistics in the
Semiconductor Industry: A Competitive Necessity", American
Statistician, v. 49(2), pp. 245-249]. It is only a slight
exaggeration to say that in 1980, almost none of the semiconductor
manufacturers were using designed experiments to understand and
improve yields. By 1990, virtually all used designed experiments.
The companies that were too slow to adopt experimental design
techniques had a slower rate of improvement in yield and went
out of business. The semiconductor industry may be somewhat unique
in its rate of change and therefore its need for a high rate of
learning. However, in almost any field, experimental design techniques
can help increase the rate of acquisition of new, useful knowledge.
If none of your competitors are making serious use of these techniques,
then you will not be at a competitive disadvantage by not using
them either. On the other hand, anything you do to increase the
effectiveness of your engineering team can increase your success
in the marketplace.
To introduce experimental design techniques to an organization
that currently makes little or no use of them, we recommend experimental
design training combined with projects on problems identified
as important by a diagonal slice of the organization, using the
Quick Start, Quick Results Total Quality
For a quick introduction to designed experiments, ask for our
free "Do-It-Yourself Experimentation Education Kit."
This booklet outlines two experiments with paper helicopters that
you can cut out, fold up, and use to understand one common, powerful
experimental approach. This is suitable for kids from age 10 to
100. Call (408)294-5779 fax: (408)294-2343, or e-mail: firstname.lastname@example.org
for your free copy.
For more information on Design of Experiments, check out the Center for Quality and Productivity Improvement at
the University of Wisconsin-Madison.
Other Productive Systems Engineering programs: Product Development,
Reliability Experimentation, Tolerancing,
Quick Start Total Quality (Total Quality
Control, TQC / Total Quality Management, TQM), Statistical Process Control
(SPC); Fiber Optic Communication/Transmission Systems
(FOCS or FOTS); Control and Monitoring
(RMM); Electronics (Theory & Application);
Production Line Assembly for Technicians