It replaces the analysis of variance test with a less-assumption-dependent binomial test and should be routinely used whenever we are trying to robustly decide whether a primary factor has an effect.

This block plot reveals that in 10 of the 12 cases (bars), weld method 2 is lower (better) than weld method 1. From a binomial point of view, weld method is statistically significant.

• Vertical axis: Response variable Y
• Horizontal axis: All combinations of all levels of all nuisance (secondary) factors X1, X2, ...
• Plot Character: Levels of the primary factor XP

1. weld strength (2 levels)
2. plant (2 levels)
3. speed (2 levels)
4. shift (3 levels)

In the above chart, the ordering along the horizontal axis is as follows:

• The left 6 bars are from plant 1 and the right 6 bars are from plant 2.
• The first 3 bars are from speed 1, the next 3 bars are from speed 2, the next 3 bars are from speed 1, and the last 3 bars are from speed 2.
• Bars 1, 4, 7, and 10 are from the first shift, bars 2, 5, 8, and 11 are from the second shift, and bars 3, 6, 9, and 12 are from the third shift.

In the block plot for the first bar (plant 1, speed 1, shift 1), weld method 1 yields about 28 defects per hour while weld method 2 yields about 22 defects per hour--hence the difference for this combination is about 6 defects per hour and weld method 2 is seen to be better (smaller number of defects per hour).

Is "weld method 2 is better than weld method 1" a general conclusion?

For the second bar (plant 1, speed 1, shift 2), weld method 1 is about 37 while weld method 2 is only about 18. Thus weld method 2 is again seen to be better than weld method 1. Similarly for bar 3 (plant 1, speed 1, shift 3), we see weld method 2 is smaller than weld method 1. Scanning over all of the 12 bars, we see that weld method 2 is smaller than weld method 1 in 10 of the 12 cases, which is highly suggestive of a robust weld method effect.

What is the chance of 10 out of 12 happening by chance? This is probabilistically equivalent to testing whether a coin is fair by flipping it and getting 10 heads in 12 tosses. The chance (from the binomial distribution) of getting 10 (or worse: 11, 12) heads in 12 flips of a fair coin is about 2%. Such low-probability events are usualy rejected as untenable and in practice we would conclude that there is a difference in weld methods.

• A quantitative procedure (analysis of variance) is replaced by a graphical procedure.
• An F-test (analysis of variance) is replaced with a binomial test, which requires fewer assumptions.

1. Is the factor-of-interest significant?
2. Does the factor-of-interest have an effect?
3. Does the location change between levels of the primary factor?
4. Has the process improved?
5. What is the best setting (= level) of the primary factor?
6. How much of an average improvement can we expect with this best setting of the primary factor?
7. Is there an interaction between the primary factor and one or more nuisance factors?
8. Does the effect of the primary factor change depending on the setting of some nuisance factor?
9. Are there any outliers?