What is gained by using effect coding rather than dummy coding to analyze the data from a factorial experiment?

Everything we say about factorial experiments on this website is based on using effect (-1,1) coding. However, a lot of behavioral scientists have been trained to use dummy (0,1) coding. Dummy coding has its place, particularly in one-way ANOVA and in non-experimental situations, but for factorial optimization trials in MOST it is essential to use effect coding. It may not seem like using (-1,1) instead of (0,1) would be a very big deal, but it turns out that it is. Oddly, this difference seems to be discussed only rarely and not very explicitly. Whether you use effect codes or dummy codes to perform an ANOVA within a regression framework (say, PROC GLM in SAS), you will be interpreting the b-weights associated with the vectors of codes. When effect codes are used, these b-weights correspond to the textbook definitions of main effects and interactions. However, when dummy codes are used, the b-weights do not necessarily correspond to these textbook definitions. (Note the implication here: If you are doing hypothesis testing based on dummy codes, you may not be testing hypotheses about main effects and interactions.) In fact, under most circumstances, dummy-coded effects should not be referred to as main effects and interactions. We prefer to maintain a clear distinction by calling dummy-coded effects first-order effects, second-order effects, etc. (For simplicity, from now on we will refer to second-order effects, third-order effects, etc. as higher-order effects.)

People trained in using dummy coding sometimes make two inaccurate assertions about factorial ANOVA. The first inaccurate assertion is that it is impossible to interpret main effects if there are any substantial interactions. The second inaccurate assertion is that there is always less statistical power for tests of interactions than for main effects, with power decreasing as the number of factors involved in the interaction increases (e.g., less power for three-way interactions than two-way interactions). These statements may be attributable to a failure to distinguish between first-order effects and main effects, and between higher-order effects and interactions. It is true that when dummy coding is used, it is impossible to interpret first-order effects if there are any substantial higher-order effects. When dummy coding is used, the first-order effects and higher-order effects can be highly correlated. You can see how this could make interpretation of a first-order effect difficult if the higher-order effects were substantial.

However, effect-coded main effects and interactions are not usually the same as dummy-coded first-order effects and higher-order effects. When there are equal ns in each experimental condition, all of the effect-coded main effects and interactions are uncorrelated. Although interactions must always be taken into account thoughtfully when interpreting main effects, this means that a main effect is not necessarily contaminated or rendered uninterpretable by interactions. In addition, when effect coding is used to analyze the data from a 2^kfactorial experiment, if there are equal ns in each experimental condition the power associated with a regression coefficient of a given size is identical, irrespective of whether the coefficient represents a main effect or interaction of any order. (This is explained in detail in Chapter 4 in Collins (2018)). Thus, with effect coding there is not necessarily less power available for interactions than for main effects.

There are two caveats. First, very little is known about interactions in behavioral science, so we don’t know whether they are likely to have effect sizes comparable to main effects, or whether the effect sizes for interactions are likely to be smaller. If the effect sizes for the interactions are smaller than those for the main effects, the power associated with the interactions will be correspondingly smaller.  Second, two subtly different ways of mathematically defining the interaction are in wide use, and they are rarely distinguished.  This has led to a lot of confusion about the power associated with interactions.

To learn more about the difference between effect coding and dummy coding, read the Kugler et al. chapter in Collins & Kugler, (2018).  For more about interactions in factorial analysis of variance, including an explanation of the two different definitions of the interaction, read Chapter 4 in Collins (2018).

References

Collins, L.M. (2018). Optimization of behavioral, biobehavioral, and biomedical interventions: The multiphase optimization strategy (MOST). New York: Springer.

Kugler, K.C., Dziak, J.J., & Trail, J. (2018). Coding and interpretation of effects in analysis of data from a factorial experiment.  In Collins, L. M., & Kugler, K. C. (Eds.), Optimization of behavioral, biobehavioral, and biomedical interventions: Advanced topics. New York: Springer.