Under what circumstances might I want to use a fractional factorial design for a factorial optimization trial?

Fractional factorial designs require fewer experimental conditions than a complete factorial. Sounds great, right? These designs can be highly efficient and have a lot to offer. In fact, we believe they could be used much more often in intervention science. However, to be blunt, if you do not know what you are doing when selecting a fractional factorial design, you can get into trouble. Before considering a fractional factorial design, please note these facts:

1. Fractional factorial designs require EXACTLY THE SAME SAMPLE SIZE as the “parent” complete factorial. The economy of these designs comes from having fewer experimental conditions to manage; the same N you would have used in a complete factorial is divided among fewer experimental conditions. In other words, if you are hoping to find a design that requires fewer subjects than a complete factorial (and, good luck with that), a fractional factorial will not help you.

2. The experimental conditions that are removed from a complete factorial to create a fractional factorial are selected on purely statistical grounds. In other words, you as the investigator will not review a list of experimental conditions and decide which ones to remove.

3. There are no free lunches in statistics; there are tradeoffs associated with the use of a fractional factorial design. First, in a fractional factorial design you cannot estimate all of the effects you would be able to estimate in a complete factorial. Second, fractional factorial designs require assuming that certain interactions are negligible in size (not necessarily zero, but small enough not to disrupt decision-making). Typically these are the highest-order interactions in a design (e.g., in a design involving five factors, the four-way interactions and the five-way), but this varies. You might want to consider a fractional factorial experiment if a complete factorial design is conceptually suitable for your research questions and either or both of the following are true:

• You are planning a 2^k optimization trial where k≥5, and you can obtain enough participants to power the experiment. However, you are concerned that you cannot manage all of the experimental conditions in a complete factorial design. (There is a fractional factorial design for 2⁴ experiments, but in our view it has some undesirable properties.)
• You have to use cluster randomization and you can obtain a sufficiently large overall N to achieve an acceptable level of power, but you don’t have enough clusters to populate a complete factorial design. This can occur if the clusters are large, e.g. entire schools. (More about this can be found in Dziak, Nahum-Shani, & Collins, 2012.)

We make the following two recommendations:  First, we recommend that before you consider using a fractional factorial design you read Chapter 5 in Collins (2018).  Second, we recommend that you do not select a fractional factorial design by finding one that was used in a different published study.  Instead, we strongly recommend that you go through the process of identifying the fractional factorial design that is right for your study by using software such as Proc FACTEX in SAS or similar.  There are subtle differences between fractional factorial designs, and it is worth it to take the time to find the right one.

References

Collins, L.M. (2018). Optimization of behavioral, biobehavioral, and biomedical interventions: The multiphase optimization strategy (MOST). New York: Springer.

Dziak, J. J., Nahum-Shani, I., & Collins, L. M. (2012). Multilevel factorial experiments for developing behavioral interventions: Power, sample size, and resource considerations. Psychological Methods, 17(2), 153-175.