Dear Caddy,

I am considering conducting a factorial optimization RCT. However, I’ve been told that the factors have to be independent. Is this true? It seems to me that it would be pretty unusual for the components in an intervention to operate independently.

Signed,

Wondering in Wisconsin

---

Dear Wondering,

Depending on what is meant by the term “independent,” yes, the factors in a factorial experiment are assumed to be independent. BUT this term can have different meanings. Let’s explore two different possible meanings of this term.

One possible meaning that is sometimes wrongly applied here

One possible meaning of the term “independent” is that the components operate independently, that is, the presence, absence, or level of a component has no influence on the performance of any others. By this definition, independent means that there are no statistical interactions. I agree with you that this is unlikely; in fact, I cannot think of a behavioral or biobehavioral intervention I have ever encountered where this assumption would be plausible. Moreover, it is an empirical question whether interactions are present, and it is a distinct advantage of factorial experiments that they enable the investigator to address this empirical question!

Another possible meaning that is correctly applied here

Factorial experiments require that a particular level of a factor be implemented in exactly the same way irrespective of the levels of the other factors in any experimental condition. Another way to put this is that all the factors can be manipulated independently. This is what is meant when we say that the factors in a factorial experiment must be independent.

To take a simple example, imagine you are developing an intervention to prevent excessive alcohol use in college students. You plan to examine the following components (both on or off): descriptive norms and alternative strategies. Your experimental design has two factors, and includes the following four conditions:

Condition	Descriptive Norms	Alternative Strategies
1	Off	Off
2	Off	On
3	On	Off
4	On	On

Suppose you have determined that Descriptive Norms can be implemented exactly the same way whether or not Alternative Strategies is included, and Alternative Strategies can be implemented exactly the same way whether or not Descriptive Norms is included.

Now, imagine you wish also to examine the effect of role-playing to reinforce the skills to carry out alternative strategies. This adds another factor to the experiment, and the design would be:

Condition	Descriptive Norms	Alternative Strategies	Role-Playing
1	Off	Off	Off
2	Off	Off	On
3	Off	On	Off
4	Off	On	On
5	On	Off	Off
6	On	Off	On
7	On	On	Off
8	On	On	On

Consider Conditions 2 and 6, in which participants who have not received the Alternative Strategies component are provided with the Role-Playing component. Does it make sense to provide the Role-playing component to participants who have not received the Alternative Strategies component? Perhaps you are thinking, “You can fix this by just adding a little material in Conditions 2 and 6 to orient the participants and give them some basic knowledge about alternative strategies.” In considering this, remember that the Role-Playing component must be EXACTLY THE SAME in the experimental conditions where Alternative Strategies is Off and where Alternative Strategies is On. In other words, it would be a violation of assumptions to add content to the Role-Playing component only in Conditions 2 and 6.

Thus, if Conditions 2 and 6 are not workable without making changes to the Role-Playing component that are specific to those conditions, this factorial experiment is not a good idea. However, it is likely that a different, more satisfactory experimental design can be arrived at with careful thought.

You can read more about this topic in Chapters 3, 4, and 5 in Collins, L.M. (2018). Optimization of behavioral, biobehavioral, and biomedical interventions: The multiphase optimization strategy (MOST). New York: Springer.

Summary

If you are considering a factorial optimization RCT, it is fine if you hypothesize that the components do not operate independently; in fact, an appropriately designed experiment will enable you to test any hypotheses you may have about this.

However, you must be able to manipulate each and every factor in the experiment independently. In other words, each level of each factor must be implemented exactly the same way in every experimental condition.

Sincerely,

Caddy