QUESTION

Is it possible to conduct a sufficiently powered factorial experiment if I have a multilevel structure to contend with?

ANSWER

Yes! This is possible even when cluster randomization is necessary. For more about this see Dziak, Nahum-Shani, and Collins (2012), Nahum-Shani, Dziak, & Collins (2018), and Nahum-Shani and Dziak (2018).

Note that if one or more intervention components are group-based, there is probably a multi-level structure. This is discussed in Nahum-Shani et al. (2018).

References

Dziak, J. J., Nahum-Shani, I., & Collins, L. M. (2012). Multilevel factorial experiments for developing behavioral interventions: Power, sample size, and resource considerations. Psychological Methods, 17(2), 153-175.

Nahum-Shani, I., Dziak, J. J., & Collins, L. M. (2018). Multilevel factorial designs with experiment-induced clustering. Psychological Methods, 23(3), 458-479.

Nahum-Shani, I., Dziak, J. J. (2018). Multilevel factorial designs in intervention development. In Collins, L. M., & Kugler, K. C. (Eds.), Optimization of behavioral, biobehavioral, and biomedical interventions: Advanced topics. New York: Springer.

Share This…

All Frequently Asked Questions

FAQ: Getting Started

OK, I took your advice and read the preface and first chapter of Collins (2018). Do I now know enough to write a successful grant proposal based on MOST?

FAQ: Very Recent Thinking About MOST

Do you see MOST intersecting with implementation science, and if so, how?

FAQ: Very Recent Thinking About MOST

Everything I have read about MOST, and all the presentations I have seen, treat the three phases as if they are all essential. Yet, I wonder whether it is always necessary to conduct an evaluation trial.

FAQ: Writing Grant Proposals Involving MOST

Can I expect reviewers of grant proposals to understand MOST? If not, how should I handle this in the application? There is not enough room to provide a lot of background.

FAQ: Writing Grant Proposals Involving MOST

I work in the prevention field, in which the horizon on the outcome is very far away. How can I optimize under these circumstances? It seems like it would take decades.

FAQ: Writing Grant Proposals Involving MOST

How can I estimate how long my trials will take?

FAQ: Writing Grant Proposals Involving MOST

I have conducted the preparation phase and now wish to write an R01 application. Should I propose to complete both the optimization and evaluation phases, or only the optimization phase?

FAQ: Writing Grant Proposals Involving MOST

I am a junior scientist who does not yet have any training in MOST, but see intervention optimization as essential to my research in the coming years. Do you think it’s possible to obtaining training in MOST via a K01 or similar?

FAQ: Writing Grant Proposals Involving MOST

Should I apply for funding to support all three phases of MOST?

FAQ: Writing Grant Proposals Involving MOST

How can I find out what other studies using MOST have been conducted in my area?

FAQ: Writing Grant Proposals Involving MOST

How can I obtain the background I need to write a grant proposal involving MOST?

Decision-making: FAQ

I have read Chapter 7 in Collins (2018), and I still do not feel confident I know how to select components and component levels for the optimized intervention.

Decision-making: FAQ

I’ve conducted an optimization trial and analyzed the data. Now my team is going to use the results to optimize our intervention! I am excited but nervous. Do you have any advice?

FAQ: Questions About SMARTs

I have heard Dr. Collins say that the SMART is NOT an adaptive trial design. I don’t understand this, can you explain?

FAQ: Questions About SMARTs

What is the difference between MOST and SMART? How are they related?

Factorial Experiments: FAQ

In a factorial ANOVA there may be many effects estimated. Should I use a correction to control the Type I error rate?

Factorial Experiments: FAQ

Under what circumstances might I want to use a fractional factorial design for a factorial optimization trial?

Factorial Experiments: FAQ

What is gained by using effect coding rather than dummy coding to analyze the data from a factorial experiment?

Factorial Experiments: FAQ

Is it possible to conduct a sufficiently powered factorial experiment if I have a multilevel structure to contend with?

Factorial Experiments: FAQ

Can I use stratified random assignment in a factorial experiment?

Factorial Experiments: FAQ

How do I randomly assign participants in a factorial experiment?

Factorial Experiments: FAQ

You claim that factorial experiments make highly economical use of experimental participants, but I was taught factorial experiments require massive sample sizes to power. Can you explain this discrepancy?

FAQ: Getting Started

Are there any trainings offered on MOST?

FAQ: Getting Started

I’d like to use MOST in my research, but I don’t know much about it. How do you suggest I get started?

FAQ: Getting Started

What is the current go-to citation for MOST?

Search FAQs

Further Learning

Whether you are looking for additional support as you prepare a grant proposal involving MOST or practical information helpful in managing your optimization trial, this section provides resources for a deeper dive into intervention optimization.

REDCap with Most

The goal of this manual is to show how one might setup a REDCap project to support a research study with multiple conditions, such as factorial experiments common in the Multiphase Optimization Strategy (MOST) framework.

Establishing a conceptual model and understanding optimization

Sometimes the conceptual model is not a model of a health behavior per se, but a model of maintaining treatment fidelity, promoting adherence or compliance, or the like. The conceptual model is explained in more...

Common misconceptions about factorial experiments

A factorial experiment is essentially an RCT with a lot of experimental conditions, and therefore is extremely difficult to power.

Informal introduction to factorial experimental designs

The purpose of this page is to clarify some concepts, notation, and terminology related to factorial experimental designs, and to compare and contrast factorial experiments to randomized controlled trials (RCTs). A more in-depth introduction can...

LET’S STAY IN TOUCH

Join the CADIO Mailing List

Keep up to date with the latest news, events, online courses, and resources from CADIO.