Optimization Phase

In this phase of MOST, the investigator optimizes the intervention. This involves identifying the set of components and component levels that best meet the optimization objective, and therefore are designated the optimized intervention. To do this it is necessary to gather empirical information. The approach to gathering this information depends on what information is required. Usually estimates of the individual and combined effectiveness of a set of intervention components are required for optimization. This information is typically gathered by means of an efficient randomized experiment called an optimization trial.

There are many choices of experimental design for the optimization trial. This web site emphasizes factorial and fractional factorial experiments, but alternatives include the sequential, multiple assignment, randomized trial (SMART), micro-randomized trial (MRT), and system identification methods. The results from the optimization trial form the basis for making decisions about component selection and formation of the optimized intervention (Chapter 7 in Collins, 2018).

At the end of the optimization phase, the investigator has selected the intervention components and component levels that make up the optimized intervention. Figure 1 shows a diamond immediately after the optimization phase, indicating that a decision is required. At this point the investigator has a rough sense of the likely overall effectiveness of the optimized intervention. If, based on the effect size estimates obtained in the optimization trial, it appears that the optimized intervention will have an effect that is sufficiently large (by statistical and/or clinical criteria) to justify evaluating it with an RCT, it would make sense to proceed to the evaluation phase. On the other hand, the results of the component selection experiment may indicate that the optimized intervention is likely to have a very small effect. For example, the optimization objective may call for an intervention consisting of only active components, and perhaps the experiment indicates that there is only one such component. In this case, it may not make sense to proceed to the evaluation phase. As Figure 1 indicates, it may be advisable to return to the preparation phase and reconsider the theoretical model, pilot test some new intervention components, and so on rather than go immediately to the evaluation phase and an RCT.

What is meant by “optimized”?

Collins (2018) defines intervention optimization as “the process of identifying an intervention that provides the best expected outcome obtainable within key constraints imposed by the need for efficiency, economy, and/or scalability.” Note that optimized does not mean best in an absolute or ideal sense. Instead, the optimized intervention is the best intervention that can actually be implemented.

So isn’t optimization just deciding what is “good enough”? How is this so different from the treatment package approach? Every clinical trial uses this “good enough” approach.

Simply saying “well, this is good enough” is not the same as optimization. In fact, the optimized product may not be good enough, and a product considered good enough, or even the ideal, may not be optimal.

Let’s take a straightforward hypothetical example from the field of product development and manufacturing. Suppose you were charged with building and optimizing a global positioning system (GPS). The ideal would be a GPS that can pinpoint your location. The way to come closest to this would be to spare no expense in building the GPS. However, suppose the optimization objective is to build an affordable GPS that costs $100 or less to manufacture. Note that the ideal does not meet this objective. The $100 constraint on manufacturing costs means you cannot choose the most precise and, therefore, most expensive components for your GPS. In this example, optimization means finding the combination of components that comes closest to the ideal without exceeding the $100 limit. Maybe that means going with all moderately-priced components; maybe it means one key component has to be on the expensive side and the others will have to be cheap. Suppose the best GPS you could manufacture for $100—that is, the optimized GPS—can identify locations only within a 400-foot range. This might not be “good enough” even though it has been optimized. Then the company might decide to change the optimization objective by changing the $100 limit and allowing, say, $150 in manufacturing costs. Or, you might be charged with developing cheaper and more effective components that potentially could be used to make a GPS that will be both optimized and “good enough.”