A new approach to effect heterogeneity for clinical trials

VanderWeele provided two separate definitions of effect heterogeneity, which he referred to as “effect modification in distribution” and “effect modification in measure”. The standard epidemiological approach, which is based on effect modification in measure, is associated with a number of well-described shortcomings, and no consensus exists about the conditions under which investigators can assume effect homogeneity on either the additive or the multiplicative scale. More recently, Bareinboim and Pearl [2] introduced a new graphical framework for transportability, based on effect heterogeneity in distribution. These graphs are an elegant solution to many of the problems associated with traditional approaches, but they require the investigator to make strong assumptions about the data generating mechanism: In particular, it is not sufficient to control for those variables that are associated with the effect of treatment; investigators using this approach are required to account for all causes of the outcome that differ between the populations.

In light of these limitations, we propose a new definition of effect heterogeneity, based on “counterfactual outcome state transition parameters”, that is, the proportion of those individuals who would not have been a case by the end of follow-up if untreated, who would have responded to treatment by becoming a case; and the proportion of those individuals who would have become a case by the end of follow-up if untreated who would have responded to treatment by not becoming a case. Effects are said to be equal between populations if and only if these proportions are equal between the populations. Although counterfactual outcome state transition parameters are generally not identified from the data without strong monotonicity assumptions, we show that when they stay constant between populations, there are important implications for model specification, meta-analysis, and research generalization.