BEMP: Methodological Research for Comparative Effectiveness Research (CER)

Comparative effectiveness research (CER) is the conduct and synthesis of research comparing the benefits and harms of different interventions and strategies to prevent, diagnose, treat, and monitor health conditions in "real world" settings. CER includes both real world (pragmatic) clinical trials and observational studies. The purpose of CER is to improve health outcomes by developing and disseminating evidence-based information to patients, clinicians, and other decision-makers, responding to their expressed needs, about which interventions are most effective for which patients under specific circumstances. However, those conducting both randomized and observational CER face many statistical challenges. A couple of examples are described briefly below.

An important epidemiological and statistical tool used in observational studies is matching, and the application of matching methods especially for large existing databases, raises a number of challenges. One of the motivations for matching is to identify a subset of the population where the causal effect can be estimated without relying on huge extrapolations. For many studies, this involves calculating a propensity score at baseline, and matching treated and untreated subjects who have similar scores. However, this situation is more complicated for CER studies relying on claims or health record data. In those settings, subjects have repeated measurements over time corresponding to encounters with the health system, but the number and timing of the observations varies substantially. Furthermore, the number and timing may be related to both measured and unmeasured characteristics that are related to the outcome. Methods for matching in this setting, which are both statistically valid and computationally feasible, are needed.

Another issue that arises particularly in CER is time-varying heterogeneity of treatment effects. Randomized and non-randomized treatment effectiveness studies frequently include analyses to identify subgroups in which the treatment may be more or less effective. However, similar to above, such analyses may not take into account that some of these stratification factors or potential effect modifiers may vary across time, often in response to treatment. A feedback loop between effect modifiers and treatment violates crucial assumptions of standard analyses of effect modification. Although methods involving instrumental variables have been developed, theoretical details need to be resolved because this approach, which typically uses only baseline covariates measured at or before the instrumental variable, does not include factors that are impacted by the treatment exposure and temporally occur after the occurrence of the instrumental variable.